People interviewing for a tech job had their genders masked. It made things worse for the women.

It is well-trod territory at this point that biases against women’s technological abilities hold women in technology back. Study after study has shown bias persists at every point of the employment process. So the start-up interviewing.io decided to try and do something about it. It masked women’s voices to sound like men’s and vice versa during online interviews to see if interviewers would like them better.

It was inspired to do the experiment because it was seeing some alarming data. Interviewing.io is a platform that allows people to practice technical interviewing anonymously and, hopefully, get a job in the process. After amassing data from more than a thousand technical interviews, the company noticed a troubling trend, writes founder Aline Lerner in a blog post:

“Men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either—men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.”

The company decided to build a voice-masking tool to see if that made a difference in how candidates fared. They developed a few different types of voice modulations, and tested them out on 234 different interviews. They were surprised to find, however, that the gender swapping had no significant effect, and actually achieved the opposite of what they had hoped. Here’s Lerner:

“Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women.”

Lerner’s conclusions fly in the face of previous experiments around gender bias and hiring practices in male-dominated fields. In the 1970s, when the nation’s leading symphony orchestras faced a problem like the one Silicon Valley faces today, orchestras began concealing applicants behind screens and drapes. By the mid-1990s, the number of women in the five leading orchestras in the U.S. had increased fivefold. By 2003, more than a third of players in the top 24 orchestras were female.

In science and technology, like in the orchestras of the not-so-distant past, the idea that men are inherently more suited to the field is a deep-seeded and persistently reinforced cultural belief. Many studies have shown that the advent of double-blind scientific reviews, in which both the name of the reviewer and a study author are not known, has significantly increased the number of works by female scientists accepted for publication. It only stands to reason, then, that similarly concealing gender would improve the luck of female tech job applicants.

Lerner dug into her data and came up with her own guess for the cause of the surprising results: women were leaving the platform after having one or two bad interviews. In other words, women, feeling discouraged, seemed to be just giving up on interviewing altogether. “Once you factor out interview data from both men and women who quit after one or two bad interviews,” she writes, “the disparity goes away entirely.”

Lerner’s findings here do correlate to some things academic research has also shown. She pointed to one study that found that after giving a scientific reasoning test to male and female undergrads and asking them how they fared, women underrated their own performance.

The sample size here was not very large, nor was its design very scientific (it assumes, for example, that voice pitch and sound are the only things that give gender away). So while it may be tempting to take this anecdote and radically rethink the way we approach closing the gender gap in science and technology, it’s probably best to view the lack of disparity Lerner found as an aberration. There is just too much research—across many different fields—showing the opposite of what her experiment found.

But her voice-masking experiment does remind us of a point perhaps too-often overlooked: as women, it is not only the biases of the world we must surmount, but also those biases we have against ourselves.