Studying Competition and Gender Through Chess

INTRO VOICE-OVER: Data Skeptic features interviews with experts on topics related to data science, all through the eye of scientific skepticism.

Kyle: Peter Backus has a PhD in Economics and is currently a Lecturer in Economics at the University of Manchester and a Fellow at The Institut d'Economica, de Barcelona. Peter is also a former guest of the show from back in 2014 when we discussed the economics of charitable giving and other interesting economic topics. His research has covered topics including the economics of charities, the private provisioning of public goods and the study of gender differences in competition. His recent work has explored gender differences in competition, specifically in chess. And it's that topic that I have invited him here to discuss today.

So, Peter welcome back to Data Skeptic.

PETER: Thanks for having me back Kyle.

Kyle: Yeah this is a really interesting paper.

I know we live in an age where many people don't read past the headline. I'd be a bit uncomfortable myself speaking about a result which said, "women are intrinsically not as good at chess compared to men." Although, I don't think that's the interpretation anyone would get if they actually read your paper, but to perhaps quell interpretations like that before they start, could you give me a high level summary of your results and maybe touch on what one should or shouldn't infer from them?

PETER: Sure, just to give a little background on the paper, what we want to contribute to this literature on gender and competition - where some very big name economists have worked - is to try to understand how many women respond differently to competitive environments, which is important for all sorts of reasons, primarily labor market outcomes.

The example that I usually gave is: imagine the boss comes down and there are male and female employees and he says, "okay whoever has the most number of sales at the end of the month is going to get a promotion."

Well essentially it's a competition. If you look at a lot of the management literature on best practicing management, they encourage managers to create competitive environments in order to increase the productivity of the employees. It is important to understand how gender and how men and women interact and engage within a competitive environment, not just because of chess or other sorts of sports competitions, but also because of things that actually matter, like labor market outcomes.

However, it is very difficult to do in a sort of employment environment because there are all sorts of confounding factors. It's very difficult to control for things. It's very difficult to measure certain components of that environment.

So chess gives us a chance to study that and you're absolutely right. It would be a misinterpretation of the paper if people would say, "women are less good at chess" or "women are less good at difficult cognitive tasks" because that's not actually what we find, at all.

The thing that we find is that, when men and women compete against each other, even if those two players, even if the male and female player have the exact same abilities, the man tends to do better and the woman tends to do worse, in terms of the outcome of those gains. So women lose more often than they should even when the men and women are of equal ability. And that's an interesting result because it raises question like: Why? Why is that? What's going on? What's different about playing against a man than playing against a woman, or playing against somebody of your own gender than playing against someone of the opposite gender?

Kyle: So, I chose two labor markets as an interesting case. That's where it's critical to understand these things, but also very hard to study because of all those confounding factors. So we'd like to find an environment or a situation that has limited confounding factors, I guess you could have picked just about anything from drag racing, to arm wrestling, or who knows what. Why is chess a really good thing to study?

PETER: Chess is great for a number of reasons. Chess players - so I'm not a big chess player myself one of my co-authors is a very accomplished chess player so I've learned a lot about the game from him - but one of the things that chess players are great at is collecting data.

They're just data hogs. They collect everything that goes on during a game. Every move that happens, they have a very elaborate rating system, or an ELO rating, which some people may have heard of and some may have not, which is a measure of how well you've performed in past games and who you've been performing against.

They're able to track the same player for a long period of time because they're registered with this international chess federation, so you can see how different people progress over their chess careers, how they get better, or how they get worse, and how often they're playing.

So we have all this fantastic data that we can use. There are sort of two key features, one that we have this Elo rating, we have this way to control for the ability of the people that are in the competition. So you can go back and think about the example I gave about the boss comes down and the male and female employee are in a competition about sales. It's difficult to measure the ability of those two people as sales people because we don't have a reliable metric for that whereas in chess we do.

We can measure and observe the ability of the each player in the competition. The other thing that's really important is that in these tournaments, and it's something shown in the paper. In these tournaments, the person that you're playing against is what we say, as good as random. So we're able to use that random variation in the gender of your opponent to then identify what happens when a man plays against a woman versus when a man plays against another man.

So that's something else that would be difficult to do in a real world setting in sort of a labor market setting where you have, a lot of professions are heavily gender either they're male dominated or female dominated. So the person they're playing against or in competition with might not be completely random, so it's difficult to say what happens if the gender wins, otherwise. So chess is a great place to study because of the data that's available.

The other reason that chess is really good to study competition is that it is a competition in which men and women compete on equal footing. If there's some chess players that are listening to this, they might be rolling their eyes or say that I'm crazy because there's a lot of sexism in chess. In the paper, we've produced a couple of quotes just at the beginning to try to demonstrate this.

Kyle: Yeah, I was a little horrified. I didn't know it was so bad. PETER: (Laughs)

Yeah, its' really bad and we had, I don't know 5 pages of quotes and it took us some time to think about which ones to include and in the end we chose some back from the 60s where maybe you think, "oh, these were the madmen days and of course, there was sexism." Yet, the most recent one was from 2015 where you'd have hoped that in society had moved on, but there's still this perception that women are not good at chess. Men are superior players of chess, but when you look at the data and evidence that just isn't born out. You just don't see that. Men have a higher rating on average but that has more to do with sampling than anything to do with innate ability.

So we have this competition; we have this setup. We have lots of data, really good data and we have a competition where men and women compete on equal footing. So, any physical sport wouldn't be able to do that. It has to be something that's cognitive, where one sex doesn't have an inherited advantage over the other, and chess gives us that.

Kyle: The other point you made in the paper that maybe it's obvious to some, that occurred to me is that there's really no luck in chess, that any variance introduced by the role of the dice is gone. This is purely a performance-based management you can use.

PETER: Yes, it helps simplify our analysis because luck doesn't play a role. I mean chess is just a computational problem. It's why computers ended up being really good at it. Ultimately, there will be a computer that "solves" chess for lack of a better phrase, because by a backward induction you can work it out exactly what you should do at every point of the game. In that, some people might say that there's luck because your opponent might make an error and that means that you're lucky.

In that extent yes, that's kind of true, but there's no luck that would determine the outcome of the game that's also inherently apart of the game like in poker where you don't know what cards will be turned over. That's just the turn of the card or roll of the dice, that's completely random. You don't know what's coming. So that nicely simplifies our analysis.

Kyle: So, I imagine we could do a whole show or series of shows just on ELO. It is very well studied which is one of its strengths then for you to leverage it. For anyone not familiar with it, could you talk a little bit about some of its important features and how to interpret it?

PETER: So, the ELO rating has been used in chess for a long time. The people over at 538, which I'm sure you're familiar with and probably your listeners are well, Nate Silver's website, they've started using it for other sports. So they now have an ELO for NBA teams and they have ELO for NFL teams. The benefit of an ELO is, it's better than just looking at the win-loss record because it takes into account the strength of the opponents that you've faced in the past.

If you have two players of exactly equal abilities, so their ELO is exactly the same. There's a 50% chance that player 1 should win and a 50% chance that player 2 should win. If player 1 defeats player 2, player 1's ELO will go up by some amount or a small amount but if the players are of different abilities, say player 1 is of a much higher ability than player 2 and player 2 defeats player 1, so the worst player wins. Player 2's ELO will increase by quite a bit more and if player 1 loses to player 2, so the better player loses to the worst player. Player 1's ELO will fall by more.

So it doesn't just depend on the outcome of the game but it depends on the strength of your opponent and the outcome of the game. So it gives us a better sense of the ability of the individual players because we see how they've rated against other players in the past of varying abilities.

Kyle: I think I know what your answer is going to be but in the interest of completeness, I'm thinking about that as a measure and I'm reminded of academic performance tests. At least, in United States we have this system called the SAT. I don't know the data on this, but I'm told it has racial biases in some way. Like you had a disadvantage depending on your race. Is there any possibility ELO could have a disadvantage to gender that could explain this?

PETER: So, we show a little bit of that in the paper actually in the next iteration of the paper, we go a bit further, strengthen the evidence that it doesn't and we'll talk about a little bit is the within-game-error. ELO predicts the within-game-error equally well for men and women, which tells us that ELO is measuring the same ability. So ELO say of 2000 means the same thing for a woman as it does for a man.

Kyle: Hmm. Of course, ELO is your historical record. It's everything, every game you've won or lost up-to-date but you're really looking at the next game in a sense. Can you look at it with some granularity or do you just have to take the match as a whole?

PETER: So you mean within a particular game?

Kyle: Right.

PETER: We can look at it with more granularity because the chess community is so great at this and they record every move that's made. We're able to look at not just the outcome of the particular game, but we can look at the quality of every move made within that game and that's really the value added of this particular paper because other people who used chess to try to study gender differences and competitiveness and they've looked at just the outcome of the games and they find the same thing that we find sort of in the first part of the paper.

The innovation in our paper is to go a step further and try to understand what's going on within those games. Why is it that we keep observing women performing worse when they play against men or men perform better when they play against women? And that's actually a key question. If you just look at the outcome of the game and you see that women are doing worse - conversely men are doing better in terms of wins and losses - you don't know whether that's because men are somehow playing better or men are pursuing a different strategy when they play against women; or if that's women are somehow playing worse or pursuing a sub-optimal strategy when they're playing against men.

So you can't untangle those two just by looking at the outcome of the game. The real value out of our paper is to look at those individual moves, look at what's happening inside of a game between a man and a woman versus a woman and a woman and try to measure whether or not the quality of play is different from one game to another.

Kyle: Yeah that's really interesting. I'd love to delve in a little more. Could you talk about some of the methodology for how you try to determine the quality within-game-play?

PETER: This is the piece that we were all very excited about when we came up with this because we thought this was very clever. We all gave ourselves a big pat on the back. So we'll see what referees think when it comes time to try to publish. But the idea, when we started this study, we just initially looked at the outcome of the games and it was me and these two other economists that were doing that and then we were trying to think what we can do next.

We wanted to look what was happening within the game and if we could measure the quality of the play. Once you understand that chess is just a computational problem, you should be able to measure how well people are computing. How close are they getting to the optimal solution? So we turned to Matageed who's a computer science professor in Slovenia and has done a lot of work on chess. He developed this little metric. He was just writing about champions, Chess Champions throughout history, and trying to say which one is better than the other is.

But when we found his paper, we thought this is exactly what we want to do in our study of gender and competition. So what he did was build a counter factual for every move within every game. We know what moves players' actually made in a game. We observe that in the data, they keep track of that kind of thing.

What we can do is feed those moves in to a computer, like a big powerful chess engine, sort of a super computer kind of thing and we ask the computer to tell us, "Computer, given this particular board position, given this particular situation in the game. What would you have done in this particular situation?"

And the computer tells us, "I would've moved pawn here or the queen there." Then we can compare that move to the move that the player actually made. Then we look at the change in the relative advantage that a player has from that position, given the move that the player actually made relative to the move that the computer would've made if the computer had been playing that game. That tells us sort of distance that the player was from "optimal solution" to the game from that particular point.

By building this counter factual of what a computer would've done, we can then build up a measure that is how well that a player play that particular game and we can really measure how well did a player play that particular move but we average up to the game level to study the quality of play within the game.

Kyle: Yeah I think it's really novel. I like the concept but I'd also like to unpack it even a little bit more. So like, "what would Deep Blue have done?" is an interesting question to ask for your counter factual but we know chess is not yet a solved game so we can't be assured that the computer came up with the most optimal move.

How do we know that we don't run into situations where maybe the player was just a little bit smarter on average or maybe I should ask what's the rate you see in these differences comparing the computer's move to the person's move?

PETER: So that's absolutely a great point and we try to talk about it in the paper which is the computer, and that's why I said quote, unquote "optimal move." The computer might be playing the really truly optimal move but because chess hasn't been solved and because it would require even significantly more computing power and computer time to get it, the optimal optimal move. What we rely on is that the computer is better than the players and that's true pretty much across the board. The days of Deep Blue, when it was a competition between powerful chess computers and world champion chess players - those days are pretty much over.

The top chess players in the world have ELO rating of 2800 and the chess computer that we were using has an ELO rating of over 3300. It's true that the chess computers aren't necessarily the optimal and we test for sensitivity to exclude these very very good chess players who might actually have done better than the computer. So we drop those players with the ELOs at the very top of the distribution. We find the same result because those are the players that might have a chance of outthinking the computer.

But in general, given how much better the computers are at chess than human beings now, we feel safe in making the argument at least that the computer's move is superior or at least as good as the human move in basically every position.

Kyle: I think it was about a year ago, I had a chance to interview Kenneth Regan who looks for cheating in chess. Players that might be using one of those algorithms to make their move selections and he was able to find some cases and essentially the chess algorithms in some situations behave differently than people and that was measurable, statistically speaking.

I don't think that's a criticism of your method because if it diverges in that way, it ought to effect both genders equally. But what is interesting, that and I discussed, was how you account for when a move is obviously optimal versus situations like opening moves. There's a lot of different opens and there are some preferences to them, but there's likely to be a lot of differences between the algorithm and the player early on. Can you account for that in your analysis?

PETER: Chess games can be broken down to The Opening, The Middle Game and The End Game. You're absolutely right, The Opening of chess games at this level at least are choreographed (that's the wrong word) but (what I mean is) the first sort of 4 or 5 moves will be done very quickly. Players would've studied what their opponents does usually, what they like to open with. They'll prepare various responses to various versions of those openings and there's actually this thing called The Encyclopedia of Chess Openings, which is this big encyclopedia of various openings, how you counter them, what opportunities it gives you and what the cost is of that opening versus other might be. So what we do is we exclude the first 15 moves of the game.

To be safe that we're kind of leaving behind these more choreographers and we also exclude anything over 30 moves. So we look at The Middle Game, which is 15 to 30 moves. We do that because when we did it for all of the moves, you're actually right, towards the end of the game, a player might choose a sort of very sub-optimal move because they're behaving in a very defensive way perhaps or they're sort of trying to get a draw rather than trying to win.

Those two things, The End Game and The Opening are more complicated we thought. So we just take the middle of the game - where computational intensity is at its height and it's about figuring out how to best position yourself to set up the end game that you want - because it was the most important part to look at. It's sort of the safest part to look at since it avoids some of the other issues in the opening and the end game.

Kyle: So let me see if I've got this right.

It sounds like you guys are using it as a measure of inner-game- play. That's or less than a score for each game of how well a individual player did against what your expectations would be and then you can look at that score conditioned on the gender of both the players to see how inter- and intra-competitions work, is that basically the framework?

PETER: Yeah exactly, so we condition on the gender of the opponent as well as the ELO ratings of the two players. So we're holding that ability consent as well as the individual effect of the player that we're studying.

Kyle: Got it, so let's get into some results. What do you see in inner gender games, people playing someone of the same gender?

PETER: If you look at male, male games and female, female games the error, so the quality of the playing metric, that you'd call the quality of the player that error which is the bigger the error the worst the person has played. Those two are about the same, once you control for ability. They don't seem to differ if you're playing someone of your own gender (conditional ability).

Kyle: And what about cross gender play from both the male and female perspective?

PETER: So that's where things get interesting/exciting.

In addition to the main result that we find that women perform worse in terms of outcomes, that's why they lose more often when they play against men.

We find that woman's quality of play, so their errors are bigger when they play against the man but a man's quality of play doesn't vary with the gender of his opponent. It's not that men are somehow extra focused or pursuing more optimal strategies when they play against a woman. Their quality of play stays the same regardless of the gender of their opponent but women perform worse in terms of the quality of play within the game. They make more errors; they make more mistakes.

Kyle: So, when I started reading this, one hypothesis I was tossing around my head is could it be that - I can't say all women but a large enough percentage of women, so that there's a statistical phenomenon - play better for like the long haul. They're more willing to take draws or something like that, where as a man would risk having a loss and his record for the potential of a win.

But if that were true, we ought to see that when you condition for gender, so it seems to be that that can't possibly explain this result. Do you have any thoughts on, I guess this is a 6 million dollar question: what's the origin of that result or what causes that to be the case?

PETER: That's a good question. (Laughs)

So we have a couple of ideas. The thing we say is consistent with, it's not necessarily evidence of, but it's consistent with something called stereotype threat, which I don't know if you're familiar with.

Kyle: Let's go into it.

PETER: It's a phenomenon that's been studied in Psychology for a long time. I'm sure people would've heard of it in terms of, if you look at high school, girls do worse on math exams than you might expect them to do. So they are consistently doing worse than boys but some research has been done where, if you give girls a math exam and you tell them alright this is a gender neutral math exam, they catch up to boys, not necessarily all the way but their scores get a lot closer to boys. Just that simple prime to say, this is a gender neutral. You can do okay on this one.

You also see it in reading tests with African Americans versus white Americans. The idea of stereotype threat is that under these sort of conditions, like a test condition, the stress or the pressure of a negative stereotype about your own person causes you to sort of live up to that stereotype. There are different ways of thinking about it. (For example,) someone's more nervous or maybe a girl is thinking that girls are bad at math so she takes the test and she does worse on the exam and she otherwise would have. She wasn't worried about girls are bad at math and I'm a girl, not mean I'm going to do badly on this exam.

So that's sort of one explanation for what we're seeing in the chess which is a male dominated environment. It's 85-90% male at this very high level of chess play. We're looking at extremely good chess players throughout the paper. You saw the quotes and at the beginning of the paper that there's a very sexist environment and there's a common misconception that women are worse at chess than men, even though we don't see any evidence of that in the data.

So it might just be when they're sat across the table from a man, they start to think subconsciously. This isn't something that these players are choosing to do but somewhere in there brain it's going, 'I'm a woman and women are bad at chess' and that distraction, that eats some "RAM" and they are not able to commit the mental resources necessarily to the game at hand and so they end up making more mistakes.

Now, our results are consistent with that idea, but they're not a direct evidence of it. We're not able to see what's going on in brains but that's one thing, we think explains some of the results.

Kyle: Yeah, that makes a lot of sense.

I had also been tossing around the idea of, from your quotes and some of the vitriol and surprising vitriol coming out of even famous players like Bobby Fisher. I thought, maybe there could be some social pressure that someone who thinks that way also would find it beyond humiliating to be beaten by women and they might play more aggressively or be less likely to concede. Do you any phenomenon like that, that could be part of the explanation?

PETER: So to play more aggressively, I would say, we don't see evidence of that because we don't see the quality of men play depend on their gender of their opponent. So we don't see that men are playing more aggressively. There's been some other work where people say they've find men playing more aggressively but we have doubts about some of that works, mainly having to do something with P-value interpretation, which I'd love to come back to your show and talk about it. So we don't find evidence of that but we do find evidence that behavior of men does change and it's exactly what you said.

Men seem less ready to concede to a female opponent. And the way to understand that is, at this level of chess, so we have tens of thousands of games, only about 1-2% of these games actually end in checkmate, which is how most people probably think chess ends. For this level of player, they don't play all the way to checkmate. They're good enough and they understand board position well enough that they know when they've lost. They can see 5 or 10 moves ahead maybe like, "okay I'm in a terrible position. I've lost a game. I'm going to concede. So I offer to my opponent, okay I concede. You won, fine."

What we can see in our data is the number of moves in a game. In games that were conceded, how quickly did a person concede that game? After how many moves did the person concede, conditional on having conceded in the first place? What we observe is that men will continue to play longer against a female opponent. The man is going to lose to the woman, but it takes him longer to concede relative to when he's playing against a male opponent.

We build a little, tiny toy model within the paper just to try to explain why that might be. If there's an additional cost to losing to a woman, which in this very sexist, male dominated environment where women are seen as inherently inferior players, I think it's safe to argue that men do not like to lose to women.

If you want some vitriol, go on some of the message boards on chess.com and these places and talking about co-author, Matthew, he's a very accomplished chess player. He said, "absolutely." Like men will do whatever they can not to lose to a woman. It's embarrassing for them. In addition to losing the game, which is costly in terms of your ELO points, in terms of just losing which isn't a pleasant experience for anybody, but for a man to lose to a woman there's this additional cost, this additional psychic cost that I've lost, but I've lost to a woman.

If that's the case, we expect to see them play to a worse position. So they might be at say move 50, they might go, "Okay I definitely, like 98% sure I've lost this game. If I was playing against a man, I would concede, but because there's this woman and if I lose to her, I get this extra cost against me, I'm going to just keep playing so that I'm 99% sure that I've lost the game."

So they exert their extra effort to play those last couple of moves and extend the game a little bit more when playing against a woman. Now the interesting thing about the change in their behavior, the men's behavior and women's behavior, both genders change their behavior but they both change their behavior that negatively have outcomes for women. Both of those change in behavior, help explain the initial effect that we measure which is women lose more when they play against a man conditional, on their ability.

Kyle: So, I haven't yet asked about the size of the effect. Is there a way you have to quantify this decline in quality of play?

PETER: We do and that's great question.

So anytime, anybody is reading an empirical work, significance is great. We're all used to seeing one star, two start, three stars ' 1%, 5%, 10% - whatever it is, but magnitude is super, super important because it might be statistically significant but it might not be economically significant.

Our effect is small it's not enormous. It's equivalent to about a 30 ELO point differential, an additional 30 ELO point differential. When a woman plays against a man, it's as though the ELO difference between them was plus 30 points in favor of the man, which is not insignificant and I got the paper in front of me.

I'm trying to look at what the - I think that's like a 3 points difference in the expected outcome of the game if winning is a 100 say and losing is 0. It's about that. It's not huge effect, but it's big enough to matter. I think it's big enough to be a concern, 30 ELO points is not nothing.

Kyle: Yeah if it's statistically significant, which it seems to be and it's non-trivial then it's absolutely worth looking at. For a little bit more reference maybe, I know you quoted some of the ELOs of grand masters and whatnot earlier. Could you give me a ballpark, who's a grand master and what would I be if I went to play chess for the first time? What would my ELO start at?

PETER: I think everybody's ELO starts at 1000, I think it all starts at a 1000.

The grand master, someone like Magnus Carlson, who's a Norwegian chess prodigy, is the current world champion. I think he's still the world champion. He has the highest ELO ever recorded which was 2800-something it was and we'd start at 1000 and getting that high would take a long time and you'd have to beat a lot of very good players to do that.

So if you beat someone, let's just say a little bit worse than you, then you might improve your ELO by 1 or 2 points. It's a long slog to get up to the ELO rating that's that high. And I can tell you, I've played my co-author who's ELO is 2100, 2200 something like that and that game lasted about 10 moves. I don't play chess at all, it was shocking that the difference in ability and understanding. It was a friendly game but it was over very quickly, so being good at chess makes a big difference. (Laughs)

Kyle: There was one result, I think this came out of another study, but you looked through it and for your paper, and I wanted to touch on because I thought it was interesting. Someone went and studied what happens when you're playing online in which you don't necessarily know the gender of the player but then revealing or revealing incorrectly the gender of the other player had, well how did something like that effect these types of results?

PETER: Right, so that's great study and that was one of the things that sort of motivated us, when we first started thinking about this because you first start thinking about women and competition and then came to chess.

That study by Mass Detole and Catinue 2008 - which I think they're psychologist and they weren't economists but - there's a lot of overlap in all of our work anyway. They did this sort of lab experiment, which is exactly what you said; they had people playing chess online. You were playing blind so you couldn't see the person that you were playing against. They changed randomly telling people whether they're playing against man and it really wasn't a man, or maybe they'd tell you that you're playing against a man and it actually would've been a woman, or you're playing a woman and it would've been a man, and so on and so forth.

Using that random variation from within this lab experiment they find exactly what we find, which is that women perform worse when they think they're playing against the man. But they don't find that women play worse when they're playing against a man but they think that they're playing against a woman, which is sort of consistent with our interpretation and that this stereotype threat thing might be important. It's the fact that you sat across a man. It's not the fact that you're playing a man, so their results are great, really well done, lab experiment, and I think our results support each other.

Kyle: I'm familiar with ideas of looking in to labor markets where of course, these topics are much more impacting on people's everyday lives. Has this, I guess the difference from there to here is that you have a ton of more controls in place. So if you can get down to saying stereotype theory is consistent with our analysis, we've eliminated all these other confounding factors, perhaps stereotype theory explains it even in lieu the other factors. Is that community of people looking at things like, labor relations and promotions and corporate America? Are they approaching the same sort of conclusions or are there any other competing theories that they have?

PETER: So there are other theories about sort of innate gender preferences. I shouldn't say 'innate' but culturally determined gender preferences for competition. Some people might say that women just don't like competition or women are more nurturing and so they don't want to be in a competitive environment, which I think is wrong. I think it's misguided and there is evidence that suggests that it is in fact a wrong interpretation.

But even something like stereotype threat is going to be culturally determined in terms of what's actually being done in labor market is being done for promotions decisions, hiring decisions. We hope that studies like ours will help convince people that they need to think about these things when developing their hiring strategies, developing their promotions and system of promotions.

It's something that I'd bring up at my own University. All the time in economics, there's a huge under-representation of women. My own university is particularly guilty of this and this not necessarily because we're a bunch of sexist people. When we go hire but everybody has biases and those biases result in the outcomes that we see. So we held the evidence like this, helps inform this debate and helps inform the discussion that when you're making hiring and promotion decisions, you need to consider this sort of things.

So if you're going to create a competitive environment within your office or your firm, you need to be aware that men and women could be responding differently to that. You might see men rise the corporate ladder faster but not necessarily because they're productive or because they're better employees but because they respond differently to competitive environments than women.

Kyle: I think it would be hard to blind something in the hiring process, maybe early on but certainly in terms of the performance, you know the people you're interacting with. But maybe for our topic of chess. Is there a way we could put some mechanism in place to eliminate or defend against this observed quality performance issue?

PETER: The obvious thing to do is to do exactly what they did in that lab experiment in that paper that you mentioned, which is to have all players play blind where you just don't know who you're playing against.

The advantage to that is you'd eliminate this gender effect that we find. The disadvantage of that, and why chess players won't go for it, is that players prepare for each other. Players want to know whom they're playing against. Particular players have particular styles. They have particular openings that they prefer and so to go and play a game of chess where you have no idea whom you're playing would be a very different world than the world of high-level chess, as it is where, you drill and you train and you prepare for your opponents, not necessarily just for an opponent.

What we might be able to do is maybe encourage blindness at lower levels. So when you're getting girls and boys, younger people into chess, we could have those tournaments be blind. They're not quite at the level where they're studying and preparing for an individual or particular opponent, just sort of learning how to play chess and developing as chess players. Could we have those be blind so that we don't lose - women don't drop out as they see they're not performing as well as maybe they'd like. So they end up not playing at all, like they don't pursue chess careers or even chess as a hobby.

That might help this and you might see that diminished effect later on once women realize, "oh maybe this stereotype is reduced" as they've grown up in an environment where they've been playing blind. They know they can play men and women equally well and as maybe as they get older, we'd see a diminished effect but we don't have evidence of that on our own paper but that will be one approach.

They've done it for orchestra trials; it's a very famous example. There were biases in hiring people for orchestras. Orchestras are very male dominated, so they had them play behind the screens and all of a sudden the number of women spiked because women are just as good at playing instruments as men.

Kyle: Yeah that was a really dramatic result, if I'm not mistaken.

PETER: Yeah and overnight. I mean it was that simple of a thing. So you might see, we have a smaller effect because we're not looking at hiring, we're not looking at selection. We might, if there's a way of economic job market, do it better where we could do it blind but it's hard of think how you'd do it in practical terms.

Kyle: Yeah. Well, Peter is this the sort of pinnacle of this line of inquiry or this is a stepping-stone? Do you have more in this direction coming forward?

PETER: So this is definitely a stepping-stone. There's a lot of very bright people, working in this area trying to understand gender and competition and various dimensions of it. We have a companion paper; we've started working on it now, which is looking at gender differences in expectations of performance.

So, going into a tournament or going into a particular game, a player will have an expectation of how well they will do and we're interested in understanding whether there's gender differences in the relationship between the actual performance and the expectations of performance.

Kyle: Yeah, so if listeners wanted to check out this paper or keep track of your work in general. Where is the best place to go online?

PETER: I am on Linkedin or the University of Manchester's website. You just Google my name and the University of Manchester is the first thing that pops up. All my papers are there, working papers and published papers are all there.

Kyle: Great I'll have those links in the show notes for anyone that wants to follow up. Well Peter I want to thank you again for coming on and sharing your work and talking me through this. It's an interesting paper and I hope this gets more exposure and keeps the conversation going.

PETER: Yeah me too. Thanks for having me on. It was really good to talk about it.

EXIT VOICE-OVER: For more on this episode, visit www.DataSkeptic.com. If you enjoyed the show, please give us a review on iTunes or Stitcher.