Winboard Forum

Here is another stat study from me. This time I wanted to see the connection between having the white pieces and elo values.

Using Scid, ChessCollect and ThisWeekinChess, I put together a database of 198005 games involving human players of elo 2400+. The games were played from Jan 01, 1991 up through March 2005. Short games (20 moves or less), computer, Internet, rapid, blitz, blindfold, simultaneous were eliminated. Most importantly, duplicates and twins were eliminated using Scid and pgn-extract. Each game had a result and both white and black elo values. I used my utility elopgn.exe (available for download at crafty-chess.com/down/Pollock ) to gather the stats.

Here is the statistical data:

all games 198005
white elo average = 2477.9975
black elo average = 2474.0227
average difference (white - black) = 3.9748

games won by white = 73623
white elo average = 2498.2800
black elo average = 2445.5071
average difference (white - black) = 52.7729

games drawn = 73651
white elo average = 2484.4999
black elo average = 2488.8232
average difference (white - black) = -4.3233

games won by black = 50731
white elo average = 2439.1952
black elo average = 2493.9186
average difference (white - black) = -54.7234

Interpretation of Data:

This is very subjective.

White has on average a 4 point elo advantage over all games. Considering the data is close to 200,000 games, this is a very large and surprising advantage. Tournament directors favor the better player by giving him white more often. What is the justice in that?

In white wins, he has on average a 53 point elo advantage. When black wins, he has a 55 point elo advantage. Considering that black also has to overcome an initial 4 point disadvantage, black has to increase his performance on average by 59 (55 + 4) elo points to win. Since white has a 4 point advantage to start on average, he has to increase his performance by 49 points (53 - 4) to win. Since black has to increase his performance on average by 59 points, and white only has to do likewise by 49 points, having white is worth 10 elo points when we consider wins.

Now let's look at the draws. The average elo of black is 4.3 points higher than the average elo of white in drawn games. Considering that black starts out 4 points lower than white, black has to increase his performance by 8.3 elo points to draw with white. Likewise, white can still draw with black even if he performs 8.3 points less than usual. Therefore having white is worth 8.3 elo points when we consider draws.

Putting wins and draws to together, I conclude that having white is equivalent to increasing one's performance by approximately 9.3 elo points.

The elos in the study were of players 2300+.

Hi Norm,
interesting study. Is it possible to have this large collection of games?
Thanks.

Ubaldo Andrea

Ubaldo Andrea Farina wrote:Hi Norm,
interesting study. Is it possible to have this large collection of games?
Thanks.

Ubaldo Andrea

I will be putting it up on the above mentioned site in a few weeks. I will call it normbk03. Right now you can download normbk02 which about 1/3 the size.

Hi Norm,

maybe I think here a bit wrong, but I initially thought that the advantage of white is a bit higher and would guess it by an easier way:

afaik white has around 54% usually... according to my table this indicates a difference of 26-32 Elo...

Am I doing something wrong here ?

Greets, Thomas

Norm Pollock wrote:
White has on average a 4 point elo advantage over all games. Considering the data is close to 200,000 games, this is a very large and surprising advantage. Tournament directors favor the better player by giving him white more often. What is the justice in that?

Or maybe the stronger players are smarter and more capable of converting the advantage of playing with white to a win (As one might expect).

Pallav

Let me take a new look at the stats. Here are the stats for the full pgn file:

number of games in coffee.pgn is 198005

number of games with a result is 198005

number of games without a result is 0

elo range is 2300-2851

number of games with white elo is 198005

number of games with black elo is 198005

number of games with BOTH white & black elo is 198005

average white elo value is 2477.9975354157723 for 198005 games

average black elo value is 2474.0227469003307 for 198005 games

maximum elo distance is 445 for 198005 games

average elo distance is 89.93354713264817 for 198005 games

number of white wins is 73623 ( 37.1823943839802 % )

number of draws is 73651 ( 37.19653544102422 % )

number of black wins is 50731 ( 25.62107017499558 % )

white score is 55.78066210449231 %

black score is 44.21933789550769 %

-------------------------------

Let me start over with the interpretation. A person can look at stats 100 times, and come out with a different interpretation each time.

There seems to be a big paradox. The elo difference between white and black is only 4 elo points in favor of white. Yet White scores 55.8% to Black's 44.2%. I don't have the elo table conversion (where can I find one) so let me assume that the results indicate a difference of approximately 50 elo points. Shouldn't a difference of only 4 elo points result in something like 50.5% to 49.5% instead of 55.8% to 44.2%

The answer to this paradox seems very easy to explain. There is a basic flaw in elo ratings. Elos are based on a "combined" white+black score. If I have an elo of 1500, in reality it is a combination of perhaps 1525 as white and 1475 as black. If players were given dual elos (one for white, one for black) we would not see the above paradox.

Assuming there were dual elos, the average black elo above would be let's say 2449 (25 points less), and the average white elo would be 2503 (25 points more). The difference of 54 points (2503-2449) would explain the outcome scores: 55.8%-44.2%.

I think there is a need to go to dual elos to accurately reflect a players strength since the same player could be quite stronger as white.

Norm Pollock wrote:
snipped

I think there is a need to go to dual elos to accurately reflect a players strength since the same player could be quite stronger as white.

Hello Norm,

A startling and original conclusion.

I can imagine the logical difficulties - I am the same player so my Elo should be the same to if I get the black pieces more and I win, do I gain more points?

Dual Elos may more accurately reflect the realities of a tournament player's performance but the practical implementation and consequences present formidable difficulties in my opinion.


However the ultimate measure of a plyer's strength is competence with both pieces. It seems hardly likely that there will be numerous cases wherein a 2600 player has a huge discrepancy between the performance with white as against black.


Having said that, I seem to recall that even among GM's, Victor Korchnoi had an exceptional performance with black.

As always Norm, I find your writings fascinating.

Later.

Ps. Eagerly awaiting the next update of your database!

Norm Pollock wrote:
There seems to be a big paradox. The elo difference between white and black is only 4 elo points in favor of white. Yet White scores 55.8% to Black's 44.2%. I don't have the elo table conversion (where can I find one) so let me assume that the results indicate a difference of approximately 50 elo points. Shouldn't a difference of only 4 elo points result in something like 50.5% to 49.5% instead of 55.8% to 44.2%

White player moves first. That is an advantage. Master opinion is that right to move is an advantage nearly equal to 1/3 of a pawn (perhaps 40-50 ELO). That would explain this apparant paradox quite well.

Pallav

Pallav Nawani wrote:

Norm Pollock wrote:
There seems to be a big paradox. The elo difference between white and black is only 4 elo points in favor of white. Yet White scores 55.8% to Black's 44.2%. I don't have the elo table conversion (where can I find one) so let me assume that the results indicate a difference of approximately 50 elo points. Shouldn't a difference of only 4 elo points result in something like 50.5% to 49.5% instead of 55.8% to 44.2%

White player moves first. That is an advantage. Master opinion is that right to move is an advantage nearly equal to 1/3 of a pawn (perhaps 40-50 ELO). That would explain this apparant paradox quite well.

Pallav

The stats show a 55.8% to 44.2% advantage.

Here is the table for elo differences translated scores

Rating expectancies vs. differences P D P D P D
.99 677 .66 117 .33 -125
.98 589 .65 110 .32 -133
.97 538 .64 102 .31 -141
.96 501 .63 95 .30 -149
.95 470 .62 87 .29 -158
.94 444 .61 80 .28 -166
.93 422 .60 72 .27 -175
.92 401 .59 65 .26 -184
.91 383 .58 57 .25 -193
.90 366 .57 50 .24 -202
.89 351 .56 43 .23 -211
.88 335 .55 36 .22 -220
.87 322 .54 29 .21 -230
.86 309 .53 21 .20 -240
.85 296 .52 14 .19 -251
.84 284 .51 7 .18 -262
.83 273 .50 0 .17 -273
.82 262 .49 -7 .16 -284
.81 251 .48 -14 .15 -296
.80 240 .47 -21 .14 -309
.79 230 .46 -29 .13 -322
.78 220 .45 -36 .12 -335
.77 211 .44 -43 .11 -351
.76 202 .43 -50 .10 -366
.75 193 .42 -57 .09 -383
.74 184 .41 -65 .08 -401
.73 175 .40 -72 .07 -422
.72 166 .39 -80 .06 -444
.71 158 .38 -87 .05 -470
.70 149 .37 -95 .04 -501
.69 141 .36 -102 .03 -538
.68 133 .35 -110 .02 -589
.67 125 .34 -117 .01 -677

From the table (see the red numbers), a score of 55.8% corresponds to an elo difference of 41 elo points. Let's just say 40 to use a rounded number.

This means that the "white elo" of the players playing white is 40 points higher than the "black elo" of the players playing black.

However the "full elo" of the players playing white is only 4 points higher than the "full elo" of the players playing black. There is a discrepancy of 36 elo points. Splitting it in half, we get 18.

In this database, the average "full elo" of the white players is 2478. And the "full elo" of the black players is 2474. To account for the 55.8% score of white vs the 44.2% score of black, we have to realize that the "white elo" of white is 40 points greater than the "black elo" of black. The adjustment number is 18 as computed above. Therefore, add 18 (on average) to the "full elo" to get the "white elo" (on average) for the white players, and subtract 18 from the "full elo" to get the "black elo" for the black players. This will result in a difference of 40 elo points (2496 vs 2456).

Pallav mentions that the difference between a player's average white elo and his average black elo is estimated in the range of 40-50 points. My stats calculate it to a difference of 36 points because we add 18 to the "full elo" to get the white player's white elo, and we subtract 18 from the "full elo" to get the black player's black elo.

Another point Pallav mentioned is that better players are able to take better advantage of being white. This likely is true. When I looked at white scores restricted to higher elos, it does seem that white's advantage increases. However I would have to do another study in depth to confirm the stats.

Getting back to my suggestion of a dual elo system. On average we could adjust the "full elo" of a player by +18 or -18 to get the "white elo" and "black elo" respectively. However some players do not fit the average mold. A dual elo system (actually a tri-elo system: full, white, black), might say that player A has a full elo of 2500, a white elo of 2540 and a black elo of 2460. Or it might say that player B has a "full elo" of 2500, a "white elo" of 2490 and a "black elo" of 2510. Another way to represent this to say a players elo is "2500, 25" which would indicate an adjustment of +25 for white elo, and -25 for black elo.

Btw, I'm assuming each player plays an equal number of games as white as he does as black. However the top super gms are given white more often than they are given black. This is a common practice in Swiss tournaments.

(this post was edited)

Hi Norm!
Now , how do you explain that some players have a ratio
White Wins / Black Wins far bigger than 55/45 , and others , like me , score more Black wins than white ones?
And all that is combined in overall statistics!
Statistics are not so simple!

Claude Le Page wrote:Hi Norm!
Now , how do you explain that some players have a ratio
White Wins / Black Wins far bigger than 55/45 , and others , like me , score more Black wins than white ones?
And all that is combined in overall statistics!
Statistics are not so simple!

Hi Claude,

The statistics here look at the averages, not individuals. My study was about long time-control games by players with ratings 2300+ since 1991. The study concluded that an average player in this population, who has played an equal number of games with white as with black, has a black elo 18 elo points less than his full elo, and has a white elo 18 elo points greater than his full elo.

Another way to look at a player's full elo is to think of it as the weighted average of the player's white and black elos.

Example: Suppose a player plays 1000 games as white and has a white elo of 2400. The same player plays 1500 games as black and has a black elo of 2200. The weighted average is
= (1000*2400 + 1500*2200)/(1000+1500) = 2280
This player has a white elo (2400) 120 points above his full elo (2280), and a black elo (2200) 80 below his full elo.

Getting back to your inquiry. Some individual players' data will differ substantially from the averages. Their data does not contradict the averages. They are "outlier" results. Like Bill Gates' income in a population of all US males, age 21+.

Claude Le Page wrote:Hi Norm!
Now , how do you explain that some players have a ratio
White Wins / Black Wins far bigger than 55/45 , and others , like me , score more Black wins than white ones?
And all that is combined in overall statistics!
Statistics are not so simple!

Hi Claude:
I don't score many wins with either color, but, for the few I do score, there are at least as many with black as with white - if not more. But I do know why. First, I play so poorly that the color doesn't matter much. Years ago I played the Sicilian in response to 1 e4. So did everybody else, and everybody else played it better than me. So I cast around for a new defense and came upon the Petroff. Karpov, the then world champion played it so it couldn't be all bad. I came to like that defense. I can't say I play it well, but I don't play it as badly as I play everything else.

Hoping your reasons for scoring better with black are different than mine
Dan H.

Hi Norm!
Long ago , probability and Statistics were a fair part of my profession ;
As I was interested in Psychometrics , I had occasion to think about the problems of "human statistics"
Roughly , Chess statistics are composed of 2 parts :
First is relative to objective strength of players , such as answers to a battery of test positions : it obeys to probability laws exactly as physical phenomena
Second is related to human decision , and I doubt whether they are "probabilisable" : just a simple example :
I am correspondence player ; at the beginning of a tournament , I can't guess what will be the first move of my opponents , and they were unable to guess mines : it's a decision
As for me , in some tounaments , I decided to open all my games with 1 e4 , but in others I varied my first move between e4,d4,Nf3 , c4 or g3
You surely agree that choice of the first moves has an influence on the outcome of games
As for me , I played almost exclusively with Black Open Ruy Lopez on e4,
and KID on anything else : on both I had become an expert , so I scored
heavily with Black
Others have a wide repertoire and don't score so well with Black
Then you may observe that this ratio White wins/Black wins vary with time , and with level of tournaments
150 years ago there were very few Black wins , chiefly in king's gambit , but now it's almost abandoned ,as too favourable to Black
You see , it's not so simple

Claude Le Page wrote:Hi Norm!
Long ago , probability and Statistics were a fair part of my profession ;
As I was interested in Psychometrics , I had occasion to think about the problems of "human statistics"
Roughly , Chess statistics are composed of 2 parts :
First is relative to objective strength of players , such as answers to a battery of test positions : it obeys to probability laws exactly as physical phenomena
Second is related to human decision , and I doubt whether they are "probabilisable" : just a simple example :
I am correspondence player ; at the beginning of a tournament , I can't guess what will be the first move of my opponents , and they were unable to guess mines : it's a decision
As for me , in some tounaments , I decided to open all my games with 1 e4 , but in others I varied my first move between e4,d4,Nf3 , c4 or g3
You surely agree that choice of the first moves has an influence on the outcome of games
As for me , I played almost exclusively with Black Open Ruy Lopez on e4,
and KID on anything else : on both I had become an expert , so I scored
heavily with Black
Others have a wide repertoire and don't score so well with Black
Then you may observe that this ratio White wins/Black wins vary with time , and with level of tournaments
150 years ago there were very few Black wins , chiefly in king's gambit , but now it's almost abandoned ,as too favourable to Black
You see , it's not so simple

Hi Claude,

The population I used are players with 2300+ elo starting in year 1991. These players I am sure choose their openings with care and after a great deal of study and practice. I am sure they used computer study and practice as well as book study and human practice. The point is, this population is very unlikely to choose openings on a random basis. It is likely that each player was very familiar and knowledgable about each opening he chose. You do not become a 2300 player by random choice of openings.

So as I see it now, choice of openings is not relevant to the study. Btw, the openings were distributed as follows:
A00-99: 40,661
B00-99: 63,341
C00-99: 29,712
D00-99: 33,300
E00-99: 30,991

I will make this database publicly available shortly.

-Norm

Hi Norm!
I don't discuss the care and the serious of your study , but you neglect psychological factors that for me are more important than the advantage of move
Many players , chiefly OTB , have a "complex" when they have Black :
they are content to draw or , if they need to win , they embark on risky lines
As I told you , these factors are difficult to compute in terms of probabiliy , but I think they are most important
I have made thematic tournaments between engines , starting with positions held as slightly favourable to White , as the McKenzie variation
(Smyslov-Reshevsky of cable match 1950 ) : OTB it's favourable to White , but between engines Black has an edge
In the same way ratio W/B is nearer from 50/50 in postal games , because it's possible to find in Chess books or databases an equalizing line
As I told you King's Gambit or Evans Gambit don't score as heavily for White as in past time , but Black continue to choose lines difficult to crack , but with which it's more difficult to win : e-g closed defences of Ruy Lopez
Conversely , when I play with Black a KID or a Dilworth , I am confident ,
and so I score more than 50%

Claude Le Page wrote:Hi Norm!
I don't discuss the care and the serious of your study , but you neglect psychological factors that for me are more important than the advantage of move
Many players , chiefly OTB , have a "complex" when they have Black :
they are content to draw or , if they need to win , they embark on risky lines
As I told you , these factors are difficult to compute in terms of probabiliy , but I think they are most important
I have made thematic tournaments between engines , starting with positions held as slightly favourable to White , as the McKenzie variation
(Smyslov-Reshevsky of cable match 1950 ) : OTB it's favourable to White , but between engines Black has an edge
In the same way ratio W/B is nearer from 50/50 in postal games , because it's possible to find in Chess books or databases an equalizing line
As I told you King's Gambit or Evans Gambit don't score as heavily for White as in past time , but Black continue to choose lines difficult to crack , but with which it's more difficult to win : e-g closed defences of Ruy Lopez
Conversely , when I play with Black a KID or a Dilworth , I am confident ,
and so I score more than 50%

As I understand you point of view, you are saying that in your opinion, the major reason for white's superiority is black's psychological attitude. Certainly that is part of the overall explanation. Whether it is the main reason, or whether white going first, or some other explanation is the first reason, is hard to determine from statistics. Interpretation and explanations are very subjective.

Norm Pollock wrote:The study concluded that an average player in this population, who has played an equal number of games with white as with black, has a black elo 18 elo points less than his full elo, and has a white elo 18 elo points greater than his full elo.

Another way to look at a player's full elo is to think of it as the weighted average of the player's white and black elos.

Example: Suppose a player plays 1000 games as white and has a white elo of 2400. The same player plays 1500 games as black and has a black elo of 2200. The weighted average is
= (1000*2400 + 1500*2200)/(1000+1500) = 2280
This player has a white elo (2400) 120 points above his full elo (2280), and a black elo (2200) 80 below his full elo.

Hi Norm,

I believe there is kind of a formal error in your way of separating "white elo" and "black elo". If it were possible to have such a rating system then you would not be able to compare these ratings, for the following reason:

a) You cannot compare a "white elo" to a "black elo", for obvious reasons.
b) You cannot compare any "white elos" among each other because there are no games where both players have white. The same applies for "black elos".

All "white elos" would have to be derived from games of players having white against other players having black. But which elo of such a black opponent to take for calculation of a new white elo? I think it is simply impossible, it hurts the whole rating system.

Just my two cents!

Sven

Sven Sch?le wrote:

Norm Pollock wrote:The study concluded that an average player in this population, who has played an equal number of games with white as with black, has a black elo 18 elo points less than his full elo, and has a white elo 18 elo points greater than his full elo.

Another way to look at a player's full elo is to think of it as the weighted average of the player's white and black elos.

Example: Suppose a player plays 1000 games as white and has a white elo of 2400. The same player plays 1500 games as black and has a black elo of 2200. The weighted average is
= (1000*2400 + 1500*2200)/(1000+1500) = 2280
This player has a white elo (2400) 120 points above his full elo (2280), and a black elo (2200) 80 below his full elo.

Hi Norm,

I believe there is kind of a formal error in your way of separating "white elo" and "black elo". If it were possible to have such a rating system then you would not be able to compare these ratings, for the following reason:

a) You cannot compare a "white elo" to a "black elo", for obvious reasons.
b) You cannot compare any "white elos" among each other because there are no games where both players have white. The same applies for "black elos".

All "white elos" would have to be derived from games of players having white against other players having black. But which elo of such a black opponent to take for calculation of a new white elo? I think it is simply impossible, it hurts the whole rating system.

Just my two cents!

Sven

Hi Sven,

For comparison of 2 players you would still take the full elo (the weighted average of the white and black elos) as you do now.

The practical uses of a full/white/black elo system are:
(1) You can tell if a player excels with a particular color, or on the other hand, if the color does not matter.
(2) You can better predict the outcome of a single game.

It would be impossible to have a "true" White-side World Champion because you cannot have a match where both players are playing white. But you could say that one player is the best in the world with white, and another is best in the world with black, and a third is the best overall.

-Norm

The above seems pretty consistent with this (white v black advantage outlined in the last few paragraphs):

http://mywebpages.comcast.net/danheisma ... alance.htm

Brian