Winboard Forum

Hi,

I'm doing a statistical study of chess results. In order to ensure fairness that each sample not only has approximately equally strong players, but also has on average, equally strong players playing White as it has playing Black, I need a special utility program. Scid doesn't do it.

I am therefore looking for the following type of utility program.

I need a pgn utility program that can look at a pgn file, extract the elos for white and for black, and can then calculate the average White elo and the average Black elo.

-Norm

Remi is working on a nifty new Elo calculation routine called BayesElo.

Maybe he could put in a feature to calculate Elo, and standard deviation of Elo for white and black player.

Dann Corbit wrote:Remi is working on a nifty new Elo calculation routine called BayesElo.

Maybe he could put in a feature to calculate Elo, and standard deviation of Elo for white and black player.

I'm not looking for an elo creator like elostat. I want something that extracts the existing elos from the tags in the pgn file, then averages the white elos and black elos separately.

Norm Pollock wrote:

Dann Corbit wrote:Remi is working on a nifty new Elo calculation routine called BayesElo.

Maybe he could put in a feature to calculate Elo, and standard deviation of Elo for white and black player.

I'm not looking for an elo creator like elostat. I want something that extracts the existing elos from the tags in the pgn file, then averages the white elos and black elos separately.

SCID *can* do that.

Header Search form,
Rating for white,
Rating for black.

Make them anything you like.

If you make it a narrow band, it will give those limited ranges.

You could also pull IM data, then GM data, then SuperGM data by using 100 Elo wide ranges.

Dann Corbit wrote:

Norm Pollock wrote:

Dann Corbit wrote:Remi is working on a nifty new Elo calculation routine called BayesElo.

Maybe he could put in a feature to calculate Elo, and standard deviation of Elo for white and black player.

I'm not looking for an elo creator like elostat. I want something that extracts the existing elos from the tags in the pgn file, then averages the white elos and black elos separately.

SCID *can* do that.

Header Search form,
Rating for white,
Rating for black.

Make them anything you like.

If you make it a narrow band, it will give those limited ranges.

You could also pull IM data, then GM data, then SuperGM data by using 100 Elo wide ranges.

It will put those games in that elo width into the filter. But how do I get the average white elo, the average black elo?

Suppose there are 500 games with white elo between 2450 and 2650. I want the average elos of those games by color. I could manually go through each game and manually record the white elo and black elo, and then manually average them. But I don't want to do it manually because some of the filters will have thousands of games.

Get the program I made called Manave:
ftp://cap.connx.com/pub/chess-engines/n ... MANAVE.exe

Here is the source if you want to see it (but it is trivial):
ftp://cap.connx.com/pub/chess-engines/n ... h/MANAVE.c

Here is how to use it:

E:\tmp>manave < c:2ndcraftys2004.pgn
white Elo average is 2490.381890
black Elo average is 2487.789683
Number of games was 276, number of games with WhiteElo tag was 254, Number of games with BlackElo tag was 252

Dann Corbit wrote:Get the program I made called Manave:
ftp://cap.connx.com/pub/chess-engines/n ... MANAVE.exe

Here is the source if you want to see it (but it is trivial):
ftp://cap.connx.com/pub/chess-engines/n ... h/MANAVE.c

Here is how to use it:

E:\tmp>manave < c:2ndcraftys2004.pgn
white Elo average is 2490.381890
black Elo average is 2487.789683
Number of games was 276, number of games with WhiteElo tag was 254, Number of games with BlackElo tag was 252

Thanks Dann. You did a good job, and you did it so fast! Thanks again.

As in your example, not all games have WhiteElo and BlackElo tags. With Scid, many of the missing elos can be filled in from a ratings database that is an add-on to Scid. These elo values are not stored in tag form but they are somehow attached to the players so that the game is not filtered out.

I'm considering redoing my database and only keeping games that have actual elo tags. That will decrease the number of games by less than 1%, but will make the stats more reliable.

Hi,

I'm doing a statistical study of chess results. In order to ensure fairness that each sample not only has approximately equally strong players, but also has on average, equally strong players playing White as it has playing Black, I need a special utility program. Scid doesn't do it.

I am therefore looking for the following type of utility program.

I need a pgn utility program that can look at a pgn file, extract the elos for white and for black, and can then calculate the average White elo and the average Black elo.

The result from average for White and Black won't help you here,
because it doesn't tell anything about the distance in each case
as you need it...

e.g.:

Code: Select all

sample W: 2400 vs. B: 2600  0:1
       W: 2600 vs. B: 2400  1:0
       W: 2500 vs. B: 2500  1/2
       avg: 2500/2500       50%

what is your conclusion?

Actually you need also a tool which sorts all games for you,
when the distance is very close for both opponents _in each_ game
, if you haven't done it already.

Guenther

Guenther Simon wrote:

Hi,

I'm doing a statistical study of chess results. In order to ensure fairness that each sample not only has approximately equally strong players, but also has on average, equally strong players playing White as it has playing Black, I need a special utility program. Scid doesn't do it.

I am therefore looking for the following type of utility program.

I need a pgn utility program that can look at a pgn file, extract the elos for white and for black, and can then calculate the average White elo and the average Black elo.

The result from average for White and Black won't help you here,
because it doesn't tell anything about the distance in each case
as you need it...

e.g.:

Code: Select all

sample W: 2400 vs. B: 2600  0:1
       W: 2600 vs. B: 2400  1:0
       W: 2500 vs. B: 2500  1/2
       avg: 2500/2500       50%

what is your conclusion?

Actually you need also a tool which sorts all games for you,
when the distance is very close for both opponents _in each_ game
, if you haven't done it already.

Guenther

You're right. "Distance" is a big issue. I need a utility that can input a pgn file, ask for an "elo distance" and then output a file that is a subset of the input pgn file, but containing only those games within the prescribed distance.

Dann Corbit, please pick up a white courtesy phone!

Norm Pollock wrote:

Guenther Simon wrote:

Hi,

I'm doing a statistical study of chess results. In order to ensure fairness that each sample not only has approximately equally strong players, but also has on average, equally strong players playing White as it has playing Black, I need a special utility program. Scid doesn't do it.

I am therefore looking for the following type of utility program.

I need a pgn utility program that can look at a pgn file, extract the elos for white and for black, and can then calculate the average White elo and the average Black elo.

The result from average for White and Black won't help you here,
because it doesn't tell anything about the distance in each case
as you need it...

e.g.:

Code: Select all

sample W: 2400 vs. B: 2600  0:1
       W: 2600 vs. B: 2400  1:0
       W: 2500 vs. B: 2500  1/2
       avg: 2500/2500       50%

what is your conclusion?

Actually you need also a tool which sorts all games for you,
when the distance is very close for both opponents _in each_ game
, if you haven't done it already.

Guenther

You're right. "Distance" is a big issue. I need a utility that can input a pgn file, ask for an "elo distance" and then output a file that is a subset of the input pgn file, but containing only those games within the prescribed distance.

Dann Corbit, please pick up a white courtesy phone!

I will make for you a program that does many statisical looks at the data.
I have a free statistical template that anyone can use here:
ftp://cap.connx.com/pub/tournament_software/STATS.HPP
requires also:
ftp://cap.connx.com/pub/tournament_software/Kahan.Hpp

It will be trivial to add it into an application for Elo statistics.

However, Monday is a busy day for me without much slack time.

Probably, you won't see anything until Tuesday unless some miracle happens or someone else does it.

Thanks Dann. It would be great if all are combined into one utility.

New program called manave.exe (same as before) but now the source file is called manave.cpp and this is what the output looks like:

E:\moment>manave < C:\eco\ssdf\2630.pgn
white Elo average is 2693.820516
White count :14330
White max :4000
White min :2630
White range :1370
White average :2.693820516399162595952547e3
White RMS error :2.694340021546965020365771e3
White variance :2.799177136648893663418698e3
White standard deviation :5.290725032213348416000375e1
White population skew :1.833128292529640495085457
White population kurtosis:2.559737020418757355372897e1
White coefficient of var.:1.964022844137171891308172
white Elo average is 2693.590046
Black count :14326
Black max :3021
Black min :2630
Black range :391
Black average :2.693590046070082367723021e3
Black RMS error :2.694092566267750568019730e3
Black variance :2.707419331325533557562285e3
Black standard deviation :5.203286779839771197042997e1
Black population skew :0.868550496582884548550503
Black population kurtosis:0.574211660850253208832652e-1
Black coefficient of var.:1.931729287250414436225825
Number of games was 22423, number of games with WhiteElo tag was 14330, Number of games with BlackElo tag was 14326

Hi Dann,

Thanks again.

Is there any way you can make the progam filter out pgns of games that exceed an inputted elo distance (absolute value of white elo minus black elo)? Then output the filtered file, the average of the elo distances in the filtered file, and the maximum elo distance in the filtered file.

-Norm

Norm Pollock wrote:Hi Dann,

Thanks again.

Is there any way you can make the progam filter out pgns of games that exceed an inputted elo distance (absolute value of white elo minus black elo)? Then output the filtered file, the average of the elo distances in the filtered file, and the maximum elo distance in the filtered file.

-Norm

Sure, but not today.

And the program I supplied is a bletcherous hack.

If there are more than 5 million games with WhiteELo or BlackElo, it will crash.

The right way to do it is to use <vector> but I just threw something together that I could to in 5 minutes.

You could use SCID to create narrow Elo bands, and then use the existing program in the meantime.