CCRL Testing Group's Debut Rating List

[Tuvoc]

CCRL (Computer Chess Rating Lists)

We are a group of testers formed with the goal of producing further rating lists that will go hand in hand with other established rating lists in showing the relativity in engine strength.

At present CCRL has the following testers:
Graham Banks, Ray Banks, Sarah Bird, Kirill Kryukov, Tom Logan, Andreas Schwartmann, Charles Smith and Chris Taylor.

We have concentrated at this stage on a 40 moves in 40 minutes repeating rating list, calibrated to Athlon 64 3800+ hardware, but also intend to produce a blitz list (40 moves in 8 minutes repeating) shortly. Future plans include the possibilty of "endgame rating lists" at the same time controls based on play proceding from a series of endgame positions.

Each engine uses a generic opening book with an upper limit of 12 moves per side. Ponder off, no learning, EGTBs, hash size 128mb or 256mb each (deep engines get twice this size). For blitz it will be 64mb hash

For our first 40/40 rating list, we took a list of 18 engines with the intention of running 30 game matches (all play all) so that we could establish a base from which to develop.

We now present to you the first CCRL 40/40 rating lists at the following link:

http://kd.lab.nig.ac.jp/chess/CCRL-4040/

There is a best versions list, an all versions list and a pure list that removes rating distortions.

There are also many stats tables available for your interest at the above link including:
- pairwise results
- game length statistics
- engine correlation statistics (expected move table, evaluation difference table, engine distance table).

In addition we have opened a CCRL discussion board. You are all welcome to join so that you can discuss our testing and make comments or suggestions on an ongoing basis.

http://kd.lab.nig.ac.jp/chess/discussio ... um.php?f=7

Please note though that general questions about engine testing, not directly related to CCRL, are better posted in other discussion boards, for example WB Forum or here. You will certainly reach more people and get better answers there.

Comments and questions are welcomed.

[Roger Brown]

CCRL (Computer Chess Rating Lists)

We are a group of testers formed with the goal of producing further rating lists that will go hand in hand with other established rating lists in showing the relativity in engine strength.

At present CCRL has the following testers:
Graham Banks, Ray Banks, Sarah Bird, Kirill Kryukov, Tom Logan, Andreas Schwartmann, Charles Smith and Chris Taylor.

[snip]

Comments and questions are welcomed.

Hello Tuvoc,


Welcome.


The list certainly looks fascinating and I am sure it will generate much interest and comment .


Please post future announcements, results etc in the Tournaments sub-forum.


Later.[/b]

[Graham Banks]

Hi Roger,

I'll ask Ray and the others to make future posts in the Tournament sub-forum as requested.
The list of engines tested will grow to include many Winboard engines over time.

Regards, Graham.

[Robert Allgeuer]

From the ponder hit and eval difference statistics:
Rybka correlates most with Fruit and Fritz, the least with Shredder.

Robert