Hi
,all testers should please use this unified abbreviation (Amateur Engines Grand Test or Tournament) from now on, so that all people know the event where the results (that will be given from all testers individually here) belong to. As most already will have understood by now, this is a common tournament, where a lot of testers combine their efforts and CPU power in a big tournament.
First of all I repeat the names of testers and the group or groups they are testing in. New testers are welcome at any time, if they agree to the rules and conditions we have voted for and they would make possible: more games for each engine, more participants and more reliable results.
Testers (organizers)
-------------------
Master Class or A:
Olivier Deville
Igor Gorelikov
Christian Koch
Pedro Eckmann
Heinz van Kempen
Slobodan Stojanovic ? (not sure, was the only one who did not fill in the voting sheet)
Volker Pittlik (can decide where to test and more help is needed, in A or B)
Higher Class or B:
Thomas Mayer
Luis Smith
Roger Brown
Olivier Deville (second computer)
Heinz van Kempen (second computer)
Volker Anuss (is not allowed and does not want to test his own engine Hermann, what will be done by others)
Simultaneous gauntlets (for "in-between group")
Heinz van Kempen (third computer, maybe also two faster ones from a friend)
possibly after the initial tournaments also others will help
A lot of testers can probably run their machines for 16 or more hours daily.
The number of engines for the start groups had to be decided. Votes ranged from 8 to 17 in each group and the average gave 12. So this is the number of those who come first in each tournament.
We decided to use a point system for group A:
I gave the following in the voting list...
for the first step I pick my favourite engines and give:
20 points to .....
18 points to .....
......
2 points to .....
So everyone could vote for ten engines and points for each engine were summated.
Authors felt free to apply various criteria for voting ranging from absolute strength, results in their respective previous tournaments, progresses this year and playing style to the point what engines they guessed or knew would be stronger with more time. Results from other tournaments we read, like those from Leo´s site, also might have had some influence
The following 12 received the most points from all together and will play at first. Each engine two games with colours reversed against all others from all 5-7 testers up to now, so each one will have to play 10-14 games against any other (from all testers combined).
Participants for Master Class (A)
----------------------------------
(in alphabetical order)
AnMon
Aristarch
Crafty
Delfi
Gothmog
List
Quark
Ruffian
SmarThink
Tao
Thinker
Yace
-------------------------------------------------------------------------------
(What now follows is my personal goal - or possible project for more and exact rules are not fixed - not those of all testers, that I still did not ask about, because most are not sure how long they want to continue).
The next ones that also partly got a lot of points will form the "in-between" group 1 the first ones added by gauntlets playing the same amount of games all others already have against all the participants from Master Class (A) above.
The first gauntlets will begin simultaneously to the other tournaments and results will be added to the overall crosstable at once. I think authors here should decide themselves if they want to be in earlier or want to do first improvements before being added.
(in alphabetical order)
Amy
Amyan
Dragon
El Chinito
Fruit
Green Light Chess
Jonny
King of Kings
Little Goliath
Movei
SlowChess
SOS
Ufim
WildCat
A third group is an undefined one, where it is not clear if to put them to Master Class (A) or Higher Class (B). To those group would belong for example The Baron, Terra, Frenzee, Averno and much more. I do not know if a lot of testers we have now want to continue for a longer time with this tournament and it will depend also on the success the tournament might have and the time and own plans for tournaments, but for me I want to continue adding engines, even if it lasts one year.
-------------------------------------------------------------------------------
Now back to that what is for sure that we will do all together.
Finally we have Higher Class or B. Here votings were less complicated. Every tester for that group could give up to three votes and fast improving engines were mainly preferred here. We did not exclude private engines, some will be released anyway maybe this year or next. I got permission from the authors of Spike, Snitch and Cerebro that all testers will receive from them a new version for this tournament. Also those testing in A because there might be a change at every time that they are needed for help in B.
(Notw to Roger ....Sorry, I miscalculated number of engines here, we should also only have 12 in the beginning. So I have to cling to your words: "I am willing to test whatever engines eventually get picked in the weaker group". You will be the first one to decide the new ones later).
Participants in Higher Class (B)in loose order:
----------------------------------------------
DanChess
Trace
Big Lion
Snitch
Spike
Bruja
Black Bishop
Alarm
Cerebro
Djinn
Booot
Hermann
Those at weaker level still I frankly speaking cannot promise that they will be included later. I will try but with the time control of 40 moves in 40 minutes (based on 2 Ghz CPU) we agreed for both groups it seems almost impossible to test 200 engines more with many games, if not a lot testers will come and help us in the future.
Engine versions (valid for both tournaments):
---------------------------------------------
We had evenly split votings here. Some want only take the latest available version (or one the respective author prefers or sends to us), others think that we should also be allowed to take earlier versions, if a few testers are convinced that it must be stronger (based on their tournaments). So for example Igor is convinced that Ruffian 1.0.1 will be the strongest free Ruffian version. I think we vote this for all versions before the tournament starts. In any case a programmer who wants to have other version in than his last release, should post this in the forum and/or write to one of us via email.
Updates
-------
no updates are allowed, many games should be played with the same version, although it might be obsolete after only a few weeks. If we replace in case of a very buggy version was not voted. Authors should give or recommend us a stable version that is already sufficiently tested. Later on, when all strong engines are in, it might be possible to replace older versions by newer ones
Settings for each engine
------------------------
should be unified as much as possible, authors are encouraged to give us hints, which we will discuss when they differ from what we thought would be apted.
---Book and position learning is allowed by a vast majority of votes and will therefore be granted by all testers
---Hash for those engines that support the scheme 32...64 should be 64 MB or 128 MB for those who can give that (Igor can only give 38 max.)
...and for those engines who support 24...48...96 it should be 48 MB or 96 MB for those testers who can give it
---EGTB and cache---those who have 5 men EGTB should use them and give 8 MB cache, others use all 3 and 4 men and give 2 or 4 MB
---pawnhash is allowed if supported and is voted to be included in the maximum amount of hash and not additionally
---logfiles...we keep logfiles, in case that there are questions or bugs, but delete them after two weeks or even earlier, because they are partly very big, sometimes over 100 MB after a short while
other rules
-----------
---exceeding time limits...when an engine loses on time the game will not be replayed unless it is obvious that something strange happened with hardware, GUI etc., a bug report will be send to the author
---books...own books, recommendations from authors can be given I think, if there are diverse...Nunn positions are allowed, results might be kept separately if they differ a lot (personally I suspect that the allowed learning will have more influence when using Nunn positions)
Time control
------------
This should be seen together with hardware, a 40/40 game on a 3400 Mhz computer is of course very different from one on 1 Ghz. So with the help of Axon benchmark we will all simulate a 2 Ghz CPU. If computers are faster they can give a bit less time than 40/40 if they are slower they have to add the respective amount of time, might even have to play a 40/120 games in case of very slow hardware.
Starting date
-------------
Not fixed. Anyone can start when he wants and has time. Noone should start until Sunday I think, so that the authors can recommend us settings. From authors of Snitch, Spike and Cerebro I know already that they are busy with a new version that will not be ready before Saturday.
Availability of results and games for download
----------------------------------------------
All testers agreed that they will post their individual results here in Winboard Forum themselves. Moreover results and downloads of games will be available from different homepages of testers. Those who do this will give a link underneath their postings when there is something new for download. I propose that every tester can post his results as soon as he has a bunch of games (a dozen or twenty) at least. It will be welcome when he gives some comments about observations what happened in some games, bugs, won positions that were finally lost, etc.. Shortly annotated games in postings for those who have the time to do that are highly appreciated, but not a must.
For my fellow testers. Surely I forgot some things, because I did not get a lot of sleep last night. You all have all voting sheets, so please add things I gave not completely, wrongly or could be misunderstood.
Best Regards
(and have a lot of fun with all those intersting games, they will mainly be on high level).
Heinz