Are tablebases usefull?

[Bernhard Bauer]

Hallo,
many have found that tablebases are not usefull.
IMHO this is due to the fact that testers normally use slow disks.
Even SCSI with Raid 0 is slow compared to memory.
Here some results from Walter Eigenmann for his endgame test.

Code: Select all

Hardware:  Athlon64/3000+
Hash: 128MB
3-5-men-Nalimov-, und Bit-Bases bei einem TB-Cache von 64MB
Zeit: 60 Sekunden/Position
Programm          Lösungen    Lösungen
                  mit Bases   ohne Bases
1. Shredder 10        68           62
2. Rybka 2.2            64           61
3. Fritz 10                63           61
4. Deep Frenzee       61           61
5. Hiarcs 11.1          59           57


For comparison I tried Crafty21.6 without use of tablebases
and with 3-4-5 men tablebases read from memory.
Here my results

Code: Select all

Xeon 3GHz, no use of tablebases
1 min/pos
Crafty v21.6 4 threads
total positions searched..........         100
number right......................               71
number wrong......................             29
total nodes searched.............. 41608456044
average search depth..............        17.1
nodes per second..................    13925512
total time........................       49:47

Xeon 3GHz, 3-4-5 men tablebases in memory
1 min/pos
Crafty v21.6 4 threads
total positions searched..........         100
number right......................               80
number wrong......................             20
total nodes searched.............. 39029180921
average search depth..............        17.0
nodes per second..................    13349242
total time........................       48:43


There is hardly any slowdown in nps.
With use of tablebases 80 positions are solved.
Without tablebases 71 positions are solved.
This compares well with W. Eigenmanns results for Schredder,
where 68 positions are solved using bitbases and only 62 positions
without bitbases.
Summary: Use of bitbases and tablebases are usefull at least for W. Eigenmanns endgame test when read from memory.

kind regards
Bernhard

[Volker Pittlik]

...

...Xeon 3GHz, 3-4-5 men tablebases in memory...

You have 8 GB RAM?

Volker

[Bernhard Bauer]

No, 16

so I can use large hash tables too.

kind regards
Bernhard

[Dann Corbit]

I believe that if you keep the EGTB tablebase files in RAM you will see a modest Elo improvement (perhaps slightly larger than bitbase files alone).

Generally speaking, the strong programs will solve simple 6 man checkmates anyway, so it is not surprising that most experiments have shown little benefit from using Tablebase files.

The biggest problem for Tablebase files adding strength is that most games do not go that deep. I guess that the vast majority of chess games have ten or more chessmen at the time of mate or resignation. So the possibility to add strength is limited to the minority of games that make it to a sparse board.

I suggest that you run a contest between a single program that can use tablebase and bitbase and run this configuration:
1. Program + bitbase + tablebase
2. Program + bitbase
3. Program + tablebase
4. Program with no bitbase or tablebase.

Run the games at high speed (e.g. game in one minute) which will give 1440/2 games in one day (unless you have multiple CPUs and can increase either the time or number of games played).

720 games would give 360 games for each engine, which should be enough to see a statistical difference (I guess that the advantage should be just barely large enough to prove with that number of games)

My guess is:
Program + bitbase + tablebase all in ram = +75 Elo
Program + tablebase in ram = +60 Elo
Program + bitbase in ram = +50 Elo
Program alone = baseline

[Uri Blass]

I think that you overestimate the value of tablebases.
I guess that you will not earn more than 20 elo by tablebases even when they are in ram.

My guess is that the gain may be bigger in blitz time control but I guess that even in blitz time control you will not see more than 25 elo.

Uri

[Dann Corbit]

I think that you overestimate the value of tablebases.
I guess that you will not earn more than 20 elo by tablebases even when they are in ram.

My guess is that the gain may be bigger in blitz time control but I guess that even in blitz time control you will not see more than 25 elo.

Uri

I saw one study that showed 50 Elo for bitbase files in RAM. I think regular EGTB tables will be better. I am fairly certain that they should also be at least 50 Elo, therefore.

I guess that when the board gets as sparse as 16 chessmen you will start to get some EGTB hits on long qsearches. But (of course) the value will get better and better as the chessmen come off.

I do not think that there is enough data to really know for sure. That is why I asked for the experiment. I think it is a really interesting question.

[Bernhard Bauer]

Your suggestion running test looks ok.
But which program may I use?
1. The program should run on a linux 64-bit system.
2. The program should be able to use tablebases.
3. The program should be able to use bitbases.

There was a program (fruit?) that uses both, but use of bitbaes was poor.

If you are interested in such a test you may simply use 3-4-piece table bases. They use only 30 Mb.

On an AMD X2 5200+, 2Gb mem running crafty21.6 on 2 procs
I got the following results for the Eigenmann endgame test:

Time (sec) solved pos tablebases
..........8..............37........3-4 piece in mem and 5 piece on disk
..........8..............45........3-4 piece in mem

kind regards
bernhard

[Greg Simpson]

My guess is:
Program + bitbase + tablebase all in ram = +75 Elo
Program + tablebase in ram = +60 Elo
Program + bitbase in ram = +50 Elo
Program alone = baseline

Why do you expect an additional improvement for using tablebases and bitbases in RAM? I thought the whole point of bitbases was to get everything in RAM. If the tablebases are already there, why would adding the bitbases help?

[Dann Corbit]

Your suggestion running test looks ok.
But which program may I use?
1. The program should run on a linux 64-bit system.
2. The program should be able to use tablebases.
3. The program should be able to use bitbases.

There was a program (fruit?) that uses both, but use of bitbaes was poor.

If you are interested in such a test you may simply use 3-4-piece table bases. They use only 30 Mb.

On an AMD X2 5200+, 2Gb mem running crafty21.6 on 2 procs
I got the following results for the Eigenmann endgame test:

Time (sec) solved pos tablebases
..........8..............37........3-4 piece in mem and 5 piece on disk
..........8..............45........3-4 piece in mem

kind regards
bernhard

Scorpio can use both. That is the only one that springs to mind.

[Dann Corbit]

My guess is:
Program + bitbase + tablebase all in ram = +75 Elo
Program + tablebase in ram = +60 Elo
Program + bitbase in ram = +50 Elo
Program alone = baseline

Why do you expect an additional improvement for using tablebases and bitbases in RAM? I thought the whole point of bitbases was to get everything in RAM. If the tablebases are already there, why would adding the bitbases help?

You are right, of course. I was imagining the usual case, where the bitbases cover a much larger area than the tablebase files. But if you have the complete set and they both cover all of the cases, then there can be no additional advantage of using bitbases (except slightly faster access because of the more compact format.).

[Bernhard Bauer]

Are you sure that scorpio can use bitbases and tablebases?
In the scorpio.ini file I can see a path to bitbases but nothing is mentioned about tablebases.

kind regards
Bernhard

[Dann Corbit]

Are you sure that scorpio can use bitbases and tablebases?
In the scorpio.ini file I can see a path to bitbases but nothing is mentioned about tablebases.

kind regards
Bernhard

You are right. Daniel even made his own format of tablebase files and generated his bitbase files from those, but I think that the code to use the tablebase files is no longer included.