EGTB compression

[Stef Luijten]

Does anyone know the compression ratio of the Nalimov tables? i.e. what is the total size with and without compression?
I wonder how usefull EGTB compression really is, a well-designed index scheme shouldn't have a very high compression ratio.
Is EGTB compression mainly done for disk space savings or to speed-up the probing?
Sheers, Stef.

[Daniel Shawul]

EGTB compression is really improtant to decreases the size of your database. Nalomv's indexing scheme are the best that i know, but i am sure uncompressed tbs are much much larger than compressed tbs.
For bitbases the compression works much better.
Another advantage is that the amount of data that you get when you
with a single read of compressed block is bigger than uncompressed.
Since decomperession speed is much faster than disk read , this might give a performance gain. I think an experiment has been done that shows using compressed tbs is better than uncompressed tbs.
Daniel

[H.G.Muller]

Where can I find an unambiguous description of the compression scheme?

I would love to do some data-mining on existing TBs with Queens and Pawns, (like KqppKp and KqpKpp) to get an idea on the occurrence frequency of positions with Q on the 8th that are won but cannot be declared a win by a few simple rules (like: no enemy pawns on the 2nd with wtm), or which are a draw and would not be a win if you replace the Q by another piece.

But I have to learn how to read the Nalimov format, or generate my own 6-men, and the former seems easier. Especially since I need that statistics to develop an efficient generator of my own...

[Alessandro Scotti]

Hi H.G.,
the source code to probe Nalimov EGTB's is freely available, and using it is probably the easiest way to extract data from the databases.

[Thomas Mayer]

Hi Alessandro,

the source code to probe Nalimov EGTB's is freely available, and using it is probably the easiest way to extract data from the databases.

the only thing to be careful:
Eugene must give his permission to use them (Also Andrew Kadatch who wrote the compression algorithm). And he will not give it for open source projects which are GPLed or stuff like that, because he does not want to have his TB routines to be GPLed. (He clarified that once in CCC somewhere)

Greets, Thomas