Fast and slow thinking engines

[Marc Lacrosse]

Not so long ago (one year maybe), when Shredder 9 was an undisputed leader of the field, V. Pittlik published an interesting analysis showing that shredder was an extremely fast evaluator : when analysing a series of difficult epd test positions, shredder was not only one of those who solved the largest number of quizzes when a long thinking time was allowed but it distinguished itself mostly by a considerable superiority over all other programs when very short thinking time was afforded per position.
This correlated very well with the number of brilliant victories won by shredder in blitz tournaments.

As I wondered how the newer stars would perform from this point-of-view, I selected 87 difficult positions within various well-known epd test-suites
(ECMGCP, LPTII, ...). I had them analysed by six programs at 10 seconds / position and then at 180 seconds /position on a Pentium IV 2200Mhz PC.
The programs were Rybka 1.0 beta 32, Shredder 9 UCI, Fruit 2.2, Toga II 1.1a, Ktulu 7.0a and Gandalf 6.0.

For each program I evaluated the ratio between the number of positions solved at 10 seconds and the number of positions solved with 3 minutes thinking time per position.

This ratio was :

Code: Select all

Rybka       57 %
Shredder    57 %
Gandalf     55 %
Ktulu       53 %
Toga        39 %
Fruit       36 %


So we see that Rybka is one of the fastest searchers (and its superiority should be specially pronounced at blitz) whereas Fruit and Toga are incredibly weaker when a too short thinking time is given.

Marc

PS : the absolute number of positions solved by the diferent engines with 180 seconds per move was : Rybka and Shredder 68, Toga and Fruit 61, Gandalf 60, Ktulu 55.