Whether or not to implement endgame knowledge

[Onno Garms]

I'm currently thinking if I should implement endgame knowledge in my engine or if it is sufficient to support tablebases.

I found only a short discussion on it and that is very old:
http://wbforum.vpittlik.org/viewtopic.php?t=3125

Reasons to implement endgame knowledge:
- Access to 5 (or even 6) men tablebases is slow. Endgame knowledge is faster.
- People might use the engine without tablebases.

Reasons not to implement endgame knowledge:
- Implemetation takes a lot of time, most likely more time then implementing a tablebases access. Implementation time is the scarcest ressource that limits an engine's playing strength. I might better invest my time in something else.
- Wait a few years and 5 men tablebases will fit into the RAM. On expensive machines they might fit today.
- Looking at the Fruit sources, there aren't many endgame positions that are covered by endgame knowledge but not by 5 men tablebases. Some recognisers respond to positions with 6 and more men, but I doubt those positions often occur in practical games.
- Implementation of endgame knowledge is error prone. For example, Fruit's recognizers seems to have many shortcomings.

Did anybody make tests how many elo internal endgame knowedge is worth when tablebases are present?

[Onno Garms]

Suppose we want to implement endgame knowledge on KPK. The best way to do this might be to create a tablebase before the game starts.

We could also safely classify some positions as draw, but a static evaluation that can also report wins without search might be a bit harder. The latter would report wins faster.

It would be nice to create KPK without KQK and KRK before.

Are the following rules true?

1. Position is a win for white if and only if he can enforce promotion in a way that does not allow the new piece to be captured.

2. In all positions other than 8/1P6/8/8/8/K7/8/k7 b - - and symmetric white wins by acutally promoting (to a rook or alternatively to a queen if the latter does not result in a stalemate as in 8/k1P5/8/1K6/8/8/8/8 b - -)

[Tord Romstad]

Reasons to implement endgame knowledge:
- Access to 5 (or even 6) men tablebases is slow. Endgame knowledge is faster.
- People might use the engine without tablebases.

- Implementing endgame knowledge without using tablebases may help you discover heuristics which could be effective in more complicated endgames.

- Mobile phones and other handheld devices are likely to become more relevant computing platforms than desktop computers in the near future, and you won't be able to fit EGTBs on those any time soon.

- Throwing gigabytes of memory at the problem of evaluating basic endgames is a very inelegant solution.

Reasons not to implement endgame knowledge:
- Implemetation takes a lot of time, most likely more time then implementing a tablebases access. Implementation time is the scarcest ressource that limits an engine's playing strength. I might better invest my time in something else.

This is only true if you can use somebody else's code for generating, compressing and probing the tablebases. Getting somebody else's permission to use their code can be problematic, and even if you can, you might not like to depend on other people's code for the game-playing parts of your program.

Implementing a tablebase generator and an efficient compression scheme is a lot of work.

- Wait a few years and 5 men tablebases will fit into the RAM. On expensive machines they might fit today.

See above. Wait a few years, and only a few hardcore geeks will use desktop computers.

Tord

[Onno Garms]

Another great thing about adding endgame knowledge is that you can learn some chess stuff from studying the positions.

I am not interested in improving my own playing strenght, unless required to write a strong chess program. I had my time almost 20 years ago. I saw my potential and my limits and I retired before reaching the age of 18.

[Onno Garms]

Thank you for the additional pros you gave. Good arguments, in deed.

I wasn't aware that getting a license for standard table base support is so difficult. Who holds the license for the compression scheme that was used and the code fragments to access the tablebases? What's the procedure to get a permission? How come that all major engines do support the standard tablebases?

Using an own compression scheme is a bad idea IMO (unless it is more efficient like Shredderbases), because people would need to have the tablebases in different formats on their disks and I would have to provide a generator or a broadband download for the tablebases.

[Ron Murawski]

Thank you for the additional pros you gave. Good arguments, in deed.

I wasn't aware that getting a license for standard table base support is so difficult. Who holds the license for the compression scheme that was used and the code fragments to access the tablebases? What's the procedure to get a permission? How come that all major engines do support the standard tablebases?

Using an own compression scheme is a bad idea IMO (unless it is more efficient like Shredderbases), because people would need to have the tablebases in different formats on their disks and I would have to provide a generator or a broadband download for the tablebases.

You need to contact Eugene Nalimov for permission. I sent one email a month for 6 months before I received permission. If you need his email address I can pm it to you.

I have no idea what compression method is used. I've looked at Eugene's egtb code and it will always remain one of the great mysteries of life for me. I made enough changes to Eugene's code to stop all compiler warnings and I never want to look at it again!

You can look at the code of any program supporting egtbs to see how to probe the tablebases. (Be aware that several of them have small bugs!) I believe that the probe code does not need any permission at all as long as you don't copy and paste licensed code.

Ron