Unexpected hash behaviour with WB.

[Trahald]

I'm using WinBoard 4.4.0 with Polyglot 1.4.50b which I have renamed polyglot.exe replacing the 1.4.46b version that was included in the installation package. Anyway in the global engine settings in WB, I have hash set to 64mb and EGTB hash set to 4mb. I think these were the default settings.

It seems when using Rybka, which supports the Nalimov Tablebases, that everything works as expected. When using Stockfish however, another UCI engine which does not support the Tablebases, when I used the Engine #1 settings to save the Polyglot ini file, the hash was set to 68mb, which I presume to be the sum of the two hash sizes. After manually editing the ini file to make it 64mb and reloading the engine, I notice from enabling the debug setting within PG that although Polyglot initially loads the engine and sets the hash size to 64mb, WinBoard still insists on changing it to 68mb.

Is this expected behaviour for an engine that does not support Tablebases? In other words, where an engine does not support tablebases, does WinBoard allocate the memory normally reserved for the endgame hash to regular hash instead?

I understand that we are only talking about a 4mb difference here, but the difference could be much more exaggerated if for instance I was using an endgame hash of say 256mb, which is the largest that Rybka supports.

[Trahald]

Is this expected behaviour for an engine that does not support Tablebases? In other words, where an engine does not support tablebases, does WinBoard allocate the memory normally reserved for the endgame hash to regular hash instead?

I guess it is so, just an an experiment I tried changing the EGTB size in WinBoard to 256mb. Leaving the hash at 64mb. Sure enough, loading Stockfish gives me a hash size of 320mb.

[Roger Brown]

Is this expected behaviour for an engine that does not support Tablebases? In other words, where an engine does not support tablebases, does WinBoard allocate the memory normally reserved for the endgame hash to regular hash instead?

Hello Trahald,

I am sure the H.G. will set me straight but from the enhanced specs the gui is behaving correctly. I quote:

memory N
This command informs the engine on how much memory it is allowed to use maximally, in MegaBytes. On receipt of this command, the engine should adapt the size of its hash tables accordingly. This command does only fix the total memory use, the engine has to decide for itself (or be configured by the user by other means) how to divide up the available memory between the various tables it wants to use (e.g. main hash, pawn hash, tablebase cache, bitbases). This command will only be sent to engines that have requested it through the memory feature, and only at the start of a game, as the first of the commands to relay engine option settings just before each "new" command.

So the gui sets the total memory size, leaving to the engine how this total is to be allocated among the various aspects of memory.

I hope this helps.

Later.

[Miguel A. Ballicora]

Is this expected behaviour for an engine that does not support Tablebases? In other words, where an engine does not support tablebases, does WinBoard allocate the memory normally reserved for the endgame hash to regular hash instead?

Hello Trahald,

I am sure the H.G. will set me straight but from the enhanced specs the gui is behaving correctly. I quote:

memory N
This command informs the engine on how much memory it is allowed to use maximally, in MegaBytes. On receipt of this command, the engine should adapt the size of its hash tables accordingly. This command does only fix the total memory use, the engine has to decide for itself (or be configured by the user by other means) how to divide up the available memory between the various tables it wants to use (e.g. main hash, pawn hash, tablebase cache, bitbases). This command will only be sent to engines that have requested it through the memory feature, and only at the start of a game, as the first of the commands to relay engine option settings just before each "new" command.

So the gui sets the total memory size, leaving to the engine how this total is to be allocated among the various aspects of memory.

I hope this helps.

Later.

It is done by design, and it is the fair way to allocate memory. Same amount for the engines, and they can do whatever they want with it.

I strongly agree with the concept!

I personally do not like the way it is shown in the interface for winboard engines; but, apparently it is done to make uci engines happy, as I understood. I would prefer one parameter in the dialog = "memory", and the engine is responsible to allocate egtb cache or whatever. The current dialog is a bit misleading, particularly for winboard engine users. The engine will directly receive the command "memory 68, whether it plays with egtb or not. Winboaard/Xboard ignores 64+4 and sends 68. The user thinks it has control over the egtb cache, but it doesn't. IMO, the option should not even be there if later is ignored in the protocol. This may be the price we have to pay for UCI compatibility.

Miguel

[H.G.Muller]

Indeed, as Roger already pointed out, this is intended behavior. Engines that are not using an EGTB cache are allowed to use that memory for hash tables, or any other purpose they see fit. That the total amount of memory that can be used is subdivided in the Global Settings menu as hash size and EGTB size is only for the benefit of UCI engines, as these are not able to decide for themselves how to allocate their memory.

Perhaps I should replace the "Hash Table" control in the "Global Settings" dialog by a "Max Memory Use" control, and calculate the UCI hash-table size as MaxMemory minus EGTBcache. This might reduce the confusion level. The way the dialog represents it now is for historic reasons (it was inherited from Winboard_x), and not really by design.

I am sorry that this confusion exist, but we have to deal with the practical situations that virtually all UCI engines implment EGTB in a non-compliant way: they do use the memory for EGTB cache on top of the memory used for hash tables. I don't know where they got the idea that they were allowed to do that. Certainly not from the UCI specs, which says the value of the Hash option says how much memory the engine can use for "hash tables" (plural), implying it is not just the main hash size, but all hash tables together. And the EGTB cache is obviously a hash table.

So we bowed to reality, and assume UCI engines use an amount of memory equal to the sum of their Hash and NalimovCache. As indeed is the case for Rybka. So in the mentioned case everything works perfectly.

WinBoard does not support playing matches with memory odds. If you want to handicap one of the engines by reducing its memory use, you should make it ignore the WinBoard memory command. This can be done for UCI engines by using a Polyglot version that does not support it, and for any engine by suppressing the memory=1 feature being sent from engine to GUI, e.g. using the InBetween adapter.

[Teemu Pudas]

So we bowed to reality, and assume UCI engines use an amount of memory equal to the sum of their Hash and NalimovCache. As indeed is the case for Rybka.

Actually, Rybka (as of version 3) uses about cores*70 MB more than that.

[H.G.Muller]

OK. Then it cheats in more way than one, as I cannot imagine one reason for allocating an extra 70MB.

[Roger Brown]

Actually, Rybka (as of version 3) uses about cores*70 MB more than that.

Hello Teemu,

What are you saying here?!?!?

That an engine that I set for 100 mb of hash is actually using 100+70 mb of hash?

Why does a world beater like Rybka behave like that? Take more hash than I intended to give it?

Perplexed regards.....

[Teemu Pudas]

It's a bug. I guess it's just a typo in the size of some array.

That an engine that I set for 100 mb of hash is actually using 100+70 mb of hash?

170 MB of memory. It's unclear what the 70 MB (per process) block is for, though a hashtable of some sort, perhaps pawn hash, would be the most logical explanation.

[Michel]

perhaps pawn hash,

A 70Mb pawnhash???

[Teemu Pudas]

~4MB sounds more reasonable, all it takes to get to 70 mb is a single additional zero.