Winboard Forum

Good evening.

I would like to know which program is responsible for setting the amount of size of the hashtables being used: Is it Xboard or polyglot when playing around with uci engines?

Polyglot 1.4-3:

-Size of Hash regulated here (ini) will be accepted by xboard as it can be seen in the logs and with 'top'

Polyglot 1.4.56:

-Size of Hash is rejected by Xboard

"Adapter->Engine: setoption name Hash value 128
...out of nowhere
GUI->Adapter: memory 68
POLYGLOT setting the amount of memory to 68Mb"

Xboard: 4.4.2-1 (Debian package)

Is it a bug in polyglot or a change in the behavior of xboard related to uci engines?

Code: Select all

[PolyGlot]

EngineName = Stockfish1.6.2
EngineDir = /home/me/chess/stockfish
EngineCommand = ./stockfish162

Book = false
Book File = stockfish-exp.bin
Log = true
LogFile = stockfish.log

Resign = true
ResignScore = 600

[Engine]

Hash = 512
Threads = 1
OwnBook = false


and polyglot log:

Code: Select all

Setting PolyGlot option "EngineName=Stockfish1.6.2"
1265228775.329 POLYGLOT Setting PolyGlot option "EngineDir=/home/me/chess/stockfish"
1265228775.329 POLYGLOT Setting PolyGlot option "EngineCommand=./stockfish162"
1265228775.329 POLYGLOT Setting PolyGlot option "Book=false"
1265228775.329 POLYGLOT Setting PolyGlot option "BookFile=stockfish-exp.bin"
1265228775.329 POLYGLOT Setting PolyGlot option "Log=true"
1265228775.329 POLYGLOT *** SWITCHING LOGFILE ***
1265228775.329 POLYGLOT NEW LOGFILE "stockfish.log"
1265228775.329 POLYGLOT *** LOGFILE OPENED ***
1265228775.329 POLYGLOT Setting PolyGlot option "LogFile=stockfish.log"
1265228775.329 POLYGLOT *** SWITCHING LOGFILE ***
1265228775.329 POLYGLOT NEW LOGFILE "stockfish.log"
1265228775.329 POLYGLOT *** LOGFILE OPENED ***
1265228775.329 POLYGLOT Setting PolyGlot option "Resign=true"
1265228775.329 POLYGLOT Setting PolyGlot option "ResignScore=600"
1265228775.329 Adapter->Engine: setoption name Hash value 512
1265228775.329 Adapter->Engine: setoption name Threads value 1
1265228775.329 Adapter->Engine: setoption name OwnBook value false


Perhaps it's a polyglot problem, as I do see in the logs from xboard in debug-mode no sign of the size of hash (but all the other options.) regulated by polyglot has reached the GUI. What I'm doing wrong?

This is no problem, but intended behavior. UCI options, including hash, are all set by the GUI. Polyglot is merely an adapter, relaying the instructions from GUI to engine. So when the GUI is set to run engins with 68MB memory, they will use 68MB memory.

To change the hash size you should alter it in the XBoard Options->General Settings menu, or use the command-line option -defaultHashSize with the desired number of MB on the XBoard command line. If you don't do either of that, the engines will run with the default memory, which is 64MB hash + 4MB EGTb cache = 68MB.

The GUI is always in charge. For non-standard UCI options this is not so obvious, as the GUI has no default settings for those. (It does not even know what they mean). But the same holds there: what the user specifies in the Engine Settings menu will overrule what was in the polyglot.ini file. The latter values are only the initial values, which will remain in force until the GUI specifies other values. Which for default options like memory and nr of CPUs it always does.

Hi,

ok then. But I think it's interesting, that the old polyglot (1.4.3*) does override the settings of the GUI then (I made none, so the default applied, as you explained) and perhaps the man page of the new polyglot would need an update on this - but you may not be the right person to address this issue to.

I do think it's confusing too, as I expect polyglot to handle _all_ of the parameter of an UCI engine (and no mixing with winboard: "Threads" vs. "smp") and then passing them to the xboard - I actually don't understand the reason for overriding them. When was this feature (filtering of smp and hash) introduced, because it seems somewhat new?

But good to know anyway; and thanks for your hints.

greetings

Alex Berger wrote:Hi,

ok then. But I think it's interesting, that the old polyglot (1.4.3*) does override the settings of the GUI then (I made none, so the default applied, as you explained) and perhaps the man page of the new polyglot would need an update on this - but you may not be the right person to address this issue to.

I do think it's confusing too, as I expect polyglot to handle _all_ of the parameter of an UCI engine (and no mixing with winboard: "Threads" vs. "smp") and then passing them to the xboard - I actually don't understand the reason for overriding them. When was this feature (filtering of smp and hash) introduced, because it seems somewhat new?

But good to know anyway; and thanks for your hints.

greetings

Hello Alex Berger,

I am of course intruding in my own ignorant way but let me see if I can assist.

In his ruthless quest to improve the Winboard/Xboard user experience, H.G. implemented a number of features to make it easier to configure engines. Traditionally and painfully, before Alessandro Scotti's Winboard, a user would have to fiddle with individual Polyglot ini and Polyglot executables - in individual folders.

Perhaps there were ways to centralise it but it must be admitted that being able to specify the common settings centrally made Winboard/xboard easier to use.

H.G. expanded the idea so that upon invoking an engine, Polyglot would translate the various settings into an interactive dialogue box which the user could adjust and save with a few easy clicks. No more messing around with ini files. Winboard is still generating the ini files (Polyglot still needs them as far as I know) but this process is hidden from the user.

The dialogue box is fairly comprehensive as I believe that it could accommodate Chest UCI and that chess software has settings galore.

So the end result is that the user can now set various parameters with ease and the Polyglot step is now hidden from his/her eyes which is no small mercy.

For UCI engines, the gui is really Winboard/Xboard invoking Polyglot.

I hope that this helped more than it annoyed.

Later.

Alex Berger wrote:Hi,

ok then. But I think it's interesting, that the old polyglot (1.4.3*) does override the settings of the GUI then (I made none, so the default applied, as you explained) and perhaps the man page of the new polyglot would need an update on this - but you may not be the right person to address this issue to.

I do think it's confusing too, as I expect polyglot to handle _all_ of the parameter of an UCI engine (and no mixing with winboard: "Threads" vs. "smp") and then passing them to the xboard - I actually don't understand the reason for overriding them. When was this feature (filtering of smp and hash) introduced, because it seems somewhat new?

It never worked that way (Polyglot passing parameters to XBoard). Polyglot is sitting in between XBoard and the engine, and it passes instructions that XBoard gives it to the engine. The only thing that is passed back to XBoard are moves.

But older XBoard versions failed to set many engine parameters, mainly because WB protocol lacked the commands to pass them. (WB protocol was developed before things like hash tables and tabebases even existed.) But UCI engines needed to receive those parameters, so Polyglot had to fill in the gap by providing them from an ini file. Now that the necessary commands have been added to WB protocol, XBoard has been enhanced with a menu for specifying them, and then sends them to the engine. If it happens to be a UCI engine, this is done through Polyglot. What the GUI does not send (e.g. because it is an older version, not supporting all commands) is still taken from the polyglot.ini file.

Older Polyglots don't support the WB protocol commands to receive the information from XBoard, though, so they have to stick with what is in the polyglot.ini file. If this is causing confusion, I think indeed the Polyglot documentation would be the place to elaborate on it. For XBoard, Polyglot + a UCI engine is just another engine speaking WB protocol. And if it woud not react to setting the memory size and CPU usage, it would be perceived as a broken engine. On any other GUI (e.g. Arena, Shredder) a UCI engine would obey the GUI's memory setting. Why should XBoard be different?

Hi,


this sounds plausible - so thank you both of you for explaining. So older versions of Polyglot override nothing - they just don't understand the expanded command scheme. Aha.

Point is: I did not use xboard's graphical menus...and did not know that you can change settings klickwise (that's the power of xboard, that is what it makes most useful and more powerful then most of the guis available*: events are scriptable and you can organise tournaments/matches very efficient; klicking through some options is slower not only for me I would guess). So indeed it was hidden and I wouldn't see it.

*for the *nix OS types anyway

greetings of the season

Well, we will certainly keep that property of scriptability: for everything that can be set through menus, we will always supply a command-line option as well. for the options that XBoard will always set these are -defaultHashSize, and -smpCores.

Hi,

I did run into two problems now:

I could not specify the -defaultHashSize just for one engine; for example Rybka uses processes (opposed to threads)-so the Hash size would be multiplied by the cpu-cores (no problem whatsoever with dualcore; but on a quad with longer timecontrols this could lead to annoyences). Old polyglot did let me specify the Hash differently for each engine.
How could that be done now?

Second: I have no Idea how to use the egtb cache feature as intended. Again, I cant' specify it for each engine (so it seems) - but the memory get claimed in any case. Whether both engines are able to use egtb does not interest xboard (is this assumption wrong?).
My case: letting some of the RobboLitos play with his own Tablebases. There is a cache option (up to 2048 mb) for that which the engines communicates. The 'cache' (from the polyglot.ini) value won't get accepted by xboard, but the directory will, so after changing it in commandline (setting -defaultCacheSizeEGTB 2048) xboard will claim an amount of 4096Mb memory because a match is played - besides as far as I can see, the cache won't get filled with useful stuff, it's just there.
Could you help me here H.G.M? How about a switch, to let xboard, when asked specifically, accept all of polyglot's option to play with? Would that be very ugly?

Alex Berger wrote:for example Rybka uses processes (opposed to threads)-so the Hash size would be multiplied by the cpu-cores

No, it's not. Task Manager just makes it look that way.

(But Rybka 3 uses ~70MB per core for another purpose. It's a bug).

Alex Berger wrote:I could not specify the -defaultHashSize just for one engine; for example Rybka uses processes (opposed to threads)-so the Hash size would be multiplied by the cpu-cores (no problem whatsoever with dualcore; but on a quad with longer timecontrols this could lead to annoyences). Old polyglot did let me specify the Hash differently for each engine.
How could that be done now?

If it does, then it is a clear Rbka bug. UCI SMP engines are not supposed to multiply their hash size by the number of cores they use. There are many methods to pamper non-cmpliant engnes so they can be used; The InBetween adapter can be very useful for that. As a general principle, I try to avoid putting features in XBoard that only serve to correct / compensate errors that ly with the engine.

Second: I have no Idea how to use the egtb cache feature as intended. Again, I cant' specify it for each engine (so it seems) - but the memory get claimed in any case. Whether both engines are able to use egtb does not interest xboard (is this assumption wrong?).

In the version of XBoard you are using (4.4.2), the EGT cache size and polyglotBook info from the dialog is only passed to Polyglot if you invoke the engine with the option -fUCI or -sUCI. Never if you invoke Polyglot explicitly by writing something like -fcp "polyglot fruit.ini". This is done because in that version (and older WB versions, upto Winboard_x), the -fUCI option lets XBoard prepare a polygot.ini temporary file, with the engine neme, directory, hash and cache size in it and pass that to Polyglot. As XBoard does not automatically send EGTB cache size to the engine (this is not a concept that WinBoard engines know, they just are told their memory limit and decide for themeselves how much of that to use for EGTB cache), Polyglot will smply use what is in the polyglot.ini file. (Either the synthetic one, or the one you provided). So it is easy to specify the EGTB cache per engine: just invoke Polyglot explicitly, with a poygot.ini file that specifies the cache size you want.

My case: letting some of the RobboLitos play with his own Tablebases. There is a cache option (up to 2048 mb) for that which the engines communicates. The 'cache' (from the polyglot.ini) value won't get accepted by xboard, but the directory will, so after changing it in commandline (setting -defaultCacheSizeEGTB 2048) xboard will claim an amount of 4096Mb memory because a match is played - besides as far as I can see, the cache won't get filled with useful stuff, it's just there.

Not completely sure what you are trying to explain here. XBoard never 'accepts' anything from the polyglot.ini file, it is completely unaware of it, unless it is a polyglot.ini file it made itself in response to the -fUCI / -sUCI option. The newer Polyglots, however, are aware of the meaning of the memory command XBoard sends them, and how that differs from the UCI option 'Hash'. So they will use the EGTB cache size specified in the polyglot.ini to send to the engine, and subtract that from the total amount of memory that XBoard says they can use, and communicates the left-over as Hash to the engine. Because hystorically the WinBoard dialog that specified hash size and EGTB cache size only worked for UCI engines, the total allowed memory usage is calculated by XBoard through adding those. So if you want both engines to run with 2GB, it really does not matter if you set -defautHashSize to 2048 and -defaultCacheSizeEGTB to 0, or set them both to 1024. (Unless you use the -fUCI option, which transmits them separatey to Polyglot through the synthetic polyglot.ini file. But if you don't want that, simply do not use the option.)

So after giving each engine 2048MB of memory, you can divide that up on a per-engine basis into cache and hash, by specifying the EGTB cache size in the polyglot.ini. So you could set it for Robolito to 2048, so it would run with 2048MB cache and 0MB hash, and for a UCI opponent to 4MB, so it will run with 4MB cache and 2044MB hash.

I am not sure what to make of your last remark. If an engine does not use the memory allocation it is granted, or fills it with useless data... That sounds pretty much like an engine bug to me. Not much XBoard or Polyglot could do against that.

Could you help me here H.G.M? How about a switch, to let xboard, when asked specifically, accept all of polyglot's option to play with? Would that be very ugly?

If you want Polyglot to ignore the memory and cores commands that XBoard sends it, why don't you simply use an old Polyglot version that does not support these options? It makes little sense to equip Polygot with an option "behave as a defective Polyglot" or XBoard with an option "make a good Polyglot behave as if it was a defective one" if there are already so many defective Polyglots around...

If you want Polyglot to ignore the memory and cores commands that XBoard sends it, why don't you simply use an old Polyglot version that does not support these options? It makes little sense to equip Polygot with an option "behave as a defective Polyglot" or XBoard with an option "make a good Polyglot behave as if it was a defective one" if there are already so many defective Polyglots around...

Hi Alex,
you can find a defective polyglot on my website(http://www.geenvis.net/pg.html).
It ignores the memory and cores commands (just as 99.9% of the winboard engines do ) and
reads the settings from the .ini file.
You can still set hash and egtb table size seperatly.
Best
Fonzy

@Teemu

Thanks for the info. 'Top' and 'free' tell me that a certain amount of memory is claimed for each process, when in fact it's shared. I did not know that (Rybka 2.3.2 a mp on Linux with microwine).

@F. Bluemers

Thank you for the offer. But I'm using already an old polyglot locally.

@H.G.M

Thank you for your endurance with me.

"So they will use the EGTB cache size specified in the polyglot.ini to send to the engine, and subtract that from the total amount of memory that XBoard says they can use, and communicates the left-over as Hash to the engine."

Now I got it.