WinBoard Protocol extensions

[Zach Wegner]

Indeed, for convenient operation of PSWBTM there must be an xboard.ini. I will see if I can still add one before we release, even if it cannot be saved yet.

Or perhaps, as a temporary kludge, I should provide an adater for invoking xoard by PSWTM, that does nothing else but reading xboard.ini, put whatever is in there as arguments before the arguments with which it was called, and then call xboard (through an exec call, no forking).

I'd say just leave it for now. I think everyone can wait a little while for this one feature, while many others are waiting for the release.

How exactly is the

feature option style list stupid patzer mediocre weak
feature option lmr_history_percent int 0 - 100

better then

askuser style Enter style: stupid patzer mediocre weak
askuser lmr_history_percent Give lmr history percentage:

?

From the viewpoint of the engine, it seems quite immaterial if you have to send one or the other.

Many reasons. Say you want to have a bunch of engines configured a certain way, and have the configuration saved externally. Having the GUI directly manipulate the parameters allows this. Also, the GUI can display all the available options in one place, and display their current values. The user can look at all of them at once and decide which one to change, rather than having to either enter a sane value into every dialog box that pops up (potentially quite a few), or not entering anything, while expecting the engine to properly handle blank input, etc. GUIs can validate input as well, so the user can't enter "one" for the history percent, etc. Finally, I think the askuser command is pretty ugly. Instead of standardizing the parameters as communication from the interface to the engine, it bypasses the interface and goes directly to the user, who might not want to answer all the dialog boxes, or type in "stupid", or who might not even be looking at the computer.

Also, I think prepending all of these option commands with "option" or something similar would be best, from an engine writer's point of view. If you look at Fruit, it doesn't keep integer representations of all its options, but merely an array of strings for option types and strings for their values. So a setoption command doesn't have to know anything about the options themselves, just that a command started with "setoption".

I disagree with Matthias about using the UCI syntax directly. So are we supposed to use options like UCI_hash? I think the syntax I proposed, of clearly delineating which options are standard and which are engine-specific, is much cleaner. And how hard is it really to program this stuff?

Code: Select all

char *c;
if (protocol == UCI)
    c = "setoption";
else if (protocol == XBOARD)
    c = "option";
...etc...

[Dann Corbit]

The reason I suggested a combination is that existing code could be folded together to handle both types of options.

I proposed a data model for this long ago.
Here is the ER/Win diagram:
http://cap.connx.com/chess-program-model.ER1
Here is a sample SQL file generated from the model:
http://cap.connx.com/chess-program-model.sql
It can be targeted to any SQL database.

[H.G.Muller]

I thought some more about this, and for options that are non-standard, I start to doubt what would actually be the use. Such options cannot be saved in the winboard.ini file, as the next time you run winboard with another engine, that won't understand any of them, or have different values for the options. So you would have to reconfigure the engine every time you play it (in human-engine play).

It seems to me that this is much more a task for an engine manager, that is capable of storing the settings of many engines simultaneously. That engine manager would then call WinBoard with one of the engines from its library, and could instruct WinBoard to send the rquired option commands to the engine. (E.g. as the initString, or computerString.) There really would be no need at all to see the Chess board (and hence invoke WinBoard) while you are configuring the options of the engine.

[Matthias Gemuh]

I thought some more about this, and for options that are non-standard, I start to doubt what would actually be the use. Such options cannot be saved in the winboard.ini file, as the next time you run winboard with another engine, that won't understand any of them, or have different values for the options. So you would have to reconfigure the engine every time you play it (in human-engine play).

It seems to me that this is much more a task for an engine manager, that is capable of storing the settings of many engines simultaneously. That engine manager would then call WinBoard with one of the engines from its library, and could instruct WinBoard to send the rquired option commands to the engine. (E.g. as the initString, or computerString.) There really would be no need at all to see the Chess board (and hence invoke WinBoard) while you are configuring the options of the engine.

ChessGUI shall be interested only in options that are common to both engines. The UCI convention handles them fine. "UCI_" can be changed to "WB_". Interesting options are WB_Hash, WB_NalimovPath, WB_NalimovCache, WB_BookPath, WB_Threads, all transmitted in one uninterrupted sequence (UCI style/format) to make it easy for GUI to build options window as for UCI engines.
The WB feature command is too messy to always provide these options in one piece.

Matthias.

[Miguel A. Ballicora]

I thought some more about this, and for options that are non-standard, I start to doubt what would actually be the use. Such options cannot be saved in the winboard.ini file, as the next time you run winboard with another engine, that won't understand any of them, or have different values for the options. So you would have to reconfigure the engine every time you play it (in human-engine play).

It seems to me that this is much more a task for an engine manager, that is capable of storing the settings of many engines simultaneously. That engine manager would then call WinBoard with one of the engines from its library, and could instruct WinBoard to send the rquired option commands to the engine. (E.g. as the initString, or computerString.) There really would be no need at all to see the Chess board (and hence invoke WinBoard) while you are configuring the options of the engine.

Winboard do not need to store the parameters. The engine should be responsible for updating "their own ini file" or whatever. The engine is responsible for providing default parameters, winboard may display them when the user request them, the user modifies them, winboard communicates that to the engine, and the the engines stores them or do whatever they want. This will be really fantastic.

I do not need Winboard to do any job for my engine, I just want WB to be an interface.

Miguel

[Zach Wegner]

I thought some more about this, and for options that are non-standard, I start to doubt what would actually be the use. Such options cannot be saved in the winboard.ini file, as the next time you run winboard with another engine, that won't understand any of them, or have different values for the options. So you would have to reconfigure the engine every time you play it (in human-engine play).

It seems to me that this is much more a task for an engine manager, that is capable of storing the settings of many engines simultaneously. That engine manager would then call WinBoard with one of the engines from its library, and could instruct WinBoard to send the rquired option commands to the engine. (E.g. as the initString, or computerString.) There really would be no need at all to see the Chess board (and hence invoke WinBoard) while you are configuring the options of the engine.

The big problem with this is that it's not standardized. Firstly, how would the user know what options there are, short of looking through the documentation for each individual engine, if it even exists? Second, how does the engine manager tell the engine to use the options? It has to know all the options and their syntax. This all seems to implicate some type of communication format, which I think the xboard protocol would handle much better.

I don't know what Winboard.ini is like, but the saving of options is just an extra. I would imagine that this would be saved in a separate file anyway. The reason why this is important to me is so that a user can, say, fire up the new Winboard with Fairy-Max, look at all the different variants it supports, and choose one. Or they can fire up the newest Crafty and look at the evaluation settings, or whatever... This has the distinct advantage of having a standard communication protocol that engine settings can be saved with, for tournaments etc. What Miguel says about not wanting Winboard to do that is understandable, but this is really about the communication protocol, not Winboard itself.

As a side note, I really like Dann Corbit's idea of a standard engine database, though his SQL code is pretty long. Such a database could hold option names and default settings, etc. etc. If we could get multiple GUIs to handle such a thing, then I think it could be quite useful. You could have manipulation programs to handle large amounts of engines' settings at once, and have them work in any GUI, or you could even zip the whole thing up along with a bunch of engines and have it just work. But I kind of doubt that the idea will take hold...

[Zach Wegner]

ChessGUI shall be interested only in options that are common to both engines. The UCI convention handles them fine. "UCI_" can be changed to "WB_". Interesting options are WB_Hash, WB_NalimovPath, WB_NalimovCache, WB_BookPath, WB_Threads, all transmitted in one uninterrupted sequence (UCI style/format) to make it easy for GUI to build options window as for UCI engines.
The WB feature command is too messy to always provide these options in one piece.

Matthias.

I'm not really sure what your objection is. Are you not satisfied with my proposal, or just H.G.'s? The syntax would be mostly the same except that the standard options that you mention would be included in one feature command, simply indicating that the engine supports the command. The reason is so they are implemented in exactly the same way in every engine. What's to stop an engine author from having this string in their options:

Code: Select all

option UCI_Hash spin Big Medium Small

?

Of course, engines may be different in what hash sizes they allow, so they should respond to such commands by specifying what they actually set the hash size to, a la UCI.

So how is this:

Code: Select all

feature stdoptions="hash,EGTBcache,nalimovpath,bookpath"
feature option style list stupid patzer mediocre weak
feature option lmr_history_percent int 0 - 100

...any easier to parse than:

Code: Select all

option UCI_hash ...
option UCI_NalimovPath ...
...

?

Additionally, I feel obligated to point out that the protocol is not _winboard_, so the WB prefix is not appropriate. If anything it should be XB, as xboard came first, and the protocol is specified by the string "xboard", but I think the fact that there are two different names means we should not use either. Because this is specific to the x/winboard protocol, why not just use hash, egtbpath, etc. The feature command defining them as the standard options sets them apart from the normal options enough IMO.

[H.G.Muller]

ChessGUI shall be interested only in options that are common to both engines. The UCI convention handles them fine. "UCI_" can be changed to "WB_". Interesting options are WB_Hash, WB_NalimovPath, WB_NalimovCache, WB_BookPath, WB_Threads, all transmitted in one uninterrupted sequence (UCI style/format) to make it easy for GUI to build options window as for UCI engines.
The WB feature command is too messy to always provide these options in one piece.

It seems to me that for these standard options, there is no need at all for the engine to tell the GUI what the possible settngs are. They are either numeric arguments or filenames. if there is a maximum, and the user would request mor than the maximum then engine can just use its maximum.

For these standard options, the option window is already there in the WinBoard menu. There really is not much to build, except that I could perhaps gray-out the items that WinBoard knows the engine does not provide. Except that the window now is used for both engines. But that is a GUI matter, not a protocol matter. It would be as easy to make the same dialog for each engine separately. (But do we really want to run hash-odds matches???)

So for the standard options, would there be really any advantage to not use the existing mechanism of sending

feature memory=1 cores=1 nalipath=1 bookpath=1

? For the numeric options, in stead of 1 the engine could give the maximum allowed value, if their is one. Like cores=4. (This would raise the problem of how to indicate infinity, though.)

Non-standard options could be handled in a completely differet way. I think there certainly is some merit to Miguel's idea of making each engine responsible for remembering its own settings. We should not dismiss that too easily.

[Zach Wegner]

Personally I think the option=maximum convention is a bit ugly, but it's interesting and worthy of consideration. I think -1 would be the best way to represent infinity (unless you want to use an INF string or something). But I think it's more intuitive to represent these standard options with just one feature, similar to the 'variants' feature.

Now for something completely different: I might be in the minority here, but I am strongly against the UCI standard's "endorsement" of Nalimov and Shredder bases. An open standard should have no association towards any proprietary standards. Yes, the Nalimov code is open source, but it is still very restrictive (e.g. it cannot be integrated into xboard due to the licenses, if anyone would want to commit such a heinous act). I would therefore want a way to define as a string the type of bases it uses, and the path as such. Perhaps egbpath for 'endgame base' would be best, as it is tablebase/bitbase neutral? Or am I just way out in left field? The engines could send a string egbtype=nalimov/scorpio/etc. and xboard/whatever can send the path string if it knows the egbase type.

I'm also not too fond of the use of "cores", as that term has only recently come into active use with the advent of multi-core CPUs. Maybe "processors" or "cpus" would be better.

Of course, these are all just my opinions. Feel free to disagree!

[H.G.Muller]

The big problem with this is that it's not standardized. Firstly, how would the user know what options there are, short of looking through the documentation for each individual engine, if it even exists? Second, how does the engine manager tell the engine to use the options? It has to know all the options and their syntax. This all seems to implicate some type of communication format, which I think the xboard protocol would handle much better.

OK, there definitely should be communication between engine and engine manager, and the standard I/O is a convenient channel for this. The engine manager would start up the engine, send it

xboard
protover 2

to evoke the response of sending the features command in a compatible way with existing engines, and look for a feature options=N for the number of settable non-standard options. It would then send the command 'settings' to the engine, and the negine could then send N lines describing the options and their alowed values. The engine manager would likely use this to build an option dialog popup, perhaps even askng what the current or default setting is (if this is not standard included in the option descriptor), and use the option commands specified by the engine to configure the latter.

As far as I am concerned, we would leave WB protocol after sending the 'settings' command. I don't know how the configuration of UCI engines is done, but I cannot see any reason not to do it in exactly the same way (making it easier for engine authors to support both protocols). The engine could remain sensitive to ordinary WB-protocol commands, (if the engine hadn't redefined them as option names) so that GUIs that are engine manager and GUI at the same time can start a game immedately after configuring, if they desire so.

I don't know what Winboard.ini is like,

winboard.ini is just a huge list of command-line options, one option per line. It is in effect pre-pended to the actual command-line option list, before the lot is processed.

but the saving of options is just an extra. I would imagine that this would be saved in a separate file anyway. The reason why this is important to me is so that a user can, say, fire up the new Winboard with Fairy-Max, look at all the different variants it supports, and choose one.

Well, this is already arranged in the feature variants={...} command. I do not grey-out choices that the engine doesn't support in the WB menu,, though. (Mainly because i don not know ho to do that...). There is a good point here, though: I defined 'fairy' as a kind of catch-all variant, allowing all pieces known by WinBoard to participate. It is quite conceivable that some engines know several variants that WB does not, and it could send a sub-variant option to let the user choose.

Or they can fire up the newest Crafty and look at the evaluation settings, or whatever... This has the distinct advantage of having a standard communication protocol that engine settings can be saved with, for tournaments etc. What Miguel says about not wanting Winboard to do that is understandable, but this is really about the communication protocol, not Winboard itself.

It is really up to the engine if it wants to remember its own settings the next time it is fired up, or revert to its defaults. An engine manager could take over the task of remembering what settings we want for every engine, and explicitly configure the engine every time we start it. (So that engines that do not remember behave s if they do.) Playing through WB, that would mean that the tournament manager would invoke WinBoard with options specifying exactly what to send to the engine in order to configure it.

The mechanism to do this already exists in WB / WB protocol: the InitStrings are meant for this. A tournament manager that knows Crafty is installed with the option Playing_Style=patzer, would simply invoke WinBoard with the option

/firstInitString="new\nrandom\nPlaying_Style=patzer\n"

[Roger Brown]

[SNIP]

Now for something completely different: I might be in the minority here, but I am strongly against the UCI standard's "endorsement" of Nalimov and Shredder bases. An open standard should have no association towards any proprietary standards. Yes, the Nalimov code is open source, but it is still very restrictive (e.g. it cannot be integrated into xboard due to the licenses, if anyone would want to commit such a heinous act).

Hello Zach,

I seriously never thought about this before but I believe that you are absolutely correct here.

I would therefore want a way to define as a string the type of bases it uses, and the path as such. Perhaps egbpath for 'endgame base' would be best, as it is tablebase/bitbase neutral? Or am I just way out in left field? The engines could send a string egbtype=nalimov/scorpio/etc. and xboard/whatever can send the path string if it knows the egbase type.

No, you are not way out in left field - well at least not alone if that offers any degree of comfort! So you envision the user having Nalimov, Shredder, Scorpio, Yace, Edwards and even Thompson bases and the engine would send the string indicating which base it uses.

Then the gui would send back the path for the various base indicated.

I think it is good to be neat and formal up front as well as allowing for maximum flexibility.

I'm also not too fond of the use of "cores", as that term has only recently come into active use with the advent of multi-core CPUs. Maybe "processors" or "cpus" would be better.

A matter of taste and again I agree with you on the terminology. Multiple names for the same thing invites ambiguity which is not a good thing.

Later.

[Zach Wegner]

It is really up to the engine if it wants to remember its own settings the next time it is fired up, or revert to its defaults. An engine manager could take over the task of remembering what settings we want for every engine, and explicitly configure the engine every time we start it. (So that engines that do not remember behave s if they do.) Playing through WB, that would mean that the tournament manager would invoke WinBoard with options specifying exactly what to send to the engine in order to configure it.

The mechanism to do this already exists in WB / WB protocol: the InitStrings are meant for this. A tournament manager that knows Crafty is installed with the option Playing_Style=patzer, would simply invoke WinBoard with the option

/firstInitString="new\nrandom\nPlaying_Style=patzer\n"

That is pretty much what I was thinking. The main point is that there is a standardized way of setting options. Whether Winboard itself is capable of remembering them isn't important IMO (though I would like it to), though I wouldn't want to expect the engines to do so. The tourney manager is a good place for this, but perhaps a different way to communicate the options to x/winboard should be investigated.

[H.G.Muller]

Now for something completely different: I might be in the minority here, but I am strongly against the UCI standard's "endorsement" of Nalimov and Shredder bases.

Although in principle I side with you here, we must remain pragmatic. No matter how much I dislike Nalimov's creations, I haven't put him out of business quite yet.

Even if there would be competing formats, it is unrealistic to expect a computer will install the complete sets of all of them. Even if disk space continues to sky-rocket, people will prefer to have complete sets of 6- or 7-men EGTBs in one format, rather than installing 10 different formats. So in practice engines wil be written such that they can handle several formats, and use the one the current tournament computer happens to have available.

This is why I wanted the feature nalimov=1, rather than egtbpath=1. An engine written to support only Nalimov EGTBs will not want to receive a pathname for a folder of Edwards tablebases, that would be completely useless to it.

So the mechanism I had in mind was to have the engine request the different formats separately. I was thinking about somthing like

feature scorpio=1 thompson=1 nalimov=1

and the GUI would respond by sendingg

scorpiopath ...
thompsonpath ...

But I agree that we should perhaps organize it more like the variants feature, where the GUI can treat the name of the variants like arbitrary text that is not really part of the protocol. So

feature egtb="scorpio,nalimov"

and the GUI would reply by sending

egtbpath nalimov H:\chess\nalimov
egtbpath scorpio H:\chess\bitbases

i.e. in general

egtbpath EGTBTYPE PATHNAME

where EGTB type would match one of the strings the engine sent. This makes it possible to accomodate future formats without canhging the GUI or the protocol. WinBoard would know the pathmase through options like

/egtbPath="nalimov:H:\chess\nalimov;scorpio:H:\chess\bitbases"

[Zach Wegner]

Although in principle I side with you here, we must remain pragmatic. No matter how much I dislike Nalimov's creations, I haven't put him out of business quite yet.

Even if there would be competing formats, it is unrealistic to expect a computer will install the complete sets of all of them. Even if disk space continues to sky-rocket, people will prefer to have complete sets of 6- or 7-men EGTBs in one format, rather than installing 10 different formats. So in practice engines wil be written such that they can handle several formats, and use the one the current tournament computer happens to have available.

This is why I wanted the feature nalimov=1, rather than egtbpath=1. An engine written to support only Nalimov EGTBs will not want to receive a pathname for a folder of Edwards tablebases, that would be completely useless to it.

So the mechanism I had in mind was to have the engine request the different formats separately. I was thinking about somthing like

feature scorpio=1 thompson=1 nalimov=1

and the GUI would respond by sendingg

scorpiopath ...
thompsonpath ...

But I agree that we should perhaps organize it more like the variants feature, where the GUI can treat the name of the variants like arbitrary text that is not really part of the protocol. So

feature egtb="scorpio,nalimov"

and the GUI would reply by sending

egtbpath nalimov H:\chess\nalimov
egtbpath scorpio H:\chess\bitbases

i.e. in general

egtbpath EGTBTYPE PATHNAME

where EGTB type would match one of the strings the engine sent. This makes it possible to accomodate future formats without canhging the GUI or the protocol. WinBoard would know the pathmase through options like

/egtbPath="nalimov:H:\chess\nalimov;scorpio:H:\chess\bitbases"

This looks pretty good to me, though I still think "egb" is a little more accurate. Supplying different paths for different formats is an important point that I hadn't thought of. I only know of Yace, but possibly there are other engines that support multiple types. The flexibility is important IMO, so that GUIs don't have to be updated with whatever new formats come out. They would be responsible for aggregating the different EGB types based on the strings, so I think we should keep a list of standard names for the formats, so different engines don't report Nalimov, nalimov, egtb, or whatever weird string someone can think up. Pragmatism in action!

[H.G.Muller]

Agreed, we would have to enforce some standardizaton of names, so that all engines will use the same name for a certain format. But I guess formats will not appear so frequently and rapidly that this would be a problem.

I suppose you did not like the 't' in egtb necause you consider bitbases not really tablebases? Let me suggest then we use the term with which Guy Haworth refers to them in general: EGT for end-game tables. So we would use 'feature egt' and 'egtpath'.

[Miguel A. Ballicora]

Agreed, we would have to enforce some standardizaton of names, so that all engines will use the same name for a certain format. But I guess formats will not appear so frequently and rapidly that this would be a problem.

I suppose you did not like the 't' in egtb necause you consider bitbases not really tablebases? Let me suggest then we use the term with which Guy Haworth refers to them in general: EGT for end-game tables. So we would use 'feature egt' and 'egtpath'.

What would happen if I develop my own EGTs for my own engine? Do I need the protocol to be updated? How will that work?

Miguel

[Zach Wegner]

I suppose you did not like the 't' in egtb necause you consider bitbases not really tablebases? Let me suggest then we use the term with which Guy Haworth refers to them in general: EGT for end-game tables. So we would use 'feature egt' and 'egtpath'.

Sure, I don't really care that much. EGT is fine. My point is that EGTB and EGBB are used pretty much universally. But the word "tablebase" in the first place is pretty stupid, so maybe EGT is a better term, especially if it is already used.

[Miguel A. Ballicora]

Agreed, we would have to enforce some standardizaton of names, so that all engines will use the same name for a certain format. But I guess formats will not appear so frequently and rapidly that this would be a problem.

I suppose you did not like the 't' in egtb necause you consider bitbases not really tablebases? Let me suggest then we use the term with which Guy Haworth refers to them in general: EGT for end-game tables. So we would use 'feature egt' and 'egtpath'.

What would happen if I develop my own EGTs for my own engine? Do I need the protocol to be updated? How will that work?

Miguel

EGTs should not be very different that an opening book, there are supporting files of a specific engine. I do not see the need to be so specific with the names of EGTs hardwired in the protocol (unless I misunderstood something).

Miguel

[Zach Wegner]

What would happen if I develop my own EGTs for my own engine? Do I need the protocol to be updated? How will that work?

Miguel

No, as the GUIs don't store won't store any predefined types. It will just say "For each engine that uses 'Gaviota-bases' look in this folder: ...". So each engine can just use feature egt="gaviota", and the GUI will match that string to whatever path the user has provided.

If you do that though, you should publicly post a name for your format, and we can keep a list somewhere of standard names, so that any other engine that might use them can use the same path.

[Miguel A. Ballicora]

ChessGUI shall be interested only in options that are common to both engines. The UCI convention handles them fine. "UCI_" can be changed to "WB_". Interesting options are WB_Hash, WB_NalimovPath, WB_NalimovCache, WB_BookPath, WB_Threads, all transmitted in one uninterrupted sequence (UCI style/format) to make it easy for GUI to build options window as for UCI engines.
The WB feature command is too messy to always provide these options in one piece.

Matthias.

I'm not really sure what your objection is. Are you not satisfied with my proposal, or just H.G.'s? The syntax would be mostly the same except that the standard options that you mention would be included in one feature command, simply indicating that the engine supports the command. The reason is so they are implemented in exactly the same way in every engine. What's to stop an engine author from having this string in their options:

Code: Select all

option UCI_Hash spin Big Medium Small

?

Of course, engines may be different in what hash sizes they allow, so they should respond to such commands by specifying what they actually set the hash size to, a la UCI.

So how is this:

Code: Select all

feature stdoptions="hash,EGTBcache,nalimovpath,bookpath"
feature option style list stupid patzer mediocre weak
feature option lmr_history_percent int 0 - 100

...any easier to parse than:

Code: Select all

option UCI_hash ...
option UCI_NalimovPath ...
...

?

Additionally, I feel obligated to point out that the protocol is not _winboard_, so the WB prefix is not appropriate. If anything it should be XB, as xboard came first, and the protocol is specified by the string "xboard", but I think the fact that there are two different names means we should not use either. Because this is specific to the x/winboard protocol, why not just use hash, egtbpath, etc. The feature command defining them as the standard options sets them apart from the normal options enough IMO.

Nobody uses it, but AFAIK the name of the protocol is CECP (Chess engine communication protocol) accoring to Tim Mann is his website.

Miguel