PSWBTM engine manager: engine options

[H.G.Muller]

We are currently brain-storming about how to handle engine options in the WinBoard package. Currently they can only be handled for UCI engines, by the cumbersome method of making polyglot.ini files, perhaps with the aid of an external utility. I would like to have this discussion here in public.

One necessary ingredient is to have a standard way of engines announcing the options they need or accept, like UCI engines already do. This requires an extension of WB protocol, which in itself can be easily made.

The main problem is how to use it. As an experiment I have added a menu popup in XBoard, which displays all options the engine defined. The user then can enter new values, and they are set on the engine. For some types of options it is convenient to be able to change them interactively from the menu, (e.g. playing style), mainly in human-engine play.

For options with a more permanent character (for customizing the engine to a fixed personal taste), it is almost useless, as the settings disappear when you exit XBoard. There is currently no way to specify such options through the XBoard command line: unlike the standard options, the engine-defined options are different for all engines. WinBoard does not keep a database of engines where it remembersa large number of settings for each engine. (There is a primitive list for the benefit of the start-up dialog, though.)

A much more logical place to do this would be in the PSWBTM engine manager. There it could be really useful. What I imagine is this:

The PSWBTM engine-manager dialog gets an extra button, "Configure Engine". When pressed, PSWBTM starts up the engine directly (without using WinBoard as an intermediary), and interrogates it for options. It does this by sending it an uci or xboard + protover command, to figure out if it is UCI or WB, and records the option or feature commands the engine replies with. It then creates a popup dialog where the user can alter the engine settings through suitable control elements, like spins, combo boxes, etc.

PSWBTM then maintains, as part of its engine database, for each engine the value of the WinBoard InitString, which by default is "new\nrandom\n". For any option that the user has altered compared to the default setting, PSWBTM prefixes this default InitString by "option NAME VALUE\n". The corresponding value of the InitString would then be passed to WinBoard as value for the -firstInitString or -secondInitString option (as appropriate) whenever PSWBTM invokes WinBoard with that engine.

If the engine turns out to be UCI, PSWBTM would automatically replace it with polyglot. (For this it would not only need the path to the WinBoard executable, but also to Polyglot, to be entered in its own configuration menu.) Under Windows it would use a polyglot.ini file in the engine directory as argument to polyglot (and create a dummy one there if it did not exist). This would basically be used only for communicating the engine command to Polyglot; everything else in it would be overruled by the option commands WB will send to Polyglot at the beginning of every game, as specified in the PSWBTM engine database.

What do you think of this scheme?

WinBoard itself would not really have to know anything about the option features; PSWBTM would essentially bypass it by hiding them in the InitString. (It might confuse the engine, though, if WinBoard first sends rejected option, and then send option commands anyway, so it is probably better to use a WinBoard that does know the option feature and accepts it.) An alternative would be to equip WinBoard with command-line options -firstOptions and -secondOptions which could contain (as a text string) a list of options to be send at startup, so that future WinBoard versions, that do have a menu to set engine options, could be aware of the fact that engine default settings have been overruled, by scanning the option strings and extracting the values to display them as current settings. Then interactive users could have the best of both worlds, launching the engine from the PSWBTM engine manager to get their customized option settings at start-up, and changing volatile options through the WB menu during the session.

[Zach Wegner]

What do you think of this scheme?

IMO it's pretty ugly. One of the most important aspects of this option scheme is that it can eliminate engine-specific init files, including polyglot.ini files. Creating new .ini files or putting all options on the command line are to me very ugly hacks. This exposes an IMO severe limitation of using a separate engine manager that doesn't directly handle the engines.

[H.G.Muller]

I am not sure I understand your objections. What is wrong with passing information as command-line options? It is what command-line options are meant for, not? So why would that be an ugly hack?

I always felt that it would be much more natural to pass info that is only needed at start-up, rather than interactively, through command-line arguments, rather than through a pipe. E.g. to tell Polyglot which engine to run, I would prefer a command-line option to Polyglot over sending it through WB protocol.

[Miguel A. Ballicora]

We are currently brain-storming about how to handle engine options in the WinBoard package. Currently they can only be handled for UCI engines, by the cumbersome method of making polyglot.ini files, perhaps with the aid of an external utility. I would like to have this discussion here in public.

One necessary ingredient is to have a standard way of engines announcing the options they need or accept, like UCI engines already do. This requires an extension of WB protocol, which in itself can be easily made.

The main problem is how to use it. As an experiment I have added a menu popup in XBoard, which displays all options the engine defined. The user then can enter new values, and they are set on the engine. For some types of options it is convenient to be able to change them interactively from the menu, (e.g. playing style), mainly in human-engine play.

For options with a more permanent character (for customizing the engine to a fixed personal taste), it is almost useless, as the settings disappear when you exit XBoard.

Shouldn't the engine be in charge of remembering its own setup?

Miguel

There is currently no way to specify such options through the XBoard command line: unlike the standard options, the engine-defined options are different for all engines. WinBoard does not keep a database of engines where it remembersa large number of settings for each engine. (There is a primitive list for the benefit of the start-up dialog, though.)

A much more logical place to do this would be in the PSWBTM engine manager. There it could be really useful. What I imagine is this:

The PSWBTM engine-manager dialog gets an extra button, "Configure Engine". When pressed, PSWBTM starts up the engine directly (without using WinBoard as an intermediary), and interrogates it for options. It does this by sending it an uci or xboard + protover command, to figure out if it is UCI or WB, and records the option or feature commands the engine replies with. It then creates a popup dialog where the user can alter the engine settings through suitable control elements, like spins, combo boxes, etc.

PSWBTM then maintains, as part of its engine database, for each engine the value of the WinBoard InitString, which by default is "new\nrandom\n". For any option that the user has altered compared to the default setting, PSWBTM prefixes this default InitString by "option NAME VALUE\n". The corresponding value of the InitString would then be passed to WinBoard as value for the -firstInitString or -secondInitString option (as appropriate) whenever PSWBTM invokes WinBoard with that engine.

If the engine turns out to be UCI, PSWBTM would automatically replace it with polyglot. (For this it would not only need the path to the WinBoard executable, but also to Polyglot, to be entered in its own configuration menu.) Under Windows it would use a polyglot.ini file in the engine directory as argument to polyglot (and create a dummy one there if it did not exist). This would basically be used only for communicating the engine command to Polyglot; everything else in it would be overruled by the option commands WB will send to Polyglot at the beginning of every game, as specified in the PSWBTM engine database.

What do you think of this scheme?

WinBoard itself would not really have to know anything about the option features; PSWBTM would essentially bypass it by hiding them in the InitString. (It might confuse the engine, though, if WinBoard first sends rejected option, and then send option commands anyway, so it is probably better to use a WinBoard that does know the option feature and accepts it.) An alternative would be to equip WinBoard with command-line options -firstOptions and -secondOptions which could contain (as a text string) a list of options to be send at startup, so that future WinBoard versions, that do have a menu to set engine options, could be aware of the fact that engine default settings have been overruled, by scanning the option strings and extracting the values to display them as current settings. Then interactive users could have the best of both worlds, launching the engine from the PSWBTM engine manager to get their customized option settings at start-up, and changing volatile options through the WB menu during the session.

[H.G.Muller]

Shouldn't the engine be in charge of remembering its own setup?

This was may original idea (mainly to shove the problem onto someone else's plate ), but this idea meets with much resistance. I even added a special 'save button' class of options (next to the 5 types known from UCI), which acts as normal buttons (instantly sending an option command to the engine as they are pressed), but force XBoard to flush all other option settings that were not OK'ed yet to the engine first. Engines could use this to define one or more save commands (e.g. for different groups of options) if they wanted.

I cannot deny that having the engine do this does lead to multiplication of ffort, which can be avoided by centralizing the storage of the option settings in GUI or TM.

[Miguel A. Ballicora]

Shouldn't the engine be in charge of remembering its own setup?

This was may original idea (mainly to shove the problem onto someone else's plate ), but this idea meets with much resistance.

Correct me if I am wrong, but who store the information is not very relevant to the protocol itself, but relevant to the implementation. Correct? In fact, the engine can choose to store it every time, and even a GUI can choose to remember it too, if designed that way.

Maybe it is possible to add a feature command "GUI_stores_the_options=1 or =0" (or whatever name). After the engine sends it, then the GUI can accept it or reject it. With this in mind, engines will know what the GUI will do. The engine will know the GUI capabilities.

A GUI writer can choose to support this feature or not (or delay it for a future development) using "reject". In case it is not supported (command rejected), the engine writer can choose to store the information is too important to remember. If the engine programmer wants the engine to remember this no matter what, the engine would not even send the command or send "GUI_stores...=0" etc., (depending on the default)

In this way, you can go step by step. Winboard may not store the information now, but it may in the future.

Just our of curiosity, what UCI does with this?

Miguel

I even added a special 'save button' class of options (next to the 5 types known from UCI), which acts as normal buttons (instantly sending an option command to the engine as they are pressed), but force XBoard to flush all other option settings that were not OK'ed yet to the engine first. Engines could use this to define one or more save commands (e.g. for different groups of options) if they wanted.

I cannot deny that having the engine do this does lead to multiplication of ffort, which can be avoided by centralizing the storage of the option settings in GUI or TM.

[H.G.Muller]

You are fully correct: this is not a protocol matter. The protocol should be flexible enough to allow any implementation that GUI or engine authors prefer. I think the current proposal to add the commands

feature option="NAME -check VALUE"
feature option="NAME -spin VALUE MIN MAX"
feature option="NAME -string VALUE"
feature option="NAME -combo VALUE /// VALUE ..."
feature option="NAME -button"
feature option="NAME -save"

(all engine -> GUI) and

option NAME VALUE

(GUI -> engine) provide that flexibility.

Engine authors that want their engine to remember its own settings, or part of its settings, can implement this by defining one or more save buttons in the engine-options menu.

I am not sure if it is useful for the engine to know if the GUI saves the options or not. For one, it might not be the GUI that saves them, but the tournament manager, and the GUI might not know it. But if an engine can save its settings, and the user asks it to do so by pressing the corresponding buttons why would you want the engine to refuse it depending on what the GUI does? Stored GUI options will always override the stored engine options, as the engine loads the latter at start-up, and the GUI will sent its own settings later, overruling those. So for GUIs that save, it is irrelevant what the engine does.

But there are perfectly legitimate reasons why a user could want an engine to save its own settings, even on a GUI that also saves settings: Perhaps he is preparing to use the engine without GUI, or on another GUI that does not save its settings.

[Miguel A. Ballicora]

You are fully correct: this is not a protocol matter. The protocol should be flexible enough to allow any implementation that GUI or engine authors prefer. I think the current proposal to add the commands

feature option="NAME -check VALUE"
feature option="NAME -spin VALUE MIN MAX"
feature option="NAME -string VALUE"
feature option="NAME -combo VALUE /// VALUE ..."
feature option="NAME -button"
feature option="NAME -save"

(all engine -> GUI) and

option NAME VALUE

(GUI -> engine) provide that flexibility.

Engine authors that want their engine to remember its own settings, or part of its settings, can implement this by defining one or more save buttons in the engine-options menu.

I am not sure if it is useful for the engine to know if the GUI saves the options or not. For one, it might not be the GUI that saves them, but the tournament manager, and the GUI might not know it. But if an engine can save its settings, and the user asks it to do so by pressing the corresponding buttons why would you want the engine to refuse it depending on what the GUI does?

No, that is not what I intended to say. In fact, NAME -save could do (at least partially) what I mentioned.

Correct me if I am wrong:

When the engine sends the feature option NAME -save, it is saying "I am ready to save the info if you want me to". When the GUI sends back "option NAME" is saying, "Yes, I want you to save the information".

If the GUI, the user, the manager or any other interacting program is willing to save the info, then the GUI will not send NAME back. If GUI is saving anything, particularly if the user wants to save the info, the engine should know about it. Then, the GUI, the manager, or whatever, should send NAME to tell the engine to save it. This is obviously triggered by the user clicking "save" (with one exception), but it could be triggered by the GUI itself if the GUI is setup or designed not to save anything. The exception should be when the user clicks "save" when the GUI is setup to save the whole thing. Then the GUI, can choose not to send NAME back, since the GUI is saving the information.

Is this the spirit? I think it's fine. Then, the protocol would provide the tools for engine and GUI writers to do whatever is most convenient. The rest is implementation (which is another discussion altogether).

You may decide that in winboard is better to do one thing or another, but another GUI writer may choose to do the opposite. This flexibility is good, IMHO.

Miguel

Stored GUI options will always override the stored engine options, as the engine loads the latter at start-up, and the GUI will sent its own settings later, overruling those. So for GUIs that save, it is irrelevant what the engine does.

But there are perfectly legitimate reasons why a user could want an engine to save its own settings, even on a GUI that also saves settings: Perhaps he is preparing to use the engine without GUI, or on another GUI that does not save its settings.

[H.G.Muller]

The way I currently implemented it is this: when the engine sends feature option="NAME -save" the GUI does not take any automatic action, but just remembers it amongst the other option features. Only when the user clicks the "Option -> Engine #N Settings" menu item for that engine, the first effect of this option materializes, in the form of a button in the popup dialog with the text NAME on it. Only when the user presses that button, all other settings in that menu dialog that were not OK'ed yet will be sent to the engine in option OTHERNAME VALUE commands, followed by option NAME. The engine should then do what NAME suggests, and can be sure that its option settings are completely up to date at this time.

In a GUI that would implement saving of engine options itself, I imagine that the "Options -> Engine #N Settings" popup would contain a GUI-defined 'Save' button, that would store the settings in the way the GUI likes to store them, without the engine knowing anything about it. In my XBoard trial version, the only GUI-defined controls in this popup dialog are an 'OK' and a 'cancel' bottom.

I don't think it is a good idea to have such a GUI suppress the sending of the option NAME command to the engine, or the appearance of the 'NAME' button. At least, I would never want to use such a GUI that prevents me from controlling the engine: GUIs are there to facilitate me doing wat I want done, not to patronize me and sabotage my efforts. And apart from that, there would be a huge practical difficulty of recognizing the options. The fact that they are of the '-save' type does not prove they are actually 'Save' options, only that the engine wants to force the GUI to flush all option settings before it treats the command. It could, for example, be a 'Change Pawn-Hash Size Now' button, that forces the engine to change tha Pawn hash during the current search, where it would normally only do it after the game, which wanted to make sure that the value you just typed in the Pawn-Hash Size field would be the one used as the new size. And the NAME might be something like 'Alter Engine Defaults', or 'Remember Settings' or 'Make Permanent'...

If I would implement autoUpdate of the WinBoard start-up-dialog engine lists, I would not do it in these menu popups, but make a more general menu item for it in the File menu. (e.g. "File -> Install Engine #N".) This because installing the engine in the start-up list is not tied to engine-defined options per se: it would already be a useful feature with all currently existing engines, as it could add the settings of all volatile WinBoard options (-fd, -fUCI, -variant, -NoOwnBookUCI, ICS options) with the start-up entry for this engine (in so far they are not at their default setting).

[Michel]

I still to not see why the engine saving the options is such a good idea.

If the engine plays against itself with different settings the saved settings would be useless and the TM of GUI would have to set the settings anyway.

To implement this on the engine side the engine should be able to save settings under different names (or personalities) and the GUI would supply such names.

But it seems to me that this is shifting more and more burden upon the engine (and the resulting duplication of effort) which naturally belongs to the GUI.

I had the following proposal.

xboard would get new options -firstChessProgramPersonality -secondChessProgramPersonality

(short: -fp, -sp)

xboard would use these personalities for persistence of options. If these options were not supplied xboard would use the default engine name.

This would entail a minimal amount of code for xboard to implement persistence. No engine changes would be necessary.

A typical command line would be

xboard -fcp fruit -scp fruit -sp enhancedkingsafety