Polyglot 1.4.30+

[H.G.Muller]

I see that the patch-level number of Polyglot is sky-rocketing.

Is it described somewhere what all these new versions do compared to their predecessors? There now also seem to be strange new options, like UCI_aboutbox. I don't think that such options belong in the Engine-Settings dialog: options starting with UCI_ are supposed to be general otions understood and handled by the GUI. They should be translated to the corresponding WB commands in stead. For UCI_aboutbox the equivalent WB-protocol command seems to be feature myname="...".

[Michel]

The changes are discussed in the changelog.

I don't know about UCI_aboutbox. It does not appear in the source of PG. This must be something from an engine.
PG's equivalent of myname is the EngineName option which is currently only settable from the config file (I think it does not
make sense for xboard to set the EngineName).

I should probably filter out all UCI_ options and try to inteprete the ones I can.

I only added two genuine new non work around options. BookDepth and BookTreshhold. I would like to add more such options such
as the ability to use a hierarchy of books (I noticed that the aquarium adapter has this feature). You know Polyglot also understands the UCI protocol and serves as a book adapter for polyglot books for UCI gui's such as Arena and ChessBase.

I think xboard should either filter out all PG options or none at all. In the first case (which was my original point of view) it could set some options internally and leave the rest to people who want to use config files (I for one am quite comfortable with them and Marc Lacrosse told me once he prefers config files over other ways of setting options).

I don't think it is workable do discuss for every single PG option if it has the right to appear or not on the limited screen space that is available.

[Michel]

These are the options rybka 2 sends which are not yet filtered. They are all defined in the protocol.

feature option="UCI_ShowRefutations -check 0"
feature option="UCI_LimitStrength -check 0"
feature option="UCI_Elo -spin 1200 1200 2400"
feature option="UCI_Opponent -string <empty>"
feature option="UCI_EngineAbout -string http://www.rybkachess.com"

UCI_EngineAbout It seems to be that this information
should be communicated to the user somehow although it is not obvious how.

UCI_Opponent is already handled internally by PG so it should be filtered.

UCI_LimitStrength -check 0"
UCI_Elo -spin 1200 1200 2400"

These appear to be useful. Is there an equivalent for these in the WB protocol?

UCI_ShowRefutations -check 0

Not sure what this does.

The standard defines one more UCI_ option.

UCI_ShowCurrLine, type check,

This gives the line the engine is currently calculating.

[H.G.Muller]

Ah, of course the cange log that is in the debian packages.

OK, the aboutbox must be my mistake then. I saw it when I was trying Glaurung on Linux, and because I did not recall having seen in in the Windows Glaurung version I jumped t the conclusion is was from the new Polyglot. But the Glaurung versions could have been different, of course. (And I might have mis-remembered the name, which could very well have been EngineAbout; at the time I was typing the message I had already switced back to Windows.)

I agree the book options are good to have; In the current scheme of things (with XBoard only supporting common GUI book) Polyglot is still responsible for managing private engine books.

Some of the UCI_ options seem indeed useful. I will give it some more thought.

[H.G.Muller]

A small side-track: I see that the default setting of the weight threshold is 5. Wouldn't it be better to make it 2, to reproduce old behavior as much as possible? Then it supports the Scid kludge of setting 1 for weights of moves that it doesn't want to be played by default. But moves with weight 2 and higher would still be played. There might be books around with weights 3, 4, 5, 6 for four moves, and not playing the 3 & 4 anymore is kind of a huge change. (And for books made from PGN collections with current weight assignment there will always be a transition region where these frequencies occur.)

It is not so much a point if it would be in general better to ignore such weights, but more to provide maximum backward compatibility.

(My first impression would be that a uniform choice in the weight case 3, 4, 5, 6 would be more justified as making it 0, 0, 5, 6. Or does the filter threshold only act on the sum of the weights?)

[Michel]

I see that the default setting of the weight threshold is 5.

The weight treshold is relative to the total weight. There are many books which contain absolute weights of 1.
For example I seem to recall that performance.bin has (c4,1)(d4,1)(e4,1) in the opening position.

5=0.5%. This means PG will ignore moves which are shown as 0 in the SCID gui (SCID shows only two significant digits).
For example well known large books such as Elo2400.bin and TDDBII.bin contain many moves with weights <0.5% (such as a3 and h3) and users
are then quite surprised if the engine suddenly decides to play them. Of course this happens extremely rarely so the effect on ratings should be negligeable. But it is
psychologically a problem.

Since SCID is the defacto PG book tuning utility it seemed to me that 5 is a reasonable default.
In my opinion moves with very low (relative) weights are somehow artifacts of the automated book making process and you don't
really want to consider them. I you truly want such moves to be played you can set the weight higher manually (or set BookTreshhold to zero).

I wonder if Marc Lacrosse has any input on this.

[H.G.Muller]

I see. It is a percentage. Perhaps this should be made obvious from the name of the option? Like calling it "Polyglot BookThreshold(x0.1%)"? WB protocol puts no restriction on the characters you can use in an option name.

I guess a standard way for probing polyglot books (and tuning that probing through options) should be developed, so that we can WB and Polyglot probe the books in the same way. It should not matter if a book is probed by the GUI on behalf of both engines, or by Polyglot. But I guess we first have to develop a better prescription of deriving the weights, to solve the problems Marc pointed out.

OK, back to the main topic. I actually do think it is worthwile to review all the Polyglot options to see which would be useful in the Engine Settings dialog and which would only cause distraction / confuson. After all, there aren't that many. We currently have the following control groups:

1) Log + LogFile
2) UseNice + NiceValue
3) Chess960
4) Resign + ResignMoves + ResignScore
5) MateScore
6) Book + BookFile + BookRandom + BookDepth + BookThreshold + BookLearn
7) KibitzMove + KibitzPV + KibitzCommand + KibitzInterval + KibitzDelay
8) ShowPonder
9) UCIVersion
10) PromoteWorkAround
11) CanPonder
12) SyncStop
13) RepeatPV
14) ScoreWhite
15) WbWorkArounds + WbWorkArounds2

Things one would obviously want to have in the Engine Settings dialog, because it is exactly meant for that kind of stuff, would be:
4) Resign control
6) Book control

This probably also holds for the somewhat more "technical" options needed for engines that somehow give you trouble:
1) Logging
8) ShowPonder
9) UCIVersion
10) PromoteWorkaround
11) CanPonder
12) Syncstop
Users would not change these setting for a given engine on a daily basis, but if this dialog also serves as an interface to configure the engines after installation (which is the intention), these options should be there so people cn set it to the unusual non-default needed to suppress problems.

That leaves the following options that could be discussed:
2) Nice Control
This does seem a task for the GUI (as it is not clear why UCI engines would have to be niced, and WB engines not). The OS provides means to impose this on the engines, so I think it is a good idea to let the GUI indeed impose it on the engines, as opposed to requesting the engines to nice down themselves through protocol commands. So it seems that this option is only implemented in Protocol as work-around for a fundamental defect in GUIs.
Nicing down engines potentially affect their playing strength, usually in an unpredictable way. So it is not really useful as a device for handicapping engines. Therefore it would be best if there existed a global GUI setting to control the engine priority. It might be useful to set this interactively, through a menu, making the engines would react immediately to any change of the setting.
Now there might be a problem with this ideal: The GUI would set the priority of an adapter, not knowing that it was an adapter, and the engine might not be affected. I have seen that Polyglot uses setpriority() for the purpose of changing the engine priority in real time, ad that this call supports setting of the priority of process groups. Could this be a solution to the problem? (i.e. have Polyglot create the engine process in the same group it is in, so that the GUI can set the nice of both engine and adapter at once?)

3) Chess960
This option duplicates (or interferes with) the variant and feature variants="..." and should never be transmitted to the GUI.

5) MateScore
Unfortunately WB protocol does not define a default for mate scores. Even if a mate score is known to have a value, it is not defined how the engine would count towards that value (per ply or per move). Once WB protocol would define or recommend a way to represent matescores, we would not want to provide a user with the option merely to do things in a non-compliant way. But even now, without a standard for mate scoring, I see very little use for this option when WB is the GUI. So my first impression is that the default value of 10000 is fine and doesn't require user tuning. (Or is this centi-Pawn and not Pawn?) So no need for this option in the Engine Settings dialog.

7) Kibitz
Under WinBoard (at least under all versions that would understand this feature), I consider this a very damaging option package. We don't want people to decide (e.g.) what ICS command to use for kibitzing, there should be a standard for that, controlled by the GUI. So other software can recognize the kibitzed info and process it. Kibitzing in fact duplicates information that was already sent as Thinking Output. So having engines kibitz themselves is actually poor design. (And by extension that also holds for adapters, as they are supposed to mimic engines.) I can imagine that this option is useful for GUIs that do not support standard kibitzing, but as these GUIs don't implement the option feature it would be enough to control that via the polyglot.ini.
Apart from that, this reminds me of another issue: It is important to have WinBoard understand the kibitz format of Polyglot when it sees it from opponents foolhardy enough to use it through the ICS. Preferably the Polyglot format should be adapted to facilitate recognition of the final kibitz line that belongs to the played move. Otherwise the process of incorporating opponent scores in the PGN would become quite error prone, as ICS are wrought with timing problems.

13) RepeatPV
I am not sure what this option is for. I see no advantage for WB when the PV is repeated. It would just lead to duplication of info in the Engine-Output window. Yet the default of this option is "on".

14) ScoreWhite
This option duplicates a GUI feature to invert the black score. Logically, this is a GUI task, as native WB engines that invert the black score exist as well. I would like to see this option dropped as quickly as possible from the Engine Settings dialog, as it only invites people to do things the non-compliant way. Of course the option itself can be useful as a workaround for GUIs that do not support optional flipping of the black score. (I.e. as an option in the polyglot.ini, as such GUIs typically also reject the option features.)

15) WorkArounds
The WbWorkArounds do not seem useful in the Engine Settings dialog. The first one could only be set after the damage is already done. The second one has been solved. No official WB release that understands these option features would have the problems they are supposed to provide work-arounds for.

[F.Huber]

OK, back to the main topic. I actually do think it is worthwile to review all the Polyglot options to see which would be useful in the Engine Settings dialog and which would only cause distraction / confuson.

Since you said you don't know how to create such tabbed windows (as I mentioned in an earlier post), what about just splitting the original engine settings and the polyglot settings (for each engine) into different menu entries in the Options menu in WB?

Something like that:
Engine #1 settings ...
Polyglot #1 settings ...
Engine #2 settings ...
Polyglot #2 settings ...

[Michel]

I see. It is a percentage. Perhaps this should be made obvious from the name of the option?

Well the option in described in the manpage and readme file.

BookTreshold (default: 5)
Do not play moves with a weight (probability) lower than this (in per mil).

( EDIT: Silly me: BookTreshold appears to be the correct spelling. Not BookTreshhold as I momentarily thought)

OK, back to the main topic. I actually do think it is worthwile to review all the Polyglot options to see which would be useful in the Engine Settings dialog and which would only cause distraction / confuson.

I don't really see the point of such a discussion. Winboard has full control over which options it wants to show (if any).I made it easy for the GUI to filter out the options it is not interested in (this is even stated explicitly in the manpage).

Options which normally appear in the [PolyGlot] section of the config file (see below) are exported as options with their name prefixed by "Polyglot". This
makes it easy to filter them in the GUI.

The meanings of the options are completely described in the manpage and readme file (if not then it is a bug). As I see it: PG controls the UCI engine and the GUI controls PG.

I should point out that winboard is not the only gui for polyglot. icsdrone is another one. Furthermore Polyglot can also be used in Arena and Chessbase as UCI Polgylgot book adapter. It would be unreasonable to ask Polyglot to make the options it sends depend on the GUI it is playing under.

[H.G.Muller]

Well, who ever reads a manual...?

I agree it would be unreasonable to expect Polyglot to make its behavior dependent on the GUI. But it is just as unreasonable to require the behavior of the GUI dependent on the adapter. There are also more adapters than just Polyglot. (e.g. UCCI2WB.) So I think we should strive for a global design where the GUI does not have to know about the existence of adapters, and vice versa. And I think the current situation is still far from optimal in this respect.

IcsDrone uses Polyglot, but does it have an Engine Settings dialog that allows the user to make real-time changes to the Polyglot settings? I am not familiar with it, but the name suggests differently. Is there any need to alter, say, kibitz format from the IcsDrone menus? Because that is the only reason these option features exit at all: to alter the option settings from the GUI menu.

Similarly, when Polyglot is used as an UCI2UCI book adapter, it communicates with the GUI in UCI, where it can do whatever pleases it (or UCI GUIs). What I say has only bearing on its behavior as a WB engine, which doesn't affect any of that. So the arguments you give above are really not valid. Are there any other GUIs that depend on the WB mode of Polyglot, and for which would be broken if they cannot set the kibitz or matescore options through the option-feature mechanism? If the answer is "no", Polyglot can use invariant behavior by witholding those options from any GUI. This would have the additional benifit that it makes it more unlikely that such GUIs would ever come into existence!

BTW, the correct spelling is "threshold". And anything that has to do with it is by definition silly, as the first synonym given by the dictionary for threshold is "sill"!

[Michel]

I really don't see the problem.

Before WB invoked Polyglot by cooking up appropriate config files. A very ugly solution in my opinion. I decided to simplify that by giving the WB the opportunity to set the PG options directly and run PG without a config file. I never asked for an engine setting dialog for PG options and was in fact quite surprised when you implemented one (given as you have pointed out on numerous occasions that a lot of functionality of PG is
duplicated in WB, some of it on my suggestion BTW).

Icdroneng (a non graphical "GUI" let us say:-) of course has no engine settings dialog but the user can send arbitrary commands to the engine at run time. Including setting of options. I have had no opportunity yet where I had to tweak the mate score (for example) but I assume since the option exists it must be useful to some people. Making all options run time settable does no harm and provides for a lot of flexibility in the interaction between a "GUI" (in a generalized sense) and PG.

[H.G.Muller]

Indeed cooking up the ini files is an ugly kludge. I would like to get rid of it (but hey, it works, and there are so many thing that do not work yet, so I tend to give those priority over cosmetically improving existing code). Eventually this should be replaced by using Polyglot specifying an engine on its command line with the -ec option. Problem is that a WB using that method would not work with older Polyglots, so that it would force people to replace their Polyglot, without getting anything visible for it in return. This won't be appreciated. With the Engine-Settings dialog implemented, they will be much more eager to replace Polyglot for a new one.

Did IcsDrone ever had to set the Chess960 option? Does it rely on the engine for kibitzing?

[Michel]

over cosmetically improving existing code

Seems we are different then. The thing I like most is refactoring code.

Did IcsDrone ever had to set the Chess960 option? Does it rely on the engine for kibitzing?

Icsdrone relies on the engine for Kibitzing (this is apparently the tradition in the XBoard engine world; other
engines on FICS seem to do their own kibitzing as well).
icsdroneng does its own kibitzing (not yet in the standard format you are proposing but I will fix that).

Icsdrone does support wild but I never experimented with it. I am currently talking to E.M. to implement
it properly.

Can we let this discussion rest for a while? I really need to finish some urgent stuff. We can come back to it later.

[H.G.Muller]

Seems we are different then. The thing I like most is refactoring code.

Well, if you run ever out of material with Polyglot, you can always join the XBoard team, because most of the code is a total mesa after so many people have been working on it (often hardly knowng what they were doing, like when I started).

Can we let this discussion rest for a while? I really need to finish some urgent stuff. We can come back to it later.

Of course. You don't have to respond immediately. Just come back when you have time. That is what I like about forums...

[Somnus]

So if I understand the posts above... Polyglot has book learning?

If so, what kind of learning is it? Result? Positional? Opponent based?