WB protocol

[Michel]

The UCI protocol is (almost) stateless so it is trivial to describe. However the Wb protocol is
not.

I think there should be a description of the Wb protocol as a state machine rather than just as a long list of commands. This would be easier to understand.

If you implement the WB protocol you have to implement is as a state machine anyway.

Michel

[H.G.Muller]

I think I have seen a diagram that did exactly that, somewhere. But I don't remember where.

[Alex Guerrero]

http://www.ascotti.org/programming/chess/xboard.htm

[Michel]

Very very nice!

This should be part of the official XB/WB documentation I think.

Michel

[H.G.Muller]

I can add it to the document defining the protocol extensions (which is in html), if I can obtain Alessandro's permission to use it.

[Michel]

The diagram is not entirely complete it seems to me. There seems to be nothing in the documentation preventing you from going from the "waiting" state back to the "observing" state (force mode) by issuing the [force] command.

Also Gnuchess and Crafty seem to enter the diagram in the "waiting" state rather than in the "observing" state....

Michel

[Pedro Castro]

I made my own chart to the engine.

http://www.euskalnet.net/pecas/chess_engine_img.gif

[Christopher Conkie]

I think I have seen a diagram that did exactly that, somewhere. But I don't remember where.

Hi Harm,

I think it was Pradu who had such a diagram, just like Pedro's one. I know I used the one he gave me.

Merry Christmas

Christopher

[Michel]

I can add it to the document defining the protocol extensions (which is in html),

Can you please make the link to that document a sticky? Google cannot find it.

[H.G.Muller]

In any case, this is the link:

http://home.hccnet.nl/h.g.muller/engine-intf.html

I will try to find agood place to put a sticky thread for it.

[Michel]

The requirement that the cores command can be issued at arbitrary times is currently impossible to meet for any version of Toga except Cyclone.

The good news that it is not very hard to change Toga to make it work.

For the moment it seems Toga would have to announce itself as not supporting the cores command.

[H.G.Muller]

I have no experience with SMP at all. Would it, in your opinion, be better to relax the standards for the cores command, and arrange it such that it will only be sent, e.g., at the beginning of the game? I proposed the current definition, because from the user point of view it seems that nr of CPUs is similar to pondering (which can be sent at any time). And from the engine point of view, it seemed to me that it would not in principle be hard to simply shut off a number of threads semi-instantaneously, in most search algorithms.

I could relax the definition such that in reaction to a cores command the engine should make an effort to comply with the requested nr of cores at the eirliest convenient time, leaving it open if this means the next time a thread became idle during the current search, after completion of the current irteration, after completion of the current search, or at the beginning of the next game.

I think it is better that even in the current state it is better if Toga would send cores=1; in principle the features command for GUI options only affects if it is safe for the GUI to send the command. It does not suppress the appearence of the command in any menu, as it is in the GUI menu specifying system limitations, and will have to remain present there for the benifit of the opponent engine, or indeed any later engines. (The setting is remembered in the winboard.ini file in WinBoard.) So it makes little difference if the engine does not comply because it does not react properly to the command, or if it doesn't comply because WinBoard refrains from sending the command. Except that in the latter case, it is guaranteed to never comply, which seems inferior to having iot comply when it can...

[Michel]

I think it is better that even in the current state it is better if Toga would send cores=1

Ok. The problem is that Toga fixes the number of threads after the first isready or search command. I think this is really a bug and should be fixed (it is not hard to fix). In the current state of things the "cores" command would not work at all for Toga.

Polyglot has to send "isready" to the UCI engine at some point and since there is no corresponding command in the xboard protocol Polyglot does it after it has read its configuration file, fixing the number of threads for Toga.

One could try to code around this in Polyglot but it would be hacky and messy and only for the benefit of one engine. People that really want to use multiple threads in Toga could still do it via the engine section of the polyglot.ini file.


This being said I think that the requirement that an engine should react immediately to the force command while pondering is a very strong one, and impossible to implement for UCI engines.

I would actually prefer that the GUI sent the cores command only in force mode or after the new command. At that time the engine is idle. This is what UCI engines do (in a sense force mode is their default state).

[H.G.Muller]

Well, it would be easy enough to have WinBoard preceed every cores command by a force command, but somehow I think this is not what you mean. At least, I don't think how it would solve anything. The force command would still have to interrupt pondering, and Polyglot could prefix the UCI command required to interrupt pondering itself. Or the engine could add it as well.

[Michel]

Well I guess if received a force command during pondering Polyglot could interrupt the engine, execute the cores command and restart the pondering. This would require
PG to save the parameters of the go ponder command.

I would be much happier if the cores command was only sent when the engine is idle.
(i.e. if I understand correctly: after force or new). I prefer less state rather than more.

[H.G.Muller]

Why do you want to restart the pondering? No one says the engine should not suffer from it it you change the nr of CPUs during a game. I imagine this is something you would do only in case of an emergency, i.e. if you discovered dring a tournamnt that you forgot to switch on all search threads initially, or accidentally had set too many for the machine you run on now. Just leave the engine in the idle state.

Would it help if it was guaranteed that WinBoard deferred sending the 'cores' command until just before sending the opponent's move to the engine? This is what I do for WB 'draw' offers, because I also hate to unnecessessarily interrupt pondering of WB engines.

Concerning the emergency character of an intra-game CPU-nr change, I doubt if that would be good for the engine, though. If someone discobvers he has goofed on the nr of CPUs, it would likely be better for the engine if it was changed instantly, than to let it finish its ponder with wrong settings.

If I let WB always send force before a cores command, it would always put the engine in the idle state if this occured during ponder. There would be no way for WinBoard to have the engine resume pondering, it would have to wait for it to be the engines turn again, and send a go then. I think it would be better to leave that decision with the engine. Autors that do not like many states of their engine can opt for leaving it in the idle state. But if they care enough, they can implement the cores command such that it does not abort pondering.

[Michel]

Why do you want to restart the pondering?

Ok if it is allowed not to resume pondering then it is no problem.

Would it help if it was guaranteed that WinBoard deferred sending the 'cores' command until just before sending the opponent's move to the engine?

I don't think so. If the opponent's move is a ponder hit then the "GUI" (PG in this case) (which has told the engine on which move to ponder) is supposed to send "ponderhit" to the engine. The pondering then transforms into a normal search with time control (pondering is just a non returning untimed search).

If you have interrupted the ponder command to change the number of threads then the pondering is lost (except for perhaps hash table entries).

But all this is quite confusing:-)

[H.G.Muller]

I guess this is dependent of the engine implementation. Joker is not a UCI engine, but if it were a UCI engine I would see no reason to treat ponder hits differenty from the way it does it in WB: the incoming move always aborts pondering, whether it is a hit or miss. On a ponder hit (which it has to conclude itself by comparing the incoming move to what it was pondering, rather than having the GUI do that comparison, but that is an insignificant detail) it starts an ordinary search. In practice, this source immediately picks up where the aborted ponder has left off: because of the hash table it does not search again what was already searched during ponder. There is virtually no time lost because of this: The new search starts with iterative deepening at one ply, but then reaches depth 17 in just a few hundred nodes.

If someting other than a move came in, it would process it and then resume the pondering, which would pick up similarly.

[Michel]

it starts an ordinary search. In practice, this source immediately picks up where the aborted ponder has left off: because of the hash table it does not search again what was already searched during ponder.

I guess that is what would happen if the hash table is big enough.

So if the engine is pondering PG should simple save the "cores" command until xboard sends the opponents move. Then it should send "stop" to the engine, execute the appropriate setoption command and start a new search, hoping that the results of the aborted ponder were not lost.

This might work.

[H.G.Muller]

Usually hash tables have some preference for keeping entries close to the root. So even if the table is only large enough to hold a 10-ply search in its depth-preferrred part, it only has to reconstruct the last 7-ply of the tree it was interrupted in to pick up a 17 ply search. In practice this takes totally negligible time compared to all other side branches of the interrupted branch that were searched before.

So I think the implementation you suggest would work very well. But I still don't see that much is gained compared to immediately interrupting the ponder, and than order a new ponder search on the same move as before. It is just a repetition of the last thing that Polyglot has done towards the engine, so it must still have all required info to do it.