Starting to use the Winboard Protocol

[Reinhard Scharnagl]

Actually I intend to write a new engine or adapter using the winboard protocol,
though I am not sure, whether it will be able to fully correspond to my needs.
Maybe this forum could be able to give hints step for step for me to become
more familiar with some details of how to use that protocol.

Maybe I later will write a German language translated "how to use Winboard 2",
if I finally would have sufficient information about that protocol and its details.

01) Is it save to start with the so called version 2 as reported by H.G.Muller?
Is it ok then to demand for an appropriate GUI to be existing, that is to refuse
any engine's working, when no protover 2 or higher command was received?

02) Is it save to ignore any signaling then, or will I still have to disable signaling?
Does this depend on different operating systems?

03) Is it possible to communicate via the STDIN / STDOUT pipe between Winboard
GUIs and engines despite of their nature as 32 bit or 64 bit engines?

Thank you in advance, R. Scharnagl.

[Reinhard Scharnagl]

Well, I still do not understand e.g. Winboard's need for to have such a lot of variants, where mostly a sent FEN or XFEN string would be sufficient for to describe the whole nature of the variant. Thus I still have problems with this protocol. Why it is not possible to start a new variant game e.g. by "new <X-FEN>"?

I think it may be an alternative to simply use UCI interpreting it compatibly for X-FEN positions. Unfortunately there no GUI still seems to be supporting such a behavior also by 10x8 boards.

[Tony Hecker]

Hi Reinhard,

01) Is it save to start with the so called version 2 as reported by H.G.Muller?
Is it ok then to demand for an appropriate GUI to be existing, that is to refuse
any engine's working, when no protover 2 or higher command was received?

I do not think it's recommended to refuse to run based on protover. It doesn't hurt anything to at least attempt to run. I think protover 2 engines often run without problems under a protocol version 1 GUI (such as Chessmaster, as I recall), though some functionality may be limited.

02) Is it save to ignore any signaling then, or will I still have to disable signaling?
Does this depend on different operating systems?

My engine sends "feature sigint=0 sigterm=0" in both Windows and Linux versions, and I ignore signals. I don't know if this is the best way, but I'm not aware of any issues (although I'm not very experienced with Linux).

03) Is it possible to communicate via the STDIN / STDOUT pipe between Winboard
GUIs and engines despite of their nature as 32 bit or 64 bit engines?

Yes, as far as I know. I haven't seen any issues running both 32-bit and 64-bit engines under the same 32-bit GUI, running on 64-bit OS.

Well, I still do not understand e.g. Winboard's need for to have such a lot of variants, where mostly a sent FEN or XFEN string would be sufficient for to describe the whole nature of the variant. Thus I still have problems with this protocol. Why it is not possible to start a new variant game e.g. by "new <X-FEN>"?

I guess that could work for some variants but not all of them. To be consistent, it makes sense for the protocol to use the explicit "variant ..." command.

Any WinBoard experts, feel free to correct any mistakes I made.

Tony

[Reinhard Scharnagl]

Thank you, Tony, for having reduced some of those difficulties.
Because I intend to write a follow-up for my SMIRF engine, there
will be merely a need to support such variants, which could be
fully described by their X-FEN. Thus there is no need to distinguish
by a variant name, which has more than only a comment function.

So I will postpone writing a Winboard (2) interface.

Maybe there will be a 8x8/10x8 Chess GUI using the UCI protocol,
but not that early. I actually intend to rewrite SMIRF using the UCI
protocol, compatibly extended for the use of XFEN. Indeed I will
have a time, where that engine could be tested merely in its 8x8 play
when using a GUI, or via a console, where additional UCI commands
also will enable its 10x8 playing.

The GUI has to be very unintelligent, it should know nothing on how
to play or to generate moves. My coming engine also will have a kind
of a referee mode. This will enable a GUI to ask for possible moves
and for XFEN strings after a done move. Thus the GUI will only have
to be able to translate XFEN into a picture and vice versa. Move entering
by users will be nothing more than to select one of the possible moves
retrieved by asking a referee engine for that.

E.g. I also intend to implement free castlings at least for 10x8 chess,
thus I will have to use a more flexible algebraic notation for castings:
related to promotions, where the fifth place is for selecting different
moves for the same source/destination combination, here the column
letter of the involved rook will be used, which establishes a compatible
way also to distinguish castlings from simple King moves e.g. at Chess960.
The last letter might be dropped, when nothing is to be distinguished.

[H.G.Muller]

Sorry for the late response, but I was on holiday the past 2 weeks, with only a smartphone to go on-line, so it was not feasible for me to write any longer messages. As to your questions:

01) Is it save to start with the so called version 2 as reported by H.G.Muller?
Is it ok then to demand for an appropriate GUI to be existing, that is to refuse
any engine's working, when no protover 2 or higher command was received?

Like Tony remarked, there are some GUIs around which still only support v1 of the protocol. So this is basically a question only you can answer: do you want the engine to run on these GUIs. WinBoard/XBoard and Arena support v2, I don't know about others. Some GUIs don't support WB protocol at all, and can only run WB engines through the WB2UCI adapter. I think that adapter supports v2 too. My new engine Spartacus would not run correctly on a v1 GUI if it wanted to set up a position; it would only understand setboard for that, which you cannot use in v1.

02) Is it save to ignore any signaling then, or will I still have to disable signaling?
Does this depend on different operating systems?

Signalling in Windows does not exist. I fall for this every time when I port one of my engines to Linux: suddenly mysteriously it crashes after having made the first move... The reason then always is that I forgot to send the sigint=0 feature. Note that even for Windows binaries it makes sense to do that, because some people run Windows binaries on Linux, using wine as Windows emulator, and it turns out that they are killed even when they did turn themselves insensitive to SIGINT using a 'signal' system call. (This should be considered a wine bug...) So best to sent sigint=0 even if you only intend to prepare a Windows binary!

03) Is it possible to communicate via the STDIN / STDOUT pipe between Winboard
GUIs and engines despite of their nature as 32 bit or 64 bit engines?

Yes, no problem at all. Pipes (like files) are completely architecture-independent things.

[H.G.Muller]

Well, I still do not understand e.g. Winboard's need for to have such a lot of variants, where mostly a sent FEN or XFEN string would be sufficient for to describe the whole nature of the variant. Thus I still have problems with this protocol. Why it is not possible to start a new variant game e.g. by "new <X-FEN>"?

I think it may be an alternative to simply use UCI interpreting it compatibly for X-FEN positions. Unfortunately there no GUI still seems to be supporting such a behavior also by 10x8 boards.

Unfortunately it is in general not possible to recognize the variant from a FEN. This because piece-ID letters are in general ambiguous. E.g. B can be either a Bishop or an Elephant (the latter if you play Shatranj (wild 28) on FICS).

I agree that some of the variants are redundant, and there existence can only be understood in a historic context: before I got involved with it, WinBoard did not keep track of castling rights. (A so-called 'documented limitation'.) So the only way to make it refuse castlings when the Rook and King were on their usual squares in the initial position was to create an extra variant 'nocastle'. Also note that in variant 'wildcastle' (playable on FICS and ICC) you can castle a King on d1 with a corner Rook, and that this has a different effect from when you would castle in Chess960 (the King ends up on b1 or f1, and I think O-O means a-side castling there in SAN!). So it is not possible to play these wildcastle positions as fischerandom, they really need to be a different variant, although the opening FEN would look the same.

Note, however, that this 'over-specification' of variants only applies to a small minority of the variants. WinBoard supports some 30 variants (and the 'Alien Edition' even more), and for most of those the opening position cannot be expressed as X-FEN at all. One of the design goals of WinBoard is also to support all variants on non-standard board size, which makes it even more difficult to recognize the variant from a FEN. E.g. from 1rnbqkbnr1/1pppppppp1/8/8/8/8/1PPPPPPPP1/1RNBQKBNR1 w - - 0 1 if this is Capablanca Chess, Janus Chess or merely normal Chess on a 10x8 board. While these are not equivalent, because they have different promotion rules.

Note that in UCI Chess960 and normal chess are also considered different variants, that you have to choose between through a UCI option (setoption UCI_chess960 true, I believe). In this respect it is not really different from WB protocol. It is just that it defines far fewer variants.

[Reinhard Scharnagl]

Hello Haarm Geert,

one of my goals is to write an engine using language C for my Mac and for Windows, compilable for 32-Bit and 64-Bit using one and same source. At this stage today it still seems to be possible.

The UCI distinction between Chess960 positions and traditional Chess and Nocastle Chess is unnecessary. This had been shown in unfortunate discussions with the UCI author. I have repeatedly explained a compatible extension for FEN called X-FEN, where such artificial distinctions could be avoided. Nevertheless I will support that UCI flag, despite of my engine will by default support X-FEN, hoping, some day there will be a 8x8/10x8 UCI engine on the Mac supporting this. I will add a new castling prefix for free castling, which will force to have an extended algebraic notation for some castlings, but this extension also will be compatible to traditional chess. Janus Chess is indeed a little bit different. When any 'J' will be inside of a X-FEN string or the symmetric castling prefix without any 'C' or 'A' pieces at the board, I will assume the absence of the 'C' in absolute, that is, it will not became able to be promoted into. In 99.9% of all situations this will be sufficient, because that will be analyzed mostly from the beginning of a game.

[Reinhard Scharnagl]

When I wrote my own SMIRF GUI it has been rarely noticed, that it has been a little demo for a new kind of GUIs. Because the SMIRF GUI does know nothing about how to generate moves or how to perform valid moves. Instead it merely has the ability to translate board positions into X-FEN and vice versa. It asks the engine in its role as a referee for a list of possible moves and how the X-FEN is changed by a move and has the user select one of those. Thus the GUI is held stupid by intention.

A multi variant GUI like HGM's Winboard seems to be forced to implement the intelligence for every variant inside, before it will become able to interact with such an engine. It supports a lot more of variants than those which could be condensed into the use of an approach as X-FEN. Thus it would make sense to say good by to such a GUI concept. Instead it seems there is a need for another kind of GUI. That type of a GUI has to ask the engine for which variants it has been built, what starting positions there are, which piece letters or symbols should be used on which board geometry. Then it has to display that as wanted by that engine. The GUI has to match two engines to make them play together. It has to choose one of both (or a recommended third) to be the referee in questions of which moves are possible in each position. Acting like that the GUI will no longer have to be changed, when engines would provide new variants.

Therefore I am convinced, that making a GUI intelligent is the wrong way to support variants. Unfortunately there is no other GUI yet reflecting those thoughts above. But if someone would have the motivation and the time to write a new GUI maybe also for the Mac it seems to be a very welcomed approach to do it like that and to create an appropriate protocol by discussion with engine writers.

[H.G.Muller]

The UCI distinction between Chess960 positions and traditional Chess and Nocastle Chess is unnecessary.

But how does that help you if you still need to distinguish it from WildCastle and Shatranj? The best you could acheive is reduce the variant list from 30 to 27 by grouping two or three of the current variants under 'normal'. Big deal...

The distiction between the variants you mention is 'unnecessary' in UCI in exactly the same sense as it is 'unnecessary' in WB protocol: in both cases the specs require it, and you could design an arguably more elegant dialect that would leave it out, but is unfortunately incompatible with virtually all existing engines and GUIs. The only difference between UCI and WB in this respect is the number of implemented variants (see however http://www.talkchess.com/forum/viewtopic.php?t=26140 ).

You seem to prefer elegance over correctness and functionality, which I think is a disastrous choice. For a GUI being able to handle only 99.9% of all cases is extremely low quality. And what makes it even more sad is that the 'elegance' is in something the users would never see or even be aware of, let alone care about, namely the protocol used to communicate with the engine. The user only cares about how he has to communicate with the GUI, and whether the GUI then does what was asked. And it is of course perfectly possible to make a GUI for 'unified chess' that groups normal, FRC, CRC, Capablanca, Gothic, Janus as a single variant in the user menus, accepting non-standard FENs of your own design (which solves all ambiguities that crop up) to set up positions from the user, and then uses standard protocol to communicate with the engine (deriving the variant command needed to make the engine function properly for the given initial position, and send it to the engine before the setboard).

Btw, WinBoard actually evolves towards your 'dumb-GUI' concept. Unlike UCI, WB protocol has always had the possibility to deligate a lot of tasks to the engine, including legality checking (Presumably initial WB versions were also oblivious of game rules.) Not having to know game rules makes a GUI more generally useful for supporting a very wide variety of board games. The WinBoard 'Alien Edition' is most advanced in this respect. Some GUI features, which originally did require rule knowledge, such as highlighting of target squares of a picked-up piece, can be delegated to the engine (but in a way somewhat different from how you proposed it).

In the Alien Edition multi-move variants (e.g. Marseillaise Chess) are also supported in a general way (separating the individual moves of a single turn by commas), and this can also be used to specify non-standard castling (e.g. Kb1,Rad3)

[Reinhard Scharnagl]

... In the Alien Edition multi-move variants (e.g. Marseillaise Chess) are also supported in a general way (separating the individual moves of a single turn by commas), and this can also be used to specify non-standard castling (e.g. Kb1,Rad3)

May be there is a lack of information.

Best regards, R.S.

[H.G.Muller]

I am sure there is, as the Alien Edition is an experimental version, which was developed in interaction with Daniel Shawul (who was responsible for development of the corresponding engines), and which is still not fully crystallized into its final form. Basically the only information on it can be found at http://hgm.nubati.net/alien.html .

WB protocol historically supported legality checking by the engine only through the possibility of the engine to refuse moves with an "Illegal move" command, on which the GUI would take the move back, and relay the message to the user. This does not work as smoothly, though, as when the GUI is already aware of the rules (especially with click-click moves, and animate-moving on).

In the Alien Edition we inform the engine when the user grabs a piece, so the engine can send the GUI a command to inform it where the piece would be allowed to move to. It does this in the form of a 'color-FEN', where it can assign different colors to squares that result in a capture vs non-capture (or leave them unmarked). The GUI can then immediately reject any move where the user drags the piece to an unmarked square. The engine is also informed when the user hovers the piece over a square that was marked red, so it can highlight the pieces that would be captured in that case (which might not coincide with the to-square, e.g. in e.p. capture, but also in checkers).

[H.G.Muller]

I implementing Seirawan Chess, we enountered the problem that it is necessary to know which pieces on the back-rank have moved, or not. This because gating of in-hand pieces is only allowed on the first move of every back-rank piece. The castling field gives this information oly partially, even for King and Rooks (if you have no a-castling rights, you don't know if it is because Ra1 was moved, or the King). The complete virginity information would imply castling rights, though.

My first idea is to add a new field to the FEN (between e.p. and 50-move counter) listing the files of all pieces on the back-rank that are not virgin. This means it could be left out for opening positions, providing good compatibility with normal FEN there. Tricky point is if all non-virgin pieces should be mentioned, or just those which could conceivably be non-virgin. E.g. a Rook on e1 is obviously non-virgin. The latter would not work for shuffle variants of Seirawan Chess, however. (But they don't officially exist, so who cares?).

There also is some redundancy: the info in this field makes the castling field superfluous. Rooks or Kings marked as non-virgin can obviously not castle. So perhaps the 'virginity field' should replace the castling field?

Do you have any ideas on this, Reinhard? (And perhaps any plans to implement Seirawan Chess in SMIRF?)

[Reinhard Scharnagl]

The FEN for Seirawan Chess has to list possible still placable pieces, because such
pieces types could already have been placed and lost. Thus this information is
not contained within the traditional FEN.
Maybe detailed castling info is partially redundant then, but not everything in it,
e.g. the potential to have castings at all (or the type of castling in variants).
A virgin flag type information seems to contain too much information, because
that info will be needed only twice in a game. Also it seems to be used only
immediately after a piece had left its starting array. Thus the info is not only
related to a piece before moving but also to a square after having moved.
Therefore there would be a need for to use the e.p. info here also for a just
left base square.
As a result a virgin flag would be related to squares not to pieces. Thus it
seems to be misleading to encode it via piece symbols at all. One idea could be
to bundle those eight bits into a two digit hex number (for each base row) and
to add it somewhere to the FEN (or substitute e.g. the castling part by that).

This have been some quickly written thoughts on that variant.

SMIRF probably will not be continued. I am very slowly (about 50 lines a week)
rewriting a monochrome approach for a new chess engine. This time I will
only use C language instead of C++ and try to have it compile on different
operating systems. Therefore it also will have a new name.

For now I intend to cover merely such variants densely related to traditional
chess (8x8/10x8). But the future still is not written.

[H.G.Muller]

Beware that in Seirawan Chess the placing of the new piece is one in the same move that evacuated the back-rank square. So there is no need to remember the previous move, like in e.p. capture. There does not have to be a FEN for after the board move, but before the placement (just like there doesn't have to be a FEN for after the King move but before the Rook move in castling).The sitution is very similar to castling actually, as you are not allowed to 'pass through check' here as well: if evacuting the square could put you in check, the gating is illegal even if the pledpiece would block the check again!

So the only thing that has to be remembered is if the piece has been moved during the entire game. This is very similar to castling rights. Except that a castling right is lost if either the King or the Rook moves, while the gating right gets lost only when that particular piece moves.

You are right for the need to list the pieces that are still 'in hand'; I had planned to do that in the same way as in Crazyhouse and Shogi, by listing the holdings content between square brackets beteen the board and stm field. Like

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR [HEhe] w KQkq - 0 1

(E=Elephant, H=Hawk, also such an annoying incompatibilitywith C and A...) I hesitate a bit to expose players to binary or hexadecimal representations. I was thinking about using the file indicators in stead, like in Shredder- and X-FEN. It seems a bit inconvenient, however, to have to write an extra

ABCDEFGHabcdefgh

with the opening position, however. So it seems better to list only non-virgin pieces. Then you would not have to list anything in the opening position. And if you would only list non-virgin pieces that are not obviously non-virgin (because they are not on their initial square), you would not list many in general, as it is not very common for pieces to return to their starting square. You could also stop listing them if there are no more pieces in hand. Problem is that this would not work at all in shuffle games, where every back-rank piece could in principle be on its initial square. So it seems hard to design a system that would work both for 'Seirawan960' and normal Seirawan, in the same spirit as X-FEN works both for Chess960 and orthodox Chess. Unless you accept an awful lot of overhead in normal Seirawan.

Another issue is that the castling-rights field seems to become completely redundant once you know of every piece whether it moved or not. If (and only if) a King and Rook are both listed as virgin (i.e. not listed as non-virgin), you will still have the corresponding castling right. So it could be logical to replace the castling-rights field by the virginity field. You could then continue to list virginity of King and Rooks even after all pieces had been gated, to imply the castling rights. Problem is that it will be very hard to maintain any form of comptibility with the X-FEN castling rights if you list non-virginity, which corresponds to absense of rights.

One could of course make the interpretation of the castling-rights field dependent on the presence of pieces to gate, and if a side has pieces to gate, use file letters to indicate non-virginity, while if he is out of pieces, use X-FEN convention, where a letter indicates virginity of that Rook, and implies virginity of the King.

In hind-sight,it would have been better if X-FEN would have indicated Rooks that are not allowed to castle, next to the KQkq rights. Normally (i.e. in absence of under-promotion) there can only be ambiguity if the right on the other side is destroyed, so you would still never have more than 4 characters in a castling field.

[Reinhard Scharnagl]

Well, you are right, there is no need to extend the use of an e.p. field.

Concerning the virgin flags, for me it seems unnatural to encode the absence
of extra possibilities. That would be in contrast to current castling flags. And
that would establish a need to still encode such absence when all extra pieces
have been placed, that is, where those flags have become completely irrelevant.

Because of that I think, that information needs to be encoded positively as long
as there are still extra pieces at hands. Providing a list of all virgin column let-
terse looks overdone for my feeling. Thus I would prefer a hexadecimal encoded
four character combination replacing the castling rights part. Indeed that would
not at all be a solution simply readable by human people. But those normally are
not the addressee of this encoding, but engines.

To unify those different FEN encodings there seems to be two ways: as long as
there are to be placed extra pieces, there could be used the virginity encoding,
after that the traditional castling flags word. Or put those flags word e.g. into
a pair of round brackets. Last idea seems to have a more universal power and
I would vote for that. (I have not checked the consequences for 10x8 chess.)

[Reinhard Scharnagl]

Addition: I would not call those unified flags virgin flags but extended abilities.
That is making clear, that virgin pieces would not have to be encoded, when
there is no ability left for a piece to castle or to put extra pieces after a move.

I vote for an encoding as (nr1.nr2) where the flags of both colors are separated
by a dot, enhancing computer readability and make it usable also for 10x8 chess.

In contrast to prior thoughts on how to encode piece types in FEN I argue for to
always use same letters for same gaits-pieces that is A for N+B and C for N+R.
It would be a job for the GUI to translate that letters into variant specific sub-
stitutes. Such substitution could be placed also into a PGN as a special option.

P.S.: I see, having a separating dot will put away a need for enclosing brackets.

[H.G.Muller]

OK, after also discussing this with others, and some rethinking, I am no leaning to a format for this 'extended abilities field' (genralized castling field) that simply lists the file indicators for every piece with a (real rather than formal) extended ability. Use of KQkq to imply virginity of both King and (outer) Rook would remain acceptable at all times. So in the Seirawan opening position we would write KQBCDFGkqbcfg. The verbosity seems the least of all evils. Advantages are that:
1) this system is fully capable of handling shuffle variants of Seirawan Chess.
2) it converges to standard FEN after all pieces have been gated in (and the holdings field would be omitted, rather than written as [-]).
3) with the convention that mentioning a Rook in the absence of pieces to gate implies that Rook can castle, it would also be compatible with X-FEN / Shredder FEN.
4) it is easy to read for humans.

As to the piece IDs, I guess we will have to live with the fact that a 1:1 mapping between pieces and letters cannot be globally made with the latin alphabet. It is a bad idea anyway to compare FENs as ASCII strings, without interpretation (e.g. e.p. field ambiguity, order of letters in castling field).

[Reinhard Scharnagl]

Well this will work. There only one point remains unclear: the encoding of placing after castlings.
My suggestion would be: regular lower case letter -> king's source field, upper case letter -> rooks source field.