'usermove' bug ??

[vlad_tudose]

Hello !

I'm developing a chess engine in C ,I'm currently testing the move generetor and I encountered the following problem:
sometime Winbord sends me the 'usermove' command without a newline at the end of the command,I find this strange considering that Winboard specification say "All commands from xboard to the engine end with a newline (\n), even where that is not explicitly stated. All your output to xboard must be in complete lines; any form of prompt or partial line will cause problems. ".

Is this normal or it is it a bug ?
I'm using _write and _read instructions to recive and send command and I'm testing my engine by playing games my engine vs my engine.
Here's an example from my logfile:
"<move d6
>time 29991
>otim 29980
>usermove >d6"

'>' commands recived by my engine
'<' commands send by me engine
I'm using Winboard 4.4.2

[H.G.Muller]

usermove d6 is to be considered a single command. That not all characters of the command were present on the first read call is something that can happen with pipes. I don't think the mechanism of pipes promises you anything about the synchronicity of reading and writing. The specs that the WB commands all end with newline is introduced to solve this. As long as you haven't seen the newline yet, you should continue reading, because the command is not complete yet, and more could come. In this case the d6. If there is no newline after d6, still more could come (e.g. d6d7).

[wing]

My engine also sometimes misses the usermove command and then looses on time because it just keeps pondering.
This is how I peek the pipe during search:

Code: Select all

noPonder:
   if ((XB_MODE) && (PeekNamedPipe(GetStdHandle(STD_INPUT_HANDLE), NULL, 0, NULL, &nchar, NULL)))
   {
      for (CMD_BUFF_COUNT = 0; CMD_BUFF_COUNT < (int)nchar; CMD_BUFF_COUNT++)
       {
         CMD_BUFF[CMD_BUFF_COUNT] = getc(stdin);
         if (CMD_BUFF[CMD_BUFF_COUNT] == '\n')
         {
            CMD_BUFF[CMD_BUFF_COUNT] = '\0';
            if (CMD_BUFF=="" || !CMD_BUFF_COUNT) return;
            sscanf(CMD_BUFF, "%s", command);
            if (!strcmp(command, ".")) return;
etc...


And a sample from the debug file:

Code: Select all

19567 >first : time 995
19567 >first : otim 1253
book hit = (NULL)
19567 >first : usermove 19568 >first : d6e7
19663 <first :     5   -194       15     98857 Be7 Qd5 Qc7 Rg1 Bd7
19669 <second:
19669 <second: Hint: ......
19669 <second: 1 -69 0 1 Qxd8+ Bxd8 a3 Bb6 axb4 Bxf2 Nxg5 Nxb4
19671 <second: 2 -69 0 36 Qxd8+ Bxd8 a3 Bb6 axb4 Bxf2 Nxg5 Nxb4
19672 <second: 3 -69 0 71 Qxd8+ Bxd8 a3 Bb6 axb4 Bxf2 Nxg5 Nxb4
19673 <second: 4 -69 0 106 Qxd8+ Bxd8 a3 Bb6 axb4 Bxf2 Nxg5 Nxb4
19674 <second: 5 -69 0 141 Qxd8+ Bxd8 a3 Bb6 axb4 Bxf2 Nxg5 Nxb4
19675 <second: 6 -40 1 11512 Qxd8+ Bxd8 g3 Bd7 Bg2 Be7 gxh4 gxh4
19676 <second: 7 -41 8 62338 Qxd8+ Bxd8 g3 O-O Bg2 Be6 Kb1
19677 <first : #<winglet : in peek, CMD_BUFF=time 995 command=time
19678 <first : #<winglet : in peek, CMD_BUFF=otim 1253 command=otim
19678 <first :     6   -204       18    130629 Be7 Qd5 Qc7 g3 hxg3 fxg3
19868 <second: 8 -40 30 271632 Qxd8+ Bxd8 g3 Be7 gxh4 gxh4 Rg1 Be6
19911 <first :     6   -204       47    365966 Be7 Qd5 Qc7 g3 hxg3 fxg3
20143 <first :     7   -197       70    575231 Be7 Qxd8+ Bxd8 Rd5 Be6 Rc5 Kd7 Nxe5+ Nxe5 Rxe5 Bxa2
20285 <second: 9 -49 72 713042 Qxd8+ Bxd8 g3 Be7 gxh4 gxh4 Rg1 Be6 Rg6 Bd7
21061 <second: 10 -38 149 1459443 Qxd8+ Bxd8 g3 Be7 gxh4 gxh4 Rg1 Be6 Kb1 a5
21412 <first :     7   -197      197   1486820 Be7 Qxd8+ Bxd8 Rd5 Be6 Rc5 Kd7 Nxe5+ Nxe5 Rxe5 Bxa2
22844 <second: 11 -38 327 3337910 Qxd8+ Bxd8 g3 Be7 gxh4 gxh4 Rg1 Be6 Kb1 Rd8 Rxd8+ Kxd8
23032 <first :     8   -197      359   2824250 Be7 Qxd8+ Bxd8 Rd5 Be6 Rc5 Kd7 Nxe5+ Nxe5 Rxe5 Bxa2
TC string = ':15'
mps=0 tc=15000 inc=0
GameEnds(26, Black wins on time, 4)
GE(26, Black wins on time, 4) bare king k=12 color=22
Interrupting first
29530 >first : result 0-1 {Black wins on time}
Interrupting second
29530 >second: result 0-1 {Black wins on time}
29531 >first : force
29531 >first : ping 4
29531 >second: force
29531 >second: ping 2
29532 <second:
29532 <second: pong 2
29595 <first : #<winglet : in peek, CMD_BUFF=result 0-1 {Black wins on time} command=result
29596 <first :
29596 <first : #>winglet : result 0-1 {Black wins on time} (from peek)


A related question: do I need to goto noPonder after having received a time or otime command from winboard during search, like is done in H.G.Muller's model winboard driver?
Would be great if someone can post working winboard driver code to 'PeekNamedPipe' during search...

[H.G.Muller]

I added the "goto noPonder" in the model driver to solve a problem that at some point suddenly appeared in Spartacus, while the same executable had not shown that problem before. I diagnosed it as the following:

WinBoard sends the time, otim and usermove so rapidly after one another that they are all waiting when the engine starts actually reading the input, after PeekNamedPipe told it there was input. Apparently it reads all available input then, storing it in a buffer. My driver then processed the time command upto the closing newline. When I set it pondering after that (which it would if the goto had been a continue), during that ponder it would again try PeekNamedPipe, and there would be nothing in the pipe! The otim and usermove have already slipped in, and are kept somewhere in limbo by the input library functions. So the engine keeps waiting forever.

So the goto noPonder was used to directly jump to the place where the program starts reading, bypassing any pipe peeking, which at that point did not work reliably because the commands you are waiting for are already in, invisible to PeekNamedPipe. The code you show subverts that idea: there is still a PeekNamedPipe that stands between the goto noPonder and the actual reading.

I guess a more fundamental solution would be to disable this kind of buffering, or use a low-level input function that never does any buffering (read?).

[wing]

I seem to receive usermove commands without newline in the pipe, especially in *very* fast games. So this is my ugly workaround - the previous version is commented.
It seems to work OK now:

Code: Select all

noPonder:

   if ((XB_MODE) && (PeekNamedPipe(GetStdHandle(STD_INPUT_HANDLE), NULL, 0, NULL, &nchar, NULL)))
   {
      for (CMD_BUFF_COUNT = 0; CMD_BUFF_COUNT < (int)nchar; CMD_BUFF_COUNT++)
       {
         CMD_BUFF[CMD_BUFF_COUNT] = getc(stdin);
//         if (CMD_BUFF[CMD_BUFF_COUNT] == '\n')
         if (((CMD_BUFF_COUNT+1)==(int)nchar) || CMD_BUFF[CMD_BUFF_COUNT] == '\n')
         {
//            CMD_BUFF[CMD_BUFF_COUNT] = '\0';
            if (CMD_BUFF[CMD_BUFF_COUNT] == '\n') CMD_BUFF[CMD_BUFF_COUNT] = '\0';
            else CMD_BUFF[CMD_BUFF_COUNT+1] = '\0';

            if (CMD_BUFF=="" || !CMD_BUFF_COUNT) return;

            sscanf(CMD_BUFF, "%s", command);
            #ifdef WINGLET_DEBUG_WINBOARD
               std::cout << "#<winglet : in peek, CMD_BUFF=" << CMD_BUFF << " command=" << command << std::endl;
            #endif

            // do not stop thinking/pondering/analyzing for any of the following commands:
            if (!strcmp(command, ".")) return;
            if (!strcmp(command, "?")) return;
            if (!strcmp(command, "bk")) return;
            if (!strcmp(command, "easy"))
            {
               XB_PONDER = false;
               return;
            }
            if (!strcmp(command, "hint")) return;
            if (!strcmp(command, "nopost"))
            {
               XB_POST = false;
               return;
            }   
            if (!strcmp(command, "otim"))
            {
               sscanf(CMD_BUFF, "otim %d", &XB_OTIM);
               XB_OTIM *= 10;  // convert centiseconds to miliseconds;
               goto noPonder;
            }   
            if (!strcmp(command, "post"))
            {
               XB_POST = true;
               return;
            }   
            if (!strcmp(command, "time"))
            {
               sscanf(CMD_BUFF,"time %d",&XB_CTIM);
               XB_CTIM *= 10; // convert centiseconds to milliseconds
               goto noPonder;
            }
            timedout = true;
            CMD_BUFF_COUNT = (int)strlen(CMD_BUFF);
            XB_DO_PENDING = true;
            return;
         }
      }
   }
   return;
}


[H.G.Muller]

The usermove command is often received in two parts. So the initial PeekNamedPipe might tell you there are 9 characters waiting, but then,when you read a single character, and would peek again, it would tell you that there are 0 characters waiting. Because the remaining 8, plus 4 or 5 for the move, and one for the linefeed, will all have been stashed in a buffer, ready to be read, but unpeekable. So if your input loop only reads the 9 announced characters, and then starts peeking again, you will be dead.

Relying on the count returned by PeekNamedPipe is just asking for trouble. If there is any input at all, you can safely read until you encounter a linefeed, because WB protocol guarantees that anything command will end in a linefeed. If PeekNamedPipe says there are 9 characters waiting, and you read 9, and there was no linefeed among them, and PeekNamedPipe now says there are 0 characters, you should still continue reading. Because WB protocol is reliable, and everything upto the linefeed will be there, while PeekNamedPipe can be lying.

[wing]

Windows pants on fire!
So basically I have to ignore nchar and just rely on PeekNamedPipe's return value (true/false)?

[H.G.Muller]

That is what I did. But there might be cleaner solutions. If it would be possible to prevent getchar() from buffering anything through some initialization call, or by using an input routine that that never does any buffering by default, the problem would be solved. An alternative way to solve the problem is to check if there is something in the getchar buffer before using PeekNamedPipe; it is really the sum of the value returned by PeekNamedPipe and the number of currently buffered characters you are interested in.

Perhaps using setbuf or setvbuf on stdin would solve the problem. I did not try that.