New WinBoard alpha version

[H.G.Muller]

I posted a new winboard.exe at http://home.hccnet.nl/h.g.muller/winboard.zip .

This version implements the -fUCI and -sUCI options in a different way than previous WinBoard versions: there now is a new WinBoard option /adapterCommand="...", where "..." gives the command-line WinBoard has to use for starting up the adapter (i.e. Polyglot). The default value for the adapterCommand is "polyglot -noini -ec %%cp -ed %%d".

This requires some explanation: Most of what you write in the adaptorCommand is use literally. You could have written "polyglot polyglot.ini", or even simply "polyglot", and it would be able to work with old-fashioned Polyglots. But if there is a % in the adapterCommand, the word after it is interpreted as the name of a WinBoard command-line option, and the value of this option is substituted for it. So if the adapterCommand is "polyglot -noini -ec %fcp", and Fruit is running as first engine, it would translate to "polyglot -noini -ec Fruit". With Polyglot versions 1.4.46b or higher, "-ec Fruit" would specify that the EngineCommand Polyglot has to use is Fruit. (So no polyglot.ini file is necessary anymore to instruct Polyglot what to do, which is what the -noini says.)

Now the adapterCommand must not only be useful for the first engine, but also for the second. So you don't know if you would have to write %fcp or %scp in it, as value of the Polyglot -ec option. To solve that, if the first character after the % is again a %, this second % is replaced by the letter f or s, as the occasion requires. So "%%cp" translates to "%fcp" when you run the first engine with -fUCI, and to %scp if you run the second engine with -sUCI. Similarly, "%%d" translates to "%fd" or "%sd" for supplying the EngineDir (-ed) to Polyglot.

Any string, int or boolean option of WinBoard can be used in the adapter command this way. So you could add "-log %debug" to make the production of a polyglot.log dependent on if WinBoard produced a debug file: it would translate to "-log true" or "-log false" depending on the setting of the WinBoard -debug option.

Currently, the adapterCommand appears in the Options->Global Settings dialog in stead of the polyglotDir. I am not sure if this is a good idea; it might be better to hide this option from the naive user, and only allow its setting through command-line options (or ini files). The /polyglotDir option still exists, btw., and specifies the folder where the polyglot.exe comamnd should be given. (i.e. where polyglot.exe is.) So evenually I will probably keep that in the Options->Global Settings dialog. Users are much more likely to change that. But for testing it was a bit more convenient to have the command itself available.

Please let me know if there are any problems with this new method!

[H.G.Muller]

Michel,

how about engine names with spaces? Does Polyglot understand quoting? I guess for WinBoard the adapter command should really be:

-adapterCommand {polyglot -noini -ec "%%cp" -ed "%%d"}

so that Polyglot would be called as

polyglot -noini -ec "Fruit_21" -ed "C:\Program Files\Chess\Fruit"

On Linux this might fail, though. In tht case XBoard and WinBoard would need different default values for the adapterCommand, and names with spces in it will not work in XBoard. (I guess names of the engine binary with spaces in it have never worked, so this might not be a big loss...)

[Michel]

Executables with spaces currently do not work on Linux. This is because PG breaks the engine command according to the spaces
it contains and considers all words except the first one as arguments. I don't think it is a problem to include extra quotes
in the ec argument. They just won't do anything.

In Windows it normally works fine since PG just passes the engine command to CreateProcess.

Directories with spaces should not be a problem on both Windows and Linux.

[H.G.Muller]

Executables with spaces currently do not work on Linux. This is because PG breaks the engine command according to the spaces
it contains and considers all words except the first one as arguments.

OK, this is pretty much what XBoard does too. But that still leaves the problem of engine commands with spaces in them (because the engine needs arguments). Polyglot probably won't like it if it gets passed the engine command "superengine --io-mode uci" as three different elements of argv[]. Or is it smart enough to join everything that is not recognized as an option name?

I don't think it is a problem to include extra quotes
in the ec argument. They just won't do anything.

In Windows it normally works fine since PG just passes the engine command to CreateProcess.

Well, that would again depend on if it can properly identify the engine command from its command line. If it gets called with the command line

polyglot -ec "superengine --io-mode uci"

would it pass

superengine --iomode uci

to CreateProcess? I guess it might, because the C startup file under Windows is responsible for breaking up the command line into argv chunks, and it might be smart enough to recognize the quotes, and put everything between them in a single argv[] element.

Directories with spaces should not be a problem on both Windows and Linux.

[Michel]

Well it is happy with

polyglot -noini -ec "superengine --io-mode uci"

in the shell. Since the shell removes the quotes I guess this means that superengine --io-mode uci is passed as one argv argument.

Is there any reason why xboard would not pass the engine command (the argument to fcp or scp) as a single argv argument?

About spaces: Windows tries to be smart about this. If you issue the command

super engine -io-mode uci

and there is a file in the current directory called "super engine.exe" and no file "super.exe" then it will take "super engine.exe" as
the file name.

This leads to suprises since Rybka 3 apparently comes with a bunch of files

Rybka.exe
Rybka Human.exe
...

So if you issue Rybka Human.exe it will silently pick Rybka.exe.

[H.G.Muller]

Well it is happy with

polyglot -noini -ec "superengine --io-mode uci"

in the shell. Since the shell removes the quotes I guess this means that superengine --io-mode uci is passed as one argv argument.

Is there any reason why xboard would not pass the engine command (the argument to fcp or scp) as a single argv argument?

The reason is that this would not work on most engines. If I type

xboard -fcp "gnuchesx xboard"

on the command line, the argument to -fcp is

gnuchesx xboard

If this was passed as a single argv argument (argv[0]) to the exec call, it would lead to the error message:

gnuchesx xboard: not found

because Linux would try to find a binary with the name "gnuchesx xboard". So it is essential that "gnuschesx" goes in argv[0], and "xboard" in argv[1]. So that a binary with the name "gnuchessx" is started, and it can see that its first argument was "xboard".

So if the argument to -fcp would have been "polyglot -noini -ec glaurung" it will be passed to the negine process as

argv[0] = "polyglot";
argv[1] = "-noini";
argv[2] = "-ec";
argv[3] = "glaurung";

That is OK, because it is just what Polyglot expects. But if you would have had -fcp "polyglot -noini -ec superengine --io-mode uci", this would be

argv[0] = "polyglot";
argv[1] = "-noini";
argv[2] = "-ec";
argv[3] = "superengine";
argv[4] = "--io-mode";
argv[5] = "uci";

and after -fcp 'polyglot -noini -ec "superengine --io-mode uci"' you would get

argv[0] = "polyglot";
argv[1] = "-noini";
argv[2] = "-ec";
argv[3] = "\"superengine";
argv[4] = "--io-mode";
argv[5] = "uci\"";

Polyglot (under Linux) has no say in this: the line is chopped up this way before it is passed to the execv call that invokes it. So unless it does something special to join argv[3-5], e.g. based on the fact that argv[3] started with " and argv[5] ended with it, Polyglot will not pass the proper argv[] array to the execv call that invokes the engine. In fact, in the last example it is likely to mess it up to the point where is not even able to find the engine binary, as it will be looking for a file the name of which starts with ".

In Windows this works quite differently, as there is no chopping up before the CreateProcess call, and the whole command line would be passed as a single text string

polyglot -noini -ec "superengine --io-mode uci"

Now polyglot.exe itself is responsible for the chopping up, and the filling of the argv[] array. And I think the C startup code that does this is smart enough to recognize the quotes as special, so that it chops:

argv[0] = "polyglot";
argv[1] = "-noini";
argv[2] = "-ec";
argv[3] = "superengine --io-mode uci";

and likely Polyglot will then pass this entire argv[3] to CreateProcess, so that the engine (if it is a C program) will find at its inception:

argv[0] = "superengine";
argv[1] = "--io-mode";
argv[2] = "uci";

[Michel]

But it seems to me that if xboard invokes an adapter it should simply not break up the argument to -fcp. It should leave the breaking up to the adapter if it chooses to do so.

It is extremely unusual in Linux for programs to mess around with their arguments. It is the caller's job (e.g. the shell) to pass the arguments to the program. In this case the caller is xboard. So it should pass proper arguments to polyglot.

I would just specify that in the adapter template the replacable strings are not broken up according to spaces but passed as single arguments
to the adapter. That would take care of this issue.

[H.G.Muller]

But it seems to me that if xboard invokes an adapter it should simply not break up the argument to -fcp. It should leave the breaking up to the adapter if it chooses to do so.

It is extremely unusual in Linux for programs to mess around with their arguments. It is the caller's job (e.g. the shell) to pass the arguments to the program. In this case the caller is xboard. So it should pass proper arguments to polyglot.

Yes, I guess that is the root of the problem. XBoard is not nearly as smart as the shell. WinBoard can even do less, as C programs in Windows have built-in shell functionality, and parse their own command line in a shell-like manner. But a Linux compile of the same code will not.

One option would be to call the engine process through the shell: let XBoard start up a shell, and then feed it the engine comand line through the pipe to its standard input. This involves a lot of overhead, though. An alternative would be to have XBoard do shell-like processing of the command line.

I would just specify that in the adapter template the replacable strings are not broken up according to spaces but passed as single arguments to the adapter. That would take care of this issue.

In the current mplmentation thi is a bit difficult, as by the time the engine process is created XBoard has no idea that the command line was the result of expanding the adapterCommand. The expansion happens at startup, and replaces the original -fcp / -scp argment. If I would defer expanding the command to there, the expansion would also take place for other engine command-lines (not running through an adapter).

It would also be too specific for the adapter command; the same problem could occur when I want to run a UCI engine not through the -fUCI flag, but by explicitly invoking Polyglot. Neither writing

xboard -fcp "polyglot -noini -ec superengine --io-mode uci"

nor

xboard -fcp 'polyglot -noini -ec "superengine --io-mode uci"'

will work there now. So the problem is really unrelated to the expansion. So I guess building in some shell-like treatment of the engine command line into XBoard is the only good solution.

Of course we can wonder if this is not an illusory problem. Are there really UCI engines that need arguments in Linux? If not, it might be perfectly acceptable to just ignore the problem, as long as the adapterCommand does not attempt to do any quoting. Apparently this is the policy that was adopted for XBoard so far. In WinBoard, OTOH, it would be better to have it do quoting in the adaperCommand. Not that UCI engines there are more likely to need arguments, but there the program or directory names themselves are likely to contain spaces, which would break things without quoting. And the exec mechanism of C under Window is such that quoting is handled automatically.

winboard -fcp {"polyglot 1.4.46b" -noini -ec "'super engine' --io-mode uci" -fd "C:\Program Files\super"}

would giv no problems at al there. The argument to -fcp,

"polyglot 1.4.46b" -noini -ec "'super engine' --io-mode uci" -fd "C:\Program Files\super"

would be passed as a complete line to Polyglot, which would find then in its argv due to C-startup processing as

polyglot 1.4.46b
-noini
-ec
'super engine' --io-mode uci
-fd
"C:\Program Files\super"

So it would pass to the engine process the full line

'super engine' --io-mode uci

which would be broken down by the startup routine of the engine as

super engine
--io-mode
uci

Due to the Windows mechanism for finding files that you sketched, the quotes around "polyglot 1.4.46b" and 'super engine' might not even be needed (depending on the names of ther files in the same folder).

[Michel]

I have no time to think about this today.

However it seems that at least on Linux the problems could be solved by passing the adapter command (or any engine
command for that matter) through wordexp. This will ensure that shell quoting rules are applied. It then becomes
the responsibility of the user to use proper quoting.

For maximum portability the source of wordexp could be included with xboard. Googling I found for example the following

http://www.minidvblinux.dyndns.org/svn/ ... /wordexp.c

I compiled and tested it and it works fine.

I am considering doing the same for processing the EngineCommand in Polyglot. This should solve the filenames-with-spaces-issue once and for all.

[H.G.Muller]

Wordexp would indeed do it, but I wonder if that isn't too heavy gunnery for what we want. As I understand it, wordexp would also expand wild-cards like *, and might result in a list of engine commands.

Especially if we would have to iclude sources for portability, I might as well write someting for it myself. The current code looks for the next space, and replaces it by a terminating NULLCHAR for that argv[] element. It would be easy enough to first test if the next character is " or ', and if so, skip it and look for the next " or ' to terminate the argv[] element, rather than for a space.

That would not allow escaping of quotes within the quotes, but would we really need that? Even the quoting itself would hardly be used.

current code:

Code: Select all

    /* We do NOT feed the cmdLine to the shell; we just
       parse it into blank-separated arguments in the
       most simple-minded way possible.
       */
    i = 0;
    strcpy(buf, cmdLine);
    p = buf;
    for (;;) {
   argv[i++] = p;
   p = strchr(p, ' ');
   if (p == NULL) break;
   *p++ = NULLCHAR;
    }
    argv[i] = NULL;