Buggy engines and building a bug base

[Guenther Simon]

Hi Guenther

No problem at all with OpenOffice, I don't use MS Office suite either.

I was not able to reproduce the hanging bug of ChessAlex here with ponder off (Dell Core2Duo).

I'll start testing for the Promo group soon, many bug reports to come

Olivier

Hi Olivier,

May be there was a missunderstanding, but I would like to know how exactly you tested it
and which version? Note that it seems it won't happen against Humans. I noticed in 40/15 games,
in the task manager that it always used 25% of my CPUs(= 1CPU) despite being ponder off.
Actually it was not really pondering because it was not thinking or sending any debug output,
but it seems it did some polling stuff and only on my quad not on my single cpu machine.

Guenther

Hi Guenther

I ran a few games at 40/5 vs other engines with ponder off on my Dell Core2Duo. Version of ChessAlex was 2.0r4. Recent versions of Winboard_F and Fonzy's Polyglot were used. Nothing was hanging in the task manager after the test was over.

Olivier

Aha Olivier, that's not really what I described. I did not say it was hanging between the games but that
it locks one cpu _during_ the games which is practically the same effect as it would ponder in a ponder off
game.
(Please look at the Taskmanager during the games, when ChessAlex' opponents are thinking)

Guenther

[Miguel A. Ballicora]

By the way, I was reported that Gaviota failed to accept certain SAN moves from Arena. The problem was that Arena did not send proper SAN moves (it sent Bb5 rather than Bb5+, for instance). I hope this was fixed in Arena in the last release 2.0. Anybody knows if it was? Otherwise I should make Gaviota to accept this buggy behavior from the GUI, but I would prefer not to do it.

It is always a bad idea to make your engine produce pedantic error messages, like refusng input that in principle has all information it needs, based on a technicality (like +# in SAN, or a missing move count in a FEN). It can serve no other purpose than to cause failure.

I did not mean to be pedantic ever. I just followed the standard, thinking that other people would do too, particularly GUI writers, who are supposed to be very strict about their output. Why do we have a standard if the interfaces are not going to follow it? In fact, IIRC, xboard can deal with other type of moves that are simpler to parse; but I chose to follow the trend and I implemented SAN. Now some GUIs decide to send non-compliant moves and I should adapt again to their bugs? Yes, I am going to do it, but I wonder what the purpose of standardizing this was. Where do I draw the line? Should I accept nd4 rather than Nd4? should I accept Rbc1 when Rc1 is sufficient and correct? 0-0 instead of O-O? How relaxed should the parser be?

Parsing SAN moves is straightforward if you have a SAN generator and the people adhere to the standard. You generate all the SAN moves in a loop until one of them matches the input (good enough for an engine input, not for a PGN reader). I just counted SAN2move() is only 12 lines of code. This was about economizing resources and reducing code that is not critical to the speed of the engine. Reducing this code reduces potential bugs and memory footprint. if the SAN generator is bug free, the parser will be bug free too.

This is not really about laziness, because II have another faster SAN parser that I wrote to parse PGN files. However, I was always reluctant to use to parse input moves because of the reasons stated above.

Anyway, it looks like Gaviota may be the only one being strict with SAN input because I did not see complains about other engines.

Miguel

[Miguel A. Ballicora]

This is a fantastic project. It will help testers and authors to centralize this information. I am glad not to see Gaviota there for now

By the way, I was reported that Gaviota failed to accept certain SAN moves from Arena. The problem was that Arena did not send proper SAN moves (it sent Bb5 rather than Bb5+, for instance). I hope this was fixed in Arena in the last release 2.0. Anybody knows if it was? Otherwise I should make Gaviota to accept this buggy behavior from the GUI, but I would prefer not to do it.

Miguel

Hi Miguel,
Gaviota somtimes sends fake SAN moves (I don't mean the "+","#" suffix) !!
I stopped using Gaviota in ChessGUI for that reason. I will test again to get an example for you.
regards,
Matthias.

Sorry ! Current ChessGUI does not append "+","#" to any SAN move anymore, so Gaviota will not run under ChessGUI.

Matthias.

I will try to make Gaviota to accept non-standard SAN (standard algebraic notation), but it would be nice if you append + and # etc. to make your GUI fully winboard compliant . Looks like Gaviota is alone here, so the standard is to follow a non-standard standard algebraic notation

Miguel

[H.G.Muller]

My reccommendation is that a parser should always be as relaxed as possible. I now the WinBoard parser is, it accepts amost anything that looks remotely like a move (0-0, O-O, o-o, OO, oo, Nf3, Ng1f3, Ng1-f3, Ng1xf3, Ng1Xf3, Ng1:f3, g1f3, N/g1-f3...).

Your way of processing the SAN input is original, but it is not really a parser. WinBoard protocol was not really designed to use SAN; a parser of the long algebraic moves could be just as simple as what you have now. In principle the protocol does not guarantee that you will get moves in SAN: if you send feature san=1 it is perfectly legitimate within the protocol for the GUI to respond rejected san. And then what would you do?

The WB protocol specs contain a warning that other GUIs than WinBoard might not implement all the features WB does, and might not even understand SAN on input. This has always been a bit of a problem with the WB protocol description, IMO: it mixes protocol specs with a description of what WinBoard does, (and sometimes even what GNU Chess does!), often without clearly making the destinction about what is protocol, and what is merely a quirk of a particular implementation.

[Miguel A. Ballicora]

My reccommendation is that a parser should always be as relaxed as possible. I now the WinBoard parser is, it accepts amost anything that looks remotely like a move (0-0, O-O, o-o, OO, oo, Nf3, Ng1f3, Ng1-f3, Ng1xf3, Ng1Xf3, Ng1:f3, g1f3, N/g1-f3...).

Yes, I think that is a good policy for a GUI that has to deal with hundreds of engines.

Your way of processing the SAN input is original, but it is not really a parser. WinBoard protocol was not really designed to use SAN; a parser of the long algebraic moves could be just as simple as what you have now. In principle the protocol does not guarantee that you will get moves in SAN: if you send feature san=1 it is perfectly legitimate within the protocol for the GUI to respond rejected san. And then what would you do?

The most sensible thing to do is to come back to the old winboard notation, which I agree, is simple to be parsed. But, if an engine receives "accepted san", I think the engine can assume that it will receive proper SAN.
Miguel

The WB protocol specs contain a warning that other GUIs than WinBoard might not implement all the features WB does, and might not even understand SAN on input. This has always been a bit of a problem with the WB protocol description, IMO: it mixes protocol specs with a description of what WinBoard does, (and sometimes even what GNU Chess does!), often without clearly making the destinction about what is protocol, and what is merely a quirk of a particular implementation.

[Matthias Gemuh]

I will try to make Gaviota to accept non-standard SAN (standard algebraic notation), but it would be nice if you append + and # etc. to make your GUI fully winboard compliant . Looks like Gaviota is alone here, so the standard is to follow a non-standard standard algebraic notation

Miguel

Appending a "+" or "#" is too expensive for the value it brings.
The move must be made, attack boards generated, InCheck() called, eventually a check evasion made, and finally the move is unmade. All this extra work should be done by GUI for each move of the game. That's bad.
A GUI should use as little CPU as possible.
When ChessGUI wants to parse moves (from engines, pgn files, etc.), it first deletes all "!","+","?","#", etc.

Matthias.

[Dave Gomboc]

Appending a "+" or "#" is too expensive for the value it brings.

I think that's a false economy. The software doesn't care about the nanoseconds it takes to determine check or checkmate (which certainly does not require full attack-table generation, by the way). However, those symbols can help a human who's debugging things.

Either the GUI is operating in a SAN-compliant mode, or it isn't. Gaviota is perfectly justified to expect that confirmation that SAN will be used means that SAN will actually be used.

Dave

[H.G.Muller]

The efficiency argument seems a it far fetched, as it is intrinsically highly inefficient to generate SAN (compared to simply sending the coordinate notation). Just make sure that it goes at the expense of the time of the engine requesting th SAN, by starting his clock before you do the conversion.

Testing for check is a side effect of testing for mate, and a proper GUI would have to do that anyway to know if the game is finished. If not, t might hang with enginses that do not send result messages after their own mating move. (Which is not a requirement of WB protocol.)

[Matthias Gemuh]

Appending a "+" or "#" is too expensive for the value it brings.

I think that's a false economy. The software doesn't care about the nanoseconds it takes to determine check or checkmate (which certainly does not require full attack-table generation, by the way). However, those symbols can help a human who's debugging things.

Either the GUI is operating in a SAN-compliant mode, or it isn't. Gaviota is perfectly justified to expect that confirmation that SAN will be used means that SAN will actually be used.

Dave

Other GUIs can choose to waste precious time on expensive "+" and "#".
ChessGUI will most likely NEVER do that.
Almost all UCI and WB engines run perfectly in ChessGUI as it is.

Matthias.

[Andres Valverde]

[SNIP]


A little problem though is that I plan to do it in ods format(which is a free excel equivalent in OpenOffice,
but better and more compact), as I don't use MS Office here.

Guenther

Hello Guenther,

This should not be a problem as OpenOffice was developed to interact with us poor MS Office product users. You can save the sheet as Excel which can be used by those of us who have not yet made the switch.



Of course, maybe it would be simpler to output it as text which was suggested in this thread as well.

Thanks for the willingness to do this free work for the rest of us.

Later.

Hi Roger,

Yes I know I can export OO formats to excel, but this way I cannot save write protection for sheets.
OTH you and Chris are right that it makes not much sense to use the OO format for the read version at least!
I guess I will change the read version to a cleaned up htm table as in my chronology pages and use csv for
transfers or I other htm and excel export to allow the user sorting himself after he had downloaded
the excel version. Hopefully there will be such a version available tonight and it will also contain
already quite some content then!

Best regards,
Guenther

Hi Güenter,

Have you tought in making a Gmail sheet? . Write/read permissions are ok and they can be viewed from any computer without MS/O office installed.

Saludos, Andrés

[Miguel A. Ballicora]

Appending a "+" or "#" is too expensive for the value it brings.

I think that's a false economy. The software doesn't care about the nanoseconds it takes to determine check or checkmate (which certainly does not require full attack-table generation, by the way). However, those symbols can help a human who's debugging things.

Either the GUI is operating in a SAN-compliant mode, or it isn't. Gaviota is perfectly justified to expect that confirmation that SAN will be used means that SAN will actually be used.

Dave

Other GUIs can choose to waste precious time on expensive "+" and "#".
ChessGUI will most likely NEVER do that.
Almost all UCI and WB engines run perfectly in ChessGUI as it is.

Matthias.

If you want to save time, and you think that SAN consumes too much, why do you send "accepted SAN"?
Would not it be better to send "rejected SAN" and use the old "e2e4" winboard notation?

Miguel

[Matthias Gemuh]

If you want to save time, and you think that SAN consumes too much, why do you send "accepted SAN"?
Would not it be better to send "rejected SAN" and use the old "e2e4" winboard notation?

Miguel

Many engines use "feature" but don't know "accepted"/"rejected".
So they would just ignore "rejected SAN" and expect SAN moves.
WinBoard/Arena/ChessGUI/WB2UCI do a lot of weird things to support as many "buggy" engines as possible.

Matthias.

[Dave Gomboc]

Other GUIs can choose to waste precious time on expensive "+" and "#".
ChessGUI will most likely NEVER do that.
Almost all UCI and WB engines run perfectly in ChessGUI as it is.

Well, it's your software. If you're happy with it being buggy, that's your choice. <shrug/>

Dave

[Matthias Gemuh]

Other GUIs can choose to waste precious time on expensive "+" and "#".
ChessGUI will most likely NEVER do that.
Almost all UCI and WB engines run perfectly in ChessGUI as it is.

Well, it's your software. If you're happy with it being buggy, that's your choice. <shrug/>

Dave

Good bugs are wonderful

Matthias.

[H.G.Muller]

IMO it is both a mistake to send sub-standard SAN, as well as to require perfect SAN on input.

Come to think of it, even a protocol allowing the engine and GUI to communicate in SAN was a bad mistake...

[Miguel A. Ballicora]

If you want to save time, and you think that SAN consumes too much, why do you send "accepted SAN"?
Would not it be better to send "rejected SAN" and use the old "e2e4" winboard notation?

Miguel

Many engines use "feature" but don't know "accepted"/"rejected".
So they would just ignore "rejected SAN" and expect SAN moves.
WinBoard/Arena/ChessGUI/WB2UCI do a lot of weird things to support as many "buggy" engines as possible.

Matthias.

So, to micro-optimize the software and at the same time include buggy engines, you leave out the ones that process SAN correctly.
All right, Gaviota may have to do lot of weird things to support as many "buggy" GUIs as possible

Miguel

[Matthias Gemuh]

So, to micro-optimize the software and at the same time include buggy engines, you leave out the ones that process SAN correctly.
All right, Gaviota may have to do lot of weird things to support as many "buggy" GUIs as possible

Miguel

Gaviota does not need to support any "buggy" GUIs.
The "buggy" GUIs are living a quite happy life without Gaviota

regards,
Matthias.

[Ilari Pihlajisto]

Other GUIs can choose to waste precious time on expensive "+" and "#".
ChessGUI will most likely NEVER do that.
Almost all UCI and WB engines run perfectly in ChessGUI as it is.

Matthias.

The all-knowing programming god Donald Knuth said that "premature optimization is the root of all evil". Have you actually profiled ChessGUI with and without the check and mate notation? I'm curious because I've done a lot of profiling with Cute Chess (a work-in-progress GUI), and full SAN support is not even close to being a bottleneck. Updating the graphics on the visual chessboard takes way more cpu cycles, and even that's not significant if it's done right.

[Ilari Pihlajisto]

WinBoard/Arena/ChessGUI/WB2UCI do a lot of weird things to support as many "buggy" engines as possible.

Matthias.

In my opinion this is one of biggest problems with Winboard engines and GUIs. If an engine fails to obey the chess protocol, the GUI should simply reject the engine. This would keep the GUI codebase shorter, simpler and less buggy, and it would help engine authors as well, because it would be easier to detect bugs and non-standard behavior.

[Miguel A. Ballicora]

IMO it is both a mistake to send sub-standard SAN, as well as to require perfect SAN on input.

It is desirable to be strict in your output, and flexible in your input. However, there is a fundamental difference between the two. One is a necessity and the other is a feature.

Come to think of it, even a protocol allowing the engine and GUI to communicate in SAN was a bad mistake...

I agree. In fact, I think I just increased the support of non-compliant GUIs with the modification of one character in the source code.

I replaced

Code: Select all

printf( "feature san=1\n");

with this

Code: Select all

printf( "feature san=0\n");

I guess I will drop SAN support. The console mode will still accept it, but I am starting to see that there is no point in using it with the winboard/xboard protocol, particularly when there are GUIs that do not follow it.

Miguel