Computer tournament has "time per move" issues

[stevenaaus]

I've been writing a Computer Tournament feature for my project, Scid vs. PC http://scidvspc.sf.net. The latest code is in subversion

Code: Select all

svn co https://scidvspc.svn.sourceforge.net/svnroot/scidvspc

and requires tcl 8.5 (available on *nix and winodws) to run. It is very powerful, automatically running large tournaments and saving results to a SCID database. [RobbLito just beats out Stockfish and Shredder Classic 4 in my testing].

It all runs runs fairly good, but has time control issues with some xboard engines, especially xchenard and sjeng. The problem is, using a "time per move" system, when these engines are playing each other they make the first few moves fine, but then speed up dramatically. I don't know the xboard protocol well, but i've tried every combination of the "st TIME" and "time TIME" command, with TIME being seconds or centiseconds, without resolving the issue.

My control flows aren't sophisticated. I just issue "st" and/or "time", "go", then wait for a line like "my move is" from the engine.
Some xboard engines work fine - Crafty, Scorpio and Gnuchess 5. The establuished UCI engines all work great too.

Any ideas ?

[H.G.Muller]

This is obviously a problem within the mentioned engines. I can only answer on a similar issue for my own engines: micro-Max can only use a very small fraction of the time in fixed-time-per-move mode. Because it has a non-interruptable search, it can only check for the time and decide to play a move after an iteration finishes. Because the time it will take to finish is extremely variable and unpredictable, the average time can be set to at most 10% of the maximum time. If you would set it any higher, it would exceed the maximum time about once every 10 moves, which would make it lose 99+% of all games on time.

[stevenaaus]

This is obviously a problem within the mentioned engines.

It seems that way, but they're much better behaved when playing against an UCI engine, when they only make rapid moves half the time. Against each other - after the first few moves - they constantly make rapid moves. My code must have bugs of it's own i think.

the average time can be set to at most 10% of the maximum time. If you would set it any higher, it would exceed the maximum time about once every 10 moves

Makes sense, but there is no time related defaulting in my tournament (apart from the configurable 60 second move time-out).

[stevenaaus]

Oh yeah...

Issuing the "hard" command seems to make Sjeng and Chen much better... but it has the side effect of making other engines totally abuse their time slice, and often get timed out.

[H.G.Muller]

Makes sense, but there is no time related defaulting in my tournament (apart from the configurable 60 second move time-out).

That might be, but the engines don't know that. They are programmed to stay with better than 99.9% probability per move to within the specified searchTime. And on some of the more primitive time-management schemes, this requires enormous safety margines.

Indeed pondering can alter the behavior dramatically. If the engine has already reached a high iteration during ponder, it is going to try to finish that one. This might on the average take much more time than when you have them iterate from scratch. So this is normal and expected behavior.

What would be strange if it depends on the opponent. Do you do this on a single-CPU machine, or with SMP engines configured to use all available CPUs, so that the pondering of one would compete with the thinking of the other?

[stevenaaus]

I run an smp enabled linux-2.6.33.4 Core 2 Duo, but haven't investigated the threading capabilites of any of this yet. Running pstree i get "tkscid─stockfish─7*[{stockfish}]", while sjeng and xchenard seem only single-threaded.

Issuing the "hard" setting, sjeng and xchenard always go great. I'm thinking i might do some generic string matches for the engine name, and issue "hard" unless they are known to misbehave aka Phalanx, which doesn't have any inbuilt time control afaics.

[H.G.Muller]

Well, if you only have a Dual, and some engines use both CPUs, you should not be surprised if the opponent affects the behavior of an opponent with ponder on. That you get the impression that it matters if the opponent is UCI or WinBoard is probably just accidental correlation between which engines are UCI and which are SMP.