Resignation thresholds

[sje]

Early chess programs did not have a resignation facility; a program losing a game played until mate or until its operator resigned on its behalf. I believe that Belle by Ken Thompson was the first program to be able to resign on its own and it would throw in the towel after it saw it was a Rook (or equivalent) down.

Some programs like Crafty have a more sophisticated resignation mechanism that takes into account the opponent's rating/title, the amount of time remaining on the clock, and the current expectation. It would be possible to add other factors such as the stage of the game, whether or not tablebases were available, and perhaps how much time had already been spent playing to hopes of wearing down a human opponent.

Symbolic has had its resignation facility changed several times. One component that it has always had is to resign if the opponent would have a mate-in-one move after a search; so the program has never been checkmated. Unlike Crafty, Symbolic does not do any opponent modelling when determining when to resign; as the phrase goes, it plays the board and not the man. This may at times result in a premature resignation, but I have chosen this route because it makes it easier to compare games against different opponents. Likewise, Symbolic has no provision for an opponent specific contempt factor.

Symbolic has a C++ class CTDRTrack (Draw/Resign tracker), and a single instance of this class is used by the command processor in use (e.g., the xboard interface) to determine when to offer/accept draws or to resign. The instance keeps track of the last N search result scores (currently N=5); only those results that have real scores are retained. Searches that return book moves or only one move available are not tracked. If the tracker sees N scores in a row such that 1) the first score is below the resignation threshold (currrently six pawns) and that 2) each successor score is lower than its prior score, then the command processor will issue a resignation.

The above seems to work okay. It helps a lot in endgames when the tablebases indicate a long loss by determining if the opponent has sufficient endgame knowledge to win.

The idea of a score tracker can be useful in other ways. For example, it could also be used to politely suggest that an opponent should resign when the score is high and steadily increasing. If chess had a doubling cube like backgammon, a score tracker could determine when to double or when to give up after the opponent doubled. Another example would be to use the tracker to modify the nominal search time allocation, increasing allocation when the program was getting into a downward slide or decreasing it when the opponent was starting to get into trouble.

For programmers: how does your program determine when to resign?

[H.G.Muller]

My engine Usurpator unintendedly resigns when the score was worse than checkmate (i.e. if it lost material on top of it)!

uMax does not resign at all. In self-play, however, I declare a game lost if the score of the best move is worse than -10. Before I did that, I saw it draw once against TSCP when two Queens plus one Knight behind! (Due to a 'mad Rook', which could continue to check way longer than TSCP could see a repetition or 50-move draw, so it just went for the material advantage.) So you might wonder if resigning is ever justified.

[sje]

One of my goals with Symbolic is to emulate strong human style play, and that includes resigning in positions where a strong human player would also resign.

Another reason for supporting appropriate resignation is that there is likely another game waiting to be played and so there's no purpose in wasting time on a nearly certainly lost position.

[bob]

My approach is to resign when score < threshold for N moves. Threshold and N are set based on things like the rating of my opponent, whether he is a GM, IM or something else, etc. Against a GM, I set N=3 so that I resign pretty quickly, and I also set threshold to something reasonable. I don't expect to win or draw against a GM if the eval drops below some number like -3.0, for example.

[H.G.Muller]

Aren't you afraid you woud miss some opportunities to draw by sacrificing material, and subsequently keeping the opponent in eternal check? (Not fo the kind where he is forced to alternate between two squares, which would be easily recognized and scored as a repetition draw, but where his King can be really chased around on a significant fraction of the board, so that he can avoid repetitions all the way to the horizon).

Or do you specifically recognize such a situation, and express the inability of the opponent to escape check within the horizon in the score (to an extend where it never exceeds the resign limit)?

[bob]

If I can see the repetition, I certainly go for it since drawscore is way above -threshold. And the weaker the opponent, the lower -threshold goes to prevent resigning at (say) -4 against a 1500 player where we are doing something like playing with rook odds.

I simply trust the search depth to uncover the repetition, but that's also why I require that the score be below -threshold for N consecutive moves, since I have seen lots of games where we find a deep tactical shot for our opponent, but he simply misses it and doesn't find it, or he makes the right first move, but then does not follow-up correctly. Hence the N consecutive moves where the score is bad..

[H.G.Muller]

For the most common eternal checks the search depth will probably more than sufficient. But I can imagine many mundane cases where a King with a piece used as a shield (say Rook or Bishop) harrassed by a Queen can avoid repetitions for more than a hundred moves. Even for a King alone there might easily be 16 squares available, and then it would take 32 ply to force the first repetition. With other Queen(s) on the board such depths will be difficult to reach.

[diag]6QQ/4kp2/6p1/8/q7/8/2K5/8 w[/diag]

E.g. this (after h7-h8Q, Qd4-h4+) might not be a good time to resign. How long can the white King stroll around before it is forced to repeat?. As long as it does not repeat, the score stays at +7...

[bob]

I'm not certain, but I thing Little Goliath used to do something about this. If there were n consecutive checks, it called it a draw, period. And it lost many games because of this "bug" or "feature" as well. And regularly reported draw scores that were dead wrong...

But it is an idea...

[H.G.Muller]

Well, it obviously depends on how large n is. If it is too small you will indeed see many unjust draw scores. If it is larger than the regular search depth, it has no effect. It should never be used to prune the branch, like a repetition draw would. It should only affect the evaluation at the end of the branch, if the path leading to it consists almost exclusively of check evasions.

A more fundamental way to solve it would be a static routine to look for a square where the King could withdraw safely, and declare the draw only if it can't find one. Not so easy...

The funny thing is that engines tend to play the checking moves as 'spite checks', just to delay the inevitable losses they can see in the other branches. This means they usually do draw in such positions, even if they don't realize it while playing the moves. Despite the wrong score they still play the good move. This makes it risky to resign on the score.

Another alternative would be to not change the evaluation, but just pass an extra bit of information towards the root along a branch that ended in n consecutive checks, if there are no moves found in that node that score better than a draw. If that bit reaches the root, don't ever resign.