Static evaluation of 'quiet' positions

[Leen Ammeraal]

When tuning the evaluation functions of our engines, it would be very helpful if there were a set of positions, with a reasonable static score for each of them. Then we as programmers could use this as a reference. This would be especially useful for those chess programmers, like me, whose are primarily programmers, not chess experts.

For example, here is a position in the Dutch book 'Practische schaaklessen, part 4' by Euwe (I have been using this book as a reference for more than 50 years!):
[diag]r2q1rk1/pb1nbppp/1p3n2/2pp2B1/3P4/2NBPN2/PPQ2PPP/2KR3R[/diag]
White to move
r2q1rk1/pb1nbppp/1p3n2/2pp2B1/3P4/2NBPN2/PPQ2PPP/2KR3R w - - 0 0
(Rubinstein-Teichman, Vienna 1908, after 10. ... c5)
According to Euwe, White's position is much better than Black's because White's pieces are much more active.

How many points (centipawns) is this this advantage of White worth?
I just tested my chess engine Queen and it gave 84 as a static score, which
seems to be rather on the high side. What do you think?
Notice that my question is about STATIC position evaluation, not about thinking moves ahead.
It would be nice if the number of position in the set in question would be greater than 1.
Are there any volunteers who can supply some more 'quiet' positions, along with static scores?
Before posting this message, I asked myself if this belongs in the programmer's section, but I think it does, since it is my experience that
only chess programmers can properly distinguish between static and tactical evaluation of positions. Those who can only play chess, immediately start making moves to evaluate the positions that will occur after these moves!
Leen

[David Weller]

Hi Leen ,

1st, There is a work in progress, though I am ashamed to say I cant remember who is doing it...

And, Crafty has a command to print out evaluations of a set of positions in an epd file.

try searching ccc

I will let you know if I remember anything....

[José Carlos]

Hi Leen. Averno says 0.03 for that postion. I'm not sure why Euwe and Queen think white is so much better. As a decent chess player (ELO 2100-2200), I think both sides have chances.
I'd be interested in other programs static eval and strong chess player opinions.

[Ralf Schäfer]

Hi Leen and Jos?,

Spike says that white has a slightly advantage of 0.2 pawns. But this seems to be a result of the piece square tables only, all other positional aspects are nearly the same for black and white: white has a slightly better king attack, but black can compensate this with a good pawn-structure.

Spike does not know about mobility, center control or strong and weak squares yet, so maybe he misses some important positional aspects here...

best
Ralf

[David Weller]

Here is that thread .... seems you were already there
http://chessprogramming.org/cccsearch/c ... _id=350542

I wonder if the test set got any bigger...

[Tord Romstad]

When tuning the evaluation functions of our engines, it would be very helpful if there were a set of positions, with a reasonable static score for each of them. Then we as programmers could use this as a reference. This would be especially useful for those chess programmers, like me, whose are primarily programmers, not chess experts.

This is an excellent idea, which was also suggested by Sergei Markoff on the CCC a few weeks ago. Unfortunately, few people seemed to be interested.

In your position, Glaurung 0.1.4 thinks that white has an advantage of 0.32, while Gothmog 1.0 beta 11 actually thinks *black* has a small advantage, the score is -0.21.

Tord

[Leen Ammeraal]

Thanks to all who reacted. It just crossed my mind that a strong program that enables the user to enter any position and then can display the static score would be better than long tables. Although Queen is not as strong as I should like, it offers this facility. For those who are interested, use the GUI version queen.exe (not the WinBoard version queen-wb.exe), set up a position using Ctrl+U (or used Ctrl+O to read a fen file and then still use Ctrl+U to enter the set-up mode), and press the space bar. The static evaluation score for that position is then displayed in the top-left corner of the screen. Which other programs (preferably strong ones) can do something similar? Would it not be a good idea for you, other chess programmers, to add this option to your engine if you have not done so already? There was a posting about Crafty being able to display static scores. I will try to find out about this. Any help would be welcome.
Leen Ammeraal

[Richard Pijl]

The Baron's static evaluation is 0.20 in this position, mainly due to bonusses for the white bishops and rooks, but it gets a few minus points for the slightly worse king safety/pawnshield.
Richard.

[Uri Blass]

Note that I was not interested in the question because movei's evaluation is simply not static.

It means that the evaluation is not dependent only on the leaf position
but also on the path.

It is probably better to get rid of part of the non static evaluation and replace it by static evaluation but I still did not do it.

The part of the non static evaluation that it is better to replace is the evaluation of mobility.

Movei simply use in most cases the number of legal moves of the side to move and number of legal moves of the opponent in the previous ply and the phase of the game to evaluate mobility

Exceptions:
1)If the side to move is in check in it goes 2 plies backward until it finds a position that it was not in check and use the number of legal moves of the side to move in that ply to evaluate mobility.

2)If the opponent was in check in the previous ply it goes 2 ply backward
and use the number moves of the opponent in that ply.

Even in the case that both sides were not in check the last move has influence on the mobility evaluation of movei so it cannot give static evaluation of mobility.

Uri

[David Weller]

GES gives a score of -36

but if I turn on my [simple] mobility term, it goes up to -3

so it seems mobility [/activity?] IS a substantial contributer to the position

[Michael Yee]

There was a posting about Crafty being able to display static scores. I will try to find out about this. Any help would be welcome.

There was also a flurry of posts in CCC a little while ago about static eval tests... (I think they were initiated by SmarThink's author, Sergei Markoff.)

In any case, I can't seem to locate them now, but here is one post where Sergei shows how to get SmarThink's static eval:

http://chessprogramming.org/cccsearch/c ... _id=356342

(Note: I think crafty and yace also let you do this.)

[Vladimir Medvedev]

GreKo's static evaluation is -0.26, due to potentially dangerous position of white King.

[Marek Strejczek]

Nesik says it is 0.02 for white, but I'm unable to tell now what factors it rewards and penalizes for each side in this position.

[fierz]

i get a static eval of +0.15 for white which breaks down to approximately the following:

-0.30 king safety
+0.09 pawns
+0.35 pieces

honestly, i don't really know how to evaluate this position properly. the "instant mobility" which is probably something which we are all using can be very misleading. i guess the lack of any active plan is blacks problem here but that is very hard to code in a static eval

cheers
martin

[Leen Ammeraal]

Since the static score of Queen was 0.84, I tried to get this somewhat lower by changing some parameters about mobility. So far, the beta versions obtained in this way lost all matches against the release version with this high score of 0.84 for this position.