Evaluation function -- important factors?

[ffao]

Hi everyone.

A FM has challenged me to write an engine that could beat him. Being so, I cannot afford to have an engine that plays unsound positional chess, otherwise it's going to be crushed. So I started writing and have mainly worked on the search part of it.

When I'm playing against engines like Fritz 5.32 and Comet B68, it's not uncommon for the opponent to think that he is a full pawn ahead, while my engine thinks the game is tied. For example:

6k1/6pp/r1bR1n2/prB1p3/1pp1P3/5P1P/PPBR2PK/8 w - - 0 29

Bubble 1.3 -0.03

Rybka 3.0 +1.00
Crafty 22.0 +1.53
Comet B68 +1.49

If that problem really is eval-related, then I must be missing something. My eval is currently mainly comprised of piece-square tables, so I tried to add mobility/isolated and doubled pawns/pawn shield, but it started playing really crazy/unbalanced chess.

Question is then, how would you go about creating an eval function from scratch? What is the minimal amount of factors you need to add (such as pcsq tables, mobility, passed pawns, etc) to have a decent evaluation function? And, last but not least, how do I balance the numerous weights of such an evaluation function?

Thanks for any suggestions -- I'm really lost here!

[H.G.Muller]

From micro-Max I have learned that the most-important pawn structure term (apart from hefty bonuses for being on 6th or 7th rank) is to avoid 'holes' in front of your Pawn chain. i.e. avoid pushing e3 if you don't have another Pawn covering f4 (i.e. on g3).

There is an engine called FUSC# which has 6000 eval parameters, developed by the Free University in Berlin. Even the most simple version of micro-Max (of 1433 characters) beats it easily.

[Rohan Padhye]

Just piece square tables and mobility will make the engine play a semi-decent game of chess, especially in the middlegame. However, endgames are a different story, because the positional value of the piece may not always be the best, and there are other threats to worry about (such as promotions).

I learnt this the hard way. More than 80% of the games that Frittle loses its ELO range are because my engine tries some tactical manuveurs gifting the opponent a passed pawn which cause havoc in the endgame with an eventual promotion. I really need to plug in some endgame knowledge, sigh.

Also, if you are not using bitboards, mobility calculation may be pretty expensive. If your mobility = move generation, then you are spending almost enough to time to search an extra depth without the mobility calculations.

[H.G.Muller]

Recognizing passers is not really end-game knowledge. A game can be decided by it in the middle-game, by making an unnecessary wrong Pawn trade.

I don't know what the Elo range of your engine is, but micro-Max proves that it is possible to get above 2000 on the CCRL scale without passer recognition, without true piece-square tables, and without mobility.

[Rohan Padhye]

Recognizing passers is not really end-game knowledge. A game can be decided by it in the middle-game, by making an unnecessary wrong Pawn trade.

I don't know what the Elo range of your engine is, but micro-Max proves that it is possible to get above 2000 on the CCRL scale without passer recognition, without true piece-square tables, and without mobility.

That is true, although many times because of lack of endgame playing ability, my engine just tries to get the king in the centre during endgames and tries to gain material (maybe a doubled pawn which wasn't worth much anyway) rather than avoiding a passed pawn threat (or some other silly move like that). I have assumed that the problem lies in the eval, because the passed pawn threat is created at such a distance that the ease of promotion cannot be seen at the search depth (at least in the quick sample games).

I think if the search depth was higher it would have realised that the route leads to easy promotion or loss of the defending piece and avoided leaving the passed pawn anyway.

My engine is wayy down somewhere in the 1200-1400 range (I have just started working on it), so maybe it is a completely different experience, since all engines in the Elo range have weak points.

PS: I just looked over some parts of the Micro-max source.... It's insane!!! (in a good way)... So what do you use in your eval function? And no, I can't figure it out from the source

[ffao]

Also, if you are not using bitboards, mobility calculation may be pretty expensive. If your mobility = move generation, then you are spending almost enough to time to search an extra depth without the mobility calculations.

True. I did some testing today and in my engine it's actually a 2-ply difference (just looping through the pieces causes a lot of overhead, not to mention generating moves for them). Maybe that's why the pcsq version performs so much better... Even so, I haven't seen any 0x88 program that actually doesn't account for mobility. Either my mobility count is too slow, or they search high enough that it doesn't matter.

So, I'll try to make an evaluation function as lightweight as possible to preserve the speed. I'm thinking basically of pawn structure (that "hole" idea sounds good, does it work indirectly on passers?) and primitive king safety, because the majority of games I lose are due to attacks on the king.

Not that the speed of my program is stellar -- with only material/pcsq eval, it used to search 600 kN/s, and today I've been able to improve it by 50% by changing only 7 lines of code (to 900 kN/s).

PS: I just looked over some parts of the Micro-max source.... It's insane!!! (in a good way)... So what do you use in your eval function? And no, I can't figure it out from the source :)

There is an explanation at http://home.hccnet.nl/h.g.muller/eval.html. I wouldn't say it's easy to read, but it definitely beats reading the source!

Wish me good luck on the match! I want Bubble to give him some serious trouble :)

[H.G.Muller]

In uMax 4.8 ( the 'strong' version, in terms of Elo/char) the most important eval terms responsible for its strengts (apart from material) are Pawn bonuseds for being on the 6th and 7th rank of nearly 1 or 2 Pawns, respectively, and a general Pawn-push bonus that increases from 9 to 19 centiPawn as the material on the board decrease. (This could have been implemented as a PST that is adjusted at the root bfore every search. But the uMax evaluation is differential, so I just add this bonus for every Pawn push.) I give a 9 cP penalty for a Pawn missing two squares right or left (because pushing the Pawn then leaves a square undefended by a Pawn.)

For King safety I freeze the King in the middle-game, by giving a 20cP penalty for every King move. And I freeze the Pawn in frnt of a King in a similar way, as a kind of minimal Pawn shield.

That's about it. The bonus for Pawns on 6th is large enough that it does give priority to preventing opponent passers from getting there. Even a Pawn on 2nd can reach 6th in only 3 moves, so usually it is within the horizon.

[Grant Osborne]

Hi H.G.

Do you have a pawn hash table in uMax?

If a pawn position and it's score is saved in the pawn hash table, then subsequently retrieved after pieces have been captured, the score you retrieve would be incorrect even though no pawns have moved. Or perhaps you have Zobrist Keys for each game stage?

Grant

[H.G.Muller]

There is no Pawn hash table in uMax. Apart from that it would take extra characters, it would probably just slow it down. I would not even know how to implement one, as the evaluation is entirely differential, and there is not a separate differential Pawn evaluation.

There is so little Pawn evaluaton in uMax, that updating a Zobrist key for Pawns would already take many times what it takes to update the evaluation itself.

The main hash in uMax also suffers from many inconsistencies, though. E.g. by penalizing King moves the same position has a different evalution if it is reached through a path where the king took a detour. And pushing a Pawn two steps only harvests a single Pawn-push bonus, while if you get to the same square in two single steps youget it twice. Not to mention that any piece gets piece-square points, but I don't take the trouble to subtract them again when the piece is captured. Also, the variable Pawn-push bonus is determined from the material situation in the root. So its contribution to the score in the main hash is not what it should be after, say, a Queen trade at game lvel.

In practice there was no measurable effect from all these inconsistencies on playing strength.

[pimidrez]

Hi everyone.

A FM has challenged me to write an engine that could beat him. Being so, I cannot afford to have an engine that plays unsound positional chess, otherwise it's going to be crushed. So I started writing and have mainly worked on the search part of it.

When I'm playing against engines like Fritz 5.32 and Comet B68, it's not uncommon for the opponent to think that he is a full pawn ahead, while my engine thinks the game is tied. For example:

6k1/6pp/r1bR1n2/prB1p3/1pp1P3/5P1P/PPBR2PK/8 w - - 0 29

Bubble 1.3 -0.03

Rybka 3.0 +1.00
Crafty 22.0 +1.53
Comet B68 +1.49

If that problem really is eval-related, then I must be missing something. My eval is currently mainly comprised of piece-square tables, so I tried to add mobility/isolated and doubled pawns/pawn shield, but it started playing really crazy/unbalanced chess.

Question is then, how would you go about creating an eval function from scratch? What is the minimal amount of factors you need to add (such as pcsq tables, mobility, passed pawns, etc) to have a decent evaluation function? And, last but not least, how do I balance the numerous weights of such an evaluation function?

Thanks for any suggestions -- I'm really lost here!

Hi Ffao, I was wondering if you could improve your evaluation function.
My evaluation (material and PST) is very similar to yours and would like to know which parameters you've been adding.

cheers!

[Dann Corbit]

I suggest looking at Olithink.
You will find extremely scanty evaluation and yet incredible strong play (for such a tiny eval).

The focus is on mobility, which is clearly very important.

[pimidrez]

yes, thats right I was thinking to implement mobilty evaluation, for keep pieces in the center I give +0.35
but which is the score for each cell you can move?

pimidrez

[Dann Corbit]

yes, thats right I was thinking to implement mobilty evaluation, for keep pieces in the center I give +0.35
but which is the score for each cell you can move?

pimidrez

Try a range of values and choose the one that works best for your program

[Edmund]

yes, thats right I was thinking to implement mobilty evaluation, for keep pieces in the center I give +0.35
but which is the score for each cell you can move?

pimidrez

Try a range of values and choose the one that works best for your program

Maybe you want to try a nonlinear increase. For example it is extremely bad to have a piece with a very low mobility.

[Dann Corbit]

I have also seen mobility calculations where the bonus is a function of both the piece type and the game stage.
Don't know if it helps a lot, but the program that did this was pretty strong.

[pimidrez]

thanks both!
Now Im testing a mobility evaluation scoring by +0.01 per cell you can go for every piece, but I do not know if I should count the cells occupied by enemy pieces to or not, and also if that cells occupied by enemy pieces should have the same score than the others.

pimidrez

[Laszlo Gaspar]

Hi all,

My new engine Nimrod just evaluate mobility in non-linear way, which means little values have negative eval, big values have big bonus. I count the reached squares for the sliders only and doesnt count horizontal squares (for rook and queen).
So, my experiment is that removing it gives -40ELO. Considering the computation requirements, even with specialized data structure, I m not sure if it worth the effort. Maybe there are better ways than simple counting, or just my params were not tuned well.

Best regards,

László