Shall I fix PV line truncating?

[voroskoi]

Hi,

I am new to this forum and started chess programming a couple of months ago. I have a working engine, I am quite happy with it, but noticed that the PV line sometimes shorter than expected. After googling around a bit I have found a couple conversations when this issue comes up.
They usually say that it happens because of the transposition table and I can confirm that switching off the TT makes this issue go away, but the engine gets slower and the PV line and best score also vary between the TT enabled and disabled version.

It is unclear to my if PV line truncation is just a cosmetics issue or it has other deeper implications which changes the behavior of the engine on the long term?
If it has to be fixed my plan is to generate zobrist hash after each move and get best move from the TT until I reach the necessary PV length. Is that the right way to do it?

Thanks,

[H.G.Muller]

It is mainly a cosmetic issue as far as the quality of the move that is selected at the root goes. But ik you want to use the engine for analysis, rather than to make it play games at the maximum strength, PV truncation can be pretty annoying.

Retrieving the PV from the TT after the search is one way to avoid truncation on a hash hit. But since TT entries can be overwritten this doesn't guarantee you will get the same PV as the one the search used to propagate the leaf evaluation to the root. So you can for instance get mate scores while the PV does not end in checkmate, or the other way around. And if the PV contains a repetition, the retrieval process would follow the loop forever.

A poor-man's solution to this is to make the search refrain from making a hash cutoff in a PV node. (I.e. when you get a hit of the required depth with an exact score within the {alpha, beta} window.) In theory this makes the engine slightly weaker, though, e.g. making it impossible to see 'over the horizon' checkmates through grafting the TT results of deeper searches on the current tree.

Another solution would be to have a special hash table for PV nodes, where you store the entire PV rather than just the first move; when you get a hash cutoff by a hit on any of those you can copy the PV back to the tri-angular array that you use to construct it, to avoid it contains a truncated one.

[voroskoi]

Thank You for the detailed answer!

Another solution would be to have a special hash table for PV nodes, where you store the entire PV rather than just the first move; when you get a hash cutoff by a hit on any of those you can copy the PV back to the tri-angular array that you use to construct it, to avoid it contains a truncated one.

Just to make sure I understand it correctly: You suggestion is to create a hashmap with best_move from the TT as a key and full PV-line as the value and I should create/update the entry whenever I find a PV-node.
Am I right, that I should only add entries to this hashmap when alpha<score<=beta (in other words when I write an exact TT entry) and I only need to append the pv-line from the hashmap when I return an exact entry from the TT.
Otherwise it would grow huge, isn't it? And I assume I should clear this hashmap like the TT after each search.

[H.G.Muller]

No, the key to this extra table should be the normal hash key. It needs far fewer entries than the normal TT, because it is only used for storing PV nodes, and these are very rare. You could have the entries age in the same way as in the normal table. (E.g. discard anything that was not probed in the latest N searches.)