Smirf and nullmove does not go together

[Reinhard Scharnagl]

Hi Jos?,

But maybe you have a secret technique to deal with hash collisions...

may be I am disabled seeing such problems. On which type of hash collisions you are targeting? Do you mean a used cache entry? That is no problem at all, because I am using my TT multi associative.

Reinhard.

[Uri Blass]

Hi Uri,

The problem is that I may look only 4 plies backward or more than it to detect repetition because I know that repetition cannot happen after 2 plies.

that seems to be a very bad technic. I am not looking back along the game line. After deciding to expand a node, it will be locked within its cache entry and unlocked when done. If I will have to decide, whether a node has to be expanded, I will see when it might be locked already. That is the moment to answer with a zero evaluation.

(Oh bad, again I have given away another secret of mine ...)

Concerning Nullmove repetition: it would be very unlikely that for both sides it would be best to include a zero evaluation into the PV.

Reinhard.

Hi Reinhard,
I do not think that you gave away another secret because of the following reasons:
1)I am not sure if I understand you
2)I know that there are faster ways to detect repetition that are not used by me(I guess that you say that it is bad because of speed) but I decided that they will probably not give me a lot of speed and it is better to have something simple without bugs).

I could write something faster but decided not to do it because there are more important things to do and testing how much it is faster is not simple because probably in most positions it will give almost no improvement because fifty is going to be small when in some positions when long lines with no conversion happens it may give more significant improvement.

Uri

[Reinhard Scharnagl]

Hi Uri,

1)I am not sure if I understand you

that seems to be a problem. I do not know how to avoid that. I try to do my very best to be understood. May be it helps to point out, what might be unclear.

2)I know that there are faster ways to detect repetition that are not used by me(I guess that you say that it is bad because of speed) but decided that they will probably not give me a lot of speed and it is better to have something simple without bugs).

the method I have descripted does not at all target on speed though it might not be slower. It is used because of reasons for to have a better integrity of the search tree, e.g. to protect positions of the actual game line to be overwritten in their cache entries at the wrong time. The detection of repetitions is a welcomed indirect gift of that approach.

Reinhard.

[Uri Blass]

Hi Reinhard,

It seems that I do not understand you or maybe you misunderstood me.

The way that I use is a safe way to detect repetition unless different positions get the same 64 bit key in the hash.

I did not talk about when I use it inside the search but I need some function to decide if to detect repetition of previous position.

I use it also not inside the search when I want to detect 3-fold repetition and this reason is the only reason that it does not return only 0 or 1.

I did not talk about when I use it but if you are interested in it
I use it inside the search after every move that I make in the search(no reason to expand a move if there is repetition and I simply return 0 without storing the information in the hash).

I can think of possible problems of wrong scores in the hash for programs that use hash for pruning but today movei still does not use hash for pruning but only for order of moves(I have my reasons to do it because there are ideas that I have that contradict using hash for pruning and I prefer to do things in my way).

Movei is different than other programs at similiar level by the fact that it does not use hash for pruning and I believe that a big improvement is possible even without doing it.

I believe that using hash for pruning can be productive also for movei but I also believe that designing it is not a simple task and I prefer to give priority to other tasks.

Uri

[Reinhard Scharnagl]

Hi Uri,

The way that I use is a safe way to detect repetition unless different positions get the same 64 bit key in the hash.

your method would be save if you would be stepping back only one step instead of two (in the case you would use Nullmove prunings). Or am I in error with that? P.S.: I think it could work, if positions were differently encoded when different colors are to move. P.P.S.: you are not detecting double nullmove repetitions only because of starting 4 plies above?

Movei is different than other programs at similiar level by the fact that it does not use hash for pruning

well I mainly have spoken to use cache entries for the detecting of position repetitions. On the other side why should one store anything into the cache when you would not trust that results gained sometimes with immense effort.

Reinhard.

[Uri Blass]

Hi Reinhard.
1)positions have different hash entry when it is different side to move so this is no problem.

2)I do not like using hash for pruning not because I do not trust positions with the same hash key to be the same.

The point is that I do not trust the score of searching to the same depth to be the same so it may be better to search to get better information.

Note that the last sentence is mainly for movei because of my ideas and less correct for other programs.

Uri

[Uri Blass]

Hi Uri,

P.S.: I think it could work, if positions were differently encoded when different colors are to move. P.P.S.: you are not detecting double nullmove repetitions only because of starting 4 plies above?
Reinhard.

If it was not clear from my last post
positions are differently encoded with different colors and
for your question I do not do double null move in the search.
If I add the possibility to do it then it means that I will not detect repetition after 2 null moves.

Uri

[Reinhard Scharnagl]

Hi Uri,

1)positions have different hash entry when it is different side to move so this is no problem.

then you must be able to also detect nullmove repetitions, or what is the point which is against that?

2)I do not like using hash for pruning not because I do not trust positions with the same hash key to be the same.

there has been another thread at a cached Perft, where I have tried to point out, that you have chances to have better distributed keys. Though I do not believe in suche heavy hash collisions, the consequences might nevertheless to be tolerated, especially if you only take evaluation values and not stored moves without being regenerated (such do I).

Reinhard.

[Uri Blass]

Hi Reinhard,

Hi Uri,

1)positions have different hash entry when it is different side to move so this is no problem.

then you must be able to also detect nullmove repetitions, or what is the point which is against that?

I may only if I change my repetition detection function to start from 2 plies before the board position but detecting a repetition after 2 null moves seems to be wrong and it seems to be better simply not to search it.

2)I do not like using hash for pruning not because I do not trust positions with the same hash key to be the same.

there has been another thread at a cached Perft, where I have tried to point out, that you have chances to have better distributed keys. Though I do not believe in suche heavy hash collisions, the consequences might nevertheless to be tolerated, especially if you only take evaluation values and not stored moves without being regenerated (such do I).

Reinhard.

I know and having perft with hash is on my list of things to do.

Uri

[Reinhard Scharnagl]

Hi Uri,

but detecting a repetition after 2 null moves seems to be wrong

why that? It could be that this is the point of may be different opinions.

Reinhard.

[Uri Blass]

Hi Reinhard,If I detect repetition after 2 null moves I may miss threats.

The idea of null move is to prune moves that have no threats.
If you evaluate null null as draw then it means that you do not prune no threats of the inferior side.


suppose white is inferior.

White play a move 1.xx that threats nothing.

What null move is going to say.

1.xx null 2.null draw.

meaning 1.xx null is at least a draw for white(white can draw by null and maybe can do better than it)
It means that 1.xx threats a draw and is not going to be pruned.

Uri

[Reinhard Scharnagl]

Hi Uri,

may be I am in error. But as far as I understood (and temporarily implemented) pruning would only occur then, when the evaluated nullmove would be better than beta. How could that be for both identical positions (only different by the right to move)? Moreover I am subtracting in my evaluation a special value to compensate the right to move.

Finally that situation would look like that no one of both could make any progress, thus validating it as zero would be consequent. Nevertheless this value is part of a node evaluation above the two nullmove nodes. And if there would be no possibility to find a better evaluated move, having zero would be fine.

Over all I see, that - which way you ever will do - nullmove pruning will influence the outcoming evaluation to be less precise (keeping the original depth). So why should that detail above be that important? It is much more important to make ANY position repetitions impossible, especially when you are using a cache to store results. Otherwise a node above could not prevent his working data to be overwritten. One of both nodes will kill the information provided by the other. That seems to be a real problem and should be avoided.

I actually do not trust the nullmove pruning idea. Maybe that will change (but only when there would be an improvement using it).

Reinhard.

[Uri Blass]

Hi Reinhard,
I will try to explain better
Let take your code

// Null Move pruning
if ( !GetPlySchach() && anz > 3
&& beta > -MATTWERT && beta < MATTWERT
) {
TZug zg;
zg.FullMove = 0;
DoMove(zg);
// bleibt der Gegner unter der beta-Schwelle?
value = -Bewerte(-beta, 1-beta, tiefe-2);
ReMove();
// ggf. Pruning
if (value >= beta) {
// Remis-Wert nicht ?ndern
if (value)
value = beta;
pL->E.minVal = value;
goto _EndeAktiv;
}
}

supoose beta>0 in the above code

My claim is that if you allow 2 null move in a row you have always
Bewerte(-beta, 1-beta, tiefe-2)>=0 in that code

The reason is that null null is evaluated as a draw by repetition and
other moves after null can only increase the score.
it means value<=0 and it means that you do not prune.

I think that it is not logical not to prune in that case because the inferior side can have bad move that threats nothing.

Uri

[Alessandro Scotti]

But as far as I understood (and temporarily implemented) pruning would only occur then, when the evaluated nullmove would be better than beta. How could that be for both identical positions (only different by the right to move)? Moreover I am subtracting in my evaluation a special value to compensate the right to move.

It could be possible because those positions are searched with different depths (in case of consecutive null moves).

[Dan Honeycutt]

If I detect repetition after 2 null moves ...

Reset the half move clock after a null move.

Dan H.

[Uri Blass]

If I detect repetition after 2 null moves ...

Reset the half move clock after a null move.

Dan H.

I do not understand what you suggest.

I do not do 2 null moves in a row.
I also do not look for at position that was 2 plies before the board position
in searching for repetition.

I only claim that allowing 2 null moves in a row and detecting repetition after them is illogical and may lead
to not pruning by null move pruning in a lot of cases.

Uri

[Dan Honeycutt]

I do not understand what you suggest.

I look back in the tree for repetition detection, but only as far back as the half move clock. By treating a null move like a capture or pawn advance it never leads to a draw.

Dan H.

[Uri Blass]

I understand now that you mean number of moves with no conversion by half move clock but I see no problem with treating null move as no conversion.

I also do not look back more than the number of moves with no conversion but I use the variable fifty for what you call the half move clock.

I admit that the name fifty is not the best name for that variable but I decided about it because it is connected to the fifty move rule.

Uri

[Sune Fischer]

2 null moves in a row cause repetition of a previous position and it seems not logical to analyze position that you already visited again.

Actually this isn't detected as a repetition draw in my code, the first repetition must be 4 moves back. Otherwise you are right that it would not make much sense to do if beta<0.

In principle even a single nullmove could trigger a draw score, because if my opponent believes his best choice is not to make any moves, then how is he ever going to mate me?

It also seems to me that you do effectively search two nullmoves in a row, compare e.g. the lines:
A) null Ng1f3 null Nf3g1 and
B) null null

Not much difference except the remaining depth.
Now since the object here is to see if the first nullmove is a FH or a FL, if A works then so should B. If you don't want to search B for principle reasons then you should try and avoid searching A too, imho.

Of course I've always used nullmove this way, so perhaps it works better because everything else has been developed around it, if you know what I mean.
The engine is becomming so complex it hard to figure out why it actually works, but whatever the reason it seems slightly better (10 Elo?) I'd say, at least in my engine.

-S.

[Dan Honeycutt]

I see no problem with treating null move as no conversion.

I don't either. I assume by "conversion" you mean a non-reversible move where you set your fifty variable back to zero. Treating the null move as a conversion avoids having the null move lead to a draw by repetition but it might tell you that you are not close to a 50 move draw when you really are. In this case treating it as a no conversion might be better.

Dan H.