request for correct statistics about nalimov tablebases

[Tim Foden]

Well, I think that this is still posible (no large lookup tables).

Thanks Tim. I also agree about all the numbers you have given. I have to think of one thing still: Is one 36 byte table enough? Where I see a small problem at the moment, is in detecting which king squares are smaller than the piece square for correction of the piece square (which gives you the multiplier of 62 instead of 64 for the first piece).

No, I think you are right, 36 bytes isn't enough It would have to be a 64 byte table, as we are mapping from 64 --> 36, to get an index in the large triangle.

Cheers, Tim.

[Uri Blass]

>Here is some statistics for

>I am not going to repeat the process for other tablebases because every
>5 piece tablebases takes hours.

Which process? Just to generate the statistics? If it took hours, your program is very inefficient. My program just generated kqrkq in 20 minutes. This included a final scan through all postions to generate the statistics. Statistics alone should not take more than a couple of minutes.

Regards,
Dieter

20 minutes mean 1200 seconds for almost 2^30 positions.
It means that your program probe almost 10^6 nodes per second.

I wonder if you check all relevant conditions for general fen or you have special function for the nalimov tablebases.

There are many conditions that you do not need to check when you do a loop in the nalimov tablebases like checking that you do not have 2 white kings or checking that you do not have 8 pawns and 3 knights(I calculate the number of promoted pieces and pawna and return error in case that the number of pawns and pronmoted pieces are at least 9(for example 3 white queens 3 white knights and 6 white pawns)

I wonder if you initialize some information of variables like
numblackknights in your code or you save only part of the information and in case that you need it for analysis you are able to calculate the rest of it.

[Anonymous]

No, I think you are right, 36 bytes isn't enough It would have to be a 64 byte table, as we are mapping from 64 --> 36, to get an index in the large triangle.

Actually, it can be done without additional lookup table besides the KK table (that, as you have mentioned, has to map the positions with both Ks on the long diagonal to a contigious index space - at the lower or the upper end). You can calculate the piece part of the index as

Code: Select all

/* everything already mirrored, so that wK is inside the a1-d1-d4
    triangle, bK in a1-h1-h4 when white wK on a1-h8.*/
if (both Ks on a1-h8-diagonal)
{
  if (sq_piece in a2-a8-f8 triangle)
    sq_piece = sq_piece mirrored on a1-h8;
  r = row(sq_piece);  /* Assuming row starts C-like from zero */
  piece_idx = sq_piece - (sq_piece > sq_white_king) + (sq_piece > sq_black_king);
  piece_idx -= (r * (r+1))/2. /* to avoid lookup table */
  tb_idx = (kk_idx[sq_white_king][sq_black_king] - 441)*34 + piece_idx
               + 441*62;
}
else
  tb_idx = kk_idx[sq_white_king][sq_black_king]*62
               + sq_piece - (sq_piece > sq_white_king)
               + (sq_piece > sq_black_king);   


And similar for TBs with more than 3 men.

Cheers,
Dieter

[Anonymous]

>Here is some statistics for

>I am not going to repeat the process for other tablebases because every
>5 piece tablebases takes hours.

Which process? Just to generate the statistics? If it took hours, your program is very inefficient. My program just generated kqrkq in 20 minutes. This included a final scan through all postions to generate the statistics. Statistics alone should not take more than a couple of minutes.

20 minutes mean 1200 seconds for almost 2^30 positions.
It means that your program probe almost 10^6 nodes per second.

I wonder if you check all relevant conditions for general fen or you have special function for the nalimov tablebases.

There are many conditions that you do not need to check when you do a loop in the nalimov tablebases like checking that you do not have 2 white kings or checking that you do not have 8 pawns and 3 knights(I calculate the number of promoted pieces and pawna and return error in case that the number of pawns and pronmoted pieces are at least 9(for example 3 white queens 3 white knights and 6 white pawns)

I wonder if you initialize some information of variables like
numblackknights in your code or you save only part of the information and in case that you need it for analysis you are able to calculate the rest of it.

Uri, you misunderstood. The 20 minutes was for generation of the TB from scratch, not just for collecting the statistics of already calculated Nalimov TBs. That takes typically in the order of magnitude of one minute here for a 5-men TB, when my "scanner" uses the symmetry.

Indeed, I will not setup all data structures for the position then, just enough, so that my incheck function works. Setting up the position is really little overhead. It looks inside the inner loop a bit like:

Code: Select all

  /* Inner loop for looping through all positions of one 3-2 material constell. */
  for (sq5=0; sq5<64; sq5++)
  {
    if (sq5==sq4 || sq5==sq3 || sq5==sq2 || sq5==sq1)
      continue;
    piece_list[black][1] = sq5; /* [black][0] is the king */
    color[sq5] = black;
    board[sq5] = piece3;
    if (!incheck(stm^1))
    {
      /* probe TB and collect statistics */
    }
    /* else broken position */
    board[sq5] = 0; /* Just reset the board for the last piece,
                                remaining stuff won't matter */
  }


Perhaps there is also another misunderstanding. Of course my probing will not go through FEN. It will use some much more efficient native structures. And things like number_of_knights do not change at all while scanning through all positions of one material constellation. So they are setup once outside the loops.

Cheers,
Dieter

[Uri Blass]

Hi Dieter,

Yes

There is a misunderstanding.

I wanted to check that I have no bugs in my probe tablebase code and also no bugs in my functions to check if fen is legal and I thought that running a loop on all tablebases positions can be a good idea(it helped me to discover a bug that I have in the function to check if a fen is legal).

I thought to use the same tool also to get statistics about tablebases.
It seems that you plan to use some new tablebases when I thought to use existing nalimov tablebases in a productive way and not to build new tablebases because I think that nalimov will always have better hardware than me to generate tablebases and I cannot compete with him so the question is how to use the information that he gives in a productive way.

I guess that you also have the KQR vs KR information all in the RAM of the computer at generation time and it may help to explain how you can generate them very fast(still I did not expect you to be able to generate them so fast and if I remember correctly only generating KQ vs KR nalimov tablebases took more time on slower hardware when the slower hardware was maybe 10 times slower than My A3000 but not more than it when generating the KQ vs KR should be something like being 60 times faster than generating the KQR vs KQ tablebases).

Uri