table indexing Math help

Programming Topics (Computer Chess) and technical aspects as test techniques, book building, program tuning etc

Moderator: Andres Valverde

table indexing Math help

Postby Daniel Shawul » 08 Oct 2008, 14:15

To reduce size of bitbases like kPPPk with similar pieces, we can enumerate
all the unique legal placements of the simiar pieces. This way we can get
a saving of k!. For two similar pieces the required table size is reasonably small
however for 3 pieces or more it takes more than 1mb. I guess most just incorporte
this tables directly for simplicity.
This enumerations can be avoided by use of polynomial functions.
I found an explanation,with which I have difficulty to understand, given in a paper.
I am wondering if some one can help me with the math used to generate the polynomial
with 3 or more like pieces. Here is the explanation

Code: Select all
4.10.2 Enumeration of k like pieces

This is somewhat simpler as there is no symmetry involved. As I have restricted
my focus on endgames with at most 5 pieces of which 2 are kings, we will only
need enumerations of 2 and 3 pieces. The definitions below are only valid when
the input are sorted in ascending order. I.e. i<j => si < sj is assumed in the
definition of Xn.
Intuitively, a sequence of the positions of k like pieces (s1, . . . , sk) is mapped to
the index in the lexicographic sorting of {(s1, . . . , sk) | i < j ⇒ 0 ≤ si < sj < 64}
X2(s1, s2) = {


            X2(s1, s2 − 1) + 1 if s1 + 1 < s2
            X2(s1 − 1, 63) + 1 if s1 > 0 ∧ s1 + 1 = s2
                 0 if s1 = 0∧ s2 = 1
}
X3(s1, s2, s3) = {

            X3(s1, s2, s3 − 1) + 1 if s2 +1 < s3
            X3(s1, s2 − 1, 63) + 1 if s1 +1 < s2 ∧ s2 + 1 = s3
            X3(s1 − 1, 62, 63) + 1 if s1 > 0 ∧ s1 + 2 = s3
                 0 if s1 = 0∧ s2 = 1∧ s3 = 2
}
It is possible to express the above enumerations as polynomials by counting the
number of ways to place the pieces in a lexicographically smaller way. I will
show how to do this for X2(s1, s2), and simply state the result for the other
enumerations.

X2(s1, s2) = [SUM s1−1 i=0 SUM 63 j=i+1 1] + [SUM s2−1 i=s1+1 1]
= [SUM s1−1 i=0 ((63 + 1) − (i + 1))] + [((s2 − 1) + 1) − (s1 + 1)]
= [SUM s1−1 i=0 (63 − i)] + [s2 − s1 − 1]
= [(((s1 − 1) + 1) − 0) ∗ 63 –SUM s1−1 i=0 i] + [s2 − s1 − 1]
= [63 ∗ s1 – 1 / 2 ((s1 − 1) ∗ ((s1 − 1) + 1))] + [s2 − s1 − 1]
= [ 1 / 2 (127 ∗ s1 – s1^2)] + [s2 − s1 − 1]
= 1/2(125 ∗ s1 − s1^2) + s2 – 1

X3(s1, s2, s3) = −64 + 1/6(11531 ∗ s1 − 186 ∗ s1^2 + s1^3) + 1/2(125 ∗ s2 − s2^2) + s3

In my implementation the piece enumerations for 2 like pieces are simply
tabulated. For 3 like pieces such a tabulation would use 1 MB of memory and
I decided to use the polynomial expression instead. The polynomial expression
can be written f1(s1)+f2(s2)+f3(s3) and can hence be optimised by tabulating
f1, f2 and f3.




SUM X Y == summation (from Y, to X)

I want to know how the polynomial for 3 like pieces is arrived. The given polynomial is
for (64 * 63 * 62) / 2 placements. I want to derive other similar polynomials for (62 * 60 * 61) / 2 placements etc.
These polynomials are used to map the set of squares (s1,s2,s3) in to a table index.
Another problem, how to do the inverse process? Ordinarily I use a table here as well, but the table is big in this case.

thanks,
Daniel
User avatar
Daniel Shawul
 
Posts: 366
Joined: 28 Sep 2004, 09:33
Location: Ethiopia

Re: table indexing Math help

Postby Harald Johnsen » 09 Oct 2008, 09:36

There is some hints here http://kirill-kryukov.com/chess/discuss ... 880#p38397 and following posts

HJ.
User avatar
Harald Johnsen
 
Posts: 43
Joined: 20 Aug 2007, 17:01
Location: France

Re: table indexing Math help

Postby Daniel Shawul » 09 Oct 2008, 15:12

want to know how the polynomial for 3 like pieces is arrived. The given polynomial is
for (64 * 63 * 62) / 2 placements. I want to derive other similar polynomials for (62 * 60 * 61) / 2 placements etc.
These polynomials are used to map the set of squares (s1,s2,s3) in to a table index.
Another problem, how to do the inverse process? Ordinarily I use a table here as well, but the table is big in this case.

There is a typo here. It should be (64 * 63 * 62) / 3!

The math required for 3 or more pieces becomes complicated to do it by hand.
Therofre I solved the problem with a combinatorial math package ( I used Maple's).
Here are the two statements needed to produce same result as the one given the paper.

Code: Select all
> simplify(eval(sum(sum(1,j=i+1..63),i=0..s1-1) + sum(1,i=s1+1..s2-1)));
                    125      1   2         
                    --- s1 - - s1  + s2 - 1
                     2       2             
> simplify(eval(sum(sum(sum(1,k=j+1..63),j=i+1..63),i=0..s1-1)  + sum(sum(1,j=i+1..63),i=s1+1..s2-1) + sum(1,j=s2+1..s3-1) ) );
      11531           2   1   3   125      1   2         
      ----- s1 - 31 s1  + - s1  + --- s2 - - s2  - 64 + s3
        6                 6        2       2             



The inverse map problem still remains unsolved though. This is necessary to loop through
only the unique positions only during bitbase generation.
There is some hints here http://kirill-kryukov.com/chess/discuss ... 880#p38397 and following posts

Thanks I will look into it.
regards,
Daniel
User avatar
Daniel Shawul
 
Posts: 366
Joined: 28 Sep 2004, 09:33
Location: Ethiopia

Re: table indexing Math help

Postby H.G.Muller » 09 Oct 2008, 16:48

I think that to loop through it, you simply have to restrict the order of the pieces, by making the uper limit of one piece equal to another:

Code: Select all
index = 0;
for(s1=0; s1<64; s1++) {
   for(s2=0; s2<s1; s2++) {
       for(s3=0; s3<s2; s3++) {
           // element (s1,s2,s3)  now found at index
           index++;
       }
   }
}

If there are already other pieces on the board, you would have to lower the 64 loop end to the number of empty squares, but you would have to increment the s1, s2 and s3 that are >= an occupied square.
User avatar
H.G.Muller
 
Posts: 3453
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: table indexing Math help

Postby Daniel Shawul » 09 Oct 2008, 17:55

I used that for my old bitbase generation code. That was ugly since I had to write separte
code for different number of pieces. Now I am trying to write a common generator for N piece endgame.
So the loop is just
Code: Select all
           for(i = 0;i < size;i++) {
                get(i)
           }

get(i) is a function to get legal placement of pieces. So far I was able to get
location of squares by division and taking the remainder to detemine the placement for
the next piece. For pieces bundled together like the two kings (KK_WP with pawns, KK without pawns)
and bundled similar pieces(KBBKNN), I do a direct look up of primary locations of squares.

Then I sort all squares and adjust the locations (increment square if there are prior squares
before the current square). This is what my code looks like

Code: Select all
bool ENUMERATOR::get(UBMP32 j) {
   UBMP32 temp;
   
   /*determine primary locations of kings/pieces/pawns*/
    int i,squares,ispawn;
   temp = j / divisor[1];
   if(n_pawn) squares = KK_WP_square[temp];
   else squares = KK_square[temp];
   square[0] = (squares >> 6) & 63;
   square[1] = (squares & 63);
   j -= (temp * divisor[1]);
   
   for(i = 2;i < n_piece; i++) {
      ispawn = (PIECE(piece[i]) == pawn);
      
      if(index[i] == 1) {
         /*three similar pieces*/
         if(index[i + 1] == 1) {
            temp = j / divisor[i + 2];
            if(ispawn) {
               squares = SSS_square[1][i - 2][temp];
               square[i] = SQ4864((squares >> 12) & 63);
               square[i + 1] = SQ4864((squares >> 6) & 63);
               square[i + 2] = SQ4864(squares & 63);
            } else {
               squares = SSS_square[0][i - 2][temp];
               square[i] = ((squares >> 12) & 63);
               square[i + 1] = ((squares >> 6) & 63);
               square[i + 2] = (squares & 63);
            }
            j -= (temp * divisor[i + 2]);
            i+=2;
         /*two  similar pieces*/
         } else {
            temp = j / divisor[i + 1];
            if(ispawn) {
               squares = SS_square[1][i - 2][temp];
               square[i] = SQ4864((squares >> 6) & 63);
               square[i + 1] = SQ4864(squares & 63);
            } else {
               squares = SS_square[0][i - 2][temp];
               square[i] = ((squares >> 6) & 63);
               square[i + 1] = (squares & 63);
            }
            j -= (temp * divisor[i + 1]);
            i++;
         }
      } else {
         /*single piece*/
         temp = j / divisor[i];
         if(ispawn) square[i] = SQ4864(temp);
         else square[i] = temp;
         j -= (temp * divisor[i]);
      }
   }

   /*legal placement based on other piece location*/
   for(i = 2;i < n_piece; i++) {

      /*start and finish*/
      int k,l,start,finish,temp;
      ispawn = (PIECE(piece[i]) == pawn);
      start = (ispawn ? 2 : 0);
      finish = i;
      if(i > 2 && index[i - 1] == 1) {
         finish = i - 1;
         if(i > 3 && index[i - 2] == 1) {
            finish = i - 2;
            if(i > 4 && index[i - 3] == 1) {
               finish = i - 3;
               if(i > 5 && index[i - 4] == 1) {
                  finish = i - 4;
               }
            }
         }
      }
        /*legal placement of pieces/pawns*/
      for(k = start; k < finish;k++)
         t_square[k] = square[k];
      for(k = start; k < finish;k++) {
         for(l = k + 1; l < finish;l++) {
            if(t_square[k] > t_square[l]) {
                    temp = t_square[k];
               t_square[k] = t_square[l];
               t_square[l] = temp;
            }
         }
      }
      for(k = start; k < finish;k++) {
         if(square[i] >= t_square[k]) square[i]++;
         else break;
      }
      /*illegal pawn placement on kings' square*/
      if(ispawn) {
         if(square[i] == square[0] || square[i] == square[1]) {
            return false;
         }
      }
      
   }
   return true;
}

Ofcourse I wouldn't need this if I just looped over all positions, and did not want a better looking generator code :)
I guess that this is more important for checkers egbbs where you have only two piece types (King,pawn) with
big repetitions resulting in big k!. I wonder if they still use the table approach?

Daniel
User avatar
Daniel Shawul
 
Posts: 366
Joined: 28 Sep 2004, 09:33
Location: Ethiopia

Re: table indexing Math help

Postby Dann Corbit » 09 Oct 2008, 20:53

Why not create a perfect hash for every legal board position?
You just give the hash function the board position and it spits back the slot.

Create a distinct perfect hash function for every tablebase you want to generate.
Dann Corbit
 

Re: table indexing Math help

Postby Daniel Shawul » 10 Oct 2008, 06:03

It looks like another polynomial mapping is possible and with a way to do the inverse map as well! It is given in the link Harald provided. I will quote the relevant part of the post made by syzygy (who is he?)

Code: Select all

Re: Explanation of the unique triangle
by syzygy on Wed Sep 10, 2008 5:56 am

For kqqk, the indexing scheme can be further improved by removing the symmetry caused by having two pieces of the same kind. Each positions k,q1,q2,k corresponds to a position k,q2,q1,k.

To get rid of this symmetry, only index pairs q1,q2 (of squares) with 0 <= q1 < q2 <= 63. There are 64*64 pairs (q1, q2) (and 64*63 pairs with q1!=q2) but only 64*63/2 pairs (q1, q2) with q1 < q2.

There is an easy way to map pairs of integers 0 <= q1 < q2 to an integer 0 <= Q < 64*63/2:

Q = q1 + q2*(q2-1)/2.

Going back from Q to (q1, q2) is as simple as:
CODE: SELECT ALL
q1 = Q;
q2 = 1;
while (q1 >= q2) {
  q1 -= q2;
  q2++;
}


More generally, three queens (of same colour) can be indexed by

Q = q1 + q2*(q2-1)/2 + q3*(q3-1)*(q3-2)/6

and so on.


And the proof
Code: Select all
 came up with it myself, so I don't have a reference. The general formula works with binomials.

Let Bin(n, k) denote the number of k-element subsets of an n-element set. Then Bin(n, k) = n! / (k! x (n-k)!). This is basic combinatorics. The values Bin(n, k) are the values in Pascal's triangle.

To index sequences 0 <= x_1 < x_2 < ... < x_k < N = 64, take

Idx(x_1, x_2, ..., x_k) = Bin(x_1, 1) + Bin(x_2, 2) + ... + Bin(x_k, k).

The nice thing is that this doesn't depend on N, so you can use the same formula if N = 62 (e.g. two white queens on a chess board with two kings and the empty squares numbered 0, 1, 2, ..,. 61).

To prove that the formula works, we will show that Idx(x_1, ..., x_k) maps sequences 0 <= x_1 < x_2 < ... < x_k = N one-to-one to the range 0, 1, 2, ..., Bin(N, k) - 1. We use induction on k.

For k=1, this is trivial: Idx(x_1) = x_1 maps 0 <= x_1 < N to the range 0, 1, ..., N-1 = Bin(N, 1) - 1.

For k > 1, let M = x_k. Look at sequences 0 <= x_1 < x_2 < ... < x_(k-1) < M. These sequences correspond to (k-1)-element subsets of {0, 1, 2, ..., M-1}, so there are Bin(M, k-1) such sequences.

By the induction hypothesis, we know that Idx(x_1, ..., x_(k-1)) maps these sequences one-to-one to the range 0, 1, ..., Bin(M, k-1) - 1.

Therefore Idx(x_1, ..., x_(k-1), M) = Idx(x_1, ..., x_(k-1)) + Bin(M, k) maps these sequences one-to-one to the range Bin(M, k), ..., Bin(M, k-1) + Bin(M, k) - 1 = Bin(M+1, k) - 1. (To see that Bin(M, k-1) + Bin(M, k) = Bin(M+1, k), either look at Pascal's triangle, or observe that there are Bin(M, k-1) k-element subsets of 0, 1, ..., M that include 0 and Bin(M, k) k-element subset of 0, 1, ..., M that do not include 0.)

So Idx(x_1, ..., x_k) with x_k = M fixed, maps 0 <= x_1 < ... < x_(k_1) < M one-to-one to the range Bin(M, k), ... , Bin(M+1, k) - 1.

Observe that these ranges for varying M are exactly adjacent. So as M runs from k-1 (which is the minimum possible value of x_k) to N-1, Idx(x_1, ..., x_k) maps sequences 0 <= x_1 < x_2 < ... < x_k < N one-to-one to the range Bin(k-1, k) = 0, 1, 2, ..., Bin(N, k) - 1. Q.E.D.
User avatar
Daniel Shawul
 
Posts: 366
Joined: 28 Sep 2004, 09:33
Location: Ethiopia


Return to Programming and Technical Discussions

Who is online

Users browsing this forum: No registered users and 39 guests