Winboard Forum

I wonder if there is a correct statistics about tablebases positions.

I do not like the files that I got from tony werten that seems to consider every possible symmetry about tablebases positions so I have only the number of tablebases position in the table and not the number of possible positions when I do not consider symmetric positions as the same.

In kpk it was easy to get the right number by assuming that the pawn is only in file a-d but in knk kbk it is not so easy and I see no simple calculation like dividing by 2 to get the correct number.

I want to debug my new fen function and my tablebases functions and I need statistics about all correct not broken tablebases positions (one of the reasons is to see if I reject only the positions when the king of the side not to move is under threat and another reason to see if I have no error in getting the score when I probe the tablebases).

Uri

Eugene Nalimov posted the statistics data for all of his files as they were published.

The statistics files have the extension: .tbs

Here are the files:
ftp://cap.connx.com/pub/chess-engines/n ... ch/TBS.ZIP

Here is what knk looks like:
E:\programme\winboard\Nalimov>type knk.tbs
wtm: Draws: 26282
btm: Draws: 28644

Here is what krkr.tbs looks like:
E:\programme\winboard\Nalimov>type krkr.tbs
wtm: Lost in 0: 609
wtm: Lost in 1: 300
wtm: Lost in 4: 6
wtm: Lost in 5: 24
wtm: Lost in 6: 30
wtm: Lost in 7: 46
wtm: Lost in 8: 41
wtm: Lost in 9: 172
wtm: Lost in 10: 177
wtm: Lost in 11: 323
wtm: Lost in 12: 600
wtm: Lost in 13: 1151
wtm: Lost in 14: 1745
wtm: Lost in 15: 2190
wtm: Lost in 16: 1376
wtm: Lost in 17: 385
wtm: Lost in 18: 81
wtm: Lost in 19: 20
wtm: Draws: 968077
wtm: Mate in 19: 148
wtm: Mate in 18: 1049
wtm: Mate in 17: 8419
wtm: Mate in 16: 32482
wtm: Mate in 15: 65453
wtm: Mate in 14: 63380
wtm: Mate in 13: 56130
wtm: Mate in 12: 44995
wtm: Mate in 11: 32073
wtm: Mate in 10: 28161
wtm: Mate in 9: 23575
wtm: Mate in 8: 11640
wtm: Mate in 7: 9576
wtm: Mate in 6: 8129
wtm: Mate in 5: 3204
wtm: Mate in 4: 1416
wtm: Mate in 3: 3648
wtm: Mate in 2: 3315
wtm: Mate in 1: 4490
wtm: Broken positions: 270560
btm: Lost in 0: 609
btm: Lost in 1: 300
btm: Lost in 4: 6
btm: Lost in 5: 24
btm: Lost in 6: 30
btm: Lost in 7: 46
btm: Lost in 8: 41
btm: Lost in 9: 172
btm: Lost in 10: 177
btm: Lost in 11: 323
btm: Lost in 12: 600
btm: Lost in 13: 1151
btm: Lost in 14: 1745
btm: Lost in 15: 2190
btm: Lost in 16: 1376
btm: Lost in 17: 385
btm: Lost in 18: 81
btm: Lost in 19: 20
btm: Draws: 968077
btm: Mate in 19: 148
btm: Mate in 18: 1049
btm: Mate in 17: 8419
btm: Mate in 16: 32482
btm: Mate in 15: 65453
btm: Mate in 14: 63380
btm: Mate in 13: 56130
btm: Mate in 12: 44995
btm: Mate in 11: 32073
btm: Mate in 10: 28161
btm: Mate in 9: 23575
btm: Mate in 8: 11640
btm: Mate in 7: 9576
btm: Mate in 6: 8129
btm: Mate in 5: 3204
btm: Mate in 4: 1416
btm: Mate in 3: 3648
btm: Mate in 2: 3315
btm: Mate in 1: 4490
btm: Broken positions: 270560

Is that what you need or do you need something else?

Hi Dan,

these are the files I sent to Uri, so that's not what he wants.

Hi Uri,

I'm afraid you have to generate these yourself. I needed stuf like that to. fe KQkbb: how often do the 2b win without capturing an undefended queen (my qsearch will pick that up) and without 2 bishop of the same color etc.

The idea was to get a "normal" percentage with wich I could give some fuzzy evaluation adjustment.

Tony

Uri Blass wrote:I wonder if there is a correct statistics about tablebases positions.

I do not like the files that I got from tony werten that seems to consider every possible symmetry about tablebases positions so I have only the number of tablebases position in the table and not the number of possible positions when I do not consider symmetric positions as the same.

In kpk it was easy to get the right number by assuming that the pawn is only in file a-d but in knk kbk it is not so easy and I see no simple calculation like dividing by 2 to get the correct number.

I want to debug my new fen function and my tablebases functions and I need statistics about all correct not broken tablebases positions (one of the reasons is to see if I reject only the positions when the king of the side not to move is under threat and another reason to see if I have no error in getting the score when I probe the tablebases).

Uri

I could probably produce these for the 3 piece tablebases... but not from the Nalimov ones... either from the Edwards ones or from the ones GLC now generates.

The 4 piece tablebases would also be possible from the Edwards bases (GLC's 4 piece tb's aren't debugged yet).

Would you like me to do this and send you the info Uri?

Cheers, Tim.

yes and thanks in advance

I hope that the Edwards tablebases include correct information about the distance to mate and also about cases that the weaker side wins.

I remember that old Fritz that used some old tablebases(I think thompson's tablebases) had problems in some endgames because the tablebases included information only for cases that the strongest side wins.

Uri

please send your mail to uriblass@hmail.com

replace hmail by gmail.

I can get emails faster in that email and I decided never to post the
exact email directly because I read in the past about people who use that tactics to protect themselves from garbage emails.

Uri

Uri Blass wrote:yes and thanks in advance

I hope that the Edwards tablebases include correct information about the distance to mate and also about cases that the weaker side wins.

I remember that old Fritz that used some old tablebases(I think thompson's tablebases) had problems in some endgames because the tablebases included information only for cases that the strongest side wins.

Uri

No, the Edwards ones should be OK. We'll see. I have to go out soon, but I'll try to do it tonight.

Cheers, Tim.

I already verified that the 3 pieces are correct and I think that I have correct code to count position in the nalimov tablebases by simply count twice positions that both kings on the same main diagnol and later divide by 8.

for KPK I simply need to count all positions and divide by 2
I am not sure if it solves the general problem for more pieces at this monent.

Uri

Uri Blass wrote:I already verified that the 3 pieces are correct and I think that I have correct code to count position in the nalimov tablebases by simply count twice positions that both kings on the same main diagnol and later divide by 8.

This may not be the case if all the pieces are on the main diagonal.


Anyhow, I've emailed you the 3 piece some stats for you to check if they are OK (I may have a bug in my stats generation code).

I've started running through the 4 piece TBs, but it's going to take a while as it takes about 8 mins for each table. So far I've done KNKN, KNNK WTM, KNNK BTM, KBKN WTM. I'm currently running KBKN BTM. I'll continue with KBKB, KBBK WTM and KBBK BTM tonight, and e-mail you the stats. Tomorrow I'll try to get the rest of the 4-piece TBs done.

Cheers, Tim.

timfoden wrote:

Uri Blass wrote:I already verified that the 3 pieces are correct and I think that I have correct code to count position in the nalimov tablebases by simply count twice positions that both kings on the same main diagnol and later divide by 8.

This may not be the case if all the pieces are on the main diagonal.

Correct, Tim. Unless I misunderstand the Nalimov TBs, only the KK position is used for the symmetry. "White" K will be mirrored to a1-d1-d4 triangle, and if on the diagonal, black K will be mirrored to a1-h1-h8. Tis means, that there are two positions in the TB for Ks on a1, c3 and say rook on d2 once and b4 once. With all pieces on the diagonal. there is no additional symmetry operation possible.

BTW, Any idea for an indexing scheme, how this could be avoided (two positions for Rd2 vs. Rb4)? I don't see anything.

Cheers,
Dieter

Uri Blass wrote:for KPK I simply need to count all positions and divide by 2
I am not sure if it solves the general problem for more pieces at this monent.

For the TBs with pawns, this should be correct in the general case. Don't forget to count positions with ep possibility.

What you will not be able to count easily is the broken positions inside the TBs - but that is not interesting anyway. Positions with triple check (and other illegal positions) are inside the TBs and have distance to mate or draw.

Cheers,
Dieter

Dieter B?r?ner wrote:

Uri Blass wrote:for KPK I simply need to count all positions and divide by 2
I am not sure if it solves the general problem for more pieces at this monent.

For the TBs with pawns, this should be correct in the general case. Don't forget to count positions with ep possibility.

What you will not be able to count easily is the broken positions inside the TBs - but that is not interesting anyway. Positions with triple check (and other illegal positions) are inside the TBs and have distance to mate or draw.

Cheers,
Dieter

I am not sure about the ep possibility

I guess that it is relevant only when both opponents have pawns in the right squares.

should I count the following fen that is obviously illegal because last move of black could not be a7-a5?

8/k7/8/pP6/8/K7/8/8 w - a6 0 3


Uri

Dieter B?r?ner wrote:BTW, Any idea for an indexing scheme, how this could be avoided (two positions for Rd2 vs. Rb4)? I don't see anything.

Cheers,
Dieter

Well, I don't see any reason why it wouldn't be possible to use 3 pieces for the initial symmetry, in the same way as 2 kings are used. Say 2 kings + 1 piece. It would just be more difficult to calculate.

I guess Nalimov might be doing this. I don't really know, as I've not looked as the code deeply enough or recently enough.

For KK symmetry I have a 64 * 64 = 4096 entry table.
For KKx we would have 64 * 64 * 64 = 256K entries lookup table -- maybe a bit too big.
Or we could look up the kings to get 462, then have a 462 * 64 = 29K entries table -- I guess this is managable, without too much extra work to generate an index.

You could have different tables for the side to move who has the extra piece, and then remove contact checks from the index. -- This sounds similar to the Nalimov ones now!

Cheers, Tim.

Here is some statistics for
KQR vs KQ(stronger side to move) at the bottom of this post.

I hoped to find mate in 70 that I could translate without doubt to a draw because after conversion it is mate in 19 or shorter mate but
unfortunately the shorter mate is mate in 67 that still not clear that the stronger side does not win because the winning line may be

1.xx xx
...
50.xx first conversion(capture of the queen of the stronger side when the weaker side has no choice)
51.mate in 17 from KR vs KQ tablebases
...
67.mate

Alternatively it may be

1.xx xx
...
49.xx first conversion(capture of the queen of the stronger side when the weaker side has no choice to prevent mate in 1 or decisive conversion in the nexxt move)
50.mate in 18 from KR vs KQ tablebases
...
67.mate




I am not going to repeat the process for other tablebases because every 5 piece tablebases takes hours.

If I can get at least table of the maximal distance to mate in every configuration of pieces then it can be productive.

Another productive thing is to get table of the maximal distance to mate for every configuration of pieces and pawn structure in case that the tablebases include pawns.

Is it possible to get the data from the nalimov tablebases without doing a loop on all possible positions?

number of draws is 11718360
mated in 0 - 234256
mated in 1 - 128616
mated in 2 - 77336
mated in 3 - 45400
mated in 4 - 28136
mated in 5 - 15128
mated in 6 - 5056
mated in 7 - 1528
mated in 8 - 992
mated in 9 - 544
mated in 10 - 424
mated in 11 - 312
mated in 12 - 208
mated in 13 - 120
mated in 14 - 152
mated in 15 - 192
mated in 16 - 288
mated in 17 - 488
mated in 18 - 416
mated in 19 - 552
mated in 20 - 744
mated in 21 - 1032
mated in 22 - 1544
mated in 23 - 2024
mated in 24 - 3740
mated in 25 - 5720
mated in 26 - 10452
mated in 27 - 18144
mated in 28 - 29788
mated in 29 - 46640
mated in 30 - 68692
mated in 31 - 86412
mated in 32 - 93212
mated in 33 - 62116
mated in 34 - 33684
mated in 35 - 10800
mated in 36 - 576
mated in 37 - 88
mate in 67 - 24
mate in 66 - 16
mate in 65 - 88
mate in 64 - 152
mate in 63 - 168
mate in 62 - 248
mate in 61 - 256
mate in 60 - 376
mate in 59 - 264
mate in 58 - 248
mate in 57 - 144
mate in 56 - 176
mate in 55 - 312
mate in 54 - 464
mate in 53 - 728
mate in 52 - 968
mate in 51 - 1240
mate in 50 - 1576
mate in 49 - 2192
mate in 48 - 2720
mate in 47 - 3424
mate in 46 - 3024
mate in 45 - 3520
mate in 44 - 4512
mate in 43 - 4696
mate in 42 - 5584
mate in 41 - 6216
mate in 40 - 7464
mate in 39 - 9408
mate in 38 - 12608
mate in 37 - 16216
mate in 36 - 22064
mate in 35 - 26904
mate in 34 - 33416
mate in 33 - 42624
mate in 32 - 55228
mate in 31 - 70672
mate in 30 - 94720
mate in 29 - 127256
mate in 28 - 168132
mate in 27 - 225884
mate in 26 - 302000
mate in 25 - 402592
mate in 24 - 535612
mate in 23 - 706384
mate in 22 - 942608
mate in 21 - 1248888
mate in 20 - 1654356
mate in 19 - 2195828
mate in 18 - 2885800
mate in 17 - 3726548
mate in 16 - 4533860
mate in 15 - 5095116
mate in 14 - 5478396
mate in 13 - 6273764
mate in 12 - 7507804
mate in 11 - 9751804
mate in 10 - 12438120
mate in 9 - 15021276
mate in 8 - 17207460
mate in 7 - 19524656
mate in 6 - 28175848
mate in 5 - 66032080
mate in 4 - 90139200
mate in 3 - 56030596
mate in 2 - 33218192
mate in 1 - 20532432

>>BTW, Any idea for an indexing scheme, how this could be avoided (two
>>positions for Rd2 vs. Rb4)? I don't see anything.

>Well, I don't see any reason why it wouldn't be possible to use 3 pieces
>for the initial symmetry, in the same way as 2 kings are used. Say 2
>kings + 1 piece. It would just be more difficult to calculate.

Ok, yes. I meant without the use of rather large lookup tables.

> I guess Nalimov might be doing this.

I don't think so, although I do not understand most of his code. He uses some larger and complicated lookup tables, to weed out some of those "unblockable check" positions from the index space. I fear, this would not mix well with the 3 piece index tables.

Cheers,
Dieter

>Here is some statistics for
>KQR vs KQ(stronger side to move) at the bottom of this post.

>I hoped to find mate in 70 that I could translate without doubt to a draw
>because after conversion it is mate in 19 or shorter mate but
>unfortunately the shorter mate is mate in 67 that still not clear that the
>stronger side does not win because the winning line may be

There are many mates in Nalimov KQRKQ that are really draw. See my statistics, that is distance to conversion at the end of the post (actually to conversion or pawn move, but that is the same here). It also gives the maximum positions.

>I am not going to repeat the process for other tablebases because every
>5 piece tablebases takes hours.

Which process? Just to generate the statistics? If it took hours, your program is very inefficient. My program just generated kqrkq in 20 minutes. This included a final scan through all postions to generate the statistics. Statistics alone should not take more than a couple of minutes.

>If I can get at least table of the maximal distance to mate in every
>configuration of pieces then it can be productive.

I thought, you got the tbs files? Then you have the max distances. If you don't have all tbs files, I think you can download them from Dann. I also have all 3/4/5 men tbs and most 6men tbs, and could send them by mail.


>Another productive thing is to get table of the maximal distance to mate
>for every configuration of pieces and pawn structure in case that the
>tablebases include pawns.

>Is it possible to get the data from the nalimov tablebases without doing
>a loop on all possible positions?

For the pawn structure - no. You can decompress the tables with datacomp.exe and just read the bytes inside the file. I don't know the exact translation from the byte to mate in at the moment. But you would find out easily by comparing the counts for all byte values with a tbs file.

Regards,
Dieter

KQRKQ(wtm):

Win in 60 3 0.00001%
Win in 59 2 0.00000%
Win in 58 11 0.00002%
Win in 57 19 0.00003%
Win in 56 21 0.00004%
Win in 55 31 0.00006%
Win in 54 32 0.00006%
Win in 53 47 0.00009%
Win in 52 33 0.00006%
Win in 51 31 0.00006%
Win in 50 23 0.00004%
Win in 49 27 0.00005%
Win in 48 44 0.00008%
Win in 47 61 0.00011%
Win in 46 72 0.00013%
Win in 45 99 0.00018%
Win in 44 133 0.00024%
Win in 43 179 0.00033%
Win in 42 193 0.00035%
Win in 41 250 0.00046%
Win in 40 246 0.00045%
Win in 39 244 0.00045%
Win in 38 340 0.00062%
Win in 37 461 0.00085%
Win in 36 513 0.00094%
Win in 35 547 0.00101%
Win in 34 578 0.00106%
Win in 33 728 0.00134%
Win in 32 887 0.00163%
Win in 31 1126 0.00207%
Win in 30 1306 0.00240%
Win in 29 1558 0.00286%
Win in 28 2008 0.00369%
Win in 27 2660 0.00489%
Win in 26 3320 0.00610%
Win in 25 4059 0.00746%
Win in 24 4990 0.00917%
Win in 23 6397 0.01176%
Win in 22 8410 0.01546%
Win in 21 11721 0.02155%
Win in 20 16355 0.03006%
Win in 19 22360 0.04110%
Win in 18 30988 0.05696%
Win in 17 42474 0.07808%
Win in 16 58414 0.10738%
Win in 15 79922 0.14692%
Win in 14 108818 0.20003%
Win in 13 146312 0.26896%
Win in 12 197211 0.36252%
Win in 11 267047 0.49089%
Win in 10 367373 0.67532%
Win in 9 506518 0.93110%
Win in 8 655807 1.20552%
Win in 7 801335 1.47304%
Win in 6 1021855 1.87841%
Win in 5 1670904 3.07151%
Win in 4 3200331 5.88295%
Win in 3 4418467 8.12216%
Win in 2 7951435 14.61657%
Win in 1 31154303 57.26879%
Draw 8059 0.01481%
lost in 0 29893 0.05495%
lost in 1 78911 0.14506%
lost in 2 13579 0.02496%
lost in 3 6173 0.01135%
lost in 4 1773 0.00326%
lost in 5 526 0.00097%
lost in 6 129 0.00024%
lost in 7 16 0.00003%
lost in 8 1 0.00000%
Min. Draw 103246 0.18979%
Unknown 1386193 2.54814%
all draws 1497498 2.75275%
broken 50436902 92.71466%
brokeni 0 0.00000%

max win in 60
8/8/8/8/q7/6k1/8/KR5Q w - - 0 1
max loss in 8
8/4Q3/8/5q2/8/2k5/6R1/1K6 w - - 0 1


KQRKQ(btm):

Win in 9 2 0.00000%
Win in 8 23 0.00003%
Win in 7 282 0.00040%
Win in 6 1348 0.00193%
Win in 5 4202 0.00602%
Win in 4 20088 0.02880%
Win in 3 65628 0.09409%
Win in 2 524958 0.75263%
Win in 1 16424216 23.54734%
Draw 295877 0.42420%
lost in 0 608151 0.87190%
lost in 1 10282234 14.74160%
lost in 2 3028573 4.34205%
lost in 3 4159250 5.96310%
lost in 4 3983360 5.71093%
lost in 5 2936527 4.21009%
lost in 6 2315968 3.32040%
lost in 7 2074810 2.97465%
lost in 8 1825348 2.61700%
lost in 9 1510468 2.16555%
lost in 10 1205726 1.72865%
lost in 11 946967 1.35766%
lost in 12 734868 1.05358%
lost in 13 571123 0.81882%
lost in 14 434956 0.62359%
lost in 15 329362 0.47221%
lost in 16 243380 0.34893%
lost in 17 180749 0.25914%
lost in 18 134548 0.19290%
lost in 19 97410 0.13966%
lost in 20 71621 0.10268%
lost in 21 51746 0.07419%
lost in 22 38107 0.05463%
lost in 23 28055 0.04022%
lost in 24 22694 0.03254%
lost in 25 18041 0.02587%
lost in 26 14634 0.02098%
lost in 27 11067 0.01587%
lost in 28 8363 0.01199%
lost in 29 6287 0.00901%
lost in 30 5320 0.00763%
lost in 31 4459 0.00639%
lost in 32 3597 0.00516%
lost in 33 2808 0.00403%
lost in 34 2684 0.00385%
lost in 35 2641 0.00379%
lost in 36 2502 0.00359%
lost in 37 2201 0.00316%
lost in 38 1522 0.00218%
lost in 39 1029 0.00148%
lost in 40 938 0.00134%
lost in 41 916 0.00131%
lost in 42 767 0.00110%
lost in 43 650 0.00093%
lost in 44 398 0.00057%
lost in 45 279 0.00040%
lost in 46 188 0.00027%
lost in 47 215 0.00031%
lost in 48 205 0.00029%
lost in 49 146 0.00021%
lost in 50 156 0.00022%
lost in 51 207 0.00030%
lost in 52 165 0.00024%
lost in 53 185 0.00027%
lost in 54 143 0.00021%
lost in 55 140 0.00020%
lost in 56 88 0.00013%
lost in 57 83 0.00012%
lost in 58 44 0.00006%
lost in 59 20 0.00003%
lost in 60 31 0.00004%
Min. Draw 9889071 14.17793%
Unknown 4614947 6.61643%
all draws 14799895 21.21856%
broken 35087278 50.30451%
brokeni 0 0.00000%

max win in 9
2q5/4Q3/8/8/8/2k5/6R1/1K6 b - - 0 1
max loss in 60
8/8/8/8/8/6k1/8/KR1q3Q b - - 0 1

Dieter B?r?ner wrote:>Here is some statistics for
>KQR vs KQ(stronger side to move) at the bottom of this post.

>I hoped to find mate in 70 that I could translate without doubt to a draw
>because after conversion it is mate in 19 or shorter mate but
>unfortunately the shorter mate is mate in 67 that still not clear that the
>stronger side does not win because the winning line may be

There are many mates in Nalimov KQRKQ that are really draw. See my statistics, that is distance to conversion at the end of the post (actually to conversion or pawn move, but that is the same here). It also gives the maximum positions.

>I am not going to repeat the process for other tablebases because every
>5 piece tablebases takes hours.

Which process? Just to generate the statistics? If it took hours, your program is very inefficient. My program just generated kqrkq in 20 minutes. This included a final scan through all postions to generate the statistics. Statistics alone should not take more than a couple of minutes.

>If I can get at least table of the maximal distance to mate in every
>configuration of pieces then it can be productive.

I thought, you got the tbs files? Then you have the max distances. If you don't have all tbs files, I think you can download them from Dann. I also have all 3/4/5 men tbs and most 6men tbs, and could send them by mail.


>Another productive thing is to get table of the maximal distance to mate
>for every configuration of pieces and pawn structure in case that the
>tablebases include pawns.

>Is it possible to get the data from the nalimov tablebases without doing
>a loop on all possible positions?

For the pawn structure - no. You can decompress the tables with datacomp.exe and just read the bytes inside the file. I don't know the exact translation from the byte to mate in at the moment. But you would find out easily by comparing the counts for all byte values with a tbs file.

Regards,
Dieter

Some questions and comments:

1)Did you build special tablebases to calculate the distance to conversion?

2)I agree that my program was not efficient.

I thought that it is not efficient because it made a loop on all possible positions and so it may consider one position 4-8 times so it could take 20 minutes to calculate the statistics for KQR vs KQ(white to move) but I find that this is not all of the story.

I did not care about speed of my program to setup position
because I remembered that probing the nalimov tablebases is slow(otherwise people could have no problem with probing the tablebases at the root)

It turns out that I intialize a lot of arrays and it seems that it is the main reason that it is slow and not probing the nalimov tablebases.

Movei did something like 100,000 probes per second that
I did not consider slow but it turns out that 100.000 probes per second is very slow.

I am surprised because I expected tablebases to be significantly slower after reading in the past about cases when programs became weaker because of probing tablebases.

I wonder if there is an advantage in speed when the program probe the same tablebases again and again.

3)I got tb files only for 4 pieces and not for 5 pieces and at hyatt's site I found only statistics about 6 pieces.

I will see later if I can download 5 from Dan's site.

Uri

Uri:
> 1)Did you build special tablebases to calculate the distance to
> conversion?

Yes.

> 2)I agree that my program was not efficient.

> Movei did something like 100,000 probes per second that
> I did not consider slow but it turns out that 100.000 probes per second
> is very slow.

> I wonder if there is an advantage in speed when the program probe the
> same tablebases again and again.

Yes. It also depends, in which order you probe the TBs. Say, you have 5 nested loops to generate all 5-men positions. Something like:

Code: Select all

for (sq1=0; sq1<64; sq1++)
  for (sq2=0; sq2<64; sq2++)
    for (sq3=0; sq3<64; sq3++)
      for (sq4=0; sq4<64; sq4++)
        for (sq5=0; sq5<64; sq5++)
        {
           /* setup the five pieces on sq1-sq5,
               probe the position in the TB
               and collect statistics */
        }

I left out some things like checking for double occupied squares, side not to move in check, symmetry, check by king.

Above code snippet may run much faster, when you use sq1 and sq2 for the two kings, compared with using sq4 and sq5 for the Kings. This is because in the Nalimov scheme, all positions with the same KK squares are stored together. So the file reading and caching will be much more efficiently, when the kings are the outer loop. Basically, you will not need a disk access for most probes. In real play with large tables, probes will lie father apart and spread over many files, and caching will be less efficient.

About your ep-position. I don't know. I guess, you have to count the pos with the impossible ep target. I cannot see, how one could use the fact, that some squares must be empty in indexing.

Regards,
Dieter

Dieter B?r?ner wrote:>>BTW, Any idea for an indexing scheme, how this could be avoided (two
>>positions for Rd2 vs. Rb4)? I don't see anything.

>Well, I don't see any reason why it wouldn't be possible to use 3 pieces
>for the initial symmetry, in the same way as 2 kings are used. Say 2
>kings + 1 piece. It would just be more difficult to calculate.

Ok, yes. I meant without the use of rather large lookup tables.

Well, I think that this is still posible (no large lookup tables).

Out of the 462 possible king permutations, there are 21 with both kings on the main diagonal. So if we are to try to include a third piece, we have 2 cases:

1. One or both of the kings are not in the main diagonal. This gives 441 x 62 = 27342 possible positions. For this we need to map the 462 --> 441 positions (924 byte table), and adjust the position of the remaining piece due to the kings. We may even not need this extra lookup table if we organise the values in the main 4096 --> 462 mapping to map the off-main-diagonal positions first, followed by the on-main-diagonal ones.

2. Both kings are on the diagonal. We now look to see if the remaining piece is not in the large triangle, and if so, we adjust the folding to get it into the large triangle. There are now 34 possible positions of this extra piece, giving 21 x 34 = 714 extra positions, which is a simple lookup (in a 36 byte table, or in a 64 byte one that may already exist) + adjustments due to king positions.

This seems fairly quick and low in terms of lookup requirements.

Thus the index size would be 27342 + 714 = 28056.

The size of this table compares to the "normal" case of 462 x 62 = 28644, for a 2.05% reduction, which I guess is fairly insignificant. I guess much larger savings can be made by removing the contact check positions.

Cheers, Tim.

timfoden wrote:Well, I think that this is still posible (no large lookup tables).

Thanks Tim. I also agree about all the numbers you have given. I have to think of one thing still: Is one 36 byte table enough? Where I see a small problem at the moment, is in detecting which king squares are smaller than the piece square for correction of the piece square (which gives you the multiplier of 62 instead of 64 for the first piece).

First I also liked your word "contact check". Note that Nalimov also weeds out knight checks.

Cheers,
Dieter