Winboard Forum

by **Fabien Letouzey** » 04 May 2004, 17:08

Geschrieben von:/Posted by: Fabien Letouzey at 04 May 2004 18:08:58:

Hello,
Fruit would like to participate in the next WBEC-Ridderkerk, but it
would need an opening book!
It will enter as a WinBoard engine, so BookThinker needs to be used.
I don't know anything about BookThinker or making books in general,
so I was just hoping to find somebody here

Is it a single file, or are there settings specific to BookThinker?
Anybody?
Fabien.
PS: Oh, and the deadline is on Thurday

by **Roger Brown** » 04 May 2004, 17:37

Geschrieben von:/Posted by: Roger Brown at 04 May 2004 18:37:12:
Als Antwort auf:/In reply to: book for Fruit? geschrieben von:/posted by: Fabien Letouzey at 04 May 2004 18:08:58:

Hello Fabien,
I do know that Leo has built other books for other non-book WB engines. Maybe you could consult with him. For a moderate fee I suppose he could help you with it.
I was joking about the moderate fee comment. Leo charges an exorbitant sum!
:-)

More importantly is the pgn base used to construct the book and your assessment of your engine's playing style. That should be a solid array of GM games and comp-comp games at long timecontrols.
Dann Corbit's ftp site is a good place to start.
There are a number of book-builders around who have built and tested BookThinker books. Mogens and Volker come to mind. They actually seem to know what they are talking about.
BookThinker produces a single file ending with dat. Should all else fail I would be willing to assist. Of course, you should pay very close attention to all else fail.
:-)

Later.

by **Volker Pittlik** » 04 May 2004, 18:09

Geschrieben von:/Posted by: Volker Pittlik at 04 May 2004 19:09:40:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Roger Brown at 04 May 2004 18:37:12:

...

Thank you very much, but don't forget Günther and Leo. There are most likely many others here.
My general recommendation for building books is: Avoid to have bad or dubious moves in it! Only take high rated human or long time computer games from reliable resources! (For example SSDF, Leo, Günther and other well known testers.) Exclude all blitz, blind, simul and so on stuff.
Add some "special lines" as gambits or strange moves as 1. Nh3 to be prepared the opponent plays it.
Check the PGN carefully with Scid, PGN_Extract and George Lyapko's PGN verifier to prevent having illegal moves in the PGN.
IIRC Bookthinker's input file is somehow limited (16 MB or so). If so keep that in mind.
I'm rather busy this days. If it is necessary I can generate a book for Fruit.
Regards
Volker

by **Uri Blass** » 04 May 2004, 22:19

Geschrieben von:/Posted by: Uri Blass at 04 May 2004 23:19:19:
Als Antwort auf:/In reply to: book for Fruit? geschrieben von:/posted by: Fabien Letouzey at 04 May 2004 18:08:58:

Why?
It can participate without book.
Movei started to play in wbec with no book.
I believe that I do not know more than you about making books.
The book of movei is clearly primitive book that does not know about tranpositions and has only small number of lines that were generated manually.
Movei also does not do an efficient search to find book moves and the time that it needs to find book moves is proportional to the size of the book but fortunately the book is very small so today is not a problem.
I plan to change it but it is not an easy task for me.
I find it unfair if other programmers are going to use bookthinker when I have to work hard only to have an inferior book(I probably need to work some hundreds of hours only to have book that knows about transpositions and is result of the same lines of movei's book including efficient search to find if a position is in book.

I think that only thinker and the commercial programs should be allowed to use bookthinker.
Thinker because the programmer already worked to write the relevant code and
the commercial programs because they have the problem of not being able to use their original book under winboard.
Uri

by **Lance Perkins** » 04 May 2004, 22:47

Geschrieben von:/Posted by: Lance Perkins at 04 May 2004 23:47:28:
Als Antwort auf:/In reply to: book for Fruit? geschrieben von:/posted by: Fabien Letouzey at 04 May 2004 18:08:58:

You can ask Leo to use the default book that comes with BookThinker. Its the same book that Thinker uses in Leo's tournament and elsewhere.
You can also ask Leo to use the custom book that he built which he is now using to play TheKing in the premier division. With this book, TheKing is currently in 1st place in that division. With the same book, TheKing finished 2nd place in the pervious edition of the premier division.
If you want your own book, you need to download and install ChessThinker. This comes with the MakeBook program which you will use to create the opening book for BookThinker.
The usage is simple: MakeBook PgnFile

by **Lance Perkins** » 04 May 2004, 23:30

Geschrieben von:/Posted by: Lance Perkins at 05 May 2004 00:30:43:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Volker Pittlik at 04 May 2004 19:09:40:

The current MakeBook limit is 1 million games. Assuming that each game is 200 bytes, that would translate to a 200 MB PGN file.
BookThinker itself can load any book file size, as long as the machine's memory permits so.

by **Dann Corbit** » 05 May 2004, 04:15

Geschrieben von:/Posted by: Dann Corbit at 05 May 2004 05:15:11:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 04 May 2004 23:47:28:

Your program has lots of problems with perfectly legal pgn files.
After stripping to the bare essentials, a few defects show up.
It does not understand underpromotion.
It does not know how to deal with an unfinished game. (*)
It does not understand quoting rules (e.g. this causes a problem):
[Black "CM 9000 [Utz12n], XP1800+ 64MB"]
The underpromotion thing makes it really broken. The others are easily fixed by a filter. I suspect that by making the book cut off early enough it will not get to the underpromotion rules.
Here is some sample output:
E:\PROGRA~3\winboard\thinker>parse < book.pgn |analyze |makebook
MakeThinkerBook 2.1a
usage : MakeThinkerBook [-max-plies] pgn-file
example: MakeThinkerBook -30 GM_Games.pgn
Error at line 3127, move 83(W): h8=N+
Error at line 4205, move 54(B): f1=B
Error at line 5027, move 25(B): *
Error at line 5406, move 1(W): ,
Error at line 8263, move 41(W): *
Error at line 9221, move 1(W): ,
Error at line 9695, move 74(W): h8=R
Error at line 9743, move 55(W): b8=B
Error at line 10904, move 74(W): f5#
Error at line 12590, move 72(B): c1=R
Error at line 12661, move 63(B): d1=N
Error at line 13907, move 1(W): ,
Error at line 13925, move 1(W): ,
Error at line 14130, move 100(W): e8=N+
Error at line 14342, move 1(W): ,
Error at line 14380, move 82(W): e8=B
Error at line 14450, move 46(B): c1=R
Error at line 14543, move 73(W): *
Error at line 14794, move 91(W): g8=N
Error at line 15218, move 90(B): h1=R
Error at line 15985, move 41(B): e5#
Error at line 16093, move 58(W): a8=R
Error at line 18883, move 50(W): g4#
Error at line 19473, move 81(W): d8=R
Error at line 20740, move 52(B): d1=R
Error at line 22167, move 63(W): h8=B
Error at line 23862, move 48(W): d8=N+
Error at line 25336, move 86(W): f8=N
Error at line 27400, move 67(B): *
Error at line 27803, move 73(B): *
Error at line 31054, move 56(W): g8=R+
Error at line 32422, move 119(W): d8=N+
Error at line 33853, move 18(B): *
Error at line 35374, move 74(B): e1=B
Error at line 36087, move 113(W): h8=R
Error at line 37990, move 64(B): h1=R
Error at line 38997, move 60(B): a1=R
Error at line 39090, move 107(W): b8=R
Error at line 39481, move 61(W): h4#
Error at line 39550, move 101(B): e1=R
Error at line 39713, move 82(W): b8=N+
Error at line 40934, move 57(B): d1=R+
Error at line 42721, move 55(W): h8=N
Error at line 42924, move 1(W): ,
Error at line 44369, move 109(B): d1=N+
Error at line 45756, move 52(W): g8=B+
Error at line 46248, move 88(B): g1=N+
Error at line 46297, move 1(W): ,
Error at line 47606, move 63(W): c8=R
Error at line 47728, move 56(W): a8=R
Error at line 48672, move 1(W): ,
Error at line 48951, move 65(W): c8=R+
Error at line 50108, move 107(B): g1=R
Error at line 52877, move 1(W): ,
Error at line 54608, move 65(W): h8=R
Error at line 57638, move 1(W): ,
Error at line 57981, move 45(W): a8=R
Error at line 60444, move 169(B): g1=R
Error at line 61681, move 53(B): e1=N+
Error at line 61885, move 1(W): ,
Error at line 63770, move 69(W): f8=N+
Error at line 64243, move 1(W): ,
Error at line 67587, move 76(B): h1=N+
Error at line 68653, move 1(W): ,
Error at line 68677, move 1(W): ,
Error at line 68726, move 1(W): ,
Error at line 68748, move 1(W): ,

my ftp site {remove http:// unless you like error messages}

by **Dann Corbit** » 05 May 2004, 04:32

Geschrieben von:/Posted by: Dann Corbit at 05 May 2004 05:32:04:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 04 May 2004 23:47:28:

input PGN file:
05/04/2004 08:23p 12,542,661 book.pgn
Creates this file after:
parse < book.pgn > book.out
05/04/2004 08:24p 8,419,219 book.out
And then:
analyze < book.out > book.ana produces this:
05/04/2004 08:24p 8,384,286 book.ana
And then makebook book.ana produces this:
05/04/2004 08:27p 121 Thinker.dat
I know you like things small, but that's a bit ridiculous.
IMO-YMMV.
;-)

my ftp site {remove http:// unless you like error messages}

by **Dann Corbit** » 05 May 2004, 04:39

Geschrieben von:/Posted by: Dann Corbit at 05 May 2004 05:39:22:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 04 May 2004 23:47:28:

I figured it out, finally. Thinker's book generator does not use the old support programs any longer. The current readme.txt does not reflect this.
Here is what I got using makebook directly on the PGN:
E:\PROGRA~3\winboard\thinker>makebook book.pgn
Error at line 216286, move 23(W): Rcc1
resulting in this file:
05/04/2004 08:34p 562,013 Thinker.dat
Here is the game that makebook did not like:
[Event "5moves 2004 Ath 1.3/64 90m+30s"]
[Site "Zurich SUI"]
[Date "2003.12.25"]
[Round "1.6"]
[White "Ruffian 2.0.0"]
[Black "Junior 8"]
[Result "0-1"]
1. d4 Nf6 2. c4 g6 3. Nc3 d5 4. Nf3 Bg7 5. cxd5 Nxd5 6. e4 Nxc3 7. bxc3
c5 8. Bb5+ Nc6 9. d5 a6 10. Ba4 b5 11. dxc6 bxa4 12. O-O Qb6 13. Qxa4
Bxc3 14. Rd1 O-O 15. Bh6 Bxa1 16. Bxf8 Kxf8 17. Rxa1 Bg4 18. Nd2 Qb2 19.
Nb3 Be2 20. c7 c4 21. Qd7 cxb3 22. c8=R+ Kg7 23. Rcc1 bxa2 24. h4 Rb8
25. Qxe7 Bg4 26. Qg5 Be6 27. h5 h6 28. Qg3 Rc8 29. Rf1 g5 30. Kh1 Rc2
31. f4 Bc4 32. Rfd1 Rd2 33. Rg1 Rd3 34. Qg4 Be6 35. f5 Bb3 36. Raf1 Kh7
37. Rc1 Re3 38. f6 Be6 39. Qd1 Rxe4 40. g4 Qa3 41. Qc2 Kh8 42. Rcd1 Bf5
0-1
And the position in question:
r1R5/3Qppkp/p5p1/8/4P3/1p6/Pq2bPPP/R5K1 w - - bm Rcc1;

my ftp site {remove http:// unless you like error messages}

by **Dann Corbit** » 05 May 2004, 04:45

Geschrieben von:/Posted by: Dann Corbit at 05 May 2004 05:45:32:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 04 May 2004 23:47:28:

Here is the book that I built.
ftp://cap.connx.com/pub/chess-engines/n ... hinker.dat
If the program does not keep track of any statistics, then I should make another one that is truncated at some move count instead.
If it keeps track of wins/losses/draws from a given point, it should be perfectly fine as is.

my ftp site {remove http:// unless you like error messages}

by **Lance Perkins** » 05 May 2004, 04:48

Geschrieben von:/Posted by: Lance Perkins at 05 May 2004 05:48:02:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Dann Corbit at 05 May 2004 05:15:11:

Those messages you are seeing is not from ChessThinker's MakeBook. They are either from 'parse' or 'analyze'. MakeBook does not read input from stdin. In your example, MakeBook terminated early with a 'usage' output.
In your example, you should just do: MakeBook book.pgn.
MakeBook should be able to handle nested/quoted "[]" and unfinished games.
As for underpromotion, there may be a problem there. I'm not sure yet. I'll investigate that.

by **Norm Pollock** » 05 May 2004, 05:36

Geschrieben von:/Posted by: Norm Pollock at 05 May 2004 06:36:41:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 05 May 2004 05:48:02:

I get similar messages from MakeBook book.pgn
If I could capture them I would post them here, but it would a very, very long post. There is something seriously wrong with MakeBook!

by **Fabien Letouzey** » 05 May 2004, 10:00

Geschrieben von:/Posted by: Fabien Letouzey at 05 May 2004 11:00:59:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Uri Blass at 04 May 2004 23:19:19:

Dont worry I don't really care about books, but I need a way to avoid repeted games due to the four-game matches.
I fully agree with your argument. I wouldn't even have thought about using BookThinker if Leo didn't mention it on his web site (UCI-only info link, I think).
Note that I have written book algos before (with book making, transpositions, etc ...). I will add that to PolyGlot when I have time, as in UCI design engines are not required to handle the book themselves. Therefore PolyGlot has the responsibility of transforming an UCI engine (just a searcher) into an xboard one (full chess player as in the FIDE rules).
We are talking about UCI engines here. I don't see in which way ChessBase engines are different in this respect.
---
I think we can easily start one of those useless xboard-vs-UCI threads from here. Instead I would prefer the tournament director of each competition to state his opinion/conditions clearly, so we can adapt accordingly.
Fabien.

by **Fabien Letouzey** » 05 May 2004, 10:24

Geschrieben von:/Posted by: Fabien Letouzey at 05 May 2004 11:24:28:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Lance Perkins at 04 May 2004 23:47:28:

That sounds fine to me (assuming some randomness). Is there a risk a super-long line is played if both engines are using this book?
There are tricky ethical issues, let's hear Leo about that. I think the first solution is just perfect for Fruit.
I don't have Windows, hence my post.
Fabien.

by **Uri Blass** » 05 May 2004, 10:32

Geschrieben von:/Posted by: Uri Blass at 05 May 2004 11:32:26:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Fabien Letouzey at 05 May 2004 11:00:59:

You can have a book only for the first move when you have only 1.e4 or 1.d4 with white and 2 replies for every legal move.
In the first tournament when movei needed to play 4 games I told it
to change the first move in every game so even after winning it could not repeat the same game.
Later I added learning that told it to change the first move only after a loss.

I talked about programs that have a book not under winboard(not only chessbase programs).
Chessmaster is not a chessbase engine but it is using a book when it does not run under winboard.
Shredder7.04 also can use a good book when it does not run under winboard.
In both cases I guess that using bookthinker is not a significant advantage.
Uri

by **Anthony Cozzie** » 05 May 2004, 14:02

Geschrieben von:/Posted by: Anthony Cozzie at 05 May 2004 15:02:47:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Fabien Letouzey at 05 May 2004 11:24:28:

Hooray for a fellow Linux user! Let's make this the Xboard forum

anthony

by **Tord Romstad** » 05 May 2004, 14:37

Geschrieben von:/Posted by: Tord Romstad at 05 May 2004 15:37:57:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Anthony Cozzie at 05 May 2004 15:02:47:

You are not the only one. You can add Tom Likens, Andrew Williams and myself
to the list of Linux and Xboard users. I'm sure there others as well. :-)

Tord

by **Lance Perkins** » 05 May 2004, 16:49

Geschrieben von:/Posted by: Lance Perkins at 05 May 2004 17:49:36:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Dann Corbit at 05 May 2004 05:45:32:

Win/loss info is stored in Thinker.dat, and BookThinker uses that when choosing the moves.

by **Lance Perkins** » 05 May 2004, 16:56

Geschrieben von:/Posted by: Lance Perkins at 05 May 2004 17:56:09:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Dann Corbit at 05 May 2004 05:39:22:

The problem is that MakeBook cannot recognize Rcc1. Had this been c8c1, it would have been fine.
I'm making the fix and will update the readme too.

by **Leo Dijksman** » 05 May 2004, 19:27

Geschrieben von:/Posted by: Leo Dijksman at 05 May 2004 20:27:46:
Als Antwort auf:/In reply to: Re: book for Fruit? geschrieben von:/posted by: Uri Blass at 04 May 2004 23:19:19:

I dont see a big problem if the "uci only" engines use BookThinker because they usually use a "GUI" book!?
Ofcourse none of the engines should use the Thinker engine book, not even the commercials, I have made the book for The King myself.
For the uci engines it looks to me as if the authors make a Thinker book thereself or send me a pgn file to make one is fine?
If there are more WB authors that dont like the idee please let me know, here or by email.
Leo.

WBEC Ridderkerk homepage.

Winboard Forum

Who is online