Polyglot now supports the UCI protocol:-)

[Zach Wegner]

Well for one, it makes a lot more sense then "cores". It seems to me that the problem is with polyglot then. "CPUs" would be the only other workable option; I really can't believe there's any disagreement over this. Some engines have "threads" or a similar option because they use threads, but that is no good for a standard option.

[Michel]

I really don't see the point of every engine stubbornly choosing its own name when there are already so many to choose from. The name of the thread option is really not all that important.

Some engines have "threads" or a similar option because they use threads, but that is no good for a standard option.

I am neutral on this but for the OS a thread or a process is the same thing. It is just a question of how much memory is shared by default.

[abik]

I really don't see the point of every engine stubbornly choosing its own name....

Wait a minute, if that is addressed to me, then that seems a bit unfair

If you read http://www.talkchess.com/forum/viewtopi ... _view=flat
(admittedly a time-waster at times) then you will see I tried to convince others of the importance of a standard keyword.

After the discussion sadly drifted away into space, I decided the most reasonable proposals were" Processors" or "CPUs" and arbitrarily picked the former. I still have no strong feelings on the exact keyword (but I do have strong feelings about a standard). Since no consensus was reached on standardizing the UCI protocol in that earlier thread, however, I am afraid you will have to deal with a lot of more "stubborn" choices by other engines in the future....

[Michel]

Wait a minute, if that is addressed to me, then that seems a bit unfair

Sorry, it was just a general remark and not meant to be offensive (although it may
have sounded that way).

I was just thinking that since two major engines already use "cpus" (and rybka uses
"max cpus") there is no need to introduce yet another name "processors".

It is trivial to add it to Polyglot but I am just afraid that further engines will introduce "max processors", "number of processors", "cores" "max cores"...

[abik]

Sorry, it was just a general remark and not meant to be offensive (although it may have sounded that way).

No offense taken. Like you, I am baffled why it is so hard to get consensus on this option name
But until Stefan Meyer-Kahlen cuts the knot in the UCI standard, I guess we will see a lot of creative option names....

[Michel]

But until Stefan Meyer-Kahlen cuts the knot in the UCI standard, I guess we will see a lot of creative option names....

Well what I do in case of lack of a standard it to pick something which is already being used rather than invent something new.... I do not see why it is necessary to invent a new name...

[H.G.Muller]

Well, there already seems to be a de-facto standard: 'Threads' is used far more than any of the alternatives, and Shredder is amongst those using it. In addition, most of the alternatives do contain the word 'threads'. Only 2 of 13 don't, their name for the option contains 'CPUs'.

[Eric Mullins]

Yes, this would be nice to become standardized. I like 'threads' myself. But anything will do, even 'foobars' so long as it can be uniform.

[Michel]

b14 is here

http://alpha.uhasselt.be/Research/Algeb ... t-release/

I started by implementing Eric Mullins' code for setting the priority of the engine on Windows (for consistency with the already existing Posix option).

But then I decided to refactor the some code so that the Windows and Posix part
of the source are now more uniform. This has allowed me to eliminate many
#ifdef _WIN32's.

I also eliminated a number of potential busy loop scenarios on Windows (the most notable
one in uci_open).

Finally I reenabled SIGINT on Windows. So you can do ^C while testing in a terminal.

Since a lot of code has been shuffled around and I have not done extensive testing
I have called the binary "polyglot_dev.exe"

[Michel]

b15 is here

http://alpha.uhasselt.be/Research/Algeb ... t-release/

Lots of refactoring.

One very important and embarrassing change. I turned out that (through lack of history knowledge) I had omitted one major contributor to the current code base from the AUTHORS file ("Morning Yellow").

I have corrected this and removed all earlier versions.

[ulver]

On my 4 GB RAM quad-core machine i'm keep getting an error:

300000 games ...
310000 games ...
allocating 1536MB ...
my_realloc(): realloc(): Not enough space
Terminate batch job (Y/N)?

polyglot cannot allocate more than 1536Mb of memory, with default book-making settings, it is a bit frustrating, when a book making limited to 300000 of games

[Michel]

On my 4 GB RAM quad-core machine i'm keep getting an error:

300000 games ...
310000 games ...
allocating 1536MB ...
my_realloc(): realloc(): Not enough space
Terminate batch job (Y/N)?

polyglot cannot allocate more than 1536Mb of memory, with default book-making settings, it is a bit frustrating, when a book making limited to 300000 of games

Is this Linux or Windows? 32 bit or 64 bit. If the latter did you try to compile a 64 bit version of PG?

I fear that if the realloc system call fails there is not much PG can do about it internally. PG basically
needs to allocate a big hash table to build the book.

EDIT: The internal size accounting in PG seems to be in 32 bit signed numbers. But since
1536 Mb is less than 2Gb you should not be hitting this yet. Only at the next stage.

EDIT2: The error message "realloc(): Not enough space" really comes from the system.
So this error is not due to calling realloc() with a negative number say.

[ulver]

it is a polyglot.exe, not compiled. OS is Windows Server 2008 R2 x64 (beta). I will try to test compiled polyglot under Gentoo x64 (gcc-4.3.2)

[ulver]

Here is an output of one of my books:
Lines for white : 35284
Lines for black : 48331
Positions of lines for white : 70706
Positions of lines for black : 94810
Unreachable white positions (?) : 13403
Unreachable black positions (?) : 7986
Isolated positions : 19474

can anyone bring more information about those unreachable and isolated positions?Are those positions-"patterns of positions" that created artificially during book merging and will never occur in any single book-testing game or is it some hidden knowledge about positions excluding of which will cause decreasing of book strength. Interesting to test book vs pruned (freed from unreachable positions, if it's possible) book.

[Michel]

can anyone bring more information about those unreachable and isolated positions?

I have tried to explain it in the manpage of polyglot. An asci rendered version is here

http://alpha.uhasselt.be/Research/Algeb ... anpage.txt

As is explained Isolated/unreachable positions are an artifact of the merge process. An unreachable position is defined as a position which can be deleted from the book without affecting game play.

Unfortunately it is hard to identify such positions.

Unreachable positions can become reachable when you insert moves in the book (e.g. using Scid). So they do represent information in some sense.

I

[ulver]

I tried one experiment and a result not so obvious as stated above. probably it is not related to book-merging process. So i took a white.pgn consist of 842 only white wins (1-0) and black.pgn (842 games black wins only 0-1) and here is an output of what i got...Same result from compiled polyglot which i tried under gentoo x64 and polyglot_dev, which i used on windows server 2008 x64 R2 (results taken from windows):

1) polyglot_dev make-book -pgn test-white.pgn -bin white.bin

polyglot_dev info-book -bin white.bin -exact

PolyGlot 1.4W10UCIb18 by Fabien Letouzey
Lines for white : 186
Lines for black : 0
Positions on lines for white : 805
Positions on lines for black : 0
Unreachable white positions(?) : 7
Unreachable black positions(?) : 0
Isolated positions : 19


2) polyglot_dev make-book -pgn test-black.pgn -bin black.bin

polyglot_dev info-book -bin black.bin -exact

PolyGlot 1.4W10UCIb18 by Fabien Letouzey
Lines for white : 0
Lines for black : 169
Positions on lines for white : 0
Positions on lines for black : 664
Unreachable white positions(?) : 0
Unreachable black positions(?) : 8
Isolated positions : 28


3) polyglot_dev merge-book -in1 white.bin -in2 black.bin -out book-test.bin

polyglot_dev info-book -bin book-test.bin -exact


PolyGlot 1.4W10UCIb18 by Fabien Letouzey
Lines for white : 186
Lines for black : 169
Positions on lines for white : 805
Positions on lines for black : 664
Unreachable white positions(?) : 7
Unreachable black positions(?) : 8
Isolated positions : 47

So, it is a polyglot book making technique, not book merging, as all isolated/unreachable positions merged in output book too. is it some polyglot bug, or feature or anything else? I know that polyglot book making efficiency is probably the best from all known, i can't even imagine what are the results of .ctg dumping .

[Michel]

I am out of town so I can't give an elaborate reply right now.

Transpositions can also yield isolated positions although this effect is much less pronounced.

Suppose you have move sequences

abce
cbae

and make a book with -min-game=2. Then e will be retained but a,b,c will not. So you end
up with an isolated position (which may very well be unreachable).

I think I will make a utility to prune isolated positions. It is obvious that many people feel uncomfortable about them.

[ulver]

Not at all!! As for book making I am completely satisfied with polyglot, the only problem, how polyglot book behaves freed from isolated/unreachable pos. In some cases it is about 50% of book. It is always interesting to find something new about book format.

[H.G.Muller]

I think I will make a utility to prune isolated positions. It is obvious that many people feel uncomfortable about them.

Why would you do that? Even if you mean unreachable in stead of isolated: they would only be unreachable for engines that play by the book from the opening position. But they might still be reachable in tests that play from a number of fixed positions close to the opening position, like Nunn, or in games started from a truncated GUI book, when you want the book to be used as own book of an engine.

[Edsel Apostol]

I think isolated positions are still valuable data in the book. It doesn't mean that if a certain book position cannot be reached by the other moves in the book it is already worthless, it is possible that this can be reached by actual game play and the book move can still be played.

Polyglot book entries I think are not concerned with lines but with positions. The case of isolated positions appear when you optimized your book by creating a book for white and a book for black and merging them.

For example:
white available book moves:
start position: e2e4,
position after e2e4 c7c5: g1f3

black available book moves:
position after d2d4: d7d5

If you're going to merge this, the black move d7d5 is isolated as it cannot be reached by playing white book moves (there is no moves for white that is d2d4 in the book) but it is still important as the opponent might play d2d4 so you must reply with d7d5.

Somebody correct me if I'm wrong with my analogy here.