Problem to run Scorpio on a Core 2 Duo

[Volker Pittlik]

...
To Jim:
I have sent the source code at the email address provided on your website.

Daniel

I can't see it anywhere. You already send me the source code. Weren't it a possibility to host it at sourceforge.net?

Volker

[Jim Ablett]

Hello Volker,

I can't see it anywhere. You already send me the source code. Weren't it a possibility to host it at sourceforge.net?

I received the original email which was sent to You, Me and Leo , but with no attachments. I sent an email to Daniel, but he hasn't got back to me yet.

Jim.

[Andreas Guettinger]

It's also hasn't appeared yet on Leos site.
Only the 5 piece bitbases.

[Daniel Shawul]

Hi Jim
I did send a second email with the attachement May be it is in your bulk folder? Anyway i set up a free website www.igotfree.com
(thanks Ubaldo!) .
You can download the source here
http://www.igotfree.com/dshawul/scorpio/
regards
Daniel

[Jim Ablett]

Hi Daniel,

did send a second email with the attachement Confused May be it is in your bulk folder?

I think my spam filter must have ate it

Sorry for all the bother.

Jim.

[Andreas Guettinger]

Great, thanks.

I still have two questions. Does scorpio profit from >2 cpus?
And second, on my PowerPC Mac, the provided book is not recogized. Probably a big endian issue. Is it be possible to create a book from a pgn file that would be recognised?

[Jim Ablett]

Better late than never ......



Scorpio 1.84:

Intel compiler 9.1 32 bit PGO build

on my homepage.

Jim.

[Andreas Guettinger]

Daniel,

there is still problems with scorpio on some 64bit systems. For example, in 64 bit linux 'long int' is 64 bit, which conflicts with you definition in scorpio.h:

#define BMP32 long int
should be
#define BMP32 int

The conflict then arrises in eval.cpp, where you re-address the WBMP8 arrays with long pointers, that have the wrong size now (64bit instead of 32 bit). Changing the longs to ints seemed resolve the problem, at least partially, but maybe there are more hidden problems.

long* const PWB = (long *) WB;
to
BMP32* const PWB = (BMP32 *) WB;
etc.

Compiling scorpio with gcc and the -m32 flag, which forces longs to 32bit, resolves the problem temporarily.

Just in case you want to look into this...
Andy

[Daniel Shawul]

HI Andreas
Thanks for the suggestions! Now i have modified the source code to handle 64bit integers. Now defining ARC_64BIT at the begining of the scorpio.h will fix these problems. In eval it uses bitboards 64bits for
readreasing the attack tables so there won't be a problem. I am surprised i haven't noticed this while trying to compile 64 bit scorpio Surely earlier versions of 64bit scorpio never worked properly!

Yes scorpio works on dual cores. I think by now I am
almost sure that there are no problems on dual cores.
It will probably work on quads but i am not sure.

As to the book, I don't know why it doesn't work. I am not an expert in big endian/little endian stuff, but will there be a problem as long as I read/write in words?

regards
Daniel

PS: Scorpio 1.9 will be relased today,where i tried to fix the problem.

[Andreas Guettinger]

HI Andreas
Thanks for the suggestions! Now i have modified the source code to handle 64bit integers. Now defining ARC_64BIT at the begining of the scorpio.h will fix these problems. In eval it uses bitboards 64bits for
readreasing the attack tables so there won't be a problem. I am surprised i haven't noticed this while trying to compile 64 bit scorpio Surely earlier versions of 64bit scorpio never worked properly!

Not necessarily. Because to the different data models of 64-bit systems it can run fine on some systems/compilers and not on others. I'm not sure about windows, but 64-bit Mac uses LLP64 and have long still 32-bit and long long 64-bit, whereas 64-bit Unix/Linux uses ILP64 and have 64-bit sized long. Thus your source run fine on my Mac even when compiled in 64-bit, but not on Unix. Therefore it's good to just define int32, int64 etc at the beginning and use them in the code (like you did almost everywhere). If found this page useful to read about the different data types models:

http://www.unix.org/whitepapers/64bit.html

As to the book, I don't know why it doesn't work. I am not an expert in big endian/little endian stuff, but will there be a problem as long as I read/write in words?

regards
Daniel

PS: Scorpio 1.9 will be relased today,where i tried to fix the problem.

Well, about the endianess, I also don't know how to solve this. Maybe somebody else can help there? Normally one is dependant on the system to handle file I/O, and they probably do it differently on big- or little endian architecture (see below). It probably depends on the architecture of the book.
Normally, the endian problems go away if the book is created on the same architecture as it is used. So if your book author has the chance to have access to a Mac sometime, then he could create a book for the Mac.
By the way, word size on 32-bit and 64-bit systems may differ, so is the old book still compatible in 64-bit?

PS: found this about file i/o and endianness:
Typically 8 consecutive bits are treated as an entity called a byte. Further, most computer systems internally handle data in chunks of two consecutive bytes sometimes referred to a "word". When a piece of data is stored in memory or on a file and it is larger than a byte, the two bytes within each word can be stored in either of two orders. Depending on whether the most significant part is stored before the less significant, the scheme is referred to as "big-endian" (big end first) or "little-endian".

[Daniel Shawul]

Not necessarily. Because to the different data models of 64-bit systems it can run fine on some systems/compilers and not on others. I'm not sure about windows, but 64-bit Mac uses LLP64 and have long still 32-bit and long long 64-bit, whereas 64-bit Unix/Linux uses ILP64 and have 64-bit sized long. Thus your source run fine on my Mac even when compiled in 64-bit, but not on Unix. Therefore it's good to just define int32, int64 etc at the beginning and use them in the code (like you did almost everywhere). If found this page useful to read about the different data types models:

http://www.unix.org/whitepapers/64bit.html

Yes the old code just assumed long to be 32 bits.
But i am confused about something else. Now in some places like for example the hashtable entry if i want a certain known size.

typedef struct tagHASHREC {
HASHKEY hash_key;
BMP32 move;
BMP16 score;
UBMP8 depth;
UBMP8 flags;
}HASHREC,*PHASHREC;

Now i am assuming that the total size is 128bits per entry. But the BMP32 could actually be 64bit and the BMP16 could be 32 bits on some systems. So it means i have different hashtable sizes. Using the word size of the specific processor is good for making fast lookups but i am not sure if this can compensate for the loss in performance due to decreased number of hashtable entries !?

Normally, the endian problems go away if the book is created on the same architecture as it is used. So if your book author has the chance to have access to a Mac sometime, then he could create a book for the Mac.
By the way, word size on 32-bit and 64-bit systems may differ, so is the old book still compatible in 64-bit?

I understand now. I have a code like sizeof(int) in the read and write, so a book created on 32bit systems can not be used on 64 bit systems, since the read will be performed with a different size than the write !? But since i don't specifically address the high/low bytes of the word in the book code i don't think the difference in endianess will cause a problem.

regards
Daniel

[Andreas Guettinger]

Yes the old code just assumed long to be 32 bits.
But i am confused about something else. Now in some places like for example the hashtable entry if i want a certain known size.

typedef struct tagHASHREC {
HASHKEY hash_key;
BMP32 move;
BMP16 score;
UBMP8 depth;
UBMP8 flags;
}HASHREC,*PHASHREC;

Now i am assuming that the total size is 128bits per entry. But the BMP32 could actually be 64bit and the BMP16 could be 32 bits on some systems. So it means i have different hashtable sizes. Using the word size of the specific processor is good for making fast lookups but i am not sure if this can compensate for the loss in performance due to decreased number of hashtable entries !?
regards
Daniel

I have to make a small correction to what I posted above, UNIX uses LP64, mac+gcc LLP64, that means in Unix long is 64 bit, int 32 bit, and on Mac int and long are 32-bit, long long 64-bit.

So there should be no problem for your hash entries, just make sure on all windows your BMP32 uses a 32 bit int. For the BMP16, if you use short, there should be no problem according to the table on the link I posted before. Alternatively, you could use BMP32 for score to make it "big" on all systems, but I think your hash entries are fine and should have the same size on most common systems...

[bob]

The easiest way to fix the endian issue is to read/write bytes, and piece the data together inside your program. Otherwise, you can never write a 64 bit int on a little-endian machine and read it successfully on a big-endian machine. Additionally, even going from little-endian 32 bits to little-endian 64 bits can be a problem as, for example, gcc pads structs differently on the two architectures.

Feel free to look at the Crafty source, particularly book.c and utility.c... utility.c contains routines to read/write 32 bit ints, 64 bit ints, and floating point values, and it works in an endian-independent way so that I can now share the same binary opening book between 32/64 bit machines and between little/big-endian machines as well...

[Daniel Shawul]

My mistake was that I assumed that the bytes are in reversed order only in registers in the little-endian representaion. If the bytes will be reversed when 'transpoting' them to RAM, why not also reverse them when transporting to disk?? Now i understand that the bytes are written to disk in the same order as they are in registers, but not in RAM. This ofcourse causes porting problems but I don't understand why there isn't a convention on the byte order in disks. Thanks for the help!

Daniel