SSDF / crafty

Archive of the old Parsimony forum. Some messages couldn't be restored. Limitations: Search for authors does not work, Parsimony specific formats do not work, threaded view does not work properly. Posting is disabled.

SSDF / crafty

Postby pete » 07 Apr 2000, 23:31

Geschrieben von: / Posted by: pete at 08 April 2000 00:31:01:
I am about to go to bed but just read about the latest SSDF list at ICC where posting is not possible at the moment .
Crafty is at the same level as Hiarcs7.32 and Nimzo99 :-)
This is though suspected for awhile by a few Internet guys a rather big sensation I think .
Too tired to post more ; just thought some people here would need no more than that and felt like sharing ...
pete
pete
 

Re: SSDF / crafty

Postby Pete Galati » 08 Apr 2000, 01:20

Geschrieben von: / Posted by: Pete Galati at 08 April 2000 02:20:27:
Als Antwort auf: / As an answer to: SSDF / crafty geschrieben von: / posted by: pete at 08 April 2000 00:31:01:
I am about to go to bed but just read about the latest SSDF list at ICC where posting is not possible at the moment .
Crafty is at the same level as Hiarcs7.32 and Nimzo99 :-)
This is though suspected for awhile by a few Internet guys a rather big sensation I think .
Too tired to post more ; just thought some people here would need no more than that and felt like sharing ...
pete
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
Pete Galati
 

Re: SSDF / crafty

Postby Frank Quisinsky » 08 Apr 2000, 07:35

Geschrieben von: / Posted by: Frank Quisinsky at 08 April 2000 08:35:48:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: Pete Galati at 08 April 2000 02:20:27:
I am about to go to bed but just read about the latest SSDF list at ICC where posting is not possible at the moment .
Crafty is at the same level as Hiarcs7.32 and Nimzo99 :-)
This is though suspected for awhile by a few Internet guys a rather big sensation I think .
Too tired to post more ; just thought some people here would need no more than that and felt like sharing ...
pete
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
Hi Pete,
and in Germany a lot of people say that Crafty has not more than 2450 ELO :-)) I think people say this because Crafty is a free program :-)
In the WB tourneys from Kai, Christian and me with longer time controls can you
see that Crafty is a very strong program, 2500 ELO is the rating list on my webpage (over 600 games).
The SSDF results are to high in my eyes, I think that all SSDF ELO ratings are to hight (50-100 ELO) !
Right is, that Crafty is ~75 ELO weeker than Fritz or Junior.
Very good results for Prof. Dr. Robert Hyatt and Crafty play stronger and stronger with longer time controls or dual boards. Stronger then the most other programs with longer time controls.
I think on a PC with 1000 MHz or more is Crafty 50 ELO weaker than Fritz or Junior (not more) !!
I know which SSDF rating get AnMon :-))
When Crafty play with 2625 SSDF ELO then AnMon play with ~2550 ELO and 10 WB Engines play with over 2500 ELO (Comet, SOS, Insomniac, LGoliath, Gromit, AnMon, Phalanx, Patzer, Amy and Zarkov over 2625 SSDF ELO).
For me is not important 2500 ELO or 2600 ELO. For me is important the style from the program. Here I must say that a lot of WinBoard programs play fantastic chess.
Chess Genius is the German football team (safe and boring) :-)
AnMon is the Argentina football team (active and interesting) !
Fritz and Junior is the Brazil football team and Crafty is the team from England (hard and good) :-)) Shredder is the team for the last 10 minutes and Chess Tiger has a good strategy for the second half time ! ColChess, Averno played at the moment with 8 players (not 11) and The Crazy Bishop with 10 players :-)
And Phalanx ?
The Spain football team !
(easy with fantastic combinations)
But now Amyan from Chile is coming and the others ...
Regards
Frank
Frank Quisinsky
 

Re: SSDF / crafty

Postby pete » 08 Apr 2000, 11:20

Geschrieben von: / Posted by: pete at 08 April 2000 12:20:55:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: Pete Galati at 08 April 2000 02:20:27:
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
I can't follow here . Inconsistant equipment ? Why ?
To the crafty-version used : I think what we see is pretty much a "pure" crafty ; what Chessbase has said to be done is nothing more than port the engine to their own interface protocol ; when you look at the speed and the main lines chosen it seems obvious that that there is no weakening of crafty when compaired to the winboard version.

Also the Fritz5.32 general.ctg is said to be a book created out of GM games without any special tricks and from the games I have seen this seems to be true ( for example this is not true about the F6 book which is said to be a special tuned one . This is the exact way how all the crafty books were made , so if there is a difference it is probably only the different book learner ; I think this won't matter very much ; the crafty one is quite efficient ; and the Chessbase one is not known to be bad either.

Pretty strong : yes this is well-known but I think the data of SSDF still comes as a surprise to many ( ie up to Hiarcs level ) although it might change a little when more games were played ( although this usually doesn't happen ) .
The improvement from say 16.6 to say 17.7 is really quite amazing ; and when you look at crafty's branching factor it seems the faster the hardware the closer crafty will get .
From a chessplayer's view I still think crafty plays an ugly chess when compaired to programs like Shredder 4 or Hiarcs and when it is about things as piece mobility it seems to understand very little so my personal guess about its strenght when not playing at a quad xeon ( ;-) ) was lower than what we see now .
Crafty obviously knows how to make its points ; no brilliancy , no positional superman but _efficient_ ; and yes : I think it _does_ make a difference that it is a free program :-)) .
pete
pete
 

Re: SSDF / crafty

Postby Colin Frayn » 08 Apr 2000, 16:48

Geschrieben von: / Posted by: Colin Frayn at 08 April 2000 17:48:24:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: Frank Quisinsky at 08 April 2000 08:35:48:
Chess Genius is the German football team (safe and boring) :-)
AnMon is the Argentina football team (active and interesting) !
Fritz and Junior is the Brazil football team and Crafty is the team from England (hard and good) :-)) Shredder is the team for the last 10 minutes and Chess Tiger has a good strategy for the second half time !
ColChess, Averno played at the moment with 8 players (not 11)
Nice analogy :)
LOL :))))
Oh well... I'll have to work hard on ColChess in the near future. I haven't coded much on the engine since version 6 came out a while ago.
Cheers,
Col

ColChess Homepage
Colin Frayn
 

Re: SSDF / crafty

Postby Frank Quisinsky » 08 Apr 2000, 18:40

Geschrieben von: / Posted by: Frank Quisinsky at 08 April 2000 19:40:04:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: Colin Frayn at 08 April 2000 17:48:24:
Chess Genius is the German football team (safe and boring) :-)
AnMon is the Argentina football team (active and interesting) !
Fritz and Junior is the Brazil football team and Crafty is the team from England (hard and good) :-)) Shredder is the team for the last 10 minutes and Chess Tiger has a good strategy for the second half time !
ColChess, Averno played at the moment with 8 players (not 11)
Nice analogy :)
LOL :))))
Oh well... I'll have to work hard on ColChess in the near future. I haven't coded much on the engine since version 6 came out a while ago.
Cheers,
Col
Maybe, 9 players :-)
You have an interesting program, yes !
In my next tourney I will see ColChess :-)
Regards
Frank
Frank Quisinsky
 

Re: SSDF / crafty

Postby Pete Galati » 08 Apr 2000, 20:05

Geschrieben von: / Posted by: Pete Galati at 08 April 2000 21:05:34:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: pete at 08 April 2000 12:20:55:
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
I can't follow here . Inconsistant equipment ? Why ?
A group of programs running on 450mhz, another group on 200mhz, and another group running on 90mhz computers. Well, we know that the programs running on the 450mhz computers (with the most mem btw) are going to come out on top.
Pete
Pete Galati
 

ssdf / crafty

Postby pete » 08 Apr 2000, 23:58

Geschrieben von: / Posted by: pete at 09 April 2000 00:58:53:
Als Antwort auf: / As an answer to: Re: SSDF / crafty geschrieben von: / posted by: Pete Galati at 08 April 2000 21:05:34:
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
I can't follow here . Inconsistant equipment ? Why ?
A group of programs running on 450mhz, another group on 200mhz, and another group running on 90mhz computers. Well, we know that the programs running on the 450mhz computers (with the most mem btw) are going to come out on top.
Pete
and ??
I still can't understand . The chess programs at 90mhz are supposed to lose ; right ; but the ELO system is good at handling this : statistics about programs on different hardware .
the problem you see is probably that a computer program running on slower hardware will score less than what can be expected by ELO standards ; this is well possible , but I have not seen much data to back up this argument yet.
pete
 

Re: ssdf / crafty

Postby Pete Galati » 09 Apr 2000, 00:39

Geschrieben von: / Posted by: Pete Galati at 09 April 2000 01:39:53:
Als Antwort auf: / As an answer to: ssdf / crafty geschrieben von: / posted by: pete at 09 April 2000 00:58:53:
Crafty is pretty strong, we all know that, but I don't take SSDF very seriously.
They put out a constant effort, and that's good, but there's allways questions about the equipment they use, very inconsistant. I'm glad SSDF is doing their work, but I don't know if it's a very valid indication of Crafty's strength. And I should probably mention that this is not the Winboard version of Crafty, this is the Chessbase verion because of auto232 problems with running the Winboard version, it's also a Fritz opening book, that must speak well for the quality of the Fritz book.
Pete
I can't follow here . Inconsistant equipment ? Why ?
A group of programs running on 450mhz, another group on 200mhz, and another group running on 90mhz computers. Well, we know that the programs running on the 450mhz computers (with the most mem btw) are going to come out on top.
Pete
and ??
I still can't understand . The chess programs at 90mhz are supposed to lose ; right ; but the ELO system is good at handling this : statistics about programs on different hardware .
the problem you see is probably that a computer program running on slower hardware will score less than what can be expected by ELO standards ; this is well possible , but I have not seen much data to back up this argument yet.
Almost without exception, the list had 3 groups of program, the first group was the programs that had the highest rating & percentage of wins, they were on the fastest (450mhz) computers. Then the group with the next highest ratings & percentage of wins who were on the next fastest (200mhz) computers, Then the last group of programs who had the lowest group of ratings & percent of win who were on the slowest (90mhz) computers.
What are you asking about, I don't have any data on the ssdf, I'm just looking at what was posted at CCC.
Pete
Pete Galati
 

Re: ssdf / crafty

Postby pete » 09 Apr 2000, 18:54

Geschrieben von: / Posted by: pete at 09 April 2000 19:54:21:
Als Antwort auf: / As an answer to: Re: ssdf / crafty geschrieben von: / posted by: Pete Galati at 09 April 2000 01:39:53:
Almost without exception, the list had 3 groups of program, the first group was the programs that had the highest rating & percentage of wins, they were on the fastest (450mhz) computers. Then the group with the next highest ratings & percentage of wins who were on the next fastest (200mhz) computers, Then the last group of programs who had the lowest group of ratings & percent of win who were on the slowest (90mhz) computers.
What are you asking about, I don't have any data on the ssdf, I'm just looking at what was posted at CCC.
Pete
I think the results of SSDF have to be interpreted : sure you can't compare apples with pears ( don't know if this translates well )
Sure you can't compare Fritz on AMD450 with Chessmaster6000 on P200 or Tiger on P90 if you want to come to conclusions about program strength .
But I think you can compair the results of the programs in the _same_ group very well even if the opponents have been out of the other group .
if Program A ( on 450 ) beats program C ( on 200 ) 13:7 ; and Program B ( on 450 ) bests Program C ( on 200 ) 19:1 I think it is possible to compare the results of A and B .
The results are still consistent ; there are two variables : the hardware and the chosen program ; a crafty on a AMD450 _is_ stronger than a Tiger on a P90 and even the margin seems to be rather predictibable .
Two reasons are posted over and over against the SSDF list :
1.) the autoplayer is buggy
I won't engage in that but as I see it at least for crafty this can't be a big problem as it uses the chessbase autolayer in its games ; and in fact if you leave out Rebel there have never been many facts to back up this argument ( and the Rebel versions that had problems don't participate )
2.) The system itself is wrong .
I more or less agree to this partly ; I think though the results are still consistent when you take into consideration what they in fact are ; programs playing long series of tournament game matches ( with or without autoplayer ) against each other . And it seems the manually played matches conducted in a similar way correlate very well with the SSDF list ( for example the Erizaquoi tournaments ) .
This won't necessarily mean it correlates well with say a 100 round swiss tournament between the 50 best SSDF programs ( double-round ) ( for example learning is then no major issue anymore ) .
What _I_ don't like about the list is that I expect that a program on a 450 will win too many games against the program on the 90 ( more than expected ) ; so the list is inflationary in itself and then it _would_ make a huge difference if you give it opponents on a 450 a 200 or a 90 .
This is IMHO the interesting question and the reason why I would like to see some facts ( but it seems not enough as else I could simply calculate a little :-)) .
The results of the current SSDF seem to suggest that Crafty on a 450 in longer matches against other programs is playing at the same level as Hiarcs or Nimzo ; and _yes_ : this still surprises me a little and within the posted ( still quiet high ) error margin it is more or less statistically proven so you can't even discuss about it :-) .
pete
pete
 

Re: ssdf / crafty

Postby Pete Galati » 09 Apr 2000, 21:37

Geschrieben von: / Posted by: Pete Galati at 09 April 2000 22:37:03:
Als Antwort auf: / As an answer to: Re: ssdf / crafty geschrieben von: / posted by: pete at 09 April 2000 19:54:21:
Almost without exception, the list had 3 groups of program, the first group was the programs that had the highest rating & percentage of wins, they were on the fastest (450mhz) computers. Then the group with the next highest ratings & percentage of wins who were on the next fastest (200mhz) computers, Then the last group of programs who had the lowest group of ratings & percent of win who were on the slowest (90mhz) computers.
What are you asking about, I don't have any data on the ssdf, I'm just looking at what was posted at CCC.
Pete
I think the results of SSDF have to be interpreted : sure you can't compare apples with pears ( don't know if this translates well )
Sure you can't compare Fritz on AMD450 with Chessmaster6000 on P200 or Tiger on P90 if you want to come to conclusions about program strength .
But I think you can compair the results of the programs in the _same_ group very well even if the opponents have been out of the other group .
if Program A ( on 450 ) beats program C ( on 200 ) 13:7 ; and Program B ( on 450 ) bests Program C ( on 200 ) 19:1 I think it is possible to compare the results of A and B .
The results are still consistent ; there are two variables : the hardware and the chosen program ; a crafty on a AMD450 _is_ stronger than a Tiger on a P90 and even the margin seems to be rather predictibable .
Two reasons are posted over and over against the SSDF list :
1.) the autoplayer is buggy
I won't engage in that but as I see it at least for crafty this can't be a big problem as it uses the chessbase autolayer in its games ; and in fact if you leave out Rebel there have never been many facts to back up this argument ( and the Rebel versions that had problems don't participate )
2.) The system itself is wrong .
I more or less agree to this partly ; I think though the results are still consistent when you take into consideration what they in fact are ; programs playing long series of tournament game matches ( with or without autoplayer ) against each other . And it seems the manually played matches conducted in a similar way correlate very well with the SSDF list ( for example the Erizaquoi tournaments ) .
This won't necessarily mean it correlates well with say a 100 round swiss tournament between the 50 best SSDF programs ( double-round ) ( for example learning is then no major issue anymore ) .
What _I_ don't like about the list is that I expect that a program on a 450 will win too many games against the program on the 90 ( more than expected ) ; so the list is inflationary in itself and then it _would_ make a huge difference if you give it opponents on a 450 a 200 or a 90 .
This is IMHO the interesting question and the reason why I would like to see some facts ( but it seems not enough as else I could simply calculate a little :-)) .
The results of the current SSDF seem to suggest that Crafty on a 450 in longer matches against other programs is playing at the same level as Hiarcs or Nimzo ; and _yes_ : this still surprises me a little and within the posted ( still quiet high ) error margin it is more or less statistically proven so you can't even discuss about it :-) .
pete
I'm one of those people still using a 586-90 computer, so when I compare the search depths that can be reached on my computer with Crafty analysis, with for example the search depths that Dann can reach analyzing the same position on his newer/faster computer, then I see a very drastic problem with the difference in computer speeds, because while I can only reach a few search depths and take a long time doing it, Dann can easilly go sometimes go 8 or 10 deeper than me, and do it in a very _small_ amount of time. So, I understand that's it's a very real situation that allot of people are using slow computers like me, I don't think it's _really_ fair to the programs to have to run on them.
My 586-90 does pathetic slow TSCP and Crafty benchmark tests compared to everyone else using the newer computers.
I might have left the impression that I don't think Crafty should have done that well, and scored up there with the strong comercial programs, but that's not really the case. I think Crafty's one of the strongest programs out there.
You have to figure that if you look at the top ten programs in any tournament that involves comercial and freeware software, if there's any freeware programs in that top 10, then that's a very strong program.
And even though I complain about SSDF, I do realize that's there's always problems in tournaments, things are never exactly the way they should be, but then the opinion of exactly how things should be will change from person to person. You can't please everybody.
But I do agree with the SSDF results about Crafty being that strong, I just wish SSDF could run all on the same equipment.
Pete
Pete Galati
 

Re: ssdf / crafty

Postby Dann Corbit » 10 Apr 2000, 18:08

Geschrieben von: / Posted by: Dann Corbit at 10 April 2000 19:08:33:
Als Antwort auf: / As an answer to: Re: ssdf / crafty geschrieben von: / posted by: Pete Galati at 09 April 2000 22:37:03:
[snip]
But I do agree with the SSDF results about Crafty being that strong, I just wish SSDF could run all on the same equipment.
This stems from not understanding how the SSDF works. In fact, doing as you suggest would render the SSDF results nearly useless.
The SSDF results are valuable because there have an enourmous number of games played at a particular time control with a particular program on particular hardware. What this causes is an opponent of precisely known strength. We can use this opponent, of whatever strength, to gauge precisely the strength of other opponents.
Hence, a certain configuration has value precisely because the strength is known to a high degree of precision. If (instead) we were to dump the old hardware and use new hardware, we would have completely *UNKNOWN* strenght and (hence) our results would be completely without value of any kind for determining the strength of a new computer setup.
The SSDF knows exactly what they are doing and the criticism of their methods is almost always uninformed.


My FTP site
Dann Corbit
 


Return to Archive (Old Parsimony Forum)

Who is online

Users browsing this forum: No registered users and 27 guests