I hate Hiarcs!

Discussions about Winboard/Xboard. News about engines or programs to use with these GUIs (e.g. tournament managers or adapters) belong in this sub forum.

Moderator: Andres Valverde

I hate Hiarcs!

Postby Tord Romstad » 08 Dec 2005, 00:06

Being the author of a weak chess engine, I am used to seeing my program lost most of my test matches, often by horrible margins. I can handle that. Even though the numbers themselves are ugly, I can usually find something to cheer me up when looking at the games. Among the few wins, there are sometimes some really beautiful or spectacular games, even against giants like Fruit or Shredder. And in the innumerable lost games, there is often a good fight, and Glaurung displays considerable resourcefulness.

The one exception is Hiarcs. Except in some of the wildest king pawn openings, where Glaurung is sometimes able to score a half decent score (but of course never more than 50%), the games are terrible to behold. My poor little engine never has the shadow of a chance. Hiarcs invariably secures a tiny edge directly after the opening, and the eval relentlessly climbs by a few centipawns for every move, eventually reaching +5 or +6 somewhere between move 40 and 60. Glaurung never achieves anything resembling counterplay.

Today I had a shiny new Glaurung version ready for testing. I decided to run a Noomen match against Hiarcs 9.6. For a short while, it looked like I had finally managed to create a version which could hold its own against the mighty Hiarcs. After 36 games, when I had to leave the computer for a few hours, the standings were 18-18.

When coming back, I was of course eager to see how the match had developed. 53 games had been played while I was away. Imagine my disappointment when I found that Glaurung had lost 32, drawn 19, and won just 2 of these games. As usual, looking at the games didn't do much to raise my spirits. It doesn't look like Hiarcs playing against Glaurung at all, but more like Karpov at his prime playing against some slow dedicated computer chess unit from the middle of the 1980s. Disgusting beyond words.

Glaurung was slightly more successful in the last few games of the match, which ended 65.5-34.5 (+48,=35,-17) in favor of Hiarcs. In the end, not much better than the usual disastrous results. It looks like I still have some hard work to do before I can release a new and improved version of my engine. :-(

There are several other engines which are as strong or stronger than Hiarcs 9.6, but I have never seen any other chess program which is able to consistently win games in such a polished and impressive style. There are many chess engines I admire, but Hiarcs is without doubt the one I most would have liked to have written myself. Waiting for Hiarcs 10 fills me with an uncomfortable mixture of excitement and dread.

Tord
User avatar
Tord Romstad
 
Posts: 639
Joined: 09 Oct 2004, 12:49
Location: Oslo, Norway

Re: I hate Hiarcs!

Postby Dann Corbit » 08 Dec 2005, 00:11

Perhaps you can find some pattern that Hiarcs is exploiting again and again.

I suggest looking at the evaluation for both programs and search for large disagreements (e.g. Hiarcs sees + 200 and Glaurung sees -4).

You might find some missing term in your evaluation (or something of that nature).

A real nemesis is very valuable to find something that needs attention. You have discovered that something does need attention. Probably, if you scour the logs, you will find out what it is.
Dann Corbit
 

Re: I hate Hiarcs!

Postby Tord Romstad » 08 Dec 2005, 00:36

Dann Corbit wrote:Perhaps you can find some pattern that Hiarcs is exploiting again and again.

Perhaps, but so far I have not noticed any recurring pattern. Hiarcs simply seems to be better than Glaurung at absolutely everything (including tactics, despite the fact that Hiarcs is much slower, in nps count as well as in search depth).

I suggest looking at the evaluation for both programs and search for large disagreements (e.g. Hiarcs sees + 200 and Glaurung sees -4).

You might find some missing term in your evaluation (or something of that nature).


Yes, I am trying.

A real nemesis is very valuable to find something that needs attention. You have discovered that something does need attention. Probably, if you scour the logs, you will find out what it is.

Technically speaking, it would be somewhat imprecise to call Hiarcs a nemesis for Glaurung. Hiarcs doesn't score much more than it should, considering the big rating difference (Hiarcs 9.6 is much stronger than Hiarcs 9.0 for Windows). The remarkable thing is how Hiarcs wins.

Glaurung's true nemesis is Pepito. Glaurung is rated much higher than Pepito on every rating list I have seen, but I have still never seen Glaurung win or draw a long match against Pepito.

Tord
User avatar
Tord Romstad
 
Posts: 639
Joined: 09 Oct 2004, 12:49
Location: Oslo, Norway

Re: I hate Hiarcs!

Postby Alessandro Scotti » 08 Dec 2005, 10:35

Hello Tord,
I would love to have a "weak" engine like Glaurung! :D
Until a while ago, I thought it could be useful to play against a stronger engine and try to get some of its "secrets". Now, I'm not so sure anymore.
An engine that is rated hundreds of points above Kiwi and defeats him consistently (of which there is plenty!) gives me very little information in fact. Was Kiwi outsearched, "outbugged", outplayed by better or extra knowledge? Hundreds of factors and it's almost impossible to tell these apart.
On the other hand, there is much to learn from losing to a "weaker" engine. There you can often exclude many of the above factors, and be left with useful information to work with.
So maybe Pepito will be more useful than Hiarcs in showing areas of improvement for Glaurung...
User avatar
Alessandro Scotti
 
Posts: 306
Joined: 20 Nov 2004, 00:10
Location: Rome, Italy

Re: I hate Hiarcs!

Postby Tord Romstad » 09 Dec 2005, 23:11

Hi Alessandro!

Alessandro Scotti wrote:Hello Tord,
I would love to have a "weak" engine like Glaurung! :D
Until a while ago, I thought it could be useful to play against a stronger engine and try to get some of its "secrets". Now, I'm not so sure anymore.
An engine that is rated hundreds of points above Kiwi and defeats him consistently (of which there is plenty!) gives me very little information in fact. Was Kiwi outsearched, "outbugged", outplayed by better or extra knowledge? Hundreds of factors and it's almost impossible to tell these apart.
On the other hand, there is much to learn from losing to a "weaker" engine. There you can often exclude many of the above factors, and be left with useful information to work with.

I think both sorts of tests are useful, and try to run matches against engines of a wide range of strengths.

So maybe Pepito will be more useful than Hiarcs in showing areas of improvement for Glaurung...

I don't test against Pepito very often, because it is not entirely stable on my machine (it segfaults once every 200 games or so). The last Glaurung-Pepito match I can find in my archives was played with a development version very similar to Glaurung 0.2.1. Pepito won this match by 57-43.

Today I decided to try again with my current version. To my surprise, Glaurung managed to win by 64.5-35.5! Still not a great result, but a tremendous improvement. I must have done something right over the last half year. :D

Tord
User avatar
Tord Romstad
 
Posts: 639
Joined: 09 Oct 2004, 12:49
Location: Oslo, Norway

Re: I hate Hiarcs!

Postby Robert Allgeuer » 10 Dec 2005, 00:18

Tord,
below are the results of Glaurung Mainz from YABRL.
According to this Glaurung has very good results against:
- Gandalf
- Ruffian
- Pseudo (it actually crushed it!)
- SoS
- and old Zappa versions

Glaurung has relatively low scores against:
- Fruit 2.2.1
- List
- WildCat
- Pepito
- Quark
- Dragon

Robert


Code: Select all
23 Glaurung Mainz            : 2623  1098 (+480,=291,-327), 57.0 %

Fruit v2.2.1                  :  20 (+  1,=  4,- 15), 15.0 %
Toga II v1.0                  :  20 (+  3,=  6,- 11), 30.0 %
Shredder 9 UCI dcbk           :  20 (+  4,=  4,- 12), 30.0 %
Chess Tiger 15.0 normal       :  20 (+  1,=  9,- 10), 27.5 %
Chess Tiger 2004 normal       :  20 (+  1,=  8,- 11), 25.0 %
Gandalf v6.0WB                :  20 (+  8,=  4,-  8), 50.0 %
Pro Deo v1.1                  :  20 (+  4,= 10,-  6), 45.0 %
Spike v1.0a Mainz             :  20 (+  5,=  7,-  8), 42.5 %
Ruffian v2.1.0                :  20 (+  9,=  2,-  9), 50.0 %
List v5.12                    :  20 (+  6,=  3,- 11), 37.5 %
Ktulu v7.0                    :  20 (+  6,=  6,-  8), 45.0 %
Jonny v2.82                   :  20 (+  7,=  7,-  6), 52.5 %
DeepSjeng v1.6ntb             :  20 (+  6,=  8,-  6), 50.0 %
Aristarch v4.50               :  20 (+  7,=  5,-  8), 47.5 %
Pseudo v0.7c                  :  20 (+ 13,=  5,-  2), 77.5 %
SmarThink v0.17a              :  20 (+  7,=  5,-  8), 47.5 %
Thinker v4.6b                 :  20 (+  5,=  8,-  7), 45.0 %
Patriot v1.2.3cpbk            :  20 (+  6,=  5,-  9), 42.5 %
SoS 5.1                       :  20 (+ 11,=  3,-  6), 62.5 %
Delfi v4.6                    :  20 (+  8,=  6,-  6), 55.0 %
Wildcat v5.0                  :  20 (+  6,=  3,- 11), 37.5 %
Pharaon v3.3                  :  20 (+  9,=  4,-  7), 55.0 %
Crafty v17.14DC               :  19 (+  6,=  9,-  4), 55.3 %
Zappa v1.1                    :  20 (+  9,=  7,-  4), 62.5 %
Yace v0.99.87                 :  20 (+  7,= 10,-  3), 60.0 %
Crafty v19.13RA               :  20 (+  8,=  5,-  7), 52.5 %
Crafty-MPC v18.15DC           :  19 (+  8,=  4,-  7), 52.6 %
Gothmog v1.0 beta 10          :  20 (+  8,=  8,-  4), 60.0 %
Anmon v5.53                   :  20 (+  9,=  7,-  4), 62.5 %
Little Goliath 2000 v3.9      :  20 (+  9,=  8,-  3), 65.0 %
Baron v1.6.1                  :  20 (+  8,=  5,-  7), 52.5 %
Pepito v1.59 profile          :  20 (+  7,=  3,- 10), 42.5 %
Green Light Chess v3.01.2.2   :  20 (+ 10,=  5,-  5), 62.5 %
Movei v0.08.295               :  20 (+  9,=  6,-  5), 60.0 %
Tao v5.6                      :  20 (+ 13,=  2,-  5), 70.0 %
Ufim v7.00                    :  20 (+ 10,=  5,-  5), 62.5 %
Quark v2.35                   :  20 (+  8,=  7,-  5), 57.5 %
LambChop v10.99               :  20 (+ 11,=  5,-  4), 67.5 %
Gromit v3.8.2                 :  20 (+ 15,=  5,-  0), 87.5 %
Comet B44-2                   :  20 (+ 12,=  5,-  3), 72.5 %
Dragon v4.4.3                 :  20 (+  8,=  5,-  7), 52.5 %
Amy v0.8.3                    :  20 (+ 14,=  1,-  5), 72.5 %
KnightDreamer v3.2            :  20 (+ 16,=  3,-  1), 87.5 %
Comet B62-3                   :  20 (+ 12,=  4,-  4), 70.0 %
Francesca M.0.0.9             :  20 (+ 13,=  5,-  2), 77.5 %
PostModernist v1.007          :  20 (+ 14,=  2,-  4), 75.0 %
Leila v0.53h                  :  20 (+ 14,=  3,-  3), 77.5 %
Tcb v0045                     :  20 (+ 15,=  3,-  2), 82.5 %
Arasan v7.4                   :  20 (+ 13,=  5,-  2), 77.5 %
Resp v0.19                    :  20 (+ 14,=  4,-  2), 80.0 %
Delfi v4.5                    :  20 (+  7,=  6,-  7), 50.0 %
Zappa v1.0                    :  20 (+ 12,=  3,-  5), 67.5 %
Jonny v2.70                   :  20 (+ 11,=  4,-  5), 65.0 %
Spike v0.9                    :  20 (+  7,=  8,-  5), 55.0 %
Glaurung v0.2.1DC             :  20 (+ 10,=  7,-  3), 67.5 %
Robert Allgeuer
 
Posts: 124
Joined: 28 Sep 2004, 19:09
Location: Konz / Germany


Return to Winboard and related Topics

Who is online

Users browsing this forum: No registered users and 40 guests