Info on testing a new engine sought

[Anonymous]

My new engine is getting close to being ready for some limited testing. What types of matches, (which will have to be with weak engines), would you recommend for a new engine?

I don't have the engine set up for any engine to engine interface yet, but that's on the list. Can Winboard handle automated testing with the help of bat files, or a utility?

Any info you can toss my way, I will be most grateful for. I can already see that the testing part of working with the engine, will involve quite a lot of time and effort.

Thanks a lot.

David

[Edsel Apostol]

My new engine is getting close to being ready for some limited testing. What types of matches, (which will have to be with weak engines), would you recommend for a new engine?

I don't have the engine set up for any engine to engine interface yet, but that's on the list. Can Winboard handle automated testing with the help of bat files, or a utility?

Any info you can toss my way, I will be most grateful for. I can already see that the testing part of working with the engine, will involve quite a lot of time and effort.

Thanks a lot.

David

Hi there David,

I suggest that you implement winboard or UCI protocol in your program.

Download some chess engines out there with source code available and

install them all in Arena GUI together with your program. Play automated

tournaments in how many games you want with different time controls for

you to have different data results to compare. You could tweak your

source code according to the results of the tournament and you could

based it on the source code of the opponent of your program. It's just a

suggestion. I'm just a novice programmer. Hope it could help.

Edsel.

[Joachim Rang]

My new engine is getting close to being ready for some limited testing. What types of matches, (which will have to be with weak engines), would you recommend for a new engine?

I don't have the engine set up for any engine to engine interface yet, but that's on the list. Can Winboard handle automated testing with the help of bat files, or a utility?

Any info you can toss my way, I will be most grateful for. I can already see that the testing part of working with the engine, will involve quite a lot of time and effort.

Thanks a lot.

David

as long as your engine is weak and you are not interested in a rating for your engine but just in making it better selfplay is the best way. Play superfast blitz and tune some parameters, only when you don'T find any more improvements I would let it play against other engines and with longer time control.

You shoul play many games to get statistically significant results though.

regards Joachim

[Roman Hartmann]

My new engine is getting close to being ready for some limited testing. What types of matches, (which will have to be with weak engines), would you recommend for a new engine?

I don't have the engine set up for any engine to engine interface yet, but that's on the list. Can Winboard handle automated testing with the help of bat files, or a utility?

Any info you can toss my way, I will be most grateful for. I can already see that the testing part of working with the engine, will involve quite a lot of time and effort.

Thanks a lot.

David

Hi,
I recommend to implement either winboard and/or UCI. UCI seems to be a bit simpler to implement (at least that's my experience). The advantage is that testing get's easier with a nice GUI like winboard than under the console. If you test your engine in engine vs engine matches don't forget to use a book or they will play the same games again and again.

You can setup batch files to play engine-engine matches with winboard. Here is a nice link to that topic: http://www.aarontay.per.sg/Winboard/batch.html

Roman

[Richard Allbert]

Hi,

Yes - you should implement either the winboard or the UCI protocol. I agree that UCI is the easier of the two, although it is essential that your engine can.... 1. be interrupted during the search and 2. understand an FEN string. Otherwise you'll have to use the winboard protocol.

Once a protocol has been implemented, an alternative method of testing is to use EPD suites (FEN understanding required) - there are lots available on the internet, and on Dann Corbit's site. This is often quicker than playing lots of games - to get a reliable result from games you need over 100, and at least 25 minutes for all moves. AN epd suite of 1000 positions with 10s per position takes a few hours...

For a new engine, WAC.epd or ECM98.epd are good ones to start with.

GUI's such as Arena (free!) and Fritz run these suites automatically.

Regards,

Richard