I just looked at you posting and thought about it a little.
I have some test positions that I regularly run after some changes, to ensure that I haven't broken the engine with a change. I run them from a batch file
and I have some test at increasing depth. When I reach a depth that starts to take long enough that I get impatient, I have PAUSE statements in the batch file to allow me to duck out there if I want to get on with development.
But my tests are either extremely shallow - eg
depth one: test that queening a pawn is prefered to taking a minor piece, etc
depth three: find king and queen knight fork, and pin and capture in endgame position, skewers, etc
or possibly deeper but very specific, eg a pawn needing to run immediately to queen, or a king needing to chase immeidately to prevent an enemy pawn queening.
The tests I have of the depth you mention are just a few checkmates that I have taken from my engines games, mate in 3, mate in 4, and mate in 5.
There are plenty of those around, they are easy to create by just taking finished games and going back a few moves, and I don't believe they are that useful for a couple of reasons. 1. They are usually from an already "clearly" won position. Since checkmate is actually possible I suppose most of the alpha-beta cutoffs are going one way in some sense.
I also have some sanity tests with AVOID MOVE (am) used in EPD files instead of BEST MOVE (bm).
They are on the lines of simply testing that pieces are not thrown away needlessly, eg king not stepping away from its pawn or rook in certain KPK and KRK endgame.
Anyway, for whatever reason, I have noticed that the checkmate tests, the basic sanity tests and the chess tactics test are complemetary. Some coding errors will leave all the checkmates working fine, but throw up errors in pawn race tests, and vice versa.
The reason I don't have many test around depth 6 is difficulty in seeing the chess logic for myself clearly - you soon get into depths with many combinations and you start wondering if there isn't a deeper insight that means the less obvious move is actually better. I have seen this with well known test positions (Bratco-Kopec, excuse errors if I've remembered the names incorrectly), and running Stockfish and Houdini on them. When these engines start disagreeing with some standard analysis at depth 19, who is really to say whether they have found a deeper insight, or merely failed to spot the solution.
What did come to mind thinking about your question was a file that I have downloaded called ECM.EPD I think. As I understand it, the positions are from a book called The Encyclopedia of chess middlegames. I then looked at the file and decided that many of the positions are actually deeper than what you are asking for. But there are nearly a thousand positions and you are not looking for that many, so chances are there are enough positions in there of the right depth.
So which are the right ones?
I played a little with a Houdinin and ECM.EPD and the first few positions seem to be a mixture of things Houdini spots in 5 ply, 7 ply and 9 ply. It does seem to be biased toward the deeper problems, but it seems there will be enough of the 5 or 7 ply type problems to satisfy you if you have the patience to click analyse with a good fast engine to classify a problem and edit it into your own specific file if it meets your criteria.
With nearly 1000 problems I don't intend to continue looking at them for too long but Houdini found positions 7 and 9 in 5 ply, and positions 2 and 3 in 7 ply searches.
- Code: Select all
3rr1k1/pb3pp1/1p1q1b1p/1P2NQ2/3P4/P1NB4/3K1P1P/2R3R1 w - - bm Rxg7+; id "ECM.007";
3r4/1b2k3/1pq1pp2/p3n1pr/2P5/5PPN/PP1N1QP1/R2R2K1 b - - bm Rxh3; id "ECM.009";
7r/1p2k3/2bpp3/p3np2/P1PR4/2N2PP1/1P4K1/3B4 b - - bm Bxf3+; id "ECM.002";
4k3/p1P3p1/2q1np1p/3N4/8/1Q3PP1/6KP/8 w - - bm Qb5; id "ECM.003";
Nothing much seems to be found on an even ply search like 6. This is probably because at even depth the opponent is on the move. Sometimes a search does thro up the good move on an even ply during analysis. I guess this is where there are a few candidate moves and an opponent move refutes the current best move, allowing another to rise to the top.
I may find a few more positions in the ECM.EPD file that are in the 5-7 ply range later. It should be of use to me too. I like to have things ordered by depth of search as there is no point in trying a long deep search with a clearly broken engine. So I do the short search problems first.
, SharpChess is still a relatively weak engine, and usually only searches between 6-8 plys of depth in 15-30 second search . But, many of the computer test suites expect solutions that require deep searches, or are designed to be run over long periods.