Hint: Valgrind bound checker

[Daniel Mehrmann]

Hello,

i had the problem that my engine has a lot of bound problems and my memory management leak at some points. During the last days i used the boundchecker "Valgrind", http://www.valgrind.org , and could fix a lot of errors This program is really nice and shows you your errors and is _very_ easy to use )

No much changes in my Makefile:

Code: Select all

#
# Makefile for Homer
# written by Daniel Mehrmann
#

# Compiler
CC = gcc

# Compilerflags
# -mcpu is removed since gcc 3.4
#
# valgrind debug
CFLAGS = -O0 -g -fomit-frame-pointer -pipe

#CFLAGS = -O3 -march=i686 -fomit-frame-pointer -pipe


Output looks like:

Code: Select all

daniel@fis615:~/current> valgrind --leak-check=yes ./homer
==24321== Memcheck, a.k.a. Valgrind, a memory error detector for x86-linux.
==24321== Copyright (C) 2002-2003, and GNU GPL'd, by Julian Seward.
==24321== Using valgrind-2.0.0, a program supervision framework for x86-linux.
==24321== Copyright (C) 2000-2003, and GNU GPL'd, by Julian Seward.
==24321== Estimated CPU clock rate is 1541 MHz
==24321== For more details, rerun with: -v
==24321==
Chessprogram Homer
Copyright Daniel Mehrmann 2003-2005
http://www.homer-chess.com, info@homer-chess.de
Version: 1.02
Build: 18

This program is free for personal use
compiled Oct 10 2005 14:20:53
Reading homer.ini

Hashtable    = 8 MB
Pawn value   = 100 points
Knight value = 420 points
Bishop value = 420 points
Rook value   = 620 points
Queen value  = 1240 points
Max QS depth = 12 ply
Ponder       = off
Logging      = off (homer.log)

Init hash memory. This may take a moment...

info Allocate main hashtable memory,  rc = 4125

info Allocate depth hashtable memory,  rc = 2062

info Allocate eval hashtable memory,  rc = 328

info Allocate pawn hashtable memory,  rc = 234

info Allocate memory for hashtables,  rc = 7686
Homer is ready for playing
uci
id name Homer 1.02
id author Daniel Mehrmann
option name Ponder type check default false
option name HistoryPruning type check default true
option name MultiCutPruning type check default true
option name NoiseReduction type check default true
option name HumanPlayStyle type check default false
option name NullMove type combo default secure var secure var strong var low var agressive
option name Hash type spin min 8 max 512 default 8
option name Pawn type spin default 100 min 50 max 150
option name Knight type spin default 420 min 370 max 470
option name Bishop type spin default 420 min 370 max 470
option name Rook type spin default 620 min 570 max 620
option name Queen type spin default 1240 min 1190 max 1290
option name Max_QS_Depth type spin default 12 min 0 max 16
option name Clear_Hash type button
option name MultiPV type spin min 1 max 6 default 1
option name UCI_EngineAbout type string default Homer 1.02 by Daniel Mehrmann, http://www.homer-chess.com, info@homer-chess.de
option name UCI_AnalyseMode type check default false
option name UCI_ShowCurrLine type check default false
option name UCI_Chess960 type check default false
uciok
go infinite
info depth 1 seldepth 1
info score cp 0 depth 1 seldepth 4 nodes 895 nps 3162 time 284  pv b1c3 b8c6
info depth 2 seldepth 2
info score cp 0 depth 2 seldepth 2 nodes 915 nps 3060 time 299  pv b1c3 b8c6
info depth 3 seldepth 3
info score cp 49 depth 3 seldepth 5 nodes 1211 nps 3317 time 365  pv b1c3 b8c6 g1f3
info depth 4 seldepth 4
info score cp 0 depth 4 seldepth 8 nodes 2803 nps 4062 time 690  pv b1c3 b8c6 g1f3 g8f6
info depth 5 seldepth 5
info score cp 31 depth 5 seldepth 12 nodes 19400 nps 6148 time 3155  pv g1f3 e7e6 b1c3 f8c5
info depth 6 seldepth 6
info score cp 14 depth 6 seldepth 12 nodes 34519 nps 6482 time 5325  pv g1f3 g8f6 b1a3 b8c6 a3c4
info depth 7 seldepth 7
info score cp 25 depth 7 seldepth 13 nodes 66782 nps 6756 time 9884  pv b1c3 g8f6 d2d4 b7b6 g1f3 d7d5
info depth 8 seldepth 8
info score cp 9 depth 8 seldepth 14 nodes 88191 nps 6745 time 13074  pv b1c3 g8f6 d2d4 d7d5 d1d3 b8c6 c1f4
info score cp 9 depth 8 seldepth 18 nodes 189522 nps 6497 time 29168  pv b1c3 g8f6 d2d4 d7d5 d1d3 b8c6 c1f4
info depth 9 seldepth 9
info score cp 16 depth 9 seldepth 18 nodes 236457 nps 6523 time 36245  pv b1c3 g8f6 d2d4 d7d5 g1f3 c8g4 c1f4 b8c6
info score cp 16 depth 9 seldepth 18 nodes 277341 nps 6381 time 43462  pv b1c3 g8f6 d2d4 d7d5 g1f3 c8g4 c1f4 b8c6
info depth 10 seldepth 10
info score cp 15 depth 10 seldepth 20 nodes 377338 nps 6408 time 58884  pv b1c3 g8f6 d2d4 d7d5 g1f3 b8c6 c1f4 f6h5 d1c1 c8f5
stop
bestmove b1c3
quit

info string Homer: Hope see you soon again :)
==24321==
==24321== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)
==24321== malloc/free: in use at exit: 0 bytes in 0 blocks.
==24321== malloc/free: 5 allocs, 5 frees, 7872364 bytes allocated.
==24321== For counts of detected errors, rerun with: -v
==24321== No malloc'd blocks -- no leaks are possible.
daniel@fis615:~/current>


[url][/url]

[Alvaro Begue]

Memory leaks in a chess program? Why are you allocating memory at all in a chess program? That just sounds like a bad idea.

[Daniel Mehrmann]

Memory leaks in a chess program? Why are you allocating memory at all in a chess program? That just sounds like a bad idea.

We do this for hashtable size

btw, that's funny:

Code: Select all

--31159-- supp:   15 Ugly strchr error in /lib/ld-2.3.3.so


[Alvaro Begue]

Memory leaks in a chess program? Why are you allocating memory at all in a chess program? That just sounds like a bad idea.

We do this for hashtable size

Oh, sure. But that's a single place where you allocate memory, and it shouldn't be that difficult to get right. If you are programming in C++, you can actually use a vector for this and then it's really hard to leak memory.

[Richard Allbert]

But you don't know the size the table will be until the program boots..?

Plus, a vector would be slow for access, compared with a simple array and a pointer? (Well, it was with my engine, but that has a lot of speed issues).

Richard

[Alvaro Begue]

But you don't know the size the table will be until the program boots..?

STL's vectors can be resized at runtime.

Plus, a vector would be slow for access, compared with a simple array and a pointer? (Well, it was with my engine, but that has a lot of speed issues).

Not really. You can get the address of the first element of the vector as a pointer and use it for accessing the container. Of course, this is only safe to do if you know the size of the vector is not changing.

[Pradu]

But you don't know the size the table will be until the program boots..?

STL's vectors can be resized at runtime.

Plus, a vector would be slow for access, compared with a simple array and a pointer? (Well, it was with my engine, but that has a lot of speed issues).

Not really. You can get the address of the first element of the vector as a pointer and use it for accessing the container. Of course, this is only safe to do if you know the size of the vector is not changing.

How does that make it much different...

[Alvaro Begue]

You can get the address of the first element of the vector as a pointer and use it for accessing the container. Of course, this is only safe to do if you know the size of the vector is not changing.

How does that make it much different...

Doing it that way you know that it's as fast as accessing it through a pointer, because you are accessing it through a pointer. Actually, looking at the implementation of operator[] on my compiler, it is as fast as what I just proposed.

My point is that there is no performance penalty to using vector.

[Dann Corbit]

But you don't know the size the table will be until the program boots..?

STL's vectors can be resized at runtime.

Plus, a vector would be slow for access, compared with a simple array and a pointer? (Well, it was with my engine, but that has a lot of speed issues).

Not really. You can get the address of the first element of the vector as a pointer and use it for accessing the container. Of course, this is only safe to do if you know the size of the vector is not changing.

This is never safe. It assumes a particular inner structure to an opaque object type. A vector could be created as a Judy array or a skiplist or a thousand other possibilities. And it could change tomorrow.

[Alvaro Begue]

This is never safe. It assumes a particular inner structure to an opaque object type. A vector could be created as a Judy array or a skiplist or a thousand other possibilities. And it could change tomorrow.

This is safer than you might think. This discussion explains the situation: http://groups.google.com/group/borland. ... rt=0&num=2

[Scott Gasch]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

Scott

[Anonymous]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

I agree, this is a very good method, which I am using myself. Unfortunately, it does not work for C++. Also, I do not have any gcc for Windows/DOS that does support it. Did you try it? To me, it seemed a lot of hazzle to set it up and try it out. On Linux, the typical installation method of gcc is to bootstrap it from sources. You only have to apply the patches then. For DOS or Windows, typically users only download the binaries. To really bootstrap the compiler, lots of other utilities have to be installed (which are just there on typical Linux distributions).

Regards,
Dieter

[Dann Corbit]

This sounds like a project that works with both C and C++:
http://www.doc.ic.ac.uk/teaching/projec ... ffield.pdf

Maybe it can be generally available.

[Scott Gasch]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

I agree, this is a very good method, which I am using myself. Unfortunately, it does not work for C++. Also, I do not have any gcc for Windows/DOS that does support it. Did you try it? To me, it seemed a lot of hazzle to set it up and try it out. On Linux, the typical installation method of gcc is to bootstrap it from sources. You only have to apply the patches then. For DOS or Windows, typically users only download the binaries. To really bootstrap the compiler, lots of other utilities have to be installed (which are just there on typical Linux distributions).

Regards,
Dieter

No, I apply the patch on a UNIX system and do boundschecking there. I don't know about gcc on DOS/Windows.

Scott

[Daniel Mehrmann]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

I agree, this is a very good method, which I am using myself. Unfortunately, it does not work for C++. Also, I do not have any gcc for Windows/DOS that does support it. Did you try it? To me, it seemed a lot of hazzle to set it up and try it out. On Linux, the typical installation method of gcc is to bootstrap it from sources. You only have to apply the patches then. For DOS or Windows, typically users only download the binaries. To really bootstrap the compiler, lots of other utilities have to be installed (which are just there on typical Linux distributions).

Regards,
Dieter

yeah, i'll test it tommorrow

Best,
daniel

[Daniel Mehrmann]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

I agree, this is a very good method, which I am using myself. Unfortunately, it does not work for C++. Also, I do not have any gcc for Windows/DOS that does support it. Did you try it? To me, it seemed a lot of hazzle to set it up and try it out. On Linux, the typical installation method of gcc is to bootstrap it from sources. You only have to apply the patches then. For DOS or Windows, typically users only download the binaries. To really bootstrap the compiler, lots of other utilities have to be installed (which are just there on typical Linux distributions).

Regards,
Dieter

yeah, i'll test it tommorrow

Best,
daniel

How shocking, this is a beast!
First try out of box:

Code: Select all

go infinite
info depth 1 seldepth 1
info score cp 0 depth 1 seldepth 4 nodes 897 nps 10939 time 82  pv b1c3 b8c6
info depth 2 seldepth 2
info score cp 0 depth 2 seldepth 2 nodes 917 nps 10916 time 84  pv b1c3 b8c6
info depth 3 seldepth 3
info score cp 49 depth 3 seldepth 5 nodes 1216 nps 11259 time 108  pv b1c3 b8c6 g1f3
info depth 4 seldepth 4
search.c:1407:Bounds error: array reference (16) outside bounds of the array.
search.c:1407:  Pointer value: 0x80af610
search.c:1407:  Object `PieceValue':
search.c:1407:    Address in memory:    0x80af5d0 .. 0x80af60b
search.c:1407:    Size:                 60 bytes
search.c:1407:    Element size:         4 bytes
search.c:1407:    Number of elements:   15
search.c:1407:    Created at:           extern.c, line 56
search.c:1407:    Storage class:        static


Valgrind doesn't saw it!

Best,
Daniel

[smcracraft]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

Scott

I use

gcc ... -fbounds-check ...

The gcc I use currently is 3.3.3

I tried it with -O2 optimization with my program and compared the
search speed without the bounds check and they were very close.

So I am going to -fbounds-check compile from now. This represents
an important improvement to my development praxis.

Thanks,

Stuart

[diepeveen]

Hi Daniel

Valgrind boundschecker is ugly bad, it can't even detect simple things if i try it with it, of course it is what you compare it with. Note that in C++ you can make your own templates to boundscheck arrays in a pretty easy way.

Rational Enterprise Suite is finding most of all boundscheckers over here.

In general most boundscheckers suck for parallel debugging, you'll have to write your own testbeds for it. Especially with 16 cpu's and above that's getting tricky

[diepeveen]

If you apply a patch (available on a link from the gcc homepage) to gcc sources, rebuild your compiler and then compile your chess code with -fbounds_checking then gcc will do bounds checking for you too. This is my preferred method of catching this kind of thing.

Scott

Hi Scott,

Long time not seen online!

If your engine is single threaded then this is actually a very good boundschecker for C. Not so much C++. I've used this thing for years actually, but last years it is a real problem as default diep has about 4 threads in processor 0.

Other boundscheckers had problems with multiple processes, but rational seems to be the superior debugger at the moment

Note diep3d gui is C++, only my code inside it is "extern C {"