Open Sourcing your Engine

[Tom Likens]

Hello Everyone,

Well, I'm just about to the point where I want to get serious about starting a new engine.
Djinn has been fun (it's my 2nd "real" chess engine), but I'm to the point where a new start
would revitalize me. I'm also tired of bitboards. I'm sure I haven't exhausted their possibilities
but I'd like to start something new. I'm leaning towards some type of 0x88, 12x16 or most likely
16x16 (one of Christophe Theron's old CCC posts has sold me on its advantages over
pure 0x88). Also trying (and learning) a number of new techniques would be fun.

Anyway, that's not my real question, what I'm really debating is should I open source
the engine. Both Tord and Daniel did it, not to mention Fruit and I'm wondering if they've
been happy with their decision. I'm also thinking about making Djinn's source open as
well, but again I haven't decided anything firm yet. I'm not worried about giving away secrets,
since Djinn (and I'm sure any new engine) is so weak that copying its internal workings would
only set other programs back a decade or two!!

Anyway, thoughts and comments would be appreciated.

regards,
--tom

[Dann Corbit]

There will be rewards and punishments.

People will learn from and enjoy your program. Someone will eventually make a few tweaks and stick their name on it and say "See what I did!"

Eventually, they will probably get caught. The question is if the hassle of dealing with a mess like that and various criticisms is worth the reward to you.

You will also get emails asking about this and that on the inner workings.

You may get some good ideas on rare occasion, but mostly you will just have to be a teacher.

Bottom line: Do what you want to do. Weigh rewards and penalties and choose whatever makes you happy. For sure, you will have some people up in arms no matter what you choose on either side and some happy and some neutral.

[Tord Romstad]

There will be rewards and punishments.

People will learn from and enjoy your program.

That's what we all hope, of course. I have never received very much feedback about my source code, but those few programmers who have contacted me with comments or questions about my code have mostly been very positive. I would also like to think that there are many happy users for every e-mail I receive.

At least there seems to be a great interest for the source code. One of the nice new things I can do after opening www.glaurungchess.com thirteen days ago is to inspect download statistics for the individual files. I am pleasantly surprised by the high number of downloads of the source code. Between September 22 and October 4, the source code for Glaurung Mainz has been downloaded 50 times, and the source code for Scatha 0.2.4 11 times. I guess the download frequency is much higer directly after the release of a new version (Glaurung Mainz was released in the beginning of August, IIRC).

Someone will eventually make a few tweaks and stick their name on it and say "See what I did!"

Perhaps. It is likely that you will have to wait rather long for this to happen, though. Glaurung has been open source for about a year, and I am pretty sure nobody has tried to clone it yet.

You will also get emails asking about this and that on the inner workings.

From a selfish point of view, this is the really big advantage. It doesn't happen often, but it is very nice when it does.

Bottom line: Do what you want to do. Weigh rewards and penalties and choose whatever makes you happy.

My personal experience is that the rewards are modest, but the penalties are practically non-existant. I haven't experienced any unpleasant consequences of open-sourcing Glaurung.

I would also like to add that I think the existence of strong open source programs is of great value to the community.

Tord

[Tom Likens]

You may get some good ideas on rare occasion, but mostly you will just have to be a teacher.

Thanks Dann,

I don't really mind being a teacher. I owe so many in the community who helped me in
the past, that paying back a bit would be a good thing.

regards,
--tom

[Dann Corbit]

I read the source code of every program that has source code.

I am pretty sure I have learned something from every one I looked at.

I guess that about one third have had an email from me with some kind of suggestion/tweak/complaint.

I have probably offered suggestions to 60 different authors. Maybe more. It's just a guess. Perhaps half of them have put something I suggested into their code.

[Jaime Benito de Valle]

hi,

I've always offered my source to those who were interested only rather than leaving it there for everyone to download it. Only a few were interested in checking it -one of them being Tord-, but I doubt anyone could benefit from the messy code of a weak engine. Of course, if no one was cloned Glaurung yet... who is going to clone Ayito and why?? The risk is that your engine gets so strong and popular that people can't resist the temptation, like what happened with Fabien. So... maybe a few may get some ideas and inspirations from your code (hopefully), and chances of cloning will be rather negligible, but think twice before publishing your code when your engine hangs in the top 20.
Regards,

Jaime

[Pradu]

This topic is completly a matter of personal preference and beliefs therefore any argument given here can be viewed in a different perspective.

[Roman Hartmann]

For a some time my source code was freely available as well. I thought I might get some feedback or imagined that someone who's just starting with chess programming as well might take advantage of it. I never got any feedback regarding the sources just one email with someone asking how he could compile the sources.
I took the sources down from my site after I had a glimpse into the sources of some other chess engines. It was just too embarassing. Everything looked so nice and clean and easy understandable in the other sources. I just realized that no one would take the effort and wade through my mess.

best regards
Roman

[Uri Blass]

I think that it may be interesting if people do not give only the final source code but a lot of sources so people may follow the process of programming.

Unfortunately I do not know of a way to save the process of programming
but people can at least save their source code often and later release 100 different source codes (code1,code2,code3,....code100) and people may learn not only from code100 that is the last version but from code1 that is still not a playing chess program.

Note that saving the process of programming can be also productive for programmers.

I remember reading about a case when GCP did not save his old code
but only his old exe and found that the old code performed better but could not go back to the old code to implement something because he has not the source.

This type of problem will not happen if the computer saves atomatically the old version and I do not understand why compilers do not have an option to save all the process.

Suppose that a programmer spends total time of 1000 hours on a program that is a lot of time(only in writing code and I do not include design time).

suppose that he writes 10,000 chars in an hour that is not bad.

10,000*1000<10Mbytes and it is possible to compress char by less than 8 bits because all the letters a-z and the digits 0-9 and some other things are clearly less than 128.

10 Mbytes is very little in terms of memory for computers so I do not understand what is the reason that the default option of the computer is not to remember all history of programming.

Uri

[Tom Likens]

Hey Tord,

Thanks for the feedback. I think one of the reasons the average program has become so
much stronger is because of the availability of the source code to reasonably strong
programs. It really helps a beginner to have something decent to look at so they
don't waste their time on fruitless (pun intended) endeavors. For this learning, having
*all* levels of programs and source code available is an advantage. That's why I believe
TSCP is still so popular, even though it's relatively weak.

Also I've tried the closed source thing, it might be fun to open source things for a change.

regards,
--tom

[Roman Hartmann]

I think that it may be interesting if people do not give only the final source code but a lot of sources so people may follow the process of programming.

Unfortunately I do not know of a way to save the process of programming
but people can at least save their source code often and later release 100 different source codes (code1,code2,code3,....code100) and people may learn not only from code100 that is the last version but from code1 that is still not a playing chess program.
[ ... ]

Uri

Hi Uri,
I had a similar problem once. I didn't save my old code and suddenly had a bug which was not present in older versions. Took me quite some time to track that bug down.
Since then I'm creating every day a new folder (like roce_4_10_05) and copy the sources from the previous folder into the new folder. That way I don't lose more than the work of 1 day if something goes wrong and I can always see what I did over the last days/weeks.

Roman

[Tom Likens]

I think that it may be interesting if people do not give only the final source code but a lot of sources so people may follow the process of programming.

Hey Uri,

I have three ways of backing up my source code:

1. I use CVS whenever I program
2. I will once a week archive the source code, makefile, etc.
3. I burn a DVD of the whole mess once a month

So far this has worked. You may already do this, but if you don't you should look into
some type of source control. CVS and Subversion are nice because they are free
and well proven. Once you get them set up, they are fairly non-intrusive, and they can
save your bacon if something goes wrong. You can also use them to track your progress,
since all of them have history-type commands.

One of the ideas I'm toying with is setting up a web blog of the new engine's
development. That might be of interest to new programmers, and I would
find it fun. Also that would force me to flesh out my webpage, which is something I've
been threatening to do forever.

regards,
--tom

[Tom Likens]

For a some time my source code was freely available as well. I thought I might get some feedback or imagined that someone who's just starting with chess programming as well might take advantage of it. I never got any feedback regarding the sources just one email with someone asking how he could compile the sources.
I took the sources down from my site after I had a glimpse into the sources of some other chess engines. It was just too embarassing. Everything looked so nice and clean and easy understandable in the other sources. I just realized that no one would take the effort and wade through my mess.

best regards
Roman

Hello Roman,

Yes, this will likely be my experience too. Still you never know, you might help that one
person or obtain that *one* idea that improves your program by 200 elo overnight!

BTW, I'd be interested in your source code, if you're still of a mind to let others see it.

regards
--tom

[Tom Likens]

hi,

I've always offered my source to those who were interested only rather than leaving it there for everyone to download it. Only a few were interested in checking it -one of them being Tord-, but I doubt anyone could benefit from the messy code of a weak engine. Of course, if no one was cloned Glaurung yet... who is going to clone Ayito and why?? The risk is that your engine gets so strong and popular that people can't resist the temptation, like what happened with Fabien. So... maybe a few may get some ideas and inspirations from your code (hopefully), and chances of cloning will be rather negligible, but think twice before publishing your code when your engine hangs in the top 20.
Regards,

Jaime

Hey Jaime,

Actually, Ayito seems to be getting stronger. I agree about your top 20 comment. If that
were my situation I might be more hesistant to release the code. "Luckily", I'm not in that boat.

Of course, the flip side to this is that people have cloned TSCP, so who knows.

regards,
--tom

[Roman Hartmann]

Hello Roman,

Yes, this will likely be my experience too. Still you never know, you might help that one
person or obtain that *one* idea that improves your program by 200 elo overnight!

BTW, I'd be interested in your source code, if you're still of a mind to let others see it.

regards
--tom

Hi Tom,
I don't mind to send my code to other fellow chess programmers, of course. I doubt anyone can make much use of it though as it's really a mess. Just provide me with an email adress (PM or email me) and you will get the sources.

best regards
Roman

[Dann Corbit]

For a some time my source code was freely available as well. I thought I might get some feedback or imagined that someone who's just starting with chess programming as well might take advantage of it. I never got any feedback regarding the sources just one email with someone asking how he could compile the sources.
I took the sources down from my site after I had a glimpse into the sources of some other chess engines. It was just too embarassing. Everything looked so nice and clean and easy understandable in the other sources. I just realized that no one would take the effort and wade through my mess.

best regards
Roman

I sent you several emails. Don't those count?

[Roman Hartmann]

I sent you several emails. Don't those count?

Hi Dann,
you sent me several emails? I didn't got a single one. Just give me your email with a PM and you get the sources in no time.

best regards
Roman

[Dann Corbit]

I sent you several emails. Don't those count?

Hi Dann,
you sent me several emails? I didn't got a single one. Just give me your email with a PM and you get the sources in no time.

best regards
Roman

No, my mistake.

I had you confused with another program author.

I have never seen the source code for your engine.

[Ron Murawski]

I think that it may be interesting if people do not give only the final source code but a lot of sources so people may follow the process of programming.

Unfortunately I do not know of a way to save the process of programming
but people can at least save their source code often and later release 100 different source codes (code1,code2,code3,....code100) and people may learn not only from code100 that is the last version but from code1 that is still not a playing chess program.

Note that saving the process of programming can be also productive for programmers.

I remember reading about a case when GCP did not save his old code
but only his old exe and found that the old code performed better but could not go back to the old code to implement something because he has not the source.

This type of problem will not happen if the computer saves atomatically the old version and I do not understand why compilers do not have an option to save all the process.

Suppose that a programmer spends total time of 1000 hours on a program that is a lot of time(only in writing code and I do not include design time).

suppose that he writes 10,000 chars in an hour that is not bad.

10,000*1000<10Mbytes and it is possible to compress char by less than 8 bits because all the letters a-z and the digits 0-9 and some other things are clearly less than 128.

10 Mbytes is very little in terms of memory for computers so I do not understand what is the reason that the default option of the computer is not to remember all history of programming.

Uri

I agree with Tom Likens. The way to preserve the programming process is with a Version Control System. For myself I set up a Subversion server and it has helped me tremendously. If you have an interest in researching Version Control Systems you can read this webpage

Ron

[Daniel Shawul]

Hi Tom

I have three ways of backing up my source code:

1. I use CVS whenever I program
2. I will once a week archive the source code, makefile, etc.
3. I burn a DVD of the whole mess once a month

i also back up when i make significant changes.
But i don't do step 3 of yours, and guess what happened?
My HD crashed and i lost all the source code of DanChess.
Fortunately, i got a relatively new one from the email i sent to Dann.
that is one advantage of open sourcing your engine

I usually check out all the open source engines i know.
I understand most of the engines with one exception,ApilChess!
There may not be many feedbacks on your source code.
I myself never sent any bug report ,when i see one,to any author.
So may be the feed backs will come, when i start sending one.

"bug reproting" regards
Daniel