New engine rating system : the Lars Scale of Lousyness
Posted: 05 Feb 2011, 14:09
During a massive brainstorming on OpenWar's chat, it appeared to us that a more accurate and realistic scale could be defined to rate chess engines, that would be a lot more handy than the now aging Elo scale, that has proven to not be able to grasp emotional and psychological states of mind of the testers and users.
We call it the Lars Scale of Lousyness (or LSL). The idea being that the more sucky the engine, the more Lars points it receives. The method is still in development and opened for discussion, but here are some elements to take into consideration :
[*] The strength, in Elo. This should give up to 1000 Lars points (for an engine being under 1000 Elo) and at least 1 Lars (Engine being >3000 Elo)
[*] The Engine crashes : +200 Lars
[*] The Engine doesn't start : + 500 Lars
[*] The engine has no opening book : + 100 Lars
[*] The engine is a proven clone : +100 Lars
[*] The engine is suspected of being a clone : + 50 Lars (1)
[*] The engine isn't swedish : + 10 Lars
[*] The engine lacks important functions : + 20 Lars by function
[*] Time loss in 1min game : + 5 Lars (2)
[*] The engine doesn't run on a Pentium I : + 50 Lars (3)
[*] Displays wrong version number : + 30 Lars (4)
[*] Not WB 3 compliant : + 15 Lars (2)
[*] Doesn't play variants : + 25 Lars (2)
[*] Play illegal moves : + 55 Lars
[*] Can only play as white : + 35 Lars (2)
[*] Can only play as black : + 35 Lars (2)
[*] Can't give Mate in One : + 25 Lars (4)
[*] Don't exit memory : +15 Lars (2)
[*] Steals opponent's time : +20 Lars (2)
[*] The author is unknown : +15 Lars
[*] Obfuscates informations (nps, depth...) : +30 Lars
[*] Engine not publicly released : +25 Lars (5)
[*] Engine is too memory hungry : +35 Lars
[*] Newest version is weakier than the older one : +17 Lars (6)
[*] The engine is coded in Java : +45 Lars (5)
[*] Lars told you your engine is lousy : +51 Lars (2)
[*] Who the heck is Lars ? : +15 Lars
Features to have no to get +20 Lars/each :
[*] SMP
[*] Multi-PV
[*] bitbases
[*] Both UCI & Wb compatible
[*] Ponder mode
Also, the Lars scale should be easily convertible to the Olivier Scale of Wonders (or OSW).
This project will give birth to an open-sourced Lars Calculator under GPL.
The project is intended to replace the now deliquescent Elo scale and is still in very early stage of development, so we won't insist enough on how important it is for you to contribute now by any useful (or not) suggestion.
(1) all engines start by receiving this Lars bonus, until proven not guilty
(2) Andrew Fan Contribution
(3) Fonzy Bluemers contribution
(4) Olivier Deville contribution
(5) Lars Hallerström contribution
(6) Tony Mokonen Contribution
We call it the Lars Scale of Lousyness (or LSL). The idea being that the more sucky the engine, the more Lars points it receives. The method is still in development and opened for discussion, but here are some elements to take into consideration :
[*] The strength, in Elo. This should give up to 1000 Lars points (for an engine being under 1000 Elo) and at least 1 Lars (Engine being >3000 Elo)
[*] The Engine crashes : +200 Lars
[*] The Engine doesn't start : + 500 Lars
[*] The engine has no opening book : + 100 Lars
[*] The engine is a proven clone : +100 Lars
[*] The engine is suspected of being a clone : + 50 Lars (1)
[*] The engine isn't swedish : + 10 Lars
[*] The engine lacks important functions : + 20 Lars by function
[*] Time loss in 1min game : + 5 Lars (2)
[*] The engine doesn't run on a Pentium I : + 50 Lars (3)
[*] Displays wrong version number : + 30 Lars (4)
[*] Not WB 3 compliant : + 15 Lars (2)
[*] Doesn't play variants : + 25 Lars (2)
[*] Play illegal moves : + 55 Lars
[*] Can only play as white : + 35 Lars (2)
[*] Can only play as black : + 35 Lars (2)
[*] Can't give Mate in One : + 25 Lars (4)
[*] Don't exit memory : +15 Lars (2)
[*] Steals opponent's time : +20 Lars (2)
[*] The author is unknown : +15 Lars
[*] Obfuscates informations (nps, depth...) : +30 Lars
[*] Engine not publicly released : +25 Lars (5)
[*] Engine is too memory hungry : +35 Lars
[*] Newest version is weakier than the older one : +17 Lars (6)
[*] The engine is coded in Java : +45 Lars (5)
[*] Lars told you your engine is lousy : +51 Lars (2)
[*] Who the heck is Lars ? : +15 Lars
Features to have no to get +20 Lars/each :
[*] SMP
[*] Multi-PV
[*] bitbases
[*] Both UCI & Wb compatible
[*] Ponder mode
Also, the Lars scale should be easily convertible to the Olivier Scale of Wonders (or OSW).
This project will give birth to an open-sourced Lars Calculator under GPL.
The project is intended to replace the now deliquescent Elo scale and is still in very early stage of development, so we won't insist enough on how important it is for you to contribute now by any useful (or not) suggestion.
(1) all engines start by receiving this Lars bonus, until proven not guilty
(2) Andrew Fan Contribution
(3) Fonzy Bluemers contribution
(4) Olivier Deville contribution
(5) Lars Hallerström contribution
(6) Tony Mokonen Contribution