March 21, 2012
Machine domination in graphs
When chess machines passed human performance, how fast did it happen?
Here is a plot of the Elo ratings of chess systems from the Swedish Chess Computer Association tournaments:
Of course, the data is noisy - I don't really think chess software has become worse over the last few years. In this case it might just have been a narrowing of the field.
Plotting them as a function of clock speed produces the following intriguing graph, showing some jumps in software performance on the same hardware:
Combining the first graph with List of FIDE chess world number ones - Wikipedia, the free encyclopedia we get the following relative graph:
It shows nicely how humans have not been getting much better, while the machines improve from nearly hopeless to superhuman.
Of course, this is slightly iffy: while SSDF has tried to calibrate against humans one can always question the commensurably.
One can also calculate the probability of winning against players of different ranks (again assuming commensurability):
Looking at 6 game matches the probability of winning the majority looks like this, a bit sharper:
Human amateurs were already having trouble at the start, but grandmasters could be confident at the start of the 90s... and were outclassed by the end.
Observant readers might wonder why Kasparov lost to Deep Blue in 1997. The reason is that this tournament data uses consumer machines, not the supercomputer IBM pitted against him. For 1997 the winner in the tournament was HIARCS 6.0, running on a 49 MB P200 MMX. Deep blue was a 30 node system with a 120 MHz processor per node, plus 480 special purpose chess chips. It ran at 11.38 GFLOPS on numerics. A pentium MMX at 200 MHz is around 23.53 MFLOPS. So Deep Blue was at least 484 times more powerful (and likely quite a bit more, thanks to the special chess chips and parallelism).
So the mystery of why it won so early is solved: the brute force allowed IBM to get extra performance equivalent to perhaps a decade development.
Posted by Anders3 at March 21, 2012 05:45 PM