AI Risk : an Historical Perspective Through the Game of Chess

During the last few weeks the conversation on the dangers that AI could present to humankind has turned crazy. The most egregious example is maybe the open letter of the 29th March 2023 by Eliezer Yudkowsky in TIME magazine. If you haven’t read it I encourage you to do so in order to get an idea of where is the conversation now. In this essay I am going to address some of the claims made by Yudkowsky along two axes. The first one is a purely scientific one: there are results in computability and algorithmic complexity that are not discussed. But this is very technical and not very easy to convey to a large audience. So I am going to use a historical precedent in which AIs have already reached superhuman level: chess. Chess has already been used as a proxy to study how information technologies have had an impact on society.

Exponentials all the way down

I am going to make it short, but I am open to any discussion (in the comment section or in sequels) to go further on the purely scientific part. It is very technical: it is about theoretical result in mathematics so not very fit for an essay like this. It is important to note that certain results have been proven, and as such, there is no room for debate. Debating such results would be akin to questioning the basic fact that 2+2=4. While some woke activists may choose to do so, it is not my intention to engage in such discussions here. Therefore, I will keep my discussion brief, but I welcome anyone who may have valid arguments to challenge my claims in good faith. In this article, I will address the fundamental ideas behind the claims made by AI doomers.

1. Exponential progress of intelligence: computational complexity theory is a well established fied of theoretical computer science. GPTs of the world are not going to make the complexity classes to collapse and even stupid problems will remain out of reach of any AI.

2. The size of what is actually (not in theory) computable is not that large: one principle that is not well known is the Landauer Principle. Computing is not energy free. Indeed some energy has to be used to switch a memory bit from 0 to 1 and 1 to 0. If you take the minimal energy expenditure possible (given by our understand of physics) it turns out that you can have an idea of what is possible and what is impossible. It is used in computer security to determine the length of security paramaters. It turns out that it is not possible to just enumerate all integers on 200 bits (that is counting in a computer memory 0,1,2,3 up to 2²⁰⁰) because there is not enough energy on the entirety of our universe. 200 bits is something like 30 characters. This is ridiculously small. So forget about exploring huge spaces exhaustively. This limit applies to any AI as well because they are just computers.

3. GPT will be able to program anything (including itself): this is the clearest point. There is the Rice’s theorem that states that it is impossible for Turing Machines to decide if a program meets non trivial semantics properties. AIs are special cases of Turing Machines.

4. Discontinuity in the progress of intelligence: there is this idea that once a critical point is reached there will be a discontinuity in how the AI behave. The more extreme version is the technological singularity theory of Kurzweil. The one liner summary is : once a critical point is reached the things become different. Either the machine begins to be sentient, or it self improves to infinity etc. There is simply no example of such discontinuity in the field of model of computations. You don’t change the class of what is computable using model M by simply considering larger machines of the same class M. For instance, a finite state automaton will never have the computational strength of a pushdown automaton or a Turing machine. So it is up to the ones who purports this idea to give arguments why this time is different. The basic hypothesis is: size does not matter (with relation to the class of computable functions). If you think it is not true, then just laying down your fears is not going to make it to convince me.

Chess as a proxy

Chess has been used as a proxy to study AI since the inception of computer science. In the first articles of Turing, and the work of Shannon, the question of how to program chess appeared even before computer existed. And its relevance to AI continue because, directly quoting E. Yudkowsky from his TIME open letter:

Valid metaphors include “a 10-year-old trying to play chess against Stockfish 15”

There are many reasons for this. In short (you can refer to this article for a more detailed discussion) it is interesting to study how AI have interacted with chess and the world of chess in general because:

1. It combines many intellectual aspects : from spatial visualisation, to ability to compute, memory, creativity, adaptation to new circumstances etc.

2. The game of chess has not changed: the rules, the board etc. remain the same. In that it is much more stable than any social science aspect (eg IQ measurement) because society changes, and you never have two time the same environment (just knowing the result of previous experiments change the environment).

3. It is possible to measure strength of engines quite precisely: the Elo ranking of engines gives a very reliable scale. Moreover it can be compared to human achievements in the field. The rank of the best chess player is 2853 Elo, to compare with 3575 elo of the best chess program (see here how the Elo rating system works).

4. It is restricted : of course the field is less general than real world but that is precisely the whole point! When you do science you try to isolate things in a lab to have a clearer view of what is going on. By its limited nature (fixed rules, a chessboard is a clearly delimited arena etc.) it makes it easier to see what happens.

5. AI is much stronger than humans for *decades* now, so we can observe what happened to chess and chess life (which can be seen as a microcosm version of society) following the appearance of super-humans AI It is also possible to see how those AI have evolved in this lapse of time.

So lets explore what has been learned along the last quarter of century during which AI was stronger than humans.

1- Discontinuity

The progress of chess software has been fairly linear in terms of elo rating points. The following chart is clear, and the jump (that is less wide than 10 years gap) witnessed is partially expalainable by new hardware not AI progress.

This chart (coming from here) stops in 2018 but the best avalaible, the Lc0 (Leela Chess Zero), software in 2023 around 3574 Elo so the trend is roughly going on as expected.

2-Logarithmic progress

This progress appears linear but is in fact a logarithmic progress because the rise in computing power is exponential. Moreover, as noted by K. Thompson when he was working on Belle adding an extra ply linearily augments the Elo strength of the engine. So at best, since hardware gives you this exponential progression, it means that the AI are making very slow progress (not more than logarithmic otherwise we wouldn’t have a line).

3-AlphaZero and Stockfish

One absent of the previous discussion is AlphaZero. It made the headlines in 2017 by defeating Stockfish (the then best chess software) quite convincingly while having self taught chess. Things were more complex than meet the eye because both engines were not running on the same hardware and Stockfish was refused access to opening knowledge and ending tables. The problem si that AlphaZero is not accessible to the pubic so it is hard to comment on it. But Lc0 is an available chess engine running on the same principles than Alpha Zero. It is just marginally better than Stockfish: they differ on a single Elo point: 3575 vs 3574 on the SSDF rating list. Lc0 won a match 20.5 to 19.5 vs Stockfish. So it is true that it is better but the result show that neural network technologies do not act as an alien and do not present sizeable discontinuities in chess strength with relation to GOFAI chess engines.

4-Evolution of chess community

Maybe less scientifically but more importantly is to study what happened to the chess community during this last quarter of century. Indeed an often employed argument is that: “Ok, AI cannot destroy humanity directly but it could convince people, or trick them, in order to do so”. Lets share some observations I have made on the evolution of the chess world over the last two decades:

1. Why do people continue to play chess? It was a real question just after Kasparov lost vs DeepBlue. It turns out people enjoy the game and especially when the opponent is human. The number of online chess players has never be so large. Chess.com has more than 100 million players. Something like 1/4 of the number of active Twitter users.

2. What about cheaters? The fact that the engine are stronger than humans introduce the possibility of cheating. Various egregious scandals have been uncovered. But the huge majority of players … just enjoy to play not just to win. Thinking and finding moves is sufficiently interesting in itself. You will always have a portion of the people looking for undeserved rewards but on the margin.

3. What about professional chess? There is still a professional scene. The world championship begins at the end of the week. People prefer to witness imperfect games between humans than long perfect draws between top chess engines. Here again the experiment has been done and the results are clear. Of course the format of chess event has evolved, more quick chess, no adjournament etc. but nothing really revolutionnary.

4. What about the game? The style of play has evolved on many levels. I think I could write a whole book about that but a fair summary could be: we learned that it is possible to be much more resilient than we thought. It means that position that appeared lost are actually possible to hold. This is much more frequent than positions we thought were draw that turn out to be winnable by one side. Also because it is possible to prepare much more thouroughly (you prepare the oppening and work out new ideas with an engine) there are second order effects: people avoid lengthy theoretical discussion (playing well known variant) because maybe your opponent has a better computer than you. In that Carlsen is very representative. Unlike Kasparov, who was the best example of a player who prepare his opening up to incredible depths, Carlsen plays almost anything (of course this is an exageration but the idea is clear): it makes it very hard to prepare against him (because you don’t know what opening he is going to play) and also it is possible because we know that the range of playable positions is much larger than we thought. One could say that today’s culture of chess player is at the same time wider (they can play more openings) and more superficial (they understand the system they play less deeply and have spent less time thinking about them by themselves) than yesterday’s players.

5. What about learning? AI is very good to teach you tactics. I do an automatic analyzis of all blitz games that I play on the internet. It is very interesting because I can spot on the fly tactical mistakes and themes. The machine doesn’t really help in terms of ideas but it is a very good tactical trainer. Maybe LLM will help to make more substantial and insightful comments of chess games (hint : I have actually tried it with GPT 3.5 and it was truly awful).

Temporary conclusions

This is a short review but, at the time of writing, it appears to me that most of the fears surrounding recent advances in AI are neither scientifically nor experimentally justfiable. The game of chess presents an historical experiment of what happens when AIs more powerful than humans appear on the scene. Of course this example is limited in its scope, this is part of the interest, and it is not possible to generalize directly from it. Nevertheless, it is a real example that can be used to illustrate/study many points of the ongoing debate on how AI would change our society. Lastly, make no mistake, as Joe Biden would say, our society will be impacted deeply by those technologies. But to state that those impacts will ineluctably mean the end of the world is, at this point, an egregious exaggeration to say the least.