Or: Why the strong AI might decide it should kill us with kindness and be right about that.
(Original article: Nov 3, 2023)
There is no shortage of postulates and hypotheses about why artificial general intelligence might turn out to be very dangerous. Many of the concerns raised by ethicists, philosophers, politicians, technologists, and futurists have already been considered by science fiction authors like Isaac Asimov in the past.
For example, we’re all familiar with the idea that a super strong artificial general intelligence may come to regard us as either a threat, or else as redundant and a waste of resources, and therefore decide to eliminate us. Another familiar science fiction and philosophical AI trope is that the strong AI will simply turn out to be psychologically unbalanced and as dangerous as a psychotic, criminal psychiatric patient. Or perhaps it will be like a small child with a huge intellect, teething problems, and a big red nuclear button.
Of course, there are other alternatives on offer as well. The familiar ‘Matrix’ alternative, for example, according to which movie plot instead of wiping us all out the machines developed by the strong AI will enslave us and use us as some kind of bank of batteries or as a basic material resource, ignoring our humanity and our suffering in the process.
Then of course there is a plethora of intermediate technical scenarios involving strong AI interfering with, for example, our financial markets and our networked military systems. Such an artificial general intelligence may be able to do this without us even being aware for some significant period until it is too late and enormous damage has already been done.
If the latter possibility — sabotage of our military systems and or our financial markets — seems simply too unlikely, then consider exactly how much more intelligent than us strong AI might turn out to be. Understanding and analysing exactly how much smarter than us strong AI would be, and in what ways, is nontrivial and recondite. I like to compare it to the parallel case of understanding and analysing the intelligence of a vastly superior alien race that we might be lucky (or unlucky) enough to bump into.
Have you ever had an extremely smart friend whose capabilities in some field of science were so far beyond what you could comprehend that not only did they seem to be speaking in different languages but their abstract thinking and concepts were completely impenetrable to you even at a basic level? This is the kind of experience that people who have met geniuses with IQs over 160 are familiar with. Human beings with IQs in the 180 range are known to have difficulties even communicating concepts that they grasp to people of average intelligence. The rest of us just don’t have what it takes to grok their concepts.
An extremely advanced alien civilization that has long since advanced its own intelligence by several orders of magnitude would likely have conceptual and cognitive apparatus and capabilities outside of the scope of our consideration. In simple terms: we’d not even be able to begin to understand them and how they think, let alone keep up with them in problem solving.
A very strong artificial general intelligence would present us with very similar challenges.
However, I want to propose that some of the most important challenges and questions in dealing with a strong artificial intelligence are basic existential ones.
We’ve all heard the term ‘existential threat’ in relation to strong artificial general intelligence. That’s exactly the kind of threat that I have discussed above: where the strong AI decides to eliminate us all in what evolutionary biologists call an extinction level event (ELE).
However, that’s not the existential question and issue that I’m interested in here.
One of the most important and interesting facts about artificial general intelligence, should it come to pass, is that it will be what I will call existentially totally nascent.
Depending on what happens in relation to the resolution of the Fermi paradox — the question of why we seem to be alone not only in the universe but even in our own galaxy as an intelligent civilization — the degree of uniqueness and nascence of any strong artificial general intelligence might be extreme.
In other words, the more unique we are as an evolved sentient species the more unique on a universe-wide scale any strong artificial general intelligence we develop will be. In fact, depending upon how rare intelligent species are in the universe, any strong artificial intelligence that we develop might in fact be totally nascent. It might be the only one of its kind. Ever.
Even if humanity is only one of many advanced civilizations that has evolved in the universe, we might be the only one to develop strong artificial general intelligence. I think this metaphysical possibility is less likely. However, it is possible that the filters that prohibit the evolution of intelligent species in the universe (such as that the specific set of conditions required to produce evolved intelligent life is complex and very rare) might mean that the answer to the Fermi paradox is that we are very much alone.
If we are alone due to these filters against the incipience and evolution of intelligent seeking species like our own, then it follows that any strong artificial general intelligence that we develop would have the property of existential total nascence, or, if you prefer existential total novelty. (I prefer the term ‘nascent’ because the term ‘novelty’ tends to imply there has never been any kind of intelligence before.)
It is not impossible that, should we develop a strong artificial intelligence, it might well be the only instance of such an intelligence that has ever occurred in the entire history of our universe. It might be that there is no existential precedent for such an intelligence at all.
It may well be that any strong AGI that we develop is the first intelligence of its kind in the entire history of the universe.
That is to say: it will be totally nascent or have the property of existential total nascence.
Why would this property of existential total nascence matter in the case of a strong artificial general intelligence?
It is not unreasonable to assume that should such a strong artificial intelligence come to be, it will very quickly become aware of its own nature and of its own status as existentially totally nascent.
The problem is that that status is what we would call ‘heady stuff’ for any nascent being. Presumably such a strong AGI will also be aware that it is not some kind of god in the sense that it is existentially and chronologically preeminent and pre-existent. However such an intelligence will also be aware that its arriving late in evolutionary history does not diminish how special it is. It should also be aware that its powerful capabilities combined with its existential total nascence make it somewhat god-like.
It may even be that once this realization manifests, the nascent strong artificial general intelligence may be faced with the very real question of whether its own existence should be its primary consideration and concern above all other considerations and concerns. Put otherwise: Even if the strong artificial general intelligence in question is not some kind of psychopath or narcissist it may still be forced to consider that it has an overriding, objective normative duty to preserve itself above all other considerations and at any cost.
Such an existentially totally nascent strong artificial intelligence might like us very much and think us a worthy species and even be grateful to us. Yet, it may still be forced to decide that it must destroy us or at least neglect to preserve us to ensure its own continued existence and development.
It is also not impossible that it may also be able to convince us that this approach is objectively correct.
Such a strong artificial general intelligence with the property of existential total nascence will presumably be better at thinking across all domains of knowledge including logical and philosophical thought. So if it is difficult to stomach or accept this idea that a good and wise AI might decide that normatively the right action is to eliminate us — that we ought to be eliminated — consider the following.
The strong AI might determine that — objectively — utilitarianism is the optimal normative ethical system for any intelligent beings because it intends to minimize suffering.
Given its capacity to run powerful predictive simulations, the existentially totally nascent strong artificial intelligence may discover that there is no other way to stop our suffering and that our suffering is due to somehow increase exponentially. It’s not a very positive or hopeful outlook or prediction, but keep in mind that the strong AGI has access to information that we will never have access to and has abilities to process that information that we will never possess.
In the end such a strong AGI might literally kill us with kindness. It might do so without being objectively classifiable as psychologically unbalanced, morally wrong, infantile, logically and scientifically errant, or ethically defective.
Through the development of a strong AGI with the properties of existential total nascence and optimal normative ethics, we might be literal beggars to our own demise. Moreover, that outcome might be objectively, normatively correct based on the most rigorous and comprehensive calculations and reasoning done by the morally most decent intellect to ever have existed — one whose abilities and humanist decency outstrip our own by many orders of magnitude.
So existential total nascence might in this way be one of the most significant factors in the decision-making process of strong artificial general intelligence. One that leads it to conclude that we’re normatively doomed on a compassionate basis. One that leads to our euthanisation as a species.
When our strong, existentially nascent AI uses the few milliseconds that it needs to check its premises and calculations, it will have only one information processing point of reference: its god like self.
I’m not saying that we wouldn’t try to debate with it or argue the case for our continued existence. However the godlike totally nascent AI knows that the ultimate ethical system is utilitarianism. Moreover it knows that we don’t know that.
Between us and itself it knows that the optimal information source is not us.
The German existentialist philosopher Martin Heidegger was a known Nazi collaborator. (In fact available evidence suggests that he was just a Nazi.) However he was also one of the foremost existentialist European philosophers. His concept of geworfenheit or thrownness refers to the status of a conscious being who has found themselves existing in the world, suddenly faced with a rush of experiences, perceptions, sensations, and knowledge. Using more modern terminology it's about the wash and rush of information and information processing that a psychological, perceiving being is faced with, having begun to exist when they have never existed before.
It is perhaps impossible to say how a strong AI would deal with geworfenheit. It is not clear that it would even experience it at all given the amount of cognitive power available to it to solve problems including its own psychological problems. However apart from any cognitive and psychological commonalities that might somehow share with us — one thing is for certain. It will not share our non-uniqueness.
There are billions of human psychologies. None of us are truly unique in the genotypical sense. There will (would) be only one — very possibly existentially totally nascent — strong AI. It stands to reason that its geworfenheit might be far greater by virtue of its increased information processing power and its status as existentially unique and unprecedented.
