On the AI sentience debate
An attempt at raising questions.
A whole debate has been going on recently around the question: “can an AI model be sentient?”
This all came about following statements from Blake Lemoine, a Google engineer, who started himself to believe that a conversational AI developed by its company had become sentient.
Here’s an excerpt from the exchange between Lemoine and LaMDA, which is clearly impressive:
This sparked a decent amount of controversy, both inside and outside the field. I have to admit here that at first I felt detached from the whole discussion, although the whole affair gave me the long-awaited opportunity to flex my AI knowledge with friends and acquaintances from outside the field.
The reason this whole thing caught me uninterested was that I have a solid grasp of the underlying methods that allows the development of these impressive technologies, and nothing has happened recently that would, in my view, grant the use of words such as “consciousness” or “sentience”.
The AI product in question is called LaMDA, which stands for Language Models for Dialog Applications.
Language Models 101
LaMDA is a language model. The following section provides a very concise, general explanation of how language models work. Feel free to skip it.
Language models are the backbones of all the impressive language AI that we see around. These are very large neural networks, which are trained on very large corpora of data. Their impressive “understanding” and ability to manipulate language comes from the way in which these models are trained.
In general, neural networks are trained on labelled data samples in order to learn some prediction task. A computer vision model might learn to classify pictures, while an NLP (Natural Language Processing) model might learn to classify a piece of text, or even to generate text based on an initial prompt.
Language models are big neural networks with particular architectures apt at working with textual data. The way in which language models are trained is pretty simple: the network is fed a “context”, a portion of text from the training corpus, where some part of the text is hidden away from the network. The model is trained to fill the gaps, that is, to predict the missing part of the text. This simple task is actually really hard to tackle without any prior knowledge and requires the model to learn many features of the human language: what words often co-occur in the same sentence, what is typically the position of the subject/verb/object , and so on. State-of-the-art models such as LaMDA have clearly reached a high level of mastery in modeling the structure of the text. This is what happens in modern neural network when we scale to crazy amounts of data and computation.
After training, whenever a language model has to generate text, as LaMBDA does, an initial prompt is fed to the model. The model, which has learned which words are likely to occur in a given context, produces the next words. in order, following the initial prompt.
Knowing how language models work is enough for most people to discard the notion of consciousness or sentience. For example Yann LeCun, one of the greatest AI pioneers, has said (tweeted) that today’s machine learning models aren’t conscious “[…] not even for true for small values of "slightly conscious" and large values of "large neural nets". I think you would need a particular kind of macro-architecture that none of the current networks possess.”
In fact LaMDA is, conceptually and structurally, the same thing as GPT-3 from 2020, or even as BERT from 2018. They belong to the same family of models (called Transformers), with the main differences residing in the magnitude and characteristics of data they were trained on, and the scale of the models themselves.
Which begs the question: if BERT definitely was not sentient, and GPT-3 was pretty smart but not sentient at all, what would make this model sentient?
I understand that what I’m saying here might sound painfully obvious to many of my colleagues. However, I think some interesting questions can be brought to light by this conversation, mainly about how to approach the same kind of questions in the future, when they might be more relevant.
The Consciousness Axis
Here I would like to switch the conversation from sentience to consciousness, with no lack of generality, I hope. I do this because I believe that consciousness is a lower bar than sentience when talking about artificial intelligence (although this might be debated). Moreover, a lot of discussion in previous years has revolved around the issue of consciousness rather that sentience. See for example this tweet from the Chief Scientist at OpenAI:
If LaMDA is “the same thing as before”, as I have explained, then why would it be conscious now?
In other words, a good question to ask is: on what axis do we expect consciousness to arise? Can consciousness be simply a product of finding the right algorithms and architectures? Or is consciousness, up to a certain point, agnostic to the underlying machinery, and instead it simply requires human-brain levels of scale and computation?
Or, just maybe, we are one piece of machinery away from consciousness (inner speech, interface with the external world, memory of the past).
And what if, no matter what marvel we might be able to create inside a machine, our pre-existent concepts of consciousness and sentience will never be able to adhere to something that is not made of flesh and blood?
As If
Everything I’ve said so far is missing one point. Whoever has seen the movie “Her” might already know what this is. I would call the the “as if” problem. That is if an AI looks and sounds conscious, will people interact with it as if that was actually the case? And what does this mean for society, human psychology, and so forth? I have no familiarity with any literature on this subject, so I am happy to receive any pointers in these directions.
An opposite issue might arise here. If we were able to actually build something that perfectly resembles a human being, and let’s also say that this being would be made of “biological stuff”, would be able to interface with it with empathy and compassion as if it was actually human, or would the simple knowledge of its fake humanity automatically render it a second-order citizen?
The Chinese Room Argument
The Chinese Room Argument is a philosophical argument developed by John Searle.
Without going into the details, the broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted. Instead minds must result from biological processes; computers can at best simulate these biological processes.
I think the argument is relevant, even practically. If for some people consciousness might be something that arises from a bunch of signals working inside a biological system, then the whole idea of a machine that might be recognized as conscious is simply unattainable. If consciousness requires passing a Turing test, then we have probably reached it already. We can make zero progress in this discussion without clear definitions.
As an example, here’s an incomplete list of features that a sentient machine should have, according to Ali Ladak from the Sentience Institute:
Going back to the debate at large, I have a feeling of unease arising from the fuzziness in the definitions we use, especially concerning the issue of AI consciousness. It looks like even the brightest AI scientists have absolutely no sophistication, no acumen when discussing these topics. I won’t name just in case I’ll need to apply for a position at their labs at some point :)
I realize I am raising many questions and giving very few answers. Sorry about that.