When AI researchers talk about the risks of advanced AI, they’re typically either talking about immediate risks, like algorithmic bias and misinformation, or existential risks, as in the danger that superintelligent AI will rise up and end the human species.
Philosopher Jonathan Birch, a professor at the London School of Economics, sees different risks. He’s worried that we’ll “continue to regard these systems as our tools and playthings long after they become sentient,” inadvertently inflicting harm on the sentient AI. He’s also concerned that people will soon attribute sentience to chatbots like ChatGPT that are merely good at mimicking the condition. And he notes that we lack tests to reliably assess sentience in AI, so we’re going to have a very hard time figuring out which of those two things is happening.
Birch lays out these concerns in his book The Edge of Sentience: Risk and Precaution in Humans, Other Animals, and AI, published last year by Oxford University Press. The book looks at a range of edge cases, including insects, fetuses, and people in a vegetative state, but IEEE Spectrum spoke to him about the last section, which deals with the possibilities of “artificial sentience.”
Jonathan Birch on…
When people talk about future AI, they also often use words like sentience and consciousness and superintelligence interchangeably. Can you explain what you mean by sentience?
Jonathan Birch: I think it’s best if they’re not used interchangeably. Certainly, we have to be very careful to distinguish sentience, which is about feeling, from intelligence. I also find it helpful to distinguish sentience from consciousness because I think that consciousness is a multi-layered thing. Herbert Feigl, a philosopher writing in the 1950s, talked about there being three layers—sentience, sapience, and selfhood—where sentience is about the immediate raw sensations, sapience is our ability to reflect on those sensations, and selfhood is about our ability to abstract a sense of ourselves as existing in time. In lots of animals, you might get the base layer of sentience without sapience or selfhood. And intriguingly, with AI we might get a lot of that sapience, that reflecting ability, and might even get forms of selfhood without any sentience at all.
Birch: I wouldn’t say it’s a low bar in the sense of being uninteresting. On the contrary, if AI does achieve sentience, it will be the most extraordinary event in the history of humanity. We will have created a new kind of sentient being. But in terms of how difficult it is to achieve, we really don’t know. And I worry about the possibility that we might accidentally achieve sentient AI long before we realize that we’ve done so.
To talk about the difference between sentient and intelligence: In the book, you suggest that a synthetic worm brain constructed neuron by neuron might be closer to sentience than a large language model like ChatGPT. Can you explain this perspective?
Birch: Well, in thinking about possible routes to sentient AI, the most obvious one is through the emulation of an animal nervous system. And there’s a project called OpenWorm that aims to emulate the entire nervous system of a nematode worm in computer software. And you could imagine if that project was successful, they’d move on to Open Fly, Open Mouse. And by Open Mouse, you’ve got an emulation of a brain that achieves sentience in the biological case. So I think one should take seriously the possibility that the emulation, by recreating all the same computations, also achieves a form of sentience.
There you’re suggesting that emulated brains could be sentient if they produce the same behaviors as their biological counterparts. Does that conflict with your views on large language models, which you say are likely just mimicking sentience in their behaviors?
Birch: I don’t think they’re sentience candidates because the evidence isn’t there currently. We face this huge problem with large language models, which is that they game our criteria. When you’re studying an animal, if you see behavior that suggests sentience, the best explanation for that behavior is that there really is sentience there. You don’t have to worry about whether the mouse knows everything there is to know about what humans find persuasive and has decided it serves its interests to persuade you. Whereas with the large language model, that’s exactly what you have to worry about, that there’s every chance that it’s got in its training data everything it needs to be persuasive.
So we have this gaming problem, which makes it almost impossible to tease out markers of sentience from the behaviors of LLMs. You argue that we should look instead for deep computational markers that are below the surface behavior. Can you talk about what we should look for?
Birch: I wouldn’t say I have the solution to this problem. But I was part of a working group of 19 people in 2022 to 2023, including very senior AI people like Yoshua Bengio, one of the so-called godfathers of AI, where we said, “What can we say in this state of great uncertainty about the way forward?” Our proposal in that report was that we look at theories of consciousness in the human case, such as the global workspace theory, for example, and see whether the computational features associated with those theories can be found in AI or not.
Can you explain what the global workspace is?
Birch: It’s a theory associated with Bernard Baars and Stan Dehaene in which consciousness is to do with everything coming together in a workspace. So content from different areas of the brain competes for access to this workspace where it’s then integrated and broadcast back to the input systems and onwards to systems of planning and decision-making and motor control. And it’s a very computational theory. So we can then ask, “Do AI systems meet the conditions of that theory?” Our view in the report is that they do not, at present. But there really is a huge amount of uncertainty about what is going on inside these systems.
Do you think there’s a moral obligation to better understand how these AI systems work so that we can have a better understanding of possible sentience?
Birch: I think there is an urgent imperative, because I think sentient AI is something we should fear. I think we’re heading for quite a big problem where we have ambiguously sentient AI—which is to say we have these AI systems, these companions, these assistants and some users are convinced they’re sentient and form close emotional bonds with them. And they therefore think that these systems should have rights. And then you’ll have another section of society that thinks this is nonsense and does not believe these systems are feeling anything. And there could be very significant social ruptures as those two groups come into conflict.
You write that you want to avoid humans causing gratuitous suffering to sentient AI. But when most people talk about the risks of advanced AI, they’re more worried about the harm that AI could do to humans.
Birch: Well, I’m worried about both. But it’s important not to forget the potential for the AI system themselves to suffer. If you imagine that future I was describing where some people are convinced their AI companions are sentient, probably treating them quite well, and others think of them as tools that can be used and abused—and then if you add the supposition that the first group is right, that makes it a horrible future because you’ll have terrible harms being inflicted by the second group.
What kind of suffering do you think sentient AI would be capable of?
Birch: If it achieves sentience by recreating the processes that achieve sentience in us, it might suffer from some of the same things we can suffer from, like boredom and torture. But of course, there’s another possibility here, which is that it achieves sentience of a totally unintelligible form, unlike human sentience, with a totally different set of needs and priorities.
You said at the beginning that we’re in this strange situation where LLMs could achieve sapience and even selfhood without sentience. In your view, would that create a moral imperative for treating them well, or does sentience have to be there?
Birch: My own personal view is that sentience has tremendous importance. If you have these processes that are creating a sense of self, but that self feels absolutely nothing—no pleasure, no pain, no boredom, no excitement, nothing—I don’t personally think that system then has rights or is a subject of moral concern. But that’s a controversial view. Some people go the other way and say that sapience alone might be enough.
You argue that regulations dealing with sentient AI should come before the development of the technology. Should we be working on these regulations now?
Birch: We’re in real danger at the moment of being overtaken by the technology, and regulation being in no way ready for what’s coming. And we do have to prepare for that future of significant social division due to the rise of ambiguously sentient AI. Now is very much the time to start preparing for that future to try and stop the worst outcomes.
What kinds of regulations or oversight mechanisms do you think would be useful?
Birch: Some, like the philosopher Thomas Metzinger, have called for a moratorium on AI altogether. It does seem like that would be unimaginably hard to achieve at this point. But that doesn’t mean that we can’t do anything. Maybe research on animals can be a source of inspiration in that there are oversight systems for scientific research on animals that say: You can’t do this in a completely unregulated way. It has to be licensed, and you have to be willing to disclose to the regulator what you see as the harms and the benefits.
From Your Site Articles
Related Articles Around the Web
