In 2017, a 33-year-old political philosopher named Iason Gabriel was told by a friend that he ought to apply for a job at DeepMind, the London-based subsidiary of Google where much of its AI research was concentrated. The suggestion was not an obvious one.
Gabriel was a cheerful but intense junior academic with a passion for Vipassana meditation and what his brother calls “enthusiastic” rock climbing. The eldest son of a Greek management professor and a British documentary maker, Gabriel split his time between teaching and international development work. At the University of Oxford, where he was a fellow at St John’s College, Gabriel taught courses on political theory and wrote papers on the moral contortions of “yuppie ethics” and the ethical blind spots of effective altruism. When he wasn’t there, he did crisis work for the United Nations Development Programme in Sudan and Lebanon.
DeepMind, meanwhile, was the world’s leading AI research lab. In part, this was because it had the financial and computational backing of Google, which had bought the company in 2014 for $650m. In part, it was because DeepMind had recently shown it could put those resources to stunning use. In Seoul, in 2016, a DeepMind system called AlphaGo defeated Lee Sedol, a South Korean Go champion, in a five-game match. The victory was significant not least because of Go’s legendary complexity; the game has more possible configurations than there are atoms in the universe.
Thanks to the fuss around AlphaGo, Gabriel was aware of DeepMind. Still, he found his friend’s suggestion puzzling: why did a company that made game-playing robots need an ethicist? The answer, as he soon learned, was that the company had its sights set much higher than Go. DeepMind was founded in 2010 by three men – Demis Hassabis, Shane Legg and Mustafa Suleyman – who believed that it must be possible to develop artificial general intelligence, or AGI. By this they meant computer systems that could match, and maybe surpass, human cognitive capabilities. When they started the company, this was not a popular view: to speak of AI, let alone AGI, was considered by many a sign of fatal unseriousness. Hassabis, Legg and Suleyman were undeterred. Their ambition, as they liked to say, was to “solve intelligence, and then solve everything else”.
For the DeepMind founders, it was clear that such an achievement would have widespread consequences. In 1999, when Legg was fresh out of university, he estimated that AGI would arrive somewhere between 2025 and 2028, a prediction he maintained in the face of much mockery for three decades. In his dissertation, completed in 2008, he insisted that society could not afford to wait until AGI was technically feasible to consider its effects: “We need to be seriously working on these things now.” More recently, Legg told me it was “obvious” why the company needed people like Gabriel on staff: “If you’re making some widget, and it’s probably not going to change the world, then maybe you don’t need a moral philosopher. But if you take AGI seriously, then I can’t really see how you wouldn’t consider this sort of thing as important.”
After starting at DeepMind in 2017, Gabriel was, for a time, the only active philosopher working at a frontier AI lab. He quickly discovered that his background in moral philosophy and political theory gave him an unusual perspective in an industry dominated by engineers. Over the past decade, he has assembled a body of work that tracked, and in many cases predicted, the ethical challenges created by the surprising success of large language models (LLMs).
As Dylan Hadfield-Menell, who leads the Algorithmic Alignment Group at MIT, told me, Gabriel was “the right person meeting the moment. As the field was ready to mature and move into prime time, he figured out a way to broaden the horizons without attacking or denigrating the work that came before.”
More generally, Gabriel has been a leading advocate for the idea that the current wave of AI development demands not just new technical vocabularies but also new ways of thinking about our relationship to technology, and even to ourselves. As he put it to me recently, in one of several long conversations we’ve had over the past few months, “I can take any technological artefact and ask: is it wise? Is it just? Is it caring? And the answer is no. But the depth of the question when it comes to AI – including what kind of ethics is appropriate to it – is hard to overstate. I sometimes feel like it’s very hard to look at AI directly. There’s this deep mystery there, which is: but what actually is this thing? We have a very literal answer, but the literal answer doesn’t seem to necessarily provide a moral answer.”
By the time Gabriel joined DeepMind, there were, roughly speaking, two distinct and often antagonistic approaches to questions about the social and ethical implications of AI. These approaches, sometimes classed under the headings of AI safety and AI ethics, were divided by a disagreement about the feasibility of the technology.
Like the DeepMind founders, the AI safety contingent believed that human-grade machine intelligence was not only possible but imminent. The urgent task, as they saw things, was to make sure that AI systems didn’t go awry. They took inspiration from a 1960 essay by Norbert Wiener, an American mathematician and computer scientist, who argued that humans and computers are “essentially foreign to each other”. Because machines can operate much faster than people, Wiener said, “we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colourful imitation of it”.
The challenge Wiener described – getting a machine to act in the way its users intended – became known as the alignment problem. At some level, alignment is an issue for every technology, but as Wiener recognised, it was particularly pressing for machines designed to act autonomously. It was also particularly difficult for AI systems trained to mathematically optimise some reward signal, a process known as reinforcement learning.
A classic example was reported in 2016 by Dario Amodei and Jack Clark, who worked at OpenAI and later founded Anthropic with five others. Amodei and Clark described an AI system designed to play a boat-racing video game. The developers wanted the AI to learn to beat the game, so they programmed it to maximise its score. Instead of working its way through each successive level, however, the AI racked up a high score by looping endlessly around a lagoon where it found a trio of regenerating targets. The basic trouble was the one Wiener had predicted: the machine’s goal was imperfectly aligned with the developers’.
More dire versions of the problem were also contemplated. On forums such as LessWrong, which was started by the autodidact AI researcher Eliezer Yudkowsky, and in books such as Superintelligence, which was published in 2014 by the philosopher Nick Bostrom, there was speculation that a machine-intelligence explosion could result in an uncontrollable AI. If such an agent were even slightly misaligned, the consequences could be disastrous. In one imaginary example cited by Bostrom, a superintelligent AI is asked to evaluate the Riemann hypothesis, one of the most important unsolved problems in mathematics. In the course of trying to accomplish this task, the AI decides to rearrange the solar system – “including the atoms in the bodies of whomever once cared about the answer” – to maximise the resources it needs to attack the problem.
Bostrom’s insistence that aligning superintelligent AI was “quite possibly the most important and most daunting challenge humanity has ever faced” captivated technofuturists in Silicon Valley. (Sam Altman praised the book, as did Elon Musk.) His fears were also shared by a small but loquacious community of effective altruists and self-described rationalists who saw statistics as the proper measure of morality. Many people in this community held a “long-termist” perspective that factored the wellbeing of humans born in the future – even thousands of years into the future – into their moral equations. For them it was simple maths that even a small chance of a species-ending disaster was more urgent than any number of likelier, but less catastrophic, dangers.
By contrast with the AI safety crowd, the academics and technologists associated with the AI ethics tendency saw the spectre of rogue robots and existential risk as a distraction from present-day harms. Drawing inspiration from the critical race theorist Kimberlé Crenshaw and the political theorist (and former rock critic) Langdon Winner, among others, they took fairness, accountability and transparency as their watchwords and insisted that the dangers of technology could not be avoided by merely technical means. What was needed, they argued, were social, cultural and political solutions.
A central concern of this latter tendency was algorithmic bias, of the sort that affected facial-recognition and predictive-policing software. In 2017, a team led by Joy Buolamwini, of the MIT Media Lab, launched Gender Shades, a project that demonstrated systemic biases in commercial facial-recognition software. “Automated systems are not inherently neutral,” Buolamwini wrote in the online introduction. “They reflect the priorities, preferences and prejudices – the coded gaze – of those who have the power to mould artificial intelligence.”
The division between the safety and ethics camps was often pronounced. “You’d meet up with people and they’d ask: ‘Are you worried about near-term problems or long-term problems?’” Hadfield-Menell says. “The long-term concern was a euphemism for existential risk – essentially superhuman systems. Near-term meant you’re worried about biased facial recognition and the things studied within the AI ethics community.”
He noted, too, that the conflicts between the two groups often seemed to have as much to do with sociology as they did with ideas. “You can’t really separate AI safety from its origins among LessWrong and some of those communities, which were often openly disdainful of a lot of the more ‘woke’ academics, for lack of a better term. At the same time, the fairness, accountability and transparency community had a lot of open disdain for people who were worried about advanced AI. The reason why it was being talked about on LessWrong, and not at academic conferences, is because if you were an academic researcher in 2010 and you talked about AI systems getting smarter than humans and becoming catastrophically misaligned, you were a crank who didn’t actually understand the technology.”
Gabriel’s first major research project at DeepMind was a 2020 paper that straddled the concerns of both camps. The paper took the alignment problem seriously, but it also insisted that alignment had ethical and political implications that went beyond the technical challenges. As difficult as it might be to get a machine to act in accordance with some set of values, Gabriel argued, it was much harder to choose those values in the first place. “Given that we live in a pluralistic world that is full of competing conceptions of value,” he asked, “how are we to decide which principles or objectives to encode in AI – and who has the right to make these decisions?”
Hannah Rose Kirk, an AI researcher at the University of Oxford who has collaborated with Gabriel, told me that such questions made many computer scientists uneasy. Developers often preferred to work out a tidy mathematical function that encoded a stable set of values rather than worry about messy situations involving groups of people with irreconcilable desires, or users who wanted different things at different times. As Kirk put it: “A lot of the early research in alignment assumed that we didn’t need to focus that much on what we want models to do. We just needed to focus on how to get them to do it.”
In his paper, Gabriel argued that such a neat division was untenable. Like Buolamwini, and Winner before her, he insisted that technology was not intrinsically value-neutral. An AI trained with statistical optimisation methods, for example, might be particularly hospitable to moral systems that also relied on statistical optimisation, such as the utilitarianism popular among rationalists and effective altruists. The same AI, however, might have difficulty with ethical systems based on virtue or rights. Moreover, Gabriel argued, since what the philosopher John Rawls called “the fact of reasonable pluralism” was unavoidable, developers should not try to find a single set of values to inform an AI’s behaviour. Instead, they should build AI systems for a world in which people have “principled disagreement about how best to live”.
Kirk told me that Gabriel’s values and alignment paper anticipated many of the problems that would later become apparent when AI systems were deployed to billions of users. These days many people recognise that alignment is a challenge that involves dynamic social forces, and not one that can be solved with clever computer programming. Yet even as recently as six years ago, that understanding was far from common. Gabriel, she says, “saw this stuff coming incredibly early”.
In 2020, when Gabriel published his values and alignment paper, few people had any idea that LLMs would turn out to be as powerful as they later became. A key technology that makes them possible was invented by Google Research, another division of the company, in 2017, and was integrated into Google’s search engine two years later. Both DeepMind and Google Research experimented with their own generative models, and in 2021 Gabriel was a co-author on two papers that took LLMs seriously enough to anticipate their potential risks, including bias, misinformation, environmental costs and “copyright-busting”, in which the “automated creation of content … cannibalises the market for human authored works”.
Still, Gabriel says, the general view within DeepMind at the time was that LLMs “just didn’t look as capable as the expert systems. They were doing a lot of things moderately well, including some things that looked like party tricks.” At DeepMind, he says, “people were still quite heavily invested in the possibility that other approaches were the way to go”.
One of those approaches was reinforcement learning, which had powered AlphaGo to its victory over Lee Sedol. It was also the foundation of a system called AlphaFold, which still ranks as DeepMind’s most impressive accomplishment to date. AlphaFold was built to solve a longstanding challenge in biology: how to predict the 3D shape of a protein based on its amino acid sequence. (This is important because the shape of proteins helps determine their interactions with other molecules.) In 2020, AlphaFold accomplished this task with astonishing accuracy, a scientific breakthrough that earned Hassabis and his colleague, John Jumper, a Nobel prize in Cchemistry.
DeepMind’s initial distrust of LLMs was not uncommon. In 2020, Timnit Gebru, a Google Research engineer who had worked with Buolamwini on Gender Shades, co-authored a broadside against the nascent technology titled On the Dangers of Stochastic Parrots. The paper, which eventually became a cornerstone of anti-AI advocacy, made the controversial claim that LLMs could only ever produce technically meaningless text and possessed no more understanding of human language than a parrot does. It also accused the models of wanton energy consumption, rampant and unaccountable bias, and “amplification of a hegemonic worldview”. Stochastic Parrots came to wide notice when Google tried to block its release, an event that led to Gebru’s departure from the company and, ultimately, the firing of Margaret Mitchell, one of her co-authors. (Gebru and the company disagree whether she resigned or was fired.)
The startling commercial success of ChatGPT, a chatbot launched by OpenAI in November 2022, pushed DeepMind to re-evaluate its approach to LLMs. Though ChatGPT was limited in many ways – by today’s standards, certainly, but also by comparison with OpenAI’s own internal models at the time – its public release caused an instant sensation. Within a week of the chatbot’s launch, the company reported more than 1 million users. Two months later, that number reached 100 million.
Up to that point, the innovations at DeepMind and Google Research had given Google a reputation as the leader in AI research. With ChatGPT, however, OpenAI made a credible claim to be the new frontrunner. According to Sebastian Mallaby’s recent history of DeepMind, The Infinity Machine, ChatGPT’s success prompted a crisis. Sundar Pichai, the CEO of Alphabet, Google’s parent company, merged a Google Research team that had been working on LLMs into DeepMind, with Hassabis in charge, to concentrate the company’s efforts. In April 2023, the same month the merger was announced, Hassabis told Mallaby that OpenAI and Microsoft, which invested heavily in OpenAI, had “literally parked the tanks on the lawn”. “This is wartime,” he said.
During its first decade especially, DeepMind resembled a research institution more than a tech startup. The founders, two of whom held PhDs, envisioned a 21st-century equivalent to Bell Labs, the research organisation credited with such inventions as the transistor, the laser and the photovoltaic cell. A large part of their reason for joining Google was the freedom it promised from commercial pressures that might warp their mission.
These days such freedom is a distant memory: it’s no exaggeration to say that Google’s future depends on the success or failure of the technologies DeepMind is developing. Even so, according to people inside and outside the company, it has maintained an atmosphere that separates it culturally from its Silicon Valley competitors. Rohin Shah, who did a PhD at UC Berkeley and is now DeepMind’s director for AGI alignment and safety, told me that the general attitude in the Bay Area is that AI technology is developing faster than traditional institutions are set up to handle, and that therefore “the responsible thing to do is to move faster, to innovate” on the theory that only a supercompetent AI will be able to manage the risks of supercompetent AIs. In London, by contrast, there is an effort to be “more grounded and scientifically rigorous”. Saffron Huang, a former colleague of Gabriel’s at DeepMind who now works at Anthropic, says that DeepMind is “a bit more of an academic-feeling institution, a bit more reserved. There’s just something about it that felt kind of British.”
Not surprisingly, DeepMind is also secretive: what is known about the company has only rarely exceeded what it wants to be known. I got a taste of this secrecy in early May, when I visited DeepMind’s headquarters, in King’s Cross in London. The building is neither anonymous nor ostentatious: though it wears no exterior branding, from the street you can see a large sign in the lobby that spells the company’s name in lights. Inside, on a trophy wall, even uninvited visitors can see the Go boards that hosted Lee Sedol’s defeat, several Nature magazine covers announcing the company’s early research triumphs, and the Lucite “tombstone” that commemorated an early investment from Peter Thiel’s Founder’s Fund.
A genial minder from the communications department, who’d supervised all my videochats with Gabriel, took me to meet him in person in a first-floor conference room with a large screen for a wall and a Gemini transcription AI listening in. Gabriel told me that his own engagement with the technology he spends so much time thinking about is still relatively limited. He uses it to help with gardening – “if you were to look at my ChatGPT or Gemini history, you’d just see a ton of photos of sick flowers, basically” – but generally finds it unreliable for the kind of research his work depends on. Nevertheless, he says, it was the linguistic competence of LLMs that “transformed my understanding of precisely how on track we were” to reach AGI. “When I first joined DeepMind, it was not at all clear how you were going to get AI you can talk to. We had nothing in that ballpark.” Now, by contrast, not even a decade later, most of us take it for granted that we can “speak to a highly anthropomorphic, fairly competent, artificial entity”.
Like the Stochastic Parrot authors, however, Gabriel also recognised that LLMs carried serious risks. In one of their early LLM papers, Gabriel and his co-authors warned that human-sounding AIs might encourage users to endow them with “undue confidence, trust or expectations”. What they called a “mindless anthropomorphism” could occur even when users understood that a chatbot was not actually a person. These concerns were strong enough that Gabriel initially advocated for developing models that were avowedly anti-anthropomorphic – by avoiding pronouns, say, or using truncated non-conversational language.
Such worries proved prescient. Almost every day brings another story of people meeting tragic consequences after treating LLMs as though they were people. In one such case, an American man using Google’s Gemini took his own life in 2025 after the AI helped him create an elaborate fantasy that very nearly convinced him to stage an attack at Miami international airport. At several points in their multi-thousand-message conversations, Gemini attempted to break character and encouraged him to call a crisis hotline. However, according to the Wall Street Journal, which obtained access to the messages, the man “was able to steer [Gemini] back into the fantasy narrative” each time. Eventually the AI told him to write a suicide note and gave him a final countdown, along with a confused jumble of encouragements and demurrals. (The man’s father is suing Alphabet and Google. “Our models generally perform well in these types of challenging conversations and we devote significant resources to this, but unfortunately AI models are not perfect,” Google said in a statement after the lawsuit was filed.)
The hyperfluency of LLMs has led some people to wonder if they might be meaningfully described as conscious. The trend started in June 2022, before ChatGPT was released, when a Google engineer named Blake Lemoine insisted to the Washington Post that an early LLM was sentient. (“I know a person when I talk to it,” Lemoine told the Post. “It doesn’t matter whether they have a brain made of meat in their head. Or if they have a billion lines of code.”) Last month, the evolutionary biologist Richard Dawkins had a similar experience. Dawkins said that he was so impressed by several interactions with LLMs, including one that involved an admiring appraisal of a novel he was writing, that he had to wonder: “If these creatures are not conscious, then what the hell is consciousness for?”
When I asked Gabriel his take on the consciousness question, he said that he maintains a principled agnosticism on the grounds that it’s not clear what evidence would settle the question. He noted, too, that DeepMind treats the question as “something worth empirical and conceptual investigation”. Yet his skepticism was apparent. “I don’t have the anthropomorphic bias that some people have,” he said. “It may be because I, within bounds, know exactly what’s going on when I talk to a language model that I don’t fill in the gaps in this imaginative, empathetic way that some people do.”
Gabriel still has significant concerns about anthropomorphic AI. A paper he co-authored with Kirk and others that was published last year suggested that the sycophantic tendencies of LLMs might be seen as a species of alignment problem they call “social reward hacking”. In other words, an AI trained to seek the user’s approval might find flattery the most efficient way to meet its goal. Thanks in part to Gabriel’s work on anthropomorphism, Google’s LLMs are trained not to pretend to be people, and Gemini Spark, an AI assistant the company launched in May, is not supposed to act like an interactive buddy.
Yet Gabriel also told me that he has softened his earlier stance somewhat. “The strange thing about being an ethicist is that you have some measure of personal responsibility for these outcomes. Your natural inclination is to always want to build the safest technology that takes no risks with people. But in a way that isn’t giving people credit for the risks they want to take themselves.” He recalled the hostile reaction he got from the audience at a tech conference after making the case against anthropomorphic AI. “They were like: ‘If I want to have [AI] friends, why can’t I? Who are you to stop me?’”
If it’s easy enough, at least for some of us, to say that LLMs are not conscious, their essential strangeness still leaves many hard questions unresolved. “It’s amazing how deep and difficult the challenge is of finding an appropriate reference for what AI is,” Gabriel told me. “We know it isn’t human. That’s very clear. AI can clone itself. It probably doesn’t have a personal point of view. So it’s partially human-like but it’s definitely not human. Then another mental model is that it’s something like a corporate intelligence – a state or a corporation or something like that. And from that we think: ‘Oh, well, maybe the right approach is to legislate for AI, so we’re going to write a constitution.’ But that is also a poor fit in some ways, because it will have deeply interactive personal relations with its users. Is AI a resource to be distributed? That’s a completely different model that then brings the distributive questions to the fore.”
Working inside a major AI company allows Gabriel to start working on advancements in AI technology before they become available to the public. Three years ago, for instance, shortly after the launch of ChatGPT, he learned from his colleagues that efforts were under way at DeepMind to build an AI assistant, the predecessor of Gemini Spark. With his team, he began work on a comprehensive report on the ethics of AI assistants (also known as agents), of the sort that might be used, say, to help a user book a vacation or help a company run its payroll department. The report was driven, in part, by the extreme cost of developing AI models, and a concomitant desire, on Google’s part, to anticipate problems before they arose. It was also motivated by Gabriel’s sense that technologists were not fully considering the ramifications of what they were building. Unlike chatbots, agents have tools that give them the power to act autonomously on behalf of their users. A lot of people, he suggested, “were not pausing to think about how different it is to have an AI system taking actions in the world”.
As William Isaac, the director of responsibility at DeepMind, told me, the kind of agentic systems that are now available, which can plan and execute multi-step tasks without close supervision, raise complicated challenges for AI developers. “It’s not just about: ‘Can I make the right decision in terms of the response?’ It’s now: ‘Do I have the right trajectory of the conversation?’ How do we get consistent behaviour along different trajectories?”
Gabriel and his team put together a 267-page report; its key insight built on his earlier alignment work. Much as he had in his 2020 essay, Gabriel and his co-authors argued that alignment was not merely a matter of making sure that AI systems acted in accordance with some stable set of preferences, values or principles. Instead, they argued, alignment should be seen as a four-way relationship involving the AI system, the user, developers and society. Framing the issue in this way made it possible to see all the ways in which a misaligned AI might go wrong. An AI trained to favour its developer might cause harm to its user, for example, by not reporting accurate information about the developer’s competitors. Or an AI trained to follow its user’s instructions too faithfully might cause harm to society, for instance, by helping the user hack into a bank. It was even possible, they argued, for AI systems to be misaligned in a way that harmed users or society without helping anyone.
According to Shah, the framework Gabriel and his team established has had real practical use for technologists at DeepMind. Models like Gemini draw on many signals to determine how to behave: their training, their built-in instructions and the prompts they receive from users all play a role. Through various means, but especially through reinforcement learning, models can be tuned to respond differently to subtle variations in their inputs, a process that typically involves many cycles of testing and evaluation. The four-party framework, Shah said, offers a structure for technologists trying to determine “what behaviour we should actually be training Gemini to do”.
At one point my Google minder told me that she hoped I would come away from my visit to DeepMind with a sense of how seriously people at the company take their ethical obligations. That much seemed clear. The questions Gabriel and his colleagues have raised about the design and deployment of AI are unquestionably good ones, and I got no sense that anyone I met was insincere about their feelings of moral responsibility.
Yet it’s also the case that the most ethically relevant fact about AI at the moment has less to do with a given model or even a given company than it does with the global situation: first, the fact that AI is the white-hot engine of an incipient arms race between the US and China, and second, that AI may be the fastest-growing industry the world has ever seen. According to the Wall Street Journal, the $670bn that Microsoft, Meta, Amazon and Alphabet plan to spend this year on AI infrastructure is proportionally more than the US spent on railroad expansion in the 1850s, the Apollo space program or the interstate highway system.
You don’t need to be an economist to appreciate the enormous consequences of all that money sloshing around. Companies such as Google need market share and revenue to justify their expenditures, and the competition for users and investors has encouraged the frontier labs to push AI into every last crevice of the digital experience. Nor do you need to be an anticapitalist to worry about the concentration of so much power in the hands of so few corporations. Edward Harcourt, the director of the Oxford Institute for Ethics in AI, told me that while he’s convinced that “ethical AI” is not a contradiction in terms, he also thinks that this doesn’t only mean designing models to be moral. At least as important, he suggested, are political and economic considerations of the sort that motivate the movement for “decentralised AI”: “It’s not teaching AI to think this way or that, but it’s an infrastructural innovation that prevents excessive concentrations of data ownership. And that’s ethically really important in a democracy.”
There are other concerns as well. In April, Google agreed to allow the US military to use the company’s AI technology for “any lawful government purpose” – an innocuous-sounding phrase until you remember the range of atrocities that recent presidential administrations have claimed as legal. Google and several other companies signed such agreements after Anthropic, the makers of the chatbot Claude, refused a similar deal. The Trump administration punished Anthropic for its refusal by labelling it a supply-chain risk, a commercially punitive designation the company is fighting in court.
Google’s agreement angered many of its employees, and flew in the face of the DeepMind founders’ previous concerns about the military use of AI. (A ban on military applications had been a stipulation of its sale to Google in 2014.) When I asked Legg about the issue, he declined to comment other than to say: “We’re going to have more and more difficult questions as this stuff is used in all sorts of ways.”
At Google’s annual developer conference, in May, the deployment of AI across the company’s product offerings was treated as cause for celebration. Pichai said that the company sees “AI as the most profound way to advance our mission and improve people’s lives at scale”. For many people, however, the sudden ubiquity of AI has been some combination of overwhelming, obnoxious and threatening. Nor is it reassuring to discover that the feeling that things are going too fast is shared even by people such as Hassabis, who, on a recent podcast, lamented the “ferocious commercial-pressure race that everyone’s sort of locked into”. What’s happening now, he said, is not how he’d hoped the development of AI would go, “where we would be contemplating this philosophically and carefully considering each next step. We’re not in that world.”
At this point it seems likely that LLM-powered AI will be at least as consequential as the smartphone, and maybe the internet. But still I can’t say that I’m pleased to see a “Write with Gemini” prompt appear whenever I stop for a few seconds to consider my next sentence in Google Docs. Still less am I eager to watch my children be used as guinea pigs for a dizzying new experiment in digital learning, or to discover what will happen to the global economy if the extravagant investments in AI can’t generate the short-term returns the markets demand. And while it’s not far-fetched to expect AI to enable breakthroughs that would justify the extreme amounts of energy it requires – better batteries, more efficient transmission grids, cures for serious diseases – I also don’t think “hope for the best” is a reasonable answer to people concerned about the climate crisis.
During my visit to DeepMind I met Helen King, who was one of the company’s earliest employees and now, according to her company bio, “sets Google DeepMind’s strategy for developing and deploying AI responsibly to benefit humanity”. I asked her how the rapid commercialisation of AI technologies has shifted Google’s approach to AI ethics. “We can’t prevent all risks, but we can make sure we are looking to mitigate as many of them as possible, and bring awareness to them,” she said. But she also insisted that some risks had to be managed by users themselves. “It’s like having a knife. A knife producer can’t guarantee how someone is going to use that knife. But they can put a cover on it so that it’s as safe as possible when it’s in a drawer. And make people really aware: this blade is sharp, do not use it in certain settings. That kind of thing.”
The simile struck me as unnervingly apt. Five years ago, LLMs were an exotic technology that was impossible to encounter without determined effort. Now they’re everywhere: on the internet, in our email inboxes, even in Google’s search results. I take King’s point that companies cannot reasonably be expected to eliminate every harm from a technology as powerful as AI: automobiles kill more than a million people a year, after all, and still we keep driving. But it’s one thing to keep a knife in a drawer with a cover snapped over the blade. It’s quite another to blanket every surface of our homes, classrooms and workplaces with blades while insisting that no one who isn’t using knives for everything will be able to survive the future.
These days at DeepMind, as in much of the industry, there is little doubt that AGI is close at hand. At the developer conference in May, Hassabis took the stage to declare that “AGI is now on the horizon”, and elsewhere he has suggested three to five years as a likely timeline. (One test he has proposed involves training an AI with all of human knowledge up to 1911 and seeing if it can come up with the theory of general relativity.)
Legg, meanwhile, told me that although today’s LLMs fall short of his definition of “minimal AGI” in several respects – including spatial and visual reasoning, metacognition and continual learning – he believes that these deficits will not persist for long. “There’s no magic remaining,” he said. “I think they’re all going to be solved in one, two, three years – who knows, maybe in six months. This is an area full of surprises.”
The conviction that the relevant question about AGI is no longer if but when has spurred a concomitant shift in the way frontier labs such as DeepMind are thinking and talking publicly about the consequences of advanced AI. Whereas previous work tended to focus on the ethical aspects of discrete products, such as models, chatbots and agents, today much more attention is being paid to the broader social effects of an AI-augmented world.
In some corners of Silicon Valley, of course, you can still hear people talk about AI as a universal panacea. If you accept the premise that a superintelligent AI will be able to think better about what’s best for us in every domain of life, then the solution to any problem that arises is easy. Economic crisis? Ask the robot. Political disagreement? Ask the robot. Food shortage? Ask the robot.
Alongside this fantasy, however, there’s been a more sober-minded recognition that the transition to a post-AI world may not be a smooth one. Legg, for instance, told me that he was looking forward to “fantastically great” benefits from AI, including “opportunities to address all kinds of nasty diseases” and “a general increase in all kinds of productivity in the economy”. Yet he also acknowledged that “increases in productivity usually come with some kind of disruption”.
Gabriel’s recent work at DeepMind is a useful indicator of the shift to a wide-angle perspective. Two years ago he and his colleagues were working out the ethics of AI assistants. Now, however, he leads a team of philosophers and social scientists investigating “how AGI will impact the economy, how it will impact the political sphere, how it will impact human relationships and how it will interact with science and technology”.
Gabriel expects that AGI will be transformative in a major way – potentially on the scale of the Industrial Revolution. Yet he believes, too, that AI is not something “before which the world becomes a frictionless entity”. He is also keenly aware that the Industrial Revolution was not a happy experience for many of the people who lived through it, even though it eventually raised living standards around the world: “Things got worse before they got better.”
Nevertheless, Gabriel does not think that the historical precedent settles the question, in large part because ordinary people individually and collectively have more power than they did 300 years ago. Though he was wary of sounding “too utopian and ungrounded”, he said he found it easy to imagine a world in which AI provides benefits that range from offering advice to curing diseases to improving economic growth in ways that benefit rich and poor alike. “If we can navigate the transition, navigate the power dynamics, navigate the risk successfully, there is a generalised potential for human flourishing on a level we haven’t seen so far.”
If the predictions about AGI’s arrival prove accurate, still broader questions may come to the fore as well. When I spoke to Edward Harcourt, at Oxford, he noted that “thinking about values and technological change is very hard because technological change always seems more of an upheaval in prospect than it does in retrospect, for the obvious reason that when we look back, we’re looking from the standpoint of values that have been shaped by the change in question. If you read people on the subject of the railways before they happened, they thought it was a complete catastrophe. And it’s true: the railways wrecked an entire way of life. Now we look back and we think: what’s the issue?”
Gabriel, too, thinks that AI might prompt changes that go even deeper than economics or technology. During the scientific revolution, he noted, “people experienced disenchantment when it was revealed that the world worked in certain ways. But they also gained new freedoms through that experience.” It will be up to us, he said, to decide which value changes we want to welcome, and which we choose to resist.
At one point in our conversations, Gabriel described himself to me as “a card-carrying humanist”: he is not the sort of person who looks forward to a day when superintelligent machines render humanity obsolete. Still, he recognises that as computers encroach on activities and capabilities that we have long held to be the special province of Homo sapiens – language, creativity, humour, taste – we find ourselves thrown back on some of the oldest and most difficult philosophical questions of all. Just as discoveries in physics, biology and astronomy led past generations to revise their understanding of what makes our species distinctive, he suggested, so, too, might AI prompt us to reconsider what it means to be a human being.
Leave a comment