Is Anthropic’s IPO compatible with its ‘safety-focused’ constitution?published at 16:11 GMT
Freya Scott-Turner
Live reporter
Image source, Eleanor DrageDr Eleanor Drage says that safe development by major AI companies is possible but it needs to be done at “moderate” pace
Founded in 2021 by seven ex-employees of OpenAI, Anthropic has cultivated something of a reputation as “the good guy” of AI.
“If powerful AI is coming regardless, Anthropic believes it’s better to have safety-focused labs at the frontier,” says its constitution.
But recently, former employees have accused Anthropic of neglecting this mission – including a researcher who previously told the BBC that workers were “genuinely frightened” about the speed of its advancements.
Dr Eleanor Drage, a senior AI ethics researcher at the University of Cambridge, says the idea that Anthropic needs to win an “AI arms race” and out-build competitors in the name of responsibility is “a kind of hype fest” and a “narrative” to justify a risky pace of development.
The problem is the “culture of development”, Drage tells me. AI companies “don’t want to [slow down], because they don’t know how. They’re so terrified they’ll lose their competitive edge”.
Because public shareholders enter the mix, IPOs bring in a larger group of people who want to see big financial returns. Is this compatible with safety?
“We have to believe that there is a compromise,” says Drage, who notes that engineers say that safe AI development is possible if we’re “temperate and moderate” about the systems we build. “But that’s just not what’s happening,” she adds.
Leave a comment