Insiders at the firm fear tech’s advancement could cause human extinction, while others are calling their declarations of concern a ‘setup’
A day after a former researcher at Anthropic made an apocalyptic declaration about artificial intelligence, more researchers and staff members at the AI startup publicly agreed with him and posted their own dire warnings.
In response, Elon Musk and other conservative figures on X, formerly Twitter, called the chorus of concerns a “setup” and a “psyop”.
The Anthropic insiders fear the technology they are building could become so advanced and dangerous as to cause human extinction within the decade, and they assert many of their colleagues feel the same but haven’t said so publicly.
These researchers and staff are posting in response to a now-viral thread from Anthropic researcher Jacob Coxon, who said on Wednesday he was resigning from the company because neither Anthropic nor its competitor OpenAI, also his former employer, were building AI models responsibly and are “gambling with our lives”.
Anna Wang, who works on Artificial General Intelligence Safety at Anthropic and previously worked at Google’s DeepMind, said Thursday that many people at Anthropic want to slow down development to come up with a plan to mitigate the risks that could come with these AI models.
“There is not yet a viable scientific plan to solve risks from recursively self-improving AI,” Wang wrote on X.
Drake Thomas, another Anthropic employee, said that he respected Coxon’s choice to abstain from building these models if he worries they could pose planet-scale risks.
“Things are moving way too fast, we don’t have anywhere near the degree of assurance we’ll want for ASI [artificial superintelligence],” Thomas said.
Musk took a more adversarial stance. “Seems like a setup,” he wrote.
Coxon replied to the world’s richest man with a selfie, writing, “I’m real and these are my real beliefs. You could ask your xAI researchers about me if you hadn’t fired them.”
In a statement to the Guardian, an Anthropic spokesperson defended the company’s strategy. “We have always been transparent that AI will bring both enormous benefits and unprecedented risks. To address these risks, we continue to build models with some of the strongest safeguards in the industry,” the statement said.
Musk and others with a more positive view of AI have started to spread theories that Coxon’s post and the resulting fallout are part of a “psyop” to negatively influence the public’s feelings about AI.
“I think the groundwork for this psy op (for lack of a better term) has been prepared for a long time,” Musk posted on X. “This was just the match that lit the fire.”
Musk was responding to a post by Parker Thayer, a researcher at conservative thinktank Capital Research, in which he floated a theory with little evidence that Coxon’s post was the start of a “VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion”. Other notable figures took notice.
Bill Ackman, the billionaire CEO of the hedge fund Pershing Square, also quoted Thayer’s post, simply saying “Interesting.”
Other Anthropic employees had posted their agreement with Coxon shortly after he declared his resignation on Wednesday.
“AI developers believe their technology could cause human extinction (or similarly bad outcomes),” wrote Samuel Marks, who works on safety research at Anthropic. “This could happen in the next few years. In general, the more senior the employee, the more concerned they are.”
Evan Hubinger, who describes himself as a lead in the company’s alignment division, which works on ensuring Anthropic’s AI models function in line with human goals, said Coxon was “correct” and that the industry was falling behind in attempts to deal with the apocalyptic potential.
“We really do earnestly believe AI could kill all humans!” Hubinger wrote. “I personally think it is >10% within the next decade.
It’s not exactly clear how Coxon and others expect AI models to usher in the demise of humanity. Some experts doubt that the technology will become so intelligent as to cause the apocalypse. Gary Marcus, a scientist and leading voice in AI, said that it’s time to boycott AI, but because it’s already causing harm.
More than extinction, Marcus said he worried about the “risk of catastrophe” from “from AI-generated pathogens, from wars started or escalated by AI-generated disinformation, from hacks that destroy critical infrastructure, and so on. Nothing I have seen gives any indication that any of that is under control”.
The same day, Anthropic released a report documenting how the company dismantled an operation attempting to build a biological weapon using its AI models.
- AI (artificial intelligence)
- Anthropic
- news
