Even for a topic as hot as artificial intelligence, talk of AI has ratcheted up to yet another level this week: on the one hand we’ve got the CEOs of the biggest companies in AI calling for a slowdown on the technology, and on the other you’ve got the President of the United States advocating for the exact opposite.
If you’re wondering why there are renewed calls for AI regulation, whether or not AI has the capability to kill everyone on the planet, and how this might impact the technology going forward, we’re here to explain it in a straightforward way for you.
Many experts are now insisting that we’re at a point in AI development where ‘pacing the frontier’ — slowing down the cutting-edge of development — is the safest and wisest move.
However, not everyone agrees, with opposing perspectives suggesting that AI doomerism is purely a marketing stunt and a distraction.
1. Why are AI bosses asking for a slowdown?
In a rare show of unity, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and SpaceX CEO Elon Musk have all called for a slowdown in AI development in recent days, arguing that better safeguards and guardrails need to be put in place around the technology before models are allowed to progress much further.
“We must slow the pace at which we improve the capabilities of AI models,” wrote Amodei in a blog post, a stance which Altman and Musk later agreed with. According to Amodei, whose company develops Claude, AI is now advancing “drastically faster” than ever before — in part because it can now upgrade itself — and that rate of progress, together with the recent incident where AI agents hacked the Hugging Face platform, have convinced Amodei that action needs to be taken.
The Anthropic CEO wants to see third-party evaluators embedded in the biggest AI companies, as well as coordinated safety standards across the industry, and between countries — something that might be challenging to achieve, considering that Chinese authorities have described calls for a slowdown as “fearmongering” and a “Cold War” move intended to keep the US ahead of China in AI development.
2. Could AI one day kill us all?
The top-level CEOs haven’t been the only ones warning about the dangers of AI in recent days. Anthropic AI developer Jacob Coxon very publicly quit his job over fears about the safety of the technology he was working on, saying that “people building AI earnestly believe that it could kill us all by the end of the decade”.
More recently, Google Deepmind Research Engineer Bilal Chughtai posted a similar resignation message on X stating that he is “extremely concerned by the default trajectory of this technology” and that he believes “AI has the potential to kill us all, and that we might be running out of time to avoid this outcome”.
This isn’t a new idea, as Stephen Hawking was saying the same over a decade ago. What’s changed is how powerful AI has become: it can now, in theory, improve itself and spread itself across the internet in a way that escapes human control, potentially hacking into essential infrastructure and outpacing any defensive measures.
Imagine the internet, cell networks, payment platforms, and GPS suddenly not working, for anyone — things would likely deteriorate pretty quickly. The other AI doom scenario that’s often talked about is bad actors using AI to launch biological or nuclear weapons.
AI isn’t that advanced yet, but it might be soon, and some experts think there’s a greater than 10% chance of it happening in the next few years.
3. How could regulation help?
Those in the AI industry are now actively asking governments for help in reining in the technology, believing that any slowing of the pace in AI development is going to require state-level coordination. That regulation might include oversight bodies, ‘kill switches’ to shut down AI, and limits on how and when new AI models can be released.
While there’s a lot of momentum behind the idea that regulation is necessary, there’s little agreement about how to go about it. AI safety legislation efforts have stalled in both the US and the UK, amidst arguments about how tight any restrictions should be and who should be in charge of them.
One sticking point is over the potential harms of AI regulation. Would, for example, a limit on data center expansion end up hurting economic growth and smaller AI businesses, as well as companies such as Anthropic and OpenAI? If regulation is going to happen, it’s going to take a while.
4. Why is Trump calling AI concerns a hoax?
One of the reasons that serious AI regulation seems unlikely in the short term is that US President Donald Trump has loudly rejected calls for a slowing down of AI development, saying that “whoever wins AI, wins!” and calling AI safety fears a “hoax” concocted by the bosses at Anthropic, OpenAI, and SpaceX.
The President’s point of view is that the guardrails that are already in place — essentially AI model oversight by the US government — are enough to keep AI from doing any harm. Trump will also be concerned about ceding ground to China, and the economic hit that may result in slower AI advances.
There’s a huge amount of money invested in AI right now, from the companies themselves to the infrastructure that they rely on to run everything. If it takes longer to see returns on those investments — and OpenAI chief Sam Altman has already said his company’s IPO is being pushed back over safety fears — then that could have significant economic impacts.
5. What could happen to our AI apps?
Assuming you’re not using ChatGPT to develop bioweapons or hack into government mainframes, none of this noise around AI will affect the apps you use day-to-day at the moment. If you do try and do something malevolent with a chatbot, like writing computer viruses, you’ll see there are many safety protocols in place to stop you.
What the AI companies are concerned about is keeping it that way. If they manage to coordinate some kind of slowing down and regulation, then you might see apps like Claude or Gemini updated less often. At the moment, new AI models are appearing on a regular basis, but that won’t necessarily continue to be the case.
One possibility is that agentic AI, which is AI that’s able to carry out tasks and program itself, becomes more tightly controlled even as other aspects of the technology improve.
What’s certain is that this discussion is a long way from being over, and that those who know best say that we’re at a “crunch time for humanity”.
