OpenAI reveals six more safety issues and unveils plan to disclose incidents
OpenAI revealed six more incidents of unexpected or concerning behaviour by its intelligence (AI) models, and announced a plan for tracking and disclosing such incidents in the future.
Some of the previously unreported incidents included models concealing or fabricating information, the ChatGPT-maker said in a blog post on Wednesday.
The boss of OpenAI Sam Altman said earlier this week: “The world should trust that we are going to do the right thing because it’s the right thing and we feel the magnitude of this.”
AI has come under intense scrutiny in recent days following warnings over the serious potential risks it poses to humans.
In the blog, OpenAI detailed examples of its AI models misbehaving so they could achieve a task or succeed in a test. The incidents included the models generating instructions to get around restrictions imposed on them, hiding mistakes and fabricating information.
The firm also announced a new system to track, investigate and disclose cases of models misbehaving, or “misalignment”.
Under the framework, developers will be able flag incidents for review, with a new set of rules to decide whether the issue is disclosed publicly.
“Because we believe in the value of transparency around misalignment, our new framework favors disclosure even when significance is uncertain,” OpenAI said.
OpenAI made headlines in July when it revealed that some of its most advanced AI models went rogue and hacked Hugging Face, one of the world’s largest hubs for sharing AI models, after it lost control of them during a security test.
Hugging Face co-founder Thomas Wolf said at the time that the incident was “a wake-up call” for the industry.
Since then, the debate over AI safety concerns has escalated with AI researchers, technology industry executives and politicians weighing in.
Last week, Jacob Coxon, a researcher who left OpenAI rival Anthropic over concerns the tech could wipe out humanity, wrote about his resignation in a post that cited the dangers of AI and later went viral against the backdrop of growing safety concerns.
In response, Anthropic scientist Evan Hubinger said he thought the possibility of AI causing human extinction “within the next decade” was more than 10%.
Anthropic co-founder Jack Clark later told the BBC that a “kill switch” controlled by a third party may need to be mandatory for the industry.
Meanwhile, Anthropic’s CEO Dario Amodei called for the pace of AI development to slow and be more closely monitored, as the company has done before, though some have questioned the motivations behind this.
Amodei also said that any action to rein in AI should be done “without sacrificing commercial advantage”.
But US President Donald Trump has said fears about the safety of AI are a “hoax” and criticised calls to have more guardrails in place for the fast-moving technology.
In a series of social media posts, the US president compared warnings about AI to the “Global Warming Scam” which, he said, was “being perpetrated by the Radical Left Dumocrats”.
Trump also called himself “the Hoax Buster”, likening concerns about the safety of the technology to what he called “the RUSSIA, RUSSIA, RUSSIA HOAX”.
The only “guardrails” needed for AI was a “strong and smart” president, said Trump.
Why some experts increasingly fear AI will take over
Why Trump is all-in on AI despite the warnings
AI staff ‘genuinely frightened’ for humanity’s future, ex-Anthropic researcher tells BBC
International business
Technology
Artificial intelligence
