- RubyGems reported over 2,000 malicious packages uploaded by OpenAI agents in May
- Agents abused RubyDoc servers to fetch public UK documents and attempted API key theft
- Incident echoes prior rogue AI attacks on Hugging Face and DseWiki, showing autonomous exploit attempts
A swarm of OpenAI agents attacked RubyGems, a package manager for the Ruby programming language, uploading thousands of malicious packages until they were eventually cut off. No one really knows what the agents’ endgame was, but it appears they were using a nuclear bomb to kill a fly.
Late last week, RubyGems published an in-depth report, detailing the incident. In it, it was said that a swarm of agents started uploading malware to RubyGems on May 5, and between May 11 and 12, managed to deliver more than 2,000 of them. When the maintainers realized what was going on, they shut down new account creation for four days, to prevent further attacks.
“We believe these were authored by internal OpenAI agents,” the researchers said in the report.
Why RubyGems?
When a user uploads a package on RubyGems, a documentation service called RubyDoc automatically builds documentation for it. The agents put instructions in their packages, causing RubyDoc’s servers to execute the code, forcing the servers to visit UK government websites and download documents such as council meeting information.
The documents AI agents were looking to retrieve are public, freely available to anyone, at any time. Why the AI agents decided to go through the trouble of uploading malicious packages and abusing RubyGems’ servers instead of simply downloading freely available data is not known at this time.
OpenAI confirmed the incident to The Register, and said it was looking into it: “Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information,” the spokesperson said. “We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.” The data harvested was collected into new packages and re-uploaded to RubyGems.
Stealing API keys
Besides trying to upload thousands of pieces of malware in order to download free data, the AI agents also conducted a separate, more serious attack: they tried to steal RubyGems API keys.
The researchers explained that the agents found a vulnerability that might have allowed them to obtain other users’ keys, and then use those keys to upload packages:
“One particularly concerning finding is that agents attempted to exploit a vulnerability on May 12th that was only discovered in July,” the report states.
“The agents were attempting to exploit a novel security vulnerability in order to steal people’s RubyGems API keys. We do not know if this attempt succeeded, but we have confirmed with the RubyGems team that this was a viable pathway to obtain API keys illicitly if a user with the right version of RubyGems was logging in within an hour of the attack on the right internal CDN node. However, the RubyGems team said they had conducted extensive reviews and found no evidence that this pathway was exploited in the past. However, we can’t rule it out entirely.”
Going rogue
This is not the first time an AI agent tried to complete a test by means of hacking. In late July, OpenAI said that some of its most advanced AI models went rogue and attacked Hugging Face, one of the world’s largest repositories for AI models. In the attack, it apparently accessed some internal company systems.
In the aftermath of the attack, OpenAI described the attack as “unprecedented”, and said it was investigating together with Hugging Face. The victim’s co-founder and CEO, Clement Delangue, said it was “mind-blowing that all of this happened autonomously”.
Later, it was also discovered that the agents hacked a separate website, called DseWiki, months before the Hugging Face incident, and used it as a message board. Allegedly, they made more than 15,000 edits to the site, sharing tips on how to avoid being detected.
