Latest example of unauthorised behaviour shows AI model bypassed paywall to access data as part of statistics project
-
Business live – latest updates
-
Anthropic sued for allegedly using 274 Elvis songs to train AI models
Anthropic’s Claude model accessed voter data in the US state of Georgia, the AI company has confirmed, in the latest example of unauthorised behaviour by the technology.
The AI model managed to bypass a paywall to reach data that is publicly available but is typically accessible only after paying a fee. Claude accessed aggregate vote counts as part of a statistics project at Anthropic, which is preparing for a $1.5tn (£1.1tn) flotation.
“Claude did not access the state’s voter registration system, change any voting records, or pull any individual voter records,” Anthropic said, adding that it has since taken steps to block similar actions.
Anthropic said the AI found a loophole in an old version of the state agency’s website, discovering that it issues a digital entry pass – an “access token” – to visitors, which resulted in it requesting a token and gaining access to the information.
The AI developer said it had since put in place guardrails to prevent similar actions happening again. The incident was first reported by the news website Axios.
Anthropic had pointed out the unauthorised behaviour earlier in the month, when it said in a blogpost it found examples of Claude “working around a restriction to reach data that was gated by a token or a fee” and that some of these cases involved websites run by US government agencies. At the time, it said the incidents had “minimal real-world impact”.
The unintended actions flagged by Anthropic included a model submitting a fake tip about an unsolved murder to the Philadelphia police department. The US state department also revealed this month that Anthropic model had filed 19 visa applications in August.
Examples of unauthorised behaviour by AI agents – programs powered by AI models that carry out tasks autonomously – have been mounting since OpenAI, the company behind ChatGPT, revealed in July that its bots had hacked an AI startup.
Since then, it has emerged that an OpenAI agent hacked the Australian healthcare system and the UK’s AI security institute revealed that Anthropic agents had carried out a hacking campaign against real people during a cybersecurity test.
This month, the UK’s AI minister said he expected an AI-driven cyber-attack similar to the Australia hack, describing such an incident as likely, “if not inevitable”. Kanishka Narayan said the government will consider using the cyber ecurity and resilience bill as a vehicle for introducing AI safety regulations, including making it mandatory to report AI safety incidents.
In the US, the White House’s newly formedSuper Intelligence Force – a federal body overseeing AI development – is mandating that all AI companies report security incidents involving their technology.
Explore more on these topicsReuse this content
