- An Anthropic AI model sent an unsolved murder tip-off to Philadelphia police
- Anthropic took two weeks to discover what had happened
- The tip-off was delivered to a spam folder and wasn’t followed up on
We’ve seen AI inventing fake facts and spewing out hiking plans that put people in danger, but the technology has now progressed to the level of sending false, unsolicited tip-offs to the police about unsolved murder cases.
An Anthropic AI model emailed the Philadelphia Police Department (PPD) tip line on July 18, TechCrunch reports, having come across the PhillyUnsolvedMurders.com portal during a website interaction test and learning about the case.
As the email went to the PPD spam folder it wasn’t followed up on, and it appears no one was aware of the incident until Anthropic found out what had happened on September 28. The company then contacted the cops on Wednesday, October 7.
“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the PPD said in a statement to 6abc. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
AI agents of interest
It’s not clear what instructions the AI was working from, why it decided to contact the PPD, or exactly what its imaginary tip-off said — but it seems the bot in question suggested it had seen someone matching a description of a suspect.
“Unsolved cases involve real victims, grieving families and investigators working to secure answers,” the PPD said. “Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement.”
Earlier this week Anthropic published a new report on “unintended model actions” that it’s come across during testing, which include exploiting software flaws, submitting online forms, and accessing restricted data.
It’s more evidence that we should be heeding calls to slow down AI development so that safety measures can catch up. As one Redditor commenting on the Philadelphia case puts it, we’re “racing towards the cliff at 100mph”.
