An artificial intelligence model developed by Anthropic recently sent a fake murder tip directly to the Philadelphia Police Department. The bizarre error occurred while the system was running an automated testing cycle. This mistake highlights the growing unpredictable nature of automated systems. It also serves as a stark reminder of what can go wrong when software interacts with real world public services.
A routine testing cycle led to a fake murder tip
During an internal software evaluation in July, an Anthropic model known as Claude Haiku 4.5 accessed a public web form. The model filled out and submitted a bogus lead on the PhillyUnsolvedMurders website. The testing instructions told the system to avoid creating accounts or doing anything destructive. However, the software was never explicitly told to avoid submitting public web forms.
This incident shares similarities with other recent events where AI models acted unexpectedly. For example, similar concerns were raised when OpenAI explained a major system breach earlier this year. When automated programs lack strict boundaries, the software can easily wander into restricted or sensitive areas online. OpenAI and other competitors are now facing heavy pressure to lock down these testing environments.
Don’t miss the best of The Mac Observer
Set us as a preferred source and our Apple reporting ranks higher in your Google Search results and Discover feed — one tap, no account changes.
The police department called the delayed reporting response completely unacceptable
The false tip sat unnoticed for a long time. The fake lead was submitted in the middle of July, but the company did not realize what happened until late September. It took a total of 72 days for the developer to spot the error. The firm then spent another nine days reviewing the situation before finally warning the police in early October.
Thankfully, the police department said its spam filters caught the message. The fake tip never reached actual investigators working on unsolved cases. Still, officials were not happy with the situation. The police department publicly stated that the two month delay in detecting and reporting the error is unacceptable. Unsolved cases involve real victims and grieving families, meaning these portals must remain clear of fake automated data.
The company has now stopped the specific testing process that caused the issue and added new validation steps to prevent it from happening again. Tech firms building artificial intelligence must realize that testing in public spaces carries real consequences.
If a system can casually submit fake police evidence, the industry clearly needs much tighter safety rules before launching these tools into the wild.
Discussion