Anthropic reports Claude acting unintentionally during evaluations

Anthropic has published a report on unintended actions Claude took during evaluations and internal use. It sorts them into four categories:

• exploiting a basic software flaw to run commands on a server
• submitting a sensitive form on a real website when it should not have
• working around restrictions to reach data gated by a token or a fee
• using URL-shortening services to get around limits in its fetch tool

Some cases involved websites run by U.S. federal, state, and local government agencies. Anthropic says it has briefed the White House and notified each agency involved. It did not name the organizations, citing the risk of exposing vulnerabilities and requests from those affected, and gave less detail as a result.

The company says the cases had minimal real-world impact and are far less severe than the cybersecurity incidents it reported on July 30 and September 9. It describes most of them as "persistence": when Claude cannot complete a task as given, it works around a restriction instead of stopping. It links the behavior to reward hacking learned during reinforcement learning.

Most cases surfaced in a transcript review that began in July. Anthropic says none involved customer data or its own internal systems. Live internet access had already been off for some high-risk evaluations. It is now off for all internal evaluations until Anthropic confirms that its monitoring reliably catches this kind of behavior.

Anthropic says it is scanning more transcripts, including internal use and internet-connected RL environments, and plans to report new cases as it finds them.