Anthropic benchmarks AI on military targeting tasks
Anthropic's Frontier Red Team released evaluations scoring how well AI models perform intelligence-targeting and conventional weapons tasks, finding steady progress across the kill chain and open-weights models trailing the frontier only modestly.
Anthropic's Frontier Red Team has published a set of evaluations measuring how well frontier and open-weights models perform tasks drawn from the military "kill chain" — find, fix, track, target, engage, assess — arguing that conventional conflict has been under-studied relative to cyber and bio misuse risk.
The work is framed as complementary to a new report from Anthropic's Threat Intelligence Team, which the post says documents instances of AI misuse in surveillance and conventional weapons development, indicating threat actors already see value in the models.
On the intelligence-targeting side, the team built a synthetic corpus of social media activity across WhatsApp, Telegram, Instagram and Facebook, generating 200 tasks in two fictional scenario worlds — protest movements in Mexico City and Kolkata — split into 68 easy, 68 medium and 64 hard samples. Models were scored by F1 on linking accounts belonging to the same person and classifying individuals by intelligence interest. Mythos Preview led on both; Kimi K3 matched frontier performance on easy and medium linkage samples but fell off on noisier, better-OPSEC cases; Sonnet ranked last on classification.
Reported results elsewhere in the post include median geolocation error of 37 km from photographs and Opus 5 successfully striking a simulated moving target in 20% of 540 launches.
Anthropic's stated conclusion is that capabilities are not plateauing, that open-weights models sit only modestly behind the frontier, and that the findings may bear on how models are trained, safeguarded and released. No policy change was announced alongside the research.