Google DeepMind launches MKA benchmark; Claude Fable 5.2 leads

Google DeepMind announced the Multimodal Knowledge Agent (MKA) benchmark, built with Artificial Analysis to test how agents reason across visually complex files in real-world knowledge work. Starting now, developers can explore the benchmark and evaluate their own models on Artificial Analysis, which has launched the evaluation exclusively on its platform. Google has also released a separate set of 10 public tasks on Hugging Face so anyone can inspect the format and test their own agents.

MKA has 200 private tasks, which Artificial Analysis runs and keeps private so they can't leak into training data or be found through search. Google says the tasks were created with domain experts across 10 verticals and 83 occupations, and that each one represents a multi-step project that takes a seasoned professional three to eight hours. Agents work from dense inputs such as architectural drawings, engineering schematics and clinical scans, and produce slide decks, spreadsheets, PDFs, Word documents and marked-up images. Artificial Analysis says the visual inputs are needed to complete each task rather than serving as extra context.

Expert rubrics average about 70 weighted yes/no checks, weighted 75% on factual accuracy and 25% on visual design, according to Google. Artificial Analysis splits them into baseline "Regression" checks and harder "Climb" checks; the headline metric is Climb Weighted Satisfaction Rate (WSR). Tasks run in Stirrup, Artificial Analysis's open-source agent harness, for up to 250 turns, with code execution, web search, web fetch and image-viewing tools. GPT-6 Luna (medium) grades the output as an agentic judge, and Google says domain experts agreed with its grades 96% of the time. Google's technical report found the models it tested used between 30 and 130 turns on average per task.

In Artificial Analysis's results, Claude Fable 5.2 (max) scored 57%, ahead of Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%. Fable 5.2 averaged $31.97 per task, compared with $6.29 for Gemini 4 Carbon and $1.11 for GPT-6.1 Sol. Google puts Carbon's cost advantage over the top model at 6x on average, while Artificial Analysis's per-task figures work out to about five times. All four OpenAI models tested were on the cost-performance Pareto frontier below Carbon.

No model led all 10 industries. Fable 5.2 led five, and Gemini 4 Carbon led Healthcare. Word and text-file outputs were the hardest deliverables, and no model scored above 52% on either.