Harvey and Baseten post-train 27B legal agent
Harvey and Baseten Research say post-training a 27B open-weight model moved it into the closed-source frontier band on Harvey's Legal Agent Benchmark, where top proprietary agents cost roughly $50 per task.
Harvey, working with Baseten Research, described a pipeline that post-trains open-weight models against its Legal Agent Benchmark (LAB) with a long-horizon legal-agent harness in the loop. LAB has more than 1,200 tasks across 24 practice areas. Each task is graded against an expert-written rubric, and an agent passes only if it meets every criterion.
The team says the pipeline brought a 27B open-weight model into the closed-source frontier band on a hold-out set. The post excerpt does not name the model or give its final score. On that hold-out set, Sonnet 4.6, Opus 4.7 and GPT-5.5 lead, with GLM 5.1 and DeepSeek v4 close behind. The post says even the strongest frontier models complete fewer than 10% of LAB tasks end-to-end. Reaching the top of the leaderboard costs roughly $50 and more than 20 minutes per task.
In an earlier experiment, a light GRPO run on Qwen3.5-9B used only the LAB rubric as reward. After 40 steps, its criterion pass rate on the hold-out set rose from 42.5% to 63.0%. The model's grep calls fell 67% and it read over 20% more characters per rollout. It ended up reading documents in full, the same approach the frontier models use.
To scale that approach to large client matters, the team tested a simple compaction harness. The agent periodically writes a structured memo that replaces its chat history. This added 5–7 points to the average criterion pass rate. The available text ends at that point.