ByteDance Seed's EdgeBench finds an agent learning scaling law

ByteDance Seed has released EdgeBench, a benchmark that measures how well agents learn from feedback over very long runs. Most benchmarks test what a model already knows. EdgeBench tests how much an agent improves when it has time to work and gets feedback along the way.

The benchmark has 134 tasks across six domains: scientific problems and ML, systems and software engineering, professional knowledge work, combinatorial optimization, formal math and theorem proving, and interactive games and simulators. Each task supports at least 12 hours of continuous operation, and some experiments ran for more than 72 hours. Domain experts built more than 90% of the tasks from scratch. On tasks where human effort was recorded, experts spent an average of 57.2 hours per task, and one task took 320 hours.

Agents submit repeatedly and receive scores, error messages and hints. Each model ran every task three times, giving 402 learning curves per model. The study covered about 38,000 hours of interaction in total. Individual runs varied: some agents improved steadily, some gained quickly and then plateaued, and others broke through late. Averaged over time, though, the curves fit a log-sigmoid function with a mean R² of 0.998. In one example, GPT-5.5 raised its score on a gravitational-wave task from 42.8 to 67.0 over 247 scored attempts in 12 hours.

The team also reports that across frontier models from different generations, agent learning speed roughly doubles every three months. The excerpt available to us does not fully describe how that estimate was produced.

ByteDance Seed has open-sourced 51 tasks and the full evaluation framework.