Runway world model ranks robot policies
Runway says its GWM-Robotics world model ranked eight robot manipulation policies in close agreement with real-world results, a sign that some hardware testing could move into learned simulation.
Runway reported that its GWM-Robotics world model, first announced in December, ranked eight vision-language-action policies in nearly the same order as real-world testing. The simulated and real scores had a Pearson correlation of 0.95, and the Mean Maximum Rank Violation (MMRV) was 0.033.
The study used manipulation tasks from the RoboArena benchmark, all run on a Franka Emika Panda arm. GWM-Robotics is a variant of Runway's Gen-4.5 video model. It generated 1,450 rollouts from the same starting conditions as earlier RoboArena evaluations, and each policy chose its own actions based on observations produced by the world model. Human graders scored each rollout's progress toward the task. The rollouts received more than 16,000 ratings in total, about 10 per rollout.
Runway also tested the subset of policy architectures that PolaRiS had evaluated. PolaRiS is a real-to-sim framework that rebuilds specific scenes using 3D Gaussian splatting. On that subset, GWM-Robotics reached a correlation of 0.986 with an MMRV of 0, slightly above PolaRiS's 0.98 and matching its perfect rank order. Runway says its model needs only starting images, while PolaRiS requires scanning and manual scene setup. It also says GWM-Robotics produces real-time rollouts of up to 30 seconds, compared with 8 seconds for Veo Robotics.
The graders identified pi05_droid as the best policy and paligemma_binning_droid as the worst. Rankings flipped only between policies that performed similarly.
Runway lists several limits. The study measures policy ranking, not absolute success rates. It covers only tabletop tasks on a single robot arm. And it is the company's own evaluation. Runway says it is developing the model with partners including NVIDIA and Berkshire Grey, and access is available by request.