Qwen releases Qwen-VLA robot action model
Qwen's new Vision-Language-Action model uses a single generalist policy for manipulation, navigation and cross-embodiment control, and on several benchmarks it matches or beats models fine-tuned for one task.
Qwen has introduced Qwen-VLA, a general-purpose Vision-Language-Action model built on its multimodal backbone. The Qwen model reads visual observations, language instructions and embodiment-specific conditions, and an action decoder outputs continuous actions or trajectories. Robotic manipulation and vision-language navigation share this one framework.
Training runs in four stages. In Stage I, text-to-action pretraining, the VLM is frozen and only the action decoder is trained, on language and embodiment prompts with no images. Stage II, continual pretraining, trains both parts on the full multimodal data mix and produces Qwen-VLA-Base. Stage III, supervised fine-tuning, branches into a multi-task track and a real-robot track. Stage IV uses PPO reinforcement learning in SimplerEnv only and yields Qwen-VLA-Instruct. Qwen says the RL gains carried over to environments and robot embodiments the model had not seen.
The training data includes more than 10,000 hours of public robot trajectories and over 1,000 hours of internal real-robot data. It also covers human egocentric datasets such as Ego4D, EgoDex and EgoVerse, about 7.2 million synthetic text-to-action trajectories, navigation data and general vision-language data.
Qwen reports these results for Qwen-VLA-Instruct:
- LIBERO: 97.9%, against 98.6% for the best specialist (ABot-M0)
- RoboCasa-GR1: 56.7%, against 58.3% for ABot-M0
- Simpler-WidowX: 73.7%, against 64.6% for StarVLA-OFT
- RoboTwin Easy/Hard: 86.1% and 87.2%, against 86.0% and 85.0% for ABot-M0
- VLN-CE R2R Val-Unseen: 57.5% success rate
- VLN-CE RxR Val-Unseen: 59.6% success rate
Qwen says the navigation results beat all open-source baselines. The source text available to us does not cover model size, licensing or whether the weights will be released.