All stories
Qwen launches 2.4-trillion-parameter Qwen4-Max

Qwen launches 2.4-trillion-parameter Qwen4-Max

Qwen has released Qwen4-Max, a sparse mixture-of-experts model built on a new architecture, and says it outperforms its predecessor on agentic work while cutting average cost per task by 62%.

Qwen announced Qwen4-Max, the first model on its new Qwen4 architecture. It has a 2.4-trillion-parameter backbone, activates 71B parameters per token and adds 328B N-gram embedding parameters. It supports vision natively and has a one-million-token context window.

The company describes three main architecture changes. The first pairs Gated DeltaNet layers with a new Qwen Sparse Attention (QSA) mechanism: three GDN layers compress history into fixed-size states for each QSA layer, which uses a lightweight indexer to pick relevant context. The second is a "Gated Residual" design with four parallel residual branches. The third is an N-gram embedding at the second layer, which adds capacity with little extra compute. Qwen says pre-training ran without loss spikes. On its in-house evaluations, the base model scored 5–13% lower bits per byte than Qwen3.8-Max-Base while activating 25% fewer parameters.

On Work360, Qwen's own benchmark of industry tasks covering 721 specialized fields, Qwen4-Max scored 52.69, up from 47.74 for Qwen3.8-Max. Qwen reports large drops in token use: 24–35% fewer tokens on Work360 and about 48–49% fewer on Terminal-Bench 4.0, depending on effort setting. API prices are lower than for Qwen3.8-Max, with the biggest cuts to cached-input pricing. On Alibaba's API the model costs $1.50 per million input tokens and $4.50 per million output tokens, according to Artificial Analysis. Qwen says average cost per Work360 task fell from $4.43 to $1.67.

The post also shows demos in Blender, in the Unity, Unreal and Godot game engines, and in automotive manufacturing deployments at SANF and Pacific Union.

Artificial Analysis independently scored Qwen4-Max at 43 on its Intelligence Index, above the median of 41 for comparable models. The model was very verbose in those tests: it generated 127M tokens on the index, against a median of 78.6M, and cost $3.27 per task to evaluate. Artificial Analysis lists its context window at 984k tokens.

Vals AI ranked Qwen4-Max #11 of 38 on its Vals Index (63.20%), the top Chinese model, ahead of Kimi K3.1 Pro (62.52%) and up from 59.05% for Qwen3.8-Max 0902. It placed #8 of 41 on Finance Agent (66.68%) and the Vals Finance Index (61.99%) and scored 48.49% on Terminal-Bench 4.0, but trailed on Legal Research Bench (41.35%, tied #32 of 95). Vals tested it at xhigh reasoning effort and lists 262k max output tokens.

The headline results come from Qwen's own evaluations. Artificial Analysis lists Qwen4-Max as proprietary, with weights not publicly available.

Sources

Updates

  • Added Vals AI benchmark rankings (Vals Index, finance, coding, legal) and 262k max output tokens
  • Added Artificial Analysis Intelligence Index score, per-token API pricing and proprietary-weights status