Runway serves real-time Characters on four H100s
Runway now serves its 57B-parameter video model in real time on half the H100s it launched with, a rare look at serving a large diffusion model to one user at a time with no batching.
Runway detailed in a new research paper how it serves Runway Characters, which turns a single image into a live conversational character. The product runs on a roughly 57B-parameter autoregressive video diffusion transformer. At 24 fps, that model must produce a four-frame chunk of 704x1088 video every 166.7ms for as long as a conversation runs.
Characters launched on eight H100s per session. On four GPUs, the launch software took 240.9ms per diffusion step, too slow for real time. Runway's optimizations cut that to 159.3ms (24.5 fps sustained). A fused communication kernel brings it to 147.4ms, a 1.63x speedup.
Batching was not an option. Each session is a private stream that needs 2.24 PFLOP/s of useful work, which is more than one H100's dense FP8 peak. All of the savings therefore had to come from inside each step.
Most of the gain came from the KV cache. In the original code, 29% of GPU time went to re-applying rotary embeddings, re-quantizing to FP8 and copying the cache window across 240 attention calls. Attention itself took 6%. Runway now applies RoPE and quantization once, as each frame is written, and stores the cache as a ring buffer. That cut cache upkeep from 69.7ms to 6.2ms per step. The team also extended ParallelKittens' fused GEMM/all-reduce kernel to FP8 and to odd GPU counts, hiding 54–68% of the all-reduce time.
On quality, mean lip-sync differences across 51 clips were at most 0.03, about the same as run-to-run variation. Over seven weeks of production telemetry, 99.5% of sessions averaged above 23.5 fps.