Baseten cuts delta weight sync to 15 seconds
Baseten now syncs a ~770 GB GLM-5.3 checkpoint into live rollout GPUs in 15 seconds with a 0.8-second pause, which shortens RL training loops and frees host memory for KV cache.
Baseten says it has cut delta weight syncs for a ~770 GB GLM-5.3 checkpoint from 36 seconds to 15. The request pause fell from 6 seconds to 0.8 seconds. The company calls this 7.5x faster than the previous state of the art and says it raises throughput by up to 3.2x.
The main change is that Baseten no longer keeps a full copy of the parent checkpoint in CPU memory. Its earlier design needed that copy to decode XOR-based deltas. On an 8×B300 node, the copy forced the host KV cache down from 1.1 TB to 275 GB, and updating the copy to each new policy took about 45 seconds.
The GPU now holds the only copy of the weights. A native decoder reads zstd's commands directly and keeps only the changed bytes. Each GPU then reads the old values at those positions, applies the XOR mask and checks the result against the trainer's Adler-32 checksums. Because each byte affects the checksum independently, Baseten can verify an update without rereading the unchanged bytes. The company says this made updates up to 2.3x faster: a 30 GB Kimi K3 delta dropped from ~230 to ~100 seconds. All of this preparation happens while the old policy keeps serving. The pause only covers writing the new values and publishing the version.
Other changes include splitting each delta by the GPU that owns each change and moving update planning out of SGLang's Python process.
Without the host copy, Kimi K3 (1.7 TB in MXFP4) can now run RL rollouts across two GB300 nodes. Baseten reports 75 seconds from request to live weights on an 8.4 GB delta, with a 4.1-second pause. The source text ends before Baseten's KV cache offloading measurements.