NVIDIA splits GLM-5.3 across two DGX Stations
NVIDIA has published a guide to serving its 753-billion-parameter GLM-5.3 NVFP4 model across two linked DGX Stations with vLLM, because the model is too large for one machine's GPU memory.
NVIDIA published a tutorial on serving a frontier-scale open model on local hardware. The setup connects two GB300 DGX Stations over ConnectX-8 RDMA and serves the model through one OpenAI-compatible endpoint.
The model is NVIDIA's GLM-5.3 NVFP4, a quantized version of Z.ai's mixture-of-experts model. It has 753 billion parameters, 40 billion of which are active per token. Sparse activation reduces compute, but every expert's weights still have to stay in memory, because each token can be routed to different experts. The checkpoint is about 464 GB. A single GB300 GPU has 252 GB of HBM3e, so the model doesn't fit on one Station.
The guide uses vLLM (container v0.30.0-aarch64) with tensor parallelism set to two (TP=2). This splits the model's tensors across both GPUs, and CPU weight offload is turned off. Both Stations compute every request. Station A (rank 0) hosts the API, and Station B (rank 1) runs only a worker. Each machine needs its own copy of the checkpoint and at least 464 GB of free disk.
Before deployment, the post has readers check the link between the machines: a 400 Gb/s link, MTU 9000 with jumbo-frame pings, and synchronized clocks. NVIDIA says these checks catch cabling and addressing mistakes before they show up as NCCL timeouts. Users must also accept the NVIDIA Open Model Agreement and the original GLM-5.3 license.
The tutorial ends with load testing using SPEED-Bench prompts. The excerpt reviewed here stops before those results, so throughput and latency figures aren't available.