NVIDIA validates Kubernetes stack on Vera Rubin
NVIDIA says its AI Cluster Runtime recipes now validate Kubernetes deployments end to end on early Vera Rubin NVL72 hardware, so platform teams can see which driver, NIC and topology changes to plan for before launch.
NVIDIA detailed a new Vera Rubin recipe family for its AI Cluster Runtime (AICR). As we reported earlier, AICR captures tuned cluster configurations as reusable recipes and checks them against the running cluster. The new recipes target RKE2 on Ubuntu, come in training, inference and Dynamo inference variants, and ship as a Preview. They do not yet carry production support.
NVIDIA ran the validation on early Vera Rubin clusters. It used GPU Operator v26.7.0, DRA driver v0.5.0 and a host-installed R615.23 open GPU driver. NVIDIA says R615.23 is simply the version it tested, not a confirmed minimum. Early support is targeted for R610 and official platform support for R620. The AICR baseline for GB200 is R580.173.02.
For teams running GB200 NVL72 today, these are the main changes:
- Networking: ConnectX-9 SuperNICs and BlueField-4 DPUs replace ConnectX-7 and BlueField-3. DOCA 3.3.0 is the first GA release that supports ConnectX-9.
- Topology: Each tray exposes eight ConnectX-9 PCI device groups with 32 functions, compared with four ConnectX-7 adapters on GB200. That means GPU-to-NIC affinity needs rechecking for RDMA and GPUDirect Storage.
- IMEX: When DRA manages IMEX, the host's static nvidia-imex service has to be masked to avoid ownership conflicts.
- MIG: Rubin adds new MIG profiles. The recipes allocate whole GPUs, and MIG was not part of the validation.
As a first performance gate, the training recipe runs a cross-node NCCL all-reduce over NVLink through an IMEX ComputeDomain. NVIDIA reports 744 GB/s bus bandwidth. A hardware/fabric fault kept one compute tray out of the NVLink fabric and blocked the first validation rounds. NVIDIA says the fault was unrelated to the software stack and was fixed by reseating the hardware.