NVIDIA releases validated Kubernetes recipes for Dynamo
NVIDIA's AI Cluster Runtime bundles its Dynamo inference framework with pinned GPU, scheduling and monitoring components, so teams can get a working inference endpoint on Kubernetes without assembling the stack themselves.
NVIDIA has published a walkthrough on deploying its Dynamo distributed inference framework on Kubernetes with NVIDIA AI Cluster Runtime (AICR). AICR is part of the company's DSX AI Factory Platform. It packages Dynamo and the infrastructure under it into "validated recipes." Each recipe fixes component versions and records compatibility constraints, configuration values, node selectors, tolerations, checksums and validation checks for a given environment and workload.
The recipes ship the dynamo-platform chart, pinned to version 1.4.2. The chart includes the Dynamo operator, the runtime images and the nvidia.com/v1beta1 DynamoGraphDeployment CRD, which replaces the earlier v1alpha1 API. The supporting components are:
- the NVIDIA GPU Operator
- Node Feature Discovery
- the DRA Driver for NVIDIA GPUs
- KAI Scheduler for topology-aware and gang scheduling
- Grove for orchestrating multi-component serving graphs
- NVSentinel and Nodewright for node health, where the recipe supports them
- a Prometheus stack
- agentgateway with Gateway API resources for Kubernetes-native ingress
There is one architectural change. In Dynamo 1.4, frontends, workers and Gateway API Inference Extension/Endpoint Picker (EPP) components find each other through Kubernetes resources. As a result, AICR's default path no longer uses NATS, and NATS is now opt-in. On the default path, requests travel over TCP and KV-cache events over ZMQ. Each vLLM worker publishes its cache events locally, and Dynamo republishes them.
The post compares two ways to route requests to workers that already hold the relevant KV blocks: Dynamo's own frontend router, or GAIE/EPP at the gateway layer. NVIDIA says this matters for agentic workloads, which keep extending long prompt prefixes. Dynamo also supports disaggregated serving, which lets operators scale prefill and decode capacity separately. The excerpt available ends before the full routing comparison and the deployment steps.