NVIDIA ships AIPerf, successor to GenAI-Perf
NVIDIA has published a walkthrough for AIPerf, a ground-up rewrite of its GenAI-Perf LLM load-testing client designed so the benchmarking tool itself stops being the bottleneck under real concurrency.
NVIDIA is positioning AIPerf as the designated successor to GenAI-Perf, and it is a clean architectural break rather than an upgrade: it no longer runs on top of Perf Analyzer, and NVIDIA points existing users to a migration guide for the deltas.
The core change is process architecture. GenAI-Perf, like most benchmark clients, was single-process and became GIL-bound under real concurrency or request rate — meaning measurements reflected the client's limits as much as the server's. AIPerf splits the work across worker processes that generate load and separate record-processor services that handle results, coordinated over ZMQ.
Coverage is the other pitch. NVIDIA lists support for 15+ endpoint types including chat, responses, NIM rankings and image generation; public datasets such as ShareGPT; and trace replay formats from Mooncake, Baseten and WEKA (AgentX). Load shaping includes constant, Poisson and gamma arrival patterns with tunable burstiness, gradual ramping, and vLLM/SGLang range-ratio synthetic distributions for variable input and output lengths.
Reported metrics center on time to first token, inter-token latency, request latency and output token throughput, each broken out at p25 through p99 with min, max, mean and standard deviation, written to console, CSV and JSON.
Install is via uv tool install aiperf; on aarch64 the crick dependency is source-only and needs a C toolchain. The post's walkthrough uses Qwen3-0.6B served through vLLM as a demonstration, not a benchmark result — NVIDIA publishes no performance numbers of its own.