Nvidia speeds up FLUX 3 Klein up to 4.45x
Nvidia says stacked TensorRT LLM VisualGen optimizations make Black Forest Labs' FLUX 3 Image Klein up to 4.45x faster on a single B200, which matters for teams serving high-resolution diffusion models.
Nvidia published a tutorial on running Black Forest Labs' FLUX 3 Image Klein in TensorRT LLM VisualGen, a runtime inside TensorRT LLM. In 2048×2048 tests on one B200 GPU, the combined optimizations made text-to-image generation up to 3.46x faster and image-to-image up to 4.45x faster for Klein Base. The baseline was uncompiled BF16 inference.
Klein pairs Qwen3-VL-4B-Instruct for text conditioning with a diffusion transformer (DiT) and an autoencoder. It ships in Base and distilled versions. BFL trained the model on GB200 NVL72 systems.
Denoising takes up most of the generation time: 96.8% of the 14.70-second text-to-image run for Base and 84% of the 2.71-second run for the distilled model. That is why Nvidia focused its work on the DiT. Reported gains include:
- NVFP4 weights: up to 1.62x faster text-to-image and 1.58x faster image-to-image than BF16, using BFL's prequantized checkpoints.
- FlashInfer PrimsTS GEMM kernels: tuned for FLUX 3 shapes with fused epilogues, 1.02–1.10x faster than the autotuned TensorRT LLM NVFP4 GEMM.
- FP8 attention: BF16 attention took about 60% of DiT kernel time. VC-Attention FP8 brought speedups of 1.18x for Base and 1.16x for the distilled model, with LPIPS scores of 0.086 and 0.151. cuDNN FP8 gave 1.08–1.09x.
Nvidia recommends VC-Attention FP8 for this setup. The tutorial also covers sparse attention, caching and multi-GPU parallelism, with YAML configurations included. The material provided does not say how much each of those adds to the final speedups.