Baseten gets Flux.2 image generation under one second
Baseten says its inference stack serves Flux.2 [dev] 2.3x faster than SGLang on NVIDIA B200 GPUs, cutting latency below one second for a workload where single-request speed drives both user experience and cost.
Baseten reported speedups for two open image-generation models on its Baseten Inference Stack. Against SGLang, Black Forest Labs' Flux.2 [dev] ran 2.3x faster on NVIDIA Blackwell (B200) GPUs and 1.9x faster on Hopper (H100). Qwen-Image ran 1.6x faster on B200 and 1.1x faster on H100.
The company says these gains come from a combination of changes:
- Quantization matched to the hardware: FP4 on B200, FP8 on H100
- Memory optimizations
- Optimized attention kernels and specialized element-wise kernels
- Improvements to the serving runtime
FP4 on B200 gave the best Flux.2 [dev] result and brought latency below one second. On H100, FP8 and the memory changes meant Flux.2 [dev] no longer needed CPU offload, which nearly halved latency. For Qwen-Image, FP8 helped on both GPUs, and FP4 gave the best B200 result at a 1.57x speedup over baseline.
Each benchmark measured one request on one GPU: a single 1024x1024 image, 8 inference steps and a fixed seed. Prompts ranged from 32 to 4,096 tokens. Baseten says prompt length did not meaningfully change latency, so it reported one figure per configuration. The post does not give absolute latency numbers beyond the sub-second claim, and it does not say whether FP4 or FP8 quantization affects image quality.
Baseten says the same stack can serve Qwen-Image-Layered, Flux.2 [klein] and other Flux and Qwen-Image variants. It plans more tuning for specific workloads on both GPU generations.