Pruna squeezes Flux 2 Dev onto one H100
Pruna AI says it compressed Black Forest Labs' 110GB Flux 2 Dev to fit on a single 80GB H100 at about 2.5 seconds per image, lowering the hardware needed to deploy the open-weights model.
Inference-optimization startup Pruna AI says it has compressed Black Forest Labs' newly open-weights Flux 2 Dev image model so it runs on a single H100. The work was done with Black Forest Labs and Replicate, and the optimized model is available through a Replicate endpoint.
Unoptimized, Flux 2 Dev needs 110GB of memory just to load. That is more than an 80GB H100 holds, so in practice it needs a B200 with 190GB. Pruna puts base latency for a 1-megapixel image on a B200 at 10–11 seconds, or 7–8 seconds with compilation. Its optimized version produces the same image in about 2.5–2.6 seconds on one H100. The company calls this 4.2× faster than the base model on a B200.
Pruna says it combined caching, factorization, quantization, compilation and "several proprietary tweaks." It says it explored and validated that combination in a couple of days. It has not given the final configuration or the memory footprint after compression.
The company also says the speedup comes "without compromising output quality," but the post includes no quality metrics to back that up. For now it points readers to an image gallery and says image-quality numbers will be added to InferBench, its own benchmarking site.
Flux 2 Dev supports output up to 4 megapixels, up to eight reference images, hex-code color matching and structured prompts of up to 32K tokens. Pruna's benchmarks cover only 1MP generation. It notes that latency rises with higher resolutions and more reference images.