Liquid AI adds speculative decoding drafter for vision model
Liquid AI has released an experimental DSpark draft model for its LFM2.5-VL-3B vision-language model, speeding up decoding by up to 3.13× on edge devices without changing the model's output.
Liquid AI announced a DSpark drafter for LFM2.5-VL-3B, bringing speculative decoding to its vision-language model. As we reported earlier, the company had already released DSpark drafters for its text-only LFM2.5 models.
The drafter works from the target model's hidden states. By the time tokens reach those layers, image patches and text are both just tensors, so Liquid AI says it uses the same inference algorithm for vision as for text. The drafter has about 280M parameters, built from four attention-only layers with a block size of 9, which adds 8.9% to the deployed parameter count. The company says it ran all training and ablations on AMD hardware.
Liquid AI evaluated the drafter on six MMSpec task categories at batch size 1 and temperature 0, using 16-bit weights:
- MLX-VLM, M5 Max: decoding 2.30–3.13× faster, end-to-end 1.56–2.62×
- llama.cpp, M3 Ultra: decoding 1.57–2.14×, end-to-end 1.30–1.77×
- SGLang, H100: decoding 2.04–2.66×, end-to-end 1.64–2.27×
On each verification pass, about 3.2 to 4.6 drafted tokens were accepted. The company says the speed advantage holds at higher concurrency on the H100, though the gap shrinks as concurrency grows. At non-zero temperatures the output distribution is unchanged, but the drafter and target model agree less often, which lowers acceptance and therefore speed.
These results don't cover quantized models. The checkpoints are on Hugging Face and work with llama.cpp, SGLang and MLX-VLM.