Liquid AI releases IFStruct structured-output benchmark
Liquid AI has released IFStruct, a benchmark that tests whether language models follow requested JSON and YAML schemas, and reports that its RL-trained 350M-parameter model outperforms much larger models on it.
Liquid AI has introduced IFStruct, a generative benchmark for structured output. It checks whether a model's response is valid and matches the schema it was asked for. The content of the response is not scored.
The company says constrained decoding can guarantee syntactically valid output, but it cannot make a model pick the right fields, values or escaped content. IFStruct is designed to measure that remaining gap.
Each prompt asks for a few instances of a randomized item, such as recipes or invoices, in JSON or YAML. The schema is presented in one of several styles:
- plain-English bullet lists
- raw JSON Schema
- annotated examples
- flat path glossaries
- ASCII tables
An LLM rewrote half of the samples into natural chat requests. Some of these use a "thinking-out-loud" style, where the user changes their mind before settling on a final spec. Many prompts deliberately test string escaping by asking for multiline text, embedded quotes, code snippets, file paths or stack traces.
Scoring is binary. A sample passes only if it gets every requested constraint right: format, top-level shape, item count, code fencing, whether commentary is allowed, and the exact fields, types, enums and numeric bounds. Any key that wasn't requested counts as a failure. Liquid says the difficulty is set so that frontier models can score close to 100%.
The company's main result is that the task is very learnable. LFM2.5-350M, trained with reinforcement learning on a separate training split, reportedly beats Qwen3.5-4B and IBM's granite-4.0-h-tiny. The excerpt provided does not include specific scores, dataset size or licensing details.