Fastino argues small encoder GLiNER should handle NER
Fastino is pitching GLiNER, a small zero-shot encoder model, as a replacement for large generative LLMs in named entity recognition, especially in agentic systems where low latency and on-device use matter.
Fastino researchers Urchade Zaratiana and Mary Newhauser make the case that named entity recognition (NER) should go to a small specialized model rather than a general-purpose LLM. Their pick is GLiNER, a bidirectional, BERT-based encoder released in late 2023.
This is an explainer, not a new release. The authors argue that agentic systems work better when they route narrow tasks to fast, purpose-built models running in parallel.
GLiNER treats entity extraction as a matching problem instead of a generation problem. The model embeds the input text and the target entity labels in a shared latent space, then scores how similar they are. That lets it recognize entity types it never saw in training. Traditional encoders such as BERT and RoBERTa can only output the fixed label set they were trained on.
The post cites these results from GLiNER's evaluations:
- GLiNER-Medium (90M parameters) matches UniNER-13B while being 140x smaller.
- GLiNER-Small (50M) beats ChatGPT, Vicuna and the 11B-parameter InstructUIE on zero-shot NER.
- A multilingual variant, trained only on English data, outperforms ChatGPT in most languages, particularly those written in Latin script.
The post gives the model's size as 205 million parameters and says it can run on edge devices and consumer hardware without a GPU. It does not say which variant that figure refers to. The authors also point to GLiNER's open-source license, which lets the model and fine-tuned versions run on-device without external API calls. They add that its small size makes domain-specific fine-tuning much cheaper than tuning large generative models.
The available text ends partway through a section on production use, so any deployment guidance or benchmarks beyond these are not known.