Artificial Analysis profiles Hermes 4 405B
Artificial Analysis has added Hermes 4 - Llama-3.1 405B to its benchmark listings, which puts a large open-weights fine-tune of Meta's Llama into its standard tables for intelligence, cost and speed.
Artificial Analysis has published a release page for Hermes 4 - Llama-3.1 405B. The release has two variants, one reasoning and one non-reasoning, and the page compares their intelligence, cost and latency.
According to the page, the model has 406B total parameters, a 128k-token context window, and text-only input and output. The weights are on Hugging Face under the Meta Llama 3 Community License Agreement.
The only performance number in the page text is latency. The non-reasoning variant has the lower time to first answer token of the two, at 5.14 seconds.
Because the model is open-weights and has more than 150B parameters, Artificial Analysis puts it in its "Large" open-weights class. Its intelligence, cost and speed metrics are compared only against other models in that class. The non-reasoning variant is also compared only with other non-reasoning models.
The page lists capability indexes built from several Artificial Analysis evaluations, including AA-Omniscience v1.1, AA-Briefcase v1.1, HLE-Diamond, AutomationBench-AA, GDP.pdf, Terminal-Bench 4.0 and FrontierCode-AA 2.0. The page text doesn't include the scores themselves, Intelligence Index results, per-token pricing, or output speed for either variant. It also doesn't name the developer or give a release date.