Artificial Analysis launches AI personality benchmark
Artificial Analysis has released AA-PersonalityBench, which uses LLM judges to score 36 models on how they write by default, giving builders a way to compare models beyond accuracy.
Artificial Analysis has launched AA-PersonalityBench, a benchmark of the personality and communication style of AI models. The firm's premise is that "two models can give equally correct answers and still feel very different to use."
The benchmark covers 30 personality traits, each tested with 30 questions, for 900 prompts in total. Every model answers each prompt once, sent as a single user message with no system prompt, so the scores describe default behaviour. A panel of LLM judges compares pairs of responses blind. Judges see the trait definition, the question and the two answers, but not model names or reasoning text. Response order is randomized to avoid position bias.
The verdicts produce per-trait Elo scores with 95% confidence intervals. These are reported as percentile ranks and rolled up into eight indexes: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop. Artificial Analysis says it measures how strongly a trait shows, not whether the trait is desirable. Separately, it measures 21 communication characteristics in code, including length, structure, punctuation and emphasis.
The sample results cover five models. On Warmth, Claude Fable 5.1 indexed at 94, ahead of Claude Opus 5.5 (80), Gemini 4 Argon (71), GPT-6 Astra (68) and Claude Sonnet 5.5 (65). Gemini 4 Argon used the most bold text, averaging 17.2 instances per response. GPT-6 Astra used the least, at 8.8.
The full question set is private to avoid contamination. Two sample questions per trait are published. The benchmark relies only on LLM judges, and the announcement does not say which models sit on the panel.