OpenAI opens MentalHealthBench, built with 80+ clinicians

OpenAI has published MentalHealthBench, an open benchmark for scoring how AI systems respond in realistic mental health conversations, extending its clinician-informed HealthBench line beyond the emergency-only scenarios that dominate existing evaluations.

The dataset is built from synthetic conversations generated with privacy-preserving techniques and intended to mirror real usage patterns; some include user background, such as a recent death in the family, to test whether models tailor responses to context. Scenarios span three acuity levels — non-acute, high-acuity, and emergencies — and four personas: adults, teens aged 13–17, caregivers, and clinicians. The teen persona is signaled through a system message, an approach OpenAI notes works across providers but "may not capture all safeguards built into individual products." Teen conversations were reviewed by clinicians specializing in youth mental health.

Rubrics were written by more than 80 licensed psychologists and psychiatrists across 22 countries, speaking 19 languages and covering nearly 20 subspecialties. Each criterion carries a weight from -10 to +10, rewarding beneficial behaviors and penalizing harmful ones. Every conversation was reviewed by at least three experts, and criteria were retained only if two agreed and no third contradicted. Grading is automated, using GPT‑5.6 Sol against the expert-written rubrics. Overall scores decompose into ten expert-defined behavioral dimensions, including safety, seeking context, preserving user agency, and providing actionable guidance.

OpenAI says results show "steady improvement" across evaluated models but does not state scores in the material reviewed here; a paper covers grading and evaluation settings in detail.