OpenAI's storage layer hits 70M requests per second

OpenAI published the first of a two-part engineering series on Habitat, the online storage platform it built so product engineers at the company never have to touch database management directly.

Written by members of technical staff Jon Lee, Chaomin Yu and Ben Ries, the post says Habitat now handles more than 70 million requests per second, serves over 500 petabytes of data across nearly 40 geographic regions, and backs products used by more than 1 billion people each week. It began in mid-2024 as a simple Python client-side library sitting in front of a single Azure Cosmos DB instance. OpenAI says the system has grown more than 10x year-over-year for three consecutive years — well past the usual practice of designing for one 10x step and hoping it holds.

The post's substantive argument is about why OpenAI moved Habitat from a client library to a standalone service by mid-2025. Rolling out a regional sharding change to reduce single-region blast radius required coordinating a feature-flagged client update across dozens of services; each fix took days, and a team rolling back to an older buggy client ultimately caused the outage the work was meant to prevent. Centralizing gave OpenAI one control point for deployments, observability, access control and audit logging.

Part two is promised on multi-tenancy reliability, read-path optimization and the Azure Cosmos DB partnership.