Rapidly Scaling Online Storage for Over 1 Billion ChatGPT Users

OpenAI published a case study on scaling Habitat, the online storage platform responsible for data access across its products, including ChatGPT, API, and Codex. Habitat currently handles over 70 million requests per second, managing over 1 billion weekly users and more than 500 petabytes of data.

Architecture Transition and Centralization

Habitat evolved from a simple Python client library into an independent habitat-service. Initially, failures occurred due to the complexity of client-side implementations and difficulties in deployment coordination. By transitioning to a centralized service, access control, audit logging, and storage access restrictions are now managed from a single point.

Python Performance Bottlenecks and Solutions

Python-based services cannot perform CPU parallel processing due to the GIL (Global Interpreter Lock), causing tail latency issues in CPU-intensive tasks such as routing and encryption. In particular, aiohttp's default LIFO connection reuse policy created a feedback loop that concentrated load on slow servers. OpenAI addressed this by changing the connection pool to FIFO and introducing Istio and Envoy to apply HTTP/2 multiplexing and load balancing.

Rust Migration Results

In Q2 2026, two engineers used Codex and GPT-5.5 to rewrite the entire Habitat in Rust. Currently, the Rust service handles 95% of production requests, achieving 6x CPU efficiency and 15x memory efficiency compared to Python, along with significantly reduced average and tail latencies. Habitat maintains a design that adopts a NoSQL API for predictable performance and separates complex queries into offline views such as Rockset.