PostgreSQL Scaling Strategy to Support 800 Million ChatGPT Users

PostgreSQL is the core data system underpinning ChatGPT and the OpenAI API. Even as database load has surged more than 10x over the past year, OpenAI is handling millions of QPS through a single Azure PostgreSQL flexible server primary and roughly 50 read replicas distributed across multiple regions worldwide.

However, a single-primary architecture shows vulnerabilities when write load spikes sharply. In particular, PostgreSQL's MVCC (Multi-Version Concurrency Control) approach requires copying the entire row on write operations, causing write amplification and read amplification, along with table and index bloat and complex autovacuum tuning challenges.

To overcome these limitations, OpenAI is pursuing the following strategies:

• Migrating write-intensive workloads to shardable systems such as Azure Cosmos DB
• Minimizing unnecessary writes through application logic optimization
• Configuring new workloads to use sharded systems by default

Currently, PostgreSQL remains unsharded. Sharding existing workloads is an extremely complex and time-consuming task. Instead, OpenAI is focused on maintaining its current architecture, optimized for read-heavy workloads, while securing sufficient headroom for future growth.