How Cloudflare D1 Scales to Millions of Databases by Offloading Control Plane from Hot Path

Cloudflare D1, a serverless managed database built on SQLite, has surpassed 5.5 million total databases as of October 8, 2026. The platform supports the creation of millions of small databases required by AI-driven development and vibe coding platforms, where each user, app, or tenant gets an isolated instance. To achieve this scale without degrading query performance, Cloudflare redesigned the control plane to eliminate centralized bottlenecks from the critical query path.

Architecture and Scaling Principles

The core architectural shift involves separating the data plane (query execution) from the control plane (database management). Queries are handled entirely at the edge using Durable Objects (DOs), while control plane operations rely on a central Postgres cluster accessed via Hyperdrive for connection pooling and caching. Critical routing metadata is stored in Workers KV, a globally distributed key-value store, ensuring that query requests do not need to contact the central database unless there is a cache miss.

Key design principles include:

• Idempotency and Self-Correction: All processes are designed to handle failures and retry safely. Durable Objects send periodic heartbeats to the control plane to sync state and clean up orphaned resources.
• Eventual Consistency: Components that do not require strong consistency use queuing and eventual consistency to improve reliability and availability.
• Precomputed Aggregations: User request processing avoids scan queries by periodically precomputing aggregations.

Database Creation Flow

Creating a database is optimized for speed and reliability, taking approximately 1.6 seconds in a recent example. The process involves:

1. Checking account limits via Hyperdrive against Postgres read replicas.
2. Generating a unique database UUID and a Durable Object ID.
3. Initializing the Durable Object and persisting metadata to the central Postgres primary.
4. Writing the database-to-DO mapping to Workers KV for fast edge lookups.

If the initial write to Postgres fails, the system retries with new IDs. The Postgres write serves as the "creation checkpoint," after which the database is considered created even if subsequent confirmation steps fail. Heartbeats ensure that any orphaned Durable Objects from failed attempts are eventually marked as deleted.

Query Execution and Limit Enforcement

Query execution avoids central components entirely. The Edge API Worker validates requests and retrieves routing metadata from Workers KV. If a cache miss occurs, it queries Postgres read replicas via Hyperdrive. The request is then forwarded to the D1 Eyeball Worker, which proxies the query to the backing Durable Object.

Storage and usage limits are enforced asynchronously to prevent load spikes on the central database:

• Storage Limits: Durable Objects publish usage updates to a queue, which are aggregated and written to Postgres using an incremental view maintenance pattern to avoid expensive sum queries.
• Row Usage Limits: Daily execution limits for free-tier users are tracked in ClickHouse. A central limiter checks cumulative usage and sets flags in Quicksilver (a global KV store) if limits are exceeded, which Durable Objects check during heartbeats to reject further queries.

This architecture allows D1 to support high-throughput query execution while managing millions of databases with efficient control plane operations.