20 Years of Amazon S3, the Way It Builds What's Next

Amazon S3 quietly launched on March 14, 2006, with just a one-paragraph announcement. It started from two very simple primitive operations—storing with PUT and reading with GET—but the core idea was to let developers focus on higher-level problems instead of complex infrastructure.

From the beginning, S3 was designed on five principles. Security for baseline protection, Durability at the 11 nines (99.999999999%) level, Availability designed assuming failure will happen, Performance that minimizes degradation even at nearly unlimited data scale, and Elasticity that scales up and down automatically without manual intervention.

20 years later, S3 has reached an entirely different scale. At launch it held about 1 PB, with 400 storage nodes, 15 racks, 3 data centers, 15 Gbps of bandwidth, and a maximum object size of 5 GB. Today it holds more than 500 trillion objects, handling over 200 million requests per second across 123 Availability Zones and 39 AWS Regions worldwide. The maximum object size has grown from 5 GB to 50 TB, and unit pricing has dropped from 15 cents per GB at launch to just over 2 cents today.

Throughout this journey, S3 has become not just an AWS service but the de facto standard for the S3 API. As multiple vendors began offering S3-compatible storage, the skills and tools developers learned on S3 became portable to other storage environments.

What's even more striking is that code written for S3 in 2006 still runs today. The internal infrastructure has been completely transformed through multiple generations of disks and storage systems, and the request-handling code has been rewritten, yet data and API compatibility have been preserved throughout.

Continuous engineering innovation lies behind this scale. S3 maintains durability through a microservices-based auditing system that continuously inspects every byte across the entire fleet, triggering recovery systems immediately upon detecting anomalies. S3 also applies formal methods and automated reasoning in actual production to mathematically verify the correctness of the index subsystem, cross-Region replication, and access policies.

On the performance and safety front, over the past 8 years S3 has been progressively rewriting performance-critical paths in Rust. Components related to blob movement and disk storage have already been converted, and work continues on other core components. Rust's type system and memory safety serve as an important safeguard for a system like S3, which operates at massive scale with extremely high correctness requirements.

S3's design philosophy is "Scale is to your advantage." This means the system is designed so that as scale grows, workload correlation decreases, which actually improves reliability. Greater scale doesn't disadvantage more users—it returns better characteristics to more users.

The future of S3 aims to go beyond a simple object store to become a common foundation for all data and AI workloads. The direction is to store data once and use it directly, without moving it into separate specialized systems. This reduces cost and complexity, and cuts down on redundant copies of the same data.

Representative features from recent years include the following.

• S3 Tables: fully managed Apache Iceberg tables optimized for analytics workloads
• S3 Vectors: native vector storage for semantic search and RAG, supporting up to 2 billion vectors per index with sub-100ms query latency
• S3 Metadata: centralized metadata that enables instant data discovery without recursively listing large buckets

All of these features run on top of S3's cost structure. Work that once required a separate database or specialized system can now be handled economically within the same storage layer. From 1 PB to hundreds of EB, from 15 cents to the 2-cent range, and from a simple object store to a foundation for AI and analytics—this 20-year story ultimately comes down to one thing. The principles of security, durability, availability, performance, and elasticity have not changed, and the code written in 2006 is still alive and running today.