turbopuffer Announces v3 Architecture to Decouple Storage from ANN Index

turbopuffer is introducing turbopuffer v3, a fundamental change to its storage architecture that moves the Approximate Nearest Neighbor (ANN) index from being the primary index to a secondary index. This shift aims to support a wider variety of query plans at greater scale by decoupling document storage from vector clustering constraints.

Why the Change?

The previous architecture, where documents were keyed by their ANN address, created three main bottlenecks:

• Storage Amplification: Multi-vector documents (e.g., late interaction) required duplicating non-vector data for every vector, leading to inefficient storage usage.
• Write Amplification: Updates or deletions triggered SPFresh rebalancing, which cascaded to moving full document contents and associated inverted indexes, hitting diminishing returns in indexing throughput.
• Limited Vectorization: Query engines benefit from processing large blocks of data (e.g., 2,048 rows in DuckDB), but the ANN cluster size (~100–200 docs) constrained block sizes for non-vector queries like aggregations and scans.

v1 and v2 Context

• v1: Used SPANN and later SPFresh for hierarchical clustering, keying everything by ClusterId and LocalId (the ANN address).
• v2: Added attribute filtering and BM25 full-text search. While FTS v2 improved performance by reworking postings into fixed blocks, other query plans remained constrained by the ANN-primary layout.

Current Status

As of early September 2026, 100% of CI passes on turbopuffer v3. The team notes that this milestone represents a significant performance regression compared to production, as the focus has been on foundational design and correctness. Performance tuning is now underway, with benchmarks to be published as optimizations are implemented.