turbovec: 10 Million Documents in 4GB RAM, Faster Search than FAISS

Indexing 10 million documents with float32 requires 31GB of RAM, but turbovec compresses this to 4GB. It is a Rust vector index based on Google Research's TurboQuant algorithm, allowing vectors to be added online without separate training steps or parameter tuning. It provides Python bindings for immediate integration into existing pipelines.

Using hand-written SIMD kernels such as ARM's NEON SDOT/SMMLA and x86's AVX-512 VNNI, it achieves faster search speeds than FAISS IndexPQFastScan. It is on average 3.4x faster at 4-bit precision and 23% faster at 2-bit precision. When an ID allowlist is passed during search, filtering is performed inside the SIMD kernel, reducing unnecessary computational costs. Incremental storage that saves only changed parts ensures crash safety while achieving millisecond-level add/delete latency even on large indexes.

It provides a drop-in replacement for the default in-memory vector stores of LangChain, LlamaIndex, Haystack, and Agno. IdMapIndex preserves external IDs even after deletion and supports O(1) complexity delete operations. It is suitable for building RAG stacks that consider both privacy and latency in a purely local environment where data never leaves the local machine or VPC.