Sentence Transformers Adds Support for ColBERT Style

The Sentence Transformers v6.0 update introduces a new model type, MultiVectorEncoder, which supports ColBERT-style Late Interaction retrieval.

While existing Dense embedding models compress entire texts into a single fixed-size vector, potentially causing information loss, Multi-Vector models retain embeddings for each token and use the MaxSim (Maximum Similarity) operator to perform fine-grained matching between queries and documents.

Key features include:

• Improved retrieval quality: Outperforms existing models on queries with complex requirements or specific entity matching (e.g., product codes, surnames).
• Visual Document Retrieval: Supports SOTA (State-of-the-art) capabilities that directly match text queries against page images without an OCR process.
• High compatibility: PyLate and Stanford-NLP ColBERT checkpoints can be loaded and used immediately via the familiar existing API.

This approach incurs the cost of larger index sizes compared to single-vector models, but it is highly useful for building RAG (Retrieval-Augmented Generation) and semantic search systems where maximizing retrieval accuracy is critical.