Matryoshka Embedding Technology

Embedding models typically generate fixed-size vectors, but recent models have shown a trend of continuously increasing output dimensions to improve performance. This causes reduced efficiency in downstream tasks such as search or classification.

Matryoshka Embedding was designed to solve this problem. Like Russian Matryoshka dolls, this approach trains the model to prioritize storing important information in the earlier dimensions. This allows a significant portion of the original performance to be maintained even when only part of the full embedding is used through truncation, rather than using the entire embedding.

The main advantages of this technology are as follows:

• Increased Efficiency: Reducing embedding dimensions can speed up search and save storage space.
• Flexible Dimension Selection: The same model can generate embeddings of various sizes, allowing adjustment according to service requirements.
• Performance Retention: Since key information is concentrated in the earlier dimensions, the performance loss that occurs when reducing dimensions is very minimal.

Hugging Face provides a guide on how to train such models using Sentence Transformers and how to apply them in practice.