Hugging Face Launches Embedding Container for SageMaker

Hugging Face has achieved general availability (GA) of the Hugging Face Embedding Container, which enables easy and secure deployment of embedding models on Amazon SageMaker. This container optimizes the deployment of embedding models, which are essential for building RAG (Retrieval-Augmented Generation) applications.

The container is based on Text Embedding Inference (TEI), a high-performance embedding model serving solution, and offers the following technical benefits:

• Inference optimization: Optimized code leveraging Flash Attention, Candle, and cuBLASLt, skipping the model graph compilation step.
• Efficient resource management: Small Docker image size and fast boot times, with support for token-based Dynamic Batching.
• Operational stability: Safetensors weight loading and production environment support via Open Telemetry and Prometheus metrics.

It supports various architectures including BERT, RoBERTa, NomicBert, and JinaBert, and can serve major open embedding models such as Snowflake Arctic and BGE with high performance in the SageMaker environment.