AI BriefingUEmbed: A Decoder-Only Multimodal Embedding Model Released
Key point
Researchers have introduced UEmbed, a multimodal embedding model that integrates sparse and dense retrieval in a single forward pass using a decoder-only architecture.
Details
UEmbed is a multimodal embedding model developed by researchers from the Chinese Academy of Sciences (CASIA), Alibaba, and Yale University. It maintains a decoder-only architecture while performing both sparse and dense retrieval simultaneously in a single forward pass.
To overcome the limitations of existing Learned Sparse Retrieval (LSR) models, which primarily rely on bidirectional encoders (such as BERT) or are restricted to text, UEmbed is designed to operate effectively under Causal Attention.
Core Technology: Vocabulary Splitting To address the information bottleneck that occurs when a single token must represent a massive vocabulary space, the model introduces 16 learnable special tokens. The vocabulary is divided into 16 subsets via k-means clustering, with each special token responsible for only its assigned portion of the vocabulary to generate a complete sparse vector. For dense embeddings, the hidden state of the EOS token immediately preceding the special tokens is utilized.
Key Results
- Achieved scores of 71.8 in dense mode and 71.0 in sparse mode on the MMEB-v2 benchmark.
- This marks the first reported sparse multimodal retrieval results on MMEB-v2.
- Performance in sparse mode alone surpassed several existing dense-only models.
- Weights for three model sizes—2B, 4B, and 9B—have been released.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
The Normalization of AI Translation: Principles of LLM-Based Translation and Structural Differences from NMT