NVIDIA Releases Nemotron 3.5 Embed, Rerank, and Parse Models for Agent Retrieval

NVIDIA released five Nemotron 3.5 models on October 21, 2026, including Embed 8B/1B (BF16 and NVFP4), Rerank 8B, and Parse, designed to improve multimodal retrieval and reduce inference costs for AI agents.
Nemotron 3.5 Embed 8B BF16 achieved an nDCG@10 of 61.17 on ViDoRe v3.1, ranking first among dense retrieval models as of October 20, 2026, and outperforming Qwen3-VL-Embedding-8B by 4.3 points on MIRACL Vision.
Integrating Nemotron 3.5 Rerank 8B with the Embed models reduced agent token costs by approximately 14% for 8B embeddings and 13% for 1B embeddings while improving retrieval accuracy.
Nemotron 3.5 Parse, a 631M parameter model, ranked #1 on ParseBench for models under 1B parameters with a score of 75.20 at 10.5 pages per second on an NVIDIA H100.

Performance and Efficiency

Nemotron 3.5 Embed 1B NVFP4 delivered a 27.1% throughput increase over the 1B BF16 model on NVIDIA GB200, processing 128.8 documents per second while maintaining 99.85% retrieval accuracy on ViDoRe v3.
NVIDIA NIM served 6,031 query requests per second on RTX PRO 6000 Blackwell Server Edition, roughly 2.9 times the throughput of a comparable vLLM configuration.
The 1B Embed model retains 95.4% of the 8B teacher model's score on ViDoRe v3.1 with an nDCG@10 of 58.38.

Models are available via Hugging Face, NVIDIA NIM, and partners including Baseten, DeepInfra, and FriendliAI.
The 1B Embed model is distilled from the 8B version using cosine and MSE losses, with the language component frozen to allow reuse of existing text vector indexes.
A fine-tuning recipe using synthetic data and hard-negative mining improved nDCG@10 by 18% relative for the 1B model on chart and infographic benchmarks.