Stanford and NVIDIA Release 'Contrastive Language Model' CLM with 9x Faster Inference

Researchers from Stanford University and NVIDIA Research have released Contrastive Language Models (CLM), a System One model. This model performs fast and generalized decision-making using a contrastive learning objective that links state and action.

Key Performance and Architecture

• CLM-8B achieves performance comparable to existing models (Jev) on computer use, gaming, and tool-calling tasks while delivering up to 9x lower latency.
• By disaggregating state and action, it builds ultra-efficient training and serving infrastructure that allows embeddings to be independently cached and reused. Efficiency is maximized in environments with many candidate actions or frequent action reuse.
• Through lightweight fine-tuning, it set new SOTA records on challenging agent coding benchmarks such as DeepSWE (81.6%) and Terminal Bench 2.1 (87.6%).

Training Data and Recipe

CLM undergoes a three-stage training process:

1. Pre-training: Learning broad semantic representations with 60 million Q&A pairs from Nemotron DQA.
2. Mid-training: Enhancing fine-grained discrimination capabilities with 30 million synthetic hard negatives generated by Gemini 2.5 Flash-Lite.
3. Post-training: Adapting to agent environments with 1 million agent trajectories from the Agent Data Protocol (ADP).

Scaling Laws

The researchers demonstrated that CLM's test contrastive loss decreases predictably following a power law with respect to training compute, model size, and dataset size. In particular, increasing the encoder size has the greatest impact on improving validation performance.