Cohere Introduces RCP-nDCG@10: A New Retrieval Metric Validated Against Human Judgment

Traditional nDCG metrics rely on pre-existing relevance labels (qrels) which often miss relevant documents retrieved by newer models, leading to inaccurate performance assessments. RCP-nDCG@10 addresses this by using a calibrated AI judge to grade every retrieved document against explicit relevance criteria, ensuring relevant results receive credit even if they were not in the original benchmark pool. A 46-person human annotation study validated that RCP-nDCG@10 aligns more closely with human judgment than traditional nDCG, with reviewers siding with RCP-nDCG 70% of the time when the metrics disagreed. Cohere has optimized its upcoming fifth-generation Embed and Rerank models using this new standard to better reflect real-world enterprise search quality.