Building Food Metadata Using LLM Jury

DoorDash built an AI-based food metadata platform to extract key attributes such as spice level and cuisine type from millions of unstructured food menu items. Existing manual labeling methods had limitations in terms of scale and cost, so DoorDash introduced an automated system leveraging multimodal signals (text, images, web search).

Key technical innovations include the following:

• LLM Jury system: A method where multiple LLMs verify results through consensus, improving annotation accuracy by about 20% compared to existing human review.
• Context-optimization agents: Iteratively refine prompts to increase model precision by more than 20%, while accelerating prompt development speed by 10x.
• Distributed computing: Supports large-scale LLM inference, reducing data backfill time from over a month to just a few days.
• AI-led annotation: Automatically generates high-quality training data, achieving state-of-the-art LLM-level quality at 10% of the inference cost.

This system enables cost-efficient deployment of generative AI at scale, contributing to improvements in DoorDash's search and personalization experience.