Josh Clemm on how Dash uses knowledge graph, MCP, and DSPy

Dropbox Dash is built on a context engine that brings together multiple SaaS and internal content in one place, enabling search, answers, and agentic queries.

The core flow first connects external apps via connectors, then normalizes documents, images, PDFs, audio, and video, extracting titles, metadata, links, and embeddings for indexing. Documents can be handled with text extraction alone, but images and video require multimodal understanding, and video in particular needs scene-by-scene understanding to be built and stored.

This information is also tied together into a knowledge graph. It links meetings, documents, people, transcripts, and previous notes to create cross-app context, and normalizes person IDs into canonical IDs to improve search and relevance.

The storage layer is hybrid retrieval, combining a BM25-based lexical index with a vector store. Results pass through multiple stages of ranking, apply per-user ACL, and are personalized.

For retrieval strategy, they chose index-based retrieval over federated retrieval. The federated approach allows a fast start but depends on API or MCP server quality, makes company-wide connector access difficult, requires merging and re-ranking results on the fly, and consumes more tokens. In contrast, the index-based approach leverages company-wide connectors and enables offline experimentation and advanced enrichment, but requires handling custom connectors, freshness, and storage cost issues.

MCP as-is proved too burdensome at Dash's scale. Tool definitions take up a lot of the context window, and when retrieval results are added on top, the 100,000 tokens ceiling is quickly filled, and even simple queries can slow down to as much as 45 seconds.

To reduce this, Dash was designed as follows.

• Simplified multiple retrieval tools into a single super tool wrapping the index
• Used the knowledge graph to compress and deliver only the information needed
• Stored large tool result context locally instead of putting it into the LLM context
• Split complex queries into sub-agents, with a classifier selecting an agent with a narrower set of tools

LLM as a judge is also a key pillar. In a chat environment lacking direct human feedback like search click signals, they built a judge that rates relevance on a scale of 1 to 5, improving quality by reducing disagreement with human labels. With the initial prompt, 8% disagreed, but this was lowered further through prompt improvements and a model upgrade to OpenAI o3, and to supplement internal context such as abbreviations, they even used RAG as a judge, where the judge fetches context directly.

Finally, he explained—including the prompt optimizer DSPy—how Dash systematizes retrieval, routing, and evaluation within large-scale work context.