How to Architect a Search Engine

Search isn't just a simple inverted index lookup—it quickly turns into a complex 20+ node graph once you layer on per-language localization, news-style freshness judgments, simultaneous queries to a knowledge graph and product index, and customer-specific exception paths.

On top of this, thousands of AI agents each have different requirements, and with the reality that agents even write the code, it becomes difficult to keep global constraints and requirements consistent. To solve this problem, Exa built Canon, a search pipeline orchestrator.

Canon defines search paths as a DAG, letting the executor automatically handle parallelism. As in the example, by separating retrieve and fetch_content into nodes, the executor can look at dependencies to parallelize tasks that can run simultaneously, and track the entire request as a single graph.

The core benefits provided by the DAG are as follows.

• Automatic Parallelism: Automatically runs nodes with no dependencies in parallel to reduce latency.
• Durable execution: Even if a node fails, retries can resume from that point.
• Introspectability: Since the graph itself is data, visualization, validation, and analysis are easy.
• Separation of definition / execution: The same graph can run synchronously in tests, distributed in production, and as a dry-run in previews.

However, the DAG isn't a cure-all. It doesn't fit structures centered on state and events rather than tasks to schedule, like an event loop; consensus protocols where exact ordering matters; or feedback loops that iterate until convergence.

Canon's goal is to make the entire search path visible before a query, and traceable after execution. Previously, relying on sequential function calls, if/else branches, and manual error checks made it extremely difficult to swap out a reranker, verify fallbacks, or diagnose why a URL was missing. Canon compiles the search path into a serializable graph, allowing precise tracking of exactly which node dropped a URL and why.

The runtime operates in a pull-based manner, executing the upstream subtree only when a downstream consumer requests a value. This structure naturally produces laziness and cancellation propagation, and for diamond dependencies, memoization prevents redundant computation.

Ultimately, Canon records timing, inputs, outputs, and decisions at every invocation boundary of the search pipeline, and automatically attaches the failed node and upstream context to errors. If nodes were to directly make external network calls themselves, this kind of control would become impossible—so the key is putting all logic under a common interface.