Can agents replace the search stack

On Amazon ESCI, BM25 scored 0.289 and e5 embeddings scored 0.314. Adding GPT-5-mini and letting it use search tools raised this to 0.359 on e5, 0.385 on BM25, and up to an average of 0.4101 (median 0.3743) when using both tools together. GPT-5 using both tools recorded NDCG 0.453.

The agent's behavior was surprisingly simple. In most cases it called each search tool only once, and if the results were off, it would slightly tweak the query and search again. For example, after searching 'PVC coupler' and getting off-target results, it would re-search with 'PVC pipe coupler'. Because keyword search has a clear right or wrong, the agent could easily choose its next action.

Forcing stronger exploration raises performance a bit further.

• 4 calls with no duplicates: mean 0.4290 / median 0.3948
• prohibiting similarity above 0.9 with the previous query: mean 0.4308 / median 0.4258

However, this approach doesn't work for every search problem. On MS MARCO passages, using more tool calls with the agent showed no improvement, and an already-trained embedding model performed better. The author divides this into 'finding things' and 'Deep Research'. In the former, an agent can assist exploration, but in the latter, the traditional search stack is still needed since it must fill gaps in the model's knowledge.

General-purpose LLMs tend to treat search like web search, querying once and stopping, but dedicated agentic search models like SID-1 make search quality itself the object of reasoning. Such sub-agents let the main LLM focus on the user's task, pointing toward a more flexible way of handling the search/RAG stack.