AI BriefingCan Large Language Models Improve Few-shot Retrieval Performance for Complex Question Answering
Key point
This introduces PromptRank, a technique that uses LLMs to find efficient document paths for complex Multi-hop Question Answering (MQA).
Details
To answer complex questions, Multi-hop Question Answering (MQA) capability—which requires going through multiple stages of reasoning to find scattered information—is essential. Generally, an MQA system consists of a Retriever that finds relevant documents and a Reader that generates the answer.
However, building a high-quality Question-Path dataset for MQA is very costly, and the data scarcity problem is especially severe for low-resource languages or specific domains. Existing data-efficient retrieval methods have mainly been limited to single-hop QA.
To address this, the proposed PromptRank operates in two stages.
- Stage 1 (Unsupervised Retrieval): A simple unsupervised method such as TF-IDF is used to select an initial set of document paths that are likely to be relevant to the question.
- Stage 2 (Reranking): An LLM is used to reassess (rerank) the relevance of the selected paths to the question.
The core of PromptRank lies in scoring by calculating the Conditional Likelihood—the probability that a given question would be generated given a specific document path. This makes it possible to achieve sophisticated retrieval performance in complex multi-hop reasoning settings while minimizing data construction costs.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.