Applying OpenAI's RAG Strategy

OpenAI revealed RAG experiment results for specific customers at Demo Day, demonstrating which techniques were effective. While evaluation metrics vary depending on the application, it is important to understand and apply various retrieval techniques. This is because a 'one-size-fits-all' approach is impossible, and an optimal retrieval strategy tailored to the problem type is required.

This article categorizes the methods mentioned by OpenAI into each stage of the RAG stack. First, it covers distance-based search in vector databases as the baseline. Queries are embedded into high-dimensional space, and similar documents are retrieved based on distance functions such as cosine similarity. LangChain supports over 60 vector store integrations, allowing for flexible configuration of such setups.

Next are Query Transformations techniques to improve retrieval quality. OpenAI proposed two main methods.

• Query Expansion: A method that uses an LLM to generate multiple queries from different perspectives based on user input, and takes the unique union of search results for each query.
• HyDE (Hypothetical Document Embeddings): A technique where a hypothetical document is generated instead of a real one, embedded, and then used for retrieval. The principle is that this hypothetical document may have higher similarity to the original document than the question itself.

Additionally, techniques such as Step-back prompting for reasoning tasks, which help synthesize answers based on higher-level concepts, are also introduced.