Guide to Building RAG Based on Intel Gaudi 2

RAG (Retrieval-Augmented Generation) is a technology that combines the latest knowledge from an external data store with an LLM to enhance performance, accuracy, and security.

This guide presents how to build an enterprise-optimized RAG application based on the OPEA (Open Platform for Enterprise AI) project, using Intel Gaudi 2 accelerators and Intel Xeon CPUs.

Key Components and Roles:

• Embedding Model: Runs on Intel Granite Rapids CPU. It supports AMX-FP16 instructions, providing a 2-3x performance improvement in mixed AI workloads.
• LLM: Runs on the Intel Gaudi 2 accelerator. The Optimum Habana library enables efficient loading and inference of Hugging Face models.
• Vector Database: Uses Redis as the default vector database.
• Framework: Uses LangChain's rag-redis template to connect the prompt, vector DB, and embedding model.

Users can quickly set up a development environment using the provided Dockerfile, and implement the entire pipeline through a hands-on example using Nike's financial documents.