Gemini API File Search Now Supports Multimodal RAG

Gemini API File Search has now been extended to multimodal RAG that handles images and text together. Based on Gemini Embedding 2, it understands raw image data, allowing searches using only visual tone and style descriptions instead of filenames or keywords.

By attaching custom metadata such as department: Legal, status: Final, you can narrow queries down to only the data you need. This filters out unnecessary documents to reduce search noise, boosting both speed and accuracy at the same time.

Page citations have also been added. Each indexed piece of information is tagged with a page number, allowing immediate verification of the source of PDF answers, making fact-checking and providing evidence easier.

Google also introduced real-world use cases.

• K-Dense Web: Uses it for mixed-modal search spanning Western blot, microscopy, and agent-generated plots, stating that it confirmed high search accuracy and low latency without any pre-processing.
• Klipy: Stated that understanding of text within GIFs of varying quality has improved, reducing guesswork and mitigating hallucinations.
• Code Fundi: Explained that indexing architecture diagrams, ERDs, and sequence diagrams reclaims over 50% of the agent's context window.

Developers can immediately apply File Search through the new developer guide and Gemini API documentation.