Implementing LightRAG and Navigator to Enhance Internal Document Search Accuracy

To address the issue where vector search alone failed to connect relationships between documents, leading to reduced accuracy in complex internal document queries (such as those involving additional common gateway filters), we introduced LightRAG and Navigator. LightRAG extracts entities and relations from documents to build a graph, improving search correctness from 76% to 84% through prompt optimization and Few-shot example replacement. Additionally, we developed a Neo4j-based Navigator to enhance location information for documents that are difficult to distinguish by title alone. Navigator stores the tree structures of Confluence and Jira directly into the graph DB and includes hierarchical paths in the embedding input during chunking to accurately capture document context. To prepare for company-wide scaling, we performed optimizations such as adding Neo4j property indexes, team-level batch processing, and skipping hash-based re-embedding, reducing loading time by 81% and stabilizing memory usage.