100% Achieved with Just 13KB

Oracle Forge's edge-type data agent centered on routing across PostgreSQL and MongoDB, join key correction, and handling unstructured text.

Initially it was a "search and hope it works" approach using RAG, but silent failures were frequent, so the direction shifted to making the Knowledge Base (KB) itself testable.

• Created 21 KB documents, each organizing schema, join rules, and domain terminology
• Designed a unit test for each document
• Fed only 1 document into a new Llama 3.1 8B session, then asked verification questions that could only be answered by extracting specific information from that document
• If more than 30% of key keywords were missed, it was judged a failure

The first result was only about a 60% pass rate, but after rewriting the documents themselves 13 times and changing the structure, it finally achieved 21/21, 100%.

The patterns that had a big impact were as follows.

• Tables > Prose: Converting data-heavy paragraphs into markdown tables improved the 8B model's extraction ability
• Front-loading: Placing the Action Path in the first 30% of the document, with reasons pushed to the back
• Embedded Q&A: Adding verification Q&A at the end of each document to pre-align the model on what it needs to extract
• Keyword redundancy: Arranging specific strings to repeat in headers, body text, and footers

The conclusion is clear. A KB is not a document but part of the runtime, and if an 8B model can't extract the right answer even when given just one document, the system is already fragile before it ever reaches the DB.