KDDI Optimizes Buffmee RAG App Performance: 38% Latency Reduction and 25% Groundedness Improvement

KDDI adopted Google Cloud's Agent Development Kit (ADK) and Gemini Enterprise Agent Platform Evaluation Service to optimize the performance of its consumer-facing RAG app Buffmee. Initially, high latency and hallucination issues arising from the Grounding process based on vast proprietary content were the main challenges.

Performance Improvements and Automation

As a result of optimization using an automated evaluation framework and ADK, total application response latency decreased by 38%, and Time To First Token (TTFT) improved by approximately 18%. Additionally, Groundedness scores improved by 25%, enabling the guarantee of accurate and reliable outputs.

Key Optimization Strategies

KDDI applied the following four principles to handle large-scale content libraries.

• Introduction of Binary evaluation: Implemented a pass/fail system instead of a 1-5 point scale to minimize evaluation variance and noise.
• Strategic content sampling: Classified content formats and compositions into a 2D grid to select representative samples, reducing evaluation workload by 75% while maintaining coverage.
• Thresholds based on Product judgment: Product owners reviewed automatic scores and sample answers together to set criteria for 'shippable quality'.
• Modularization with ADK Skills: Separated complex system prompts exceeding 800 lines into functional ADK Skills, applying dynamic loading to shorten response times.

This approach contributed to analyzing production logs via BigQuery Agent Analytics and an ADK log analysis agent, and resolving prompt bloat and deep-stack bottlenecks in real time.