SSG.COM's Internal Agent Development Story

The SSG.COM AI team built its own Agent to make internal wiki (Confluence) search easier, helping with document search and finding responsible persons and internal policies. External AI services couldn't be used for security reasons, and the existing keyword search tended to miss results with even slight differences in phrasing, so natural-language-based search was needed.

The core approach was turning user questions into a conversational search experience. When users enter questions like "What's the login-related policy?" or "Where is Kafka used in our infrastructure?", the system is designed to find answers from documents and continue with follow-up questions as needed.

The overall implementation was operated on a workflow basis. The full flow is as follows.

• User enters a question
• Generate a CQL search query
• Call the Confluence REST API
• Summarize document content and evaluate relevance
• Generate final response

For document search, the LLM was made to generate CQL, guided by few-shot examples for syntax, with an Evaluator-Optimizer structure repeatedly verifying query validity and search results. They leveraged CQL's case-insensitive search and text~-based fuzzy matching to cover various notations like "Kafka", "kafka", and "KAFKA".

Retrieved documents weren't used immediately; they were first summarized by the LLM, then re-evaluated for relevance to the question. Only documents matching the question were stored in memory to be used as context for the next conversation, improving both token efficiency and response quality.

The biggest challenge in actual operation was document preprocessing. Confluence documents aren't simple text—they mix tables, macros, user tags, and Jira integrations—making them hard for the LLM to understand as-is. So nested tables were converted to markdown, tags and assignee information shown as hash values were restored to human-readable form, and unnecessary tags were removed.

Performance was also a major issue. To reduce response speed, they reduced output token count based on the structure of TTFT (Time To First Token), TPOT (Time Per Output Token), and total response time, and branched simple questions to a fast model and complex questions to a high-performance model. At the same time, they added progress step indicators and loading animations to reduce perceived wait time.

Beyond document search, they also built specialized agents. The R&R (roles and responsibilities) agent reads text in images via OCR, recursively explores sub-pages, and analyzes tables together with natural language to organize project owners. The internal policy agent processes hierarchical policy documents and PDF/image attachments, and caches context that doesn't change often, including it in the system prompt to speed up responses.

Going forward, they plan to integrate with Microsoft Teams so users can search directly from the chat window, and to evolve the current workflow-centric structure into a more flexible agent-based structure.