Building a Chatbot Service

To overcome the limitations of existing chatbots, the team used OpenAI ChatGPT and Function Calling to build a structure that identifies user intent and extracts the search parameters needed from internal data.

The overall flow was designed so that after receiving a question, related content is retrieved from a search server and a Vector DB, and the results, combined with the original question and the prompt, are passed back to the LLM to generate the final answer. Both the answer generation results and conversation history were saved and used for later improvement.

To improve accuracy, RAG was expanded in stages.

• In stage 1, refined data such as Saramin help content, FAQs, and product information was stored in VectorDB and searched.
• Instead of simple splitting, documents were treated as a title–content pair, and separate embeddings were generated for each.
• When the similarity weight between title and content was set to 5:5, the accuracy of finding the correct document was highest at 87% across roughly 500 test questions.
• Some help content was expanded to be collected and vectorized via the Notion API.

In stage 2, the existing internal content search API was integrated into the RAG pipeline. This allowed reuse of verified resources without separate ingestion, and Function Calling was used to decide which API to call and which parameters to pass. Response data was organized using an Extraction Schema to filter out private information and pass only the necessary text to the LLM.

Answer quality was heavily influenced by prompt design. Clear principles were built in: answer based on Saramin content, do not guess when information is unavailable, and do not respond to questions outside the service scope. Quality was improved by applying rules such as short, clear instructions, including examples, separating sections, and ordering.

A Multi-subject feature was also introduced to reduce search failures. For a single piece of content, the original title, a similar title added by an operator, and a summary title generated by the LLM analyzing the body text were all managed together, making it easier to find the correct document even when users phrased their questions differently.

LangGraph was used to control complex workflows. A structure based on State, Node, and Edge resolved issues with question branching, iterative processing, error retries, and duplicate code between stream and normal response handling. Invalid inputs, such as short questions or profanity, were handled with fixed answers without calling the LLM, and when multiple pieces of content were needed, cyclic structures and parallel processing were used.

The operational architecture was designed as a multi-tenancy setup that separates the chat system from the bot system. Data sources, prompts, and LLM models can be configured dynamically in Admin, and currently four bots are in operation: Saramin Bot, Labor Consultation Bot, Begins Bot, and Komate Bot. Even when a single bot needs to reference multiple search servers, Function Calling was used to select the appropriate data source.

History of all questions and answers, referenced content, prompts, and processing time was stored and used for monitoring and improvement. By analyzing this together with the customer support team, gaps in content were identified and reinforced, resulting in the improvement of about 20% of content since the service launched.