How Are Toss Securities' Recommendation and Search Evolving?

The Toss Securities AI Data Platform team aims to build a data ecosystem that all AI services can utilize in common. To this end, they are designing a system based on source data pipelines, real-time and batch feature stores, RAG for natural language search, and a financial information knowledge graph.

The recommendation system has shifted from a past batch-centric clustering approach to a real-time loop structure that reflects users' real-time behavioral events and item changes. Version management and consistency maintenance of the vector store when changing embedding models have emerged as key challenges. To address operational issues such as JVM memory pressure during high-dimensional vector lookups, they are precisely monitoring mget throughput and GC trends.

In the search domain, they approached RAG from a 'search productization' perspective to resolve the fragmentation of data access methods in LLM-based services. The news RAG pipeline provides refined context through question understanding, hybrid search (text + vector), and reranking stages. To overcome the limitations of finance-specific embeddings, they introduced pre-management, metadata-based signal enhancement, and an LLM-fine-tuned reranker to improve search accuracy and reliability.