SK Planet Develops LLM-based DMP 'Seg Lens'... Automatically Segments Marketer Intent

SK Planet's Data Service Development Team has developed Seg Lens, an LLM-based DMP (Data Management Platform) system. Existing DMPs required marketers to understand complex data structures and manually input logical conditions such as age, behavior, and interests, taking an average of 2-3 hours and creating high potential for human error. Seg Lens introduces an LLM to interpret natural language sentences and automatically convert them into structured data conditions, resolving this learning curve and inefficiency.

Core Features and Architecture

Seg Lens consists of a DIA (DMP Intelligence API) module and a Vector Database (Vector DB). DIA is composed of an OpenAI GPT-4-based LLM service, a Vector Service that performs 3,072-dimensional vector similarity search, and a Keyword Service responsible for exact matching. Users can input natural language queries or images such as "Recommend laptops to women in their 30s-40s living in Seoul," and the system parses this into a logical structure such as 'Seoul AND (30s OR 40s) AND Female AND Laptop'.

Vector Search and Semantic Expansion Technology

To overcome the limitations of simple keyword search, the Semantic Expansion technique was applied. User input is expanded into synonymous, broader, narrower, and related segments before being embedded to improve search accuracy. For example, when 'car' is entered, contextually similar segments such as 'SUV', 'Hyundai/Kia compact cars', and 'car insurance' are suggested together. Search utilizes Elasticsearch's KNN (k-Nearest Neighbors) query and Cosine Similarity to return semantically close Top-K results.

Cost Optimization and Access Control

During the embedding process for over 50,000 items, the Batch API was used to reduce costs by 50% (approximately $9,000 annually) compared to real-time API and secure processing stability. Additionally, for site-specific permission management, site_resource_index information is reflected in the filter area of search queries, clearly distinguishing data access scope between the admin console and partner sites. Through this, marketers can now focus on establishing customer understanding strategies without needing data engineering knowledge.