GloZ's Case Study on Building a Natural Language Resume Search System with Amazon OpenSearch Service - Part 1: Data Pipeline and Indexing

GloZ redesigned its search architecture with Amazon OpenSearch Service to overcome the limitations of PostgreSQL-based metadata filtering and its in-house vector library, targeting approximately 100,000 translator resumes registered on E'nuff. The existing approach required running BM25 and vector search separately, and also had weak index updates, monitoring, scalability, and Korean morphological processing.

The project's goals were to achieve nDCG@10 of 0.90 or higher for natural language search, combine BM25 and vector search into a single query, and improve Korean and multilingual resume search quality using Nori and multilingual embeddings.

On Amazon Bedrock, the pipeline was built primarily around Cohere Embed Multilingual v4 and Claude Haiku 4.5. In model comparisons, Cohere was adopted after showing about a 40% advantage in nDCG@10 over Titan Text Embedding v2, and Claude Haiku 4.5—paired with source-text verification to control Hallucination—enabled stable structuring without needing a higher-tier model.

• PDFs were extracted directly with PyMuPDF, DOCX files were converted via LibreOffice, and image-based PDFs were processed with OCR.
• Extracted text was structured into language pairs, domains, tools, experience level, and more, then unified through three-stage normalization: a standard vocabulary dictionary, alias mapping, and fuzzy matching.
• Documents where text extraction failed or quality was low were supplemented with an LLM fallback.
• Hallucination was filtered out through post-processing that matched against source-text keywords, preventing tool or domain values not present in the original text from entering the index.

The embedding input method was also decided through experimentation. Embedding the full raw text scored lowest at nDCG@10 0.781, structured JSON scored 0.802, and summary-only scored 0.831. The final adopted method—summary + flattened metadata keywords—scored highest at 0.852, and the same trend was reconfirmed across about 150 evaluation queries in the production environment.

The resumes-v01 index defines embedding as a 1024-dimensional knn_vector with HNSW + L2 applied, using settings of ef_search 512, ef_construction 512, and m 16 to target fast response times. summary was indexed separately using a multilingual analyzer and a Nori-based Korean sub-field, while metadata was kept as keyword type for exact matching and filtering.

By connecting the default_pipeline with an Ingest Pipeline (resume-embedding-pipeline), the team automated the process so that a Script Processor generates search_text and an ML Connector calls Bedrock embeddings directly from OpenSearch. Part 1 focuses on data cleansing and indexing design, while Part 2 covers hybrid search optimization that combines BM25 and vector search into a single query.