Building a Pesticide Product Image Recognition System Using Amazon Bedrock Vision LLM and Amazon OpenSearch Service

㈜Kyung Nong developed a visual search system to improve convenience for elderly farmers, allowing them to find product information just by taking a photo with their smartphone, without any text input. The company focused on solving the limitations of conventional simple OCR methods, namely the difficulty of recognizing text in blurry images or complex designs.

The system operates through a 3-stage architecture.

• Stage 1 (Vision LLM): Extracts product name, registration number, manufacturer, and more from the label, with TypoCorrector correcting OCR typos.
• Stage 2 (Amazon OpenSearch Service): Searches for candidates in stages, from 'exact match' to 'partial match,' through a hierarchical fallback approach.
• Stage 3 (LLM Reranker): Re-analyzes the original image to determine the final ranking of search results.

In particular, to improve the response speed of the Vision LLM, a compressed JSON format was designed to minimize token usage, and visual context was explicitly provided in the prompt to improve recognition rates for cases with elaborate product name designs. Additionally, correction logic was implemented using the RapidFuzz library to accurately match the actual product name even when typos occur.