Blocking Fraudulent E-commerce Returns with AI: Building an Industry-Specific Guardrail with Amazon Nova Fine-tuning

The fashion e-commerce industry suffers approximately $100 billion in annual losses due to fraudulent returns such as wardrobing and false defect claims. Existing general-purpose LLMs have limitations in properly blocking such fraudulent intent due to their tendency to respond kindly to customers.

To address this, a domain-specific Custom Guardrail was built using the Amazon Nova 2 Lite model on Amazon Bedrock. This model analyzes the customer's inquiry before it reaches the AI agent, pre-classifying it as a legitimate request (Safe) or a fraudulent request (Unsafe).

As a result of Fine-tuning using a dataset of 837 entries, fraudulent intent detection accuracy improved from 73.0% to 94.6%, a 21.6%p increase. Notably, by using a cost-efficient small model instead of an expensive large model, both excellent performance and cost efficiency were achieved simultaneously.