Improving Amazon Product Listing Quality with LLMs

Amazon has deployed LLMs for quality control to organize its catalog of hundreds of millions of products and the massive volume of listings added or modified every day, faster and more accurately. Previously, separate ML models built for each category handled attribute correction, error detection, translation, and external data integration, but these had limits when it came to complex or subtle attributes.

To address this, the catalog was reorganized around product types and attribute values, compiling statistics on how often each value appears and where. For example, even if the attributes for wireless headphones are scattered as “Bluetooth”, “BT”, “BT 5.1”, and “Bluetooth version 5.1”, frequency and exposure statistics can be used to establish “Bluetooth” as the most reliable canonical value.

However, statistics alone don't solve every case. For attributes where granularity matters, such as “stainless steel” versus “440 stainless steel” for surgical instruments, the more detailed value needs to be preserved, which requires prompt tuning that embeds rules such as "the returned value must match the granularity of the candidate list" into the LLM.

In this process, the LLM is trained to present not just its response but also its reasoning process, which helps both with performance improvements and further tuning. It also handles issues such as notation consistency, like “men's shirt” versus “men shirt,” and preserving more meaningful expressions, like “4K UHD HDR” for TVs.

Once iterative prompt tuning is complete, the LLM is applied across the entire catalog to perform three tasks:

• Determining accuracy by recognizing standard attribute values
• Expanding standard values by collecting synonyms/alternative expressions
• Blocking quality degradation by detecting errors or illogical input

As a result, the latest seller values are reflected faster, within days, and thousands of hours of human review time are saved. It also allows more languages to be monitored and updated, expanding quality control coverage to deeper areas of the catalog that were previously too costly to manage with the old approach.