How GS SHOP Built a Video-Based AI Product Recommendation Platform

GS SHOP's Search & Recommendation Part set out to turn Shortpicks—the roughly 30-second clips produced from TV home shopping broadcasts—from simple supplementary content into a recommendation signal. Previously, a PD had to rewatch an entire 1-hour broadcast to remove scenes unsuitable for rebroadcast and pick out highlights, and even Shortpick recommendations ultimately ended up as a structure where video was simply attached to product recommendation results.

Video search began by integrating TwelveLabs Marengo 2.7 on Amazon Bedrock. Marengo embeds video chunks so that natural language queries can be compared directly, and it could find even Korean sentences like a shopping host placing a piece of kimchi on boiled pork and picking it up deliciously with chopsticks with high accuracy. As a result, verification work that used to take a PD several hours could be finished in seconds. This was later extended to a stage where Pegasus 1.2 reads the context of an entire video.

The existing Shortpick recommendation worked by having CF/CB pick products based on recent click logs, then only surfacing those that had a Shortpick available. Since the actual content of the video itself was barely used, GS SHOP layered Bedrock model chaining on top of Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Amazon ElastiCache (Redis) to connect video understanding results to recommendations.

The key was extracting Appeal Points. Pegasus describes the entire video, Claude Sonnet 4 extracts keywords for recommendation, and Cohere Embed Multilingual vectorizes these so that Amazon OpenSearch Service can perform similarity search using Cosine Similarity.

• AS-IS: Embedding the full video description as-is caused the system to latch onto brand names and product names, resulting in a problem where only similar sneakers were recommended.
• TO-BE: By keeping only the core context—such as breathability, fit, and comfortable mobility—the recommendation range expanded to include leggings, windbreakers, and everyday wear.

In the end, GS SHOP didn't stop at using video understanding merely as a search tool—it redesigned Shortpick recommendations into video-based product recommendations and connected them to an actual live service.