AI BriefingAirbnb introduces CASTLE framework to solve cold-start problem for natural language search
Key point
The framework achieves a KL divergence of 1.01 against real user queries, a 9.2x improvement over previous baselines.
Details
Airbnb's CASTLE (Contrastive And Seed-guided Training for natural Language sEarch) framework addresses the cold-start problem for natural language search by jointly generating synthetic queries and relevance labels from de-identified booking data. Traditional methods like InPars and Promptagator struggle with distributional mismatch, scoring between 9.33 and 26.2 on KL divergence compared to real user behavior. CASTLE closes this gap by leveraging actual booking signals to ground synthetic data generation.
Contrastive Grounding Mechanism
The core innovation uses contrastive listing pairs derived from booking sessions. When a guest books a listing after viewing alternatives, the booked listing is treated as positive and a viewed-but-not-booked alternative as negative. An LLM generates a query satisfied by the positive listing but not the negative, ensuring relevance labels are correct by construction without post-hoc judgment. This approach avoids the circularity of LLM-generated labels and aligns synthetic queries with real user intent.
Performance and Distribution Alignment
CASTLE achieves a KL divergence of 1.01 against real user queries, marking a 9.2x improvement over the best baseline (Promptagator at 9.33). The generated queries also match real-world linguistic patterns, with a word-count standard deviation of σ = 2.48, closely mirroring real users (σ = 2.49) and outperforming baselines (σ = 0.99). Human evaluation confirmed high label quality, with annotators agreeing with CASTLE’s relevance assignments at 91–93%.
Production Readiness and Future Work
The framework includes a Virtual Judge for listings without booking history and a CASTLE-R variant for environments with accumulating real traffic. By oversampling long-tail intents, CASTLE provides value even after real data becomes available. The research was accepted as a full paper at CIKM 2026, and Airbnb is exploring extensions to multi-turn conversational search.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.