While AI enthusiasts were publishing blog posts, demos, and browser extensions showing how easy it is to summarize hotel reviews with LLMs, we were busy tackling the much harder challenge of building a feature that actually helps travelers. Building an AI-powered review summarizer is no longer the hard part. Building one that works consistently across nearly 4 million properties and hundreds of millions of reviews, while measurably helping travelers make better booking decisions, is. Join me to learn about our two-year journey of experimenting with prompts, models, data sources, evaluation methods, and UX at Booking.com. We’ll explore why offline AI metrics are only the beginning, why A/B tests often tell a very different story, and the surprising lessons we learned when real traveler behavior challenged our assumptions.
Senior Product Manager @ Booking.com