Running conversational agents in production is expensive, but analyzing their unstructured logs at scale using LLMs introduces a painful "double-token tax" - burning budget during the conversation, and again during analysis. This session introduces a database-native design pattern to escape this tax and scale conversational LLM analytics to millions of interactions at a fraction of the cost. We present a two-part architecture: - Insights Surfacing: Automating category distributions and user intent mapping utilizing unsupervised clustering and drift-aware LLM labeling. - Interactive Semantic Discovery: An interactive natural-language agentic interface enabling ad-hoc semantic queries directly on the data warehouse, analyzing millions of conversations at scale with high quality by combining LLMs and light ML models. By bridging these modes, teams can extract latent user needs hidden in massive log volumes and turn raw dialogues into actionable product improvements.
Senior Data Scientist, Google