Non-deterministic outputs, predictable quality: How to set up your agent for success

We are witnessing a fundamental shift in how users interact with digital products. Static interfaces are giving way to dynamic, conversation-based experiences that mimic human dialogue. Most product teams already build AI agents. The challenge is no longer how to create an agent, but how to make it feel like a coherent, reliable product surface.In this 25-minute talk, I'll share a practical framework to set up AI conversational agents for success, whether they are internal tools or end-user facing products. Drawing from UX writing, linguistics, and content operations, I'll show how teams can design, govern, and assess conversational AI in a way that is scalable, measurable, and aligned with product goals.We'll explore why some AI interactions feel natural and trustworthy while others feel confusing or frustrating, connecting linguistic principles (Grice's maxims, pragmatics, Jakobson's functions) to concrete design decisions. Then, I'll walk through a two-part methodology: first, how to prepare your conversational agent (documentation, identity, principles, and prompt design); second, how to assess output quality through both quantitative and qualitative approaches.Poorly designed conversations increase cognitive load, break trust, and create hidden costs through rework, escalations, and user frustration. Well-designed conversational AI, on the other hand, improves efficiency, adoption, and long-term product value. This talk gives product designers and PMs the tools to achieve the latter.