LLMs in Production Gone Wrong: A Story of Prompts and Observability

LLM applications rarely fail loudly — they fail strangely. Costs are drifting, outputs are degrading, prompts are evolving reactively rather than by design, and it's becoming difficult to understand what's really going on in production. This presentation will cover essential practices for maintaining the reliability of LLM systems: end-to-end monitoring, cost and latency visibility, prompt versioning, offline and online evaluations, A/B testing, and user data protection. We'll compare the top tooling options available today and show how each one helps you measure, debug, and improve an AI product throughout its lifecycle - before small problems turn into costly surprises.