Building an agent demo takes an afternoon. Knowing whether it still works after next week’s prompt change is the hard part.

We built and shipped agentic Python systems to production, among them a multi-agent assistant for market and whitespace analysis. As features piled up and real users arrived, we learned that our usual tests say little about a system that is non-deterministic by design. A small change to a prompt or a tool description can shift behaviour in ways no unit test will catch.

In this talk we walk through the layers of evaluation we use, from cheap to expensive: deterministic checks on a single step, assertions on the trajectory and its tool calls, simulated users that hold a full conversation with the agent, and AI as a judge for the open-ended tasks where there is no single right answer.

We also cover the parts nobody warns you about: calibrating your evaluators until you and your stakeholders trust the scores, splitting train and test sets so you don’t overfit your rubrics, and keeping drift and cost under control.

The payoff is concrete. When two versions look equally good, evaluation tells you whether you really need the bigger model or the more complex architecture, or whether a prompt change already got you there.