Agent demos hide the denominator
An agent demo is usually a carefully selected journey where the tools respond, the model understands the task, and the final answer looks coherent. A product lives in the denominator: all the ambiguous requests, stale pages, malformed tool results, partial permissions, and reasonable user expectations that were not in the demo. Reliability begins when those cases are collected instead of explained away.
I define an agent as a policy that chooses actions under uncertainty. That means evaluation cannot stop at the final paragraph. We need to observe whether it selected the right tool, passed valid arguments, used evidence returned by the tool, stopped at the correct time, and communicated uncertainty. A polished answer can conceal a broken trajectory.
Build the harness around real work
A useful evaluation set starts with actual tasks and failures. Each case should include the user goal, available context, allowed tools, expected invariants, and a rubric for the final outcome. Some checks are deterministic: a raw IP must never be stored, a financial claim requires a citation, or an email must not be sent without confirmation. Other checks need a model or human judge, but their rubric should still be explicit.
Keep traces. Tool calls, retrieved evidence, intermediate state, latency, and cost make failures explainable. When the final answer is wrong, the trace tells us whether the model planned badly, retrieval failed, a tool contract was unclear, or the system had insufficient information. This is much more actionable than tuning a prompt against screenshots.
Reliability is a product loop
The harness should run during development and against production-like traffic samples. Every fixed failure becomes a regression case. Metrics should be segmented by task type because a single average hides weak capabilities. I care about task completion, groundedness, tool-call validity, safe abstention, latency, and cost, but the right mix depends on the decision the agent supports.
Evaluation does not eliminate judgment. It moves judgment into a reusable form. The team still decides what good behavior means, which errors are expensive, and where the system should ask for help. The payoff is compounding: each incident can improve the product instead of becoming a one-off patch. That is the difference between an agent that occasionally impresses and one that can carry real work.