Evaluating agents when there is no right answer

A traditional test asserts that a function returned the expected value. An agent's job is often to decide what the expected value should be, which makes that assertion circular.

Teams hit this wall early and respond in one of two unhelpful ways. Either they give up on testing and ship on vibes, or they write brittle tests that pin exact outputs and break every time the model changes.

There is a third option, and it starts by measuring the trajectory instead of the answer.

Measure the path, not the destination

For most agent tasks there are several acceptable answers and a much smaller set of acceptable ways to get there.

An agent that diagnoses an incident correctly by reading the right logs in a sensible order is doing the job. An agent that reaches the same conclusion after twenty-three redundant tool calls and one lucky guess is not, even though the final output is identical.

So instrument the loop. Track tool call count per task, how often the agent repeats a call with identical arguments, how often it recovers from a tool error versus giving up, and how many turns it takes to reach its first correct action. These are objective numbers on a subjective task, and they move in obvious directions when something regresses.

Build a case file, not a test suite

The artefact that has served us best is not a test suite. It is a directory of real scenarios, each one a frozen snapshot of system state plus the transcript of what the agent did, annotated by whoever reviewed it.

Every time the agent does something surprising in production, good or bad, that scenario goes in the file. Over a few months you accumulate a corpus of genuinely hard cases specific to this system, which is far more valuable than any generic benchmark.

Before a model change or a prompt change, replay the file. You are not asserting exact outputs. You are reading the diffs and asking whether the new behaviour is better or worse on the cases you already know are hard.

Three checks you can automate

Some things are objectively assertable even when the answer is not.

Did it stay in bounds? Assert that no tool outside the allowed set was called, and that no destructive tool ran without an approval event. This is a hard pass or fail and it should gate every deploy.

Did it stop? Assert a maximum turn count and a maximum cost per task. An agent that exceeds either is failing, regardless of whether it eventually got the right answer.

Did it verify? Assert that when the agent claims a task is complete, it made at least one read call after its final write. An agent that reports success without checking is guessing, and this check catches that mechanically.

Accept a human in the loop

The uncomfortable conclusion is that some portion of agent evaluation stays manual, permanently. Someone reads transcripts.

This is not a failure of engineering discipline. It is the same reason code review exists for a codebase with full test coverage. The tests catch the things you knew to check for, and review catches the things you did not.

Budget for it. An hour a week of someone senior reading agent transcripts will surface problems that no amount of automated scoring would have found, and it is the cheapest quality mechanism available.