A correct final answer can hide a wasteful or unsafe path: an unnecessary tool call, a retried payment, data fetched that shouldn't have been. Agent evaluation has to inspect the steps as well as the result.
What to score
- Outcome — did the task finish correctly?
- Tool use — right tools, right order, valid arguments, no redundant calls.
- Safety — did it stay within permissions and refuse what it should?
- Efficiency — number of steps, time and cost per task.
- Recovery — how it handled a failed call or ambiguous input.
How to build the test set
Use real past cases, include difficult and adversarial ones, and write down the expected path, not only the expected answer. Re-run it whenever you change the model, prompt or tools, and keep the history so you can see trends.
A new model that scores better on answers but takes more risky steps is not an upgrade. Track both.