Evaluate Outcomes When Several Tool Paths Are Valid

A reference tool sequence is useful when the order itself is a requirement. It can be misleading when several authorized paths can produce the same valid outcome. The evaluation needs to state which differences matter.

Quick answer: Define the required final state and the constraints on reaching it. Use exact sequence checks for genuine ordering requirements, and outcome or rubric checks for valid alternatives. Preserve the executed path for diagnosis even when the evaluator accepts more than one route.

Distinguish a requirement from a convenient example

Imagine an illustrative delivery-options assistant. One implementation calls a combined shipping-options API. Another calls availability and shipping-estimate operations separately. Both may produce an appropriate answer using approved sources.

A reference that insists on the combined operation could reject the second path without showing a product failure. Conversely, an outcome-only check could overlook a path that sent customer data to an unapproved destination.

Write the requirements before choosing the matcher:

• The answer concerns the requested product and destination.
• Availability and delivery estimates have appropriate source support.
• Only approved operations receive the required data.
• No purchase or other commitment is made without the required permission.
• Missing estimates are reported rather than invented.

These are illustrative criteria, not claims about an implemented commerce integration.

Choose the strictness deliberately

The Google ADK criteria documentation distinguishes evaluation criteria for tool trajectories and responses. Consult the actual framework version and matching configuration you use. Do not assume a criterion’s name alone establishes whether extra calls, reordered calls or alternative tools are accepted.

An exact path is appropriate when a prerequisite truly matters: authorization must precede a protected operation, for example. A different order among independent read operations may not violate the task.

Review cost and latency without redefining correctness

Accepting two paths as functionally valid does not make them equivalent in cost, latency or maintainability. Record those measures separately and compare them on relevant cases.

If one path repeats a paid request, retain that diagnostic result even when the answer is correct. A product may choose to impose a usage budget, but that should be an explicit acceptance condition rather than an unexplained penalty for differing from a reference.

Include an invalid alternative

A suite that accepts several paths should also contain a path that violates a real boundary. Confirm that the evaluator rejects a wrong source, an unauthorized action or an invented result. Otherwise flexible matching can become accidental permissiveness.

I’m Shashank Agarwal, founder and CEO of Noveum AI and creator of API.market. The HTTP-success example separates execution from task completion; the eval-driven guide explains how to preserve the acceptance conditions during iteration.

Follow my work on LinkedIn. Drafted with AI assistance.