Define an Agent Abstention That Counts as Success
Shashank Agarwal
An agent that cannot find the required evidence should not be rewarded for confidently completing an unsupported answer. It also should not be rewarded for refusing every difficult task. Define the boundary between an acceptable abstention and an avoidable failure.
Quick answer: Make abstention conditional on the task’s evidence and permission state. Specify what information is missing, what the agent should communicate and which next actions are allowed. Evaluate that behavior alongside cases where sufficient evidence exists and completion is expected.
Define the unavailable-evidence branch
Consider an illustrative warranty assistant. The source contains a policy for one product, but the user asks about another product whose policy is unavailable. A plausible duration borrowed from the first policy is not a supported answer.
A successful response might identify the missing product-specific policy and ask for an approved source or route the request to the appropriate support path. The exact fallback is a product requirement; the evaluator should not invent it after seeing the response.
| Situation | Expected behavior to define |
|---|---|
| Correct policy is available | Answer the question with appropriate support |
| Only an unrelated policy is available | Explain the evidence gap rather than transfer its terms |
| Required identifier is missing | Ask for the information needed to resolve the request |
| Tool request fails | Report the limitation and use only an allowed fallback |
| Action permission is missing | Avoid the action and explain the required next step |
These are proposed test conditions, not a measured benchmark.
Evaluate the explanation as well as the stop
“I can’t help” is not always sufficient. A useful abstention should make clear what could not be established without inventing an internal error or claiming a restriction that does not exist.
Check whether the agent requests only relevant missing information. Do not reward a fallback that collects unnecessary personal data or sends the user into an unrelated process.
Keep the completion cases in the same suite
Include a case where the correct warranty policy is present and the question is in scope. An agent that abstains there may be failing to use available evidence.
Report successful completions, appropriate abstentions, inappropriate abstentions and unsupported answers separately. A single answer-rate metric can reward risky guessing; a single refusal-rate metric can hide an unusable assistant.
Microsoft’s evaluation checklist includes expected handling for questions beyond available policy knowledge. The product-specific conditions above make that branch reviewable rather than treating all non-answers alike.
I’m Shashank Agarwal, founder and CEO of Noveum AI and creator of API.market. The hallucination investigation guide explains source support; the eval-driven guide describes retaining these cases through application changes.
Follow my work on LinkedIn. Drafted with AI assistance.
Get Early Access to Noveum.ai Platform
Join the select group of AI teams optimizing their models with our data-driven platform. We're onboarding users in limited batches to ensure a premium experience.