Treat the 60% success rate as a hypothetical scenario constraint. Before improving it, define what “success” means: completing the intended user outcome, not merely producing a plausible final message. An agent that says a ticket was created when the tool call failed should count as a failure even if the conversation sounds excellent.
Our AI agent succeeds only 60% of the time. How would you improve it?
Why the common answer is incomplete
Most candidates answer: “Use a better model and improve the prompt.”
That may improve planning, but an agent is a distributed workflow with model decisions, tool schemas, external systems, mutable state, retries, and security policies. The model may choose the correct action while the tool times out, or the tool may succeed while the agent loses state and repeats it. A single success percentage hides where those failures occur.
Improve reliability by converting the agent from an opaque loop into an observable state machine.
A production approach
1. Define task-level success and terminal states
For each workflow, specify the expected side effects, output, and allowed terminal states: completed, safely declined, needs clarification, waiting for approval, retryable failure, and permanent failure. Verify success using system state when possible. A calendar event ID from the calendar service is stronger evidence than the agent’s sentence claiming it scheduled a meeting.
2. Classify failures by step
Capture traces containing the plan, model decision, tool name, validated arguments, tool response, state transition, retry, and final verification. Label failures as planning, tool selection, argument construction, authorization, external dependency, state management, output interpretation, or recovery. Prioritize the largest failure class instead of tuning everything simultaneously.
3. Replace open-ended loops with explicit workflow state
Represent durable workflow state outside the model: current step, completed actions, pending approval, retry count, and idempotency keys. The model proposes an action; application code validates and applies a state transition. Enforce maximum steps, elapsed-time limits, and token budgets. This makes recovery possible after a process restart and prevents the model from silently forgetting completed work.
4. Design narrow, typed tools
Give tools clear names, minimal parameters, schemas, and machine-readable error classes. Separate reads from writes. Prefer get_customer(customer_id) over a generic endpoint that accepts arbitrary SQL or URLs. Validate arguments server-side, apply authorization using the authenticated user rather than model-provided identity, and return only fields needed for the next decision.
5. Make side effects idempotent
Every write tool should accept or derive an idempotency key tied to the workflow and intended action. Record the result so a timeout or model retry can retrieve the previous outcome rather than repeat the mutation. For operations that cannot be made idempotent, require explicit confirmation and expose the uncertainty when the response is lost after submission.
6. Add bounded recovery policies
Map errors to recovery behavior. Retry rate limits and temporary unavailability with backoff and jitter. Do not retry invalid arguments, missing permission, or policy rejection without changing the request. After a bounded number of attempts, transition to a safe terminal state with the trace preserved. A model-generated “try again” loop is not a recovery policy.
7. Use confidence gates and approvals
Require clarification when required fields are ambiguous. Require human approval before high-impact writes such as payments, account deletion, or external communication. Evaluate tool selection and arguments before execution using deterministic policies where possible. Keep approval state durable and bind it to the exact proposed action so later context changes cannot reuse old consent.
8. Build step-level and end-to-end evaluations
Test individual tool selection, argument accuracy, state transitions, recovery, and final task outcome. Include dependency timeouts, malformed tool results, duplicated messages, interrupted workflows, and adversarial content returned by tools. Run the suite against prompt and model changes, then canary production traffic. A higher model benchmark score does not guarantee a more reliable workflow.
Engineering trade-offs
Explicit workflows are more reliable and testable but less flexible than a fully autonomous loop. Human approval reduces risk while increasing completion time and abandonment. More detailed traces improve diagnosis but raise storage and privacy costs. Redundant verification catches silent failures but adds dependency calls and latency. A stronger model may reduce planning errors while increasing cost and still leaving tool failures unchanged.
Use more autonomy for reversible, low-impact tasks and more deterministic control for irreversible or regulated actions.
Common mistakes
- Measuring success from the agent’s final text instead of external state.
- Giving a generic tool broad network or database access.
- Retrying every failure with the same arguments.
- Storing workflow state only in the conversation transcript.
- Letting the model supply the user or tenant identity used for authorization.
- Reusing an approval after the proposed action changes.
- Testing only successful tools and stable network conditions.
Failure handling and observability
Track task success, safe-decline rate, clarification rate, step count, tool selection accuracy, argument-validation failures, dependency errors, retry amplification, approval abandonment, duplicate-action prevention, and cost per successful task. Segment by workflow and tool; a global 60% rate cannot reveal that one unreliable integration causes most failures.
When a worker crashes, resume from durable state and re-check the last action using its idempotency key. When a tool response is malformed, quarantine the result rather than passing it back as trusted instructions. When the budget is exhausted, stop with a clear recoverable state. Logs should record identifiers and error classes, not secret tool payloads or entire private conversations by default.
Interview insight
The interviewer is not testing whether you have used an agent framework. They are testing whether you understand workflow reliability, state, idempotency, authorization, and recovery. A staff-level design lets the model reason where flexibility helps while keeping correctness and side effects under application control.