How to Stop AI Agents from Wasting Tokens (Updated 2026)

“Token killing” is usually a system problem

When an agent consumes far more tokens than expected, the model is rarely the only cause. The system may replay the full conversation on every turn, retrieve too much evidence, expose verbose tools, use maximum reasoning for simple steps, repeat failed calls, or coordinate several workers without a shared budget. Cutting the final answer to three sentences will not fix that architecture.

Start with evidence. Break one expensive run into stages and attribute input, cached input, output, reasoning tokens, tool calls, retries, latency, and outcome. The largest visible message may not be the largest cost. Repeated prefixes, hidden reasoning, and tool observations carried into later turns can dominate.

Killer one: replaying everything

A naïve agent appends every message, plan, log, document, and tool result forever. Each new turn resends the growing history. Even when the model supports a large window, cost and latency rise, stale instructions remain active, and duplicated evidence competes with the current task.

Keep a compact working state: objective, current constraints, verified facts, unresolved questions, artifacts, and next action. Drop completed scratch work. With Responses API conversation chaining, follow the documented state pattern instead of manually duplicating items. For long-running workflows, use supported compaction after meaningful milestones.

Compaction reduces context while carrying forward task-relevant state in an opaque item. OpenAI supports server-side compaction with a compact_threshold and standalone compaction. It should be planned before the window is exhausted, not triggered after failure. Do not compact every turn; the compaction pass itself has usage and can remove useful detail if applied without a milestone.

Killer two: uncontrolled retrieval

Retrieval often returns what matches, not what the decision needs. An agent asking for “everything about this customer” may receive years of emails, tickets, documents, and CRM events. A coding agent may load a full monorepo when three files and one interface would answer the question.

Retrieve in layers. Start with metadata, titles, summaries, symbols, or search hits. Let the agent request precise slices when a hypothesis requires them. Limit result count, document length, time range, and fields. Deduplicate repeated passages and keep source identifiers so later turns can cite or fetch details without resending the raw content.

Measure retrieval precision: the share of returned material actually used in the final decision. Low precision is a token problem and a product-quality problem.

Killer three: verbose tools

Tools designed for humans often return banners, progress bars, full payloads, stack traces, and hundreds of unchanged rows. When this text enters the next model turn, it becomes input tokens. A single command can erase the savings from a carefully optimized prompt.

Give agent tools a compact mode. Return a typed status, key identifiers, bounded records, a summary, and an address for the full artifact. On failure, return the error class, likely cause, retryability, and a short diagnostic excerpt. Paginate deliberately.

For shell and logs, filter at the source. Request the failing tests, changed files, or last relevant lines rather than dumping everything and asking the model to summarize afterward.

Killer four: retries without change

Automatic retry is useful for transient failures. It becomes waste when the same call repeats with the same inputs after a deterministic error. Classify errors as transient, permission-related, validation-related, missing-input, or unknown. Give each class a different policy.

A retry must change a meaningful variable or wait for a state that can actually change. Permission failures need approval or configuration, not five identical attempts. Validation failures need corrected input. Missing business context needs a person. Set per-tool and per-error retry limits plus a total run limit.

Record why each retry occurred and whether it succeeded. A high retry rate can reveal a poor tool contract, unstable dependency, ambiguous prompt, or wrong model routing.

Killer five: maximum reasoning everywhere

High reasoning effort can improve hard tasks, but routing, extraction, formatting, and deterministic checks rarely need the maximum. Set effort by stage. Use a cheaper or lower-reasoning configuration for classification and simple transformations, then escalate difficult synthesis or debugging when evaluations justify it.

Do not infer quality from token consumption. Measure whether higher effort improves verified completion enough to offset latency and cost. If two settings pass the same tests, choose the less expensive one.

Killer six: multi-agent fan-out

Parallel workers can reduce wall-clock time on independent problems, but every worker may receive the same large prefix, call overlapping tools, and return material the coordinator must read. Four agents can cost more than four times a single focused run when coordination overhead is included.

Delegate only separable work with explicit deliverables. Give each worker the minimum context for its lane, a token and tool budget, and a no-duplication boundary. The coordinator should receive concise findings plus evidence links, not every worker’s transcript.

Compare multi-agent and single-agent versions on total task success, tokens, latency, and reviewer time. Parallelism is a tradeoff, not a default badge of sophistication.

Killer seven: no stopping rule

An agent without a definition of done may keep researching, refining, checking, or trying alternatives. Define success as an observable condition: tests pass, the destination contains the expected record, all required sources are cited, or the human approved the proposed write.

Also define non-success stops: maximum calls, maximum retries, token or dollar ceiling, deadline, repeated state, missing permission, or low-confidence decision. When a stop fires, the agent should return the best verified state, the blocker, and the smallest next action.

Use a loop detector based on repeated tool calls, unchanged results, or no progress in the verification metric. “Do not stop” is not a safe operating policy unless the system also has bounded authority and escalation.

Make caching earn its place

Prompt caching lowers the cost and latency of repeated exact prefixes. Keep stable instructions, schemas, and tools at the beginning and put variable data later. Monitor cached_tokens and, on GPT-5.6 and later, cache_write_tokens. A long prompt is not efficient merely because part of it is cached; irrelevant cached text can still complicate the task and cache writes have economics.

Use the cache for stable content that many runs genuinely need. Remove duplicated rules and unused tools first. Then optimize prefix stability. Validate that changes preserve task success.

Install a run budget

Set a global budget and stage budgets. The global budget may include maximum input, output, reasoning, tool calls, retries, wall time, and estimated dollars. Stage budgets prevent one early search from consuming everything needed for action and verification.

At each checkpoint, estimate remaining work. If the task cannot finish inside the remaining budget, summarize state, narrow scope, request approval for expansion, or stop with a blocker. Do not silently exceed a customer-facing limit.

Track p50, p90, p95, and maximum usage. Averages hide runaway runs. Alert when a workflow’s percentile distribution shifts, cached ratio drops, or cost per accepted task increases.

A 30-minute remediation sequence

Take one costly trace. Remove duplicate history and stale tool output. Add row, byte, and time limits to the noisiest tool. Set a retry cap and loop detector. Move stable prompt material before variable content. Choose a measured reasoning level. Add a compact state checkpoint after the largest completed phase. Re-run the same task and compare outcome, tokens, latency, and reviewer effort.

Repeat one change at a time so you know what helped. The goal is not the smallest prompt. It is the smallest reliable system that finishes with evidence.

How to Stop AI Agents from Wasting Tokens