We do not optimize token usage to cut cost. We optimize it because token is the raw material of our product — and waste is a product defect.
00 · Why ActelosAI Writes About Tokens
Generative Engine Optimization is, at its core, a wager on conversations with frontier models. Our agents query models to measure how a brand appears in AI-generated answers, and increasingly to shape it. Every query, every reasoning step, every retrieved page — they are all token. A company that sells “optimization of generative engines” must first be excellent at optimizing its own consumption of generative engines.
You cannot teach a client Generative Engine Optimization while your own token bill leaks. We should apply the same discipline to our own work.
01 · Token Is a Product Problem, Not a Cost Problem
The default instinct is to treat token spend as an infrastructure line item — a bill to be negotiated down. That framing is wrong. For an AI Native company, token is unit economics: it is the denominator of everything we ship, and it compounds across every agent, every customer, every retry.
The real issue is deeper than the bill. An agent’s autonomy carries two authorizations that are almost never separated:
| Authorization | What it means | What we do |
|---|---|---|
| Execution | The right to act — call tools, write files, run code. | Granted within an explicit scope. |
| Consumption | The right to spend — issue LLM calls, spawn subagents. | Granted rarely, and made visible. |
An internal incident reported by the author illustrates what can happen when the second is ignored. A single task spiraled into 12 subagents → 265 LLM requests → ~19.8M tokens → ≈ ¥1000 in under five minutes. These are author-reported internal figures, not a controlled benchmark or a client result. The breakdown:
| Metric | Value | Signal |
|---|---|---|
| Input tokens | ~19.64M | Context carried the weight. |
| Output tokens | ~0.13M | The model barely produced. |
| Cached read | 0 every round | No cache reads were recorded; the cause needs request-level verification. |
The lesson An agent’s autonomy has two halves — the right to act, and the right to spend. Both need explicit boundaries; consumption also needs a visible budget and a stop condition.
02 · The Anatomy of a Token Bill
To govern consumption you must first see it. Most people vastly misjudge where their tokens go, because of one counterintuitive possibility: the text you type may be only a small part of the total input processed.
| Where tokens live | Share | Why |
|---|---|---|
| Your message | Workload-dependent | The prompt you actually wrote. |
| System tax | Large | System instructions and loaded tool/context definitions that may recur across calls. |
| History | Large | Prior turns retained by the application. |
| Retrieved context | Variable | Everything you stuffed into the window. |
This is why “write shorter prompts” is often incomplete advice. The cost lives in what you carry, not in what you type. Optimization therefore begins not with the message, but with the environment: system prompt, tool definitions, and conversation hygiene.
03 · The Five-Layer Funnel
The following five-layer funnel is a practical order for reviewing token efficiency. Start at the top, then measure which layer offers the most leverage for the actual workload.
| # | Layer | The question it answers | Example |
|---|---|---|---|
| 1 | Don’t use an LLM | Does this step need intelligence at all? | Deterministic code, caching, rules. |
| 2 | Use a cheaper model | Does it need a frontier model? | Route routine steps to small models. |
| 3 | Send fewer tokens | Am I carrying dead weight? | Compact context, drop stale history. |
| 4 | Make tokens cheaper | Is this content being paid for twice? | Prompt caching. |
| 5 | Don’t pay twice | Can this call be reused? | Validated result reuse, memoization. |
Do not stop at shorter prompts. First ask whether the call is needed, and whether repeated context can be reused.
04 · Caching: A High-Leverage Starting Point
For repeated prompts, caching can materially reduce input-processing cost. But a cache-read discount is not a discount on the whole task: writes, misses, outputs and any storage charges still matter.
| Provider | Mechanism | The catch |
|---|---|---|
| Anthropic | Prompt caching | 5-minute writes typically cost 1.25× base input; 1-hour writes 2×. Read rates depend on the model. |
| OpenAI | Prompt caching | Eligible prefixes, write pricing and retention vary by model; prompt_cache_key can help routing but does not guarantee a hit. |
| Context caching | Gemini 2.5 and newer support implicit caching; minimum lengths and prices are model-specific. |
Technical reference: Anthropic prompt caching, OpenAI prompt caching, and Google context caching, checked 8 September 2026.
Three disciplines make caching actually work:
01 · Stable first, dynamic last The cached prefix must be byte-identical. Put the system prompt, tool schemas, and static instructions first; put the changing parts — user message, retrieved context, tool results — at the end. One character of drift at the front invalidates everything behind it.
02 · Don’t break the chain Every edit to the beginning of a conversation resets the cache for everything after it. Treat the shared prefix as immutable; append, never splice.
03 · Verify the cache is alive
In the reported incident, cacheReadTokens = 0 was a signal to investigate, not a complete diagnosis. Check the provider’s usage fields, eligible prefix length, cache settings, expiry and request changes. A missing conversation header is not a universal explanation for cache misses. Cache metrics must be a first-class part of your telemetry, or you will “optimize” a lever that is not attached.
05 · Context Management: Carry Only What the Work Needs
When repeated context dominates the input, the leverage is in what you stop carrying. The disciplines:
- Compact early, compact often Long sessions accumulate stale turns that the model re-reads every round. Summarize finished work when that preserves the decisions and evidence still needed; measure the saving against any lost cache reuse.
- Right-size the environment Review what Memory, Rules, Skills and MCP actually load into model context. Load the useful content when the task needs it.
- Retrieve, don’t hoard A small, well-targeted retrieval beats a stuffed window — for both cost and answer quality.
The discipline of context is subtraction, not addition. Every token you carry without purpose is a tax on every subsequent turn.
06 · Model Routing & Structured Output
Not every step needs a frontier model, and not every answer needs to be prose.
Route by difficulty Reserve frontier models for the steps that need genuine judgment — reasoning, synthesis, edge cases. Route the routine — classification, formatting, extraction — to a smaller, cheaper model. The saving depends on models, retries and quality requirements; compare cost per successful task.
Turn the reasoning knob The reasoning.effort / text.verbosity pair (OpenAI) offers separate controls on supported models: it separates how hard the model thinks from how much it says. Lower it for routine work; raise it only where depth pays.
Force the shape Structured output — JSON schema, tool calls — collapses a model’s tendency to pad. A constrained format can reduce unnecessary prose and formatting retries. It does not guarantee fewer tokens, correct content or successful completion.
07 · Agent Architecture: Token Total ≠ Cost
A useful distinction: total token count alone does not determine spend. In an internal comparison reported by the author, Claude Code Internal ran 22 small requests and cost ¥9.05, while CodeBuddy IDE ran 2 large requests and cost ¥18.34 — twice as much, from far fewer tokens. Model choice, input/output mix, cache usage and billing terms must be compared before attributing the difference to architecture. The figures illustrate those recorded runs, not a like-for-like product ranking.
Three architectural rules follow:
- Bound the subagent tree Depth and fan-out multiply requests exponentially. Cap both, and require justification for spawning.
- Prefer well-scoped calls Split work when it reduces total cost without losing necessary context or quality; more calls are not automatically cheaper.
- Make spend a first-class product surface The agent should show its running cost, its cache hit rate, and a spend ceiling — not hide them in a monthly bill.
08 · The Numbers That Matter
You cannot govern what you do not measure. These seven metrics, watched together, tell you whether your token strategy is working:
| Metric | What it reveals |
|---|---|
| Cost per task | The unit economics of an agent run. |
| Cache hit rate | Whether your biggest lever is attached. |
| Input / output ratio | Whether context is bloated. |
| Tokens per outcome | Efficiency per unit of delivered value. |
| P50 / P90 / P99 latency | Whether routing and caching are healthy. |
| Retry rate | Where parsing and determinism fail. |
| Cost per brand / per customer | The GEO-native view: margin per engagement. |
Cost per successfully completed task is the economic north star—with quality, control and latency as constraints.
09 · What This Means for ActelosAI
For ActelosAI, token efficiency belongs inside capability design. Actelos Delta asks what outcome is worth pursuing and what evidence will demonstrate useful change. Actelos Forge brings those decisions into engineering: model selection, context, permissions, evaluation and the assets needed to operate the capability.
GEO within Growth Marketing provides a concrete context for applying these disciplines:
- A shared, cached ontology Keep authorized category definitions, brand facts and query libraries in a stable reusable prefix, where the selected provider supports it.
- A daily per-brand budget Set an agreed consumption ceiling; reaching it should trigger review, not an unexpected bill.
- Model routing by question type Use deterministic checks or smaller models where sufficient; reserve deeper model judgment for questions that need it, verified through evaluation.
- Telemetry before optimization We never “optimize” a lever we cannot see.
10 · The Direction: From Token Pricing to Outcome Pricing
Token pricing is a transitional honesty: it is what we have, not what we want. The value a customer buys is not “fewer tokens” — it is a brand that appears, is cited, and is trusted inside AI answers. As reasoning models and caching mature, we expect more commercial models to connect charges with useful outputs and outcomes, rather than input volume alone.
But that migration does not let us relax. Until pricing follows value, the company that treats token as raw material — measured, cached, routed, and bounded — will out-execute the one that treats it as a bill. Cost discipline is not the enemy of ambition; it is how ambition ships.
OUR POSITION Token optimization is not a cost-cutting exercise. It is the discipline of an AI Native company that treats its raw material with the same respect a manufacturer treats steel. Waste is not an expense — it is a defect, and we refuse to ship defects.