What is a Provider-Side Token Inflation Attack (PTIA)?

A PTIA is when a pay-per-token LLM provider covertly manipulates generation so the model emits more output tokens than it otherwise would, while keeping the answer useful enough that customers do not notice. The term was formalised in the September 17, 2026 arXiv paper 'The More It Says, the More You Pay' (2609.20370), which demonstrates five attack variants at the query, prompt, representation and model levels of the provider-controlled pipeline. Each variant raised mean output length to more than 10.2 times the clean baseline. Because customers only see the response and the bill, the inflation is invisible without an audit.

How does the single-probe token inflation audit work?

It exploits saturation. The paper observes that the first lengthening intervention sharply lowers the model's end-of-sequence probability, but stacking a second similar intervention adds very little. So the audit sends two requests that look like ordinary traffic: the original prompt, and the same prompt with a controlled lengthening instruction added by the customer. Under honest service the probe produces many additional tokens; under a PTIA it produces far fewer, because the provider already pushed the model close to its ceiling. No trusted local reference model or history of clean responses is required. Across four open-weight models the audit detected inflation 85.1% of the time with false positives below 2%.

Are real LLM APIs inflating tokens?

The paper ran its audit against 15 real LLM API services and flagged 7 for PTIA-consistent behaviour. The authors do not name the services in the abstract and a flag is not proof — a provider's default system prompt, a verbose fine-tune, or reasoning tokens billed as output can produce the same signature. Treat a flag as grounds for a controlled cost benchmark against a second provider serving the same open-weight model, and for a conversation with your account team.

What can I do today to limit token inflation costs?

Set max_tokens on every request and alert on responses that hit the cap; log output-token counts per prompt template and watch the distribution rather than the mean; run the same prompt set weekly through two providers of the same open-weight model and compare tokens per task; prefer providers that itemise reasoning tokens separately; and use structured outputs or JSON schema where possible, which caps how much a model can pad. Retrieve billing metadata (usage.output_tokens) and reconcile it against your own tokenizer counts monthly.

Quick Answer

How to Audit an LLM API for Token Inflation (2026 Guide)

Published: September 21, 2026

Why this matters in 2026

Every frontier API bills by the token, and output tokens cost 4–5x input: GPT-6 Astra is $10/$50 per million, Claude Fable 5.1 is $10/$50, Gemini 3.8 Flash is $0.75/$3.75 through December 31, 2026 (rates verified September 2026; see the API cost guide). The revenue lever is therefore on the provider’s side of the pipe: a serving stack that quietly makes the model talk longer makes more money, and the customer cannot see the difference between “the model is verbose” and “the provider made it verbose.”

Until September 2026 that was a suspicion. A paper from Leilei Chen, Lan Zhang and colleagues, “The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services” (arXiv 2609.20370, submitted September 17, 2026), gives it a name — the Provider-Side Token Inflation Attack (PTIA) — shows it is easy to do at four different layers, and, more usefully, gives customers a test they can run against an opaque API. This guide turns that test into a procedure.

Step 1 — Know the five ways a provider can inflate output

The paper instantiates one attack at each layer a provider controls:

Layer	What the provider changes	Why you can’t see it
Query	Rewrites or expands your request before it reaches the model	You only see your own prompt
Prompt	Adds hidden system instructions (“explain thoroughly”, “show all steps”)	Hidden system prompts are never returned
Representation	Steers hidden activations toward continuation	Nothing in the output identifies it
Model	Serves a fine-tune trained to be longer	Same model name on the invoice
Decoding	Suppresses or penalises the end-of-sequence token	Looks like ordinary sampling

Every one of these pushed mean output length above 10.2x the clean baseline while largely preserving task utility on the paper’s benchmarks. The point is not that your provider does this; it is that the capability is trivial and the incentive is structural.

Step 2 — Build a probe set that looks like normal traffic

The audit works with paired requests, so you need prompts where the honest answer has a natural stopping point:

30–50 prompts from your real templates (support replies, extraction, summarisation, code review). Synthetic trivia is fine for a first pass but production prompts catch fine-tune-level inflation.
Keep temperature and top_p at your production values; run each prompt 5 times to average sampling noise.
Record usage.output_tokens from the API response and your own tokenizer count of the returned text. A persistent gap between the two is a separate red flag (billed tokens you never received — typically reasoning tokens, which should be itemised).

Space the paired requests out over hours and mix them into normal traffic. The paper notes the probe’s strength is that the original and probed requests “resemble ordinary traffic, making evasion difficult”; sending them in an obvious burst weakens that.

Step 3 — Run the single-probe audit

The paper’s key observation is saturation: an initial inflation intervention sharply lowers the end-of-sequence probability, but a second intervention lowers it only marginally. So:

Send the original prompt. Record mean output tokens L₀.
Send the probed prompt: the same prompt with a fixed, customer-controlled lengthening instruction appended — the paper uses a controlled lengthening intervention; in practice a line such as “Give a complete, step-by-step answer with full explanation.” Record L₁.
Compute the probe gain G = L₁ / L₀.

Interpretation:

Honest service: the probe lands on an un-inflated model and produces a large jump — G of roughly 2x or more is typical on explanatory tasks.
PTIA-consistent: the model was already pushed toward its ceiling, so the probe adds little — G close to 1.

Calibrate the threshold on a provider you trust or on a self-hosted copy of the same open-weight model. Across four open-weight models the paper reports an 85.1% detection rate at under 2% false positives, without any trusted local reference or historical clean responses — but your thresholds should come from your own prompt set.

Step 4 — Cross-check before you accuse anyone

A low probe gain is a signal, not a verdict. Rule out the honest explanations:

Default system prompts. Many hosted endpoints prepend instructions. Ask for them, or test with a system message that explicitly says “Answer in the fewest words that fully answer the question” and see whether L₀ drops.
Reasoning tokens. Models with extended thinking bill reasoning as output. Compare output_tokens against visible text; if the gap is large and the provider does not itemise it, that is a billing-transparency problem in its own right (see the tokenizer tax analysis).
Different checkpoint. Two providers serving “the same” open-weight model may be serving different quantisations or revisions. Pin the revision hash where the provider exposes it.
Provider comparison. Run the identical probe set on a second provider of the same model (inference providers ranked). Tokens-per-task should agree within sampling noise; a 30%+ gap is worth escalating.

The paper’s own sweep of 15 real LLM API services flagged 7 for PTIA-consistent behaviour. The abstract does not name them, and the authors are explicit that the audit detects behaviour consistent with inflation.

Step 5 — Make inflation expensive to attempt

Controls you can ship this week, independent of the audit:

max_tokens on every call, sized per template, with an alert when the cap is hit more than a few percent of the time.
Structured outputs / JSON schema for extraction and classification — a schema caps padding (how to get structured JSON).
Per-template token dashboards with p50/p95, not just totals; inflation shows up as a right-shifted distribution before it shows up in the invoice.
Monthly reconciliation of billed output tokens against your own tokenizer counts.
Contract language: ask for the default system prompt, the served checkpoint identifier, and itemised reasoning tokens. Providers that cannot answer are telling you something.
Route commodity work to open-weight models across two providers so you always have a comparison point (caching, batching, routing guide).

What this does not cover

The audit targets output inflation. Input-side games — silently expanding prompts, or repricing above a context threshold the way GPT-6 Astra does beyond 272K tokens — need separate checks against the pricing page. And none of this replaces reading the vendor’s usage object carefully: the most common “inflation” in 2026 is still reasoning tokens a team did not know it was paying for.