LLM gateway spending limits: stop a runaway agent bill?

Write down exactly what the budget covers

Before choosing a control, fill in: billed party and credential route; key, user, team or organization scope; covered models and tools; reset window and timezone; alert versus rejection; concurrent and in-flight requests; fallback billing; and who can raise or bypass the limit. Set separate ceilings for routes outside the gateway.

Use the cost estimator to estimate normal usage, the failover guide to bound retries, and the observability guide to verify which route actually ran. A budget policy is only one part of a tested spending boundary.

An alert, a rate limit, and a budget do different jobs

An agent can turn one user instruction into a long sequence of model calls. If it keeps retrying a broken tool or starts several workers, the cost comes from the work it launches after the first request. A monthly dashboard helps explain that work afterward. An unattended run also needs a rule that can refuse the next call.

The labels are unreliable. Vercel calls its budget a soft cap because a request can finish past the threshold, even though subsequent requests are blocked. Read the documented action rather than inferring it from the word “soft” or “hard”. Vercel’s budget semantics.

Six implementations and their enforcement limits

These examples were selected for the differences their documentation exposes. They are not a ranking or an exhaustive list of gateways with budgets. Each row describes a specific control; neighbouring features or editions may behave differently.

A pricing gap can become an enforcement gap. Portkey’s separate provider-budget documentation says requests whose model pricing is untracked do not count toward that cost limit. Check an actual request for every model you permit, including custom deployments, before treating the counter as complete. Provider-budget pricing limitations.

Not published: a numeric worst-case overshoot bound for these controls in the cited documentation. A missing bound is an unanswered question, not evidence that the overshoot is zero.

A working cutoff can still finish above the limit

Checking the balance before a request starts is only one part of enforcement. If its cost enters the counter after it finishes, other requests may be admitted against the same remaining allowance. Cloudflare explicitly documents this concurrency behaviour. How its spend checks work.

This example is not an upper bound. Longer requests, more workers, or a slower counter update can change the exposure. To promise an exact ceiling, a system needs a defensible maximum cost for admitted work and a shared, atomic way to reserve that allowance before launching it, then reconcile the actual charge. Ask the vendor whether its control does this.

Our recommendation is to combine the budget with limits on concurrent work, input size, output generation, and task steps. Leave headroom below the amount you cannot afford to exceed. Choose that headroom from observed behaviour and known request bounds; an arbitrary percentage is not a guarantee.

The budget must follow the spending you meant to limit

Write the intended policy in one sentence: “This background research task may spend this much before it stops.” Then identify the counter that implements it. A limit on a provider, a key, a person, and a team answers four different questions.

OpenRouter’s member-assigned budget combines that person’s keys, but gives each member their own allowance. Giving several people the same allowance therefore creates several spending buckets. A shared credit pool does not turn those individual limits into one team budget. Member-budget behaviour.

With BYOKBYOK — bring your own key: You keep your own accounts and contracts with OpenAI, Anthropic and the rest, and the gateway routes through your keys. You keep your negotiated rates and any committed-spend discounts; the gateway charges you for the plumbing, not the tokens.BYOK — bring your own keyYou keep your own accounts and contracts with OpenAI, Anthropic and the rest, and the gateway routes through your keys. You keep your negotiated rates and any committed-spend discounts; the gateway charges you for the plumbing, not the tokens., inspect the setting rather than assuming that visible spend is capped spend. OpenRouter exposes include_byok_in_limit on API keys. Vercel’s documented budgets exclude provider-key spending. OpenRouter’s accounting option; Vercel’s exclusion.

Check the time boundary too. OpenRouter documents calendar resets at midnight UTC; Cloudflare supports fixed and rolling windows. A daily allowance is renewed spending permission. For a one-off task, a cumulative allowance or an explicit task stop avoids letting the same broken run start spending again tomorrow. Reset options; Window types.

Bind budget ownership on a trusted server or through authenticated identity. If a caller can freely change the metadata value used as its budget bucket, test whether that change gives it a fresh allowance. Keep provider credentials and budget-management permissions outside the agent’s control, and verify that direct provider calls cannot bypass the intended gateway path.

A budget rejection should change what the agent does

A client that retries every error can keep a stopped task alive until its budget resets. The HTTP status alone is not a reliable classifier: Cloudflare documents a 429 for spend limits, Vercel a 402 with quota details, and OpenRouter a 403 for a guardrail budget rejection. Inspect the reason and the exhausted scope. Cloudflare; Vercel; OpenRouter.

Our preferred behaviour is to pause the affected task, save its progress, stop launching dependent work, and notify its owner. Resume after a deliberate decision or an explicitly approved reset policy. Verify what cancellation does to requests already running; stopping the local process does not establish that billing stopped.

A cheaper fallbackFailover: When the provider you asked for is down or rate-limiting you, the gateway automatically retries somewhere else. The single most valuable reliability feature these products offer.FailoverWhen the provider you asked for is down or rate-limiting you, the gateway automatically retries somewhere else. The single most valuable reliability feature these products offer. can be a useful policy, but it still spends money. Cloudflare documents routing to a cheaper model when a primary model’s allowance runs out. If the intention is to stop total spending, put both routes under an appropriate shared limit and test the whole chain. Budget-triggered fallback.

Finally, inventory the task’s other bills. Search tools, browser sessions, sandboxes, and storage may be charged outside the gateway’s model counter. Keep an application-level task budget and step limit alongside gateway controls. The application knows whether the work is making progress and what else it is buying.

Test the cutoff before the agent runs unattended

Use an isolated test workload, a small allowance, and bounded requests. Record the configuration, model, authentication path, timestamps, and settled cost. The acceptance test should demonstrate what stops, what remains billable, and how the operator finds out.

1. Cross the threshold sequentially. Confirm the next call is rejected, identify its error reason, and check that a rejected call was not forwarded upstream.
2. Repeat with concurrent and streaming calls. Measure final spend after all admitted work settles. Record overshoot and whether active streams finish.
3. Exercise every accounting path. Include BYOK, custom models, caching, and each billable modality you use. Compare gateway totals with provider usage.
4. Test the scope. Try a second key, another worker, and changed metadata. The intended parent budget should still apply; an unrelated workload should retain its own allowance.
5. Exercise retries and fallbacks. A budget rejection must not silently switch to an uncapped key or provider. Check that the agent pauses.
6. Test the clock and the dependencies. Verify reset timing and configuration propagation. For self-hosted systems, test counter-store outages and restarts in staging; establish whether spending continues when checks fail.
7. Verify the notification and recovery. Confirm an alert reaches its intended recipient, identify who may raise the limit, and resume without replaying already-completed tool actions.

Portkey illustrates why the last check matters: its workspace-policy threshold records an audit event without sending a proactive alert, while its separate provider-budget page describes email notifications. Verify the exact feature you configured. Policy thresholds; Provider-budget alerts.

Our buying criterion is straightforward: choose a gateway whose documented scope matches your workload, then require an observed cutoff under your actual billing and concurrency conditions. Keep the test result with the configuration and rerun it when authentication, routing, model pricing, or the gateway version changes.

Common questions