Self-Hosted vs Managed LLM Gateway: Which in 2026?
The Short Answer
Both approaches give you one API surface over many model providers. The difference is who carries the risk.
| Self-Hosted (e.g. LiteLLM) | Managed (e.g. hosted gateway) | |
|---|---|---|
| Per-token markup | None | None to meaningful, by vendor |
| Provider relationship | Yours | Yours or the vendor’s |
| Prompt data in third party | ❌ No | ✅ Yes |
| Uptime responsibility | You | Vendor |
| Model catalog currency | You maintain | Vendor maintains |
| Time to first request | Hours to days | Minutes |
| Failure blast radius | Your proxy = all AI features | Vendor outage = all AI features |
| Best below ~$2k/mo spend | ❌ Rarely worth it | ✅ Yes |
| Best above ~$20k/mo spend | ✅ Usually | ⚠️ Measure the markup |
Verified August 24, 2026.
The Cost Model Nobody Writes Down
The markup argument is the one people make, and it is the least important below real volume.
Take a moderate workload: 200 million input and 40 million output tokens a month. On Gemini 3.7 Flash at its introductory $0.75/$3.75 per million tokens (in effect through December 31, 2026) that is $150 + $150 = $300 a month. A 5% gateway markup costs $15. No engineer’s time is worth saving $15.
Now scale to a serious agent workload on Claude Opus 5 at $5/$25 per million tokens — 400M input, 80M output is $2,000 + $2,000 = $4,000 a month. The same 5% is $200 a month, or $2,400 a year, which is still less than the fully loaded cost of maintaining a proxy well.
The number that actually flips the decision is rarely markup. It is one of these:
- A compliance requirement that forbids prompt content leaving your perimeter. This is binary and instantly decides the question.
- Committed-use pricing you can only access by holding the provider relationship directly.
- Routing sophistication you need and no vendor implements — per-customer budgets, custom model-selection heuristics, weird failover chains.
If none of those apply, managed is the rational default regardless of size.
What Self-Hosting Actually Buys
1. Data path control. Your prompts never enter a third-party system. For regulated workloads this is the whole argument, and it does not require any cost justification.
2. Provider economics. You keep enterprise agreements, committed spend discounts, batch pricing and cache economics negotiated directly. Anthropic’s cache reads at 0.1x and DeepSeek’s off-peak window — peak is 01:00–04:00 and 06:00–10:00 UTC, with all other hours at exactly half price since August 16, 2026 — are the kind of provider-specific optimization you exploit best when you own the relationship.
3. Strategic independence. Your routing layer cannot be repriced, repositioned or acquired out from under you. Stripe’s agreement to acquire OpenRouter, announced the week of August 17, 2026 at a reported ~$7.5 billion, is a reminder that hosted infrastructure has owners and owners have strategies.
What Self-Hosting Actually Costs
1. You are the uptime. This is the failure that hurts. A single-instance proxy means one restart takes down every AI feature simultaneously. If you self-host, run at least two instances behind a load balancer and health-check the proxy separately from the models — otherwise you have swapped a vendor risk for a worse self-inflicted one.
2. You are the catalog. 2026 shipped models continuously: Gemini 3.7 Flash on August 13, Grok 4.6 on August 12, GLM-5.3 on August 14, DeepSeek V4 repricing on August 16. A stale config quietly overpays. Managed gateways absorb this churn; self-hosted ones make it a recurring chore with no deadline, which means it does not get done.
3. You are the observability team. Cost dashboards, per-customer attribution, latency percentiles and evaluation hooks all exist in managed products and all must be built or assembled in a self-hosted one.
The Hybrid That Usually Wins
The strongest 2026 pattern is not either column. It is:
Self-hosted proxy as primary, managed gateway as configured fallback.
This gets you the markup-free, data-controlled default path, and it converts your proxy from a single point of failure into a preference. When the proxy fails a health check, traffic shifts to the hosted gateway using a key that has been sitting warm and tested. You pay for that fallback only when you need it.
It costs one config block and one monthly smoke test. It removes the single worst outcome — your own infrastructure taking down your entire product’s AI surface — which is the failure that self-hosting advocates consistently underweight.
The Decision Checklist
Answer in order and stop at the first yes:
- Does a contract or regulation forbid prompt data reaching a third party? → Self-host.
- Do you have committed-use provider pricing that a reseller would break? → Self-host, or use a managed proxy that keeps your keys.
- Is your monthly model spend under about $2,000? → Managed. Optimizing this is procrastination.
- Do you have anyone who would carry a pager for a proxy? → If no, managed. A proxy nobody owns is worse than a vendor.
- Otherwise → Self-host primary with a managed fallback.
The Test That Matters More Than the Choice
Whatever you pick, confirm you can leave it:
- Does your application code use only standard OpenAI-compatible fields?
- Can you change one environment variable and hit a provider directly?
- Do you log which provider and model actually served each request?
- Do you have a live, tested direct provider key today?
Teams that pass all four can change gateways in an afternoon. Teams that fail have already made the decision permanent without noticing — which is the only version of this choice that is genuinely expensive.