What is the best cloud provider for open-source LLMs at scale?

As of October 14, 2026 it depends on how much you want to manage. For raw GPUs at the lowest on-demand price, Lambda (H100 $3.99, B200 $6.69 per GPU-hour) and RunPod (H100 SXM $3.99, B200 $7.99, B300 $8.99). For a full cloud with cheap preemptible capacity, Nebius (H100 $4.50, B200 $8.50, spot from $0.79). For large reserved clusters, CoreWeave. For managed open-model serving without running vLLM yourself, Together AI (dedicated H100 $5.49, B200 $8.99). For spiky traffic that must scale to zero, Modal at about $3.95 per H100-hour billed per second.

How many GPUs do I need to serve a large open-weight model?

Enough memory for the weights plus the KV cache. An H100 has 80 GB, an H200 141 GB, a B200 180 GB and a B300 288 GB. A 1-trillion-parameter mixture-of-experts model such as Xiaomi's MiMo-V2.6-Pro (1.02T total parameters) at 8-bit needs roughly 1 TB for weights alone, so a full 8-GPU B200 or B300 node; a 120B model like gpt-oss-120b fits on a single 80 GB GPU at its native 4-bit precision.

Is renting GPUs cheaper than a per-token API for open models?

Only at sustained high utilisation. Per-token APIs such as Together, Fireworks or DeepInfra charge nothing when idle; a dedicated 8x B200 node costs about $53 to $72 an hour whether you use it or not. Rent GPUs when traffic keeps them busy most of the day, when you need a custom or fine-tuned model, or when data must stay in your own environment.

Quick Answer

Best Cloud Provider for Open-Source LLMs at Scale (2026)

Published: October 14, 2026

The short answer

For open-source LLMs at scale, pick by how much infrastructure you want to run: raw GPUs (Lambda, RunPod, Nebius), large reserved clusters (CoreWeave), managed open-model serving (Together AI) or serverless containers (Modal). List prices read from each vendor’s pricing page on October 14, 2026, per GPU-hour, on demand:

Provider	H100 80 GB	H200 141 GB	B200 180 GB	B300 288 GB	Billing	Best for
Lambda	$3.99	—	$6.69	—	Per hour; 1-Click Clusters 16–2,000+ GPUs	Cheapest on-demand B200, simple self-serve
RunPod	$3.99 (SXM) · $2.89 (PCIe)	$5.29	$7.99	$8.99	Per second; 31 regions	Flexible single-GPU to multi-node, serverless workers
Nebius	$4.50	$5.40	$8.50	$9.50	Per hour; preemptible from $0.79–$0.99	Full cloud plus cheap spot capacity, EU regions
CoreWeave	$6.16 (8x node $49.24/h)	—	$8.60 (8x node $68.80/h)	spot ~$4.48	Per hour; reserved contracts	Large reserved clusters, GB200/GB300 NVL72
Together AI	$5.49 (dedicated)	contact	$8.99 dedicated · $8.19 cluster	contact	Per hour; managed endpoints	Managed serving of open models, no ops
Modal	~$3.95	~$4.54	~$6.25	~$7.10	Per second, scale to zero	Spiky traffic, batch jobs, Python-native teams

Modal prices are its per-second rates multiplied by 3,600. CoreWeave lists whole 8-GPU nodes; the per-GPU figure is the node price divided by eight. This page is the “GPU cloud” spoke of our best LLM inference providers ranking.

Picks by situation

Steady high traffic on one or two models: Lambda or RunPod plus vLLM or SGLang. At sustained utilisation, renting GPUs and running an open serving engine — vLLM v0.31.0 (October 5, 2026) or SGLang v0.5.21 (October 2, 2026) — is the cheapest way to serve. Lambda has the lowest on-demand B200 ($6.69); RunPod has the widest menu, including B300 at $8.99 and per-second billing. See the fastest way to serve an open-weight model.

Batch and offline jobs: Nebius preemptible. Nebius lists preemptible H100 and H200 from $0.79 and B200/B300 from $0.99 per GPU-hour. Evaluation runs, embedding backfills and synthetic-data jobs that can checkpoint and restart are the right fit.

Hundreds or thousands of GPUs on contract: CoreWeave or Lambda Superclusters. At that size you negotiate reserved prices; list prices matter less than capacity, interconnect and delivery dates. CoreWeave sells GB200 NVL72 at $42 an hour per 4-GPU slice; Lambda quotes 1-Click Clusters of 16 to 2,000+ B200 or H100 GPUs.

You want open models, not GPUs: Together AI. Dedicated endpoints run the model for you (H100 $5.49, B200 $8.99 per GPU-hour), and the same account offers per-token serverless for traffic that does not justify a GPU. Compare managed options in best dedicated inference endpoint providers.

Traffic that comes and goes: Modal. Billing stops when the container stops, so an H100 that serves for 10 minutes an hour costs about $0.66, not $3.95. Cold starts are the trade-off for large models.

How to size the cluster

Weights first. Parameters × bytes per parameter: a 1T MoE at FP8 ≈ 1 TB → one 8x B200 (1.44 TB) or 8x B300 (2.3 TB) node; a 120B model at 4-bit fits on one 80 GB GPU.
Then KV cache. Long contexts and many concurrent users need as much memory again as the weights; this is why H200 and B300 beat more H100s for long-context serving.
Then throughput. Benchmark tokens per second per GPU at your real prompt lengths, divide your peak tokens per second by it, and add 30% headroom.
Compare against per-token prices for the same model on serverless providers before signing anything; for under a few hours of busy GPU time a day, per-token is usually cheaper. See leading serverless inference providers.

Last verified: October 14, 2026.

The short answer

Picks by situation

How to size the cluster

Sources