How do you evaluate a cloud AI migration to Baseten?

Score four things before moving: packaging effort (can your current container run on Baseten unchanged through its custom Docker server support), cost at your real traffic shape (Baseten bills each running replica per minute, including replica start-up, with a default 15-minute scale-down delay), compliance (SOC 2 Type II and HIPAA are included on every plan), and lock-in (Enterprise lets you self-host in your own cloud and use existing cloud commitments). Then migrate one model behind a feature flag and compare latency and cost per 1,000 requests with your current setup.

Can I move my existing vLLM or SGLang deployment to Baseten?

Yes. Baseten deploys pre-built containers such as vLLM, SGLang, Triton or NVIDIA NIM: set base_image to your image and add a docker_server block with the start command, server port, predict endpoint and readiness and liveness endpoints in Truss's config.yaml. Port 8080 is reserved by Baseten's proxy, containers run as a non-root user (UID 60000), and the base image must be Debian-based with Python on PATH unless you use the no-build option.

How much does Baseten cost after migration?

Dedicated deployments bill per minute per replica: H100 $0.10833 (about $6.50 an hour, or about $4,680 for a 30-day month running 24/7), B200 $0.16633, A100 $0.06667, fractional H100 (MIG, 40 GB) $0.0625 and L4 $0.01414, from Baseten's instance reference on October 17, 2026. The Basic plan is $0 a month, pay as you go; Pro adds priority GPU access and volume discounts; Enterprise adds self-hosting, custom regions and data-residency control.

When should you not migrate to Baseten?

When you only call popular open models per token (a serverless catalog API is simpler and usually cheaper), when traffic is tiny and spiky enough that cold starts dominate, or when a hyperscaler commitment you must burn makes SageMaker or Vertex AI effectively free. Baseten fits best for custom or fine-tuned models in customer-facing products.

Quick Answer

Migrating AI Inference to Baseten: 2026 Evaluation Guide

Published: October 17, 2026

The short answer

Migrate to Baseten when you run your own or fine-tuned models in production and want autoscaling, compliance and help with latency without operating Kubernetes GPU clusters yourself. The move is usually light — Baseten runs your existing vLLM, SGLang, Triton or NIM container — but the bill depends on how you configure autoscaling, so model the cost on your real traffic before you switch. Facts are from Baseten’s documentation, read October 17, 2026.

The evaluation scorecard

Question	What to check on Baseten	Green light if…
Packaging	Custom Docker server (`base_image` + `docker_server`), or Truss Model class, or no-build	Your image is Debian-based with Python, doesn’t need port 8080, runs as non-root
Cost	Per-minute replica billing; start-up minutes are billable; scale-to-zero by default	Steady traffic, or spiky traffic that tolerates cold starts
Hardware	L4 to 8×B200, fractional H100 (MIG)	Your model fits an available SKU
Compliance	SOC 2 Type II and HIPAA on all plans	You need BAAs and audit reports
Lock-in	Truss is open; Enterprise self-hosts in your cloud	You can leave without rewriting serving code

How the migration works

Containerised server (most common). Point Truss at your image:

base_image:
  image: your-registry/your-vllm-image:latest
docker_server:
  start_command: vllm serve your-model --port 8000
  server_port: 8000
  predict_endpoint: /v1/chat/completions
  readiness_endpoint: /health
  liveness_endpoint: /health

Baseten layers its reverse proxy onto your image. Port 8080 is reserved (binding to it fails with “address already in use”), and containers run as user app, UID 60000.

Pick hardware with resources.instance_type, e.g. H100 or L4:4x16. Changes to config.yaml affect only new deployments.
Deploy to staging, then promote: baseten model push --environment staging, then baseten model deployment promote (or truss push --promote).
Set autoscaling deliberately (see below), then mirror a slice of production traffic.

Autoscaling and billing: the traps

Scale-to-zero is the default (min replicas 0): idle deployments cost nothing, but the first request after idle waits for a cold start, and starting and loading a replica is billable.
Default scale-down delay is 900 seconds (range 0–3,600). Every burst therefore keeps a replica billed for 15 minutes after the last request.
Autoscaling window is 60 seconds by default; replicas = average concurrent requests ÷ (concurrency target × target utilisation).
Practical rule: for a latency-sensitive product, keep one warm replica (min replicas 1) and tune the delay; for batch or internal tools, keep scale-to-zero.

What it costs

Instance	Per minute	≈ Per hour	24/7 for 30 days
L4 (4 vCPU, 16 GiB)	$0.01414	$0.85	~$611
H100 MIG (40 GB)	$0.06250	$3.75	~$2,700
A100 80 GB	$0.06667	$4.00	~$2,880
H100 80 GB	$0.10833	$6.50	~$4,680
B200 180 GB	$0.16633	$9.98	~$7,185

Plans: Basic $0/month pay as you go (dedicated deployments, Model APIs, training); Pro adds priority access to high-demand GPUs, higher Model API rate limits and volume discounts; Enterprise adds custom SLAs, self-hosted deployments, use of existing cloud commitments, data-residency control and custom regions.

Decide: Baseten, stay, or go elsewhere

Move to Baseten: custom or fine-tuned models in a customer-facing product, need HIPAA/SOC 2, and no team to run GPU autoscaling.
Stay on SageMaker or Vertex AI: large prepaid commitments or strict single-cloud policy — see Baseten vs SageMaker vs Vertex AI.
Use a serverless catalog API instead: you call popular open models per token — see Baseten vs Together AI vs Fireworks AI.
Self-host: steady 24/7 load on GPUs you already own — see how to self-host an LLM for production.

For the company itself (funding, customers, risks), see the Baseten evaluation.

Last verified: October 17, 2026. Prices are Baseten list USD.