Ray vs Kubernetes vs Slurm for AI Workloads (2026)
The Short Answer
These three are constantly compared and only sometimes compete. They sit at different layers.
| Ray | Kubernetes | Slurm | |
|---|---|---|---|
| Schedules | Python tasks and actors | Containers / pods | Batch jobs on nodes |
| Layer | Distributed application | Infrastructure | Cluster batch queue |
| Best for | Distributed Python, RL, tuning, serving | Elastic services, platform ops | Large pre-training, HPC |
| Gang scheduling | ⚠️ Via placement groups | ⚠️ Needs Kueue / Volcano | ✅ Native |
| Elastic autoscaling | ✅ Built in | ✅ Excellent | ❌ Not the model |
| Multi-tenancy | ⚠️ Weak alone | ✅ Namespaces + RBAC | ✅ Fair-share accounting |
| Serving workloads | ✅ Ray Serve | ✅ Native | ❌ Poor fit |
| Learning curve | Low for Python devs | High | Moderate |
| Runs on the others? | ✅ On K8s (KubeRay) or Slurm | — | Can host Ray or containers |
Verified August 23, 2026.
Ray: The Distributed Python Layer
What it is: a framework for turning single-machine Python into distributed Python. Decorate a function, get a remote task. Define an actor, get stateful distributed compute.
Where it wins:
- Workloads with irregular parallelism. Hyperparameter search, reinforcement learning, evolutionary methods — anything where task shape and count are dynamic. This is Ray’s home ground and nothing else comes close for developer velocity.
- Batch inference over large datasets, where Ray Data handles the pipeline and backpressure.
- Serving with per-model scaling, through Ray Serve.
- Teams whose infrastructure expertise is Python, not YAML. A researcher can distribute a workload in an afternoon.
Where it struggles:
- Multi-tenancy. Ray clusters are most naturally single-tenant. Sharing one across teams means sharing a failure domain and a security domain.
- Very large synchronous training runs. Ray can do it, but tightly-coupled multi-node training with collective communication is territory where Slurm plus a dedicated training framework is still the sharper tool.
- Security defaults. Historically deployed unauthenticated on trusted-network assumptions — the assumption that broke publicly when CISA added CVE-2025-62593 to its KEV catalog on August 17, 2026 with a three-day federal remediation deadline. Ray supports token authentication and KubeRay v1.5.1+ can configure it; enable it.
Kubernetes: The Infrastructure Layer
What it is: a container orchestrator. It manages nodes, schedules pods, handles rollouts, autoscaling, networking and secrets.
Where it wins:
- Anything that must stay up. Inference services, APIs, pipelines with SLAs.
- Elastic GPU capacity, scaling node pools up and down against demand.
- Multi-team platforms. Namespaces, RBAC, quotas and network policies give you real isolation, which Ray and Slurm do not provide on their own.
- Operational consistency, because it is probably already running the rest of your infrastructure.
Where it struggles:
- Batch and gang scheduling out of the box. The default scheduler will happily place three of your four workers and leave the job stuck holding GPUs. Fix this with Kueue or Volcano — for a GPU platform this is not optional, it is the difference between a working queue and burnt money.
- Cost. GPU nodes idling between jobs are expensive, and Kubernetes autoscaling reacts on timescales that do not always match training patterns.
- Complexity for researchers. Asking a scientist to write pod specs to run an experiment is a tax you will pay in adoption.
Slurm: The Batch Layer That Never Left
What it is: the workload manager that has run supercomputers for two decades. Submit a job to a queue, it runs when resources free up.
Where it wins:
- Large pre-training runs. Native gang scheduling, topology-aware placement, and a model built precisely for “give me 512 GPUs for 20 hours.”
- Fair-share accounting. When multiple teams contend for a fixed expensive cluster, Slurm’s accounting and priority machinery is mature in a way most Kubernetes setups are not.
- Existing HPC environments. If you have an on-prem GPU cluster and an HPC team, Slurm is already there and already understood.
Where it struggles:
- Serving. Slurm is for jobs that finish. Long-lived endpoints are not its model.
- Elastic cloud capacity. It assumes a mostly-fixed resource pool.
- Container-native workflows, though Pyxis and Enroot have made this much better than it was.
The Combinations That Actually Ship
Almost nobody runs one of these alone at scale.
Ray on Kubernetes (KubeRay) — the mainstream choice for cloud-native ML platforms. Kubernetes owns nodes, autoscaling, isolation and secrets; Ray owns the distributed program. Add Kueue for queueing across teams. This gives researchers Ray’s ergonomics and gives platform teams Kubernetes’ controls.
Slurm for training, Kubernetes for serving — extremely common at organisations with both an HPC heritage and a product. Big periodic training runs go to the Slurm cluster; the resulting models are served from Kubernetes. Two systems, two clean fits, one artifact handoff between them.
Ray on Slurm — used where the cluster is Slurm and the workload is irregular. Allocate nodes with Slurm, start a Ray cluster inside the allocation, run Ray code. Slightly awkward, entirely workable.
How to Choose
Work through these in order:
- Do you need to serve models with an SLA? → You need Kubernetes somewhere in the stack.
- Is your dominant workload very large synchronous training on fixed hardware? → Slurm, and do not fight it.
- Is your dominant workload irregular Python parallelism — tuning, RL, batch inference? → Ray, on top of whichever of the other two you already run.
- Do multiple teams share expensive GPUs? → You need real queueing: Slurm natively, or Kubernetes with Kueue or Volcano. Do not attempt this with the default Kubernetes scheduler.
- Is your team Python-first with no platform engineers? → Start with Ray on a managed Kubernetes service and defer the rest.
The Security Note That Applies to All Three
Every one of these has a control plane that can execute code across a cluster, and all three get deployed unauthenticated on trusted-network assumptions.
The August 17, 2026 KEV listing for Ray was the loudest reminder yet, but it is not Ray-specific advice. Regardless of which you pick:
- Authenticate the control plane. Ray token auth, Kubernetes RBAC with no anonymous access, Slurm with proper munge configuration.
- Keep dashboards off routable interfaces. Loopback plus SSH forwarding or an authenticated proxy with
Hostheader validation. - Scope node IAM roles tightly. A training node rarely needs permission to launch instances.
- Patch on a real SLA. These are production systems, not lab tools, whatever their origin story says.