How to Run Coding Agents Locally: 2026 Setup Guide
The Short Answer
Running a coding agent locally in 2026 takes five steps: check memory → install a runtime → pull a model → connect a harness → sandbox it. Budget an afternoon.
Set expectations first. A local 27B-class model is good at boilerplate, single-file refactors, test generation and routine edits. It is meaningfully worse than a frontier model at multi-file reasoning and long-horizon planning. Build this for privacy and unlimited retries, not as a Claude Code replacement.
Step 1 — Check You Have Enough Memory
24GB of VRAM or unified memory is the floor. That is the minimum GeForce RTX requirement Perplexity set for Portable Computer, and roughly what a 27B model at sensible quantisation needs with usable context left over.
| Your hardware | Verdict |
|---|---|
| < 16GB | Too small — small models only, poor tool use |
| 24GB | Workable — 27B-class fits, context is tight |
| 48–64GB | Comfortable — the sweet spot |
| 128GB+ | Large models and long context |
Completion criterion: you can state your available VRAM or unified memory in GB, and it is at least 24.
Agents are more memory-hungry than chat because they hold long context — file contents, tool outputs, accumulated reasoning — across many steps. Size for the context, not just the weights.
Step 2 — Install a Model Runtime
Two mature options, both free:
Ollama — CLI-first, widest model catalogue, scriptable, runs on macOS, Linux and Windows across NVIDIA, AMD and Apple Silicon. Pick this if you want control.
LM Studio — GUI, easy model discovery, and critically it exposes an OpenAI-compatible local server. Pick this if you want to point existing tools at local weights without rewriting them.
The OpenAI-compatible endpoint is the important detail. Most agent harnesses speak the OpenAI API, so a local server that mimics it is what lets you swap the backend without touching the harness.
Completion criterion: you can send a prompt and receive a response from a locally-served model.
Step 3 — Pull a Coding Model
Choose a 27B-class model if you are at the 24GB floor. Verified options in the current open-weight field include Qwen 3.8 27B (which Perplexity ships in Portable Computer alongside its own post-trained PPLX 27B), GLM-5.2/5.3-class open releases, and Nemotron 3.5 Lightning, which Perplexity has announced as coming to Portable Computer.
Two rules that will save you time:
- Prefer models with explicit tool-calling training. An agent is a tool-calling loop. A model that was not trained to emit structured tool calls will fail in ways that look like reasoning failures but are formatting failures.
- Re-check the field before committing. Open-weight models turn over roughly monthly. Any specific recommendation — including this one — ages fast. Verify what is current when you set up.
Completion criterion: your model returns a correctly-formatted tool call when given a tool schema.
Step 4 — Connect an Agent Harness
The harness is the loop: it takes a task, asks the model what to do, executes tool calls, feeds results back, repeats.
Option A — bundled. Perplexity Portable Computer (launched August 25, 2026 with NVIDIA) ships the harness, orchestrator, post-trained models and an OS-enforced sandbox as one install. Requires NVIDIA hardware on Linux — DGX Spark, RTX PRO, or GeForce RTX with 24GB+ — and a Perplexity Pro or Max subscription. Windows RTX support is stated as coming. No macOS.
Option B — bring your own. Point an existing agent framework at your local OpenAI-compatible endpoint. More work, but cross-platform, free, and you keep full model choice.
Completion criterion: the agent completes a trivial two-step task — read a file, then report something about it — without manual intervention.
Step 5 — Sandbox It Before Granting Write Access
Do not skip this. A coding agent runs shell commands and writes files, and a local model plans worse than a frontier model. The failure mode is not hypothetical.
Minimum controls:
- Filesystem scope — write access to the project directory only. Never the home directory.
- Container or OS-enforced sandbox — Portable Computer provides one; otherwise run in a container.
- Read-only first — run several real tasks with writes disabled and read the proposed diffs.
- Confirm shell commands — require approval for execution until the loop has earned trust.
- Version control — work on a branch, commit before every agent run.
gitis the real undo button.
Completion criterion: you have verified the agent cannot write outside the project directory, by testing it.
Step 6 — Set Up Escalation
The reason to keep a cloud path is that local models fail on hard tasks, and you want that failure to be recoverable rather than terminal.
Route by difficulty: single-file edits, boilerplate, test scaffolding and formatting stay local at zero marginal cost. Multi-file refactors, architectural decisions and anything requiring a large codebase in context escalate to a frontier model.
For reference when escalating (verified August 2026, per 30K-in/5K-out task): Gemini 3.7 Flash ~$0.041, Claude Sonnet 5 ~$0.11, GPT-5.6 Sol ~$0.22, Claude Opus 5 ~$0.275.
Escalating a handful of hard tasks per day costs a few dollars a month. That is the cheapest possible insurance against the local tier’s ceiling.
Completion criterion: you have a defined rule for which tasks go where, and a working cloud fallback.
Common Mistakes
Expecting frontier quality. The gap is real and largest on the hardest tasks. Judge the local tier on routine work.
Under-sizing context. Agents accumulate context fast. A model that fits in memory with 4K context may thrash at 32K.
Skipping the sandbox. The single highest-consequence shortcut on this list.
Buying hardware first. Test on what you own. A $4,699 DGX Spark equals roughly 21,000 GPT-5.6 Sol calls — prove the workflow earns its keep before spending that.
Assuming a Mac runs everything. Ollama and LM Studio are excellent on Apple Silicon; CUDA-dependent harnesses, including Portable Computer, are not available there at all.