Best Local Coding Model for a 32GB Mac or GPU in 2026
The short answer
For agentic coding on a 32GB Mac or a 32GB GPU, start with Inkling 2 Mini; keep Qwen3.8-27B as the proven alternative; skip Gemma 4 31B for agent work. Inkling 2 Mini, released November 20, 2026, leads the small open models on agentic coding in its vendor’s benchmarks and, as a mixture-of-experts model, decodes faster than a dense 27B. Qwen3.8-27B has three months of community tooling, quantizations and harness support behind it. Facts and scores were read from model cards on November 21, 2026.
The comparison
| Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B | |
|---|---|---|---|
| Maker / released | Thinking Machines, Nov 20, 2026 | Alibaba Qwen, Aug 2026 | Google, 2026 |
| Licence | Apache 2.0 | Apache 2.0 | Apache 2.0 |
| Type | Looped MoE, 26.4B total / 3.4B active | Dense 27B | Dense 31B |
| Memory to run | ~20GB native checkpoint | ~16–18GB at 4-bit | ~18–20GB at 4-bit |
| Context | Up to 1M | 262K native, extendable to 1M | 256K |
| DeepSWE v1.1 | 48.2% | 42.2% | 0.3% |
| SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% |
| Terminal-Bench 2.1 | 71.6% | 73.0% | 43.4% |
| Images in | Yes | Yes (image and video) | Yes |
| Pick it for | Agentic coding, speed | Reliability, ecosystem | General chat, writing |
Benchmark figures come from Thinking Machines’ November 20, 2026 table and are vendor-reported; setups differ by model.
Picks in detail
Inkling 2 Mini — best for agents. It loops its middle layers three times per token, which lifts reasoning on multi-step coding without adding stored parameters. The checkpoint is trained and shipped in 4-bit NVFP4, so the ~20GB file is the full-quality model, not a lossy quantization. Run it with ollama run inkling-2-mini, or vLLM and SGLang on NVIDIA. It is one day old: expect harness quirks, and check that your agent’s tool-call format works before relying on it. Details in what is Inkling 2 Mini.
Qwen3.8-27B — best all-rounder. Dense models are simpler to quantize and run everywhere, and Qwen’s tool-calling is well supported across Ollama, LM Studio, llama.cpp and agent harnesses. It edges Inkling on Terminal-Bench 2.1 and on factual recall. The cost is speed: every token reads all 27B weights, so it decodes noticeably slower on the same Mac.
Gemma 4 31B — not for coding agents. A capable general model with a 256K context and a permissive licence, but it scores near zero on DeepSWE in the same table. Use it for writing and chat, not autonomous edits.
How to set up a local coding stack
- Runtime: Ollama or LM Studio on a Mac; vLLM or llama.cpp on NVIDIA.
- Agent: OpenCode, Cline or Kilo Code pointed at the local OpenAI-compatible endpoint — see best open-source Claude Code alternatives.
- Context budget: start at 32K tokens; raise it only if memory allows, because the KV cache grows with context.
- Fallback: route hard tasks to an API model; Claude Sonnet 5.5 is $2/$10 per million tokens on our current API prices table.
Hardware first? See how much VRAM you need for a local LLM and best Mac for running local LLMs. Full walkthrough: how to run coding agents locally.
Last verified: November 21, 2026.