AI agents · OpenClaw · self-hosting · automation

Quick Answer

Best Local Coding Model for a 32GB Mac or GPU in 2026

Published:

The short answer

For agentic coding on a 32GB Mac or a 32GB GPU, start with Inkling 2 Mini; keep Qwen3.8-27B as the proven alternative; skip Gemma 4 31B for agent work. Inkling 2 Mini, released November 20, 2026, leads the small open models on agentic coding in its vendor’s benchmarks and, as a mixture-of-experts model, decodes faster than a dense 27B. Qwen3.8-27B has three months of community tooling, quantizations and harness support behind it. Facts and scores were read from model cards on November 21, 2026.

The comparison

Inkling 2 MiniQwen3.8-27BGemma 4 31B
Maker / releasedThinking Machines, Nov 20, 2026Alibaba Qwen, Aug 2026Google, 2026
LicenceApache 2.0Apache 2.0Apache 2.0
TypeLooped MoE, 26.4B total / 3.4B activeDense 27BDense 31B
Memory to run~20GB native checkpoint~16–18GB at 4-bit~18–20GB at 4-bit
ContextUp to 1M262K native, extendable to 1M256K
DeepSWE v1.148.2%42.2%0.3%
SWE-bench Verified (bash only)79.2%77.8%46.1%
Terminal-Bench 2.171.6%73.0%43.4%
Images inYesYes (image and video)Yes
Pick it forAgentic coding, speedReliability, ecosystemGeneral chat, writing

Benchmark figures come from Thinking Machines’ November 20, 2026 table and are vendor-reported; setups differ by model.

Picks in detail

Inkling 2 Mini — best for agents. It loops its middle layers three times per token, which lifts reasoning on multi-step coding without adding stored parameters. The checkpoint is trained and shipped in 4-bit NVFP4, so the ~20GB file is the full-quality model, not a lossy quantization. Run it with ollama run inkling-2-mini, or vLLM and SGLang on NVIDIA. It is one day old: expect harness quirks, and check that your agent’s tool-call format works before relying on it. Details in what is Inkling 2 Mini.

Qwen3.8-27B — best all-rounder. Dense models are simpler to quantize and run everywhere, and Qwen’s tool-calling is well supported across Ollama, LM Studio, llama.cpp and agent harnesses. It edges Inkling on Terminal-Bench 2.1 and on factual recall. The cost is speed: every token reads all 27B weights, so it decodes noticeably slower on the same Mac.

Gemma 4 31B — not for coding agents. A capable general model with a 256K context and a permissive licence, but it scores near zero on DeepSWE in the same table. Use it for writing and chat, not autonomous edits.

How to set up a local coding stack

  1. Runtime: Ollama or LM Studio on a Mac; vLLM or llama.cpp on NVIDIA.
  2. Agent: OpenCode, Cline or Kilo Code pointed at the local OpenAI-compatible endpoint — see best open-source Claude Code alternatives.
  3. Context budget: start at 32K tokens; raise it only if memory allows, because the KV cache grows with context.
  4. Fallback: route hard tasks to an API model; Claude Sonnet 5.5 is $2/$10 per million tokens on our current API prices table.

Hardware first? See how much VRAM you need for a local LLM and best Mac for running local LLMs. Full walkthrough: how to run coding agents locally.

Last verified: November 21, 2026.

Sources