Can you really run a 27B model on a 16 GB laptop in 2026?

Yes. PrismML's Ternary Bonsai 2 27B, released September 17, 2026, stores every language weight as −1, 0 or +1 with FP16 group scales, so the whole 27.36B-parameter model is 5.95 GB on disk versus ~54 GB in FP16. That leaves room on a 16 GB machine for a 32K–64K context. It retains 98.2% of full-precision Qwen3.8 27B's aggregate benchmark score, runs at about 47 tokens per second on an Apple M5 Max and 120–130 tokens per second on an RTX 5090, and is Apache 2.0 licensed.

Does Ternary Bonsai 2 work with stock llama.cpp or Ollama?

No, not as of September 20, 2026. The PTQ1_0 and PQ2_0 GGUF packings need PrismML's llama.cpp fork, which has the ternary hybrid-attention kernels and the Hadamard activation transform. Stock llama.cpp rejects PTQ1_0 and PQ2_0 as unknown types, and the model card warns that a Q2_0 file loads silently and produces garbage. Use a prebuilt release from github.com/PrismML-Eng/llama.cpp, or the MLX fork on Apple Silicon. Ollama and LM Studio bundle stock llama.cpp, so they cannot run these files until the kernels are upstreamed.

Which file should I download: PTQ1_0 or PQ2_0?

PTQ1_0 packs trits densely at 1.75 bits per weight (5.95 GB) and is the pick when memory is tightest or on Ada-generation NVIDIA cards and the L4. PQ2_0 stores each trit in a 2-bit slot (2.13 bits per weight, 7.21 GB), decodes faster on H100, A100 and Blackwell cards, processes prompts faster everywhere, and is the pack PrismML measured on Apple Silicon. On a 16 GB laptop either fits; start with PQ2_0 for speed and drop to PTQ1_0 if you need a longer context.

How much quality do you lose versus the full model?

PrismML reports 83.9 versus 85.4 aggregate in thinking mode (98.2% retained) across reasoning, math, coding, instruction following, vision and agentic tool use. Math is within half a point (96.57 vs 97.06), coding 81.58 vs 82.17, agentic tool calling 77.57 vs 79.74; instruction following is actually higher (82.66 vs 81.25). The clearest gap is knowledge and reasoning (83.95 vs 86.66) and vision (78.59 vs 81.64). For comparison, a conventional IQ2_XXS build of the same model scores 72.59 on PrismML's 14-benchmark set against 84.78 for Bonsai 2.

Quick Answer

How to Run a 27B LLM on a 16GB Laptop (2026 Guide)

Published: September 20, 2026

Why this is possible now

Until 2026 the honest answer to “can I run a 27B model on a 16 GB laptop” was “only at 2-bit, and it will be noticeably dumber.” Conventional 2-bit quantisation of a 27B model collapses on reasoning, tool use and code.

Ternary Bonsai 2 27B changed that on September 17, 2026. PrismML — a Caltech spin-out backed by Khosla Ventures, Cerberus, Google and Samsung — rewrote Qwen3.8 27B so that every weight in the embeddings, attention projections, MLP projections and LM head is −1, 0 or +1, with one FP16 scale per 128 weights and a blockwise Hadamard rotation folded into the stored weights. The result is 1.72 bits per weight, a 5.95 GB language model (down from ~54 GB), and 98.2% of the full-precision aggregate score — 83.9 versus 85.4 in thinking mode across a suite spanning reasoning, math, coding, instruction following, vision and agentic tool use.

It keeps the base model’s 262K-token context (practical on-device because Qwen3.8 27B’s backbone is ~75% linear attention), supports image input via a separate 0.63 GB Q8_0 vision pack, and ships under Apache 2.0. This guide gets it running on a Mac with 16 GB of unified memory or a PC with a single consumer GPU.

Verified September 20, 2026 against the Hugging Face model card and PrismML’s announcement. Where this guide and the Bonsai-demo repository disagree, the repository is right — it pins known-good binaries and is updated as the kernels change.

Step 1 — Pick your hardware path

Machine	Backend	Expected speed	Fits?
MacBook / Mac mini, 16 GB unified, M3/M4/M5	PrismML llama.cpp fork (Metal) or MLX fork	~47 tok/s on M5 Max; lower on base chips	Yes, with 32K–64K context
Windows/Linux laptop, RTX 4060/4070 8–12 GB	llama.cpp fork (CUDA), PTQ1_0	Tens of tok/s; full offload at 8 GB with PTQ1_0	Yes, shorter context on 8 GB
Desktop RTX 4090 / 5090	llama.cpp fork (CUDA), PQ2_0	~130 tok/s on RTX 5090, ~120 with PTQ1_0	Yes, long context
CPU only, 16 GB RAM	llama.cpp fork (CPU)	Usable but slow	Yes, for batch jobs

Memory maths on a 16 GB Mac: 5.95 GB (PTQ1_0) or 7.21 GB (PQ2_0) of weights, plus KV cache, plus macOS. The linear-attention backbone keeps KV cache small, so 32K context is comfortable and 64K is realistic. Do not expect the full 262K on 16 GB.

Step 2 — Get the PrismML llama.cpp fork (stock llama.cpp will not work)

The ternary kernels are not upstream. Stock llama.cpp rejects PQ2_0 and PTQ1_0 as unknown types, and loads a Q2_0 file without warning and produces garbage because it lacks the Hadamard activation runtime. Ollama and LM Studio embed stock llama.cpp, so they cannot run these files yet either.

Prebuilt (recommended):

# download the archive for your platform from
# https://github.com/PrismML-Eng/llama.cpp/releases/latest
mkdir -p bin
tar -xzf llama-<tag>-bin-<platform>.tar.gz -C bin --strip-components=1

Or build it:

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j   # drop -DGGML_CUDA=ON on macOS; Metal is default

The binary is ./bin/llama-cli from the archive or ./build/bin/llama-cli if you built it.

Step 3 — Download the weights

pip install -U huggingface_hub
hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir .
# memory-tightest option instead:
# hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .

The repository had passed 1.5 million downloads by September 20, 2026. Apple Silicon users who prefer MLX can use prism-ml/Ternary-Bonsai-2-27B-mlx-2bit with PrismML’s MLX fork (and the mlx-swift fork for iOS/macOS apps).

Step 4 — Run it

./bin/llama-cli -m Ternary-Bonsai-2-27B-PQ2_0.gguf \
  -ngl 99 -fa on -c 32768 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain quantum computing in simple terms." -n 256

-ngl 99 offloads every layer to the GPU (or Metal); 0 is CPU-only.
-c is context length, up to 262144. Start at 32768 on 16 GB.
The sampling values are PrismML’s recommended defaults for this model.
It is a reasoning model and thinks by default. Expect a thinking block before the answer; use the reasoning-budget flags documented in Bonsai-demo to cap it.

For an OpenAI-compatible endpoint, tool calling, image input (load the mmproj pack) and speculative decoding, use the run scripts in PrismML-Eng/Bonsai-demo; they choose the right flags for your hardware and expose llama-server.

Step 5 — Point a coding agent at it

PrismML’s launch demo ran Cline against Bonsai 2 27B on an RTX 5090. Any agent that speaks the OpenAI chat API can use llama-server from the fork the same way — set the base URL to http://localhost:8080/v1 and the model name to whatever you served. Agentic tool calling scores 77.57 (vs 79.74 full precision), which is enough for edit-test loops on a single repository; for the harness side see how to run coding agents locally.

What to expect on quality

Capability	Bonsai 2 27B	Qwen3.8 27B (FP16)
Math (AIME 2025/2026, GSM8K, MATH-500)	96.57	97.06
Coding (HumanEval+, LiveCodeBench v6, MBPP+, BigCodeBench)	81.58	82.17
Agentic & tool calling (τ²-bench, BFCLv3)	77.57	79.74
Instruction following (IFBench, IFEval)	82.66	81.25
Knowledge & reasoning (MMLU-Redux, GPQA Diamond, AA-LCR)	83.95	86.66
Vision (CharXiv, OCRBench v2, …)	78.59	81.64
Overall	83.9	85.4

The pattern: math and coding are almost free; broad knowledge and vision pay the most. PrismML’s 14-benchmark thinking-mode set puts Bonsai 2 at 84.78 versus 72.59 for a conventional IQ2_XXS build at a larger footprint — that 12-point gap is the whole reason to use this format rather than a generic 2-bit GGUF.

Troubleshooting

“unknown tensor type” — you are on stock llama.cpp. Use the fork.
Nonsense output — you loaded a Q2_0 file into stock llama.cpp. Same fix.
Out of memory on 16 GB — switch to PTQ1_0, lower -c, and quit memory-heavy apps; on Macs check sudo sysctl iogpu.wired_limit_mb if Metal refuses to allocate.
Slow prompt processing on Ada cards — PTQ1_0 decodes faster on the RTX 40-series and L4; PQ2_0 wins on Blackwell, H100 and Apple Silicon.

Alternatives if this does not fit your stack

Need Ollama/LM Studio today: run a Q4 build of a 14B–20B model instead (see best small LLMs for a single GPU).
Need the full 27B in Q4: Qwen3.8 27B UD-Q4_K_XL scores within 0.4 points of Bonsai 2 but needs roughly three times the memory — a 32 GB+ machine.
Hardware shopping: best local AI agent hardware 2026.