How do I run a local LLM on Windows?

Install one of three free tools, download a model and chat with it or call it over a local API. As of October 2026: Ollama for a background service with a command line and an API on localhost:11434; LM Studio for a desktop app with a model browser; Microsoft Foundry Local for .NET developers who want to ship a model inside a Windows app using CPU, GPU or NPU.

Ollama vs LM Studio vs Foundry Local: which should I use on Windows?

Ollama if you want a lightweight service other apps can call; it runs on Windows 10 22H2 or newer with NVIDIA or AMD Radeon GPUs. LM Studio if you prefer a graphical app; it supports x64 and Snapdragon ARM PCs and needs AVX2 on x64. Foundry Local if you are building a Windows 11 (24H2 or later) app and want models that run on NPUs as well as GPUs and CPUs.

How much RAM do I need to run an LLM locally on a PC?

LM Studio recommends at least 16 GB of RAM and 4 GB of dedicated VRAM. As a rule of thumb, a 4-bit model needs about 0.6 GB of memory per billion parameters plus room for context: a 4B-active MoE such as Gemma 4 26B A4B or a dense 27B model such as Qwen 3.8 27B wants roughly 16 to 20 GB, while 3B to 8B models run on 8 to 16 GB machines.

Can I run local LLMs on Windows without a GPU?

Yes. All three tools fall back to the CPU, and Foundry Local states that a dedicated GPU, NPU or Copilot+ PC is not required when the model has a CPU variant. Expect a few tokens per second on CPU with small models; a GPU with enough VRAM to hold the whole model is what makes local chat feel fast.

Quick Answer

How to Run Local LLMs on Windows in 2026: 3 Tools Compared

Published: October 13, 2026

The short answer

To run a local LLM on Windows, install Ollama, LM Studio or Foundry Local, download a model that fits your memory, and use it through the tool’s chat window or its local API. As of October 13, 2026:

	Ollama	LM Studio	Foundry Local
Best for	A background service other apps call	A desktop app with a model browser	Shipping a model inside your own Windows app
Windows version	Windows 10 22H2 or newer	Windows x64 or ARM (Snapdragon X)	Windows 11 24H2 (build 26100) or newer
Hardware	NVIDIA (driver 551.61+), AMD Radeon via ROCm v7 or Vulkan, or CPU	AVX2 CPU on x64; 16 GB RAM and 4 GB VRAM recommended	CPU, GPU or NPU model variants; no Copilot+ PC required
Interface	CLI + API on `localhost:11434`	GUI, local server, MCP servers, chat with documents	.NET SDK in-process; optional CLI and REST server
Admin rights	Not required	—	Not required

All three are free. For Macs, see our best local LLMs for Mac guide.

Step 1: Check what your PC can hold

Open Task Manager → Performance and note system memory and the GPU’s dedicated memory. The model has to fit, with headroom for context:

8 GB RAM, no GPU: 3B–4B models at 4-bit. Usable for drafting and summaries.
16 GB RAM or 8 GB VRAM: 7B–14B models, or small mixture-of-experts models such as JetBrains’ new Mellum2.1 (12B total, 2.5B active).
24 GB+ VRAM or 32 GB+ RAM: the 26B–27B class — Google’s Gemma 4 26B A4B (Apache 2.0, 4B active parameters) or Alibaba’s Qwen 3.8 27B (Apache 2.0, dense). These are the strongest models most single-GPU PCs can run well in October 2026.

Rule of thumb: about 0.6 GB per billion parameters at 4-bit, plus 1–4 GB for context. If a model spills from VRAM into system RAM, speed drops sharply.

Step 2: Pick the tool

Choose Ollama if you want something that runs quietly and serves other software — Open WebUI, coding agents, scripts. It installs into your home folder without admin rights, needs about 4 GB for the app, and serves its API at http://localhost:11434. Models can take tens to hundreds of gigabytes; install to another drive with OllamaSetup.exe /DIR="d:\some\location" and move model storage with the OLLAMA_MODELS environment variable.

Choose LM Studio if you want to browse, download and compare models in a window. It runs on x64 and ARM Windows PCs, requires AVX2 on x64 CPUs, supports MCP servers and document chat, and can expose a local server for other apps.

Choose Foundry Local if you are a developer building a Windows app. Add the Microsoft.AI.Foundry.Local.WinML NuGet package (.NET 9 SDK or later), and the model runs inside your app’s process — no separate server. It picks CPU, GPU or NPU variants for the hardware it finds, which makes it the only one of the three built around Copilot+ PC NPUs.

Step 3: Install and run a first model

Ollama: install, open PowerShell and run ollama run <model> — it downloads the model on first use and opens a chat. ollama list shows what is installed.

LM Studio: install, use the search tab to pick a model (it shows whether the download fits your hardware), load it, chat.

Foundry Local: follow Microsoft’s get-started guide to add the package, choose a model from the catalog in code, and call it from your app.

Step 4: Connect your apps

Ollama and LM Studio both offer OpenAI-style local endpoints, so most chat front-ends and coding tools can point at them by changing the base URL. Compare front-ends in Open WebUI vs AnythingLLM vs LibreChat, and serving engines for multi-user setups in vLLM vs Ollama vs LM Studio.

Troubleshooting

Slow on an AMD card: some RX 6000-series GPUs do not expose ROCm v7 on current Windows drivers; Ollama falls back to Vulkan, which is enabled by default.
Wrong GPU picked on a laptop: set GGML_VK_VISIBLE_DEVICES to the discrete GPU’s index in Ollama.
Out of memory: use a smaller quantization or shorten the context length before buying hardware.

Last verified: October 13, 2026.