What are the best on-device speech-to-text models for edge AI assistants?

As of October 26, 2026: Moonshine Voice for most assistants (MIT, streaming, runs on iOS, Android, desktop, browsers and Raspberry Pi with one library); NVIDIA Parakeet Realtime EOU 120M for English voice agents that need 80–160 ms latency and end-of-turn detection; Mistral Voxtral Realtime (4B, Apache 2.0) for multilingual live transcription when a GPU is available; Whisper large-v3-turbo via whisper.cpp for the widest language coverage; and Cactus Whistle (16.9 MB) for wearables and microcontrollers.

What is an on-device alternative to Whisper with higher accuracy?

Moonshine Voice: its README states that its speech-to-text models range from higher accuracy than Whisper large-v3 on the Hugging Face Open ASR Leaderboard down to tiny 1 MB models, and unlike Whisper they stream, doing work while the user is still talking. For multilingual streaming, Mistral's Voxtral Realtime reports offline-level quality at sub-second delay. Check the leaderboard yourself for your language; vendor averages use different test sets.

Why does an AI assistant need streaming speech recognition?

Latency. Whisper processes audio in 30-second windows, so the assistant cannot start thinking until the user stops and the window is processed. Streaming models emit text as the user speaks, and models with end-of-utterance detection, such as Parakeet Realtime EOU, tell the assistant when the user has finished so it can reply without a fixed silence timeout.

Can speech-to-text run on a microcontroller?

Small ones can. Cactus Whistle, released October 2, 2026, is a single 16.9 MB file that runs on the CPU with no dependencies in seven languages, and Moonshine publishes micro models down to about 1 MB. Expect lower accuracy than laptop-class models and short utterances rather than long dictation.

Quick Answer

Best On-Device Speech-to-Text Models for Edge AI Assistants

Published: October 26, 2026

The short answer

An edge AI assistant needs speech-to-text that streams, detects when the user has stopped talking, and fits the device. Batch accuracy matters less than time-to-first-word. As of October 26, 2026:

Moonshine Voice — best default for on-device assistants across phones, desktops, browsers and Raspberry Pi.
NVIDIA Parakeet Realtime EOU 120M — best for English voice agents: 80–160 ms latency plus an end-of-utterance token.
Mistral Voxtral Realtime — best multilingual streaming model when you have a GPU.
Whisper large-v3-turbo (whisper.cpp) — best language coverage; not streaming.
Cactus Whistle — best for wearables and microcontrollers at 16.9 MB.

For batch transcription accuracy rather than assistant latency, see our ranking of local speech-to-text models.

The comparison

Model	Size	Languages	Streaming	End-of-turn	Licence	Best for
Moonshine Voice	~1 MB micro models up to large	English plus others (legacy non-English models non-commercial)	✅	Via its voice-agent toolkit	MIT	Cross-platform assistants
Parakeet Realtime EOU 120M	120M	English, no punctuation	✅ 80–160 ms	✅ `<EOU>` token	NVIDIA Open Model License	English voice agents
Voxtral Realtime	4B	Multilingual	✅ delay configurable	❌	Apache 2.0	Multilingual live captions, GPU
Whisper large-v3-turbo	809M	~99	❌ 30 s windows	❌	MIT	Rare languages, ecosystem
Cactus Whistle	16.9 MB	7 (EN, DE, FR, ES, IT, NL, PL)	❌ 30 s pass	❌	Open weights	Wearables, MCUs, smart home

1. Moonshine Voice — the default

Moonshine Voice is an open-source toolkit for real-time voice agents: speech-to-text, intent recognition and text-to-speech, all on-device with no account or API key. Its models are trained from scratch for live streaming and “do work while the user is still talking”, so latency does not balloon to Whisper’s 30-second window. The project says its range spans models more accurate than Whisper large-v3 on the Open ASR Leaderboard down to 1 MB micro models, and one library covers Python, JavaScript/WASM, iOS, Android, macOS, Linux, Windows and Raspberry Pi. Code and models are MIT, except legacy non-streaming models for non-English languages, which stay under a non-commercial licence — check before you ship in another language. Install: pip install moonshine-voice.

2. Parakeet Realtime EOU 120M — English voice agents

NVIDIA’s 120M-parameter streaming FastConformer-RNNT model does two jobs: transcription at 80–160 ms latency and end-of-utterance detection, signalled by an <EOU> token. That second job is what makes an assistant feel responsive: it can reply the moment the user finishes instead of waiting out a silence timer. It is English-only and outputs no punctuation or capitalisation, so use it for the live turn and, if you need a clean transcript, re-run the audio through a larger model afterwards. Licensed for commercial use under the NVIDIA Open Model License.

3. Voxtral Realtime — multilingual streaming on a GPU

Mistral released Voxtral Realtime on February 4, 2026 as open weights under Apache 2.0 (Voxtral-Mini-4B-Realtime-2602). Its paper reports performance competitive with Whisper and ElevenLabs Scribe v2 Realtime at a 480 ms delay, and offline-level quality at about 960 ms. At 4B parameters it wants a GPU or a strong Apple-silicon machine, which makes it a fit for in-car systems, kiosks and laptops rather than phones.

4. Whisper large-v3-turbo — when you need the language

Whisper still covers about 99 languages, and whisper.cpp runs large-v3-turbo on ordinary laptops and phones. It does not stream: audio is processed in 30-second windows, so for an assistant you must chunk with voice-activity detection and accept the delay. Choose it when your users speak a language the streaming models do not cover.

5. Cactus Whistle — the smallest useful model

Whistle, released October 2, 2026, is one 16.9 MB file that runs on the CPU with no dependencies, in seven European languages, with word timings, keyword biasing and speech embeddings. It shares a C++ engine with Cactus’s Needle tool-calling model, so a single binary can turn a spoken command into a function call — exactly the job of a smart-home or wearable assistant. Its limits are a 30-second single pass (windowed beyond that) and seven languages. Full comparison: Whistle vs Whisper Base vs Moonshine Tiny.

How to choose

Phone or desktop assistant, English-first: Moonshine Voice.
English voice agent where turn-taking speed is everything: Parakeet Realtime EOU for the live turn.
Multilingual, GPU available: Voxtral Realtime.
Language outside the streaming models: Whisper large-v3-turbo with VAD chunking.
Under 20 MB, voice commands to tool calls: Whistle.
Laptop app choosing between local and cloud accuracy: test on your own audio; our hardware-based STT stack guide and best speech-to-text APIs cover both sides.

Last verified: October 26, 2026.