Best Multimodal Embeddings for Visual Document Retrieval
The short answer
For PDFs full of tables, charts and scans, embed the page image, not the extracted text — and use a late-interaction (multi-vector) model. As of October 12, 2026:
- pplx-embed-v2-late-9b (Perplexity) — best quality with a permissive licence: MIT, 7.4B active parameters, 65.2% nDCG@10 on public ViDoRe v3 image retrieval (64.7% on the Markdown version).
- pplx-embed-v2-late-0.6b (Perplexity) — best for cost: 340M active parameters, 62.3% on the same benchmark, and it shares the 9B model’s embedding space, so it can query a 9B-built index.
- ColQwen2.5 v0.2 (ViDoRe team) — the established baseline: MIT, built on Qwen2.5-VL-3B, widely supported in
colpali-engineand vector databases. - ColNomic Embed Multimodal 7B (Nomic) — the Apache-2.0 option.
- Jina Embeddings v4 — single-vector and multi-vector output from one model, 32K tokens, 30+ languages, but under the Qwen Research License — check terms before commercial use.
Perplexity published pplx-embed-v2-late on October 7, 2026, with weights on Hugging Face.
The comparison
| Model | Licence | Size (active) | Vectors | Notable | Pick for |
|---|---|---|---|---|---|
| pplx-embed-v2-late-9b | MIT | 7.4B | 128-dim per token, MaxSim | ViDoRe v3 image 65.2%; distilled from an internal 18B ColBERT teacher | Best quality, commercial use |
| pplx-embed-v2-late-0.6b | MIT | 340M | 128-dim per token | 62.3%; same space as 9B | Cheap queries, edge |
| ColQwen2.5 v0.2 | MIT | ~3B (Qwen2.5-VL-3B base) | Multi-vector | Up to 768 image patches; mature tooling | Proven default, small GPU |
| ColNomic Embed Multimodal 7B | Apache-2.0 | 7B | Multi-vector | Apache licence | Teams that need Apache-2.0 |
| Jina Embeddings v4 | Qwen Research License | ~3B (Qwen2.5-VL-3B base) | 2048-dim single vector (Matryoshka to 128) or 128-dim multi-vector | 32,768 tokens, task adapters | Research, or one model for both modes |
Scores are the vendor’s own reported numbers; they are only comparable within the same benchmark version.
Picks by situation
A commercial RAG product over visually rich documents: pplx-embed-v2-late. Index with the 9B model offline, where cost matters less, and query with the 0.6B model online. Perplexity’s model card notes two practical details: use separate encoding calls for text and image batches (mixed text+image inputs are not supported), and it needs sentence-transformers 6.0.0+ and transformers 5.4.0+.
A small team on one modest GPU: ColQwen2.5. Three billion parameters, MIT licence, and the most integrations; it is the model most tutorials and vector-database examples use.
Legal or procurement requires Apache-2.0: ColNomic Embed Multimodal 7B.
You want a single vector to keep the index small: Jina Embeddings v4 in dense mode (2048 dimensions, truncatable to 128) — if its licence fits. Otherwise, pair a text embedding model with OCR; see best embedding models.
What late interaction costs you
Multi-vector indexes are much larger than single-vector ones: every page stores hundreds of 128-dimension vectors instead of one. Plan for it:
- Use a store with native multi-vector support (MaxSim) — see best vector databases.
- Retrieve in two stages: a cheap single-vector or BM25 pass for candidates, then late-interaction scoring on the top few hundred.
- Pool or quantize patch vectors if storage dominates; test the recall loss on your own pages.
- Keep extracted text alongside so answers can quote the page — a PDF parser for RAG still matters for generation.
Last verified: October 12, 2026.