TL;DR
olmOCR is the Allen Institute for AI’s open-source toolkit for converting PDFs, PNGs, and JPEGs into clean Markdown using a fine-tuned 7B vision-language model. It scores 82.4 on olmOCR-Bench, beating Marker (76.1), MinerU (75.2), and Mistral’s OCR API (72.0) — and it costs under $200 per million pages end-to-end on your own GPU. The repo just crossed 18,600 GitHub stars with 1,200 added this week, and every serious LLM-dataset builder I know has now tried it.
The pitch is straightforward: OCR isn’t really OCR anymore. If you want text that’s actually usable for LLM training, RAG, or fine-tuning, you don’t want a bounding-box detector — you want a small vision model that reads the page the way a human does. That’s what olmOCR is, packaged as a batch pipeline that will chew through millions of pages on a single node or a Kubernetes cluster.
Key facts:
- 18,653 GitHub stars, 1,216 added this week — top-trending Python repo
- v0.4.0 (October 2025) — a new RL-trained model (
olmOCR-2-7B-1025-FP8) that pushes olmOCR-Bench to 82.4 - Backed by Ai2 and the AllenNLP team — same lab that ships OLMo, Dolma, and Tulu
- Two ArXiv papers (2502.18443 and 2510.19817), one at ICML 2025
- Apache 2.0 license on all code, models, and training data
- Ships a full benchmark suite (olmOCR-Bench, 7,000+ test cases, 1,400 documents)
- Runs via
pip install olmocr[gpu]ordocker pull alleninstituteforai/olmocr:latest-with-model
Install (GPU): pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128. Try the hosted demo at olmocr.allenai.org before you commit to a 30GB Docker pull.
Why olmOCR exists
The pitch in the paper title is “Unlocking Trillions of Tokens in PDFs.” That’s not marketing. Somewhere between 40% and 60% of high-quality technical writing on the public internet — textbooks, government reports, court filings, scientific papers — is locked inside PDFs. Traditional OCR (Tesseract, ABBYY) can pull the text, but it can’t preserve reading order across multi-column layouts, it hallucinates on tables, and it turns math into gibberish.
The commercial answer is API-based OCR: Mistral OCR, AWS Textract, Azure Document Intelligence, Google Document AI. All of them work, all of them cost meaningful money per page, and none of them let you inspect or fine-tune the model.
The AI2 answer is: fine-tune a 7B vision-language model on 260,000 PDF pages, ship the weights, ship the training code, ship the benchmark. If your data is sensitive or your volume is high, you run it yourself. If you just want to try it, hit the online demo or a hosted inference provider.
The actual pipeline
The command-line surface is deceptively simple. The full pipeline runs one binary:
# Single PDF, local GPU
olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf
# Glob of PDFs
olmocr ./localworkspace --markdown --pdfs tests/gnarly_pdfs/*.pdf
# Millions of PDFs, S3-coordinated across N worker nodes
olmocr s3://bucket/workspace --pdfs s3://bucket/pdfs/*.pdf
# Remote inference (no local GPU needed)
olmocr ./localworkspace \
--server http://your-vllm:8000/v1 \
--model allenai/olmOCR-2-7B-1025-FP8 \
--markdown --pdfs *.pdfUnder the hood: poppler-utils rasterizes each PDF page to an image, the pipeline crops that image plus an anchor-text hint from the raw PDF, sends both to a vLLM server hosting olmOCR-2-7B-1025-FP8, and streams the model’s output back into two files — a Dolma-format JSONL (structured for LLM training) and a plain Markdown file preserving your input folder structure.
The two things that matter for real workloads:
--serverdecouples inference from ingestion. You can run 40 CPU-only ingestion workers pushing pages to one shared vLLM box with two H100s. That’s the shape production data-prep pipelines actually take.- S3 workspaces are a shared work queue. Start
olmocr s3://bucket/workspace ...on the first machine to seed the queue, then run the same binary without--pdfson every other node. They’ll cooperatively drain the queue. This is how AI2 processes millions of pages on Beaker with--beaker --beaker_gpus 4.
There’s also a Docker image with the model baked in (~30GB), which is the right choice if you’re spinning up ephemeral GPU boxes on RunPod, Modal, or Lambda:
docker pull alleninstituteforai/olmocr:latest-with-model
docker run --gpus all \
-v $(pwd):/workspace \
alleninstituteforai/olmocr:latest-with-model \
-c "olmocr /workspace/output --markdown --pdfs /workspace/*.pdf"The benchmark story
The interesting bit isn’t that olmOCR is “good” — every 7B VLM fine-tune claims that. The interesting bit is that AI2 shipped their own benchmark alongside the model and it survives external scrutiny. olmOCR-Bench has 7,000+ test cases across 1,400 documents, sliced into eight categories: ArXiv, old scans, math, tables, old-scans-headers, multi-column, long tiny text, and a base category.
Here’s the leaderboard as of v0.4.0:
| System | Overall |
|---|---|
| Chandra OCR 0.1.0 | 83.1 |
| Infinity-Parser 7B | 82.5 |
| olmOCR v0.4.0 | 82.4 |
| PaddleOCR-VL | 80.0 |
| Marker 1.10.1 | 76.1 |
| DeepSeek-OCR | 75.7 |
| MinerU 2.5.4 | 75.2 |
| Mistral OCR API | 72.0 |
| Nanonets-OCR2-3B | 69.5 |
Two things stand out. First, olmOCR is inside the top three despite being fully open-source and the only one where you get the weights, the training data, the training code, and the benchmark all under Apache 2.0. Chandra and Infinity-Parser are strong but closed-weights. Second, the pairwise-preference numbers from AI2’s own human eval — olmOCR was preferred 61.3% of the time against Marker, 58.6% against GOT-OCR, and 71.4% against MinerU — matter more than the aggregate score for real workloads, because they show olmOCR is producing text you actually want to read.
The Medium recap of the paper put the human-ELO story bluntly: “Human ELO rankings (452 judgments) place olmOCR at 1,800+ Elo, far above Marker (1,600) and MinerU (1,500).”
What the community actually says
The Hacker News thread on the initial launch (item 43174298) has the usual reception curve — impressed, then skeptical, then grudgingly on board. The strongest positive was that olmOCR runs locally on a single Nvidia GPU and produces a single stream of text with correct reading order for multi-column PDFs — which is the failure mode of every non-VLM tool. Handwriting recognition surprised people who expected traditional-OCR performance.
The skepticism has a real edge to it, and it’s important to name. VLMs can and do hallucinate on unreadable text. A traditional OCR engine faced with a smudged scan will output gibberish with low confidence — you can filter it. A VLM will confidently make something up that looks plausible. Multiple HN commenters flagged this as the reason they won’t use VLM-based OCR for anything legal, medical, or scientific without a second pass.
The r/LocalLLaMA thread was more practical — the first questions were about VRAM (12GB minimum, comfortable at 24GB), context length, and whether it can be quantized further (the FP8 model already is). The most interesting real-world comparison was on r/fintech, where someone benchmarked olmOCR against Gemini 2.0 Flash for structured extraction from financial tables. Verdict: Gemini’s still better on the tightest table layouts, olmOCR is cheaper and more predictable on volume.
Two threads on daily.dev and AI Innovations Substack picked up on the training data story — 260K PDF pages, 100K unique documents, all with GPT-4o silver labels. That’s what fine-tunes a 7B model into an OCR specialist. If you want to replicate the recipe on your own domain, everything’s in the repo under olmocr/train/.
Cost math
The bit AI2 keeps repeating in the docs — “less than $200 USD per million pages” — is worth checking, because it’s the number that decides whether you self-host or pay Mistral $1/1000 pages.
At $0.09 in / $0.19 out per million tokens on DeepInfra (one of the officially-tested providers), and ~500 tokens per page average for markdown output, one million pages is roughly:
- Input: ~150M tokens × $0.09/M = $13.50
- Output: ~500M tokens × $0.19/M = $95
That’s ~$108 per million pages on hosted inference, plus the ingestion machine cost. On your own H100 running vLLM at FP8, the marginal cost per page trends toward the electricity + amortized GPU cost — well under $100/M pages if the box is saturated. On a rented H100 at $2/hr running at the paper’s ~5,700 pages/GPU-hour, that’s ~$350/M pages — higher than the marketing number, but still well under any commercial API.
The takeaway: olmOCR wins the cost argument the moment your workload is more than ~50K pages/month. Below that, the setup cost isn’t worth it and you should just call Mistral OCR.
Honest limitations
- VLM hallucinations are real. As the HN thread flagged, a 7B VLM can confidently invent text when the source is unreadable. If accuracy on scans matters — legal, medical, scientific — you need a second-pass validator (e.g., match against Tesseract output and flag disagreements). olmOCR doesn’t ship that out of the box.
- GPU-heavy. Even the FP8 model needs ~12GB VRAM minimum, and you’ll want 24GB (RTX 3090 / 4090 / L40S / A100) for real throughput. There’s no CPU fallback and no Apple-Silicon path.
- English-first, and it shows. The
--apply_filterflag literally filters to English, non-form, non-SEO-spam PDFs. Non-Latin scripts work but the training data is English-heavy, and the tables/math training slice is essentially all English. ttf-mscorefonts-installeris a real dependency. The Ubuntu install spec includes Microsoft’s core fonts because the model was trained on rendered pages that use them. Skip this and rendering-side artifacts creep in.- Not a document-understanding system. olmOCR extracts text. If you want key-value extraction, form parsing, or structured JSON output (“give me the invoice total”), you’re building that layer yourself on top of the markdown.
- The Docker image is 30GB. Includes the model weights. If you’re spinning up ephemeral cloud GPUs on every job, that pull dominates your cold-start.
How it compares
| Tool | License | Runs locally? | Bench score | Cost / 1M pages | Best for |
|---|---|---|---|---|---|
| olmOCR | Apache 2.0 (weights + code) | Yes (GPU) | 82.4 | ~$100–$350 | High-volume LLM training data, self-hosted |
| Chandra OCR 0.1.0 | Closed weights | No | 83.1 | API pricing | Highest raw score, if you don’t need open weights |
| Marker 1.10.1 | GPL | Yes (GPU/CPU) | 76.1 | ~$0 self-hosted | Fast, layout-only extraction |
| MinerU 2.5.4 | AGPL | Yes | 75.2 | ~$0 self-hosted | Scientific PDFs, formula-heavy |
| Mistral OCR API | Closed | No | 72.0 | $1,000 | Low-volume, no infra |
| Nanonets-OCR2-3B | Open weights | Yes (GPU) | 69.5 | Self-hosted | Smaller GPU footprint |
| DeepSeek-OCR | Open weights | Yes | 75.7 | Self-hosted | Multilingual, competitive on cost |
The unique seat is: highest open-weights score + real production pipeline (S3, Beaker, vLLM) + shipped benchmark. Marker and MinerU are faster on CPU but their ceiling is lower. Chandra beats olmOCR on the bench but you can’t fine-tune it. Mistral OCR is easier but 14× more expensive on volume.
FAQ
Is olmOCR free?
Yes — Apache 2.0 on all code, model weights, training data, and benchmark. You pay for compute (a GPU or an inference provider). There’s no license tier and no “open-core” catch — the same model AI2 uses internally is the one on Hugging Face.
Do I need a GPU?
For local inference, yes — 12GB VRAM minimum, 24GB comfortable. If you don’t have a GPU, install the lightweight package (pip install olmocr) and point --server at Cirrascale, DeepInfra, or Parasail. They all host olmOCR-2-7B-1025 at $0.07–$0.10 per million input tokens.
How does olmOCR compare to Mistral OCR?
On the AI2-run benchmark, olmOCR scores 82.4 vs Mistral OCR’s 72.0 — a real 10-point gap that shows up as fewer table errors and better multi-column reading order. On cost, olmOCR is ~10× cheaper per million pages at hosted inference and effectively free at scale on your own hardware. Mistral OCR wins on setup simplicity for one-off jobs.
Can I fine-tune it on my own documents?
Yes — that’s the point. The olmocr/train/ directory has a clean SFT trainer for Qwen2.5-VL plus a GRPO RL trainer (the technique behind the v0.4.0 improvement). If your domain is old scanned invoices, medical charts, or legal filings, you can fine-tune on a few thousand labeled pages and materially improve accuracy on that slice.
What about non-English documents?
It works, but performance drops. The training data is heavily English, and the --apply_filter flag explicitly filters to English PDFs. For serious multilingual OCR at scale, DeepSeek-OCR or PaddleOCR-VL are stronger picks today. That said, olmOCR still handles Latin-script European languages well because of the shared Qwen2.5-VL base.
Does it handle math and equations?
Yes, and this is one of the strongest slices. On the “Old scans math” category of olmOCR-Bench, olmOCR v0.4.0 scores 82.3 — second only to Chandra (80.3 on the same slice). Equations come out as LaTeX inside the Markdown, which is usually what you want for LLM training and RAG.
How do I run it on RunPod / Modal / Lambda Labs?
The Docker path is the fastest. Pull alleninstituteforai/olmocr:latest-with-model on an H100 or L40S template, mount your PDFs, and run the pipeline. For repeated jobs, cache the image on a persistent volume — the 30GB pull is the slowest part of a cold start.
Bottom line
If you’re building an LLM training dataset, running RAG over a document archive, or converting a corpus of PDFs into anything useful, olmOCR is now the default open-source pick. It’s not the highest-scoring OCR system on olmOCR-Bench — Chandra edges it — but it is the highest-scoring system where you get the weights, the training code, the data, the pipeline, and the benchmark under Apache 2.0.
The right sequence is: hit the hosted demo with a hard PDF from your workload → if the output is right, run the Docker image on a single-GPU box against 100 pages → if the numbers hold, wire up the S3 pipeline for the real corpus. That’s a one-afternoon evaluation for a decision that will save you five figures a year against any commercial API.
The next release (per AI2’s roadmap in the blog post) focuses on document-level QA rewards and a smaller distilled model — both of which would push this repo from “best default” to “the answer” for open PDF extraction.
Install: pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128. Star and try the demo at github.com/allenai/olmocr.