TL;DR

pdf-inspector is Firecrawl’s open-source Rust library that answers one narrow question extremely fast: does this PDF actually need OCR? Then, when the answer is no, it extracts the text itself and hands you clean Markdown. Highlights:

  • ~15.8K GitHub stars (repo opened February 6, 2026), 1,095 forks, MIT-licensed, pure Rust
  • Classifies a PDF in ~10-50ms without rendering a single page — returns TextBased, Scanned, ImageBased, or Mixed plus a confidence score and a per-page pages_needing_ocr list
  • No ML models, no GPU, no API key, no system dependencies — one Rust dependency (lopdf)
  • Bindings for Python, Node.js, browser WebAssembly, and Rust, plus two CLIs (pdf2md, detect-pdf)
  • Benchmark headline: 0.875 overall on the 200-document opendataloader-bench corpus, and it finished the whole corpus in 0.47s versus 17.1s for PyMuPDF4LLM
  • The honest catch: it does no OCR itself, the open-source build doesn’t extract figures, the benchmark is vendor-run, and there are real open bugs on CJK fonts and dense two-column layouts

If your pipeline currently pushes every uploaded PDF through an OCR or vision endpoint, this is the routing layer that stops you paying GPU prices for documents that already contain their own text. Below: how classification works, real code in three languages, the benchmark with caveats attached, where it breaks, and who should use something else.

What is pdf-inspector?

Firecrawl builds web-scraping and document-parsing infrastructure for AI pipelines. pdf-inspector is the piece of that stack they open-sourced: a from-scratch Rust PDF engine that reads a document’s internal structure — font encodings, text operators, image coverage — rather than rendering it to pixels and looking at it.

That distinction is the whole product. A conventional “PDF to Markdown for RAG” tool tends to assume the worst: a page might be a scan, so send it to OCR or a vision model and let the GPU sort it out. Firecrawl’s stated figure is that roughly 54% of PDFs don’t need OCR at all — reports, research papers, invoices, contracts, and legal filings are usually generated by software and carry a perfectly good text layer inside them.

pdf-inspector’s job is to find that out in milliseconds and, if the text is there, pull it out locally in about 150ms. Firecrawl says this routing layer is what makes their hosted Fire-PDF engine 3.5x to 5x faster than their previous pipeline: on a 200-page report where 150 pages are pure text, 150 pages never touch a GPU.

The library ships alongside a sibling project, AnyDoc, which covers 14 non-PDF formats (docx, xlsx, pptx, rtf, odt, epub, csv and friends) and embeds pdf-inspector so a single call also handles text-based PDFs. They’re deliberately separate repos: pdf-inspector is the dedicated PDF engine, AnyDoc is the everything-else layer.

How the classifier actually works

This is the part worth understanding, because it explains both the speed and the failure modes.

Detection does not load the full document. The steps are:

  1. Parse the xref table and page tree — no full object load
  2. Select pages according to a scan strategy
  3. Look for Tj / TJ (text-showing operators) and Do (image operators) inside the content streams
  4. Classify based on text-operator presence across the sampled pages

That’s it. No rasterization, no layout model, no inference. It’s why a 300-page PDF gets classified in milliseconds.

The scan strategy is configurable, and picking the right one matters more than most people realize:

StrategyBehaviorBest for
EarlyExit (default)Scan pages, stop on the first non-text pagePipelines routing TextBased PDFs to fast extraction
FullScan every page, no early exitAccurate Mixed vs Scanned classification
Sample(n)Sample n evenly distributed pages (first, last, middle)Very large PDFs where speed beats precision
Pages(vec)Only scan specific 1-indexed pagesWhen you already know which pages matter

The default EarlyExit is optimized for a yes/no gate. If you need to know which pages are scanned so you can route only those to a vision model, use Full and read pages_needing_ocr. That per-page routing is the feature that separates this from a simple “is it scanned?” heuristic — you get page-level granularity instead of an all-or-nothing verdict on the document.

The classifier also flags encoding issues explicitly. If a font has a broken or missing encoding table, pdf-inspector tells you rather than silently emitting garbage, so your caller can fall back to OCR for that page. In practice this is the single most useful safety valve in the API.

Real code: three languages, one call

The API surface is deliberately tiny. Python, via maturin:

pip install maturin
maturin develop --release
import pdf_inspector

result = pdf_inspector.process_pdf("document.pdf")
print(result.pdf_type)   # "text_based", "scanned", "image_based", "mixed"
print(result.markdown)   # Markdown string or None

Node.js:

npm install @firecrawl/pdf-inspector
import { readFileSync } from 'fs';
import { processPdf, classifyPdf } from '@firecrawl/pdf-inspector';

const result = processPdf(readFileSync('document.pdf'));
console.log(result.pdfType);   // "TextBased" | "Scanned" | "ImageBased" | "Mixed"
console.log(result.markdown);  // Markdown string or null

Rust:

cargo add pdf-inspector
use pdf_inspector::process_pdf;

let result = process_pdf("document.pdf")?;
println!("Type: {:?}", result.pdf_type);
if let Some(markdown) = &result.markdown {
    println!("{}", markdown);
}

And the one that surprised me — the same Rust core compiled to WebAssembly, running in a browser or Web Worker with embedded CMaps and no server round trip:

import init, { processPdf } from '@firecrawl/pdf-inspector-wasm';

await init();
const response = await fetch('/document.pdf');
const pdf = new Uint8Array(await response.arrayBuffer());
const result = processPdf(pdf);

console.log(result.pdfType);
console.log(result.markdown);

For a browser app where users drop in PDFs, the document never leaves the machine — a meaningful privacy story for contracts, medical records, or financial statements, and it’s the same parser rather than a separate reimplementation.

There are also two CLIs, which is how you should evaluate it before writing any integration code:

cargo install pdf-inspector

# Convert PDF to Markdown (add --json to pipe it)
pdf2md document.pdf

# Token-efficient output (collapses TOC dot leaders and similar padding)
pdf2md document.pdf --compact

# Page break markers, or a page subset
pdf2md document.pdf --pages
pdf2md document.pdf --select-pages 1,3,5-10

# Detection only, plus optional layout analysis (tables, columns)
detect-pdf document.pdf --json
detect-pdf document.pdf --analyze --json

The --compact flag deserves a mention for anyone feeding output into an LLM: table-of-contents dot leaders and similar source padding can eat a startling number of tokens on corporate reports, and collapsing them is free savings.

What the Markdown converter handles

“Converts to Markdown” is doing a lot of work in most tools’ README. Here, the detection rules are documented, which makes it possible to predict the failure cases:

ElementHow it’s detected
Headings (H1-H4)Font size tiers relative to body text, with 0.5pt clustering
Bold / italicFont name patterns (Bold, Italic, Oblique)
Lists•, -, *, ○ prefixes; 1., 1), (1); a., a), (a)
Code blocksMonospace fonts (Courier, Consolas, Menlo, Fira Code, JetBrains Mono) plus keyword detection
TablesRectangle detection from PDF drawing ops plus heuristic detection from text alignment
Captions”Figure”, “Table”, “Source:” prefixes
CleanupRejoins hyphenated line breaks, filters page numbers, merges drop caps

Note what’s implied: headings are inferred from font size, not from document structure. A PDF whose author styled a heading at the same size as body text, using only bold, will lose that heading. Code blocks are inferred from font family, so a code sample typeset in a proportional font won’t be fenced. These are reasonable heuristics — they’re also the reason a structure-aware model-based parser will sometimes win on weird documents.

The table handling is the strongest part. Dual-mode detection (rectangle-based from actual PDF drawing operations, plus alignment heuristics when the table has no ruling lines) covers both the “properly drawn table” and the “whitespace-aligned financial statement” cases, including continuation tables across pages and consolidated numeric values.

Benchmarks — and what to discount

Firecrawl evaluated pdf-inspector on the opendataloader-bench corpus (200 PDFs), restricted to local engines without model-based parsing, with OCR disabled. Results refreshed July 31, 2026, on an Apple M4 Pro:

EngineOverallReading order (NID)Tables (TEDS)Headings (MHS)Speed (200 docs)
pdf-inspector0.8750.9150.8140.7880.470s
LiteParse0.8730.9130.6930.8110.750s
OpenDataLoader0.8310.9020.4890.7392.569s
PyMuPDF4LLM0.7350.8860.4010.42417.117s
MarkItDown0.5890.8440.2730.00016.165s

Versions tested: pdf-inspector 0.2.6, LiteParse 2.10.1, OpenDataLoader 2.2.1, PyMuPDF4LLM 0.2.0, MarkItDown 0.1.5. Speed is the median of five complete corpus runs after a discarded warm-up.

Three honest caveats:

  1. This is a vendor-run benchmark. Firecrawl publishes per-document predictions, evaluator output, and a reproducible-results branch, which is more transparency than most vendors offer — but they chose the corpus configuration and they’re scoring their own product. Run it on your documents.
  2. OCR was disabled and model-based parsers were excluded. Docling, Marker, and MinerU aren’t in this table. That’s methodologically defensible (they solve a different problem at a different cost) but it means this is not “pdf-inspector beats the field.”
  3. The headline speed is per-corpus, not per-page guaranteed. 0.47s for 200 documents averages out to roughly 2ms per document on that corpus. A 400-page PDF full of heuristic tables will not finish in 2ms.

The genuinely notable numbers are the table score (0.814 versus 0.693 for the next-best local engine) and the 36x speed gap over PyMuPDF4LLM, which is the incumbent default in a lot of RAG stacks. LiteParse is statistically tied on overall quality and slightly better on headings, so if you’re already using it, the case for switching is speed and tables, not general accuracy.

Community reaction

Growth has been steep and quiet: roughly 8,600 stars in a single week in early August 2026, pushing the repo past 15K. Firecrawl’s own blog cited 13K at writing time; the count is ~15.8K now.

Notably, the traction is not coming from Hacker News. Three separate submissions in early August landed at 5, 5, and 3 points with barely any comments. The distribution has been X (Firecrawl CTO Nick Camara’s launch thread), GitHub trending, and the r/firecrawl and r/Rag communities. Repo-momentum trackers show mentions firing on one of six monitored channels.

Sentiment where discussion does happen splits along a predictable line. Teams building document pipelines like it a lot — the routing insight (“stop paying OCR for pages that don’t need it”) lands immediately with anyone who has seen a GPU bill for parsing invoices. Teams doing document understanding on hard material are more measured: an independent comparison against Docling by Laura Martel found the open-source build can’t extract figures, which is disqualifying for scientific papers where the figures are the point.

The broader r/Rag consensus in 2026 still leans toward Docling or MinerU as the default for complex and multilingual layouts, with PyMuPDF4LLM as the lightweight fallback. pdf-inspector is entering as a fast path, not as a replacement for that tier — and the README is refreshingly upfront that its best fit is “native-text PDFs where speed, reading order, and table structure matter.”

Honest limitations

It does no OCR. This is the design, not a bug, but it needs restating: pdf-inspector will tell you a page is scanned and hand you the page reference and the reason. It will not read that page. You still need Tesseract, a vision model, or a hosted OCR service behind it. If your corpus is mostly scans, this library saves you nothing.

No figure or image extraction in the open-source build. Image placeholders appear via the xobjects path, but you don’t get the figures out. For research papers, technical manuals, and anything where diagrams carry meaning, that’s a hard limit.

Font-based heuristics have a real error surface. Open issues on the repo at time of writing include garbled text (U+FFFD) for Chinese PDFs using GBK-EUC-H fonts without a ToUnicode CMap, garbled Form XObject text when /Resources is an indirect reference, <u> tags splitting words when underline flags differ mid-word, first-row duplication in compact financial tables, and an open feature request for dense justified two-column layouts with gutters under 8pt. None of these are exotic — CJK documents and tight academic two-column layouts are common.

165 open issues against a repo six months old is a lot of surface area, though the flip side is a visibly active maintenance cadence (v1.14.2 shipped August 13, 2026, and the issue tracker shows OCR-fusion and vision work in progress).

Heading detection can return nothing useful on documents that don’t vary font size. MarkItDown scoring 0.000 on headings in the benchmark is a reminder that this whole category is fragile; pdf-inspector’s 0.788 is good, not solved.

Who should use it

Use it if: you run a document pipeline at any volume, your inputs are mostly software-generated PDFs (reports, invoices, contracts, filings, papers), you’re currently paying OCR or vision costs indiscriminately, or you want local/in-browser parsing for privacy reasons. The classifier alone — used purely as a router in front of your existing stack — is worth the integration even if you never use its Markdown output.

Skip it if: your corpus is predominantly scans or photographs, you need figures and images extracted, you work primarily with CJK documents (until those encoding issues close), or you need one library to handle docx/xlsx/pptx too — in which case use AnyDoc, which embeds pdf-inspector anyway.

The strategic read: pdf-inspector isn’t competing with Docling or Marker. It’s competing with the assumption that every PDF is hard. That assumption is expensive, and for a bit more than half of real-world documents it’s simply wrong. A 20ms check that eliminates a 2-10 second OCR call on every one of those is the kind of unglamorous infrastructure win that compounds quietly at scale.

FAQ

Is pdf-inspector free and open source? Yes. MIT-licensed, with no API key and no paid tier for the library itself. It’s published on crates.io, PyPI (pdf-inspector), and npm (@firecrawl/pdf-inspector, plus @firecrawl/pdf-inspector-wasm for browsers). Firecrawl monetizes the hosted /parse and /scrape endpoints that use it, not the library.

Does pdf-inspector do OCR? No. It detects which pages need OCR and routes them out with a reason attached, then extracts text natively from the pages that don’t. You supply the OCR engine for the remainder. This is the core design decision, not a missing feature.

How does it compare to PyMuPDF4LLM, Docling, and Marker? Against PyMuPDF4LLM it’s substantially better on tables (0.814 vs 0.401 TEDS) and roughly 36x faster on Firecrawl’s 200-document benchmark. Docling, Marker, and MinerU are model-based parsers that handle scanned pages and complex layouts pdf-inspector cannot touch — they’re slower and heavier by design. The sane architecture in 2026 is pdf-inspector as the fast path with a model-based parser behind it for the pages it flags.

Can it run in a browser? Yes. The WebAssembly build (@firecrawl/pdf-inspector-wasm) runs the same Rust parser in a browser or Web Worker with embedded CMaps, so PDFs never leave the user’s device. Useful for privacy-sensitive uploads and for cutting a server round trip out of a document UI.

How accurate is the scanned-vs-text classification? It returns a confidence score from 0.0 to 1.0 alongside the type, and flags broken font encodings separately so you can fall back to OCR when text extraction would produce garbage. Accuracy depends on scan strategy: the default EarlyExit is tuned for a fast yes/no gate, while Full is what you want for reliably distinguishing Mixed from Scanned documents.

What languages and layouts does it support? It handles CID fonts via ToUnicode CMap decoding (Type0/Identity-H, UTF-16BE, UTF-8, Latin-1), multi-column newspaper-style layouts with automatic reading order, and RTL text. Known gaps: Chinese PDFs using GBK-EUC-H fonts without a ToUnicode CMap can produce garbled output, and dense justified two-column layouts with sub-8pt gutters are an open feature request.

Sources