What is the most reliable way to get JSON from an LLM in 2026?

Use the provider's native structured-outputs feature with a JSON Schema and strict mode. OpenAI (Responses and Chat Completions APIs, strict: true), Anthropic (output_config.format with type json_schema, GA on Claude Opus 5, Sonnet 5, Fable 5.1 and others) and Google Gemini (response_mime_type application/json plus response_json_schema) all use constrained decoding, so the model cannot emit a token that violates the schema. This removes JSON.parse errors and missing fields entirely; prompting 'reply in JSON' does not. Validate the result with Pydantic or Zod anyway for business rules a schema cannot express.

Should I use structured outputs or function calling for extraction?

Structured outputs when you want the model's answer itself in a fixed shape (extraction, classification, reports). Strict tool use when the JSON is an argument to something your code will execute (search, database writes, agent actions) — all three major providers support strict: true on tool definitions so arguments are schema-guaranteed. Older 'JSON mode' (valid JSON, no schema) is obsolete; do not use it for anything that parses fields.

What JSON Schema features are not supported in strict mode?

The common limits across OpenAI and Anthropic: every object needs additionalProperties: false and all properties listed in required (use nullable types for optional fields); no numeric minimum/maximum, no minLength/maxLength, no pattern regex, no default values, and limited or no recursion and $ref depth. Anthropic additionally rejects recursive schemas. Gemini supports a JSON Schema subset (string, number, integer, boolean, object, array, null, enum, required) with property-ordering control. If you need those constraints, enforce them in post-validation and retry, or use a grammar-constrained open-model runtime such as Outlines or vLLM guided decoding.

Do structured outputs cost extra or slow the model down?

No surcharge from OpenAI, Anthropic or Google; you pay normal token rates and the schema adds a few hundred input tokens. The first request with a new schema can add latency while the provider compiles the grammar (OpenAI and Anthropic cache it afterward). Output quality can drop slightly if a schema forces the model to commit to an answer before reasoning — put a free-text 'reasoning' or 'notes' field before the structured fields, or use a reasoning model where thinking happens before the constrained output.

Quick Answer

How to Get Structured JSON Output From LLMs (2026 Guide)

Published: September 15, 2026

The short answer

In 2026 you should never parse LLM output with regex or hope. Every major API now offers constrained decoding against a JSON Schema, which makes malformed output impossible rather than merely unlikely. The recipe:

Define the shape once as a Pydantic (Python) or Zod (TypeScript) model.
Send it as a JSON Schema with strict mode to the provider’s native structured-outputs parameter.
Validate the parsed result with the same model for business rules.
Retry with the validation error as feedback only when step 3 fails.

Everything below is the detail.

Method comparison

Method	Guarantees valid JSON?	Guarantees schema?	Works with	Use when
”Respond in JSON” prompt	No	No	Any model	Never in production
JSON mode (legacy)	Yes	No	Older OpenAI/Gemini settings	Legacy code only
Native structured outputs (strict)	Yes	Yes (constrained decoding)	GPT-6 Astra, GPT-5.6 family, Claude Opus 5/Sonnet 5/Fable 5.1, Gemini 3.x	Default choice
Strict tool / function calling	Yes	Yes	Same models	JSON is an action argument
Instructor (library)	Via provider + retries	Via Pydantic validation + retries	15+ providers	Cross-provider code, extra validation loops
Outlines / vLLM guided decoding / llama.cpp GBNF	Yes	Yes, incl. regex and CFG	Open-weight models you host	Self-hosted, or constraints strict mode cannot express

Provider-by-provider (September 2026)

OpenAI — GPT-6 Astra, GPT-5.6 Sol / Terra / Luna

Parameter: Responses API text.format = {type: "json_schema", name, schema, strict: true} (Chat Completions: response_format). The SDKs expose client.responses.parse(...) with a Pydantic/Zod type and return a typed object.
Tools: strict: true on each function definition guarantees argument schemas.
Rules: additionalProperties: false on every object; all properties in required; optional fields as ["string", "null"]; no minimum/maximum, minLength/maxLength, pattern, or default. Nested depth and total size are capped (roughly 5 levels, 100 properties in early limits — check the current docs).
Refusals: the response can carry a refusal instead of a parsed object; handle it.

Anthropic — Claude Fable 5.1, Opus 5, Sonnet 5, Opus 4.8, Haiku 4.5

Parameter: output_config: {format: {type: "json_schema", schema: {...}}} — GA, no beta header. (The 2025 beta output_format field and structured-outputs-2025-11-13 header still work for a transition period, but Python SDK v1.0+ rejects output_format; migrate.)
Tools: strict: true on a tool’s input_schema guarantees tool names and inputs.
Platforms: Claude API, AWS Bedrock, Google Cloud Vertex, Microsoft Foundry. Supported models include claude-fable-5-1, claude-opus-5, claude-sonnet-5, claude-opus-4-8/4-7/4-6, claude-sonnet-4-6/4-5, claude-haiku-4-5. Zero-data-retention eligible.
Rules: same additionalProperties: false + required discipline; no recursive schemas; no numeric range or string length constraints; a schema grammar is compiled on first use and cached.
Tip: the SDKs offer client.messages.parse(...) with a Pydantic model, mirroring OpenAI.

Google — Gemini 3.8 Flash, 3.7 Flash, 3.5 Pro preview, 3.1 Pro

Parameter: generation_config.response_mime_type = "application/json" plus response_json_schema (or the older response_schema OpenAPI-style object). Constrained decoding enforces the schema, not just JSON validity.
Rules: JSON Schema subset — string, number, integer, boolean, object, array, null, enum, required; propertyOrdering controls field order (matters for quality); large or deeply nested schemas can be rejected.
With tools: Gemini 3.x models can combine structured output with built-in tools (search, code execution) in one request.

Open-weight models you host (DeepSeek V4, Qwen 3.8, GLM-5.3, Kimi K3)

vLLM and SGLang support guided decoding (guided_json, guided_regex, guided_grammar) via Outlines or XGrammar backends — full JSON Schema, regex and context-free grammars.
llama.cpp / Ollama accept GBNF grammars and JSON Schema (format parameter in Ollama).
Inference clouds (Fireworks, Together, DeepInfra, Groq) expose response_format: {type: "json_schema"} on most catalog models, with the same strict-mode style limits.

Step-by-step recipe

Step 1 — Model the output.

from pydantic import BaseModel, ConfigDict
from typing import Literal

class Invoice(BaseModel):
    model_config = ConfigDict(extra="forbid")   # → additionalProperties: false
    vendor: str
    total: float
    currency: Literal["USD", "EUR", "GBP"]
    due_date: str | None                        # nullable, still required
    line_items: list["LineItem"]

class LineItem(BaseModel):
    model_config = ConfigDict(extra="forbid")
    description: str
    amount: float

extra="forbid" and explicit nullables are what make the schema strict-compatible on every provider.

Step 2 — Call with strict schema. Use responses.parse (OpenAI), messages.parse (Anthropic) or response_json_schema (Gemini) with Invoice.model_json_schema().

Step 3 — Validate again. Invoice.model_validate(obj) catches nothing the grammar allowed but your rules forbid — negative totals, due dates in the past, sums that do not add up. Add @field_validators for those.

Step 4 — Retry with feedback, bounded. On validation failure, append the error text and re-ask once or twice; Instructor automates this loop across providers. Log the failure rate; above ~1% the schema or prompt needs work, not more retries.

Step 5 — Keep reasoning out of the fields. Either use a reasoning model (Astra effort levels, Claude extended thinking, Gemini thinking) so reasoning happens before the constrained output, or add a rationale: str field first in the schema so the model can think in-band before committing to total.

Pitfalls seen in 2026 codebases

Optional fields done wrong. Omitting a key from required breaks strict mode on OpenAI and Anthropic. Make it required and nullable.
Enums as strings. status: str invites “Completed”, “complete”, “DONE”. Use Literal[...]/enum.
Huge schemas. 300-field schemas hit provider limits and degrade accuracy. Split into multiple calls or a two-pass extract-then-fill.
Numbers as strings. If the source has “1,234.50”, let the model output a number and normalize in the prompt, or accept a string and parse in validation — decide once.
Streaming. All three providers stream structured output as partial JSON; only parse on completion, or use the SDK’s typed streaming helpers.
Prompt injection through documents. A schema constrains shape, not truth. Extracted values can still be attacker-controlled text from the input; treat them as untrusted downstream.
Tokenizer drift. Claude 4.7+ and Mythos use a tokenizer that yields ~30% more tokens for the same text than Sonnet 4.6-era models; schema overhead scales with it when comparing costs.

When to use a library

Instructor (Python/TS): one interface over OpenAI, Anthropic, Gemini, Bedrock, Ollama and more; Pydantic validation with automatic retries; streaming partials. Worth it when you support multiple providers or want validation-driven retries out of the box.
Outlines / XGrammar: when you self-host and need regex or grammar constraints strict mode lacks (phone formats, DSLs, exact enumerations of thousands of values).
LangChain / LlamaIndex with_structured_output: fine if you already use the framework; they wrap the same provider features.