LLM structured output: JSON mode, JSON Schema and strict mode explained

LLM structured output is an API feature that forces a model's response to match a JSON Schema you provide, so your code can parse it without guesswork. Anthropic, OpenAI and Google all offer it, and Anthropic and Google both describe the mechanism as constrained decoding, which limits what the model can generate so the response stays valid against your schema.

This guide covers the difference between JSON mode and schema-enforced output, how each vendor's API turns it on, what schemas they accept, and the edge cases that still need handling. Everything was checked against official docs on 26 October 2026.

What is the difference between JSON mode and structured outputs?

JSON mode guarantees valid JSON. Structured outputs guarantee JSON that matches your schema. OpenAI's Structured Outputs guide puts it plainly: "Structured Outputs is the evolution of JSON mode. While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence." It adds: "We recommend always using Structured Outputs instead of JSON mode when possible."

JSON mode has a quirk worth knowing. OpenAI says you must instruct the model to produce JSON somewhere in the conversation, and "the API will throw an error if the string "JSON" does not appear somewhere in the context."

Without either feature, Anthropic's structured outputs docs list what tends to go wrong: "Parsing errors from invalid JSON syntax," "Missing required fields," "Inconsistent data types," and "Schema violations requiring error handling and retries."

How do Claude, OpenAI and Gemini APIs turn it on?

Each puts the schema in a different request field:

Anthropic describes its feature this way: "Structured outputs constrain Claude's responses to follow a specific schema, ensuring valid, parseable output for downstream processing." Here is the shape of a request, trimmed from Anthropic's example:

{
  "model": "claude-opus-5-5",
  "max_tokens": 1024,
  "messages": [
    {"role": "user", "content": "Extract the key information from this email: ..."}
  ],
  "output_config": {
    "format": {
      "type": "json_schema",
      "schema": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "email": {"type": "string"},
          "demo_requested": {"type": "boolean"}
        },
        "required": ["name", "email", "demo_requested"],
        "additionalProperties": false
      }
    }
  }
}

The SDK helpers save you from writing schemas by hand. With OpenAI's Python SDK, for example, you pass a Pydantic model and read the parsed result:

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

response = client.responses.parse(
    model="gpt-6-astra",
    input=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    text_format=CalendarEvent,
)
event = response.output_parsed

Anthropic's Python SDK has the same pattern with client.messages.parse() and an output_format argument, and Google's structured output docs show Pydantic's model_json_schema() passed as the schema. OpenAI recommends the helpers for another reason too: they stop your schema and your code's types from drifting apart.

Should you use structured outputs or tool calling?

It depends on whether the model is answering or asking your app to act. OpenAI's guidance: if you are connecting the model to tools, functions or data in your system, use function calling; if you want to structure the model's output when it responds to the user, use a structured text.format. Google's docs draw the same line: structured outputs are for "Formatting the final response," while function calling is for "Taking action during conversation."

Tool arguments can be schema-enforced too. Anthropic's strict tool use (strict: true) and OpenAI's strict function calling apply the same constrained decoding to tool inputs, and Anthropic lets you use JSON outputs and strict tools in the same request. For how the tool loop works, see what is tool calling. Tools exposed through MCP also describe their inputs with JSON Schema.

Which JSON Schema features are supported?

A subset, and the subsets differ. Check before you port a schema between vendors:

Anthropic notes that its SDK helpers can transform schemas that use unsupported constraints, and that sending an unsupported feature directly returns a 400 error. Google strips unenforced keywords such as minLength "rather than returning an error," so a Pydantic or Zod length check will not be enforced by the model.

There are size limits as well. Anthropic allows up to 20 strict tools, 24 optional parameters and 16 union-type parameters across all strict schemas in one request. OpenAI allows up to 5,000 object properties with up to 10 levels of nesting, and up to 1,000 enum values. Google says very large or deeply nested schemas can be rejected with a 400 INVALID_ARGUMENT error.

Can structured output still fail?

Yes. Schema enforcement covers syntax and shape, not truth. Anthropic says "Although structured outputs guarantee schema compliance in most cases, there are scenarios where the output might not match your schema," and Anthropic and OpenAI both list the same two:

• Refusals. If the model declines for safety reasons, the refusal can take precedence over the schema. Anthropic returns stop_reason: "refusal"; OpenAI adds a refusal field.
• Token limits. If the response hits the token limit (max_tokens on Claude, max_output_tokens in OpenAI's Responses API), the JSON can be cut off. Anthropic suggests retrying with a higher limit.

Then there are valid-but-wrong outputs:

• Hallucinated fills. OpenAI warns that "The model will always try to adhere to the provided schema, which can result in hallucinations if the input is completely unrelated to the schema." Give the model an explicit way out, such as an empty value or a not_found flag.
• Wrong values. OpenAI: "Structured Outputs can still contain mistakes." Google: "While output is syntactically correct JSON, always validate values in your application."
• Enum casing. Anthropic says its structured outputs "don't guarantee the capitalization of string enum and const values," so compare enum values case-insensitively.

Does structured output work with thinking and reasoning?

Yes. Anthropic says "Grammars apply only to Claude's direct output, not to tool use calls, tool results, or thinking tags," which lets Claude "think freely while still producing structured output in the final response."

One Claude-specific trap: do not add a field for the model's reasoning. Anthropic warns that "A property that asks for the model's thinking or step-by-step reasoning might lead to a reasoning_extraction refusal," and suggests asking for a short explanation instead. OpenAI's guide, by contrast, includes a math tutoring example whose schema has a list of solution steps, which works because the steps are part of the answer shown to the user. See chain of thought prompting for the wider picture.

Google adds that on Gemini 3 models, structured outputs can be combined with built-in tools such as Google Search grounding, code execution and function calling, a feature it labels preview.

What does structured output cost?

A little latency on first use and a few extra tokens. Anthropic says "The first time you use a specific schema, there is additional latency while the grammar compiles," and that compiled grammars "are cached for 24 hours from last use." OpenAI notes the same first-request latency only for fine-tuned models and says other models do not have this limitation.

Anthropic also injects a short system prompt explaining the format, so input tokens rise slightly, and changing output_config.format invalidates the prompt cache for that conversation. Its docs list two incompatibilities: citations and message prefilling.

How do you design a schema the model fills well?

Write it like documentation. The vendors' tips overlap:

• Clear names and descriptions. OpenAI: "Name keys clearly and intuitively." Google: "Use the description field to guide the model."
• Strong types and enums. Google recommends specific types such as integer, string and enum, which also suits classification.
• Fewer optional fields. Anthropic says each optional parameter "roughly doubles a portion of the grammar's state space," so make fields required where a sensible default exists.
• Flat over deep. Anthropic suggests flattening nested structures when you hit complexity limits.
• Still prompt. Google's best practices include "Clearly state what you want the model to do." A schema defines shape; the prompt defines the task. Few-shot examples still help with tricky fields.

Where should extraction prompts and schemas live?

Next to each other, versioned. Kipt keeps prompts as Markdown in a private GitHub repo you own, with {{name}} blanks for the input text, so the extraction instructions you tested stay easy to find alongside the rest of your library.

What should you read next?

What is tool calling covers the other half of JSON Schema in LLM APIs. The prompt engineering guide covers the instructions that go around a schema. And LLM tokens explained explains why max_tokens matters for complete JSON.