How to Design Schemas for LLM Structured Output

Start with Your Application's Data Contract

The most common mistake is designing the schema around what you think the model can produce rather than what your application needs. Start from the other direction: what exact data structure does your downstream code consume? What fields does it read? What types does it expect? What values are valid? Write the schema that your application code wants to receive, then adjust only if the model cannot fill it reliably.

For example, if your application classifies support tickets into departments, your schema should have a "department" field with an enum of your actual department names, a "priority" field with your actual priority levels, and a "summary" field of type string. Do not add a "confidence" field unless your code uses it. Do not add a "reasoning" field unless you log or display it. Every field in the schema is a commitment: the model will spend tokens generating content for it, and your code will need to handle it.

That said, there is one exception to the "only include what you need" rule: a reasoning or explanation field often improves the quality of the other fields, even if your application discards it. When the model has to explain its reasoning before committing to a classification or extraction, the classification or extraction tends to be more accurate. This is the structured output equivalent of chain-of-thought prompting, and it is worth the extra tokens.

Choose the Right Type for Each Field

String fields are the most flexible and the least constrained. Use them for free-text content: summaries, explanations, descriptions, names. The model can put anything in a string field, so the quality depends entirely on the prompt. Keep string fields for content where you genuinely want the model's judgment about what to write.

Enum fields are the most constrained and the most reliable. Use them whenever you want the model to choose from a fixed set of options: categories, labels, sentiment values, status codes, rating levels. The model cannot invent new values, which eliminates an entire class of errors. When designing enums, use clear, unambiguous labels. "positive" is better than "pos". "high_priority" is better than "1". The model reads these labels and uses them to understand what each option means.

Number and integer fields guarantee numeric output. Use them for scores, counts, prices, percentages, and other numeric values. Note that the schema constrains the type but not the range: a number field can be any number, including negative numbers, zero, or extremely large values. If you need range constraints, validate them in your application code or with a validation library like Pydantic.

Boolean fields force a true/false decision. Use them for binary classifications: is this spam? Does this contain PII? Is this relevant? Booleans are cleaner than string enums with two values because they eliminate any ambiguity about the representation.

Array fields produce ordered lists. Specify the item type to constrain what goes in the array. Use arrays for lists of extracted entities, multiple classification labels (multi-label classification), sequences of steps, or collections of facts. Consider setting a reasonable maximum length in your prompt, even if the schema does not enforce it, to prevent the model from generating excessively long lists.

Nested objects create hierarchical structures. Use them when a concept has multiple attributes: a person with name, role, and email; a product with name, price, and category; a citation with source, quote, and page number. Keep nesting shallow, ideally 2-3 levels. Deeply nested schemas are harder for the model to fill accurately and harder for your code to consume.

Nullable fields (using anyOf with null) tell the model that a value is optional, meaning the field will be present but may be null. Use these for information that might not be available in the input: an email that might not be mentioned, a price that might not be stated, a date that might not be specified. This is better than using a default value like "unknown" because null is programmatically distinguishable from a real value.

Handle Provider Constraints

All providers impose constraints on schemas used with structured output. Designing within these constraints is not optional, because schemas that violate them will be rejected.

All properties must be required. In strict mode, every property in an object must appear in the required array. You cannot have optional properties at the schema level. If a property is semantically optional, make it required but nullable: include it in the required array and set its type to {"anyOf": [{"type": "string"}, {"type": "null"}]}. The model will return null when the information is not available.

Nesting depth is limited. Most providers support 5 levels of nesting, which is sufficient for nearly all production schemas. If your data structure is deeper than 5 levels, flatten it by pulling nested objects up to higher levels. An address inside a contact inside a company inside a deal is 4 levels; adding another level of nesting would push it to 5. Restructure to keep the most important information at shallow levels.

Total property count is limited. OpenAI limits schemas to 100 properties across all levels. This is rarely an issue for individual extraction tasks, but can be hit with complex schemas that have many nested objects each with many fields. If you hit this limit, split the extraction into multiple API calls with simpler schemas.

additionalProperties must be false. You cannot allow the model to add extra fields beyond what the schema defines. This is a feature, not a limitation: it prevents the model from adding unexpected fields that your code does not handle.

Some JSON Schema features are not supported. Features like patternProperties, if/then/else, format validators (like "email" or "date-time"), minimum/maximum on numbers, and minItems/maxItems on arrays are not available in constrained decoding. Validate these constraints in your application code or validation library instead.

Add a Reasoning Field

One of the most effective schema design patterns is including a "reasoning" or "explanation" field that the model fills before the main output fields. This is not a trick; it is based on the well-documented finding that language models produce better answers when they reason before answering, the same principle behind chain-of-thought prompting.

In practice, this means placing a "reasoning" string field before your main output fields in the schema. The model generates the reasoning text first, which activates relevant knowledge and considerations, and then generates the classification, extraction, or evaluation result with the benefit of that reasoning. The improvement in accuracy is measurable, typically 5-15% on classification tasks, and the cost is a few dozen extra output tokens.

Whether to include this field depends on your use case. For simple classifications with clear-cut categories, the improvement is small and may not justify the extra tokens. For nuanced judgments, ambiguous inputs, or multi-step reasoning tasks, the improvement is significant. If you include a reasoning field, you can either use it for logging and debugging or discard it after extraction, depending on your application's needs.

Test with Edge Cases

After designing your schema, test it with inputs that stress the boundaries. What happens when the input does not contain the information the schema asks for? The model should return null for nullable fields or reasonable defaults for required fields. What happens when the input is ambiguous? The model should choose the best-fit value from the enum, and the reasoning field should explain the ambiguity.

Common edge cases to test: empty or very short inputs (the model still needs to return a complete schema-valid response), inputs in unexpected languages, inputs with contradictory information (the model needs to make a judgment call), inputs from outside the expected domain (a medical extraction schema given a legal document), and adversarial inputs designed to confuse the model. Each of these tests reveals whether your schema is robust or whether it needs adjustment.

Pay particular attention to enum coverage. If your classification schema has enums for "positive", "negative", and "neutral", test with inputs that are mixed, sarcastic, or off-topic. Does the model map these to reasonable enum values? If not, consider adding an "other" or "unclear" enum value as an escape hatch for inputs that do not fit the primary categories.