Classification and Enum Outputs with LLMs

Why Enums Are the Foundation of LLM Classification

Classification is the task of assigning a label from a fixed set of categories to an input. Sentiment analysis (positive, negative, neutral), support ticket routing (billing, technical, account, general), content moderation (safe, warning, block), and intent detection (purchase, return, question, complaint) are all classification tasks. The defining characteristic is that the output is one of a known set of labels, not free text.

Without structured output, LLM classification is unreliable in subtle ways. You prompt the model to classify a support ticket into one of your departments, and it returns "Technical Support" when your system expects "technical". Or it returns "billing/payment" when your enum has "billing" and "payment" as separate categories. Or it returns "I would classify this as technical because..." when you expected just the label. Each of these responses is a reasonable interpretation of the prompt, but none matches the exact enum value your downstream code requires.

Enum constraints in structured output eliminate all of these variations. You define the allowed values in the schema: {"type": "string", "enum": ["billing", "technical", "account", "general"]}. The constrained decoding engine ensures the model returns exactly one of those four strings. Not "Technical", not "tech", not "billing/payment", not a sentence explaining the classification. Exactly one of the four enum values, every time, with a 100% match rate against your downstream code.

This is not a small improvement. It is the difference between a classification system that requires string normalization, fuzzy matching, and fallback logic, and one that returns a directly usable label. The code that consumes the classification result goes from a chain of if/else statements and string comparisons to a simple switch on the enum value.

Single-Label Classification

The simplest and most common classification pattern assigns exactly one label to each input. The schema has an enum field for the label and optionally a confidence score and reasoning field.

A well-designed single-label classification schema includes three fields: "category" (the enum with your labels), "confidence" (a number between 0 and 1 indicating how confident the model is), and "reasoning" (a string explaining why this category was chosen). The confidence and reasoning fields are optional but highly recommended. Confidence lets you filter low-confidence classifications for human review. Reasoning lets you debug classifications that seem wrong and provides audit trails for regulated industries.

When designing the enum values, use clear, mutually exclusive labels that are unlikely to confuse the model. "billing" is better than "payment_issues" because it is simpler. "technical_bug" is better than "tech" because it is more descriptive. Include an "other" or "uncategorized" value as an escape hatch for inputs that do not fit any category. Without this escape hatch, the model is forced to assign every input to a category, even when none of them fit, which produces incorrect classifications that look valid.

The number of categories affects classification accuracy. With 3-5 categories, accuracy is typically 90-95% even with a simple prompt. With 10-20 categories, accuracy drops to 80-90% and requires more detailed category descriptions in the prompt. With 50+ categories, accuracy drops further and you should consider a hierarchical approach (described below) or a two-stage classification that narrows the candidate set before the final classification.

Multi-Label Classification

Multi-label classification assigns multiple labels to a single input. A support ticket might be both "billing" and "technical" if the customer has a billing issue caused by a technical bug. A news article might be tagged with "politics", "economy", and "international". Content moderation might flag "violence" and "profanity" on the same input.

The schema pattern for multi-label classification uses an array of enum values instead of a single enum field: {"type": "array", "items": {"type": "string", "enum": ["billing", "technical", "account", "general"]}}. The model returns an array containing zero or more of the allowed labels. Each label in the array is guaranteed to be one of the enum values, and there are no duplicates if you add that constraint in your validation layer.

Multi-label classification requires more careful prompt engineering than single-label because the model needs to understand that it should return all applicable labels, not just the most relevant one. Instructions like "assign all categories that apply to this input" and "return an empty array if no categories apply" make the model's task clear. Adding a confidence score per label (by using an array of objects with label and confidence fields instead of an array of strings) gives you even more control over the classification output.

Hierarchical Classification

Hierarchical classification handles large category sets by organizing them into levels. Instead of one enum with 50 values, you have a top-level enum with 5 categories and a second-level enum with 10 sub-categories per top-level category. The schema uses nested objects: a "category" enum for the top level and a "subcategory" field whose valid values depend on the chosen category.

The challenge with hierarchical classification in structured output is that most providers do not support conditional schemas (if/then/else) in constrained decoding. You cannot define a schema where the valid subcategory values change based on the category value. The workaround is to list all subcategory values in a single flat enum and use application-level validation to check that the subcategory is valid for the chosen category. Alternatively, use a two-pass approach: first classify into the top-level category with one model call, then classify into the subcategory with a second call that only includes the relevant subcategories in its enum.

The two-pass approach is slightly more expensive (two model calls instead of one) but produces higher accuracy because each classification step has fewer options to choose from. For category trees with 50+ leaf nodes, the accuracy improvement from the two-pass approach typically justifies the additional cost.

Confidence Scores and Calibration

Adding a confidence field to your classification schema gives the model a way to express uncertainty. The field is typically a number between 0 and 1, though some schemas use a 1-5 or 1-10 scale. The model produces a value that represents how confident it is in its classification.

It is important to understand what LLM confidence scores are and are not. They are not calibrated probabilities in the statistical sense. A confidence of 0.9 does not mean the model is correct 90% of the time when it says 0.9. LLM confidence scores are better understood as the model's self-assessment of how clear-cut the classification is: high confidence means the input clearly belongs to one category, low confidence means the input is ambiguous or could belong to multiple categories.

Despite being uncalibrated, confidence scores are useful for operational decisions. Setting a threshold (for example, routing all classifications with confidence below 0.7 to human review) effectively creates a semi-automated system where the model handles clear cases and humans handle ambiguous ones. This is often the best production setup: the model processes the 80% of cases that are straightforward, and humans focus on the 20% that require judgment.

To improve the usefulness of confidence scores, include the reasoning field. When the model explains why it chose a category and why it assigned a particular confidence, you can evaluate whether the confidence is appropriate. A low confidence with a reasoning of "the text mentions both billing and technical issues" tells you the input genuinely belongs to multiple categories. A low confidence with a reasoning of "the text is very short and does not contain clear signals" tells you the input needs more context.

Enum Design Best Practices

Use descriptive labels: "customer_retention" is better than "cr". "technical_infrastructure" is better than "tech_infra". The model reads the enum values and uses them to understand what each category means. Abbreviations and codes reduce classification accuracy.

Make labels mutually exclusive: If "billing" and "payment" are separate categories, clearly define the boundary in the prompt. Better yet, combine them into a single category unless the distinction is operationally important. Overlapping categories force the model to make arbitrary distinctions that reduce consistency.

Include an escape hatch: An "other", "uncategorized", or "not_applicable" value prevents forced misclassification. Without it, the model assigns every input to a category even when none of them fit, producing false positives that are harder to detect than explicit "other" labels.

Order matters for tiebreaking: When the model is unsure between two categories, it may be influenced by the order of enum values. Put your most common or most important categories first. This is a minor effect but can be significant at the margins.

Add descriptions in the prompt: For enums with more than 5-6 values, include a brief description of each category in the system prompt. This gives the model more context than the label alone and improves accuracy on ambiguous inputs. "billing: charges, invoices, refunds, payment methods" is more helpful than just "billing".