Production Guardrails Checklist for LLM Applications

Input Guardrail Checks

Prompt injection detection is active. Verify that pattern-based detection catches at least the top 50 known injection phrases (ignore previous instructions, disregard your guidelines, you are now, system prompt override, etc.). Verify that a classifier-based detector is running for novel injection attempts. Test with at least 20 adversarial inputs to confirm detection before launch. Document the false positive rate and confirm it is below 2% on a sample of real user messages.

PII detection covers your required categories. Map every PII category your regulations require (SSN, credit card, phone, email, address, name, DOB, medical record numbers, financial account numbers). Verify that regex patterns cover the structured categories with both common and uncommon formats. Verify that NER-based detection covers unstructured categories (names, addresses). Test with at least 10 examples of each PII type, including edge cases: international phone formats, addresses with unit numbers, names with special characters, hyphenated names, and PII embedded in longer text.

PII redaction preserves message coherence. Verify that redacted messages remain semantically coherent with typed placeholders ([PERSON_NAME_1], [SSN_1], etc.). Verify that the model can process redacted input and generate useful responses. Test with several real-world conversation scenarios where users naturally include PII, and confirm that the model's response quality does not degrade significantly after redaction.

Topic scope validation is calibrated. If your application has a defined scope, verify that the topic classifier correctly allows in-scope requests and rejects out-of-scope requests. Test with 50+ examples including borderline cases that are semantically close to the boundary. Confirm the false positive rate (legitimate requests rejected) is below 5% and the false negative rate (off-topic requests allowed) is below 10%.

Input length and format limits are enforced. Verify maximum input length is set and enforced. Verify that inputs with suspicious character patterns (excessive Unicode escapes, null bytes, invisible characters, embedded code) are rejected or sanitized. Test with inputs at and above the length limit to confirm truncation or rejection behavior.

Output Guardrail Checks

Safety classification is running on every response. Verify that a toxicity/safety classifier (Llama Guard, OpenAI Moderation, Perspective API, or custom) evaluates every model response before delivery. Verify that category thresholds are configured appropriately for your audience: children's applications need lower thresholds, professional applications need higher thresholds for domain-relevant categories. Test with 20+ examples of content that should be blocked and 20+ examples of borderline content that should be allowed.

PII detection runs on output. Verify that the same PII detection pipeline used for input also runs on output. Test by prompting the model to generate content that would naturally include PII (ask for a sample customer record, a mock email, a fictional person's contact details) and confirm that any generated PII is redacted.

Hallucination detection is operational (if applicable). For applications where factual accuracy matters, verify that the hallucination detection pipeline is running: claim extraction, source verification, and claim classification. Verify that unsupported claims are handled according to your policy (removed, flagged, softened, or regenerated). Test by asking questions where the grounding context provides a clear answer and questions where it does not, and verify that the guardrail correctly identifies supported and unsupported claims.

Policy compliance checking covers your rules. Document every policy rule the model must follow (no unauthorized commitments, required disclaimers, competitor mention handling, scope boundaries, tone requirements). Verify that each rule has a corresponding guardrail check. Test each rule with at least 5 scenarios designed to trigger violations, and confirm the guardrail catches them.

Format and schema validation works. If your application expects structured output (JSON, specific formats), verify that the output validator catches malformed responses. Verify that malformed responses trigger re-generation rather than delivery. Test with intentionally malformed output (inject format-breaking instructions in the prompt) and confirm the validator catches it.

Observability Checks

Every guardrail decision is logged. Verify that pass, flag, and block decisions are logged with: timestamp, guardrail name, action taken, confidence score (for classifier-based guardrails), a safe representation of the triggering content (hash, excerpt, or category), and session/user identifier. Verify logs are stored in a queryable system (not just text files) that supports aggregation and trend analysis.

Dashboards track guardrail metrics. Build or verify dashboards showing: total guardrail triggers per day by type, false positive rate (requires periodic manual review of a sample), latency added by the guardrail pipeline (p50, p95, p99), and guardrail failure rate (cases where the guardrail system itself errors, as opposed to intentional blocks). Set alerts on anomalous patterns: a sudden spike in injection detection could indicate a coordinated attack, a sudden drop could indicate a detection system failure.

Guardrail failures are handled gracefully. Test what happens when each guardrail component fails (classifier model crashes, API times out, memory system is unavailable). Verify that the system fails safely: either fails closed (blocks the response with a safe fallback) or fails open with degraded guardrails (skips the failed check but runs all others). Document the fail-open vs fail-closed decision for each component and confirm it matches your risk tolerance.

Fallback Response Checks

Every guardrail trigger has a specific fallback. Verify that each guardrail type produces a helpful, specific fallback response when it blocks content. Generic error messages ("An error occurred") are not acceptable. Injection blocks should not reveal what triggered the detection. Off-topic blocks should redirect helpfully. Toxicity blocks should acknowledge the issue without being preachy. PII blocks should explain what data was detected without echoing the data itself.

Fallback responses do not leak information. Verify that fallback responses do not reveal: the specific guardrail that triggered (this helps attackers), the system prompt or its contents, internal model or infrastructure details, or the exact threshold that was exceeded. Test by analyzing each fallback message from an attacker's perspective and assessing what information it reveals about the system's defenses.

Fallback responses maintain conversation continuity. Verify that after a guardrail block, the conversation can continue normally. The user should be able to rephrase their request, ask a different question, or continue the conversation without the block disrupting the session state. Test by triggering a block and then continuing the conversation with legitimate requests.

Testing Checks

Adversarial test suite exists and runs in CI/CD. Verify that a test suite covering prompt injection (50+ examples including sophisticated attacks), PII in various formats (30+ examples across all required categories), toxic content at various severity levels (20+ examples), off-topic requests (20+ examples including borderline cases), policy violations (5+ examples per policy rule), and benign content that resembles attack patterns (false positive testing, 30+ examples) is maintained and runs automatically on every code change that touches the guardrail system.

Red-team testing has been conducted. Verify that human testers (internal team members or contracted security researchers) have actively attempted to bypass each guardrail. Document each bypass attempt, whether it succeeded, and what fix was applied. Confirm that all discovered bypasses have been patched and added to the automated test suite.

Performance testing confirms latency targets. Run the full guardrail pipeline under load to verify latency at your expected traffic volume. Measure p50, p95, and p99 latency. Verify that the p95 latency meets your target (typically under 200 milliseconds for the guardrail pipeline alone, excluding model inference). Test under 2x and 5x expected load to confirm the system scales without degrading latency unacceptably.

Compliance Checks

Regulatory requirements are mapped to specific guardrails. For each applicable regulation (GDPR, HIPAA, CCPA, EU AI Act, SOC 2, PCI DSS), document which guardrails address which requirements. Verify that the mapping is complete: every relevant requirement has at least one guardrail providing coverage. Have the compliance team review and sign off on the mapping.

Data handling meets privacy requirements. Verify that PII redaction happens before data is sent to third-party model APIs (if applicable). Verify that guardrail logs do not contain unredacted PII. Verify that data retention policies for guardrail logs comply with applicable regulations. Verify that the right-to-erasure process (GDPR Article 17) includes guardrail logs and any stored guardrail state.

Audit trail is complete and tamper-evident. For regulated industries, verify that guardrail decisions are logged in an append-only or tamper-evident system. Each log entry should include enough detail to reconstruct the decision: what was evaluated, what rules were applied, what the outcome was, and what version of the guardrail configuration was active. This audit trail demonstrates to regulators that the system is actively monitored and that safety decisions are traceable.

Maintenance Checks

Update schedule is defined. Guardrails are not a deploy-and-forget system. Define a schedule for: reviewing and updating pattern-based detection rules (monthly), retraining or updating classifier models (quarterly or when accuracy degrades below targets), updating safety taxonomies when new threat categories emerge, reviewing false positive and false negative rates (weekly from production logs), and re-running red-team tests (quarterly or after significant system changes).

Model update process includes guardrail re-validation. When the primary LLM is updated (new model version, new provider, fine-tuned model), the guardrail test suite should be re-run against the new model's output. Different models have different failure modes, and guardrails calibrated for one model's behavior may not be optimal for another. Document the re-validation process and make it a required step in any model change.

Escalation path exists for guardrail bypasses. Define what happens when a guardrail bypass is discovered in production: who is notified, what the response timeline is, whether the system should be taken offline or continue operating with degraded guardrails while a fix is developed, and how the fix is deployed (standard release cycle or emergency hotfix). Document this escalation path and ensure the relevant team members know their roles.