What Guardrails Does the EU AI Act Require?

How the EU AI Act Classifies AI Systems

The EU AI Act uses a risk-based classification system that determines which guardrails are mandatory. Understanding where your application falls in this classification is the first step in determining your compliance obligations.

Unacceptable risk AI systems are banned entirely. These include social scoring systems, real-time biometric identification in public spaces (with narrow exceptions), and AI that manipulates behavior to cause harm. If your LLM application falls into this category, no amount of guardrails makes it legal.

High-risk AI systems face the most extensive guardrail requirements. This category includes AI used in critical infrastructure (energy, transport, water), education (admissions, grading), employment (hiring, promotion, termination), essential services (credit scoring, insurance pricing), law enforcement, migration management, and administration of justice. If your LLM application assists with decisions in these domains, it is likely high-risk and must comply with the full set of technical requirements in Articles 8-15 of the Act.

Limited risk AI systems have transparency obligations. Chatbots must disclose to users that they are interacting with AI. Deepfake generators must label their output. Emotion recognition systems must inform the people being analyzed. Most customer-facing LLM chatbots fall into this category at minimum.

General-purpose AI (GPAI) models have separate obligations under Article 53 that apply to the model providers (OpenAI, Anthropic, Google) rather than the application developers. However, if you fine-tune a GPAI model or deploy it in a high-risk application, the high-risk obligations apply to your deployment. You inherit compliance responsibility for the application even if the model provider handles model-level compliance.

Required Guardrails for High-Risk AI Systems

Human oversight (Article 14) requires that AI systems "can be effectively overseen by natural persons." For LLM applications, this translates into guardrails that enable human review of AI decisions before they take effect, override mechanisms that let authorized humans reverse AI actions, and monitoring dashboards that give human overseers visibility into the system's behavior. The specific guardrail implementation depends on the domain: a high-risk hiring AI might require human approval of every candidate ranking, while a high-risk credit scoring AI might require human review of every denial.

The Act specifies that human oversight must be "effective," meaning the human must have sufficient information and authority to actually override the AI's decisions. A guardrail that routes decisions to a human reviewer who lacks the context, training, or authority to meaningfully evaluate the AI's output does not satisfy the requirement. Your guardrails must present the AI's reasoning (not just its conclusion), the relevant evidence, and a clear override mechanism.

Transparency and record-keeping (Article 12) requires that high-risk AI systems automatically log their operations with enough detail to enable traceability. For LLM applications, this means logging every input, every output, every guardrail evaluation, every human override, and every system configuration change with timestamps and user identities. Guardrail systems must produce audit logs that document what rules were applied, what the results were, and what actions were taken in response.

The logging requirement is more extensive than most teams implement by default. Standard application logging that records errors and exceptions is insufficient. The Act requires logging that enables post-hoc analysis of how each decision was made, which means capturing the full context that influenced each AI output, including retrieved documents, system prompts, guardrail evaluations, and human review decisions.

Accuracy, robustness, and cybersecurity (Article 15) requires that high-risk AI systems achieve "appropriate levels of accuracy, robustness and cybersecurity" and maintain them throughout their lifecycle. For guardrails, this translates into three specific requirements: accuracy monitoring (tracking the system's error rates over time and alerting when they degrade), robustness testing (verifying that the system performs correctly under adversarial conditions, including prompt injection and jailbreaking), and security hardening (protecting the guardrail system itself from attacks that could disable or bypass it).

The "throughout their lifecycle" clause is critical. Compliance is not a one-time certification but an ongoing obligation. Your guardrails must include monitoring that detects accuracy degradation over time, and you must have processes to update guardrails when new vulnerabilities are discovered or when model updates change the system's behavior.

Risk management system (Article 9) requires a documented risk management process that identifies risks, evaluates their likelihood and severity, implements mitigation measures, and monitors residual risk. Your guardrails are the technical implementation of your risk mitigation measures. The documentation must explicitly connect each identified risk to the guardrail that mitigates it, the metrics that demonstrate the guardrail's effectiveness, and the process for updating the guardrail when the risk profile changes.

GPAI Model Obligations That Affect Application Developers

If you are building on top of a general-purpose AI model (GPT-4, Claude, Gemini, or open-source equivalents), the GPAI provisions in Article 53 create obligations that flow through to your application.

Model providers must conduct and document model evaluations, including adversarial testing. As an application developer, you should verify that your model provider has completed these evaluations and understand what risks they identified. If you fine-tune the model or use it in a high-risk application, you may need to conduct your own evaluations that specifically test the model's behavior in your application context, because the provider's general evaluation may not cover your specific use case.

GPAI models with "systemic risk" (currently defined as models trained with more than 10^25 FLOPs, which includes GPT-4 and Claude-class models) face additional requirements: red teaming, incident monitoring and reporting to the EU AI Office, and cybersecurity assessments. While these are primarily the model provider's responsibility, application developers must implement incident reporting processes on their end, detecting and reporting serious incidents involving their application to both the model provider and relevant authorities.

Practical Guardrail Implementations for EU AI Act Compliance

Translating the Act's requirements into specific guardrail implementations:

For transparency: add a disclosure message to every AI interaction ("This response was generated by an AI system"). Implement a mechanism for users to request human review of any AI-generated output. Log every interaction with the detail level described above. These are straightforward technical implementations that most guardrail frameworks support.

For human oversight: build approval workflows for high-risk decisions with configurable thresholds. Implement override mechanisms that let authorized humans reverse AI actions. Create monitoring dashboards that display system behavior metrics, guardrail trigger rates, and accuracy metrics in real time. Include the AI's reasoning chain in human review interfaces, not just the final output.

For accuracy and robustness: implement continuous accuracy monitoring by comparing AI outputs against ground truth data or human evaluations on a sampling basis. Run automated adversarial testing (red teaming) on a regular schedule. Track accuracy metrics over time and alert when they degrade below acceptable thresholds. Document your testing methodology and results as evidence of compliance.

For risk management: maintain a risk register that maps each identified risk to its guardrail mitigation, the metrics that measure the guardrail's effectiveness, and the review schedule. Update the risk register when new risks are identified, when guardrails are modified, or when the model or application changes. This documentation is what regulators will request during an audit.