Can You Detect AI Hallucinations Automatically

What Automated Detection Can Catch

Automated detection works best for hallucinations that involve verifiable factual claims. When the model says "the API rate limit is 500 requests per minute" and you have source documents that say 100 requests per minute, automated comparison catches the discrepancy. When the model cites a document that does not exist, automated checking catches the fabricated reference. When the model generates inconsistent responses to the same question across multiple runs, automated comparison catches the instability. These are the majority of hallucination cases, and they are well-suited to automated verification.

Source attribution checking is the most reliable automated technique. It verifies that claims in the generated response are supported by the documents or memories that were provided during generation. This catches both extrinsic hallucinations (claims with no source support) and intrinsic hallucinations (claims that contradict the sources). With good implementation, source attribution catches 60% to 70% of hallucinations at a false positive rate under 10%.

Self-consistency checking catches a different category of hallucination. By generating multiple responses to the same query and comparing the factual claims across responses, the system identifies claims that the model is not confident about. A specific date that changes between responses, a number that shifts, or a name that varies across runs are all strong indicators of fabrication. Self-consistency is particularly effective for detecting fabricated proper nouns, specific statistics, and precise numerical values, which are the hallucination types that cause the most real-world damage.

Entity verification catches fabricated references by checking whether named entities in the response (people, organizations, products, API methods, configuration values) actually exist in the knowledge base or knowledge graph. This catches the specific and damaging case of the model inventing plausible-sounding entities that do not exist: a library method that was never implemented, a configuration parameter that does not exist, or a research paper that was never published. Entity verification is fast (graph lookup rather than semantic comparison) and has very low false positive rates.

What Automated Detection Misses

Automated detection struggles with subtle hallucinations that are close to true but not quite right. A claim that misattributes a statistic to the wrong year, confuses two similar entities, or slightly distorts a nuanced position may pass automated checks because the key terms and semantic content match the source material closely enough. The error is in the detail, not the topic, and automated systems that check for semantic similarity rather than exact factual correspondence miss these cases.

Hallucinations about subjective or qualitative claims are also hard to detect automatically. If the model claims that a tool is "widely adopted" when adoption is moderate, there is no clear factual boundary for automated checking. If the model overstates the certainty of a research finding, the overstatement is a subtle distortion rather than a clear fabrication. These nuanced errors require human judgment to evaluate.

Synthesis hallucinations, where the model combines real facts from different sources to create a false composite claim, are the hardest category for automated detection. Each individual fact in the claim is true, but the combination is false. "PostgreSQL 15 supports the performance improvements shown in the latest benchmarks" might combine a real product (PostgreSQL 15) with real benchmarks (from PostgreSQL 16) to create a claim that is wrong despite both components being individually verifiable. Detecting these requires understanding not just whether facts are present in the sources but whether the specific combination of facts is supported.

Finally, automated detection cannot reliably catch hallucinations of omission, where the model leaves out critical context or caveats that change the meaning of a true statement. "The API supports authentication" is technically true but omits that authentication is required and that unauthorized requests are rejected with no error message. Omission hallucinations are invisible to systems that check for the presence of false claims because no false claim is present, the problem is what is missing.

Detection Techniques Compared

Each automated detection technique has different strengths, costs, and coverage areas. Source attribution checking is the cheapest and fastest, requiring only a comparison between the response and the already-retrieved context. It catches 60% to 70% of hallucinations and adds under 500 milliseconds of latency. Self-consistency checking is more expensive because it requires multiple generation calls (3 to 5 times the normal cost), but it catches fabricated specifics that source attribution misses and adds 2 to 5 seconds of latency. Entailment verification using an NLI model is moderately expensive and catches nuanced contradictions between claims and sources that semantic similarity misses. Entity verification is fast and cheap for systems that already have a knowledge graph, catching fabricated references with near-zero false positives.

The combination of all four techniques catches 75% to 85% of hallucinations. The remaining 15% to 25% are the subtle, synthesis, and omission hallucinations that require human judgment. For most applications, the automated 75% to 85% catch rate is sufficient when combined with a general disclaimer that AI output should be verified for critical decisions.

Practical Implementation

The most practical approach is tiered detection. Run source attribution checking on every response, which catches the majority of hallucinations quickly and cheaply. Apply self-consistency checking and entailment verification selectively to high-risk responses (those about specific facts, those with low retrieval quality scores, those in sensitive domains). Route the highest-risk subset to human review. This tiered approach catches most hallucinations automatically while focusing expensive human attention on the cases that automated detection cannot handle.

Trigger conditions for the higher tiers matter for cost efficiency. Good triggers include: queries that contain specific entity names or version numbers (high fabrication risk), queries where the retrieval step returned low-similarity results (suggesting the knowledge base may not cover the topic), queries in sensitive categories (pricing, compliance, security), and queries where the generated response contains claims not present in any retrieved passage (flagged by the first-tier source attribution check). These triggers keep the expensive detection techniques focused on the responses most likely to contain hallucinations.

Persistent memory enhances automated detection by providing a richer verification source. Instead of checking claims only against the documents retrieved for the current query, you can also check against the accumulated memory store, which contains verified facts from previous interactions. This broader verification surface catches hallucinations that the current query's retrieval missed, because the memory store may contain relevant verified facts that were not retrieved for this specific query but are still available for verification. Over time, the memory store becomes an increasingly comprehensive verification database, catching more hallucinations as it grows.