Hallucination in production: the technical-depth most teams skip

You will not prevent hallucinations. Not fully, not reliably, not in any system that routes real user queries to a frontier language model. The technical-depth required to deal with this problem honestly starts with accepting that premise, then building detection infrastructure that treats fabricated outputs the way a security team treats vulnerabilities: as inevitable events you instrument for, classify by blast radius, and respond to with evidence. Prevention is a research goal. Detection is an engineering discipline. This piece is about the latter.

Key Takeaways

• Frontier models still hallucinate at rates between 3.1% and 19.1% depending on task and configuration. Prevention alone cannot close that gap to zero. Detection is the operational layer that actually protects users.
• Hallucination is not one failure mode. Fabrication, citation drift, speculative completion, and agentic execution errors each require distinct detection logic, and reporting them as a single aggregate metric obscures the risk surface.
• Courts have now documented roughly 1,490 decisions worldwide involving AI-hallucinated material. The penalties increasingly target failure to verify and disclose, not the hallucination itself, making the audit trail as important as the model output.
• Most state-of-the-art detection techniques (LLM-as-judge, retrieval-grounding, self-consistency sampling) require sending outputs to a second model or external service. If you care about privacy, you need to understand where detection creates new data exposure and architect around it.
• Multi-layered detection stacks combining uncertainty estimation, guardrails, and retrieval augmentation can cut hallucination rates substantially, with one report citing up to 96% reduction, but "substantially" is not "completely."

Why can't you just prevent hallucinations?

Because the mechanism that generates useful completions is the same mechanism that generates plausible fabrications. A language model predicts the next token. When the training distribution and the retrieval context contain the answer, that prediction is often correct. When they don't, the model fills the gap with something that looks correct. It does not know the difference. You do not get one without the other.

A 2026 five-model benchmark study measured frontier hallucination rates between 3.1% and 19.1%, depending on model, task family, and whether reasoning configurations were enabled. That is a real improvement over 2024 baselines, which sat between 15% and 45%. It is also nowhere near zero. If you are serving thousands of queries per day, a 3% hallucination rate means dozens of fabricated outputs reaching users before lunch.

The instinct is to add more guardrails at the prompt level: system instructions that say "do not make things up," retrieval-augmented generation to ground responses in source documents, chain-of-thought reasoning to force the model to show its work. All of these help. None of them solve the problem. They shift the distribution of failures without eliminating the failure mode. Prevention reduces the rate. Detection catches what prevention misses.

What does hallucination actually look like in production?

It looks like a confident, well-formatted, grammatically perfect response that happens to be wrong. That is the core difficulty. Hallucinations do not arrive with a flag that says "I made this up." They arrive looking indistinguishable from correct outputs.

LayerLens documented a case where a model answered a question correctly but cited the wrong source. The statement was true. The reference was fabricated. In most dashboards, that response passes every check. The user trusts it, cites it downstream, and the fabricated reference propagates.

This matters because hallucination in production is not one problem. It is at least four distinct failure modes, each with different detection requirements:

• Fabrication: the model invents facts, names, dates, or events that do not exist.
• Citation drift: the model attributes a real claim to the wrong source, or invents a source entirely.
• Speculative completion: the model fills gaps in its knowledge with plausible but unverified extrapolation.
• Agentic execution errors: in tool-use scenarios, the model calls the wrong function, passes fabricated parameters, or misinterprets a tool's response.

A single filter cannot catch all four. A regex that catches invented URLs will miss a speculative completion about a legal statute. A retrieval-grounding check that verifies facts against a knowledge base will miss an agentic tool-call with fabricated parameters. FutureAGI's 2026 guide argues persuasively that reporting hallucination by failure mode, not as one aggregate metric, is now a best practice. We agree. Each hallucination type has a different blast radius and requires different guardrail placement. Treating them as distinct attack surfaces, the way a security team models threats, is more useful than a single "hallucination score."

How bad is the citation fabrication problem?

Worse than most teams realize, because fabricated citations are specifically designed (by the training distribution, not by intent) to look real. The Digital Applied benchmark study noted that when a model is uncertain, it fills a citation slot with plausible content rather than refusing. Invented DOIs follow real DOI formatting conventions. Fabricated author names are composites of real researchers in the field. Fake journal titles sound legitimate because they are statistically plausible recombinations of real journal names.

You cannot detect these with prompt engineering. You cannot detect them by asking the model "are you sure?" (it will say yes). You can only detect them by looking up the cited source in an actual database and verifying it exists. This is a lookup problem, not a language problem, and it requires infrastructure that most teams building with LLMs have not built.

We learned this the hard way. When we first started routing queries through our stack of frontier models, we assumed retrieval-augmented generation would handle citation accuracy. It handled grounding. It did not handle the model's tendency to cite sources that were not in the retrieval set but sounded like they could have been. The fix was a separate verification layer that checks cited references against known databases before the response reaches the user. It adds latency. It is worth the latency.

What does the legal system tell us about undetected hallucinations?

It tells us the consequences are concrete, expensive, and accelerating. A public tracker maintained by GC AI shows roughly 1,490 court decisions worldwide, more than 1,000 of them in the United States, where a party relied on AI-hallucinated material and a court responded. That number was in the low hundreds eighteen months ago.

The penalty trajectory is instructive. Fortune reported in May 2026 that a federal judge in Oregon sanctioned two lawyers $110,000 after they submitted 23 fabricated citations and eight invented quotations. That is the largest AI hallucination penalty in American legal history.

But the more important pattern is what courts are penalizing. ComplianceHub.Wiki's analysis of 2026 rulings found that in several cases, the court's severity was driven less by the hallucination itself and more by the failure to disclose AI use and the failure to verify the output. Misrepresentation to a court compounds an AI quality problem into a candor violation. The hallucination is a technical failure. The failure to detect and disclose it is a professional one.

If you are building AI products for regulated industries, this pattern should inform your architecture. The audit trail, the verifiable record of what was generated, what was checked, and what was verified, is not a nice-to-have. It is the thing that determines whether an incident is a technical bug or a compliance violation.

How do detection stacks actually work?

A production hallucination detection stack is typically composed of multiple independent layers, each catching a different failure mode with a different technique. Zylos Research reported in January 2026 that multi-layered approaches combining uncertainty estimation, self-consistency checks, retrieval augmentation, and guardrails have been shown to cut hallucination rates by up to 96% in production systems. They also concede that complete elimination remains impossible. Both of those statements are important.

The layers, roughly in order of where they sit in the request lifecycle:

Pre-generation: retrieval grounding. Before the model generates a response, you retrieve relevant documents from a knowledge base and inject them into the context window. This reduces the model's need to rely on parametric memory (where hallucinations live). It does not eliminate hallucination because the model can still ignore or misinterpret the retrieved context.

At generation: uncertainty estimation. Some detection approaches extract signals from the model's internal state during generation. Semantic Entropy Probes, for example, show that hallucination-correlated uncertainty signals can be extracted from a single model's hidden state without generating multiple samples. This is meaningful for two reasons: it is computationally cheap (no need to sample the model five times to do self-consistency checking), and it does not require sending the output to an external judge model.

Post-generation: self-consistency. You generate multiple responses to the same query and check whether they agree. If three out of five responses say the same thing and two say something different, the divergent responses are hallucination candidates. This is effective but expensive (you are paying for five inferences instead of one) and adds latency.

Post-generation: LLM-as-judge. You send the generated response to a separate model that evaluates whether the response is grounded in the provided context. Noveum.ai reports that fine-tuned judge models reach roughly 85% to 90% agreement with human reviewers on RAG hallucination benchmarks. That is strong but not perfect. One in ten to one in seven hallucinated responses will slip past the judge.

Post-generation: factual lookup. For specific claim types (citations, statistics, named entities), you verify the claim against an authoritative database. This is the most reliable detection method for citation fabrication but also the narrowest in scope.

Runtime guardrails. The detection tooling landscape in 2026 has matured into specialized categories: CI-native assertion tools that run in your test pipeline, runtime guardrails that do inline blocking with sub-200ms latency, open-source self-hosted observability platforms, and regulated-domain evaluators with domain-specific benchmarks. You pick the combination that matches your latency budget, privacy constraints, and failure-mode profile.

Where does detection create privacy problems?

Almost everywhere in the standard stack. This is the part most detection guides skip, and it is the part that matters most if you are building a privacy-respecting product.

LLM-as-judge requires sending your user's output (and often the input query and retrieved context) to a second model. If that second model is hosted by a different provider, you have just created a new data exposure surface. Self-consistency sampling means generating multiple responses, all of which contain user-context-derived content, and comparing them (often on a server you control, but still expanding the surface area of where that content exists). Retrieval-grounding checks require a knowledge base that may itself contain sensitive data. Factual lookup against external databases means sending extracted claims (which may contain private information) to third-party APIs.

The honest statement is: most state-of-the-art detection requires sending outputs somewhere. The privacy-preserving alternative is to detect using internal signals. Research like Semantic Entropy Probes and the ShED-HD framework for lightweight edge-device detection points toward a future where hallucination uncertainty can be quantified from the generating model's own hidden states, without round-tripping user data to an external judge. The overhead for probe-based detection is near zero compared to multi-sample methods.

We are not there yet in production, not fully, and we are honest about that. At Selina, memory is encrypted at rest but is NOT end-to-end encrypted, because a slice of each request reaches a frontier provider at inference. Files and transfers via SelinaSEND are end-to-end encrypted. Memory is not. We state this plainly because the tension between detection and privacy is real, and pretending otherwise helps no one. Our approach is to do as much detection as possible using internal signals and to minimize what crosses inference boundaries. Non-content operational metadata is kept for a short retention window, not stored indefinitely.

Why should you report hallucination by failure mode?

Because a single aggregate hallucination rate is nearly useless for prioritization. If your blended hallucination rate is 4%, you do not know whether that is 4% citation fabrication (high severity in regulated contexts), 4% speculative completion (moderate severity, often caught by users), or 2% of each with different downstream consequences.

FutureAGI's 2026 guide makes a point we think is underappreciated: a hallucination score of zero often means no judge was attached to the generation step, not that nothing went wrong. The absence of detected hallucinations is not evidence of absence. It is evidence of a missing detection layer.

In our weekly product reviews, we break hallucination data into categories. Citation fabrication rates. Factual fabrication rates. Context-grounding failures (where the model had the right context but ignored it). Agentic tool-call errors. Each category maps to a different guardrail, a different severity level, and a different remediation. This is borrowed directly from how security teams think about vulnerability classes. You do not report "number of vulnerabilities" as a single metric. You report by type, severity, and blast radius. Hallucinations deserve the same treatment.

What does the audit trail need to contain?

Everything a regulator, a court, or your own incident-response team would need to reconstruct what happened. The legal cases make this concrete. Courts are not asking "did your AI hallucinate?" They are asking "did you check, and can you prove it?"

A minimal audit trail for a hallucination-aware production system should capture: the query (or a privacy-safe representation of it), the retrieved context used for grounding, the generated response, the detection checks that were run and their results, and a timestamp for each. If a human reviewed the output, the review decision and reviewer identity. If a detection layer flagged a potential hallucination and the system chose to serve the response anyway (because the confidence was above threshold, or because the failure mode was low-severity), that decision and the reasoning behind it.

This is not just a compliance exercise. It is debugging infrastructure. When a user reports a hallucinated response, the first question your engineering team asks is "what did the detection stack see?" If the answer is "we don't know because we didn't log it," you have a detection system you cannot improve.

The ComplianceHub analysis noted that over 35 state bar associations have now issued guidance requiring attorneys to verify AI-generated content, and multiple federal courts mandate disclosure of AI use in filings. Financial services, healthcare, and government contracting regulators are expected to follow within 12 to 24 months. The audit trail is not a future requirement. It is a current one for legal, and an imminent one for everyone else.

How mature is the detection tooling ecosystem?

Mature enough to be useful. Not mature enough to be standardized. The 2026 tooling landscape has segmented into clear categories, but there is no converged industry standard for how to measure hallucination detection quality. New academic benchmarks keep appearing (HalluScan, OpenHalDet, HalluAudio, various RAG-specific benchmarks from Amazon Science and collaborators), which is itself evidence that the research community has not settled on a trusted detection standard. Evaluation of evaluators remains an open problem.

Practically, this means you should expect to run your own evals on your own data. Off-the-shelf benchmarks will tell you something about a detection tool's general capability. They will not tell you how it performs on your specific failure modes, your specific domain, your specific prompt patterns. If you are serious about detection, you need a labeled dataset of known hallucinations from your own system, and you need to measure your detection layers against it regularly.

What should you actually build?

Start with the cheapest, highest-signal layers. Retrieval grounding (if you are not already doing RAG, start). Citation verification against known databases (cheap, high precision, narrow scope). Uncertainty estimation from model internals if your inference setup gives you access to logits or hidden states. Then add LLM-as-judge for broader coverage, accepting the latency and privacy tradeoffs.

Do not try to build a single "hallucination detector." Build a pipeline of independent checks, each targeting a specific failure mode, each producing a typed signal that gets logged and reported separately. Instrument the pipeline the way you instrument a distributed system: with metrics, traces, and alerts, broken down by failure category.

And be honest about what you are not catching. Every detection stack has blind spots. Stating those blind spots explicitly, to your team, to your users, and if necessary to regulators, is better than pretending they do not exist. The legal cases make it clear: the penalty for "we checked and missed it" is dramatically lower than the penalty for "we didn't check at all."

What do we not claim?

We do not claim to have solved hallucination. We run a stack of frontier models routed per task, and those models hallucinate. We detect a large fraction of those hallucinations before they reach users. We do not detect all of them. We log what we check and what we find. We report by failure mode, not by aggregate score. We treat each hallucination category as a distinct threat surface with its own guardrails and its own blast radius assessment.

We also do not claim that our detection infrastructure is zero-cost to privacy. As noted above, memory is NOT zero-knowledge encrypted, because inference requires data to reach a provider. We minimize what crosses that boundary. We encrypt content at rest. We keep non-content metadata for a short retention window. We are transparent about these tradeoffs because the alternative, implying that you can have both perfect detection and perfect privacy with current technology, is not honest.

Detection over prevention is not a slogan. It is an engineering position. You cannot prevent what you cannot fully control. You can detect, log, classify, and respond. That is what production-grade means.

If you want to see how this works in practice, start a free 7-day trial, no card required.

Michael C.

Founder & Principal Engineer, Selina Labs

Michael builds Selina, a privacy-first AI that remembers you across conversations. He ships security-sensitive AI in production — real attacks, real fixes, measured in minutes and dollars — and writes about privacy, security, and LLMs from that seat. Top Rated Plus and expert-verified on Upwork.

• www.upwork.com
• x.com
• medium.com
• selinalabs.io