The Core Problem: When AI Agents Blend Facts and Guesses
If you have ever used an AI research agent to gather information for a business decision, you have likely encountered a frustrating reality: the agent speaks about everything with the same confident tone, whether it retrieved a hard data point from a source document or quietly filled a gap with its own inference. A web page might state that a market was valued at 1.2 trillion won — that is retrieved data. The agent might then describe that market as "growing fast" — that is a conclusion the model drew. Both sentences arrive in the same paragraph, in the same confident prose, with no label telling you which is which.
For casual reading, this blending is annoying. For anything you intend to act on — investment decisions, competitive analysis, regulatory compliance, medical research — it is genuinely dangerous. You cannot build reliable downstream processes on outputs where you cannot distinguish grounded retrieval from probabilistic gap-filling.
The instinctive fix most teams reach for is a better prompt. Developers add instructions like "only state facts you can cite" or "clearly distinguish between what sources say and what you infer." This approach feels logical but consistently fails in practice. Prompts are probabilistic guardrails, not hard rules. Under pressure — longer context, ambiguous queries, conflicting sources — the model reverts to confident synthesis. The problem is not that the model lacks instructions; the problem is that you are asking the model to be the judge of its own outputs.
The Structural Fix: Take the Decision Away From the Model
The correct solution is architectural, not conversational. You need to draw a hard line through your pipeline and assign each type of work to the layer best suited for it. Large language models are exceptional at extracting, summarizing, and paraphrasing unstructured text. They are unreliable as arbiters of epistemic status — they cannot consistently and reproducibly decide whether a claim meets a threshold of factual corroboration. Code can do that reliably every single time.
The design principle is simple: the LLM extracts, deterministic code judges. Once you accept that division of labor, the architecture becomes clear.
On the LLM side of the pipeline, the model is responsible for reading source documents, pulling out candidate claims, attaching source URLs or document identifiers to each claim, and producing structured output. It is doing what language models do well: reading and organizing unstructured text. It is not deciding whether any claim is a fact.
On the deterministic code side, a rule-based system receives those structured claims and their attached provenance metadata, then applies explicit scoring logic. A claim gets tagged FACT only when it satisfies a defined rule — for example, corroboration by two or more independent sources, or confirmation from a single authoritative official API. Everything that does not meet that threshold is tagged INFERENCE. No exceptions, no probabilistic judgment, no model involved in this step at all.
Why Rule-Based Labeling Matters for Provenance
The value of moving labeling into deterministic code goes beyond accuracy. It creates what engineers call reproducibility. Because the labeling logic is a set of explicit rules rather than a neural network, the same query run against the same sources will produce the same labels every single time. This is a property that probabilistic LLM outputs simply cannot offer.
This reproducibility has several practical downstream benefits:
- Auditability: When a stakeholder asks why a particular claim was labeled a fact, you can point to a rule and a source list rather than saying "the model decided." This matters enormously in regulated industries, legal contexts, and enterprise governance workflows.
- Debugging: When something is mislabeled, you debug a rule, not a prompt. The failure mode is transparent and fixable.
- Trust calibration: Downstream consumers of agent output — whether human analysts or other automated systems — can weight
FACTandINFERENCEtags differently in their own logic, because those tags carry a defined, consistent meaning. - Hallucination containment: Because the model never has the opportunity to award itself the
FACTlabel, it cannot launder a confident-sounding inference into verified information. The gap between what the model believes and what the pipeline has corroborated becomes structurally visible.
Implementing the Pipeline in Practice
In a working implementation, the pipeline typically follows four stages. First, the LLM reads retrieved documents and emits a structured list of claims, each paired with one or more source identifiers. Second, a deduplication and normalization layer groups semantically equivalent claims from different sources. Third, the scoring function counts independent corroborating sources per claim and checks whether any source qualifies as an official authoritative feed. Fourth, the labeling function applies the threshold rules and annotates each claim with its final tag before the output is assembled.
The threshold rules themselves are configurable and domain-specific. A pipeline serving a financial research use case might require three independent corroborating sources before awarding the FACT tag, and might maintain an allowlist of official regulatory data APIs that automatically qualify. A pipeline serving a news summarization use case might accept two corroborating sources but explicitly exclude sources from the same media group to ensure genuine independence.
What remains constant across configurations is the structural principle: the rules live in code, not in the model's context window.
Moving Beyond Prompt Engineering for AI Reliability
The broader lesson from this architecture is one that engineering teams building on top of large language models are learning across many domains: prompt engineering is a tool for shaping outputs, not for enforcing invariants. When you need a guarantee — a property that holds every run, not most runs — you need to move that property out of the model and into the surrounding system.
Fact versus inference labeling is one of the clearest examples of this principle. The model is excellent at surfacing candidate claims from a corpus of documents. It is structurally unsuited to be the final authority on whether those claims are corroborated. Assigning that authority to a deterministic, rule-based scoring layer does not limit what the agent can do — it makes the agent's output genuinely trustworthy and actionable in a way that no prompt instruction can reliably achieve.
For teams building AI research agents, RAG pipelines, or any system where the distinction between retrieved evidence and model-generated inference matters, this architectural split is not an optional refinement. It is the foundation that makes the rest of the system's outputs worth relying on.