Generative AI has an unusual capacity to be impressively useful and convincingly wrong at the same time. It can analyse a lengthy document, identify patterns across complex information and produce a clear explanation in seconds, then confidently provide a statistic that does not exist or attribute a finding to a paper that was never published. In other circumstances, the individual facts may be correct while the framing systematically privileges one interpretation, overlooks a particular group or reproduces assumptions contained in the information on which the system was trained.
These problems are often collected under a general warning that AI can make mistakes, but that description is not particularly helpful because different kinds of error arise for different reasons. A fabricated reference is not the same problem as an unrepresentative training dataset, and neither is the same as a model drawing an unjustified causal conclusion from accurate information. If we want to check AI-generated work properly, we first need to identify what kind of failure we are trying to detect.
Two of the most commonly discussed are hallucination and bias.
Hallucination is not simply another word for error
The term “hallucination” is now used so broadly that it sometimes means little more than an AI system being wrong. A more useful definition is narrower. A hallucination occurs when a generative model produces information that appears to be supported by reality or the available evidence but is not.
The classic example is the fabricated academic citation. Ask a model for research supporting a particular proposition and it may produce an entirely credible combination of author names, article title, journal, year and page numbers. Every element resembles the structure of a legitimate reference, yet the paper may not exist.
The same process can produce invented legal cases, fictional quotations, non-existent statistics, incorrect product features or details that were never contained in a supplied document.
These errors become easier to understand once we stop imagining the model as consulting an internal encyclopaedia. A language model generates sequences that fit the context it has been given. If the context strongly implies that a citation should come next, producing something with the characteristics of a citation is consistent with the model’s generative task even when it cannot establish that the citation corresponds with a real publication.
This is why telling a model not to hallucinate is a weak control. It is equivalent to instructing an analyst not to make mistakes without changing the conditions under which mistakes occur. Better controls change the task by supplying authoritative material, allowing retrieval from reliable sources, requiring claims to be linked to evidence, permitting the model to identify missing information and independently checking claims where accuracy matters.
Some hallucinations are easier to detect than others
An obviously impossible claim tends not to survive for long. The more difficult errors are those that sit comfortably inside what we already believe.
Imagine an AI-generated report stating that a particular government programme reduced administrative costs by 18 per cent. The number is neither implausibly large nor suspiciously small. It has the appearance of precision, and if the surrounding paragraph is well written there may be little reason for a reader to stop.
The important question is not whether 18 per cent sounds reasonable. It is where the 18 per cent came from.
The same applies to quotations. A sentence may sound exactly like something a particular economist, executive or public figure would say, but resemblance is not provenance. Unless the quotation can be traced to a source, its plausibility provides no evidence that the person actually said it.
AI-generated factual claims therefore deserve a different kind of reading from ordinary prose. Rather than asking only whether a statement makes sense, ask whether it is the kind of statement that requires external support and, if so, whether that support can be established.
Bias is a different problem
Bias is often discussed as though it means an AI system occasionally producing an offensive or prejudiced answer. That is one possible manifestation, but it is a much narrower conception than is useful for understanding how computational systems affect decisions.
Bias can enter at several points.
The information used to train a model may overrepresent some populations, languages, professions, countries or perspectives. Historical data may encode previous patterns of discrimination. A dataset collected for one purpose may be poorly suited to another. The objective used to optimise a system may privilege one outcome while ignoring another. The examples used to evaluate performance may not reflect the people who will actually encounter the system.
Even a model that performs identically across groups can be embedded in a process that produces unequal consequences.
Consider a system used to identify employees considered at risk of leaving an organisation. Historical data might show that people with particular employment histories are more likely to resign. The statistical relationship may be real, but using it operationally could reproduce the consequences of previous organisational practices rather than reveal an intrinsic characteristic of the employees themselves. If certain workers historically received fewer promotion opportunities and consequently left at higher rates, a model trained on that history may learn the outcome without understanding the institutional process that produced it.
The model does not need to contain an explicit prejudicial rule for bias to matter.
Bias can also come from the question
Not every problematic output originates in training data. Sometimes the prompt has already determined the direction of the answer.
Ask:
Why are younger employees less loyal to their employers?
and the model is being asked to explain a proposition that has not yet been established.
A useful response would challenge the premise, ask how loyalty is being measured and consider whether observed differences could instead reflect labour market conditions, occupational composition, tenure, wages or other factors. A less careful response may accept the proposition and generate a persuasive collection of reasons for it.
This matters because generative AI is exceptionally capable of explanation. Once a premise is supplied, the model can often construct a coherent account around it, even when the premise itself deserves scrutiny.
Checking AI work therefore begins before the answer is generated. Ask whether the question contains an assumption that should itself be tested.
Correct facts can still produce a biased answer
Fact checking alone cannot address every form of bias.
Imagine asking an AI system to summarise the effects of a major infrastructure project. Every statistic in the response might be correct, but the analysis could focus heavily on construction costs and travel-time benefits while saying little about displacement, accessibility, environmental effects or distributional consequences.
There need not be a fabricated fact anywhere in the answer. The bias may lie in what was selected, what was omitted and which outcomes were treated as important.
This is a familiar problem in human analysis as well. Every analytical framework places boundaries around what is considered relevant. Cost-benefit analysis, financial analysis, environmental assessment and social impact analysis can examine the same project and legitimately focus on different dimensions.
The additional difficulty with generative AI is that the analytical frame can be largely invisible. A fluent response may arrive with its categories already selected, making it easy to accept the structure of the answer without asking why those particular considerations were included.
Checking the work therefore requires examining not only whether individual claims are correct, but whether important perspectives, variables or affected groups are missing.
Start checking at the level of claims
The simplest method is to break an important AI-generated answer into claims.
Consider:
Remote work increases productivity because employees experience fewer interruptions, spend less time commuting and can structure their working day more efficiently.
This apparently straightforward sentence contains several propositions. Remote work increases productivity. Employees experience fewer interruptions. Reduced commuting contributes to productivity. Greater scheduling flexibility improves efficiency. The word “because” also makes a causal claim connecting those mechanisms to the outcome.
Checking the sentence means examining those propositions separately rather than searching for a source that vaguely discusses remote work.
Some claims may have strong empirical support, others mixed evidence, and some may depend heavily on occupation, organisation, measurement period or the particular form of remote work being studied.
This claim-level approach is particularly useful because AI-generated prose can compress several steps of reasoning into a sentence that reads as though it contains one established fact.
Check the source, not merely the citation
When AI provides a source, verify three things.
First, does the source exist?
Second, does the source contain the information attributed to it?
Third, does that information support the claim being made?
The third question is where superficial checking often fails. A genuine study may be cited correctly by title while being used to support a proposition its authors did not establish. An observational relationship may become a causal finding. Results from one country may be generalised internationally. A study of a small group may be described as evidence about an entire population. A qualification appearing in the original paper may disappear entirely from the AI-generated summary.
For quantitative claims, go one step further and check the definition. “Employment”, “unemployment”, “productivity”, “poverty”, “business failure” and “housing affordability” all have multiple possible measures. A number can be accurately copied from a source and still be inappropriate for the question being answered.
Checking provenance is therefore more than confirming that a hyperlink works.
Check calculations differently from facts
When AI produces a calculation, the useful question is not primarily whether the final number looks right. It is whether the inputs and operations can be reconstructed.
Suppose a model estimates that a proposed initiative will save $2.4 million annually. Rather than asking the model whether it is confident, identify the components of the estimate.
How many people are affected? What is the assumed saving per person? Over what period? Is implementation cost included? Are nominal and real values being mixed? Has the same benefit been counted twice? Which values are observed and which have been assumed?
A transparent calculation can be independently reproduced. An opaque number cannot.
For important quantitative work, it is often better to ask AI to show the inputs, equations and assumptions in a structured form, then perform or verify the arithmetic using appropriate analytical software. Language models can assist with quantitative reasoning, but there is little reason to rely on generated arithmetic when deterministic calculation is readily available.
Ask what is missing
A strong review does not only look for false statements. It looks for absent information that would materially change the conclusion.
If AI recommends a new product strategy, ask what customer information is missing. If it evaluates a policy, ask which affected populations have not been considered. If it interprets a research result, ask about alternative explanations. If it summarises a debate, ask which credible positions are absent.
One particularly useful instruction is:
Identify information that, if available, could materially change this conclusion.
Another is:
What assumptions does this analysis depend on, and which have not been established by the evidence provided?
These questions force attention away from the polished surface of the answer and towards its evidentiary structure.
Use AI to criticise AI, but do not confuse that with verification
One model can be useful for reviewing the output of another, and the same model can often identify weaknesses in its own previous answer when explicitly asked to adopt a critical stance.
You might ask it to identify unsupported claims, challenge assumptions, look for contradictory evidence, test a calculation or rewrite an argument from an opposing perspective. For lengthy documents, this can be an efficient first-pass quality-control process.
But AI reviewing AI is not independent verification.
If both analyses depend on similar training patterns or the same supplied information, the second model may reproduce the first model’s mistake. Agreement between two language models does not establish that either has consulted the relevant evidence.
AI review is best treated as a method for finding things worth checking rather than as the final check itself.
Not everything requires the same level of checking
The appropriate review process depends on what the output will be used for.
A brainstorm for internal discussion can tolerate speculative ideas because speculation is the purpose of the exercise. A draft paragraph can be reviewed by reading it. A summary should be compared with the source. A market statistic should be traced to current data. An academic claim should be checked against the research. A consequential legal, financial or health decision requires considerably stronger safeguards and appropriate specialist judgement.
This proportional approach is important because checking every sentence generated by AI with equal intensity would remove much of its practical value. The objective is not maximum scepticism. It is matching the strength of verification to the nature and consequence of the claim.
A useful hierarchy is to ask whether the output is creative, transformational, factual, analytical or consequential.
Creative work can often be judged directly. Transformational work should be checked against the source. Factual work requires provenance. Analytical work requires both evidence and scrutiny of assumptions. Consequential work requires all of these, together with appropriate human or institutional oversight.
A practical checking routine
For substantive AI-generated work, the checking process can be relatively simple.
Begin by identifying the claims that would matter if they were wrong. Numbers, quotations, citations, dates, named research findings and statements about current events deserve immediate attention.
Then examine the evidence supporting those claims. Follow the source rather than trusting the citation, and check whether the source supports the proposition actually being made.
Next, separate evidence from interpretation. Look for causal language, broad generalisations and conclusions that travel further than the underlying information permits.
After that, examine the frame. Ask which groups, variables, explanations or perspectives might have been omitted and whether the question itself introduced assumptions that shaped the result.
Finally, apply a consequence test. Decide what additional verification is warranted given what will happen if the answer is wrong.
None of this requires treating every AI interaction as a research project. It requires recognising that different outputs create different evidentiary obligations.
Generative AI is powerful partly because it can turn incomplete information into coherent language. That is enormously useful when the task is drafting, organising, comparing, hypothesising or developing possibilities. The same characteristic becomes a weakness when coherence is mistaken for evidence.
Hallucination reminds us to ask whether a claim is grounded. Bias reminds us to ask how the problem has been framed, whose experience is represented and what may have been left out. Checking the work requires both.
The central discipline is not to distrust everything the machine produces, but to avoid giving plausibility more evidentiary weight than it deserves. The question worth asking is not simply whether an answer sounds right, but what would allow us to establish that it is.