AI hallucinations: what every professional needs to know about verifying AI outputs

The most important thing to understand about AI tools

Large language models — the technology behind ChatGPT, Copilot, Claude, and similar tools — do not "know" things in the way a database knows things. They generate text by predicting, word by word, what is most likely to follow given their training data and the prompt they've received. They are extraordinarily good at generating fluent, confident, plausible text. They are not reliably accurate.

This gap between fluency and accuracy is what produces hallucinations: outputs that are confidently stated, grammatically correct, contextually appropriate, and factually wrong. The model isn't "lying" — it doesn't have intentions. It's producing the most statistically likely continuation of a sequence of words, and sometimes that continuation is incorrect.

For professionals — lawyers, accountants, healthcare practitioners, financial advisers — this is not an abstract technical concern. It's a liability issue.

What hallucinations look like in practice

Hallucinations aren't always dramatic. Often they're small errors embedded in otherwise accurate content — a case citation that doesn't exist, a regulation that was amended two years ago, a statistic from a study that never happened, a product specification that's slightly wrong. These are the dangerous ones, because they look right.

Common hallucination patterns include:

• Fabricated citations. Legal cases, academic papers, regulatory guidance, and news articles that don't exist but are presented with full citation details.
• Outdated information. AI models have training cutoff dates. Information about legislation, regulations, fees, or policies may be accurate as of the training data but wrong now.
• Plausible-sounding specifics. Numbers, dates, names, and technical details that sound correct but are invented.
• Confident summaries of non-existent content. Asked to summarise a document it hasn't been given, the model may produce a plausible-sounding summary of a fictional document.

Five techniques for catching hallucinations

1. Verify citations independently

Never rely on an AI-provided citation without checking it against the primary source. If the model cites a case, find the case. If it cites a regulation, find the regulation. This is non-negotiable for any professional work product.

2. Ask the model to express uncertainty

Prompting the model to flag areas of uncertainty — "indicate where you are less certain" or "note any claims that should be independently verified" — can surface potential errors. It won't catch everything, but it raises the model's own flagging behaviour.

3. Cross-reference with authoritative sources

For any factual claim that will be relied upon in professional work, verify it against an authoritative source — legislation.gov.au for statutes, the relevant court's database for cases, the OAIC website for privacy guidance, and so on.

4. Use a second model to check

Running the same factual claim through a different AI model and asking it to assess the accuracy can catch some hallucinations — though models can agree on the same wrong answer, so this is a supplement to, not a substitute for, primary source verification.

5. Apply domain knowledge skeptically

The most reliable hallucination detector is a human expert who reads the output critically. AI tools are most dangerous when used by people who lack the domain knowledge to recognise when something is wrong. If AI is being used in a specialised field, the reviewer needs to know that field well enough to catch errors.

Building verification into your workflows

Ad hoc verification doesn't work reliably. The answer is to build verification requirements into your AI workflows as a structural step — not something that happens if someone remembers to do it, but something that happens as a matter of process.

Your AI Acceptable Use Policy should specify verification requirements for different types of AI use. High-stakes outputs — anything going to a client, anything that will inform a significant decision, anything in a professional context — should require explicit verification sign-off. Lower-stakes uses — internal drafts, administrative tasks — may have lighter requirements.

The goal isn't to eliminate AI use. It's to make confident, accurate use of AI tools by building in the human oversight that makes the outputs trustworthy.