What is an AI hallucination?

Ask a boda rider for a street he does not know. A poor one will not admit it. He nods, starts the engine, and takes you somewhere with total confidence. The ride is smooth, but the address is wrong.

An AI hallucination is that ride. The model gives you an answer that reads well, sounds certain, and is false. Nothing in the tone changes, and wrong answers come out in exactly the same style as right ones.

Once you know why this happens, it stops being a shock and becomes something you can plan around.

What the word means, and why it is a poor word

Hallucination is borrowed from medicine, where it means seeing something that is not there. Used about software it is loose, because the model is not seeing anything at all.

A better plain description is confident invention. The model fills a gap with text shaped like an answer. Some engineers prefer the word confabulation for that reason. However, hallucination is the word that stuck, so it is the word you will hear.

Why it happens

Start from what the system actually does. A large language model predicts the next piece of text, over and over. It is trained to produce writing that fits, but it is never trained to produce writing that is true.

Most of the time those two things line up. Common facts appear again and again in the text it learned from, so the fitting answer is also the correct one. However, trouble starts where the data was thin.

What does the model do when it holds nothing solid? It does not stop, because nothing inside it says stop. Therefore, it produces the most plausible looking continuation, and an invented detail usually looks more plausible than a gap. A neural network has no separate store of facts to check itself against.

There is a second cause. The model must produce something for every question, and training rewards answers that seem useful. Therefore, a system pushed hard to be helpful will guess rather than refuse. That is a design choice, and it can be tuned.

The kinds you will see

• Fake sources. A citation with a real sounding title, real sounding authors, and no such paper anywhere.
• Invented numbers. A precise figure, often with a decimal point, that nobody ever measured.
• Wrong attribution. A real quote given to the wrong person, or a real feature given to the wrong company.
• Confident dates. Real events placed in the wrong year with no hedge at all.
• Invented steps. A menu item or setting in a piece of software that does not exist.
• Made up people and places, complete with convincing detail.

Notice what these share. Every one of them is specific. Specific detail is exactly what makes an answer feel checked, and that is why hallucination is so easy to miss.

When it gets worse

The risk rises under clear conditions, so you can predict most of them.

Narrow subjects are worse than broad ones, because the training data was thin there. Recent events are worse, because training stopped at some point in the past. Long answers are worse than short ones, since each extra sentence is another chance to drift. Questions that demand exact figures, names and citations are worse than questions that ask for an explanation.

Pressure makes it worse too. Push back on a correct answer and many models will fold and change it. Ask for ten examples when only three exist and you will be handed ten.

What reduces it

Nothing removes it completely today, but several things cut it a long way down.

The strongest is grounding. Give the model the source text and tell it to answer only from that text. Many products do this by searching first and writing second, which is often called retrieval. It turns a memory test into a reading test, and reading is far more reliable.

Next comes permission to refuse, so tell the model plainly that it may say it does not know. Models follow that instruction better than most people expect.

Then narrow the question. Ask for one thing. Ask for a short answer. Ask it to mark which parts it is unsure about. And keep it on the jobs it is good at, such as drafting and rewriting, where a mistake shows up at once.

How to check an answer in one minute

1. Underline every specific claim. Names, numbers, dates, titles, laws.
2. Search for one of them word for word. A source that does not exist shows up fast.
3. Ask where the claim came from, then open the source itself rather than trusting the description of it.
4. Ask the same question again in a fresh session. Answers that change between runs are the invented ones.

That last test is the most useful of the four. A model that learned something solid tends to repeat it. A model that made something up rarely makes up the same thing twice.

What is being done about it

Work is going on in several directions. Systems look things up before answering far more often than they used to. Models are being trained to express doubt in place of a flat statement. Products are adding links back to the source, so you can check without leaving the page. Some tools now run a second model over the first answer to hunt for claims nothing supports.

Expect steady improvement rather than a cure. The behavior comes from the way these systems generate text, so it will get rarer and it will not disappear. Anybody who tells you their model never hallucinates is selling something.

What to do about it in your own work

Decide in advance which jobs may use a model without checking, and which may not. A first draft of an email is low risk. A medical dose, a legal clause, a payment instruction or a public claim is high risk.

Write that line down and share it with your team. Then keep a short list of the errors you catch, and after a month you will know exactly where your tool is weak. EVERY UNCHECKED SPECIFIC IS A RISK. That is the whole discipline.

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.