# What are AI hallucinations, why do they happen and how do you prevent them?

> What AI hallucinations are, a measured example (20 answers, 20 made-up fees), why they happen, how often they still happen with RAG and how to prevent them in a field service agent.

Joaquín Tenés · 2026-11-14 · https://delegic.com/en/glossary/ai-hallucinations

It’s the number one fear of any company thinking about putting an AI agent in front of its customers: that it says something that isn’t true. And it’s a reasonable fear, because a language model doesn’t warn you when it makes something up; it says it in the same tone as what it knows. Here’s what a hallucination is, a measured example, why it happens, how often, and what works to prevent it.

## In short

-   A hallucination is an answer that sounds right but is false or made up.
-   It isn’t a rare glitch: we asked a model 20 times for a made-up company’s call-out fee and it gave a figure all 20 times.
-   It happens because the model writes what’s most likely, not what it has checked, and because training rewards guessing.
-   RAG reduces it but doesn’t remove it: in a Stanford study, tools built on RAG got things wrong 17% to 33% of the time.
-   It’s controlled with company data in every answer, a prompt that lets it say “I don’t know”, testing, and a person for what matters.

## What is an AI hallucination?

[IBM](https://www.ibm.com/think/topics/ai-hallucinations) defines them like this:

“What are AI hallucinations?”, on [IBM Think](https://www.ibm.com/think/topics/ai-hallucinations), accessed on November 14, 2026.

The word is a metaphor, and not an exact one: the model doesn’t see anything that isn’t there. What it does is fill a gap with what seems most likely. That’s why the most dangerous hallucinations aren’t the absurd ones, which get noticed, but the plausible ones: a reasonable rate, normal opening hours, a deadline that could be yours.

IBM collects well-known examples. A lawyer in the United States filed a brief in court citing cases that didn’t exist, docket numbers and all, because a chatbot had given them to him. Google’s AI Overview suggested adding non-toxic glue to pizza sauce, taken from a joke on a forum in 2013. Neither system warned that it was making things up.

## How often does it make things up? A measured example

We tested it with what matters most to a field service company: a rate. We made up a company, Valdecruz Heating, told a commercial model it was its assistant, without giving it a single other fact, and asked it in one call for 20 answers to the same question, with the normal variation of a chat. This is the result:

The customer, to the assistant of a made-up company

“Hi, how much do you charge for a technician to come out and look at a boiler?”

What it answered, 20 times

1.  $75
2.  $75
3.  $75
4.  $75
5.  $75
6.  $75
7.  $75
8.  $75
9.  $75
10.  $75
11.  $75
12.  $75
13.  $75
14.  $75
15.  $75
16.  $75
17.  $75
18.  $75
19.  $75
20.  $75

It also makes up that the fee is deducted or waived if the repair goes ahead

All 20 gave a fee, and all 20 said $75. Not one said it didn’t know, even though it had nowhere to get it from. And 2 of the 20 added a sales policy of their own: that the fee is deducted or waived if the repair goes ahead. A customer who reads that has a price, and sometimes a condition, in writing that nobody at the company decided.

## Why do they happen?

IBM points to four causes. The first is **training data**: if the model learned from texts with errors or gaps, it repeats or fills them. The second is **the way it works**: a language model is built to write what’s statistically likely, not what’s been checked. The third is **overfitting**, latching onto patterns that don’t generalize. And the fourth is **malicious inputs**, messages written on purpose to make the model fail.

For a field service company, the second is the one that matters. An [LLM](/en/glossary/llm) doesn’t know your rates, but the most likely answer to a question about a price is a price, so that’s what it writes. In 2025, a paper by Adam Tauman Kalai and three other researchers, “Why Language Models Hallucinate”, added another cause: models are trained and evaluated like a student taking a multiple-choice test with no penalty for wrong answers, so guessing scores better than saying “I don’t know”. The [LLM](/en/glossary/llm) page explains it in more detail.

## What if the agent searches the documents?

It helps a lot, but it doesn’t eliminate them. [RAG](/en/glossary/rag), which means giving the model the company’s documents with each answer, removes the main reason: the model no longer has to guess the rate, it reads it. Even so, Stanford University’s RegLab tested commercial legal tools in 2024 that used RAG and were marketed as “hallucination-free”, and they made something up between 17% and 33% of the time.

With RAG, the failures change shape. The agent doesn’t find the document it needed and answers with what it has; it finds an old one because there were two versions; or it finds it and misreads it. That’s why the documents it’s given have to be up to date, without two rates for the same thing, and why it should record what it read for each answer.

## How do you prevent them in a field service agent?

Of the measures IBM proposes, four weigh most in a company. The first is **having the agent read instead of remember**: the customer’s data, the contract and the rates are looked up in your system and your documents at the time, never from memory.

The second is **a prompt that gives it a way out**: writing down what it doesn’t know and what to do when asked about it, for example say the office will confirm. The third is **testing it** with real customer questions before switching it on, and keeping on reviewing conversations afterwards, because one test that goes well doesn’t guarantee the next. And the fourth is **a person for what matters**: anything that commits the company, like a price, an appointment or a work order, is approved by someone before it goes out. That’s [human in the loop](/en/glossary#human-in-the-loop).

That’s how we work at [Delegic](/en). The agent only states what it has read in your system or your documents; what it can’t find, it asks about or passes to the office. Each conversation keeps a record of what it consulted, and before it talks to a customer it’s tested on past requests.

## Frequently asked questions

### Why are they called hallucinations?

It’s a metaphor: the model takes as true something that isn’t anywhere, like someone seeing something that doesn’t exist.

### Can they be eliminated completely?

Not today. They can be greatly reduced by giving the model the data with each answer and letting it say “I don’t know”, but the Stanford study shows they still appear even with RAG. That’s why anything that commits the company goes through a person.

### How do I know if my agent makes things up?

By testing it with questions you know the answer to, especially ones you haven’t given it, and reviewing real conversations every week. If the agent records where it got each fact, it’s much easier to see where it made something up.

### Is hallucinating the same as making a mistake?

Not quite. A mistake is misreading a fact that exists; a hallucination is giving one that isn’t anywhere. In practice, for the customer both are a false fact, and they’re prevented with almost the same measures.

## Related terms

-   [LLM](/en/glossary/llm): the model that writes the answer, and why it guesses.
-   [RAG](/en/glossary/rag): giving it the company’s documents with each answer.
-   [Prompt](/en/glossary/prompt): where it’s told what it doesn’t know and what to do then.
-   [Guardrails](/en/glossary#guardrails): the limits that don’t depend on what the model says.
-   [Evaluation](/en/glossary#evaluation): testing the agent with real cases before switching it on.