Does AI Hallucinate?

A potential client of our Customer Support and Product Knowledge AI Agent and asked some pertintent questions relating to hallucination and if the system can be broken.

This is a summary of my response:

Obviously the objective of an AI Agent is to ALWAYS give the right answer. If it does not have the answer it’s better for the AI Agent to say “I don’t know – let me get a live human to contact you” than make up an answer.

Personally I don’t like the term “hallucination” because it sounds like the AI agent is human which it is not and it can NEVER hallucinate. Yes, it can give wrong answers. In my case I can run an audit trail to see exactly where it obtained its data and in all cases it’s because the data the client gave me was inconsistent.
We are very careful to ensure the prompt and related instructions given to the AI Agent does not say “make up an answer” or “what do you think?” – which instanty reduces any errors by the agent.

Unfortunately we cannot guarantee the AI agent will NEVER go wrong.
Even Rolls Royce cannot guarantee their car will NEVER break down.
No one can guarantee an aeroplane will NEVER crash.
No company can guarantee their own systems will NEVER go down.
No retailer can guarantee they will NEVER have shrinkage due to shoplifting.
No doctor or lawyer can guarantee they will NEVER make a mistake.
No call centre can guarantee their human agents will NEVER get it wrong.

The difference is how we and they manage the issue – how fast we can react to ensure it does not happen again or too often. In our case to update the data used by the AI Agent – in their case to update all manuals and re-train all their staff.