The short answer: AI gets things wrong most often in three ways: it makes up facts, it gives out-of-date information, and it gives one-sided answers. Before you use or share an AI answer, ask three quick questions: Is this important? Could it be out of date? Can I check it with one trusted source? If it’s important, always check.
AI assistants like ChatGPT, Gemini and Claude are helpful, fast and usually right about everyday things. But they don’t look facts up the way a person would. They predict what a good answer sounds like. Most of the time that works well. Sometimes it means an answer is confidently wrong. This guide shows you how to tell the difference, with real examples and a simple habit you can use every day.
Why AI gets things wrong
An AI assistant learned from a huge amount of text. When it doesn’t know something, it doesn’t always say so. Instead, it fills the gap with something that sounds likely. That’s why a wrong answer can look just as polished as a right one.
The people who build these tools say the same. OpenAI, the company behind ChatGPT, calls these mistakes “hallucinations,” which it describes as times when a model “confidently generates an answer that isn’t true.” In its research on why language models hallucinate, OpenAI explains that the way these models are trained and tested tends to reward guessing over admitting they’re not sure. In other words, AI is a bit like a student who writes something on every exam question rather than leaving it blank.
A real example: the lawyers and the made-up cases
In 2023, two lawyers in New York used ChatGPT to help write a court filing. ChatGPT listed several past court cases to support their argument. The cases sounded real, with names, dates and quotes, but they didn’t exist. The judge noticed, and the lawyers and their firm were fined $5,000 (Mata v. Avianca).
The lesson isn’t “never use AI.” It’s that AI can produce details that look exactly like real ones. The more specific and important a detail is, the more it needs a quick check.
The three most common mistakes
| Mistake | What it looks like | What to do |
|---|---|---|
| Made-up facts | A statistic, quote, book title or link that sounds real but isn’t | Check important facts with a trusted source |
| Outdated information | Old prices, opening hours, rules or news | Check the official website, or use an assistant that searches the web and shows sources |
| Bias | One-sided answers or stereotypes | Ask: “What other viewpoints are there?” |
Made-up facts
This is the one that catches most people. Ask for “three studies about sleep and memory” and you may get three titles, authors and years that look perfect, and one of them may not exist. Links can be broken or lead to a different page. Quotes can be close to what someone said, but not quite.
Outdated information
Many AI assistants learned from text up to a certain date. Anything that changes often, like prices, train times, tax rules, product features or who holds a job, may be out of date. Some assistants can search the web, but even then, check the date on the source they used.
Bias
AI learned from text written by people, so it can repeat people’s blind spots. You might get advice that assumes everyone lives in one country, or an example that leans on a stereotype. When a topic has more than one side, ask the AI to show the other sides too.
Warning signs to watch for
- Very specific details with no source: exact numbers, dates, names of studies or people.
- Anything that changes often: prices, schedules, laws and recent events.
- Claims that seem too neat or too surprising to be true.
- Answers that change when you ask again. If you get a different date or number each time, the AI is probably guessing.
- Links you haven’t opened. A link in an answer is only a source once you’ve clicked it and seen that it says the same thing.
Practice spotting mistakes
Step 4 of AI Essentials is a quick Spot the Mistake game.
Your 3-second check
Before you use or share an AI answer, ask yourself three questions:
- Is this important? Health, money, legal, work, or something you’ll share publicly.
- Could it be out of date?
- Can I check it with one trusted source?
If the answer to the first question is yes, always do the third. It takes seconds, and it’s the habit that separates confident AI users from everyone else.
How to check an AI answer, step by step
- Pick out the facts that matter. Names, numbers, dates, quotes and anything you’ll act on.
- Find one trusted source for each. An official website, a well-known news outlet, a government page or the original document.
- Open the links yourself. Make sure the page exists and actually says what the AI claims.
- Compare dates. Is the information current, or has it changed since?
- Fix or remove anything you can’t confirm. If you can’t check it, don’t repeat it.
What it looks like in practice
Say you ask: “What’s the best time to visit Japan to see the cherry blossoms?” The AI answers late March to early April. That’s generally right, and it’s low risk, so you can use it to start planning.
Now say you ask: “What documents do I need to enter Japan?” This is important, and entry rules change. Use the AI’s answer as a starting list, then check the official government website before you book anything.
Same tool, same day. The difference is how much is at stake and how often the facts change.
Make AI check itself
You can also ask the AI to help. Try: “Which parts of your answer are you least sure about?” or “What sources would confirm this?” It won’t replace your own check, but it often points you to the parts that need one.
Other useful follow-ups:
- “Is any of this likely to have changed recently?”
- “What would someone who disagrees with this say?”
- “Show me only the facts you are confident about, and mark the rest as unsure.”
Common mistakes to avoid
- Trusting a confident tone. How sure an answer sounds tells you nothing about whether it’s right.
- Checking with another AI only. A second AI can repeat the same mistake. Use a real source.
- Sharing first, checking later. Once a wrong fact is in a group chat or a report, it’s hard to take back.
- Giving up on AI after one mistake. AI is still great for drafts, ideas and explanations. Just check the facts that matter. A clear prompt helps too: see the 4-part prompt.
Frequently asked questions
Why does AI make things up?
AI writes the answer that seems most likely, based on patterns in the text it learned from. When it doesn’t have the right information, it can fill the gap with something that sounds right instead of saying “I don’t know.” These mistakes are often called hallucinations.
How often is AI wrong?
It depends on the tool and the question. AI is usually right about common, well-known topics, and more often wrong about specific details, recent events and niche subjects. That’s why the 3-second check focuses on how important the answer is.
Can I trust AI answers that include sources?
Sources help, but only after you open them. Make sure each link works and that the page really says what the AI claims.
What should I never rely on AI alone for?
Health, legal and money decisions, and anything you’ll publish or send to many people. Use AI to understand the basics and prepare questions, then confirm with a trusted source or a professional.
Want to go further?
Practice spotting AI mistakes in AI Essentials, our free, beginner-friendly class that ends with a certificate. It takes about 35 minutes.
