What If the AI Tells My Child Something Wrong?

Short answer: Yes, AI can be wrong — and it will be wrong in a fluent, confident voice, which is exactly what makes it hard for a child to spot. Good design reduces how often it happens and limits the damage when it does, but no one can honestly promise zero. The thing that actually protects your child isn’t a filter. It’s the habit of “shall we check that?” — and learning it at seven is worth far more than learning it at seventeen.

It’s the question we get asked more than any other, usually phrased carefully, as though it might be rude:

“What happens if it just… makes something up?”

It’s the right question. Here’s the honest answer.

Why AI gets things wrong

A language model isn’t looking anything up. This surprises people, so it’s worth being plain about it.

When your child asks “why is the sky blue?”, the AI is not consulting an encyclopaedia and reading back the entry. It is producing the words most likely to follow that question, based on an enormous amount of text it has read.

Most of the time, the most likely words happen to be the true ones — because most of what’s been written about the sky is correct. That’s why it works at all.

But the mechanism has no built-in sense of knowing versus guessing. When it doesn’t have the answer, it doesn’t stop. It produces the most plausible-sounding continuation, in exactly the same confident tone it uses for things it has right.

The useful mental image is a very well-read friend answering everything from memory, who never once says “actually, I’m not sure.”

Three different kinds of wrong

Lumping them together makes this scarier than it needs to be. They’re not the same problem.

1. The invented fact. A name, a date, a number that simply isn’t real. This is the one people mean by “hallucination”. It’s most likely on narrow, specific questions — a particular local festival, an obscure animal, the plot of a book that isn’t famous.

2. The out-of-date answer. The model learned from text up to a point in time. Ask about something that has changed since, and you get a confident answer that used to be true.

3. The badly-pitched answer. Factually fine, wrong for the listener. A technically accurate explanation of photosynthesis is wrong for a five-year-old if it leaves them feeling stupid. This one gets ignored in accuracy discussions, and it’s the kind children actually meet most often.

Which questions are actually risky

Risk isn’t spread evenly. It tracks the cost of being wrong.

Type of question	Risk	Why
”Why is the sky blue?” “Do fish sleep?”	Low	Well-covered ground, and a slightly-off answer costs nothing
Story-making, word games, “what if”	Low	There’s no fact to get wrong
Dates, populations, “how many X are there”	Medium	Exactly the specific-number territory where invention happens
Homework answers	Medium	Wrong answer copied down, plus it skips the thinking
Anything medical, any dose, any symptom	High	Don’t. Ask a pharmacist or a doctor
Today’s news, this week’s events	High	Structurally outside what the model reliably knows

Notice that the bulk of what a six-year-old actually asks sits in the top two rows. The everyday case is the low-risk case — which is a genuinely reassuring thing about young children’s curiosity, not a claim we’re making about the technology.

What we do about it

We can reduce frequency and limit blast radius. We can’t get to zero, and we won’t say we have.

• The reply is written for a child, on purpose. KidTalk is told the child’s age, and answers in two to four sentences at that level. Short answers have less room to drift than long ones, and category 3 — the badly-pitched answer — is the one this design fixes outright.
• It’s built to hand the question back. Poppy is designed to end on a hook or a question rather than a verdict. The conversational shape matters: a reply that invites “what do you think?” is one a child engages with, rather than swallows.
• It stays away from the high-risk rows. It isn’t a medical adviser, a news service, or a homework-answer machine, and it doesn’t pretend to be.
• The conversation is on the record. The full history — the actual exchange, not a summary — is kept in your account, and reading it in the app is a Premium feature. If something odd got said, it can be found and talked about rather than argued over from memory.
• There’s a daily limit. Not a billing tactic — a design position. Bounded use is safer use.

What we’re not going to tell you is that our AI doesn’t make mistakes. Any children’s AI product claiming perfect accuracy is either misunderstanding its own technology or hoping you don’t ask. See also how we think about safety generally.

The habit that does the real work

Here is the part that matters more than everything above.

Your child is going to live their whole adult life surrounded by machines that generate confident-sounding text. The protective skill is not avoidance. It’s checking. And it’s learnable young.

1. Say “shall we check that?” out loud — including when the AI is right. If you only check when you suspect an error, checking becomes an accusation. Make it routine and it becomes a reflex. A picture book, a search, a phone call to grandad — the method barely matters.

2. Ask your child to explain it back. “So what did Poppy say?” An answer they can’t reproduce was never absorbed. An answer they can reproduce is one you can now examine together.

3. Use it for questions, not answers. The best use of AI with children is generating things worth wondering about. Where it produces a finished answer to be copied down, the value drops sharply — see our take on summer projects.

4. Let them catch one. The single best moment in a child’s AI education is the day they find a mistake themselves. Don’t smooth it over. “You’re right, that was wrong — well spotted” teaches more than a hundred correct answers.

Frequently asked questions

How often does it actually get things wrong? Nobody can give you an honest number, because it depends entirely on what’s being asked. On well-trodden “why does…” questions from young children, errors are uncommon. On narrow specifics — a small local place, a precise figure, a recent event — the rate rises sharply.

Isn’t it dangerous for a child to be told something false? For most of what small children ask, an imperfect answer costs about as much as a slightly muddled explanation from an uncle. The genuine risk sits in the high-risk rows above — health especially — which is why those need an adult, every time.

Should I check every single conversation? No, and you won’t sustain it. Glance at the history now and then, and ask “what did you talk about today?” more often. The second one is worth more than the first, because it’s a conversation rather than an audit.

My child believes it completely. Is that a problem? It’s normal — young children extend trust to confident adults too. Which is why the habit of checking beats any explanation of how AI works. Show them a mistake once, and the framing changes permanently.

What if it says something upsetting rather than wrong? That’s a different problem with a different answer. Our safety post covers the content side, and how to choose an app covers what to look for elsewhere.

In summary

AI is wrong sometimes, and it’s wrong in a confident voice. That’s the honest position, from us as much as anyone.

• It predicts likely words; it doesn’t look facts up
• Everyday “why?” questions are low-risk; health and current events are not
• Design can reduce errors and cap the damage — it can’t eliminate them
• The real protection is a child who says “let’s check”

A child who learns at seven that a confident machine can be wrong is carrying something genuinely valuable into the rest of their life. Handled well, this isn’t the weak point of children’s AI. It’s one of the more useful things it teaches.