Peanut Butter and ___: Why AI Predicts Instead of Reasons

Try this before you read any further. Finish each line with the first word that comes to mind.

• Mary had a little ___
• Peanut butter and ___
• The sun rises in the ___
• Once upon a ___

If you grew up in the U.S., you probably said lamb, jelly, east and time. Almost everyone does, whatever their job or education.

Notice what you didn't do. You didn't weigh whether Mary might have had a puppy. You didn't reason your way to anything. Your brain matched a pattern it had seen many times and returned it before you noticed you'd made a prediction instead of reaching a conclusion.

That's the difference that matters. Prediction asks what usually comes next. Reasoning asks what's actually true. Most of the time they land in the same place, so we stop telling them apart.

AI does the first one

A large language model has been trained on more writing than any person could read in many lifetimes. It doesn't remember that material the way you remember a book, but it has encoded an enormous number of relationships among words and ideas. That's what lets it predict what people expect to hear next.

That's what it's doing, every time. Each response is the same question asked over and over: given everything so far, what comes next? AI researcher Gary Marcus has argued that these systems are built to produce plausible output, not to determine whether that output is true (Marcus, 2020).

It doesn't feel that way because the output is fluent, fast, well organized and confident. Those are the surface signs we associate with expertise. It also rarely says "I don't know." It fills the gap with the most plausible next words.

Now make it ambiguous

Try a harder round:

• The chips are ___
• A period ends a ___
• The largest mountain in Alaska is ___

This time you probably considered a few answers before picking one. Chips could be potato, poker or computer. A period could end a sentence, a school day or a hockey game. Plenty of people still say McKinley instead of Denali.

You didn't get less intelligent between rounds. The questions got more ambiguous, so several patterns competed and you picked whichever felt most likely from your own experience.

That's what happens when AI gets something wrong. It didn't break. Faced with uncertainty, it picked the most probable continuation, just as you did with "chips." Sometimes that's the truth. Sometimes it isn't. The more ambiguous the question, the more plausible wrong answers are available to land on.

Where the ambiguity lives in your business

Ambiguity isn't limited to trivia. It's in most business questions worth asking. "Which customers are at risk?" depends on what you mean by customer and by risk. "Summarize our compliance obligations" depends on which rules, which state and which year. "What should we charge?" depends on everything the model can't see.

When a question has one obvious answer, AI's prediction and the truth usually match. When it has several reasonable answers, AI will still give you one, with the same confidence. Nothing in the response tells you which kind of question you asked.

What to do with this

The point isn't that AI is unreliable. It's to understand what it's actually doing, so you can trust it where it's strong and check it where it isn't.

• Use it freely where you can verify the result yourself: drafting, summarizing, reformatting, and generating options you'll review.
• Slow down where the question is ambiguous or the stakes are real: anything involving numbers, legal or compliance exposure, money, or a decision that's hard to undo.
• Make your questions less ambiguous. Define your terms, give the context, and say what a good answer looks like. Less ambiguity means fewer plausible wrong answers for the model to land on.

The next time AI hands you a confident answer, ask yourself which round of the experiment you were in.