We are all dazzled by generative AI. Large Language Models (LLMs) seem capable of anything, from writing poetry to solving complex math problems. But what if the approach driving this revolution is fundamentally flawed?

Richard Sutton, one of the “founding fathers of reinforcement learning (RL)” and the recipient of the Turing Award (the Nobel Prize for computer science), is convinced that LLMs are a “dead end”.

It’s a bold claim. Here’s a non-technical look at why this AI legend thinks we’re missing the basic essence of intelligence.

1. Mimicking People vs. Understanding the World

Sutton argues there are two very different conceptual ways to approach AI.

LLMs are focused on mimicking people. They learn from trillions of tokens of text—a record of what humans said and did. Their goal is essentially “next token prediction”, which means they are very good at predicting what a person would say.

RL, which Sutton considers “basic AI”, is fundamentally about understanding your world. RL agents learn from experience, meaning the things that actually happen in their life.

Sutton questions the common belief that LLMs possess a robust “world model”. He argues that they predict what a person would say, but they lack the ability to predict what will happen in the physical world.

2. The Crucial Difference: Having a Goal

For Sutton, the most fundamental flaw in LLM design is the lack of a genuine goal.

He cites John McCarthy’s view that intelligence is “the computational part of the ability to achieve goals“.

• LLMs Don’t Have Substantive Goals: Predicting the next token is not considered a substantive goal because it doesn’t change the world or influence the tokens coming at the system. You can’t say a system has a goal if it is merely happy predicting accurately.

• RL Requires Ground Truth: In the LLM framework, there is “no definition of actual knowledge” and “no ground truth” for what is right or wrong. In contrast, reinforcement learning has a clear definition: the right thing to do is the thing that gets you reward. This allows RL agents to check their prior knowledge against actual outcomes. Without a goal, you’re just a “behaving system,” not an intelligent one.

3. The Power of Pure Experience

Sutton makes a sharp distinction between LLMs learning from static “training data” and agents learning from continuous experience.

LLMs learn from examples of what a person did, implicitly suggesting that the model “should do what the person did”. This is imitation learning.

Sutton stresses that in nature, supervised learning doesn’t happen broadly. Squirrels learn about the world through trial-and-error, not by going to school or receiving examples of desired behaviour.

The truly “scalable method” for intelligence, according to Sutton, is to learn from experience: “You try things, you see what works. No one has to tell you”. This is the experiential paradigm.

4. The Bitter Lesson and the Future of AI

Sutton is famous for his essay, “The Bitter Lesson,” which observed that methods that scale with computation consistently outperform those relying on embedding human knowledge.

While LLMs rely on massive computation, they also incorporate “lots of human knowledge” via the Internet text they are trained on. Sutton expects that systems capable of learning entirely from experience will eventually supersede LLMs, marking yet another instance of the Bitter Lesson in AI history.

He views the RL paradigm—a continuous stream of sensation, action, and reward—as the foundation and focus of intelligence.

This continual learning capability is something “all mammals have”, and it suggests that future human-level general AI (AGI) must be capable of learning throughout its deployment, rather than relying solely on a fixed training period.

The Four Pillars of Continual Learning

A continually learning agent, unlike an LLM, is designed to live and learn in the world. It requires four core components:

• Policy: What to do in the current situation.

• Value Function: A prediction of how well things are going (long-term outcome prediction, often learned via TD learning).

• Perception: Construction of the system’s sense of its current state.

• Transition Model of the World: The agent’s belief about the consequences of its actions—its “physics of the world”. This model is learned very richly from all sensation, not just from the specific reward it receives.

Sutton’s message is clear: while LLMs are impressive, true intelligence comes from actively engaging with, predicting, and changing the real world based on consequences, not just on mimicking human conversation.