AI for the Real World: A Conversation with Yann LeCun (12 min read)

Yann LeCun acknowledges today's LLMs as useful tools, but does not see them as the path to human-level intelligence. Citing a calculation that a child awake for 4 years receives roughly 10^14 bytes through vision alone, he argues that text training alone cannot match the scale of experiencing the world.

Language is only part of intelligence, and the real core is the ability to predict the outcomes of actions. He believes LLMs are strong in domains like coding and math where symbol manipulation itself is reasoning, but are structurally weak at tasks like planning, common sense, and long-term judgment that require an internal world model.

His alternative is a world model based on JEPA (Joint Embedding Predictive Architecture). Instead of matching pixels directly, it learns only the abstract representation needed for prediction to predict the next state, removing uncertain details from the representation. He said he has pushed this direction for over 15 years, and the first 10 years were spent on pixel-level prediction, which failed.

• Reasoning is closer to a search that explores an internal model, rather than producing the correct answer in one shot.
• Chain-of-thought is an inefficient workaround that forces autoregressive models to look like they're reasoning.
• Superhuman Adaptable Intelligence is a more accurate term than AGI, with the ability to solve new, untrained problems as the standard for intelligence.

This philosophy carries over into his new company, AMI Labs. Targeting domains that deal with the physical world—industrial process control, automation, wearables, robotics, and healthcare—AMI is building a general-purpose foundation model that predicts the next state rather than the next token.

In robotics, data and embodiment lock-in are simultaneous bottlenecks. Teleoperation data is highest quality but hard to parallelize, and alternatives like UMI-style hand-held grippers, wearable rigs, cross-embodiment datasets, and simulation pipelines each have gaps to overcome. Internet video, on the other hand, is large in scale but lacks action labels, and inverse dynamics and latent action models are working to unlock it.

In a similar vein, World Labs's Marble, Google DeepMind's Genie 3, 1X, GeneralistAI, NVIDIA Cosmos, and Tesla are each pushing world models in their own ways. However, AMI differentiates itself by emphasizing causally important state representations over pixel generation.

In the end, LeCun concludes that language will continue to be used, but it is merely the interface of a thinking system.