Episode Description

Can interpretability tell us whether a world model has learned physical concepts—or shortcuts that happen to produce convincing results?

For Episode 6 of AI Deep Dive, World Mechanics co-founder and CEO Sonia Joseph joins Rocket Drew to explore how understanding a model’s internal representations could help researchers train more accurate, reliable and data-efficient world models.

What does it mean for a neural network to represent velocity, direction or object permanence? Can researchers identify those concepts and use them to guide training? Drawing on her research at Meta, Sonia discusses interpretable architectures, JEPA and findings that challenge assumptions borrowed from language-model interpretability. They also examine why learning directly from sensor data matters—and why a self-improving language model may not automatically solve physical AI.

Chapters:

00:00 Intro

01:39 What counts as a world model?

07:12 Looking inside models for physical understanding

11:28 Interpretable training and the bitter lesson

19:24 Physics, language and understandable explanations

28:07 Surgical robots and manufacturing

31:10 Language models and recursive self-improvement

38:13 JEPA and specialized world models

45:25 What video models actually learn about physics