Why do language models sometimes reason flawlessly and other times confidently hallucinate?
Martin Casado Erik Torenberg and I explored this on the a16z podcast, and it led to an interesting thought experiment:
Take any LLM trained on pre-1915 physics. Give it unlimited compute. It will never derive Einstein’s relativity. It can refine Newtonian approximations. It can connect existing results creatively. But it cannot reject absolute time and propose spacetime - that requires creating a new conceptual structure, not navigating within an existing one.
We’ve been measuring how transformers approximate Bayesian inference (entropy within 0.1 bits, attention aligned with true posteriors). The math is surprisingly precise in controlled settings. But it also reveals the boundary between sophisticated interpolation and genuine discovery.
Thought-provoking discussion on what current architectures can and cannot achieve. Watch Youtube Video above to find more.