The Pre-Training Wall and the Treadmill After It

The Pre-Training Wall and the Treadmill After It

Author: Adam Gordon Bell - Software Developer May 9, 2026 Duration: 56:10

I've been confusing Don with frontier-lab links late at night for a bit.

Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall.

A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion. 

Don was confused. So he came over and we made an episode about it.

 


In CoRecursive: Coding Stories, host Adam Gordon Bell sits down with software developers to explore the human narratives woven into the technology we build. This isn't a technical interview focused on syntax or best practices, but a deeper dive into the pivotal moments, creative struggles, and personal insights that shape how code comes to life. Each conversation reveals the context behind decisions, the stories of failure and breakthrough, and the often-overlooked human elements that define our craft. As a software developer himself, Adam guides these discussions with genuine curiosity, pulling out anecdotes and reflections you won't hear anywhere else. The podcast lives in the intersection of News, Education, and How-To, but through the lens of lived experience. You'll hear about the origins of influential projects, the rationale behind unconventional solutions, and the personal journeys that lead developers to their philosophies. Listening feels like pulling up a chair with thoughtful peers, offering a richer understanding of the profession that goes far beyond the screen. This is where the abstract becomes concrete, through the stories and people behind the code.
Author: Language: English Episodes: 100

CoRecursive: Coding Stories
Podcast Episodes
Tech Talk: Crafting Interpreters With Bob Nystrom [not-audio_url] [/not-audio_url]

Duration: 56:06
Bob Nystrom is the author of Crafting Interpreters. I speak with Nystrom about building a programming language and an interpreter implementation for it. We talk about parsing, the difference between compiler and interpre…
Tech Talk: Rethinking databases and Noria with Jon Gjengset [not-audio_url] [/not-audio_url]

Duration: 58:44
Can we make databases faster and remove the need for caching reads in an external cache? Can we make a distributed SQL based relational database that outperforms memcached? Jon Gjengset and the PDOS team at MIT CSAIL hav…
Abstraction and Learning with Runar Bjarnason [not-audio_url] [/not-audio_url]

Duration: 49:29
What is abstraction? Can we have a precise definition of abstraction that, once understood, makes writing software simpler? Runar has thought a lot about abstraction and how we can choose the proper level of abstraction…
Burn out and recreational coding with Jamis Buck [not-audio_url] [/not-audio_url]

Duration: 59:30
A decade ago Jamis Buck was not loving his job. He was an important open source contributor. He worked for the hottest trendiest software company at the time, 37 signals, creator of ruby on rails. He was on top of the wo…
Tech Talk: Software as a Reflection of Values With Bryan Cantrill [not-audio_url] [/not-audio_url]

Duration: 1:19:04
Which operating system is the best? Which programming language is the best? What text editor? Bryan Cantrill, CTO of Joyent says that is the wrong question. Languages, operating systems and communities have to make trade…

«1...678910