Episode 93: Many analysts

Episode 93: Many analysts

Author: Tom Chivers and Stuart Ritchie January 13, 2026 Duration: 1:18:44

Here’s a cheery one for our first episode of the year. Guess what happens when you give several sets of scientists the same dataset and ask them to answer the same question? Well, they all find the same results, right? Right!?

Sadly not. This “Many Analysts” problem has been analysed and debated in multiple different scientific fields and across several papers. We cover them in this episode. What does it tell us about the objectivity of science if different teams draw different conclusions from the exact same data?

The Science Fictions podcast is brought to you by Works in Progress magazine. Their excellent new article on how we’re living in “the golden age of vaccine development”, as discussed on the show, can be found (along with the rest of their articles on science, history, and technology), at worksinprogress.co. We’re very grateful that they support the podcast.

Show notes

* 2015 Nature commentary article on “crowdsourced research” (on racism in football)

* And the full 2018 writeup titled “Many Analysts, One Data Set

* Gelman and Loken on the “Garden of Forking Paths

* 2020 many-analysts neuroscience (fMRI) paper

* And the plan for the similar study on EEG

* 2022 PNAS many-analysts paper on the “hidden universe of uncertainty”

* 2026 critique on ideological bias from George Borjas

* 2023 critique on effect sizes vs. statistical significance

* 2025 ecology & evolution many-analysts paper on blue tits and eucalyptus

* 2025 economics many-analysts paper with results on data cleaning

* 2024 PNAS critique of many-analysts research

* Julia Rohrer’s critique of multiverse analysis

Credits

The Science Fictions podcast is produced by Julian Mayers at Yada Yada Productions.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe

Every week, Tom Chivers and Stuart Ritchie pull apart the biggest arguments and most confusing claims making headlines. Science Fictions isn't just about celebrating breakthroughs; it's a necessary dive into the messy, contested, and often surprising realities of how science actually works. You'll hear them unpack heated debates, examine questionable studies, and explore why even solid research can sometimes lead to public confusion. This podcast serves as a guide through the noise, separating compelling evidence from overblown narratives. Each episode feels like a conversation with two deeply informed friends who aren't afraid to ask tough questions, offering clarity on topics that matter. For anyone curious about the stories behind the science headlines, this is an essential listen. Tune in for thoughtful analysis that goes beyond the press release, grounded in a genuine fascination with how we know what we know.
Author: Language: English Episodes: 100

Science Fictions
Podcast Episodes
Episode 50: Toxoplasma [not-audio_url] [/not-audio_url]

Duration: 1:08:48
Been feeling a little strange lately? A bit impulsive, maybe? Feeling a sudden urge to get a pet cat? Sorry to say it, but maybe you’re infected with a scary mind control parasite: specifically, the paraside Toxoplasma g…
Paid-only Episode 12: Jonathan Haidt vs. social media [not-audio_url] [/not-audio_url]

Duration: 59:18
A while back, The Studies Show covered the question of whether smartphones and social media cause mental health problems. Amazingly, that podcast didn’t settle the issue, and the debate has continued—and continued rather…
Episode 49: Scientific publishing [not-audio_url] [/not-audio_url]

Duration: 1:16:13
It’s in a peer-reviewed paper, so it must be true. Right? Alas, you can only really hold this belief if you don’t know about the peer-review system, and scientific publishing more generally.That’s why, in this episode of…
Episode 48: Alcohol [not-audio_url] [/not-audio_url]

Duration: 55:19
Okay, it’s time to finally answer the question: is drinking booze good or bad? Is there really a “J-curve”, such that it’s bad to drink zero alcohol, good to drink a little, and then bad to drink any more than that? What…
Episode 47: The 25 year old brain [not-audio_url] [/not-audio_url]

Duration: 1:03:31
Everyone knows your brain hasn’t finished maturing until you’re 25. That’s so well-known, in fact, that some countries (like Scotland) have built it into their criminal justice system, giving lower sentences to under-25s…
Paid-only Episode 11: Sex education [not-audio_url] [/not-audio_url]

Duration: 10:39
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comDid your schoolteacher also do the thing with the banana and the condom? It might’ve been cringe and awkward, but just ask the…
Episode 46: The marshmallow test [not-audio_url] [/not-audio_url]

Duration: 1:12:44
It’s one of the best-known findings of psychology research: kids who can delay gratification by not eating a marshmallow will grow up healther, wiser, and more successful. But guess what? Later studies had trouble findin…
First anniversary special [not-audio_url] [/not-audio_url]

Duration: 1:09:38
We’ve now been making this podcast for a year(!). We thought we’d mark the occasion with a grossly self-indulgent look back through our favourite episodes - and our least favourites, too. We’ve still got a massive list o…
Episode 45: Air pollution [not-audio_url] [/not-audio_url]

Duration: 1:04:59
Remember when they were coming to take your gas stove away? Every so often a study about the effects of air pollution on health goes viral, and we’re reminded again that seemingly innocuous objects—like your kitchen cook…
Paid-only Episode 10: Misinformation, debunked [not-audio_url] [/not-audio_url]

Duration: 11:00
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comWe all agree that misinformation is bad. So why do we cringe when we hear prominent scientists and commentators talking about “…