Episode 93: Many analysts

Episode 93: Many analysts

Author: Tom Chivers and Stuart Ritchie January 13, 2026 Duration: 1:18:44

Here’s a cheery one for our first episode of the year. Guess what happens when you give several sets of scientists the same dataset and ask them to answer the same question? Well, they all find the same results, right? Right!?

Sadly not. This “Many Analysts” problem has been analysed and debated in multiple different scientific fields and across several papers. We cover them in this episode. What does it tell us about the objectivity of science if different teams draw different conclusions from the exact same data?

The Science Fictions podcast is brought to you by Works in Progress magazine. Their excellent new article on how we’re living in “the golden age of vaccine development”, as discussed on the show, can be found (along with the rest of their articles on science, history, and technology), at worksinprogress.co. We’re very grateful that they support the podcast.

Show notes

* 2015 Nature commentary article on “crowdsourced research” (on racism in football)

* And the full 2018 writeup titled “Many Analysts, One Data Set

* Gelman and Loken on the “Garden of Forking Paths

* 2020 many-analysts neuroscience (fMRI) paper

* And the plan for the similar study on EEG

* 2022 PNAS many-analysts paper on the “hidden universe of uncertainty”

* 2026 critique on ideological bias from George Borjas

* 2023 critique on effect sizes vs. statistical significance

* 2025 ecology & evolution many-analysts paper on blue tits and eucalyptus

* 2025 economics many-analysts paper with results on data cleaning

* 2024 PNAS critique of many-analysts research

* Julia Rohrer’s critique of multiverse analysis

Credits

The Science Fictions podcast is produced by Julian Mayers at Yada Yada Productions.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe

Every week, Tom Chivers and Stuart Ritchie pull apart the biggest arguments and most confusing claims making headlines. Science Fictions isn't just about celebrating breakthroughs; it's a necessary dive into the messy, contested, and often surprising realities of how science actually works. You'll hear them unpack heated debates, examine questionable studies, and explore why even solid research can sometimes lead to public confusion. This podcast serves as a guide through the noise, separating compelling evidence from overblown narratives. Each episode feels like a conversation with two deeply informed friends who aren't afraid to ask tough questions, offering clarity on topics that matter. For anyone curious about the stories behind the science headlines, this is an essential listen. Tune in for thoughtful analysis that goes beyond the press release, grounded in a genuine fascination with how we know what we know.
Author: Language: English Episodes: 100

Science Fictions
Podcast Episodes
Paid-only episode 23: Suicide contagion [not-audio_url] [/not-audio_url]

Duration: 10:40
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comWarning: As you can tell from the title, this podcast covers a potentially distressing topic.Recent events have had us wonderin…
Episode 86: Food and climate [not-audio_url] [/not-audio_url]

Duration: 54:54
How can both of the following be true? (1) The world has record crop harvests this year; (2) climate change is ruining crop harvests and threatening food security. Does that make sense? Is it even really a contradiction?…
Episode 85: Insectageddon [not-audio_url] [/not-audio_url]

Duration: 56:16
What is that? What is that!? What is it!? Augh! No! Not the bees! NOT THE BEES! AAAAAUGGGHHHHH MY EYES! MY EYES! AAAGGHHHH!!!!This episode is about bees. And all insects, actually—are they in the process of being wiped o…
Episode 84: Brain training [not-audio_url] [/not-audio_url]

Duration: 1:07:04
5—6—3—4—3—1—7—2In the first episode under our new podcast name (it’s now the Science Fictions podcast!), we ask whatever happened to all those games that claimed to tell you your “brain age”—games that turned into a whol…
Episode 83: Dark matter and dark energy [not-audio_url] [/not-audio_url]

Duration: 56:41
Where is most of the universe? And why don't we know yet? Yes—we’re talking about dark matter and dark energy, the mysterious stuff that’s predicted by physical theory, but which still remains elusive in experiments.Afte…
Episode 82: Paper mills [not-audio_url] [/not-audio_url]

Duration: 47:07
In our last standard episode, we talked about retraction—removing papers from the scientific literature. Well, it turns out there’s an awful lot of retraction to do, in large part due to paper mills. These are fraudulent…
Unpaywalled: Diversity training [not-audio_url] [/not-audio_url]

Duration: 1:05:08
We’re very sorry about the disrupted service over this summer! It’s been hectic with work and a house move and various things. To tide you over, here’s a formerly paywalled episode: our very first one.…If you’ve ever don…
Episode 81: Retraction [not-audio_url] [/not-audio_url]

Duration: 1:11:53
RETRACTED // In this episode of The Studies Show, Tom and Stuart discuss retraction, the process of removing articles from the scientific record. How often is it due to fraud? How many papers get retracted—and is the num…
Episode 80: Mindfulness meditation [not-audio_url] [/not-audio_url]

Duration: 1:04:59
Pay attention. Focus on your breathing. Live in the moment. Accept yourself. Do you have a self? Focus on that self. And so on. This is, of course, the practice of mindfulness meditation, which seems to be everywhere: in…
Episode 79: Cancer rates [not-audio_url] [/not-audio_url]

Duration: 52:11
Are cancer rates going up or down? It seems like depending on where you look, you’ll find different answers to the question. What’s going on here — have some writers just got it completely wrong? Is it something to do wi…