Episode 93: Many analysts

Episode 93: Many analysts

Author: Tom Chivers and Stuart Ritchie January 13, 2026 Duration: 1:18:44

Here’s a cheery one for our first episode of the year. Guess what happens when you give several sets of scientists the same dataset and ask them to answer the same question? Well, they all find the same results, right? Right!?

Sadly not. This “Many Analysts” problem has been analysed and debated in multiple different scientific fields and across several papers. We cover them in this episode. What does it tell us about the objectivity of science if different teams draw different conclusions from the exact same data?

The Science Fictions podcast is brought to you by Works in Progress magazine. Their excellent new article on how we’re living in “the golden age of vaccine development”, as discussed on the show, can be found (along with the rest of their articles on science, history, and technology), at worksinprogress.co. We’re very grateful that they support the podcast.

Show notes

* 2015 Nature commentary article on “crowdsourced research” (on racism in football)

* And the full 2018 writeup titled “Many Analysts, One Data Set

* Gelman and Loken on the “Garden of Forking Paths

* 2020 many-analysts neuroscience (fMRI) paper

* And the plan for the similar study on EEG

* 2022 PNAS many-analysts paper on the “hidden universe of uncertainty”

* 2026 critique on ideological bias from George Borjas

* 2023 critique on effect sizes vs. statistical significance

* 2025 ecology & evolution many-analysts paper on blue tits and eucalyptus

* 2025 economics many-analysts paper with results on data cleaning

* 2024 PNAS critique of many-analysts research

* Julia Rohrer’s critique of multiverse analysis

Credits

The Science Fictions podcast is produced by Julian Mayers at Yada Yada Productions.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe

Every week, Tom Chivers and Stuart Ritchie pull apart the biggest arguments and most confusing claims making headlines. Science Fictions isn't just about celebrating breakthroughs; it's a necessary dive into the messy, contested, and often surprising realities of how science actually works. You'll hear them unpack heated debates, examine questionable studies, and explore why even solid research can sometimes lead to public confusion. This podcast serves as a guide through the noise, separating compelling evidence from overblown narratives. Each episode feels like a conversation with two deeply informed friends who aren't afraid to ask tough questions, offering clarity on topics that matter. For anyone curious about the stories behind the science headlines, this is an essential listen. Tune in for thoughtful analysis that goes beyond the press release, grounded in a genuine fascination with how we know what we know.
Author: Language: English Episodes: 100

Science Fictions
Podcast Episodes
Episode 78: Worm wars and mouth bacteria [not-audio_url] [/not-audio_url]

Duration: 1:15:56
Seconds out! Round one! Ding ding! And other boxing terms. We’ve found a topic on which Tom and Stuart actually disagree. Kind of. In this episode of The Studies Show, they use the examples of “the worm wars” (does dewor…
Paid-only Episode 21: Psychological sex differences [not-audio_url] [/not-audio_url]

Duration: 10:31
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comInspired by a new Nature paper on how the “maths gap” emerges between girls and boys in school, Tom and Stuart look into the ev…
Episode 77: Anti-ageing, part 1 [not-audio_url] [/not-audio_url]

Duration: 1:05:46
Whether it’s people giving themselves goat blood transfusions in an effort to maintain their youth, or yet another influencer telling you to buy XYZ miracle supplements, anti-ageing is big business. In the first part of…
Episode 76: The four-day week [not-audio_url] [/not-audio_url]

Duration: 49:58
Would you like to do 80% of your current job but for 100% of the pay? Well, of course you would. But would it be good for the economy? It’s been suggested that companies who move to four-day working weeks have happier, l…
Episode 75: Broken windows theory [not-audio_url] [/not-audio_url]

Duration: 1:14:50
We all hate “urban decay”—graffiti, litter, boarded-up buildings. But does urban decay cause crime? That’s the premise of the “broken windows” theory: seeing a dilapidated and poorly-maintained physical environment embol…
Un-paywalled: Bicycle helmets [not-audio_url] [/not-audio_url]

Duration: 55:33
Hello everyone! Thanks to Tom’s holiday and Stuart’s job we weren’t able to record this week, so we’ve put out a classic paid episode to tide you over. We hope this goes some way to scratching your Studies Show itch.Most…
Episode 74: Neurogenesis [not-audio_url] [/not-audio_url]

Duration: 1:06:23
Can adults still grow new neurons in their brains? You’d think we might know the answer to the question of adult “neurogenesis” after more than half a century of neuroscience research. But it turns out we don’t.In this e…
Episode 73: Borderline Personality Disorder [not-audio_url] [/not-audio_url]

Duration: 1:03:24
Do you know anyone who has extremely intense relationships—one minute totally in love with someone; the next, flying into a rage and calling them the worst person ever? It’s possible that they have borderline personality…
Paid-only Episode 20: The Studies Show LIVE with Jesse Singal [not-audio_url] [/not-audio_url]

Duration: 8:32
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comDid you miss our live show last week? Good news if you happen to be a paid subscriber: we’re posting the audio in full. Listen…
Episode 72: Parenting (Part 1) [not-audio_url] [/not-audio_url]

Duration: 1:17:52
The Studies Show LIVE (with special guest Jesse Singal) is this week! Friday 9 May, Conway Hall, London, 8pm. Get your tickets AT THIS LINK or at bit.ly/tss_live. Welcome to a new series of The Studies Show, all about pa…