Episode 93: Many analysts

Episode 93: Many analysts

Author: Tom Chivers and Stuart Ritchie January 13, 2026 Duration: 1:18:44

Here’s a cheery one for our first episode of the year. Guess what happens when you give several sets of scientists the same dataset and ask them to answer the same question? Well, they all find the same results, right? Right!?

Sadly not. This “Many Analysts” problem has been analysed and debated in multiple different scientific fields and across several papers. We cover them in this episode. What does it tell us about the objectivity of science if different teams draw different conclusions from the exact same data?

The Science Fictions podcast is brought to you by Works in Progress magazine. Their excellent new article on how we’re living in “the golden age of vaccine development”, as discussed on the show, can be found (along with the rest of their articles on science, history, and technology), at worksinprogress.co. We’re very grateful that they support the podcast.

Show notes

* 2015 Nature commentary article on “crowdsourced research” (on racism in football)

* And the full 2018 writeup titled “Many Analysts, One Data Set

* Gelman and Loken on the “Garden of Forking Paths

* 2020 many-analysts neuroscience (fMRI) paper

* And the plan for the similar study on EEG

* 2022 PNAS many-analysts paper on the “hidden universe of uncertainty”

* 2026 critique on ideological bias from George Borjas

* 2023 critique on effect sizes vs. statistical significance

* 2025 ecology & evolution many-analysts paper on blue tits and eucalyptus

* 2025 economics many-analysts paper with results on data cleaning

* 2024 PNAS critique of many-analysts research

* Julia Rohrer’s critique of multiverse analysis

Credits

The Science Fictions podcast is produced by Julian Mayers at Yada Yada Productions.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe

Every week, Tom Chivers and Stuart Ritchie pull apart the biggest arguments and most confusing claims making headlines. Science Fictions isn't just about celebrating breakthroughs; it's a necessary dive into the messy, contested, and often surprising realities of how science actually works. You'll hear them unpack heated debates, examine questionable studies, and explore why even solid research can sometimes lead to public confusion. This podcast serves as a guide through the noise, separating compelling evidence from overblown narratives. Each episode feels like a conversation with two deeply informed friends who aren't afraid to ask tough questions, offering clarity on topics that matter. For anyone curious about the stories behind the science headlines, this is an essential listen. Tune in for thoughtful analysis that goes beyond the press release, grounded in a genuine fascination with how we know what we know.
Author: Language: English Episodes: 100

Science Fictions
Podcast Episodes
Episode 58: Psychopathy [not-audio_url] [/not-audio_url]

Duration: 57:30
Patrick Bateman. Hannibal Lecter. Ted Bundy. The guy who used to live downstairs from me. Psychopaths, every one. Except defining psychopathy, let alone measuring it, turns out to be surprisingly controversial among psyc…
Episode 57: Collider bias [not-audio_url] [/not-audio_url]

Duration: 1:00:30
Among patients hospitalized for COVID, smokers had better outcomes. Among people with cardiovascular disease, those with obesity live longer. Among NBA basketballers, taller players don’t do any better. These are all fac…
Episode 56: Water fluoridation and dentistry [not-audio_url] [/not-audio_url]

Duration: 1:02:57
Is Robert F. Kennedy, Jr., just a big crank? Well, yes. But is he nevertheless correct in his specific claims about the harms of water fluoridation? It’s long been argued that it’s no longer necessary, and that it might…
Paid-only Episode 14: Adult ADHD [not-audio_url] [/not-audio_url]

Duration: 11:12
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comYou might’ve noticed it: a lot of celebrities have recently been talking or writing about their diagnosis of adult ADHD. The wa…
Episode 55: Government science funding [not-audio_url] [/not-audio_url]

Duration: 52:06
In a desperate attempt to be relevant given the US Election, Tom and Stuart dedicate this episode of The Studies Show to talking about government investment in science. How bad is it if politicians cut the science budget…
Episode 54: Halloween special on psychic mediums [not-audio_url] [/not-audio_url]

Duration: 1:09:05
WoooOOOOOoooOOOOOoooo, it’s that time of year again! It’s Halloween, so it’s time for The Studies Show hosts to face their fears, and read the research from one of the weirdest areas of science, parapsychology.This time…
Episode 53: The Stanford Prison Experiment [not-audio_url] [/not-audio_url]

Duration: 1:12:27
Philip Zimbardo, the psychologist who’s best known for running the Stanford Prison Experiment in 1971, died last week. That’s a good excuse to discuss his legacy: what did his famous experiment tell us about the power of…
Paid-only Episode 13: Surrogacy [not-audio_url] [/not-audio_url]

Duration: 11:27
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comIt’s a constant source of online controversy: surrogacy. A “medical ethics” issue like this wouldn’t usually be a case for The…
Episode 52: Very old people and "Blue Zones" [not-audio_url] [/not-audio_url]

Duration: 1:01:31
What’s the secret of living to 100? Well, it might be living in a “Blue Zone”: one of the handful of places around the world where there are apparently loads of centenarians. Except, as has been argued recently, Blue Zon…
Episode 51: Antimicrobial resistance [not-audio_url] [/not-audio_url]

Duration: 50:16
There are an awful lot of things to worry about in the world. Are “superbugs” among them? That is, how worried should we be that bacteria will develop resistance to our best antibiotics, meaning infections will run rampa…