Episode 93: Many analysts

Episode 93: Many analysts

Author: Tom Chivers and Stuart Ritchie January 13, 2026 Duration: 1:18:44

Here’s a cheery one for our first episode of the year. Guess what happens when you give several sets of scientists the same dataset and ask them to answer the same question? Well, they all find the same results, right? Right!?

Sadly not. This “Many Analysts” problem has been analysed and debated in multiple different scientific fields and across several papers. We cover them in this episode. What does it tell us about the objectivity of science if different teams draw different conclusions from the exact same data?

The Science Fictions podcast is brought to you by Works in Progress magazine. Their excellent new article on how we’re living in “the golden age of vaccine development”, as discussed on the show, can be found (along with the rest of their articles on science, history, and technology), at worksinprogress.co. We’re very grateful that they support the podcast.

Show notes

* 2015 Nature commentary article on “crowdsourced research” (on racism in football)

* And the full 2018 writeup titled “Many Analysts, One Data Set

* Gelman and Loken on the “Garden of Forking Paths

* 2020 many-analysts neuroscience (fMRI) paper

* And the plan for the similar study on EEG

* 2022 PNAS many-analysts paper on the “hidden universe of uncertainty”

* 2026 critique on ideological bias from George Borjas

* 2023 critique on effect sizes vs. statistical significance

* 2025 ecology & evolution many-analysts paper on blue tits and eucalyptus

* 2025 economics many-analysts paper with results on data cleaning

* 2024 PNAS critique of many-analysts research

* Julia Rohrer’s critique of multiverse analysis

Credits

The Science Fictions podcast is produced by Julian Mayers at Yada Yada Productions.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe

Every week, Tom Chivers and Stuart Ritchie pull apart the biggest arguments and most confusing claims making headlines. Science Fictions isn't just about celebrating breakthroughs; it's a necessary dive into the messy, contested, and often surprising realities of how science actually works. You'll hear them unpack heated debates, examine questionable studies, and explore why even solid research can sometimes lead to public confusion. This podcast serves as a guide through the noise, separating compelling evidence from overblown narratives. Each episode feels like a conversation with two deeply informed friends who aren't afraid to ask tough questions, offering clarity on topics that matter. For anyone curious about the stories behind the science headlines, this is an essential listen. Tune in for thoughtful analysis that goes beyond the press release, grounded in a genuine fascination with how we know what we know.
Author: Language: English Episodes: 100

Science Fictions
Podcast Episodes
Episode 44: Asteroids [not-audio_url] [/not-audio_url]

Duration: 53:18
Last week’s episode covered a man-made existential risk to humanity—nuclear war. But what about natural risks? Could there, right now, be a vast asteroid sailing through space that’ll collide with Earth, sending us go th…
Episode 43: Nuclear winter [not-audio_url] [/not-audio_url]

Duration: 1:12:45
The UK has a new Prime Minister, and one of his first acts will have been to write letters to the captains of our nuclear missile submarines, telling them what to do in the event that the UK gets obliterated by a nuclear…
Episode 42: Election special [not-audio_url] [/not-audio_url]

Duration: 1:01:48
This week it’s the UK General Election, and lots of other countries either have elections coming soon or have recently voted. Lots of pollsters and political scientists have been attempting to predict the outcomes - but…
Paid-only Episode 9: Viagra [not-audio_url] [/not-audio_url]

Duration: 10:32
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comThere’s one thing we know Viagra does very well. But what other uses does it have? Can it, as has now been claimed in three sep…
Episode 41: Criminal justice and forensic science [not-audio_url] [/not-audio_url]

Duration: 1:01:35
The criminal justice system and science are both broadly looking for the same thing - the truth. But in many cases the two don’t mix well. Whether it’s court cases that attempt to decide the truth of a scientific dispute…
Episode 40: Addiction [not-audio_url] [/not-audio_url]

Duration: 1:05:21
To be addicted to something, you’ve got to… er, actually, what does it mean to be “addicted” to something? We all agree you can be addicted to heroin, but can you also be addicted to videogames, or sex, or listening to p…
Episode 39: Peanut allergy [not-audio_url] [/not-audio_url]

Duration: 1:04:45
Should you avoid giving your child peanuts to ensure they don’t develop an allergy? If you’d asked medical authorities this question in the late 90s and early 2000s, you’d get an answer that’s completely opposite to what…
Episode 38: Lead and crime [not-audio_url] [/not-audio_url]

Duration: 56:13
Many Western countries, most notably the US, had a major decline in their crime rate in the 1990s. About 20 years earlier, the US had banned the use of lead in gasoline. Perhaps you wouldn’t think those two facts are rel…
Paid-only Episode 8: The science of Johann Hari [not-audio_url] [/not-audio_url]

Duration: 10:36
This is a free preview of a paid episode. To hear more, visit sciencefictionspod.substack.comJohann Hari is a journalist with an interesting past who has now written four very popular books on scientific topics (addictio…
Episode 37: Lead and IQ [not-audio_url] [/not-audio_url]

Duration: 1:04:51
Petrol, pipes, paint: they made a whole generation duller. That’s if you believe the research on the effects of lead on IQ. By interfering with neurological development, the lead that we used to encounter routinely has l…