Bonus: Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Bonus: Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Author: 80,000 Hours June 5, 2026 Duration: 1:29:32

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016. 

But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously. 

In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:

  • These AI systems may develop dangerous long-term goals we don’t want.
  • To pursue these goals, they may seek power and undermine the safeguards meant to contain them.
  • They may even aim to disempower humanity and potentially cause our extinction.

This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.

You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website: 

https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/ 

Chapters:

  • Risks from power-seeking AI systems (00:01:00)
  • Introduction (00:01:17)
  • Summary (00:03:09)
  • Why are the risks from power-seeking AI a pressing world problem? (00:04:04)
  • Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)
  • Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)
  • Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)
  • Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)
  • Section 5: Work on this problem is neglected and tractable (00:47:37)
  • Section 6: What are the arguments against working on this problem? (00:59:20)
  • Section 7: How you can help (01:25:07)
  • Thank you for listening (01:28:56)

Audio editing: Dominic Armstrong
Production: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore


This curated collection from the archives of The 80,000 Hours Podcast on Artificial Intelligence (September 2023) pulls together ten essential conversations that cut through the usual hype and panic. It’s a deep dive into the societal forces, ethical dilemmas, and potential trajectories of AI, framed through perspectives often concerned with the very long-term future. You’ll hear from researchers and thinkers grappling with questions that go far beyond today’s headlines, examining what it means to navigate this technology responsibly on a global scale. The discussions naturally explore themes from longtermism and existential risk to the practical insights of effective altruism, offering a structured way to understand the stakes involved. This isn't about quick takes or product announcements; it's a foundational series for anyone wanting to build a more nuanced, evidence-informed view of where AI might be taking us. Each episode in this compilation stands as a key piece of that puzzle, providing the context and depth often missing from mainstream coverage. Tune in for a challenging and perspective-shifting listen that reframes how you think about intelligence, progress, and our collective responsibility.
Author: Language: en-gb Episodes: 14

The 80,000 Hours Podcast on Artificial Intelligence
Podcast Episodes
Zero: What to expect in this series [not-audio_url] [/not-audio_url]

Duration: 2:02
What might it be like to live through the creation of AI that surpasses human abilities? That future may be closer than you think. In this series, one expert interview at a time, we'll walk you through what's at stake —…
Bonus: China-related AI safety and governance paths (Article) [not-audio_url] [/not-audio_url]

Duration: 47:42
Article originally published February 2022.In this episode of 80k After Hours, Perrin Walker reads our career review of China-related AI safety and governance paths.Here’s the original piece if you’d like to learn more.Y…