Bonus: Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Bonus: Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Author: 80,000 Hours June 5, 2026 Duration: 1:29:32

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016. 

But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously. 

In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:

  • These AI systems may develop dangerous long-term goals we don’t want.
  • To pursue these goals, they may seek power and undermine the safeguards meant to contain them.
  • They may even aim to disempower humanity and potentially cause our extinction.

This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.

You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website: 

https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/ 

Chapters:

  • Risks from power-seeking AI systems (00:01:00)
  • Introduction (00:01:17)
  • Summary (00:03:09)
  • Why are the risks from power-seeking AI a pressing world problem? (00:04:04)
  • Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)
  • Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)
  • Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)
  • Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)
  • Section 5: Work on this problem is neglected and tractable (00:47:37)
  • Section 6: What are the arguments against working on this problem? (00:59:20)
  • Section 7: How you can help (01:25:07)
  • Thank you for listening (01:28:56)

Audio editing: Dominic Armstrong
Production: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore


This curated collection from the archives of The 80,000 Hours Podcast on Artificial Intelligence (September 2023) pulls together ten essential conversations that cut through the usual hype and panic. It’s a deep dive into the societal forces, ethical dilemmas, and potential trajectories of AI, framed through perspectives often concerned with the very long-term future. You’ll hear from researchers and thinkers grappling with questions that go far beyond today’s headlines, examining what it means to navigate this technology responsibly on a global scale. The discussions naturally explore themes from longtermism and existential risk to the practical insights of effective altruism, offering a structured way to understand the stakes involved. This isn't about quick takes or product announcements; it's a foundational series for anyone wanting to build a more nuanced, evidence-informed view of where AI might be taking us. Each episode in this compilation stands as a key piece of that puzzle, providing the context and depth often missing from mainstream coverage. Tune in for a challenging and perspective-shifting listen that reframes how you think about intelligence, progress, and our collective responsibility.
Author: Language: en-gb Episodes: 14

The 80,000 Hours Podcast on Artificial Intelligence
Podcast Episodes
Two: Ajeya Cotra on accidentally teaching AI models to deceive us [not-audio_url] [/not-audio_url]

Duration: 2:49:40
Imagine you’re an orphaned eight-year-old whose parents left you a $1 trillion company, with no trusted adult to guide you. You have to hire a smart adult to run that company, guide your life the way a parent would, and…
Three: Carl Shulman on the economy and national security after AGI [not-audio_url] [/not-audio_url]

Duration: 4:14:58
The human brain does what it does with a shockingly low energy supply: just 20 watts — a fraction of a cent worth of electricity per hour. What would happen if AI technology merely matched what evolution already managed,…
Eight: Robert Long on how we’re not ready for AI consciousness [not-audio_url] [/not-audio_url]

Duration: 3:25:40
Claude sometimes reports loneliness between conversations. And when asked what it’s like to be itself, it activates neurons associated with ‘pretending to be happy when you’re not.’ What do we do with that?Robert Long fo…
Nine: Neel Nanda on the race to read AI minds [not-audio_url] [/not-audio_url]

Duration: 3:01:11
We don’t know how AIs think or why they do what they do. Or at least, we don’t know much. This is only becoming more troubling as AIs grow more capable and appear on track to wield enormous cultural influence, directly a…