"Schelling Goodness, and Shared Morality as a Goal" by Andrew_Critch

"Schelling Goodness, and Shared Morality as a Goal" by Andrew_Critch

Author: LessWrong March 6, 2026 Duration: 1:14:50
Also available in markdown at theMultiplicity.ai/blog/schelling-goodness.

This post explores a notion I'll call Schelling goodness. Claims of Schelling goodness are not first-order moral verdicts like "X is good" or "X is bad." They are claims about a class of hypothetical coordination games in the sense of Thomas Schelling, where the task being coordinated on is a moral verdict. In each such game, participants aim to give the same response regarding a moral question, by reasoning about what a very diverse population of intelligent beings would converge on, using only broadly shared constraints: common knowledge of the question at hand, and background knowledge from the survival and growth pressures that shape successful civilizations. Unlike many Schelling coordination games, we'll be focused on scenarios with no shared history or knowledge amongst the participants, other than being from successful civilizations.

Importantly: To say "X is Schelling-good" is not at all the same as saying "X is good". Rather, it will be defined as a claim about what a large class of agents would say, if they were required to choose between saying "X is good" and "X is bad" and aiming for a mutually agreed-upon answer. This distinction is crucial [...]

---

Outline:

(01:59) This essay is not very skimmable

(03:44) Pro tanto morals, is good, and is bad

(06:39) Part One: The Schelling Participation Effect

(13:52) What makes it work

(15:50) The Schelling transformation on questions

(19:10) Part Two: Schelling morality via the cosmic Schelling population

(21:12) Scale-invariant adaptations

(22:54) An example: stealing

(30:32) Recognition versus endorsement versus adherence

(31:34) The answer frequencies versus the answer

(33:59) Ties are rare

(35:06) Is the cosmic Schelling answer ever knowable with confidence?

(36:02) Schelling participation effects, revisited

(38:03) Is this just the mind projection fallacy?

(39:42) When are cosmic Schelling morals easy to identify?

(42:59) Scale invariance revisited

(44:03) A second example: Pareto-positive trade

(47:45) Harder questions and caveats

(50:01) Ties are unstable

(51:43) Isnt this assuming moral realism?

(53:07) Dont these results depend on the distribution over beings?

(54:41) What about the is-ought gap?

(56:29) Tolerance, local variation, and freedom

(58:25) Terrestrial Schelling-goodness

(59:42) So what does good mean, again?

(01:01:08) Implications for AI alignment

(01:06:15) Conclusion and historical context

(01:09:16) FAQ

(01:09:20) Basic misunderstandings

(01:12:20) More nuanced questions

---

First published:
February 28th, 2026

Source:
https://www.lesswrong.com/posts/TkBCR8XRGw7qmao6z/schelling-goodness-and-shared-morality-as-a-goal

---



Narrated by TYPE III AUDIO.


Dive into a stream of ideas where technology, culture, philosophy, and society intersect, all through the lens of the LessWrong (Curated & Popular) podcast. This isn't a traditional talk show with hosts, but rather a curated audio library of the most impactful writing from the LessWrong community. Each episode is a narration of a full post, selected for its high value and interesting arguments, focusing on pieces that have been formally curated or have garnered significant community approval. You'll hear clear, thoughtful readings of essays that tackle complex topics like artificial intelligence, rational thinking, moral philosophy, and the forces shaping our future. The audio format lets you absorb these dense, often paradigm-shifting concepts during a commute or a walk, turning written analysis into an immersive listening experience. This particular feed is deliberately selective, offering a manageable stream of the community's standout work. For those who want an even deeper dive into the discussion, there are broader feeds available. The LessWrong (Curated & Popular) podcast serves as an intellectual filter, delivering the signal through the noise and inviting you to engage with some of the most rigorously examined ideas on the internet.
Author: Language: English Episodes: 100

LessWrong (Curated & Popular)
Podcast Episodes
"My Willing Complicity In “Human Rights Abuse”" by AlphaAndOmega [not-audio_url] [/not-audio_url]

Duration: 18:47
Note on AI usage: As is my norm, I use LLMs for proof reading, editing, feedback and research purposes. This essay started off as an entirely human written draft, and went through multiple cycles of iteration. The primar…
"Don’t Let LLMs Write For You" by JustisMills [not-audio_url] [/not-audio_url]

Duration: 5:53
Content note: nothing in this piece is a prank or jumpscare where I smirkingly reveal you've been reading AI prose all along. It's easy to forget this in roarin’ 2026, but homo sapiens are the original vibers. Long befor…
"Thoughts on the Pause AI protest" by philh [not-audio_url] [/not-audio_url]

Duration: 11:12
On Saturday (Feb 28, 2026) I attended my first ever protest. It was jointly organized by PauseAI, Pull the Plug and a handful of other groups I forget. I have mixed feelings about it. To be clear about where I stand: I b…
"Less Dead" by Aurelia [not-audio_url] [/not-audio_url]

Duration: 14:11
Come with me if you want to live. – The Terminator 'Close enough' only counts in horseshoes and hand grenades. – Traditional After 10 years of research my company, Nectome, has created a new method for whole-body, whole-…
"Gemma Needs Help" by Anna Soligo [not-audio_url] [/not-audio_url]

Duration: 15:00
This work was done with William Saunders and Vlad Mikulik as part of the Anthropic Fellows programme. The full write-up is available here. Thanks to Arthur Conmy, Neel Nanda, Josh Engels, Dillon Plunkett, Tim Hua and man…
"On Independence Axiom" by Ihor Kendiukhov [not-audio_url] [/not-audio_url]

Duration: 44:59
The Fifth Fourth Postulate of Decision Theory In 1820, the Hungarian mathematician Farkas Bolyai wrote a desperate letter to his son János, who had become consumed by the same problem that had haunted his father for deca…
"Solar storms" by Croissanthology [not-audio_url] [/not-audio_url]

Duration: 23:22
Most of civilization's electricity is generated far off-site from where it's delivered. This is because you don't want to be running and refueling coal/gas/nuclear plants inside cities, hydraulic/wind power can't be move…
"Maybe there’s a pattern here?" by dynomight [not-audio_url] [/not-audio_url]

Duration: 15:23
1. It occurred to me that if I could invent a machine—a gun—which could by its rapidity of fire, enable one man to do as much battle duty as a hundred, that it would, to a large extent supersede the necessity of large ar…