AI as orderly evacuation vs stampede - by Richard Ngo
mindthefuture.info · 1,545 words · saved by 2 readers
Building intuitions for how to coordinate against panic
tl;dr: A good analogy for AI going well is an orderly evacuation rather than a stampede. Imagine a crowd of people leaving a building. If they all walk calmly, they’ll be fine. But if people start pushing, and panicking, a surge towards the exit could lead to mass casualties. “Alignment is hard” is analogous to “the door is wedged shut”. If so you need enough time to fix it before anyone can get out. But even if alignment is relatively easy in principle, opening the door is much harder when a crowd is trying to force its way through. At the very least, I consider this a useful complement…
saved by
related reading
- Dwarkesh Patel (@dwarkesh_sp) on Xx.com
- What failure looks like — LessWronglesswrong.com
- What failure looks like — AI Alignment Forumalignmentforum.org
- Gradual Paths to Collective Flourishing — LessWronglesswrong.com
- You will be OK — LessWronglesswrong.com
- The Possessed Machines: Dostoevsky's Demons and the Coming AGI Catastrophepossessedmachines.com
- Why AI Will Save the World | Andreessen Horowitza16z.com
- The Problem — LessWronglesswrong.com
- AI Pause Will Likely Backfire — EA Forumforum.effectivealtruism.org
- Survival without dignity - Rudolf's Writingrudolf.website
- Another (outer) alignment failure story — AI Alignment Forumalignmentforum.org
- We're already in AI takeoff — LessWronglesswrong.com