About this episode
Scott Alexander argues against SURA, his name for the growing view that safety-minded AI companies, their employees and even whole countries should Stop Unilaterally and Raise Awareness, with no plan for what happens if the race carries on without them. He makes the case that the backup plan, where alignment-focused labs stay in the lead and spend that lead on alignment, deserves far more care than it gets, and that walking away may hurt the odds of a global pause as well. Along the way come forecasters’ survival estimates, Frodo and the One Ring, and a worry that AI safety is drifting down the same road as climate activism. * 00:00 - Introduction * 08:12 - I. Questioning Assumptions Behind SURA * 17:53 - I.B: Is racing to build superintelligence deontologically evil? * 19:50 - I.C: If lots of alignment-pilled people and companies quit, would that raise awareness enough to make a global pause much more likely? * 21:54 - I.D: If alignment-pilled people and companies stay in the race, does that burn timeline? * 25:35 - II. Actually, most of this stuff is bad for a global pause, too * 27:01 - II.A: Getting rid of alignment-pilled people and companies makes it harder to win political fights * 28:05 - II.B: Getting rid of alignment-pilled people and companies makes it harder to activate the masses * 31:49 - II.C: Unilateral disarmament makes it harder to bring China to the bargaining table * 35:16 - (Maybe you already care a lot how alignment-pilled companies are) * 36:34 - III. What incentives are we fighting here? * 37:18 - III.A: Are corporate incentives a “One Ring” that corrupts everyone who touches them? * 41:32 - III.B: I dunno, mass politics is also looking kind of annular * 47:29 - (Exhibit 1: Sigal Samuel’s “Anthropic Could Just Stop”) * 54:12 - (A virtuous exception) * 55:12 - IV. What if we survive? * 58:47 - V. What’s The Alternative To SURA? * 59:26 - V.A: Getting companies to agree to stop in a coordinated manner * 01:00:46 - (...and then we know where we stand) * 01:01:43 - V.B: Getting companies to prepare for Plan C+/Plan C if that doesn’t work https://www.lesswrong.com/posts/xikGZ8fuhdG6DApAo/against-sura This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe
Comments
Rate this episode
Episode Details
- Author
- Askwho Casts AI
- Show Type
- full
- Audio Format
- audio/mpeg

