ITHome snapshot and original · Published 2026-09-17
Hinton and Amodei doubt the AI emergency stop button
The disagreement over an emergency stop is really about placement: is it one layer among many, or a stand-alone safety device?
Four people, four reservations
Hinton's doubt is long-term: a superintelligence could plausibly talk its overseer out of pressing the switch. Soares's doubt is about what lies beyond the boundary: once AI can copy itself or reach critical infrastructure, there is no guarantee the switch still binds.
Jack Clark and Dario Amodei represent the other side: Clark argues for mandatory, independently verifiable shutdown mechanisms; Amodei calls a switch far from a panacea, one part of a broader strategy of testing, external evaluation and deployment limits.
From a device to a system
Engineering today can deactivate self-hosted models and cut tool or compute access; the contested ground is systems beyond that perimeter. The policy question accordingly shifts to mandated retention plus independent verification rather than a single button.
The backdrop is Sam Altman's 2024 remark that there is no magic red button — safety comes from a series of concrete decisions. The debate's value is turning an abstract risk into a checkable list of mechanisms.
Background and context
Per Business Insider: Hinton argues a superintelligence could talk its overseer out of pressing the switch; Soares notes it may not bind once AI copies itself or reaches critical infrastructure; Jack Clark wants mandatory, independently verifiable shutdown; Amodei calls a switch far from a panacea.
The governance debate is changing its question, from whether to install a switch to where a switch sits among layered defenses. Policy and procurement decisions both need an answer on how verifiable shutdown is actually built.
The positions are quoted via Business Insider's report, observed here through ITHome's Chinese coverage; a stated position is not a shipped mechanism, and the debate itself is not proof of the risk's size.
Reading progress is saved in this browser. Save or add a note →
Looking back over time
02TIMELINE
Looking back over time
- Added to BitShovelEntered the BitShovel radar: 60 entries in the current ITHome feed.
Collection and archive details
Record location: live register
First discovered: 2026-09-17 (Beijing time).
Collection and source publication have distinct dates. A library addition does not establish a daily selection; retained selection dates are listed above.
Sources and references
03SOURCES
Sources and references
- Read the originalSourceITHome snapshot and original
Sources and evidence · 2 links
Each link has a purpose and scope. Link counts do not establish independent confirmation; direct support applies only to the statements specified below.