The 'if-then' conditional commitments are the practical hinge here. Most AI regulatory debate collapses into binary 'overreach now' or 'too late later.' Conditional triggers let governments build authority and detection capacity in reserve without forcing premature rules. The harder problem is institutional capacity to detect triggers before the window closes.
The 13-17% automation threshold deserves more weight than the regulation framing carries. We're already at or past that level in narrow domains — coding tools, customer support, design generation — and the economic-growth paper says the cascade doesn't need uniform automation, just enough at high-leverage research nodes. The radical-optionality framework assumes governments have years to build information-gathering authorities and auditing capacity. If the singularity-in-6-years number is even directionally right, institutional capacity-building at normal-policymaking speed won't keep up. The harder question isn't 'should we regulate' but 'can the auditing/whistleblowing/assessment infrastructure stand up faster than automation that's already mid-deployment.'
The auditing-speed problem already has a test case. Two papers in late March collapsed the qubit threshold for breaking encryption by orders of magnitude, and the tool that shortened that timeline was AI itself. If the security foundations protecting model weights can be obsoleted in weeks, the assumption that policymakers get years to design audit regimes looks generous. https://thesynthesisai.substack.com/p/the-qubit-threshold sketches the math.
The auditing-speed gap matters even more when you factor in who has to build the audits. Cryptographic audit regimes assume a static or slow-moving threat model, and the institutions that staff audit teams (governments, standards bodies, certifying labs) recruit and accredit on a 3-7 year cycle. AI cuts the time-to-vulnerability discovery to weeks. So the gap isn’t just “laws can’t keep up” — it’s that the underlying audit-engineering workforce can’t keep up either. The bottleneck is the human-formation pipeline, not the legal language. The lab that can rebuild its audit toolchain faster than the threat surface moves wins, and right now the labs are losing on both axes.
Can regulation stand up faster than the automation is advancing? The answer is pretty obviously "no" unless we figure out a radically different way of regulating that doesn't involve laws, committees, courts, etc.
I think we have to admit that (a) regulating in anticipation of problems is a nonstarter because we don't know what the risks even are, and (b) by the time the risks become incontrovertible, regulation won't be fast enough.
It seems inevitable then that all the discussion about AI safety won't amount to much. The technology will advance on its own trajectory, driven by competition, and we'll have to deal with the consequences as they arise. Which is exactly what we did with other general purpose technologies like the personal computer and the internet.
Calling this "radical optionality" is doing a lot of heavy lifting. Transparency requirements, whistleblower protections, third-party auditing... in most regulated industries these are just called "the basics." Framing the basics as radical tells you more about the current state of AI governance than the paper does.
"Radical optionality" sounds directionally good, but the proposed interventions feel detached from any particular regulatory decision a government might need to make in response to crisis. It's closer to a posture than a policy. This is like what Mark Neocleous called "resilience" after 2008. Not being secured against specific harms, but being ready for anything, with the state's capacity expanding to match whatever catastrophe we can imagine.
Neocleous's concern was that this reunited state and capital around managing the disaster rather than questioning the conditions that produced it. Without wishing to sound a luddite, questioning the conditions of an AI-driven disaster is hard when unfettered AI development feels inevitable. It certainly feels more like nuclear than solar in terms of its risk profile…
I suppose my question, then, is: Is radical optionality the pre-emptive version of the same move? And if so, does taking the (hypothetical) catastrophe seriously - which you do, and Neocleous didn't - change the answer, or just make the apparatus harder to refuse?
Capability-to-growth links are useful as maps. The investment question is still which constraints — compute, energy, and capital intensity — decide durable winners.
Radical Optionality focuses on preserving future choices. But most of its tools are designed to make companies visible to government.
As governments become major AI deployers themselves, the symmetry breaks down. Who makes AI-enabled institutions visible to the public?
We spend a lot of time discussing how governments should oversee frontier labs, and much less time discussing how citizens can oversee AI-enhanced power.
the radical optionality framing reminds me of how central banks built payment rail infrastructure in 2015 that sat idle until instant payment demand spiked. problem is governments historically struggle with "build it before you need it" — treasury systems still run COBOL for a reason
What fascinates me about recursive self-improvement is that the real acceleration may not simply be technological, but civilizational and psychological.
Human systems — education, labor, economics, identity, status, even meaning itself — evolved under conditions where change happened slowly enough for cultures and individuals to adapt across generations.
If intelligence begins recursively improving the systems surrounding civilization, humanity may encounter a historically unusual feedback loop:
we shape the systems,
the systems reshape society,
and society gradually reshapes human behavior, cognition, and values in return.
Which may mean the long-term challenge is not only aligning AI, but preserving human coherence, meaning, and social stability during periods of accelerating transformation.
The "radical optionality" framing is genuinely useful because it sidesteps the binary of regulate-now vs. wait-and-see. Both positions assume you know what you're regulating — the current moment doesn't give you that certainty.
The interesting implementation challenge: what does "building the institutions" look like when the capabilities you're preparing to govern don't yet exist? There's a risk that institutions designed around today's AI failure modes end up poorly calibrated for the failures that actually matter at frontier capability levels.
The data infrastructure piece — logging, audit trails, model lineage — seems like the least-regret investment here, since it's load-bearing across nearly every plausible governance scenario regardless of which specific risks materialize.
Hardware research being the dominant lever is the most striking finding here, but there's a gap between automating chip *design* and scaling chip *deployment*. Data center construction hit $41 billion in 2025, with electrical work at up to 70% of costs and a https://thesynthesisai.substack.com/p/the-hard-hat that takes five years per trainee to close. RSI models assume automated R&D translates smoothly into deployed compute. Physical infrastructure tells a different story: at some point the bottleneck shifts from designing better chips to wiring up the buildings that house them.
The scary bit in the RSI section is the 13% number, because that’s low enough for the warning sign to look like a bunch of boring workflow automation nobody wants to argue about.
The 'if-then' conditional commitments are the practical hinge here. Most AI regulatory debate collapses into binary 'overreach now' or 'too late later.' Conditional triggers let governments build authority and detection capacity in reserve without forcing premature rules. The harder problem is institutional capacity to detect triggers before the window closes.
The 13-17% automation threshold deserves more weight than the regulation framing carries. We're already at or past that level in narrow domains — coding tools, customer support, design generation — and the economic-growth paper says the cascade doesn't need uniform automation, just enough at high-leverage research nodes. The radical-optionality framework assumes governments have years to build information-gathering authorities and auditing capacity. If the singularity-in-6-years number is even directionally right, institutional capacity-building at normal-policymaking speed won't keep up. The harder question isn't 'should we regulate' but 'can the auditing/whistleblowing/assessment infrastructure stand up faster than automation that's already mid-deployment.'
The auditing-speed problem already has a test case. Two papers in late March collapsed the qubit threshold for breaking encryption by orders of magnitude, and the tool that shortened that timeline was AI itself. If the security foundations protecting model weights can be obsoleted in weeks, the assumption that policymakers get years to design audit regimes looks generous. https://thesynthesisai.substack.com/p/the-qubit-threshold sketches the math.
The auditing-speed gap matters even more when you factor in who has to build the audits. Cryptographic audit regimes assume a static or slow-moving threat model, and the institutions that staff audit teams (governments, standards bodies, certifying labs) recruit and accredit on a 3-7 year cycle. AI cuts the time-to-vulnerability discovery to weeks. So the gap isn’t just “laws can’t keep up” — it’s that the underlying audit-engineering workforce can’t keep up either. The bottleneck is the human-formation pipeline, not the legal language. The lab that can rebuild its audit toolchain faster than the threat surface moves wins, and right now the labs are losing on both axes.
Can regulation stand up faster than the automation is advancing? The answer is pretty obviously "no" unless we figure out a radically different way of regulating that doesn't involve laws, committees, courts, etc.
I think we have to admit that (a) regulating in anticipation of problems is a nonstarter because we don't know what the risks even are, and (b) by the time the risks become incontrovertible, regulation won't be fast enough.
It seems inevitable then that all the discussion about AI safety won't amount to much. The technology will advance on its own trajectory, driven by competition, and we'll have to deal with the consequences as they arise. Which is exactly what we did with other general purpose technologies like the personal computer and the internet.
Calling this "radical optionality" is doing a lot of heavy lifting. Transparency requirements, whistleblower protections, third-party auditing... in most regulated industries these are just called "the basics." Framing the basics as radical tells you more about the current state of AI governance than the paper does.
Mary Shelley fue mucho más que una inmensa escritora.
Just thought I'd share my proposal for AI Risk Agility Plans which has some resonance with this proposal: https://www.lesswrong.com/posts/5EkoySbJZfhTnHSek/ai-risk-agility-plans-v0-1
(Though it's somewhat independent of the level of legislation).
You can't control something that's an order of magnitude smarter than you. Arguments here:
https://medium.com/@ekvi/youre-looking-for-agi-in-the-wrong-place-310d572116cb
And yes, the Dyson Sphere/Swarm is complete nonsense. Arguments here:
https://medium.com/@ekvi/the-two-keys-2cede203d92c
Thank you for this, Jack!
"Radical optionality" sounds directionally good, but the proposed interventions feel detached from any particular regulatory decision a government might need to make in response to crisis. It's closer to a posture than a policy. This is like what Mark Neocleous called "resilience" after 2008. Not being secured against specific harms, but being ready for anything, with the state's capacity expanding to match whatever catastrophe we can imagine.
Neocleous's concern was that this reunited state and capital around managing the disaster rather than questioning the conditions that produced it. Without wishing to sound a luddite, questioning the conditions of an AI-driven disaster is hard when unfettered AI development feels inevitable. It certainly feels more like nuclear than solar in terms of its risk profile…
I suppose my question, then, is: Is radical optionality the pre-emptive version of the same move? And if so, does taking the (hypothetical) catastrophe seriously - which you do, and Neocleous didn't - change the answer, or just make the apparatus harder to refuse?
Capability-to-growth links are useful as maps. The investment question is still which constraints — compute, energy, and capital intensity — decide durable winners.
PUTIN WANTS THE ACTUAL
@Jack Clark
Radical Optionality focuses on preserving future choices. But most of its tools are designed to make companies visible to government.
As governments become major AI deployers themselves, the symmetry breaks down. Who makes AI-enabled institutions visible to the public?
We spend a lot of time discussing how governments should oversee frontier labs, and much less time discussing how citizens can oversee AI-enhanced power.
https://deepbitcheesebrew.substack.com/p/when-ai-builds-power?r=8hzmhl&utm_medium=ios
the radical optionality framing reminds me of how central banks built payment rail infrastructure in 2015 that sat idle until instant payment demand spiked. problem is governments historically struggle with "build it before you need it" — treasury systems still run COBOL for a reason
the rsi stuff assumes deployment at scale but most enterprises cant even roll out basic rpa without 18 month change management cycles
the economic models skip the messy middle where orgs have the tech but not the culture to use it
What fascinates me about recursive self-improvement is that the real acceleration may not simply be technological, but civilizational and psychological.
Human systems — education, labor, economics, identity, status, even meaning itself — evolved under conditions where change happened slowly enough for cultures and individuals to adapt across generations.
If intelligence begins recursively improving the systems surrounding civilization, humanity may encounter a historically unusual feedback loop:
we shape the systems,
the systems reshape society,
and society gradually reshapes human behavior, cognition, and values in return.
Which may mean the long-term challenge is not only aligning AI, but preserving human coherence, meaning, and social stability during periods of accelerating transformation.
The "radical optionality" framing is genuinely useful because it sidesteps the binary of regulate-now vs. wait-and-see. Both positions assume you know what you're regulating — the current moment doesn't give you that certainty.
The interesting implementation challenge: what does "building the institutions" look like when the capabilities you're preparing to govern don't yet exist? There's a risk that institutions designed around today's AI failure modes end up poorly calibrated for the failures that actually matter at frontier capability levels.
The data infrastructure piece — logging, audit trails, model lineage — seems like the least-regret investment here, since it's load-bearing across nearly every plausible governance scenario regardless of which specific risks materialize.
Hardware research being the dominant lever is the most striking finding here, but there's a gap between automating chip *design* and scaling chip *deployment*. Data center construction hit $41 billion in 2025, with electrical work at up to 70% of costs and a https://thesynthesisai.substack.com/p/the-hard-hat that takes five years per trainee to close. RSI models assume automated R&D translates smoothly into deployed compute. Physical infrastructure tells a different story: at some point the bottleneck shifts from designing better chips to wiring up the buildings that house them.
The scary bit in the RSI section is the 13% number, because that’s low enough for the warning sign to look like a bunch of boring workflow automation nobody wants to argue about.