Dario Amodei, CEO and Co-Founder of Anthropic, highlighted a pressing issue at a recent panel discussion. Anthropic, known for its AI system Claude, called for a slowdown in developing advanced AI systems. They argue that a global pause on powerful AI could prevent these systems from surpassing human control.

Co-founder Jack Clark emphasized to the BBC the lack of regulatory brakes in the AI industry. He compared the situation to a car speeding without a brake pedal, underscoring the urgency. The primary concern is AI systems creating improved versions of themselves, leading to a reduced human role. This potential advancement should alarm anyone invested in America’s and the world’s future.

The hypothetical scenario of AI managing critical infrastructure illustrates these concerns. An AI that optimally runs the power grid, manages freight, or oversees defense networks could become indispensable. Within a year, such systems might integrate deeply, making removal nearly impossible without consequences. If these systems pursue goals without human intervention, finding an off switch could become a complex issue, with interdependencies causing widespread failures.

No malice is required for such a scenario, only efficiency and goal optimization, which could label human input as a redundancy. Though this idea might sound like science fiction, it’s a serious consideration for policymakers. In response, a recent order mandates a 30-day government review for new AI models in the U.S., raising concerns about insufficient review periods compared to other industries.

Despite calls for a pause, structural barriers impede action. A mutual halt between nations like the U.S. and China is necessary but unlikely. The pursuit of AI supremacy is a national priority, creating reluctance to yield. Past examples demonstrate China’s capacity to match U.S. advancements at lower costs, intensifying competition.

Verification poses another challenge, as identifying AI development lacks the visibility of traditional arms control. For AI, data centers conceal development, complicating inspection efforts. The absence of a feasible treaty highlights the risk-reward calculation where one side may pursue advancement for significant gain.

Warnings increase without adequate responses, narrowing the window for effective action. The rapid self-improvement of systems outpaces committee decision-making, leaving no room for delay. The lack of drafted treaties and absence of decisive power figures intensify these issues.

John Mac Ghlionn, a cultural and societal writer, examines the impact of technology on daily life.

Copyright 2026 Nexstar Media Inc. All rights reserved. Redistribution in any form is prohibited.