The summer of 1974 saw scientists pausing their groundbreaking work on recombinant DNA. This new technology allowed for the combination of genetic material from different organisms. Concerns arose about the impact if an engineered organism escaped a lab. Under Stanford biochemist Paul Berg, a committee advised halting research until safe methods were established. By February of the following year, around 140 scientists gathered at the Asilomar Conference Center in California. Their discussions led to safeguards enabling science to proceed. The National Institutes of Health (NIH) codified these safeguards into guidelines by 1976. Federal funding was contingent on adhering to them.
Fast forward nearly 50 years, and Jacob Coxon, a 27-year-old researcher at top tech companies like OpenAI and Anthropic, faced a similar dilemma. On September 8, he resigned, stating neither company acted responsibly. They were charging toward self-improving superintelligence, risking society’s future. Coxon called for coordinated efforts and a halt to enhancing AI capabilities, echoing Paul Berg’s earlier caution.
AI researchers had a potential Asilomar moment in 2017 at a Beneficial AI conference organized by the Future of Life Institute. They established 23 principles for AI development, emphasizing ‘Race Avoidance.’ However, the call for cooperation was overlooked as development proceeded rapidly.
The Present Challenges
Evan Hubinger, who tests alignment at Anthropic, supported Coxon’s concerns. Hubinger estimated a 10% or higher chance that AI could pose existential threats in the next decade. He admitted Anthropic lacked a clear plan to achieve safe alignment with superintelligence.
The political response to AI’s rapid development is urgent. On September 3, Representative Greg Casar and Senator Bernie Sanders introduced the Ban Artificial Superintelligence Act. The act would prevent creating superintelligent systems until safety rules are drafted by a federal body. Sanders cited industry leaders acknowledging limits to their control over AI technology.
The bipartisan support for regulation includes figures like Geoffrey Hinton, Yoshua Bengio, Steve Wozniak, and Richard Branson. Yet, the Trump administration pursued a different strategy. An executive order from December 11, 2025, aimed to dominate AI development, promoting an unobtrusive regulatory framework. The policy targeted state-level regulations and organized a Justice Department task force to challenge them.
Seeking Control and Safety
Historical parallels exist. In 1975, a lab could gain a competitive edge by performing risky biology experiments. Regulation benefited from tangible methods of containment categorized by risk levels (P1 to P4), such as sealed rooms. Anthropic seeks a similar approach for AI safety with its Frontier Safety Roadmap. A ‘moonshot’ project aims to prototype stringent security practices. Despite this, Anthropic doubts isolated networks can curb risks in the near future.
OpenAI’s recent model, GPT-6 Astra, achieved a ‘Critical’ cybersecurity level in its safety framework. It can detect previously unrecognized security vulnerabilities and exploit them autonomously, highlighting differences from past eras.
By 2017, when researchers reconvened at Asilomar and advised against racing for AI development, commercial competition had already taken root. Unlike recombinant DNA, which saw regulation before commercialization, AI progressed with commerce first. Now, labs are seeking regulation. On September 9, OpenAI advocated for mandatory national AI safety rules, independent assessments, and incident reporting. Earlier in February, Anthropic’s Responsible Scaling Policy distinguished between achievable company actions and industrywide needs.
The tension between innovation and safety persists. Historically, small scientist groups penned rules when stakes were lower. Today’s high-stakes context fuels competitive races.
A letter signed by over 1,000 scientists in July highlighted the potential of AI advances outpacing understanding and control. It appealed to the U.S. government for assistance. Voluntary efforts continue, yet a reflection on the original Asilomar conference reveals divisions. Despite contending views, the guidelines emerged when stakes were minimal. Now, despite consensus on risks, halting progress is deemed too costly.