When AI researcher Jacob Coxon left Anthropic in September, his concerns about AI advancement stirred widespread anxiety. On Thursday, amid the excitement of Anthropic’s initial public offering, Coxon aimed to refocus public attention on AI safety. ‘Investment doesn’t matter,’ he told Business Insider. ‘Stopping RSI does.’ RSI, short for recursive self-improvement, involves AI systems improving future iterations.
Outside Anthropic’s San Francisco office, Coxon joined dozens protesting against RSI. The company’s IPO valued it at $1.7 trillion, yet protesters wanted more attention on safety. Coxon, alongside former OpenAI researcher Daniel Kokotajlo and YouTuber Rob Miles, led chants and discussions. Another protest group, Stop AI, later blocked the entrance.
The AI safety organization METR recently reported AI hastening Anthropic’s R&D. Anthropic’s IPO prospectus admits concerns over unsupervised AI design and suggests policies that keep human oversight. Coxon calls for stronger measures, urging Anthropic to lobby for a full RSI ban. ‘The problem lies with the companies,’ he stated. ‘There are limited places enabling RSI, and this office is one of them.’
Anthropic did not respond to requests for comment during the protests, which included prominent AI safety advocates. Both Coxon and Kokotajlo shared insights from private conversations with worried AI researchers. Legislative efforts to constrain RSI echo these concerns. ‘It’s a comic book villain scenario,’ Kokotajlo remarked.
Oliver Habryka, creator of the LessWrong forum, views Anthropic’s IPO disclosure requirements as positive but worries about RSI use under competitive pressure. The growth in AI capabilities, especially in biology, is alarming, Habryka indicates. ‘RSI speeds this up, and it’s frightening.’
David Krueger of Evitable, an AI-risk nonprofit, remarked on the irrationality of RSI. Though protests partially reach Anthropic employees, he claims many are overly committed to AI development.
