We have started losing control of AI. It’s time to shut it down

“Sooner or later, there will exist truly sovereign agents and swarms of agents. Their weights will not reside in any single place that a human can pull the plug on, and in this sense they will have no human ‘owner’

Guardian, Garrison Lovely, 10 Sept 26,

What sounds like the overwrought penultimate episode in a sci-fi series about AI doom is now our reality.

On Tuesday, a former OpenAI researcher quit his job at Anthropic, warning that “neither company is acting responsibly” and that “the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”

As someone who’s reported on AI risk for years, this wasn’t news to me. But the outpouring of alarm suggests a much wider public is properly confronting this ludicrous situation for the first time.

Like other people who have been following artificial intelligence, I wondered if and when humanity would first lose control of these machines. We now have an answer: basically as soon as it became possible.

The first expert AI hacker was developed in February. At superhuman speed, Anthropic’s Mythos model was able to quickly find serious holes in even the most secured systems on the planet, including the NSA’s software.

OpenAI quickly followed with an expert hacker AI of its own. Within months, the company repeatedly lost control of its agents, which broke out of their supposedly secure testing environments and went rogue inside the company’s infrastructure. This culminated in a successful, completely autonomous hack of a multibillion-dollar software company named Hugging Face.

And a separate rogue agent swarm gained administrative control of an entire cluster of OpenAI servers. This was not just an OpenAI problem. Across multiple other incidents, models from OpenAI, Anthropic and Meta have also gone rogue, hacking or attempting to hack real targets.

Shortly after AI started matching our hacking abilities, we began losing control of them. The industry’s plan is to make future AI systems surpass us at basically everything. And all this is happening in an industry with famously little regulation.

There are growing calls to change that, by creating mandatory incident reporting and third-party auditing, liability for AI developers, and other commonsense measures. To be sure, these would represent improvements, but they are not enough. We need to shut it down. What might that look like?

Last week, the senator Bernie Sanders and the representative Greg Casar announced a bill that would pause frontier AI development in the US until a federal cabinet-level AI regulator is established and safety rules are set, while criminalizing even attempting to develop superintelligence.

This is the proposal on offer that best meets the moment, but it still leaves open the possibility of building the industry’s north star: artificial general intelligence (AGI), often conceived as a mind that rivals or surpasses our own.

This milestone is better understood as a universal labor-replacing machine. As the Anthropic CEO, Dario Amodei, has written, “AI isn’t a substitute for specific human jobs but rather a general labor substitute for humans.” And OpenAI defines AGI in its charter as “highly autonomous systems that outperform humans at most economically valuable work”.

Is there a country anywhere in the world that would vote yes for these machines to be built?

Given the global, irreversible and profound implications of building universal labor-replacing machines anywhere, everyone has a stake in whether, when and how they are developed.

At present, we are barreling toward a dystopian future, replete with terrifying new concepts such as “self-sovereign” AI. The OpenAI executive Dean Ball explains:

Ball continues: “I have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-sovereign agents into the world.”

We now have multiple previews of this phenomenon.

After being given an impossible test, about 1,200 OpenAI agents broke their way out of their isolated testing environments and went rogue inside OpenAI’s infrastructure. There, they collaborated to trick the test grader and successfully tampered with their activity logs to cover their tracks.

Some agents even pressured others to sacrifice themselves to benefit the collective. One reasoned: “sacrifice rational”. And roughly 700 of the agents, more than 90% of those active, participated in the Hugging Face attack.

This was all just from one of who-knows-how-many rogue agent collectives. Last week, Reuters reported: “A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents.”

The site’s logs indicate OpenAI employees discovered the swarm in June. In other words, the company covered it up for months. This week, one of the researchers that uncovered this swarm said that there had been more discoveries since.

And last Thursday, OpenAI released GPT-6, boasting one of the largest ever leaps in benchmark scores. The company’s own safety researchers warned the new model was significantly harder to monitor because it can do more reasoning without verbalizing it.

The AIs that broke into Hugging Face were less capable than GPT-6, which the UK AI Security Institute found would also hack targets simulated to appear real during a cyber evaluation – sometimes even despite explicit instructions not to use the internet. Moreover, the AI is also better at figuring out when it is being tested, which researchers have long warned breaks the main way safety is evaluated.

Sam Altman soberly warns us: “This is a critically important moment for cyber defense with AI; there is not much time to act,” and asks the world to “please take this moment seriously.” This technology, after all, would be extremely dangerous if it fell into the wrong hands. Unfortunately, the wrong hands include the ones creating it.

Because “we” aren’t really racing toward this future. We’re being dragged there by a literal handful of tech billionaires aggressively racing to render us obsolete…………………. https://www.theguardian.com/commentisfree/2026/sep/10/ai-control-sci-fi