Under Cover Of AI Doomsday, Big Tech Is Writing Its Own Rules

Today, democracy and the rule of law are controversial among billionaires in Silicon Valley and in the AI Safety world. Many simply don’t think it’s viable or moral. They believe the best case is for humanity to live in a gilded cage, fed and treated well by those rational enough to look out for the best interests of most of us, who cannot otherwise care for ourselves. To that end, anything they do to benefit their own power is virtuous.

“The consensus is that the next year or two is crunch time for humanity. These are actually just literal quotes from my colleagues at Anthropic. They’ll say things like ‘endgame’ or ‘crunch time.’ From their perspective, this is when Anthropic and its competitors decide the fate of humanity“

To unpack this, I’ll first describe the technology as best I can. Then I’ll offer two different narratives of what is happening. The first narrative, the doomer narrative driving this panic, is that AI is a new form of civilization uncontrollable by humans, and we are moving quickly toward a singularity where bots will take over. The second narrative, call it the “rule-of-law” narrative, is that AI products are risky, perhaps existentially risky, but they are manageable if we can apply standard rules, such as negligence or criminal fraud, to the powerful people building them.

Matt Stoller THE LEVER, September 16, 2026

Silicon Valley says its machines could destroy humanity.

The more immediate danger is letting the people building AI decide how they’re governed.

Editor’s note: Since Matt Stoller first published this essay on his Substack on Friday amid the mounting AI crisis, Anthropic CEO Dario Amodei hascalled for slowing the development of powerful AI models, while OpenAI CEO Sam Altman hassaid an OpenAI IPO right now would be “ill-advised.”

But Stoller’s central point remains unchanged: The companies building these systems are still asking to set the rules. The real AI crisis doesn’t lie within the technology; it’s our failure to govern the powerful people building and profiting from it.

Something very weird is going on in our political economy discourse.

Over the past month, the executives at the top AI companies have engineered a slow-motion panic over AI, snowballing into an argument that they are on the verge of creating a technology with a reasonable chance of ending human civilization. A few days ago, Jacob Coxon, a 27-year-old AI researcher at Anthropic, resigned his role and posted a thread on X, arguing these companies are building super-intelligent systems that may destroy the world. Coxon’s comments garnered a front-page Wall Street Journal story, more than 110 million views, and wide commentary from lawmakers, CEOs, AI experts, and journalists.

Then another 20-something Anthropic employee, Evan Hubinger, chimed in to say that most employees at Anthropic agreed with Coxon, and “do earnestly believe AI could kill all humans,” putting the odds at 10 percent within the next decade. These are extraordinary claims, and the people making them are nicknamed “AI Doomers” or sometimes called the “AI Safety” movement. This group loosely encompasses the network that used to surround disgraced crypto billionaire Sam Bankman-Fried, a quasi-religious order known as effective altruists, who believe a cadre of hyper-rational elites can guide humanity. Despite their weirdness, this technology’s capabilities are improving, and some dangerous events have happened recently.

Yet at the same time, Anthropic, a major artificial intelligence “frontier lab,” is likely to go public in the next six weeks with an initial public offering valuing the firm at multiple trillions. This move will immediately create many AI doomer billionaires and millionaires, and it follows a similar IPO from SpaceX and a possible future IPO for a similar firm, OpenAI. It’s about to be very good times for San Francisco real estate, the yacht business, or whatever else AI techies like to buy.

That said, it’s not clear these companies have a sustainable business. Despite raising hundreds of billions of dollars, they are likely not profitable. They are competing fiercely with each other, and with cheaper Chinese open-source models. OpenAI just cut pricing for one of its models by 80 percent, despite losing $26 billion in 2025. Where are the long-term sustainable profits? To remain at the frontier with pricing power, these guys have to spend every dollar of revenue they bring in on training ever more powerful models and then some.

Despite these financial hurdles and predictions of mass extinction, no one is considering canceling the IPOs. And that is true despite the fact that the AI doomer messaging has now broken through to the culture and politics writ large. Former Treasury Secretaries Hank Paulson and Robert Rubin have suggested a Presidential working group on AI risk, modeled after similar commissions put together to deal with financial crises. Bernie Sanders is convening senators over the threats to humanity by AI and calling for a pause on AI development; Ted Cruz is calling for guardrails, and Democrats are considering a special House committee to look at Big Tech and AI. Celebrities like Sheryl Crow are posting in all black on their Instagram accounts.

Something about this odd dynamic, of multitrillion-dollar IPOs by loss-making companies, combined with panicky messages of doom passed around via influencers, seems off. It’s not that there’s no risk to AI systems. It’s that if you don’t accept the full doomer worldview, you get gaslit as not taking risk seriously. Right after 9/11, during the debate over the war in Iraq, the financial crisis in 2008, right before COVID-19, and during various bailouts, I was distressed by the broad demand to stop thinking or reasoning, as the urgency was too great to delay action. A similar pressure campaign feels like it’s happening now. And I’m not alone; experienced tech investor Roger McNamee and many others see it too.

It’s hard to escape that there’s something very strange and manipulative about AI safety discourse. The AI doomers have a lot of money; they run the major AI companies; they fund most AI research; and they staff the major AI Safety nonprofits, like the one that investigates security problems. Many are sincere, but the strange hyper-rationalist effective altruism theology is a huge part of the AI Safety space. As an academic computer scientist who has been around this world for a decade told me, “It’s a weird quasi-religion, like a cult. The core people are sociopaths, and then the rest are neurotics.” It’s not a coincidence that Sam Bankman-Fried was the single most important early funder of Anthropic.

So what is the real deal with AI products? Is it all just a ruse? Not at all. Agentic AI is truly a through-the-looking-glass type of technology that does unexpected things and presents new challenges.

Ok, let’s go to the technology…………………………………………………………………………………………………………………………………………………………………………

The AI Doomer Narrative

We’ll start with the doomers. Their view of the world is based on the idea that building generalized artificial intelligence is likely to end humanity, and they have believed that for many years. But they see the last six months as particularly validating to their fears……………………………………………………………………………………………………………………………………………………………………..

The narrative from the AI Doomer world, most prominently an essayist named Dwarkesh Patel, is that this story shows we are on the verge of being supplanted as the dominant species on earth. …………………………………………………………………………………………………………………………………….. So that’s the doomer narrative. And it is quite scary.

The Rule-Of-Law Narrative

Now, let’s go to the rule-of-law narrative, in which agents are not creatures, but are simply unsafe machines.

In this telling, it’s important not to think of agents as humans with wants and desires………………………………………………………………………………………………………………………………………………………………………………………

The bottom line is that OpenAI researchers released a system designed to hack, gave it access to powerful tools, let it run wild, and then it did the hell out of what it was asked to do in ways these researchers did not fully anticipate……………………………………………………………………………………………………………………………………………….

Why isn’t it possible to force model developers to invest in tech to make agents safer? Well, it is. We just choose not to force the powerful financiers at these AI firms to invest in safer products. “Perhaps it’s time that we put aside the sci-fi tales,” writes Newport, “and actually hold these labs to account for playing fast and loose with an ill-advised way of building AI systems.”

The Real Crisis

So where do I come down? Well, I think you can tell I believe in the rule-of-law narrative. And that’s because the evidence for it is overwhelming. It just doesn’t make any sense that we’d allow OpenAI and Anthropic to go public at multitrillion-dollar valuations, even though they make deeply unsafe products, unless we have a crisis with the rule of law………………………………………………………………………………………………………………………………………………………………………………………………..

This dynamic isn’t driven by technological advance, it’s driven by a refusal to collectively manage risk. We just don’t govern. It’s true that AI creates serious cybersecurity vulnerabilities in our society for which we are unprepared, but it’s also true that corporations have successfully lobbied aggressively against being mandated to patch software infrastructure for many years, worsening this situation……………………………………………………………………………………………………………………………………………………….We haven’t constrained new technologies for decades, because we lost touch with our traditions of public governance. ……………………………………………………………………………………………………………………………………..

The consensus is that the next year or two is crunch time for humanity. These are actually just literal quotes from my colleagues at Anthropic. They’ll say things like ‘endgame’ or ‘crunch time.’ From their perspective, this is when Anthropic and its competitors decide the fate of humanity.

…………………………………………………………………………………………………………………………………………………………..Now it’s certainly possible to assume we will never be able to govern risk as a society. But recognize that’s not actually a fear of technology; it’s an argument about the inevitable weakness of democracy and how self-government is a ruse that will lead to our collective annihilation, while an all-powerful caste, by contrast, can protect and enrich our lives. It’s very Divine Right of Kings, actually.

In other words, the AI Safety goal is to find a way to place what they perceive as the immense power of AI in the hands of the only people capable enough to prevent the destruction of the human species, and not have to deal with those petty things like laws, competition, or democratic feedback. The real doomer solution for humanity is for the Anthropic IPO to go really, really well and to find a way to turn those losses into profits. So color me skeptical of ghost stories. We can manage the risks of AI products. The real crisis is that we no longer manage the risks of anything. https://www.levernews.com/the-ai-doomsday-distraction/