A researcher at one of the largest artificial intelligence companies has gone viral after publicly quitting and sharing that the product “could kill us all by the end of the decade.”
Jacob Coxon has been working on pre-training research at Anthropic and OpenAI for the last three years. His X post putting the company on blast has racked up 86 million views as of Wednesday morning.
“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” the post said.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote, saying his warning was “not a marketing stunt.”
“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.”
Coxon accused OpenAI employees of not fully understanding the “civilizational stakes” involved in their work.
“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” the post continued. “Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack.”
The replies were a mix of agreement, confusion, and skepticism.
One team lead at Anthropic agreed with Coxson’s dire warning. Evan Hubinger wrote in response, “Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Samuel Marks, a technical staff member at Anthropic, also weighed in, calling Coxson’s thread “very worth reading.”
He agreed that these companies continue to work on products despite the risks due to financial incentives and the belief that getting there before others will help reduce some of the potential risks. Marks also tried to explain some of the actual issues Coxson was referring to.
“Unlike traditional software, we can’t ‘program’ AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this,” he wrote. “We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them.”
He added that working on safety research at Anthropic is his way of reducing the chance for “extinction-level bad outcomes.”
Most commenters were truly confused about how AI could bring about the end of humanity. Some posted images of plugs being pulled out of walls to demonstrate how humans could potentially fight back against the machines.
Others indicated that things weren’t that simple. They pointed to the creation of a novel virus with no cure, weapons creation, overtaking the power grid, and other potential disruptors as examples of what AI could do to cause mass destruction.
Some commenters argued that the whole controversy is akin to climate alarmism and is being overblown. Many speculated that the whole public warning was nothing more than an attempt to drum up publicity.
“Everything these frontier AI clowns do is a marketing stunt. There is always an angle. LLMs are not some sentient, destructive force. I can’t believe we have looped back into the doom hype phase. Cue Amodei and Altman saying how it will take all of our jobs again,” one commenter wrote.
Anthropic and OpenAI did not immediately respond to a request for comment.
.png)
.png)
