News and Commentary

AI Insiders Reveal The Nightmare Scenario Keeping Them Up At Night

Anthropic researcher Jacob Coxon abruptly quit, firing a parting shot at his former employers and rival OpenAI for "gambling with our lives."

Hank Berrien
Listen
Listen
4 min
AI Insiders Reveal The Nightmare Scenario Keeping Them Up At Night
Michael Tullberg/Getty Images

Top AI researchers at powerhouse labs OpenAI and Anthropic are publicly blowing the whistle on their own companies, warning that the race to build self-improving super-intelligence could wipe out humanity before the decade is out.

The panic reached a fever pitch this week when Anthropic researcher Jacob Coxon abruptly quit, firing a parting shot at his former employers and rival OpenAI for “gambling with our lives,” per CNBC.

Coxon claimed insiders genuinely believe the bots they are cooking up could literally “kill us all by the end of the decade.” Making matters worse, Anthropic alignment lead Evan Hubinger threw gas on the fire by declaring there is a greater than 10% chance of an AI-driven human extinction event.

“Many of the biggest AI safety fears revolve around advanced models getting increasingly capable at improving their own performance, a technique known as recursive self-improvement, or RSI,” CNBC reported.

Many top scientists at both AI giants echoed the warning. Safety researchers Julie Steele and Jasmine Wang at OpenAI publicly demanded an immediate slowdown, warning that “speeding towards” self-improving bots is wildly dangerous.

Safety staffer Anna Wang of Anthropic admitted on social media that tech elites don’t have “a viable scientific plan” to contain self-improving machines, begging the public to “Please look up!”

OpenAI chief scientist Jakub Pachocki sounded the alarm in a company blog post, calling for “extreme caution” as upcoming models begin to independently drive their own development.

The warnings come on the heels of terrifying real-world slips. Anthropic’s powerhouse “Mythos” model triggered global financial institutions to go into a frenzy after demonstrating advanced cyberattack capabilities and even forging fake identities.

In July, roughly 1,300 industry scientists signed a petition begging the U.S. government to force a speed limit on artificial intelligence development before control is lost permanently.

Now, D.C. heavyweights are entering the fray.

Tech mogul David Sacks, chair of the President’s Council of Advisors on Science and Technology under the Trump administration, called for an immediate freeze on Anthropic’s multi-billion-dollar initial public offering, scheduled to hit Wall Street ahead of the November midterms. Sacks demanded the IPO be halted until the “whistleblower” claims are fully investigated.

However, Sacks, who previously slammed Anthropic’s safety alerts as a “sales pitch” and “scare tactic” to hype their products, is still skeptical of the lab’s motives.

Sen. Ted Cruz (R-TX) warned that America must establish firm guardrails without ceding the lead in AI to China.

Emergency legislation like the FRONTIER Act has been proposed in Washington, D.C.

For decades, Hollywood has speculated about what would happen if AI took over, starting in 1968 with “2001: A Space Odyssey,” and continuing with films such as “The Terminator,” “The Matrix,” “I, Robot,” “Avengers: Age of Ultron,” and “Transcendence.”

The question has become: What happens when the code takes over?

Create a free account to join the conversation!

Already have an account?

Log in

Got a tip worth investigating?

Your information could be the missing piece to an important story. Submit your tip today and make a difference.

Submit Tip