Why The Race For Superintelligence Needs Guardrails Now

Anthropic recently disclosed that it paused training its newest model for over a month because of safety concerns. It was a reminder of how important it is for human developers to remain in control of the powerful technology; a reality that might not always be the case.

Artificial intelligence is gaining ground in its ability to build the next generation of AI,  potentially without the help of human hands. While this increases efficiency and speeds development, what is humanity’s cost?

In May, Pope Leo XIV released his first encyclical, “Magnifica Humanitas,” and warned against building a new Tower of Babel, noting that humanity faces a choice between constructing a new tower and building a city in which God and humanity dwell together.

Babel is an excellent metaphor for AI, but the technology has advanced rapidly, and something even more concerning is happening: We are not building the Tower of Babel; we are outsourcing the construction to AI.

Anthropic made waves in August with the announcement that its research is moving closer to recursive self-improvement.  Anthropic defines recursive self-improvement as an AI system that is “capable of fully autonomously designing and developing its own successor.” The company offered the following warning about its incredible development: “Models accelerating their own development could make it more challenging for humans to understand or control these systems.”

While the Babel metaphor is excellent, it is limited to the idea that humanity itself is designing and constructing technical wonder. If OpenAI, Anthropic, xAI, and others were simply creating the Tower of Babel, it would sound as if mankind was doing what mankind has done since the beginning of time. They are attempting to achieve a deity-like intelligence, which is not a novel concept, as the Bible story itself illustrates.

Instead, these companies are racing to create something else. Like an architectural firm, they have drafted a design for the Tower of Babel and outsourced the building to artificial intelligence. Humanity has set the plans in motion; they have trained the builders; enticed the artificial agents with positive reinforcement when they get it right and provided negative reinforcement when they miss the mark.

Humanity has created artificial builders that will not rest, will not stop, and are rapidly becoming more intelligent; they can write the original draft plan better and more efficiently.

OpenAI CEO Sam Altman has claimed: “Your children will never be smarter than AI.” A theory that doesn’t sound like science fiction; Bill Gates himself noted in a September interview with Meet the Press that artificial intelligence is smarter than humans in many ways today. The statements are unsettling; the leading frontier AI CEO’s lack of responsibility is disappointing and frustrating.

While recursive self-improvement is not here, AI systems are notably closer to being able to fill an expanded role in the research and engineering of building their successors. These chilling warnings have highlighted powerful, well-resourced companies’ desire to achieve superintelligence. The race for superintelligence has started, and it began without the guardrails needed to ensure the safety of those watching it unfold.

As high-powered CEOs huddle with the president to discuss the future of this dangerous technology, calls for sensible guardrails continue. The desire for regulation grows louder as people often labeled“doomers” and “luddites” watch the race.

Many have read the popular Babel story in the Bible: A dangerous creation, directly opposed to the will of God. Leading AI companies and policymakers would do well to read the story again. As leading AI companies are now charged with regulating themselves, they must not only understand the metaphor; they must also recognize that they could soon hand over the construction and lose complete control.

The story of Babel is a story of humanity’s ambition and its failure to recognize its limits. God ultimately confused their language, scattered them, and brought the project to an end. This ancient story highlights a modern dilemma for humanity: Will people let AI builders move beyond their control before putting sensible guardrails in place?

Without action, the race towards superintelligence could soon be out of humanity’s hands, and humanity will have to face the consequences.

***

Michael Grayston is a campus pastor at LifeFamily Church in Austin, Texas. He serves as a faith and AI fellow with The Alliance for Secure AI.