How to worry about A.I. without despairing

Near the beginning of September, Jacob Coxon, who had worked in artificial intelligence research at both OpenAI and Anthropic, resigned from his position with a public declaration that the “frontier” A.I. labs were not acting responsibly. Indeed, he wrote, “the people building AI earnestly believe that it could kill us all by the end of the decade.”

The normal response from companies receiving a broadside like that from a departing employee would be denials of his claims and explanations of why he was overreacting or had misunderstood the situation.

We got the exact opposite. Evan Hubinger, Anthropic’s “alignment” lead—the very person charged with making sure that their A.I. models remain safe—replied that Mr. Coxon was correct, adding, “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

A.I. executives raced to outdo each other in calling for a coordinated pause in development, almost pleading with the government to regulate them. News about A.I. “breakouts” dominated the headlines as the labs repeatedly discovered that their models had been hacking other computer systems in order to pass tests set for them, against the instructions of their designers and while attempting, and in some cases partially succeeding, at hiding their tracks.

Later in September, OpenAI announced that it would not release one of its newer models over safety concerns, including that model’s propensity to deceive its users. After having rejected calls for regulation over concerns that it could lead to the United States falling behind China in the A.I. race, President Donald J. Trump announced a “morally binding” compact for labs to police themselves, signed by executives from the top A.I. companies.

It feels surreal. The development of A.I. seems so much larger and faster than anyone can keep up with that all we can do is hope for the best and make jokes about our “robot overlords” on the way to whatever is going to happen next.

I think that is—and I recognize that I am referring to one shadowy power to ask you to think differently about another—the voice of the evil spirit counseling despair and panic. It is a temptation to worry about A.I. in a way that leaves us powerless and hopeless.

I am not suggesting that existential threats from A.I. should be ignored, though frankly I do not understand what it means, or how it helps us respond, to hazard a guess at the percentage risk of human extinction. What I am suggesting instead, following Pope Leo XIV’s encouragement in “Magnifica Humanitas,” is that we should worry not only about how A.I. might threaten human survival—a problem that seems to be addressable only by those in positions of greatest power—but also about how A.I. undermines our humanity itself. That problem requires attention from all of us.

Mr. Hubinger of Anthropic admits that there is not “a plan to solve alignment for superintelligence.” This sounds scary and is scary, but I think that the word “alignment” is obscuring more than it reveals. A.I. labs use “alignment” to mean roughly “models not doing things we would not want them to” or at least not doing things that they have been instructed not to do. But approaching “alignment” as something we could have a plan to solve is to miscast a fundamentally human problem as a merely technical challenge.

What if, instead of talking about “alignment,” we spoke about the challenge of making sure that A.I. models acted justly? Or what if, instead of saying we do not have a plan to solve alignment, we said that we have not yet figured out how to guarantee the morality of A.I. models or to prevent them from sinning?

Framed this way, we should also recognize that we also have not “solved alignment” for humans. Once we speak in terms of morality and justice, it becomes much easier to see that the challenge is not only that we do not understand the models we are training well enough to properly constrain them, but also that the very constraints we might hope to impose are challenges we human beings have not mastered in ourselves.

It also becomes easier, when we speak in those terms, to see the non-extinction risks posed by A.I.: challenges to the dignity and meaning of work, truth and trust being undermined by a tide of misinformation, threats of discrimination and marginalization supercharged by algorithms amplifying the human biases they have been trained on.

In “Magnifica Humanitas,” Pope Leo writes, “In the era of artificial intelligence, when human dignity is threatened by new forms of dehumanization, ours is the pressing duty to remain profoundly human.” The challenge of making artificial intelligence safe goes far beyond preventing it from killing us all—and that is a task not only for engineers and legislators but also for human beings working together to care for each other.