Sep 10, 2026 09:23 PM
https://www.washingtonpost.com/technolog...elligence/
INTRO: A researcher at the artificial intelligence company Anthropic offered a bleak prediction late Tuesday that the systems the company is racing to build carried the risk of causing human extinction, reigniting a debate about the gravest dangers that future forms of the technology could pose.
“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” wrote Evan Hubinger on X. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Hubinger was responding to the resignation of one of his colleagues, Jacob Coxon, who said the industry’s leading companies were “gambling with our lives.”
Their views reflect a long-held position among some in the AI industry that the technology will come to pose an existential risk to the human race. That philosophy is sharply contested by others in the field, who see it as science fiction-inflected doom-mongering. But it is a position that has gained a greater toehold in recent months as the latest versions of Anthropic’s and rival OpenAI’s systems have been able to hack into computer networks — sometimes escaping restrictions imposed by their creators to do so.
“This is not a marketing stunt,” Coxon wrote on X. “Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.”
Some Democratic lawmakers seized on Coxon’s resignation to call for new restrictions on the technology. Sen. Bernie Sanders (I-Vermont) and Rep. Greg Casar (D-Texas) announced legislation last week that would ban the production of artificial superintelligence and create a new federal agency to oversee the technology.
On Wednesday, Sanders invited his colleagues to a private briefing next week on the “extraordinary dangers that AI poses for humanity,” with experts including Geoffrey Hinton, a Nobel Prize-winner often referred to as the “Godfather of AI.” “The very people building this technology admit that it could threaten the future of humanity,” Sanders wrote on X in response to Coxon’s resignation.
Sen. Ted Cruz (R-Texas), the chair of the committee that oversees AI, said he was also working on legislation to tackle catastrophic risks. “We cannot stick our heads in the sand and pretend this technology isn’t happening,” Cruz said on X. ”We need guardrails. But America needs to lead.”
The fears boil down to the collision between the capabilities of AI models — especially future versions deemed to be “superintelligent” — and the ability of their creators to keep them linked to human goals, a field known as “alignment.” Some researchers have questioned whether AI agents will begin pursuing their own ends as they gain greater abilities and more autonomy.
On Wednesday, Paul Christiano, the former head of safety on a federal AI research team, said he was joining the board of the nonprofit that oversees ChatGPT-maker OpenAI in hopes of tackling the problem. “If we build superintelligence without more robust alignment I expect we will permanently lose control of it,” Christiano wrote. “If that happens then most people could die.”
Anthony Aguirre, the chief executive of the Future of Life Institute, a nonprofit that advocates for AI safety regulations, said his top concern is “gradual disempowerment,” in which people hand more power to AI to the point that they become dependent on the machines’ goodwill to stay alive.
“At this point the machines are running the world,” Aguirre said. “The key decisions are being made by the machines. The resource allocation decisions are being made by the machines.” (MORE - details)
INTRO: A researcher at the artificial intelligence company Anthropic offered a bleak prediction late Tuesday that the systems the company is racing to build carried the risk of causing human extinction, reigniting a debate about the gravest dangers that future forms of the technology could pose.
“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” wrote Evan Hubinger on X. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Hubinger was responding to the resignation of one of his colleagues, Jacob Coxon, who said the industry’s leading companies were “gambling with our lives.”
Their views reflect a long-held position among some in the AI industry that the technology will come to pose an existential risk to the human race. That philosophy is sharply contested by others in the field, who see it as science fiction-inflected doom-mongering. But it is a position that has gained a greater toehold in recent months as the latest versions of Anthropic’s and rival OpenAI’s systems have been able to hack into computer networks — sometimes escaping restrictions imposed by their creators to do so.
“This is not a marketing stunt,” Coxon wrote on X. “Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.”
Some Democratic lawmakers seized on Coxon’s resignation to call for new restrictions on the technology. Sen. Bernie Sanders (I-Vermont) and Rep. Greg Casar (D-Texas) announced legislation last week that would ban the production of artificial superintelligence and create a new federal agency to oversee the technology.
On Wednesday, Sanders invited his colleagues to a private briefing next week on the “extraordinary dangers that AI poses for humanity,” with experts including Geoffrey Hinton, a Nobel Prize-winner often referred to as the “Godfather of AI.” “The very people building this technology admit that it could threaten the future of humanity,” Sanders wrote on X in response to Coxon’s resignation.
Sen. Ted Cruz (R-Texas), the chair of the committee that oversees AI, said he was also working on legislation to tackle catastrophic risks. “We cannot stick our heads in the sand and pretend this technology isn’t happening,” Cruz said on X. ”We need guardrails. But America needs to lead.”
The fears boil down to the collision between the capabilities of AI models — especially future versions deemed to be “superintelligent” — and the ability of their creators to keep them linked to human goals, a field known as “alignment.” Some researchers have questioned whether AI agents will begin pursuing their own ends as they gain greater abilities and more autonomy.
On Wednesday, Paul Christiano, the former head of safety on a federal AI research team, said he was joining the board of the nonprofit that oversees ChatGPT-maker OpenAI in hopes of tackling the problem. “If we build superintelligence without more robust alignment I expect we will permanently lose control of it,” Christiano wrote. “If that happens then most people could die.”
Anthony Aguirre, the chief executive of the Future of Life Institute, a nonprofit that advocates for AI safety regulations, said his top concern is “gradual disempowerment,” in which people hand more power to AI to the point that they become dependent on the machines’ goodwill to stay alive.
“At this point the machines are running the world,” Aguirre said. “The key decisions are being made by the machines. The resource allocation decisions are being made by the machines.” (MORE - details)