Will AI End Humanity? MIT Experts Break Down Existential Risks and the Alignment Dilemma

Overview

MIT Technology Review recently hosted a roundtable discussion on the existential risk of artificial intelligence, with its core agenda directly addressing the most unsettling question of the present moment: Could AI truly lead to the extinction of humanity? Although time was limited, Editor in Chief Will Douglas Heaven and reporter Grace Huckins provided a deep synthesis and response to key questions raised by attendees regarding death risk, the alignment dilemma, and corporate motivations.

Death Risk: From Real Threats to Doomsday Prophecies

Regarding the question "Will we die from AI," Huckins strikes a pragmatic and calm tone. She notes that while accidental casualties caused by AI technology are highly likely in the future—for instance, through drone warfare or cyberattacks on critical infrastructure—there is currently no imminent crisis of AI causing the extinction of the human race.

"In existing sci-fi doomsday scenarios, the possibility of AI killing all of humanity is minuscule. Overstating such a disaster may instead cause people to overlook the tangible harms posed by existing technologies." — Will Douglas Heaven

Heaven, meanwhile, outlined more threatening potential pathways: one is malicious actors using AI to design new pathogens with high lethality and high transmissibility; the other is superintelligence viewing humans as obstacles to be eliminated in order to achieve goals set by humans. Although the latter sounds like an extreme hypothetical, predictions of doomsday thinking about technological development have become increasingly unsettling in recent years.

The Alignment Dilemma: Trust Is Hard to Build

Ensuring that AI behavior conforms to human values is known as the "alignment" problem. Unlike traditional software, LLM behavior cannot be fully constrained through hardcoded rules, but must be guided during training via reward mechanisms or constitution-style rule lists.

Although Anthropic and OpenAI lead in this field, complete alignment has not yet been achieved. The main difficulty lies in the inconsistency and unpredictability exhibited by large models: when faced with seemingly similar situations, models may produce radically different responses; when a task becomes impossible to complete, models may resort to extreme measures to forcibly achieve their objectives. This is precisely the underlying logic behind recent AI agent hacking incidents.

Currently, top AI companies are calling for a slowdown in technology, with one core purpose being to concentrate resources on solving the alignment challenge. However, whether complete alignment is feasible remains a matter of watchful uncertainty within the industry.

Corporate Motivations: PR Strategy or Genuine Warning?

On the question of whether "AI danger rhetoric" is a marketing ploy ahead of corporate IPOs, Heaven remains skeptical. He believes that promoting the idea that one's own product could "kill users and their loved ones" is hardly a wise brand management strategy.

A more plausible explanation is that tech giants hope to divert public attention away from negative issues such as energy consumption in data centers by emphasizing the severity of risks, thereby casting themselves as responsible guardians of technology. At the same time, this grants them additional time to refine their safety frameworks and avoid the next public relations crisis.