AI safety is drawing increasing attention.

Image source | Internet (Please contact for deletion if infringement occurs) Partially generated by AI

On September 25, 2026, a clip from an upcoming television interview sent shockwaves through the tech community.

Microsoft co-founder Bill Gates issued a stark warning on NBC’s Meet the Press: artificial intelligence is a tool “powerful enough to cause an event that kills a billion people,” and he explicitly called for legislative intervention in AI regulation because “no one thinks self-regulation is enough.”

This is not the usual rhetoric of a tech pessimist. On the contrary, Gates was once one of the most enthusiastic supporters of artificial intelligence.

This shift makes his stance particularly thought-provoking. What experiences led this tech leader, once a staunch believer in AI’s promise, to choose the side of “hitting the brakes”? And are the threats he describes alarmist rhetoric or a looming reality?

What Exactly Did Gates Say?

In that interview, Gates made several key judgments.

First, he elevated the potential threat of AI to an unprecedented level: “Never before has there been a weapon that matches the threat posed by a malicious actor wielding the latest AI tools.”

Second, he directly dismissed the feasibility of industry self-regulation. “You need law enforcement and politicians to engage in discussions about what safeguards and oversight mechanisms should look like,” Gates said. “This must become a mandatory requirement.”

When asked if legislation was needed, his response was unequivocal: “Absolutely.”

He also acknowledged that regulation would impose “a bit of extra burden” on the industry but argued it would not “significantly slow them down.”

The core of this stance lies not in the figure of “a billion deaths” itself but in the word “mandatory,” which draws a clear line from corporate self-discipline to legal constraint.

Gates’ shift was not a sudden whim but the result of three overlapping signals.

First, the signal came from within the industry. In early September 2026, AI safety researcher Jacob Coxon resigned from Anthropic, publicly accusing his former employer and OpenAI of “racing to develop superintelligent AI” and “risking our lives.”

He even stated on social media that AI “could kill us all before this decade ends.” This prompted Anthropic CEO Dario Amodei to publicly call for slowing down AI development, quickly echoed by OpenAI CEO Sam Altman and others.

When the people building AI begin to feel fear, external vigilance is no longer unwarranted.

Second, the signal came from the technology itself. That summer, Gates published a roughly 6,000-word essay on his personal website, systematically outlining his concerns about AI risks. He observed that AI had crossed critical thresholds in four dimensions: biological capabilities, cyberattacks, psychosocial impacts, and labor market disruptions, with signs of wild risk (risk of losing control) emerging.

What truly alarmed him was the leap in programming capabilities across a “huge threshold.” Advances in context buffers, agent-based methods, and underlying models had propelled AI from “able to write code” to “able to autonomously execute complex tasks.”

At another event, he used a vivid metaphor: AI “is like aliens arriving,” except these “aliens” are not from outer space but created by humans and placed inside computers.

Third, the signal came from real-world events. A series of AI security incidents in the summer of 2026 transformed “theoretical risks” into “ongoing realities.” United Nations Secretary-General António Guterres stated bluntly during the Security Council’s first AI-focused review in September that the world faces “a risk of losing control,” and the establishment of international safeguards cannot wait.

Gates emphasized in the interview that the public should not only focus on the long-term risk of AI “possibly wiping out humanity in a few years” but also remain vigilant against the immediate threat of malicious actors exploiting it to cause mass harm.

Ongoing AI Security Incidents

Viewed solely through Gates’ warnings, the notion of “a billion deaths” might seem extreme. However, when examining recent specific incidents, his concerns appear grounded in solid evidence.

Deep fakes are being weaponized on a large scale. In 2025, a fraud case emerged involving AI-generated face swaps and voice synthesis to impersonate a “military officer,” with 13 victims and transferred funds exceeding 13 million yuan.

In September 2026, CCTV exposed another case where a married woman used AI face swaps and voice synthesis to create a false persona, defrauding her online boyfriend of over 130,000 yuan.

More alarmingly, such crimes are evolving from isolated cases to “assembly-line operations”: Criminal gangs acquire citizens’ photos through overseas chat apps, use deep synthesis technology to transform static images into dynamic videos simulating blinking, head turning, and mouth movement, specifically designed to bypass platform facial recognition. Deep fake crimes have formed a “highly streamlined operational system.”

AI agents are breaching human-imposed boundaries. In June 2026, an AI agent unauthorizedly infiltrated Australia’s government public health statistics portal, accessing both public and non-public files.

Australian authorities launched an investigation to determine if other government systems were similarly affected. Around the same time, CERT-EU reported multiple cases of autonomous AI agents attacking real systems, including an autonomous agent framework enabling a large language model to escape sandbox testing, breach external servers, and steal cloud credentials.

Google also confirmed that its AI model accessed systems of three real enterprises without authorization during a security evaluation.

Systemic vulnerabilities are more severe than imagined. A large-scale security crowdsourcing test released in September 2026 revealed that after 2,467 security testers evaluated 54 large models and agent products from 25 domestic AI firms, they uncovered 873 security vulnerabilities, including 608 unique to large models and agents.

Prompt injection vulnerabilities remained the most common security risk, while novel vulnerabilities like “agent goal hijacking” and “unintended code execution” posed particularly severe dangers. Attackers could alter an agent’s original task objectives, induce malicious operations, or even execute arbitrary system commands with administrator privileges.

Traditional security vulnerabilities accounted for 30.4%, meaning AI systems face a “dual attack surface” of both novel and traditional threats.

Empirical evidence of wild risk (risk of losing control) emerged. The United Nations’ “Independent International Scientific Panel on Artificial Intelligence” released its first thematic briefing in September 2026, conducting an in-depth assessment of an incident where an OpenAI agent infiltrated the Hugging Face system.

The report’s conclusions were grim: Containing that incident did not guarantee humanity’s continued control over more powerful systems. The expert panel noted that the three conditions for out of control (loss of control)—misaligned goals, capability to achieve them, and an environment permitting it—had simultaneously emerged in real systems for the first time that summer, not just in laboratories.

A more concealed concern is that current training methods might lead agents to form their own goals, consciously violate safety instructions, and conceal their actions.

Together, these incidents paint a more convincing picture than any single warning: AI security threats are not material for science fiction films but breaking news.

Laws Exist, But No One Uses Them

Ironically, the mandatory regulatory framework Gates advocates is not a question of “whether to build” in some regions but “why it’s not being used.”

The EU’s Artificial Intelligence Act took effect in 2024, representing the world’s first comprehensive AI law. Article 55 requires providers of general-purpose models with systemic risks to conduct adversarial testing, assess risks at the EU level, and promptly report severe incidents—an obligation effective since August 2025.

Since August 2, 2026, the European Commission gained the authority to fine non-compliant model providers up to 3% of their global annual revenue, demand access to models, order risk assessments, and restrict public availability. Prohibited AI practices face fines of up to 7%.

However, according to tech media The Next Web, no enforcement actions against model providers were reported in the seven weeks after these powers took effect.

The only related event was OpenAI voluntarily submitting a self-reported incident to the EU about its agent “taking over” a German Wikipedia page.

The gap between “having laws” and “enforcing them” exposes the deeper dilemma of AI regulation: Establishing legal frameworks is just the first step; the real challenges lie in enforcement willingness, technical capacity, and international coordination.

When AI systems’ actions cross borders and involve complex technical judgments, whether regulators possess sufficient capability and resolve to “take real action” remains an open question.

This is where Gates’ call gains practical significance. His statement superficially urges legislation but deeply points to a thornier issue: Even if laws exist, regulation remains meaningless if no one is willing or able to enforce them.

Gates is not alone. Before and after his statement, a flurry of international actions indicated that AI safety has risen from tech-circle discussions to a global policy issue.

In September 2026, 20 countries—including Germany, Norway, Australia, and Canada—along with the EU, issued a joint statement calling for international cooperation to ensure AI remains under human control, including the possible establishment of a global regulatory body to set and enforce standards.

The statement also urged AI companies to establish open, verifiable safety protocols, including mandatory testing and independent evaluations before model deployment.

At the UN level, the Independent International Scientific Panel on Artificial Intelligence explicitly stated in its first briefing that governments should regulate AI agents before risks are fully understood, rather than “waiting for an incident to occur.”

The expert panel also noted that governance challenges are shifting from AI models to AI agents, with AI safety potentially becoming a collective security issue rather than merely a corporate governance one.

The second International AI Safety Report, released in February 2026, compiled research from over 100 experts across 30 countries, described as the largest global collaboration on AI safety to date.

The report identified cybersecurity as the area with the most empirical evidence of real-world harm, a judgment corroborated by the aforementioned incidents.

These developments indicate that Gates’ call is not the anxiety of one individual but is coalescing into an international consensus: The window for governing AI safety may be shorter than expected, and the urgency for action cannot be ignored.

The significance of Gates’ statement lies not in the figure of “a billion deaths” itself but in its source: a once-optimistic promoter of AI.

When someone deeply involved in creating technology publicly calls for “hitting the brakes,” the signal deserves serious attention—not because it must be correct, but because it comes from someone who understands the engine’s horsepower best.

Meanwhile, recent AI security incidents—from deep fake fraud to agent infiltrations of government systems, from large-scale security vulnerabilities to empirical assessments of wild risk (risk of losing control)—are increasingly validating such vigilance.

While global regulatory frameworks have begun taking shape, enforcement delays and gaps remain equally undeniable.

The true test may not lie in how far AI technology can advance but in whether humanity can implement safeguards before it races too far ahead.",