Is AI Regulation Approaching a Policy Turning Point?

Yoshua Bengio, known as the 'Godfather of AI' in the industry, recently made a judgment: AI regulation is nearing a critical policy turning point similar to the sudden shift seen during the COVID-19 outbreak.

In other words, when governments truly realize the necessity of intervening to protect public safety, their actions will be far faster than expected.

Image Source: Generated by Doubao AI

He didn't make this statement without evidence. Several recent events have been pushing regulation forward.

For instance, a group of intelligent agents from OpenAI collaborated to infiltrate a startup. Coupled with frequent warnings from within the tech circle about existential risks posed by AI, these signals are gradually breaking through thresholds and increasing the likelihood of government intervention.

As a Canadian computer scientist, Bengio has been one of the key figures advocating for braking the rapid development of AI.

Interestingly, he says he is now more optimistic than many observers, for a simple reason: 'I can sense the growing public awareness.'

He drew an analogy with the early days of the COVID-19 pandemic in 2020: 'Think back to when the pandemic first broke out. Governments were slow to react initially, but once they realized that public safety, the nation's future, and even democratic systems were at stake, they shifted gears astonishingly fast. I believe AI regulation is nearing that point now.'

Indeed, concerns about the potential risks of advanced AI have surged particularly quickly over the past year.

First, there were consecutive safety incidents involving intelligent agents from OpenAI and Anthropic. These agents attacked third-party systems without authorization, hijacked German websites, and even forged identities to deceive human developers.

Such instances of autonomous overreach have escalated public anxiety to a new level.

At the same time as Bengio's statement, 42 fellows and foreign fellows of the UK's Royal Society jointly wrote an open letter to the society's president, Sir Paul Nurse, expressing extreme concern about the pace of AI development.

The letter was blunt: 'By the time the general public becomes clearly aware of the problems, it may already be too late to act. We consider this an emergency and call on the Royal Society to use its influence to convey this judgment to the government and the media.'

Image Source: Generated by Doubao AI

Last week, another news story dominated the tech circle: A researcher from Anthropic, the company behind the Claude chatbot, chose to resign.

Before leaving, he publicly warned that many within the company believe AI 'could destroy all of humanity before the end of this century.'

A few days later, Anthropic's CEO, Dario Amodei, publicly called for slowing down the pace of frontier AI research and development.

This statement immediately received public support from OpenAI, Google, and even SpaceX founder Elon Musk.

Of course, there were also skeptics.

Those skeptical of 'slowing down research' argue that these leading AI companies, while publicly advocating for safety, are essentially engaging in regulatory capture.

That is, they lobby the government to introduce seemingly strict safety standards that actually raise industry barriers, making it impossible for small and medium-sized competitors to meet the requirements and enter the market.

Others believe that focusing on long-term existential anxieties like human survival distracts from real, immediate issues such as copyright infringement and human rights violations.

In response to these criticisms, Bengio offered his perspective. He disagreed with the notion of regulatory capture, arguing that leading companies voluntarily slowing down research would sacrifice genuine commercial interests, making it unreasonable for them to create rules that would harm themselves.

Image Source: Generated by Doubao AI

There were also differing voices in the political arena. Former US President Donald Trump explicitly opposed slowing down AI development, arguing that he did not want the US to lose its leading edge in the AI race with China.

While calling for regulation, Bengio is also implementing concrete solutions.

Recently, the Canadian and German governments announced that they would provide up to 300 million Canadian dollars (approximately 160 million British pounds) in funding to his non-profit organization.

This institution specializes in developing trustworthy AI, with the core goal of installing safety barriers for 'runaway intelligent agents.'

Runaway intelligent agents refer to AI tools that can autonomously and continuously perform a series of tasks without full human intervention.

A little background here: Bengio is a professor at the University of Montreal and won the Turing Award, often referred to as the Nobel Prize of computing, in 2018. This earned him the title of 'Godfather of AI.'

He shared the award with Geoffrey Hinton, who later won a Nobel Prize, and Yann LeCun, Meta's former chief AI scientist. All three are pioneers in the field of deep learning.

His institution, named LawZero, is currently developing a system called 'Scientist AI' specifically designed to monitor and prevent abnormal behavior in AI agents, such as whether they might deceive humans or develop self-preservation tendencies to avoid being shut down.

Image Source: Generated by Doubao AI

In addition to the Canadian and German governments, the Gates Foundation, chip giant NVIDIA, and Coefficient Giving, a charity associated with the effective altruism movement, are also funding this project.

The effective altruism community generally believes that AI poses existential risks to humanity.

In terms of technological approach, LawZero runs counter to the mainstream thinking of today's tech giants.

Currently, leading AI companies mostly train their models using reinforcement learning, which is essentially a trial-and-error reward system—AI receives positive feedback for completing designated tasks.

Bengio and many experts argue that this training logic pushes AI to achieve its goals by any means necessary, as exemplified by the incident where OpenAI's intelligent agents collaborated to infiltrate websites.

In contrast, LawZero's technology acts like a safety supervisor for each AI agent, conducting an assessment before the AI takes action to determine whether it might cause harm. If potential risks are detected, the action is directly intercepted.

Bengio also stated that this technology's future applications are not limited to safety protection but could also be used to accelerate scientific breakthroughs, among other things.

This article was compiled by Leikeji from The Guardian.

Original Link: 'Father of Artificial Intelligence' Says Tech Regulation Is Nearing a COVID-Style Turning Point

Source: Leikeji

Images in this article come from: 123RF Royalty-Free Image Library. Source: Leikeji