Unlike the security proposition in the era of cloud computing, security threats in the era of AI come not only from the outside but also potentially from the inside, and the nature of these threats is more complex and diverse. It is evident that security challenges are expanding across the entire lifecycle of enterprise AI systems, necessitating urgent upgrades in protection measures.

Author | Dou Dou

Editor | Pi Ye

Produced by | Industrialist

In just 10 minutes, a whopping 4.3 million yuan was swindled away. This is just the tip of the iceberg of criminal cases caused by AI-powered face swaps, voice changes, and other technologies after the popularization of AI technology, directly affecting consumer safety.

According to QiAnXin's '2024 Artificial Intelligence Security Report,' deepfake fraud cases based on AI surged by 3000% in 2023.

Generative AI not only brings benefits such as speed, scale, precision, and advancement to enterprises but also lowers the technical threshold for attackers. In the past, it might take hackers months to create an attack virus, but now with AGI (Artificial Intelligence Generative Model) tools, it can be generated in minutes, significantly increasing the efficiency and scope of hacking attacks.

Another set of data from DeepInstinct's fourth edition report indicates that 75% of security professionals witnessed an increase in cyberattacks in 2024, with 85% of these attacks driven by generative AI.

Hackers not only leverage AI technology to threaten enterprise organizations but also attempt to attack enterprises' AI models, using the enterprises' own AI against them. For example, they might make the enterprise's AI make incorrect predictions about the supply chain or make chatbots generate hateful sentiments. They can also 'crack' the enterprise's LLM using language-based prompts, causing it to leak confidential financial information, create vulnerable software code, or provide incorrect cybersecurity response suggestions to the enterprise's security analysts.

Furthermore, hackers covet enterprises' large model infrastructures. For instance, at the beginning of 2024, a computing power cluster with thousands of servers in the United States was hacked and used to mine Bitcoin.

AI is posing an all-around 'threat' to enterprise security, encompassing cybersecurity, software security, data security, host security, ethical security, and hostage security, among others, exhibiting complexity and diversity. This forces many enterprises to increase their budgets to cope with the various variables brought about by the AI era.

According to IBM's 'Cybersecurity in the Era of Generative AI Research Report,' executives stated that their AI cybersecurity budgets in 2023 increased by 51% compared to 2021 and are expected to increase by another 43% by 2025.

In the new round of technological revolution led by AI, technology service providers need to leverage AI technology to enhance security capabilities and operational efficiency while helping combat new risks brought about by new technologies. For enterprises, keeping up with the times is crucial, but avoiding dangers along the way has become an urgent issue to be addressed.

How to solve the problem of security in the era of AI?

I. The 'dark side' of AI technology development

'In a narrow sense, data security means protection against tampering and destruction, but in a broader sense, it encompasses data reliability, security, and the security of service content. In the era of AI, the connotation of data security will be expanded, and the risks will also be amplified,' said Wu Hequan, Academician of the Chinese Academy of Engineering.

A fact is that in the era of AI, the definition of data security is changing. The vulnerability of AI technology and its strong dependence on data make enterprises face unprecedented data security challenges while enjoying the convenience brought by AI.

The first step for enterprises to achieve AI empowerment is to collect a large amount of diverse and representative training data. However, the sources of these data may not be secure, and there is a risk of theft during the collection process, which may even violate privacy policies and laws without consent.

After data collection, it needs to undergo cleaning, labeling, and enhancement processes. If malicious data is injected during this process, it will directly lead to deviations in model training, thereby affecting the accuracy of model decisions. Liu Qianwei, Chief Scientist of QiAnXin Data Security, once emphasized, 'The tamper-proof requirement for labeled data is very high because it is the most critical guarantee for ensuring the quality of large models.'

Next, based on the underlying large model training, proprietary data is used to fine-tune and train the base model and save it as a new version.

In the case of collaborative models, such as those developed jointly by universities and enterprises or between enterprises, data resources need to be shared to enhance model training effectiveness and conduct joint model training and optimization.

Regardless of the model, certain data security issues will arise. For example, in the underlying large model training and collaborative models, both parties may want to utilize each other's data but be reluctant to fully share their own. If data is entrusted to a third party, there is no guarantee of impartiality and security.

Furthermore, model design, training, tuning, testing, and deployment stages cannot be overlooked. Especially in cloud deployment models, enterprises need to choose between private clouds, community clouds, public clouds, or hybrid clouds based on their business needs. However, each deployment model faces different data security issues.

For private clouds, if local stored data lacks effective encryption and access control, it may be accessed by unauthorized users. Employees with access to sensitive information may inadvertently or intentionally leak data.

Public clouds are even more vulnerable, as their cloud-based data is more susceptible to hacking and data breaches. Users of public cloud services must configure authentication and access control correctly to prevent data leaks.

Hybrid cloud architectures often involve infrastructure from different cloud computing providers, and data can be intercepted during transmission. In hybrid cloud environments, authentication and authorization become complex, requiring unified identity and access management tools to ensure security. In hybrid clouds, data loss risks increase significantly if proper redundancy and backup measures are not in place.

It can be said that in the face of the complexity and dynamism of the AI era, traditional data security measures are no longer adequate to address new challenges. Enterprises need to adopt comprehensive security strategies and continuously improve their technologies to minimize data security risks. This requires enterprises to remain highly vigilant throughout data collection, processing, storage, and transmission to ensure data security and reliability.

II. AI versus AI

In the context of the AI era, enterprise data security has become a significant issue that cannot be ignored. To ensure data integrity and confidentiality, a series of strategies and technologies have emerged.

Data encryption, as the foundational defense in data governance, acts like an impenetrable protective suit for data, effectively resisting external threats during data transmission and storage. Additionally, the introduction of homomorphic encryption technology adds a new dimension to data encryption protection. It allows data analysts to process data without decryption, safeguarding data security while meeting analytical needs.

In data labeling and enhancement, access control and multi-factor authentication mechanisms play crucial roles. They function like guards at a castle, strictly controlling data access to ensure only authorized users can access the data. Meanwhile, data masking technology adds a veil of mystery to sensitive information, effectively protecting data privacy even during necessary exchanges.

In model security, defensive mechanisms against adversarial attacks and model watermarking technologies provide safeguards. They act like traps and sentinels within the castle, constantly vigilant against potential threats, ensuring models are protected from malicious attacks and unauthorized use.

Addressing the challenges of data sharing in the AI era, privacy computing and multi-party homomorphic encryption technologies offer effective solutions. Through specific key controls, they enable encrypted data computation, ensuring data is both usable and invisible during sharing. Additionally, the combination of distributed storage and key control technologies facilitates on-demand data retrieval and decrypted computation.

It is noteworthy that in the era of generative AI, data security challenges are becoming increasingly complex and diverse. Generative AI, with its powerful creativity, continuously generates new data and content, posing new risks to data security. These risks are often dynamic and unpredictable, making traditional security detection methods potentially ineffective.

Faced with this challenge, enterprises cannot rely solely on manual or fixed detection methods; instead, they must leverage AI to combat AI.

Currently, an increasing number of enterprises are applying AI technologies to data security, using machine learning and deep learning to monitor network traffic in real-time, identify abnormal behaviors, and respond quickly to potential security threats. AI not only automates the formulation and implementation of security policies, improving the efficiency and accuracy of security protection, but also learns normal network behavior patterns to identify unusual behaviors and enable early threat detection.

Research from IBM's Institute for Business Value shows that 64% of executives have identified cybersecurity as the top priority for generative AI use cases, with 84% indicating they plan to prioritize generative AI cybersecurity solutions over traditional ones. This trend signifies the growing mainstream adoption of AI in data security.

Take Alibaba Cloud's real-person verification product as an example. It effectively counteracts Deepfake attacks, offering eight Deepfake detection capabilities, blocking up to 250,000 attacks daily with a 99% attack blocking rate. The model also updates itself every few minutes, adapting to evolving offensive and defensive techniques to ensure data security.

In conclusion, amidst the ever-changing risk landscape, technology service providers must harness the power of AI to address new data security challenges. By continuously developing and optimizing AI security products and technologies, they can better protect enterprise data security and support the healthy development of AI technology.

III. Security lowlands amidst technological peaks

A thought-provoking set of data reveals that three-quarters of China's 26 listed cybersecurity companies were in the red in 2023. However, just in the first half of 2024, multiple far-reaching data breaches were reported, involving a staggering number of records exceeding one billion.

This stark contrast highlights a grim reality: while enterprises have an urgent need for security technologies, technology providers generally face losses.

What lies behind this?

The '2022 China Enterprise Data Security Status Survey Report' sheds light on the crux: Over half of enterprises believe that market offerings for data security products and services meet less than 60% of their needs. This indicates significant shortcomings in existing data security solutions, leading to underwhelming investments in data security that fail to effectively mitigate data breaches and cyberattacks, ultimately resulting in financial losses.

There is a severe mismatch between demand and supply.

Moreover, in an increasingly complex security management environment, enterprises often grapple with numerous fragmented security issues. Public data shows that enterprises deploy an average of 76 security products and tools, while the penetration rate of hybrid clouds is also rising year by year, undoubtedly increasing the complexity of security management.

Therefore, a comprehensive solution to AI security issues is crucial.

A fact is that unlike the security proposition in the era of cloud computing, AI-era security threats emanate not only from the outside but potentially from within, such as misjudgments, biases, and data breaches in AI systems. These threats are more complex and diverse, encompassing data privacy, algorithmic biases, model security, and more. Security challenges extend across the entire lifecycle of AI systems, from design, research and development, training, testing, deployment, usage, to maintenance.

Hence, AI security protection measures require further enhancement. This enhancement involves not only normative improvements but also specific technical and management measures.

For instance, stringent data privacy protection regulations should be formulated to clarify the scope and purpose of data usage, preventing data misuse and breaches. Uniform AI security certification standards and norms should be established to evaluate and certify AI technologies and systems. An AI security standard system should be created to ensure all AI systems adhere to corresponding security norms.

On September 9, the National Technical Committee for Standardization of Cybersecurity officially released the 'Artificial Intelligence Security Governance Framework' version 1.0 (hereinafter referred to as the 'Framework'), proposing AI security governance principles. The Framework classifies AI security risks into endogenous and application-based risks and outlines technical countermeasures for each. Its release will facilitate the secure and orderly development of the AI industry, accelerating the standardization of AI technologies.

In terms of specific measures, several AI security solutions have emerged in the market. For example, QiAnXin Group has launched China's first comprehensive AI security response plan, encompassing AI security frameworks, solutions, assessment services, and detection tools.

Simultaneously, large model vendors have introduced full-stack security protection systems. Alibaba Cloud offers a full lifecycle security protection system for models, while Baidu Intelligent Cloud provides full-stack service capabilities. These solutions encompass data collection, model design, training, evaluation, deployment, and usage, providing a range of security products and technical support to help enterprises build a comprehensive data security protection system.

Going forward, as more AI applications are deployed, the definition of data security may broaden, and the challenges will only intensify. Only by remaining vigilant, continuously innovating, and strengthening collaboration can we collectively build an impregnable wall of data security.