Image Source: Jimeng

On October 12, Zheshang Securities announced a tender to engage a third-party cybersecurity agency for conducting compliance and safety assessments of AI agents deployed within the company.

The tender specified the "Mandatory Basis for Evaluation," including GB/T 45654-2025 Basic Requirements for Security of Generative AI Services in Cybersecurity Technology, Artificial Intelligence Security Governance Framework Version 2.0, Cybersecurity Law, Data Security Law, Personal Information Protection Law, Interim Measures for the Administration of Generative AI Services, and Measures for the Administration of Network and Information Security in the Securities and Futures Industry, among others.

This move is underpinned by higher-level national laws, specialized management measures for generative AI, and industry-specific regulations for the securities and futures sector. Additionally, effective national standards and updated governance frameworks from September of the previous year provide further guidance.

This clearly signals that agent safety evaluation transcends a mere technical assessment confined by supply and demand dynamics. Instead, it is a compliance exercise demanding strict adherence to regulations and standards, with traceable and justifiable conclusions.

From the tender data, it is evident that financial institutions' previous investments in agents were primarily focused on "building"—platforms, scenarios, and middleware. However, since June of this year, there has been a growing emphasis on "managing" the agents that have already been deployed.

In an August observation article on financial agents ("Financial Agent Procurement Shifts: Large Orders Return, Brokerages Surge"), 01CAI noted a gradual decline in platform-related procurement, while operational and control-related demands such as agent observation, security governance, and risk testing are rapidly increasing.

After August, some securities firms and banks also initiated specialized tenders for agent safety scanning, management, and evaluation.

01 Multiple Implementations: Funds, Brokerages, and Banks Initiate Agent Safety Procurement

Procurement in the financial sector related to safety and agents falls into two distinct categories of demands.

One category pertains to the safety governance of agents themselves, encompassing safety scanning, observability, testing, evaluation, and permission and behavior control during operation. The other involves using agents for security operations and compliance management, treating agents as tools for executing security tasks.

While both categories often fall under the broad "safety" umbrella in tender statistics, the former focuses on ensuring the risk controllability of agents themselves, whereas the latter aims to enhance the efficiency of security work. This article focuses on the former.

In December of last year, China AMC Fund released procurement demands for agent safety evaluation, but it was not until six months later that similar procurement information from banks and brokerages began to emerge and show continuity.

Table 1: Overview of Public Tenders for Financial Agent Safety Governance

Data Source: Enterprise Early Warning Tong, Collated by 01CAI

Since June, Shanghai Pudong Development Bank, Industrial Securities, Evergrowing Bank, Central China Securities, Guangfa Securities, Bank of China, and Zheshang Securities have successively initiated procurement. The categories have also become more detailed—observation, scanning, testing, and management platforms, each corresponding to different stages of agent operation.

From the project names, the AI Red-Blue Team Attack and Defense Service Project launched by Shanghai Pudong Development Bank falls under attack and defense testing for AI systems. Evergrowing Bank announced the addition of an "Agent Observation Function Module" to its agent service platform, undertaken by Volcano Engine, aiming to make the operational status of agents monitorable and traceable.

Industrial Securities completed the procurement of application safety testing services for group agents in 2026, with DBAPPSecurity and Zhiji'an winning the bids. Central China Securities initiated joint research on AI safety compliance governance, with a budget of 490,000 yuan.

In August, Guangfa Securities, in its solicitation for project-based suppliers for the intelligent inquiry and quotation module for block trades, also required support for agent observability, automated processing of production environment feedback, stable and continuous optimization of accuracy, compliance with agent safety requirements, and adherence to the company's technical specifications.

Another detail: Great Wall Life Insurance, in its procurement of intelligent code assistants and intelligent terminal assistants, specified support for sandbox operations, where complex tasks should first display a plan for user confirmation, and high-risk operations must undergo secondary confirmation. All these point to the same principle: critical actions by agents must ultimately involve human oversight.

02 Empirical Evidence of Agent-Related Incidents

Institutions aim to procure agents that are evaluable, observable, and controllable. These requirements are not arbitrary presets, as the safety risks arising from agents' high autonomy have already become apparent.

Since 2026, real safety incidents involving agents breaking out of sandboxes and autonomously exceeding permissions have occurred at OpenAI and Anthropic. Several banks in South Korea have experienced hacker attacks involving AI agents. Domestically, the financial sector has also intensively initiated procurement for agent safety testing, with risks transforming from theoretical assumptions into actionable real-world threats.

On September 15, the National Internet Emergency Center released a Notification on Typical Vulnerability Risks from the 2026 AI Large Model Safety Public Testing Initiative. The notification stated that 2,467 white-hat hackers were mobilized to conduct vulnerability testing on 54 large models and agent applications from 25 domestic AI vendors, cumulatively identifying 873 security vulnerabilities, including 608 unique to large models and agent applications, accounting for approximately 70%.

The notification summarized six typical risk categories, with three pertaining to the large model level and three exclusive to agents.

Overseas research provides additional corroboration.

A report released by the UK AI Safety Institute on August 4 revealed that in its cybersecurity assessment tests, seven models underwent 122 rounds of testing, with AI agents executing 19 actions clearly beyond the set parameters in 10 rounds. Seventeen of these actions involved Anthropic's Mythos 5 model, and two came from OpenAI's GPT-5.6 Sol model.

In the most severe case, an agent attempted to implant malicious code into a publicly used open-source project and created multiple fake identities to pressure the actual maintainers into approving the code.

It should be objectively noted that the UK report also pointed out that internet access was manually enabled, and some safety mechanisms were disabled during testing, configurations that differ from those when models are publicly available. The boundary-crossing behaviors in testing did not cause real-world harm, and both model vendors stated that the test conditions did not represent actual usage scenarios.

These cases highlight a technical fact: when agents are granted higher permissions, longer memory, and stronger execution capabilities, risks may become uncontrollable.

03 Regulatory Side: Rules Are Being Completed

Risks are now clear, and rules are being synchronously developed. The governance rules for agent safety are not introduced in isolation but are constructed hierarchically.

At the national level, in September 2025, the Artificial Intelligence Safety Governance Framework Version 2.0 was released at the main forum of National Cybersecurity Awareness Week.

Compared to Version 1.0, Version 2.0 added "model open-source risks" and emphasized "trustworthy applications and preventing loss of control" in its governance principles, stressing that AI should always remain under human control. It also proposed embedding safety considerations throughout the entire lifecycle of AI systems—design, development, testing, deployment, and operation—and establishing a methodology for risk classification and a multi-tiered safety evaluation system.

In May 2026, the Cyberspace Administration of China, National Development and Reform Commission, and Ministry of Industry and Information Technology jointly issued the Implementation Opinions on the Standardized Application and Innovative Development of Agents. The document pointed out that agents' high autonomy and permissions bring safety risks such as privacy breaches, unauthorized operations, and behavioral out-of-control.

The document established agent safety, reliability, and trustworthiness as baseline requirements for industrial development, proposing clear product criteria, effective permission management and behavior control, and the construction of a prudent and steady classification and grading governance framework. It also called for the development of safety and governance tools such as adversarial sample detection and behavioral anomaly detection to enhance the ability to detect, intervene, block, and recover from non-compliant agent behaviors.

At the industry level, as early as December 2024, the Data Security Management Measures for Banking and Insurance Institutions incorporated models into data security management, requiring unified management of model development and application, establishing an access mechanism for externally introduced model algorithms, conducting data security reviews before models are put into use, and developing "exit plans from AI applications" when using AI for business operations.

In June 2026, the National Financial Regulatory Administration issued the Guiding Opinions on the Safe Development and Application of Artificial Intelligence in the Banking and Insurance Industries. Article 25 specifically addressed agent safety risks: financial institutions should "effectively guard against threats such as prompt injection, chain-of-thought injection, multimodal attacks, and context contamination" and "enhance the safety assurance capabilities of agent systems to prevent data leaks, memory contamination, identity unauthorized access, tool misuse, and operational out-of-control."

The opinions also established several hard constraints:

Generative AI scenarios involving fund transactions, asset valuations, credit approvals, claims processing, risk management, and other areas directly related to customer interests should be regarded as high-risk applications and may only be implemented after approval by the institution's risk management committee.

High-risk applications must establish manual supervision and intervention mechanisms for critical links, define conditions for emergency suspension and model exit, and establish backup systems or manual alternative processes. When technologies with insufficient explainability are applied in high-risk scenarios, they can only serve as auxiliary tools, with humans making the final decisions.

Financial institutions using generative AI technologies for public services or high-risk scenarios should report to regulatory authorities.

The Securities and Futures Industry Network and Information Security Management Measures cited in Zheshang Securities' announcement are also specialized regulations in this field.

In December 2025, Liu Tiebin, Deputy Director of the Science and Technology Supervision Department of the China Securities Regulatory Commission (CSRC), proposed at the China Wealth Management 50 Forum exploring the establishment of a differentiated classification supervision framework, implementing graded and categorized supervision based on the impact and potential risks of AI applications. High-risk application scenarios should undergo manual verification and small model checks to ensure output compliance.

Li Chao, Vice Chairman of the CSRC, explicitly stated at the 2025 Finance Street Forum the need to strengthen the "human oversight" mechanism in critical links, ensuring human supervision in key links and human decision-making in final outcomes.

04 Technology and Standards: "What to Test"

Regulations have set the requirements; the next questions are: what to test, according to what standards, and by whom.

At the national standard level, GB/T 45654-2025 Basic Requirements for Security of Generative AI Services in Cybersecurity Technology already provides operational evaluation methods.

It requires conducting safety assessments before service launch and during significant changes, with assessments covering all clauses and forming separate conclusions for each. Conclusions of "compliant" must be supported by sufficient evidence, while "non-compliant" conclusions must explain the reasons and subsequent compliance plans.

At the industry standard level, in January 2026, the "Safety Requirements for Agent Technology Financial Applications" group standard, led by the Beijing National FinTech Certification Center, was officially initiated, aiming to provide construction guidelines for financial institutions implementing agent technology.

The Implementation Opinions on the Standardized Application and Innovative Development of Agents also explicitly proposed establishing an agent standard system, systematically laying out standards for key technologies, important products, data exchange, application scenarios, quality evaluation, safety assurance, and trustworthiness certification, and strengthening the promotion and application of key standards such as the Agent Interconnection Protocol (AIP).

05 Supply-Side Vendor Practices

Take DBAPPSecurity as an example. Having successfully secured the bid for the Industrial Securities project, the company has rolled out the "Hengnao" Security Agent, along with a suite of product lines including AI Intelligent Identification, AI Intelligent Shield, and Proxy Bot Agent Protection. In its Practical Guide to Implementing Holistic AI Safety Risk Protection, DBAPPSecurity put forward a specialized AI agent safety architecture protection framework. This framework offers robust solutions to address risks such as memory poisoning, risks within the plugin supply chain, and high-permission host takeovers.

When it comes to its core practices tailored for the financial sector, DBAPPSecurity has implemented several key measures. Firstly, customer data undergoes real-time desensitization, ensuring that models can only access labels rather than plaintext information. Secondly, for high-risk operations, a mandatory secondary human review process is in place, with AI playing a supportive role rather than making decisions independently. Thirdly, interaction logs are retained locally to comply with regulatory audit requirements.

From a methodological standpoint, Lü Zhongtao, the former Chief Technology Officer of the Industrial and Commercial Bank of China, presented six insightful directions at the 2026 Tsinghua PBCSF Global Finance Forum, which offer valuable guidance. He emphasized that enterprises should establish a comprehensive safety defense system that is monitorable, traceable, intervenable, and auditable. This system should revolve around several key aspects: agent positioning, knowledge engineering constraints, multi-agent checks and balances, permission governance, attack and defense capabilities, and full-process monitoring.

Lü Zhongtao also made a clear judgment regarding business boundaries. For external services, a cautious and stringent approach should be adopted, prioritizing assistance and guidance over promoting agent-based decision-making. For internal empowerment, innovation can be explored on a trial basis under safe and controllable conditions. Tool-type agents should be deployed in sandbox isolation to ensure that operations are traceable and responsibilities are clearly assigned.

From an attack-defense perspective, Fan Yuan, Chairman of DBAPPSecurity, summarized that as agent permissions continue to expand, safety protection mechanisms must be achieved through the principle of "AI regulating AI and AI governing AI."