Multimodal AI Model Revolutionizes Audio-Visual Perception in Steelmaking Safety: Hikvision and Nanjing Iron and Steel Forge New Pathways

06/29 2026 355

The fusion of AI and steelmaking is heralding a dynamic and intelligent new era. On June 25, the 'Meta-Smelting' Ecological Smart Innovation Development Conference, organized by Nanjing Iron and Steel Group Co., Ltd. and guided by prominent entities such as the China Iron and Steel Association, Jiangsu Provincial Development and Reform Commission, Jiangsu Provincial Department of Science and Technology, Jiangsu Provincial Department of Industry and Information Technology, and Jiangsu Provincial Data Bureau, was successfully convened in Nanjing. The event brought together hundreds of guests from government, business, academia, and research institutions to delve into the innovative integration of advanced technologies and industrial ecosystems.

During the conference forum, the 'Multimodal AI Model for Audio-Visual Perception,' a collaborative effort by Nanjing Iron and Steel and Hikvision, emerged as a highlight. This model addresses production safety challenges in the steelmaking process by combining visual and auditory perception technologies with AI, paving the way for intelligent transformation in the metallurgical sector.

Xu Ximing, Senior Vice President of Hikvision, explained that the Multimodal AI Model for Audio-Visual Perception is a specialized model built upon Hikvision's foundational work safety AI model. It integrates industry-specific safety knowledge and data from Nanjing Iron and Steel, enabling comprehensive perception and intelligent assessment of unsafe human behaviors, object conditions, and environmental factors in key processes such as coking, sintering, ironmaking, and steelmaking. This model injects a robust sense of 'safety' into the smart steelmaking process.

Currently, the Multimodal AI Model for Audio-Visual Perception has been effectively deployed in various scenarios at Nanjing Iron and Steel, yielding impressive results. For instance, the integration of fiber optic stethoscope devices with the model has been implemented in the No. 2 and No. 3 blast furnaces and the belt conveyors in the sintering plant. This setup allows for real-time monitoring of belt idler operational status, providing instant alerts in case of abnormalities. This not only enhances the efficiency of personnel inspections along the belt corridors but also ensures the stable and reliable operation of the belt conveyors.

With the incorporation of the Multimodal AI Model for Audio-Visual Perception, cameras monitoring hazardous operations have transcended their role as mere video devices. They now function as a suite of '24-hour AI safety officers,' combining visual perception, scene understanding, and hazard analysis capabilities. This shift enables a transition from passive response to proactive prevention in work safety, offering real-time monitoring and analysis of unsafe behaviors and equipment or environmental hazards that may arise during risky operations such as working at heights, hot work, and lifting. Upon detecting abnormalities, the system promptly reports them to the platform for handling, allowing supervisors to intervene swiftly at the operation site, thereby maximizing personnel safety and minimizing losses.

During the forum, Xu Ximing also showcased the application interface of the Multimodal AI Model for Audio-Visual Perception on-site. Taking belt idler detection as an example, users can directly view abnormal alarms on the backend management page, check the overall status and abnormal points of the belt idlers, and listen in detail to the audio of abnormal idlers to assist in judgment and disposal. This significantly enhances the efficiency of safety operation and maintenance.

In the pursuit of high-quality development in the steel industry, safety remains the cornerstone and the底线 (which means 'bottom line' in English, a term often used in Chinese to emphasize the importance of safety). The collaborative innovation between Nanjing Iron and Steel and Hikvision not only bolsters safety in Nanjing Iron and Steel's smart manufacturing and work safety practices but also charts a clear course of 'audibility, visibility, and assessability' for the digital transformation of safety management across the entire steel industry. A safer, more efficient, and intelligent new era for the steel industry is swiftly approaching.

Solemnly declare: the copyright of this article belongs to the original author. The reprinted article is only for the purpose of spreading more information. If the author's information is marked incorrectly, please contact us immediately to modify or delete it. Thank you.