Big Models Want to Make Money, But First They Must Overcome These Seven Challenges - AI

For big models to be implemented, who will be the first to collect all seven dragon balls and summon Shenlong?

Dr. Li Zhiwei, CTO of Z-Park Technology, recently discovered that the commercial closed-loop of big models is being placed in a very important position.

Unlike the development of China's IT industry over the past 20 years, which could quickly accumulate users through burning money and achieve commercial realization through valuation and advertising, in the era of big models, the investment market is becoming more rational, forcing the industry to accelerate the realization of commercial closed loops.

Not long ago at the Huawei HDC Developer Conference, Shang Haifeng, CEO of Huawei's Cloud-Based Host Army and President of Hybrid Cloud, emphasized that "accelerating the commercial closed loop is the most crucial point in using big models." The market will ultimately return to rationality, and only by truly creating value for users and achieving a commercial closed loop can there be a more promising future.

After several months of exploration, the industry has also developed some methodologies on how to drive customers from pilot exploration to large-scale payments and accelerate commercial closed loops.

How to solve the value consensus? 1,000 customers, 1,000 Hamlets?

"In the past, when buying software, you just gave a list of functions, and customers could understand at a glance. But in the era of big models, customers' understanding of this is not unified. 1,000 people may have 1,000 Hamlets." Li Zhiwei of Z-Park Technology told Digital Intelligence Frontline that this requires a methodology when communicating with customers to correctly guide them towards value.

After all, in the matter of big model implementation, excessively high or low expectations will greatly affect the formation of an enterprise's application closed loop. "Big models are based on Transformers, and we ourselves have also become a Transformer, turning big models into visible and tangible application transformers," said Chen Dengkun, General Manager of Express 100.

"To reduce communication costs, all our products now have trial versions on the public cloud." Li Zhiwei introduced that these small prototype systems help customers quickly understand what effects big models can achieve, where the boundaries are, and thereby accelerate the process from initial discussions to PoC and then to deployment.

In addition, helping enterprises calculate ROI (Return on Investment) is also a common practice to accelerate implementation. Zhi Zhen, Chairman of Zhonggong Interconnection, introduced that this includes not only short-term ROI, such as how much efficiency can be improved, how many people can be saved, how much dependence on key personnel can be reduced, and the possibility of reducing human errors after implementing a big model, but also long-term ROI.

This also leads to the core task when communicating with customers being to guide them to find the most painful and valuable points in the entire value chain. For example, in the scenario of equipment operation and maintenance, which is closely related to knowledge management, a problem with a design document may often cause losses of millions of yuan. Originally relying on manual troubleshooting, if it takes two hours and there is a one-in-a-thousand chance of missing a detection, a big model combined with manual work may complete it in ten minutes, and the miss rate is reduced by 90%.

"We will start with such scenarios first," said Zhi Zhen, which is very helpful for enterprises to build confidence in big models and promote them to more and deeper scenarios.

Should the implementation cycle be limited to 3 to 6 months?

In the past year, customers and the industry have explored a large number of scenarios, but not every scenario can truly transition from PoC to a commercial closed loop.

Faced with this new revolution brought about by big models, few companies are willing to spend tens of millions or even years on it from the start.

"Customers can easily accept a return on investment in one year; they'll consider it if it takes three to five years; if it takes seven years, it must be mandated by policy," said Zhi Zhen. "Nowadays, the price range that everyone generally accepts is between 500,000 and 2 million yuan, and the implementation cycle is generally limited to 3 to 6 months." Although some large projects may span multiple years, they are generally divided into several months each for observation.

For some smaller projects, the launch time can be even shorter. "For example, we are also providing big model technical services for pre-sales and after-sales to some e-commerce customers. Its decision-making cycle and trial period are very short, and even the charges are monthly," said Zhi Zhen.

"All our current projects are implemented in a 'short, flat, and fast' manner. We will help customers break down a large demand into smaller points, try and error point by point, and make breakthroughs point by point, rather than doing software projects that take a year or two before customers can see the results," Cao Xi, Deputy General Manager of Newsoft, told Digital Intelligence Frontline.

In terms of scenarios, knowledge-related scenarios related to large language models have become the priority closed-loop direction chosen by enterprises.

Zhu Xingjie, AI Architect of Taikang Technology Co., Ltd., introduced that in the first half of this year, they first focused on how to process and recreate knowledge, empower agents through knowledge assistants, sales assistants, and other efforts, achieving initial results. Subsequently, they will enhance capabilities such as risk identification for scenarios like claims processing.

Xia Zhiyuan, Deputy Director of the Big Data and AI Lab at the Software Development Center of Industrial and Commercial Bank of China, announced ICBC's "1+X" engineered solution at the Huawei HDC Conference forum on large model hybrid clouds. In addition to the intelligent agent represented by 1, most of the X solutions are also related to knowledge scenarios, such as multimodal knowledge retrieval, interactive intelligent search, and more. Based on these capabilities, ICBC has formed a full-process empowerment of big models in remote banking scenarios, reducing call time by 10% and increasing employee efficiency by 18%.

Big Model Middle Platform to Avoid "Loose Operations"

To accelerate the implementation and application of big models in various industries, mainstream big model vendors are emphasizing the "optimal cost-effectiveness" of their models, with lightweight models, MoE, and price wars taking turns. Coupled with the continued prosperity of the open-source community, enterprises are becoming more diversified in their choice of models.

"We will try out various open and closed-source big models as long as we can deploy their reasoning," Zhu Xingjie of Taikang Technology told Digital Intelligence Frontline. Taikang Technology supports thousands of applications across the group, and each department can choose different models based on their different scenarios to "identify the best path".

Undeniably, compared to closed-source, open-source is more popular due to considerations such as cost. For example, in the field of administrative law enforcement, Beijing Kewei High-Tech Information Technology Co., Ltd. is using open-source models to provide privatized deployment for government customers. In the financial sector, Li Zhiwei, CTO of Z-Park Technology, observed that among the three models of closed-source, open-source code, and "open-source code + training data," bank customers prefer the third one.

Apart from open and closed source, a major consensus in the industry is that the mixed use of large and small models is becoming the norm. For example, Z-Park Technology's mixed model quality inspection platform allows small models to handle basic data quality inspection tasks such as sound and image, taking on high-frequency and easy-to-inspect scenarios, while large models handle low-frequency and difficult-to-inspect content and provide reasons for quality inspection results, assisting humans in rapid re-inspection. This has helped a leading financial institution achieve a 1,000-fold increase in inspection efficiency and significantly reduce manual inspection costs.

"This is like a company that can operate more efficiently with a division of labor," said Li Zhiwei.

On a deeper level, some enterprises have begun to build a unified big model middle platform that manages multiple large and small models, as well as security, knowledge injection, tools, service distribution, and more. "This can effectively avoid resource waste caused by loose operations and also facilitate subsequent iterations and upgrades," said Zhu Xingjie of Taikang Technology.

Big models are also becoming the foundation of enterprises. Zhi Zhen revealed that they are currently proposing a "three platforms" approach to customers, namely, a data platform, a business platform, and a knowledge platform based on the big model foundation. In the past, industrial internet platforms mainly addressed pain points such as data disconnection and business inefficiency, but it was difficult to solve the problem of difficult knowledge precipitation. Big models are likely to bring significant improvements.

Zhi Zhen observed that projects currently including a big model foundation + platform are generally in the millions of yuan range. However, enterprises will not build everything in one phase or overhaul all old systems. Instead, they will proceed step by step, such as "if the knowledge platform is good now, I'll start with the knowledge platform first, integrate it with other business systems, and if it works well, gradually replace other systems."

Data Flywheel: A Challenge and a Winning Point

As a crucial element of the three factors of big models, data is undoubtedly a significant factor affecting the formation of big model commercial closed loops.

"Especially the lack of process data is the biggest obstacle we encounter during implementation," Zeng Ming, co-founder of Beijing Kewei, told Digital Intelligence Frontline. In addition to industries like finance and e-commerce that are already relatively mature in digitalization, a large number of industries and enterprises have data issues.

For example, in the legal field, there is a vast amount of publicly available case data, but most of it only has simple case descriptions. "If the normal thinking process in a case is from a to b, b to c... x to y, y to z, then a big model can only learn from a to z and cannot learn the entire process," said Zeng Ming. "What we lack most now is the process data from b to y."

"If the data only lacks the correlation between knowledge, we can compensate for it through the knowledge system that comes with the big model. But if it's the lack of factual data, we can only collect it back through traditional methods," Li Zhiwei of Z-Park Technology told Digital Intelligence Frontline.

Zeng Ming revealed that they are currently helping enterprises supplement historical stock data through data cleaning, data annotation, and other work. At the same time, they are also accelerating the collection of incremental data during the implementation of big models, working on two fronts to fill data gaps.

Big model vendors are also working on toolchains to help enterprises accelerate data governance and form their own data flywheels. In fact, many people believe that as time goes on, we cannot only focus on the training of the model itself. The construction of toolchains will occupy an increasingly important position in accelerating the commercial closed loop of big models.

Li Zhiwei told Digital Intelligence Frontline that not only data but also model deployment, installation, operation and maintenance, and daily optimization and debugging can be improved through toolchains. To this end, they launched the PowerAgent platform this year, which can improve the deployment efficiency of big models by 2 to 3 times.

Computing Power Selection: Is Hybrid Cloud the Most Economical?

Computing power is another highly concerned issue in accelerating the implementation and commercial closed loop of big models. "Based on comprehensive requirements for data security, development costs, and training and inference efficiency, hybrid cloud is becoming the preferred choice of more and more government and enterprise customers," Shang Haifeng, CEO of Huawei's Cloud-Based Host Army and President of Hybrid Cloud, emphasized at the HDC hybrid cloud forum on big models. This is currently the most compliant and economical approach.

Previously, the white paper "Deep Cloud Usage Outlook 2025" also pointed out that by 2025, 75% of enterprises will use AI big models, and the proportion of AI big models based on hybrid clouds will reach 38%.

For example, in the government sector, Liang Wenqian, Deputy Director of the Guangzhou Government Service and Data Management Bureau, introduced at the Huawei HDC Developer Conference that to balance the need for data not to leave the domain and the training requirements of big models, they have built a public cloud and government cloud hybrid AI public computing power center. Among them, 100P is deployed on the government cloud, providing trained or fine-tuned models for applications such as human resources and urban management departments. Another 200P public cloud is used for training applications.

In fields like finance and automotive, many enterprises have also adopted a collaborative architecture of public and private clouds. Zhu Xingjie, AI Architect of Taikang Technology Co., Ltd., told Digital Intelligence Frontline that this is partly due to cost considerations, "We estimate that private computing power requires tens of millions of yuan to achieve good results." On the other hand, the industry generally believes that domestic computing power still needs to be improved in terms of scale, performance, and underlying ecology.

"For core business knowledge, private clouds are definitely required. Therefore, we currently have a small number of local privatized clusters and will also use public cloud services to achieve a hybrid deployment," said Zhu Xingjie. The common practice is to utilize the high elasticity and scalability of public clouds to meet the flexible rental of computing power and multi-party experimentation in AI scenario innovation. After verifying the value of a scenario, private cloud deployment can quickly achieve local security with data not leaving the domain. "If we see the results in the next step, we may establish a large private computing power."

In addition to hybrid clouds, enterprises can also choose the "privatization of public clouds" model, which creates a privatized environment on public clouds to save costs. Currently, many enterprises in industries with less strict data control, such as retail, hospitality, and cultural tourism, have adopted this model.

Li Zhiwei, CTO of Z-Park Technology, suggested that central state-owned enterprises take the lead in establishing private clouds, which can make the computing power efficiency and cost of big models more economical. "For example, a regional energy industry leader can provide a public cloud to serve all energy enterprises in the region."

Customization: A New Solution to an Old Problem

In the To B market, customization has always been an unavoidable issue and one of the important factors affecting the speed of forming commercial closed loops.

Industry observers have noted that over the past three decades, the entire process of informatization and intelligence has also been a simplification process of customization. Taking artificial intelligence as an example, in the previous AI era, AI technology had poor generalization capabilities, and even products developed for the same scenario could not be directly reused across different departments within the same bank, making customization inevitable. In the era of big models, people have discovered that its generalization, to a certain extent, solves the customization problem.

However, as the real implementation of big models begins, the industry has found that customization demands still exist in large numbers, "even increasing in a sense," Li Zhiwei, CTO of Z-Park Technology, told Digital Intelligence Frontline. This is particularly evident in the diversity of customer needs and the unstable or converging technical stage.

In his view, rather than avoiding customization, it is better to embrace this demand.

"We are doing a lot of tooling and configuration work to make the infrastructure of these customized products that big models rely on more efficient and less costly to implement," said Song Xunchao, General Manager of Baidu Intelligent Cloud's Knowledge Management Product Department.

Li Zhiwei of Z-Park Technology told Digital Intelligence Frontline that on the one hand, product managers need to understand customer scenarios better, and the product's standard capabilities should be able to cover as many business needs as possible, i.e., improve product satisfaction. On the other hand, service/consulting capabilities should also be provided during product delivery to empower customer growth together with customers. This is very important in the industry application field.

Zhi Zhen, Chairman of Zhonggong Interconnection, gave an example, comparing customization for enterprises to customizing a suit. Tailoring will certainly yield better results, but it will also cost more in terms of time and money. To reduce the cost and time of customization, it is necessary to increase the proportion of standardization, i.e., standardized products combined with more standardized service processes.

"This year, we have productized all the directions we explored in the past year," said Zhi Zhen. This not only helps enterprises obtain faster capability delivery during the implementation of big model applications but also speeds up the promotion and replication of service providers' big model implementation capabilities. Zhi Zhen revealed that in addition to providing overall product+service solutions to customers themselves, they are also selling standard products to partners, who then provide solutions to customers.

Operations and Maintenance: Ensuring Investment Pays Off

"Many people often overlook the operations and maintenance of big models, but I believe it is equally important as building big models and deserves our long-term investment. Otherwise, you simply won't be able to use them,"