The smart customer service industry is experiencing a surge of interest in large models.

"Now customers immediately ask if we can use large models to make calls. If we don't have this capability, they won't even talk to us," Wei Jiaxing, CEO of CloudBat, told Digital Intelligence Frontline. Market awareness of large model customer service is accelerating.

The industry also regards smart customer service as one of the first typical scenarios where large models can be applied. Large models can better understand user questions, generate responses that match user needs, and even predict user behavior and provide personalized recommendations. This significantly improves customer service efficiency and quality, while also saving companies a lot of manpower and time costs.

Cheng Yin, Research Manager at IDC China, told Digital Intelligence Frontline that in this field, enterprises investing in large models expect them to replace humans, not just assist them. However, many scenarios beyond customer service are not yet mature, largely because large models can only assist humans and cannot accurately measure ROI.

Rapid technological advancements and sharply reduced large model inference costs are undoubtedly accelerators for the application of large models in smart customer service. "Last year, the cost of a single call on ChatGPT was still a few cents, and the quoted price for a customer service call was 1.2 cents. At that time, the business model simply didn't work. Now, with the sharp drop in inference costs, a single outbound call using a large model only costs 0.2 cents," Wei Jiaxing mentioned, noting that this sudden drop in costs has accelerated commercial applications. He added that the current trend of technological upgrades and cost reductions is ongoing.

Amid this boom, two application paths are emerging in the market: established smart customer service companies are actively integrating large model capabilities, while large model companies are quickly entering the smart customer service arena.

However, large model smart customer service, still in its early stages of development, faces a series of challenges to overcome. For example, text-based customer service must overcome hallucinations, while voice-based customer service faces challenges with latency and anthropomorphism. Some companies are adopting a gradual strategy, prioritizing product launches in less demanding scenarios.

The era of non-idiotic smart customer service may be upon us.

01

Inference Costs Decline, Accelerating Large Model Adoption in Smart Customer Service

Wei Jiaxing has been in the smart customer service market for seven years. He has observed that since last year, large model technology has evolved rapidly, and more and more business opportunities and demands in the smart customer service field are related to large models. "Just like the intelligent driving feature in cars, customers may or may not use it in the end, but they won't consider it without it," he said.

The booming market demand can also be glimpsed from the bidding data for large models from central and state-owned enterprises. Digital Intelligence Frontline previously counted that in Q3 of this year, there were 9 projects related to smart customer service among the biddings for large model-related projects, out of a total of over 360 projects. Although this does not account for a high percentage of the total, the demand for large model smart customer service has emerged and formed a certain scale.

A relevant person from a central enterprise related to national economy and people's livelihood previously told Digital Intelligence Frontline that they are looking for cooperation suppliers for the large model smart customer service in a super app under their brand, with an expected project scale of tens of millions of yuan. They have surveyed the smart customer service product capabilities of multiple vendors on the market, including Alibaba and iFLYTEK, and are currently in the product selection stage.

The market space is accelerating, as can be seen from IDC's report data in August this year. The overall market size of smart customer service solutions reached 3.08 billion yuan in 2023, an increase of nearly 36.9% from 2022.

Industry observers believe that the booming demand is firstly related to the significant upgrades to smart customer service capabilities brought about by the rapid progress of large model technology over the past two years.

"For smart customer service driven by large models, enterprises do not need a large number of small NLP models. Large models can solve key NLP tasks, and there are significant leaps in the robot's understanding and language generation capabilities," Zhang Hongguang, General Manager of Baidu Intelligent Cloud's Smart Customer Service and Smart Content Product Department, said at a media briefing in September this year.

Currently, large models improve customer service experiences in scenarios beyond just direct communication with users, such as voice and text robots. For example, in smart quality inspection and analysis scenarios, large models can be used to confirm whether the service provided by human or robot customer service meets the SOP defined by the enterprise. They can also tag conversations structurally to improve customer service.

Cheng Yin told Digital Intelligence Frontline that in addition to traditional smart customer service scenarios such as text/voice robots, smart quality inspection, agent assistance, digital human customer service, video customer service, smart analysis, and agent training, there is also increasing demand for large model smart customer service in other areas closely related to customer service, such as marketing, training, operations, and decision-making, where user interaction is required.

Wei Jiaxing observed that technological advancements are enabling voice robots to unlock scenarios that were previously impossible, such as CloudBat's exploration of using large model-based agents for telephone interview services. This new scenario requires a high level of understanding and natural language expression from the robot, and the service quotation is also exponentially higher compared to traditional scenarios.

The rapid integration of smart customer service and large models is also closely related to the rapid decline in model invocation costs.

At the beginning of last year, the cost of a single invocation of the GPT3.5 interface was still a few cents. At that time, Wei Jiaxing judged that only when the cost of a single invocation dropped to around 1 cent could the industry's cost accounting work out. Now, almost 18 months later, the cost of a single invocation of a voice robot's large model interface has long fallen below that figure.

This is not just happening overseas; domestic model vendors also initiated intense price reductions in the first half of this year. Alibaba CEO Wu Yongming mentioned in September this year that the cost of model inference has decreased exponentially, far exceeding Moore's Law. Kai-Fu Lee, CEO of Sinovation Ventures, mentioned an aggressive figure at an event a few days ago: over the past year and a half, the price of large model inference costs has differed by 500 times, while model capabilities have also significantly improved.

Other large model vendors have also taken various price reduction measures. The apparent downward trend in inference costs has accelerated the adoption of large models by smart customer service industry service providers this year.

02

Measurable ROI vs. Overcoming Obstacles

Since last year, leading enterprises have regarded the smart customer service field as a pioneering scenario for the application of large models.

At last year's Yunqi Conference, Alibaba Cloud launched eight industry-specific large models based on Tongyi, including Tongyi Xiaomi, an intelligent customer service robot that can customize customer service chatbots based on enterprise needs. NetEase Wisdom released a business and large model for customer service in September last year, used in smart quality inspection, customer service robots, smart outbound calls, precise insights, and other customer service scenarios.

This year, starting from March, Baidu Intelligent Cloud has continuously upgraded its three benchmark applications, including Keyue, an intelligent customer service product reconstructed and upgraded based on large models.

Li Qiang, Vice President of Tencent Group and President of Tencent's Enterprise Services Business Group, also mentioned in the media briefing session at this year's Tencent Digital Ecology Conference that compared to complex scenarios such as production and research and development, large models will progress faster in scenarios with relatively higher fault tolerance, such as smart customer service.

There are many enterprises exploring and applying large models in the customer service field. Technology service providers are entering the market in droves, partly due to the significant cost reduction and efficiency enhancement brought by large models in this scenario.

IDC's Cheng Yin explained that through many in-depth interviews, they found that the introduction of smart customer service has significantly improved customer service efficiency and quality, saved enterprises a lot of manpower and time costs, and brought measurable benefits.

Shen Dou, Executive Vice President of Baidu Group and President of Baidu Intelligent Cloud, mentioned at the Cloud Intelligence Conference that Keyue has significantly improved the self-service resolution rate in customer service. This is an important indicator in the field of customer service services. "The industry's self-service resolution rate is generally around 80%. After the large model reconstruction and upgrade, Keyue's self-service resolution rate exceeded 92%, making customer service smarter, more anthropomorphic, and more understanding of users," he said.

KD100, a courier and logistics information service provider, applied large model technology to customer service scenarios this year. CEO Chen Dengkun told Digital Intelligence Frontline, "The synergistic effect between the large model and customer feedback and system rules has been significantly enhanced, jointly resolving up to 90% of work order issues. Only 10% of issues require manual intervention, and the one-time resolution rate for customer complaints is as high as 99.4%."

A representative from Ronglian Cloud's large model product team mentioned that in insurance service scenarios, they have achieved insights and mining of a large amount of recorded content from 400 telemarketing calls based on large models. After application, the number of potential customer complaints has been reduced by about 10%.

Telemarketing calls used to only be analyzed based on the number and duration of calls, but in reality, the content of the calls contains many potential needs of users. With large models, potential user needs can be extracted, and high-quality responses from experienced service personnel can be mined from call content to provide suggestions and references for subsequent services.

These are all advancements in improving the smart customer service experience through large models. However, on the flip side, there are still many challenges to be solved.

IDC told Digital Intelligence Frontline that since 2023, leading technology suppliers have been productizing their offerings, but commercialization progress has not been significant, and the market is still dominated by traditional AI customer service. The universal application of large model-enabled customer service products in the industry still faces challenges such as coupling with the original system, limited understanding ability, lack of context awareness, insufficient interactivity and flexibility, and insufficient personalized services. These require continuous training and optimization based on specific scenarios, as well as investment in data and computing resources.

A senior executive in the enterprise services industry lamented that while marveling at technological breakthroughs, the industry also needs to solve many problems, such as hallucination control in real communication scenarios and the costs of integrating new technologies into enterprise business processes and management.

Taking voice-driven agents powered by large models in voice scenarios as an example, the current level of anthropomorphism in voice is insufficient, and customers can easily tell that the real-time generated voice is not human. At the same time, due to the high requirements for low latency in telephone scenarios, the problem of hallucinations is also difficult to solve.

Some large-scale projects also involve the costs of subsequent construction and operation and maintenance. Regarding the tens of millions of yuan smart customer service project mentioned earlier at a central enterprise, a relevant person mentioned that they were very cautious in product selection due to the high costs of subsequent construction and operation and maintenance of the project. Therefore, although they are familiar with the capabilities and characteristics of various vendors, they have not yet finalized a cooperation partner.

In addition, a senior executive in the smart customer service industry mentioned that in addition to various technical limitations, she has also seen scenarios that do not encourage completely "dehumanized" customer service. "Now AI may have improved its ability to solve language problems and may also be able to solve math problems. However, many customer service scenarios involve customer complaints, which may involve legal, psychological, and social issues. In many cases, users still prefer to receive calls from real people," she said.

03

Gradual Approach: Prioritizing Usable Scenarios

Upgrading products based on large models has a significant characteristic: a leap in basic model capabilities leads to improvements in product performance and experience. However, before more capable models arrive, technology service providers need to find an appropriate pace for productization and commercialization based on current technological limitations.

Wei Jiaxing mentioned that their strategy is to prioritize business scenarios with lower requirements for indicators such as latency and anthropomorphism before ideal solutions to these issues are found. "For example, for 400 phone calls, you can accept issues with latency and the lack of anthropomorphism, where it sounds like a robot. These are the scenarios to use now," he said.

A software service provider in the automotive parts industry uses agents to serve special model repair and spare parts quotation requests from car manufacturers and end-users. Currently, 80% of customer service inquiries are handled by agents.

To address uncontrollable factors such as agent hallucinations, they have also adopted a service model combining agents and human supervision to better serve customers. "The quotation agent communicates with customers in the group, but there are still salespeople in the group performing some supervisory actions," the service provider said.

Kong Miao, VP of Ronglian Cloud's Industrial Digital Cloud, mentioned that for issues such as computing costs and hallucinations brought about by the full replacement of large models, large models can be used to solve local problems first.

"For example, using it to solve front-end generalization problems and then converting them into specific tasks, which are then solved using efficient small models. Adopt a strategy that combines large and small models," Kong Miao said at an industry forum a few months ago.

As IDC said, the development of large model-enabled smart customer service is still in its initial stages, and commercialization still needs to be promoted. However, some enterprises have also observed current market preferences and are trying to find commercialization strategies.

Wei Jiaxing found that although many enterprises are aware of the functions of large models, compared to native agent products of large models, they prefer the previous generation of explainable products. "In the short term, large models are a sales feature," he said.

The exploratory value of native agent-type products of large models lies in the fact that, on the one hand, they represent the future direction of development. On the other hand, they can decouple some capabilities for upgrading previous-generation products, thereby enhancing user experience and retention and ultimately achieving indirect commercial conversion.

Taking the telephone interview scenario as an example, it may involve the collaborative work of N agents. The first AI reads unstructured resumes, turning them into standardized small fields for subsequent AI reading. The second AI retrieves the highest-level GPT-level model to teach the calling AI to train its cache system. The third AI is responsible for making calls and initiating them. The fourth AI extracts standardized conclusions from call records.

In this process, the four AI capabilities can be decoupled, and part of the technical solutions can empower previous-generation products. "Currently, much of the commercialization revenue comes from the revenue growth brought about by the reuse of technical solutions in previous-generation products," Wei Jiaxing said.

Currently, whether it is traditional AI customer service companies integrating large model capabilities into scenarios or large model companies entering the smart customer service arena with their tools, the popularity of this field is increasing.

"Platform-level companies represented by cloud services have research and development advantages in underlying large models and new-generation smart customer service products. Moreover, some cloud vendors have businesses closely related to customer service. They have a certain brand reputation and user base, and what they need to strengthen is customized services for small and medium-sized enterprises. Traditional customer service companies that entered the market earlier also have market awareness and have accumulated a certain number of customers. What these companies need to strengthen is the underlying research and development capabilities of large model technologies," IDC's Cheng Yin said.

An entrepreneur in the AI customer service field believes that they differ from giants in pricing power and service capabilities. Small companies have flexibility, while large companies, due to labor costs, find it difficult to serve projects with quotations below 500,000 yuan after involving delivery teams.

Regardless, competition is intensifying. However, the market is not a zero-sum game.

Earlier this month, OpenAI's GPT-4o Realtime API public beta version was launched. Wei Jiaxing immediately promoted the development of a new generation of smart customer service products based on this interface within the company.

For voice-based smart customer service, this is an important upgrade in model capabilities that can greatly address the latency issues of voice robots. Previous models combined speech-to-text and text-to-speech functions, while the Realtime API is an end-to-end model that performs more naturally in voice interactions.

This is also a major benefit of the intelligent customer service product empowered by large models. In the long run, technological advances and cost reductions will bring about qualitative breakthroughs in the product, and new market spaces will also be further opened up.