AI Video? AI Filters!

Since OpenAI introduced Sora, the pace of development in the video generation field has significantly accelerated. Many companies, both domestic and international, have begun investing in research and launching large models specifically for video generation. At the same time, they have integrated their technologies into user-friendly AIGC products, making these technologies accessible to a broader audience.

Compared to Sora, which is still in a standby state, the entry of new players such as Runway Gen-3 Alpha and Kuaishou Keling has intensified the competition in the video generation field. While most players are focused on improving productivity, a small group has taken a different approach, choosing to enter the video generation race from the pan-entertainment direction.

Tencent Zhiying Launches AI Feature, Turning Ordinary Videos into Anime Instantly

Recently, Tencent's intelligent creation platform, "Zhiying" mini-program, announced the official launch of an innovative feature called "AI Video Stylization Service." With just one click, users can generate stylized videos, instantly transforming ordinary videos into creative works of different styles.

According to Tencent Zhiying, video stylization technology leverages AI algorithms to conduct in-depth analysis and re-creation of the original video. Its aim is to enhance the interest and appeal of content, thereby promoting its widespread dissemination and sharing.

Currently, the platform offers a Japanese anime style template. Users simply need to upload their video materials to the Zhiying mini-program, select their preferred anime style template, and click "Generate with One Click" to obtain a uniquely styled video work. It supports processing video content up to 10 seconds in length.

Since this feature is temporarily available for free, it has been warmly received by users since its launch. When I opened the mini-program, I saw many high-quality popular works on the feature page. From soccer girls sweating on the green field to dancers hitting the beat in the practice room, after AI stylization, they all became anime characters, giving people a sense of being immersed in the world of manga.

Image Source: Zhiying

In addition to the Japanese anime style, Tencent Zhiying plans to introduce more style templates, including but not limited to retro, sci-fi, and cinematic looks, to meet the creative needs of different user groups and achieve diversification and personalization of video content.

Recently, I have tried out many AI-generated products, some of which have been delightful, while others have been disappointing. Let me take you on a journey to explore the level of Zhiying's AI video stylization.

Surprising Character Processing Effects, but Lackluster in Other Scenes

From the moment I generated my first video, I could feel the popularity of Zhiying's AI video feature. Just how popular is it? Including material upload, AI generation, and queue time, it takes an average of two hours to generate a stylized video.

At first, I thought it was due to an upload error, but when several colleagues encountered the same situation, I realized it was not that simple. I specifically consulted the platform's customer service and was told that due to heavy usage, please be patient.

Image Source: Zhiying

While it is understandable that users are enthusiastic about a new feature, I still hope that Zhiying can optimize the computing power distribution of its AI video function. After all, long waiting times can significantly impact the user experience.

Returning to the video, after a long wait, the first work was finally ready. The original video showed a couple standing by the window watching the sunset. After AI stylization, the overall video style changed dramatically, but the camera movements, character movements, and clothing remained essentially the same. The AI strictly adhered to its principle of "changing only the style."

As can be seen, not only the girl whose face is visible but also the boy facing away from the camera has been transformed into a Japanese anime style. In terms of the environment, the halo created by light rays has also been processed, making the overall video style more unified and natural. Overall, this is a video worth sharing on social platforms.

Of course, it is not yet perfect. Upon closer inspection, we can still find many small flaws. The first problem is hair processing, which is a common mistake made by many AIGC video applications. The girl's hair in the video initially appeared normal, but when she turned her head, the hair on her face underwent a virtual teleportation and eventually disappeared. The second problem is color tone, which is particularly evident in the boy's clothing. The white shirt became a patchwork, and his right hand, which was in the shade, was magically transformed into black, which was quite incongruous.

Subsequently, I tested several different types of videos. According to my observations, Zhiying can stylize content such as landscapes, animals, and characters, but the best processing effects are achieved with character videos. In contrast, the stylization of landscapes and animals feels somewhat lacking.

For example, in this video, the original shows an explorer walking alone on a forest path. After AI stylization, it gives the impression of a game scene. The overall atmosphere is there, but the character's feet and body are distorted to varying degrees.

Image Source: Zhiying

As we all know, making generated videos conform to physical laws has always been a headache for large AIGC video models. Compared to generating images, generating videos requires more complex considerations, not only involving the movement trajectories and limb coordination of different subjects but also integrating real-world physical characteristics such as gravity and lighting for comprehensive processing.

Although Zhiying's AI video stylization function is essentially a secondary creation, it also faces the same challenges. Judging from my experience, Zhiying's AI video obviously needs to put more effort into conforming to physical laws.

Overall, as a feature that focuses on social sharing, AI video stylization is entertaining enough, and the quality of generated videos is above average for AI filters. However, compared to mainstream large video generation models in terms of AIGC, there is still a gap, mainly due to instability. While portrait video processing quality is quite good, scene processing still needs improvement.

Leaving aside the issue of generation efficiency, Zhiying's new AI feature is commendable for its free trial, but considering that the official is likely to upgrade it to a paid feature in the future, I have doubts about its dissemination effect. A feature that is not yet perfect in its current state may be a fatal blow to user enthusiasm once usage costs are attached.

How Far Can Video AIGC Go in the Pan-Entertainment Race?

Launched in March 2023, Tencent Zhiying is an intelligent creation tool integrated with AI creation capabilities, offering functions such as digital humans, text-to-speech, AI painting, AI image expansion, and online video editing.

When I saw Zhiying's AI video function, I immediately thought of CapCut and Remini. They also entered the AI field from creative tools and gained popularity by launching AI features with social fission properties.

Image Source: Jimeng

In my opinion, Zhiying's AI video stylization function has also taken the same path. Rather than being an AI video, it is more like an AI filter. Compared to traditional video AIGC, AI filter-style videos are obviously more in line with the social sharing habits of ordinary users. Once they go viral on social media, their effect may be more effective than other large video generation model promotions.

Everything has two sides. While focusing on fun social sharing methods may bring Zhiying unexpected promotional effects, the heat of the pan-entertainment race comes and goes quickly. Predecessors such as Remini, Lianmeng, ZAO, and Miaoya Camera have already taught us this lesson: short-cycle products that rely on a single function to gain popularity are difficult to retain users.

The novelty brought by product functions will eventually fade, especially for functions with low technical barriers. When I searched for AI video stylization, I found many apps with similar selling points, indicating that the threshold for this technology is not high.

Secondly, Zhiying's AI video stylization feature is currently available for free for a limited time. When other software introduces free or low-cost similar functions, even if Zhiying can create a personalized effect, it will still be difficult to differentiate itself from other apps. Users are attracted to the AI video stylization function, and during the free trial period, they can essentially try out all the videos. How many users are actually interested in long-term use? Even if the official launches other style templates in the future, it will be difficult to convince users to pay for new experiences.

Image Source: Zhiying

For generative AI products targeting the consumer market, how to retain users and achieve commercialization based on short-term popularity is the real challenge that needs to be addressed. Perhaps it's through a succession of new features, or through marketing on social platforms... By referring to cases such as Remini and Miaoya Camera, it is not easy for such applications to maintain their popularity. Without bringing new stimuli to users, they are likely to be just a flash in the pan of AI history.

No one knows how far video AIGC can go in the pan-entertainment race. What Tencent Zhiying may need to consider is the integration of AI with existing creative functions to create a true AI creation tool. Only in this way can it solidify its product positioning while pursuing more hot opportunities.

The first half of 2024 has been tumultuous in the tech industry.

Large models are accelerating their deployment, with AI phones, AI PCs, AI home appliances, AI search, AI e-commerce, and other AI applications emerging in an endless stream;

Vision Pro went on sale and entered the Chinese market, reigniting the wave of XR spatial computing;

HarmonyOS NEXT was officially released, changing the mobile OS ecosystem;

The automotive industry has fully entered its "second half," with intelligence becoming a top priority;

E-commerce competition has become increasingly fierce, with price wars escalating into service wars;

The wave of going global has surged, with Chinese brands embarking on a journey of globalization;

...

As July heats up, Leitech's Mid-Year Review Special is online, summarizing the brands, technologies, and products worth noting in the first half of 2024 in the tech industry, recording the past, and looking forward to the future. Please stay tuned.

Source: Leitech