AI telephoto lens becomes the major direction for mobile phone imaging upgrades, will computational photography "kill" optical imaging?

10/31 2024 460

Have you guys noticed that the new phones released in October seem to have reached a consensus?

Whether it's the standard version or the Pro version, they have almost all started to prioritize "telephoto upgrades."

Take Xiaomi 15 series released last night, the Pro version directly adopts the 50MP, 5x optical zoom IMX858 periscope telephoto lens that was previously exclusive to the Ultra models.

Going back a bit, the vivo X200 series, the Pro version uses the 200MP, 3.7x optical zoom HP9 periscope telephoto lens that was previously exclusive to the Ultra models.

As for the OPPO Find X8 Pro, it is equipped with a 3x optical zoom periscope telephoto lens and a 6x optical zoom periscope telephoto lens, allowing consumers to enjoy the pleasure of dual periscope telephoto lenses beyond their expectations.

(Image source: vivo)

What about those models that prioritize slim and compact bodies?

Surprisingly, even the Xiaomi 15 and OPPO Find X8 this time around are equipped with fully functional upright telephoto lenses, and the vivo X200 Pro mini even uses a 50MP, 3x optical zoom LYT600 periscope telephoto lens.

Interestingly, even so, these flagship new models don't seem to have much more advantage in telephoto lenses.

Looking back at 2024, realme took an unconventional approach, bringing the OV64B large-sensor periscope down to the 2000 yuan price range; the once-proclaimed largest-sensor periscope telephoto lens, the IMX890, is now easily accessible to cost-effective sub-brands.

When even the once-rare periscope telephoto lens can be found in standard versions and even mid-range phones this year, you can see just how competitive the domestic mobile phone market is.

(Image source: realme)

At this point, some of you handsome folks might be wondering.

In this situation, how will flagship new models emphasize their advantages?

Don't ask, the answer is "AI."

That's right, not only for imaging super-sized models, but it can be said that this year's end-of-year batch of flagship phones will all use AI large models for super-resolution algorithms to enhance resolution at high zoom levels.

As we stand at the forefront of this trend, we may be witnessing the arrival of the era of AI imaging.

AI fully unleashes the potential of telephoto lenses

In my memory, the first to integrate AI large models into imaging was...

Probably Xiaomi.

At the beginning of this year, at the Xiaomi 14 Ultra launch event, Lu Weibing spent almost half the time emphasizing the imaging capabilities of this super-sized flagship, featuring a full-frame Leica optical large aperture quad-camera and the world's first "AI Large Model Computational Photography Platform" AISP.

(Image source: Xiaomi)

According to Lu Weibing at the time, AISP can not only optimize the algorithm to make the image more aesthetically layered in terms of brightness, depth of field, color, and more; more importantly, it can directly use AIGC technology to redraw the image in 30x or higher optical zoom scenarios, making distant objects instantly clear from fuzzy pixels.

This is what is known as AI telephoto.

The specific principle is actually quite straightforward. I'm sure you've all heard of generative AI, which is the ability to generate novel text, images, music, and software by analyzing large amounts of digital material and then follow up with keywords/prompts to create new content.

By applying generative AI, AI can supplement missing details in the originally relatively blurry telephoto photos without changing the overall feel of the photo, making the photo clearer and more natural while maintaining authenticity, theoretically allowing users to better appreciate the enlarged details.

As for the specific effect...

Let me put it this way, my Xiaomi 14 only has a 3x optical zoom lens.

Randomly taking a photo of the park from a high place, you can see that the red flag in the distance and the park name further away are basically unrecognizable.

(Image source: Leitech)

And then, without changing the phone, simply zooming in to 30x and automatically activating the large model telephoto algorithm, this is the result.

(Image source: Leitech)

Since the quality of the telephoto lens inside the original body is not high, it is difficult to provide clear images for the large model. Even so, we can still see the details of the leaves, windows, and red flags restored by the large model, as well as the park name further away, which is also clearly visible.

Although there is a slight smear effect, it is still much better than the previously blurry effect.

Perhaps for this reason, other manufacturers have also followed suit. In the X100S and X200 series, vivo uses AI large models to learn from over a billion object data points, combined with AI multi-frame high-resolution algorithms, to achieve high-definition long-distance photography; the OPPO Find X8 series even directly advertises the AI Long Range Telephoto, which utilizes a new AI computation method at 20x-120x zoom to significantly improve base resolution and detail.

Through AI upgrades to imaging, mobile phone photos have indeed become clearer.

Mobile phone imaging improvements are "doing the best with limited resources"

As one of the core selling points of flagship phones, imaging hardware configuration has always been the focus of competition among manufacturers, which I think few readers would dispute.

Over the past decade, the competition in mobile phone imaging hardware configuration can be roughly divided into three parts.

The first part is the competition in sensors, constantly pursuing larger main camera sensors until a one-inch sensor became the standard configuration for high-end imaging phones.

The second part is the competition in the number of lenses, from the traditional basic rear single camera to the extreme pursuit of multi-camera combinations adapted to various scenes, until the basic combination of "main camera-telephoto-ultra-wide-angle" was finalized in recent years.

The third part is the competition in lens focal lengths. After finalizing the basic combination, manufacturers naturally turn their attention to expanding the focal length range of the lenses, from ultra-wide-angle to telephoto, achieving 5x or even higher zoom periscope telephoto photography, while also adding 2x and 3x mid-range lenses to achieve optical zoom relay.

(Image source: OPPO)

Of course, I understand why manufacturers relentlessly pursue imaging hardware configuration:

Hardware configuration is the foundation for taking a good photo. Only stronger hardware can deliver better image quality, higher light intake, and richer details.

The problem lies in the physical form of the mobile phone itself.

Users' demands for portability, hand feel, and aesthetics are becoming increasingly demanding, while manufacturers need to find ways to cram larger batteries and more components into limited space to achieve richer functions (such as satellite communication). Users' constant complaints about "packing" are actually highly technical feats, akin to "doing the best with limited resources".

This also makes it increasingly difficult to improve mobile phone imaging.

Limited by the internal space of the phone, the telephoto lens is doomed not to be mindlessly stacked, as there is an "upper limit" to the thickness of the camera module that does not affect the grip feel.

The Zeiss 200MP APO telephoto lens on the vivo X200 Ultra is basically the upper limit that a phone can accommodate.

However, telephoto lenses inherently have a need for telescopic vision and compressed spatial composition, and 3.7x optical zoom is obviously insufficient to satisfy photography enthusiasts.

That's why we can see that in 2024, various manufacturers have equipped telephoto lenses with a new set of AI telephoto algorithms, which allows phones to avoid excessive competition in hardware and instead unleash the power of existing hardware to a greater extent.

Partnering with AI is undoubtedly the mainstream direction for mobile phone computational photography today and in the future.

The key to good AI telephoto is reducing the AI feel

Above, we've talked a lot about the advantages of AI telephoto.

At this point, I think attentive readers must have noticed, "Couldn't it be that this so-called AI telephoto doesn't have a single flaw?"

Haha, of course not.

We novices may not be too picky, but there are many seasoned photographers on platforms like Xiaohongshu who have come out in opposition to what manufacturers are doing quite vocally—

"The details calculated by AI are not real, can it still be considered photography?"

Well... that's not entirely wrong, I suppose.

Readers who have read our review of the vivo X200 series should know that the advantages and disadvantages of this AI telephoto algorithm are quite obvious.

When the zoom level is too high, the algorithm can add clear texture lines and noise reduction to the originally degraded image.

It looks pleasing to the eye, but as can be clearly seen from the image below, when Xiao Lei zooms in to 20x, the textures on the lighthouse are all reconstructed by AI post-processing, which does give an indescribable weird feeling.

(Image source: Leitech)

In addition, just as AI painting can sometimes mess up text and hand details, current AI telephoto lenses can easily go haywire when processing details of distant faces and various shooting scenarios with text, even distorting the text that should be straight and clear after processing by the super-resolution algorithm, making it difficult to distinguish.

In some cases, it even directly displays gibberish.

(Image source: Leitech)

It can only be said that it's unrealistic to expect AI to fill in details that can't be captured by the lens.

However, I believe that with the advancement of large models and the improvement of mobile phone ISP computing power, AI algorithms will become more mature, which is the direction that mobile phones with limited internal space should develop towards.

With experience accumulation, mobile phones will become more sensitive to whether algorithms are involved and should be able to minimize situations where users feel the uncanny valley effect.

As for those voices opposed to AI telephoto...

I can only say that with the development of large models, there will only be more and more AI functions used in mobile phones in the future. It is recommended to keep up with the trend of the times.

Source: Leitech

Solemnly declare: the copyright of this article belongs to the original author. The reprinted article is only for the purpose of spreading more information. If the author's information is marked incorrectly, please contact us immediately to modify or delete it. Thank you.