The battle for mobile "super resolution".

Super Resolution (hereinafter referred to as "Super Res"), may not be unfamiliar to many avid gamers who pursue image quality. After years of development, Super Res technology has gained widespread support and recognition on PCs and has become a regular feature at mobile phone launch events.

Now, Arm is also getting into the "Super Res" game.

Recently, Arm officially announced the launch of its Arm ASR (Accuracy Super Resolution) technology in a community post, which enhances game graphics resolution and frame rates while reducing mobile phone power consumption.

As we all know, modern video games are constantly pushing the boundaries of realism, and gamers' pursuit of image quality, frame rates, lighting, and other aspects is never-ending. However, frame rates and image quality are like fish and bear's paws – it's difficult for GPUs to have both under heavy loads.

In practical terms, we find that many mobile phones encounter overheating, frame drops, and increased power consumption when running demanding games at high graphics settings, significantly impacting the user experience.

In this dilemma, AI computing has emerged as an important breakthrough.

"DLSS is a revolutionary breakthrough in AI-driven graphics," Image/NVIDIA

In 2018, NVIDIA pioneered the DLSS technology, which uses neural network operations in the Tensor Cores of GPUs to enhance game performance, sparking a graphics processing revolution. AMD and Intel have since released similar FSR and XeSS Super Res technologies.

In the mobile space, Arm is not the first chipmaker to introduce Super Res technology. In 2022, MediaTek launched its Game AI-SR, and Apple introduced MetaFX Super Res at WWDC the same year. Qualcomm officially launched its Snapdragon Game Super Resolution (GSR) technology last year.

With the introduction of Arm ASR, mobile Super Res technology is entering a new stage of development. Meanwhile, the once-prevalent external display chips are facing an awkward situation where they may no longer be needed.

ASR Officially Tested to Outperform AMD and Qualcomm?

According to Arm, its ASR technology is based on AMD FSR 2 and is also open-source under the MIT license.

Official tests show that Arm ASR delivers higher frame rate improvements on mobile platforms than AMD FSR 2 and better image rendering quality than Qualcomm GSR.

On devices equipped with Arm Immortalis-G720 GPU and a 2800 x 1260 display, ASR technology can increase frame rates by 38%-53%. In comparison, AMD FSR 2 can only increase frame rates by 36% at 2x upscaling.

Frame Rate Performance, Image/Arm

Additionally, official tests show that Qualcomm GSR can increase frame rates by 47% at 1.3x upscaling.

It's worth noting that compared to Qualcomm GSR's spatial-based processing, ASR's temporal-based approach guarantees better image quality. Arm states that ASR excels in handling dynamic scenes, reducing tearing and latency issues compared to spatial-based Super Res solutions.

What exactly do spatial and temporal mean? To explain the difference, AMD FSR serves as the best example:

As AMD's first-generation Super Res solution, FSR 1 is spatial-based, improving resolution by enhancing spatial details in a single frame. It leverages existing pixel information within a single frame to interpolate or reconstruct a higher-resolution image.

As the second generation, FSR 2 shifts to a temporal-based approach, utilizing information from multiple frames to enhance resolution. It not only leverages spatial information but also temporal information, i.e., motion and changes between adjacent frames, for image reconstruction.

Compared to AMD FSR 1, AMD FSR 2 significantly improves image quality, often indistinguishable from native rendering by the naked eye.

Theoretically, spatial-based Super Res techniques often cause blurring and ghosting in fast-moving scenes. However, temporal-based ASR performs better in dynamic scenes, effectively reducing blurring and ghosting issues by incorporating temporal frame information, providing clearer and smoother visuals.

When handling high-contrast scenes, temporal-based Super Res solutions also better preserve details and layering, enhancing overall visual effects.

Furthermore, ASR's power consumption performance is promising. Arm states that compared to native 1080p gaming, ASR significantly reduces power consumption during the process of upscaling from 540p to 1080p on Dimensity 9300-powered devices.

Power Consumption Performance, Image/Arm

For users who engage in long gaming sessions and demand high performance, power consumption is undoubtedly a significant advantage. Reduced power consumption not only extends device usage time but also mitigates overheating issues, enhancing the overall user experience.

ASR's power consumption advantage stems from its efficient algorithms and optimized processing workflows.

According to Arm, through intelligent scheduling and resource management, ASR minimizes processor and GPU loads while ensuring image quality, reducing power consumption and avoiding overheating caused by high loads, thereby enhancing game stability and device longevity.

Evolution of Mobile Super Res: From Pixelworks to ASR

Today, NVIDIA DLSS and AMD FSR have strongly demonstrated the revolutionary impact of Super Res technology on game graphics rendering. Even the most advanced graphics cards can offer better experiences through Super Res.

Digital Foundry previously tested RTX 4090 running Marvel's Spider-Man at 4K resolution with DLSS 3 enabled, finding performance could reach 203.6% of what it was without DLSS, while maintaining image degradation below human perception thresholds.

In a nutshell, the core of Super Res technology lies in using complex algorithms and powerful computing capabilities to upscale low-resolution images to near-high-resolution levels, enhancing user experience without significantly increasing hardware burdens.

From this perspective, mobile phones are even more in need of Super Res technology, and indeed, chipmakers have been attempting to bring it to mobile devices in recent years.

1. Pixelworks: The "External" GPU Preferred by Mobile Phone Manufacturers.

Image/realme

Once upon a time, from Honor, OPPO, vivo, Xiaomi to ROG, many gaming-focused models adopted external Pixelworks display chips to achieve Super Res gaming experiences. OPPO and vivo were once customers that contributed over 10% of Pixelworks' revenue.

The Pixelworks X7 "standalone" GPU has appeared in many manufacturers' product launches as a major selling point. Many people still remember it vividly, even in the latest OnePlus Ace 3 Pro, which features the latest Pixelworks X7 Gen 2.

According to official introductions, the Pixelworks X7 Gen 2 integrates ultra-low latency MotionEngine technology and efficient AI game Super Res technology, allowing mobile phones to render as low as 1/4 of game frames and image resolution while delivering up to 120 fps and 1.5K resolution immersive visual experiences on the display end.

2. MediaTek: The First Major Chipmaker to Apply Super Res Technology to Core Chips.

MediaTek was an early adopter of applying Super Res technology to mobile gaming scenarios.

Image/MediaTek

With Dimensity 9000, MediaTek introduced Game AI-SR technology, which leverages the APU (AI Processing Unit) and AI models to enhance game image resolution through complex calculations.

According to MediaTek, Game AI-SR optimizes image processing workflows through intelligent analysis of game scenes, maintaining good image quality under high loads. It also excels in reducing power consumption and enhancing processing efficiency.

Unfortunately, MediaTek has not disclosed many technical details about Game AI-SR, and we haven't seen any games supporting it yet.

3. Qualcomm: Focusing on Building a Super Res Ecosystem.

Zhulian Mobile Game with Qualcomm GSR enabled, Image/Qualcomm

Early last year, Qualcomm announced the Snapdragon GSR technology, an optimized single-pass spatial upscaling technology for Snapdragon Adreno GPUs.

Qualcomm's technology quickly gained support from popular games, including Activision's Call of Duty: Warzone, Tencent's Return of the Continent, NetEase's N.E.O., Naraka: Bladepoint Mobile, and Perfect World's Zhulian Mobile, all of which will support Snapdragon GSR.

However, as mentioned earlier, while Snapdragon GSR can be expected to improve frame rates, there's still room for optimization in ensuring image quality. In fact, Qualcomm's press releases emphasize Snapdragon GSR's advantages in latency and power consumption more than image quality.

4. Apple: MetalFX's Comprehensive Strength in Bridging Hardware and Software Ecosystems.

Digital Foundry's Test of MetalFX, Image/YouTube

Like Arm ASR, Apple's MetalFX is a Super Res technology based on AMD FSR 2. As part of Apple's Metal graphics framework, MetalFX aims to provide developers with more efficient image processing capabilities.

Through MetalFX, Apple devices can achieve high-resolution image output while maintaining low power consumption. Unlike MediaTek and Qualcomm, Apple truly bridges hardware and software, applying it not only to Macs but also to iPads and iPhones, represented by games like Resident Evil Village and Death Stranding.

In fact, without MetalFX, it would be almost impossible to run Resident Evil Village and Death Stranding smoothly on Apple devices, especially the iPhone 15 Pro.

As for Arm ASR, it's the latest player in mobile Super Res technology.

Super Res Technology Trends Towards Native Support, Signaling the Decline of Mobile "Standalone GPUs"

If the goal of Super Res technology on PCs is to achieve higher game frame rates, the core of mobile Super Res technology may be to reduce chip power consumption while ensuring near-native image quality. External chips inherently have power consumption disadvantages compared to integrated chips.

This may explain why OnePlus, which equips its Ace 3 Pro with an "e-sports-grade gaming standalone GPU chip," has remained silent about Pixelworks.

More importantly, external third-party display chips are becoming increasingly unnecessary.

This Generation of Flagship SoCs, Image/Geekbench

On one hand, mobile GPUs have made significant strides in gaming performance. From Qualcomm's Adreno 750 GPU on Snapdragon 8 Gen 3 to MediaTek's Immortalis-G720 GPU (Arm public version) on Dimensity 9300, and even Apple's A17 Pro, all have significantly upgraded mobile gaming experiences.

Take Genshin Impact, once known as a "mobile GPU killer," as an example. Nowadays, various flagship SoCs can run it at near-full frame rates with native maximum graphics settings.

On the other hand, with major chipmakers like Qualcomm, MediaTek, and Arm natively supporting Super Res technology, optimizing the synergy between hardware and software can theoretically achieve even better Super Res effects, simplifying developers' work and enhancing overall gaming experiences.

According to rumors, Snapdragon 8 Gen 4, to be released in October this year, will feature GSR Super Res technology, supporting native game graphics Super Res and frame interpolation, even compatible with Snapdragon 8 Gen 3 and Snapdragon 8s Gen 3. As @DigitalChatStation tweeted:

"I'm wondering what flagships with dual-chip platforms will do. If games enable standalone GPU frame interpolation, the latency control won't be as good as Qualcomm's native solution, which is a bit awkward."

Image/Weibo

Compared to external chips, mainstream chipmakers directly leverage the synergy between GPUs and NPU/APUs within their chips, better utilizing computational resources and energy efficiency optimizations to provide more stable and efficient image processing capabilities. Whether in dynamic scenes or under high loads, natively supported Super Res technology can deliver consistent performance, avoiding compatibility and stability issues that external chips may bring.

In contrast, the limitations of external display chips like Pixelworks are becoming increasingly apparent, likely to be gradually abandoned by mobile phone manufacturers and even fade into history.

But for mobile Super Res, this is just the beginning.

Source: Leitech