Adobe and NPU vs. GPU

#95

Heterogeneous computing, which involves using different types of processors and specialized hardware to maximize performance and efficiency, is indeed being adopted, though the pace varies across industries.

Adoption of NPUs (Neural Processing Units) has been slower in some sectors, primarily due to the limited marketplace and the current focus on mobile devices. Companies are not always transparent about where and how these technologies are used, which can contribute to the perception of slow adoption.

Powering AI features on battery is a significant challenge. Many AI features, especially those requiring intensive computation, can drain battery life quickly. However, advancements in energy-efficient hardware, like the NPU we now have available in most CPUs, are driving the adoption of AI features that are optimized for battery life. Several CPU manufacturers are actively investing in Neural Processing Units (NPUs) to enhance their AI processing capabilities:

• Intel: Their latest NPU, integrated into the Lunar Lake architecture, offers significant improvements in AI processing power and energy efficiency.
• AMD: AMD’s XDNA 2 chip boasts high performance with 50 TOPS (Tera Operations Per Second) and is designed for dynamic programmability.
• Qualcomm: Known for their Snapdragon X Elite, Qualcomm has entered the laptop market with NPUs that offer competitive AI processing capabilities.

What is the NPU good for?

Over time, we can expect more features to trickle into NPUs as the technology matures and becomes more widespread. This gradual integration will likely enhance the capabilities of NPUs, making them more versatile and efficient for a broader range of applications.

One of the absolute front runners, on utilizing the NPU, is Adobe. Especially the Media and Entertainment vertical has been quick to adapt the new features available in the NPU. Below is a list of the different applications, offered by Adobe, and what features are supported for the NPU usage:

Photoshop:

Neural Filters - These filters use machine learning, powered by Adobe Sensei, to generate new pixels in your images, allowing for non-destructive edits and creative adjustments , like Portrait adjustments, skin smoothing, colorization and JPEG artifacts removal.

AI-driven Content-Aware Fill - This tool analyzes the image and generates realistic fills, making it easier to achieve clean edits without manual cloning or patching.

Lightroom:

AI-based Subject and Object Detection - Powered by Adobe Sensei, significantly enhances photo editing by automating the selection process, like Selection of Objects, Sky and Backgrounds.

Super Resolution - A feature that effectively doubles the linear resolution, meaning the enhanced image will have twice the width and twice the height of the original, resulting in four times the total pixel count.

Noise Reduction - A feature that helps to minimize the grainy appearance (noise) in photos, which is often caused by shooting in low light or using high ISO settings. Examples of noise is Luminance noise and Color noise.

Premiere Pro:

Auto Reframe - A feature designed to automatically adjust the aspect ratio of your videos to fit different platforms and devices. This tool is particularly useful for creating content for social media, where different platforms often require different aspect ratios (e.g., square for Instagram, vertical for TikTok, and 16:9 for YouTube).

Scene Edit Detection - A feature that uses AI to automatically detect and mark the cut points in a video. This is particularly useful when working with a single, continuous video file that contains multiple scenes or edits, such as a compiled video or a final cut without access to the original project files.

These functions are also written for GPU usage, and will boost the performance even more, as the most basic entry RTX 500 Ada graphics card from Nvidia features well over 150 TOPS.!

GPU is the Performance King

Other, more demanding functions, are still heavily depended on the GPU. Here are some examples of functions within various Adobe apps:

Photoshop:

GPU-accelerated Denoising - reduces noise in images more efficiently and quickly. This feature uses advanced algorithms to analyze and clean up noise, which is especially useful for high-ISO images or photos taken in low-light conditions.

Image Upscaling and Filters (also available in Lightroom) - Upscaling has been explained under the NPU section, but gets even more powerful with the addition of a GPU. In addition filters like Blurring and Sharpen filters are heavily depended on a GPU from a Graphics Card.

Premiere Pro:

Mercury Playback Engine - The Mercury Playback Engine in Adobe Premiere Pro is a technology that leverages the power of your computer’s GPU (Graphics Processing Unit) to accelerate video editing and rendering tasks. It enhances the Real-time playback and Rendering.

Lumetri Color Engine - A powerful toolset designed for color correction and color grading. It provides a comprehensive suite of features that allow you to adjust and enhance the colors in your video projects, such as Basic corrections, Curves and HSL adjustments.

After Effects:

Real-time 3D Rendering - Real-time 3D rendering in After Effects, powered by the Advanced 3D renderer, allows you to preview changes to your 3D designs instantly. This feature is particularly useful for animators and motion graphics artists who need to see the impact of their adjustments without waiting for a full render.

Motion Graphics Rendering - This involves the process of creating the final output of your motion graphics projects. This process converts your compositions into a playable video file or sequence of images.

Media Encoder:

Hardware-accelerated Exporting and Encoding - leverages your computer’s GPU to speed up the process of rendering and exporting video files. This feature is particularly useful for handling high-resolution videos and complex projects, as it significantly reduces the time required for these tasks.

Wrapping up

So, to sum it all up, the NPU is not the answer to all our prayers. Though it may speed up some processes, at a low power usage, is it no match for a decent Graphics Card, when it comes to raw performance. NPUs are handy in a small notebook with an entry level power saving CPU, to take advantage of the superior calculation performance. But users of an ISV solutions, like the applications provided by Adobe would benefit more from a discrete Graphics Card, preferably in a Mobile Workstation, or even a Desktop Workstation, if you produce very heavy Video content.

As mentioned in this article, Adobe is one of the players at the very edge of AI and NPU utilization, but that is not to say that they are alone in the landscape. BlackMagic Design with their powerful video editor Davinci Resolve Studio has also embraced the NPU, and written several functions and features to utilize the NPU, but this would be basis for another article in the near future. When it comes to AEC and CAD users, we see almost usage of the NPU, but that's not to say, that it won't come in the near future. We just have to be patient and wait and see.

#Adobe #photoshop #lightroom #premierepro #blackmagicdesign #davinciresolve #ai #npu #cpu #gpu #thinkstation #thinkpad #workstation #wearelenovo