Recap of the NAB SHOW 2022
Are you a part of the video production industry? Then, the NAB Show is undoubtedly familiar to you!?Did you miss it this year? Don't worry, we will keep you updated in our posts!
This year, the event was held in the Las Vegas Convention Center from April 23rd to 27th. Several speakers from the OTT, broadcast, sports, and streaming media industries have?discussed?technical and business challenges ranging from ingestion and transcoding to media management and playback in order to provide the ecosystem with the highest quality experience.
In this post, we try to recap some of the NAB 2022 hot topics. In fact, we mainly?focus on the conferences and presentations that deal with video coding issues:
At C2M, one of the topics that we follow is the codecs, whether AI-based or not. So keep an eye on our next posts to stay updated about the latests analyses, tests and news, if you're passionate about these subjects, or others ;)
Real-World Use of AI for Better Video Compression (Tony Jones, Principal Technologist from MediaKind)
In this session, Tony Jones discussed how artificial intelligence (AI) is being used by MediaKind to improve the balance between video compression efficiency and processing power (CPU load). Their main goal is to use AI to increase the compression performance of encoders by implementing existing compression standards such as AVC and HEVC.
This talk explains how neural networks (NNs) can be used to perform this type of optimization, with results demonstrating improvement to the trade-off quality?vs?bitrate. It's worth noting that the costs of neural network processing are included into the overall CPU budget, which explains where and why AI might be a good fit. When AI is applied to some specific areas with huge data sets, the results demonstrate that significant reductions in bit rate (and/or CPU required) are attainable.
This session focuses on specific AI applications for video compression, at a high level (sequence of images) as well as at a very low level of decision-making. Let's dive a little deeper into the technical aspects and talk about?the NNs applications.
Different types of content produce different visual artifacts, which are reflected in the video's quality. A fast-action sports video clip, for example, is not the same as an animated film or a news show!
There are often variable limits that can be imposed to prevent processing requirements becoming excessive.?Recognizing the differences in the characteristics of different types of content, it is possible to classify compression processing into several categories of encoding features, i.e., broad categories of compression tools, such as: Motion Estimation, Motion Precision, Partition, Transition (scene change), Spatial Allocation, Sequencing (picture type, order and referencing) ...
The goal is to adaptively use all available compute resources, employing the most beneficial balance between the potential encoding characteristics, so that processing power is spent where it is most effective for that type of content. When the right balance is established, an encoder may deliver improved compression efficiency while using less processing power, intelligently adapting as the content changes. The graphics below depict how allocating processing resources to different encoding characteristics may be useful.
This produces AI-based mappings along the lines of the Figure?below.
Clearly, there is a significant match between each encoding feature task and the percentage of the resource usage.
Traditional approaches employ a pre-defined set of scale values for each of the tools they implement, making?the compute required predictable – which is a good thing, but may not reflect the best way to use that resource across all content types.
MediaKind showed its ability to make high-level decisions about how to best use a constrained CPU budget by using neural networks to learn the relationship between content features and compression tool set effectiveness. This means they can be applied in real time, dynamically adapting to the nature of the content.
Another approach is, as we said before, at lower level of decision-making, like for example, decisions for HEVC when performing Coding Tree Unit (CTU) splitting, dynamically based on detailed content characteristics. Nice, but how?! MediaKind developed a NN to predict the probability of achieving the lowest rate-distortion cost by splitting or not each single CU in the coding tree. Options that are unlikely to be optimal can be skipped, saving precious resources with negligible effect on compression efficiency.
In both cases, whether at high or low level decisions, the goal is to use the characteristics of the pre-processing procedures to process the information in real time and train the neural networks to learn how to make real-time decisions. This makes better use of the CPU resources available, either improving the quality of the bit rate obtained or lowering the CPU cost and environmental effect while maintaining similar performance.
To sum up, the results suggest that applying the appropriate sort of AI to applicable issues results in significant improvements, like for instance, an increased density case (increased channel density?by up to 50% for some cases) and an increased compression efficiency (reaching 40% for some cases) ...
Many other sessions adressed similar topics related to video compression. We can cite Harmonic talking about Green-AI based video compression, V-NOVA explaining their LCEVC... and many more. We will be covering these presentations and summarize them in a future post. Now let's talk about AV1!
领英推荐
AV1 video codec advances and real-world implementations
At this year’s NAB Show, diferent AV1 video codec advances and real-world implementations were presented by different industrials:
Visionular and AMD
The Visionular and AMD teams presented the newest AV1 performance results for live streaming applications such as broadcast, sports, and real-time communication (RTC). Using AMD EPYC and the Aurora1 AV1 encoder, it was demonstrated how to encode eight simultaneous live streams at 4Kp60 broadcast quality while maintaining the efficiency of standard AV1 encoding.
ATEME
ATEME demonstrated its premium AV1 file transcoder in the cloud enabling tremendous bitrate savings on film grain content. ATEME demonstrated this technology on a UHD OLED TV at the NAB Show.
Amazon Web Services (AWS)
This year, at the NAB Show, Amazon Web Services (AWS) introduced AV1 encoding, for both live streaming and VOD applications, offering excellent video quality at low bit rate compared to other video codecs. AWS Elemental MediaConvert, the VOD encoding service, now supports AV1 frame sizes up to 4K with 10-bit color depth, allowing you to create High Dynamic Range (HDR) UHD content for display on modern, color-rich 4K displays and devices.
Intel
Intel shared the latest AV1 HW and SW advancements commercially available through two sessions.
First session: Inspiration Session: Unleashing Intelligent, Interactive and Immersive Visual Experiences in the Cloud and at the Edge
Intel's Nagesh Puppala, General Manager Edge & Cloud Video Division at Intel, explained how Intel's innovations in video processing, AI, and edge computing are unleashing a whole new class of cloud-native deployments in the areas of live/remote production, video compression/streaming, AI video enhancement and analytics, and immersive video.
Second session: Innovation Session: ?Towards Ubiquitous AV1 encoding with SVT-AV1 1.0 ?
In this session, Intel announced the complete readiness and availability of its " powerful and flexible software encoder " SVT-AV1 for comercial deployments. Foued Ben Amara, software engineering manager at Intel, talked about new presets optimized for low latency and real-time communications, as well as the advances in VOD shot-based encoding.
iSIZE
iSIZE?is?a?deep-tech?firm?that?uses?deep?learning?to?produce?efficient,?intelligent,?and?long-lasting?video.?The?patented?iSize?technology?is?driven?by?the?most?cutting-edge?AI?advancements?to?provide?better?experiences?and?lower?the?financial?and?environmental costs?of?video?streaming. They use a deep neural network that evolves over time to minimize latency, increase video quality, and seamlessly integrate with any current codec without breakinf any standard. For video entertainment platforms, gaming, VR/AR, IoT, VOD, and live streaming services, this equates to significant bandwidth, energy, and cost savings.?
Video preprocessing and denoising solutions based on AV1 (SVT-AV1) as the underlying encoding technique were showcased by iSIZE.
NETINT
NETINT Technologies is an innovator in the development of 8K video processing systems based on the AV1, HEVC, and H.264 ASIC for real-time, low-latency video transcoding on x86 and ARM servers. When?compared?to?GPU-?and?CPU-based?software?encoding?solutions,?users?of?NETINT?solutions?obtain?software-grade?encoding?with?hardware?performance,?as?well?as?a?20-fold?improvement?in?encoding?density?and?a?40-fold?reduction?in?carbon?emissions.
NETINT presented the first enterprise hardware video encoder with built-in AI engine, as well as the world's first AV1 encoder for the data center and 40 1080p60 broadcast quality live streams on a 1RU server.
Last but not least,?Do not miss our next posts!
At C2M, we try our best to stay up to date with the latest news concerning the newest codecs. Particularly, we are following some tests with AV1 using SVT-AV1 and Aurora1 by Visionular, as well as the recent MPEG codecs: VVC, EVC and LCEVC. So, do not miss our next posts!