For Video Streaming Services, The Internet Is A Shared Resource To Preserve

TL;DR 

Post COVID-19, the work from home trend will continue, and this will extend the pressure on the Internet from video traffic. Even with the EU Commissioner's call for video services to reduce their traffic by 25%, as Internet traffic patterns shift from corporate networks to mobile, fixed wireless, and broadband networks, the need to reduce video bandwidths will continue beyond COVID-19. Consumers will still demand the highest quality, and those streaming services meeting their expectations while delivering video in as small a footprint as possible will dominate the market. Now is the time for the streaming video industry to play an active role in adopting more efficient codecs and content-adaptive bitrate technology so that streaming video services can ensure a great user experience without disrupting the Internet. 

The Internet is a shared resource to preserve.

For the video streaming industry, Thursday, March 19th, marked the day of reckoning for runaway bitrates and seemingly never-ending network capacity. On March 19th, Thierry Breton, the European Commissioner for the Internal Market tweeted, "let's #SwitchToStandard definition when HD is not necessary." The result is that most of the best known US video services, including Facebook and Instagram, agreed to a 25% reduction in bandwidth used for video delivered in Europe, UK, and Israel. With other countries rumored to follow suit. 

We can blame COVID-19 for the strain as a result of closed schools and businesses, leading to increased use of video conferencing, streaming video services, and cloud gaming. Verizon reported that total web traffic is up 22% between March 12th and March 19th, while week-over-week usage patterns for streaming video services increased by 12%. However, it's easily predictable that these numbers are trending even higher as the quarantine and shelter in place orders expanded, as evidenced by Cloudflare reporting Internet traffic is 40% higher than pre COVID-19 levels.

The purpose of this article is to provide a framework for how video streaming services may want to think about the Internet post COVID-19 where video streaming services and video-centric applications will need to consider their utilization of the Internet as a shared and not an unlimited resource.

Content-Adaptive Encoding is no longer a nice to have for streaming services. 

There are multiple technical and technology options available for reducing video bitrate. The fastest to implement, however, is to drop the resolution of the video. By manipulating the video playlist (called a manifest) that organizes the various resolutions and bit rates that enable the video player to adapt to the speed of the network, a video service can achieve immediate savings by merely serving a lighter weight version of the video. Standard-definition (SD) instead of high-definition (HD). This approach is what most of the complying services have taken, but, it is not a sustainable answer since dropping resolution impacts video experience negatively.

A more advanced technique known as Content-Adaptive Encoding works by guiding the encoder to adapt the bitrate allocation to the needs of the video content. 

Reducing resolution is not what consumers want, and this will make content-adaptive encoding essential for many video encoding workflows. Because content-adaptive encoding solutions require integration, for some services, it was relegated to the "nice to have" list. But now, with the sweeping changes to video consumption that is driving network saturation, those services that must compete with high visual quality, are shifting the priority to "must-have." 

Effective tools and methods to be a good citizen of the Internet.

If we are going to be a good citizen of the Internet, we should understand what tools and methods are available to preserve this precious shared resource while delivering a suitable UX and visual quality. 

Engineering for video encoding is about tradeoffs. The three primary levers are 1) bitrate, 2) resolution, 3) encoder performance. These levers are interconnected and dependent. For example, it's not possible to achieve high bitrate efficiency at higher resolutions without affecting encoder performance (increasing CPU cycles).  

From a video quality perspective, lever one and lever two are the levers available to most video encoding engineers. From an operational point of view, the third lever is what most impacts bitrate and quality.

The tools that we can use to reduce bandwidth include the use of advanced video codecs such as HEVC. HEVC (H.265) provides up to 50% reduction in bitrate at the same quality level as H.264, the current dominant codec used around the world. The other tool available is advanced technology, such as content-adaptive encoding, implemented inside the encoder. 

Beamr's Content-Adaptive Bitrate (CABR) rate-control is an example of advanced technology that brings an additional 20-40% reduction in bitrate. Using HEVC and CABR, a 4K HDR video file can be as small as 10Mbps, an added savings of as much 6Mbps without CABR. With the promise of a 50% bitrate reduction using HEVC, and over 2 billion devices supporting HEVC decoding in hardware, it's the obvious thing to do for a video service concerned about the sustainability of the Internet.

If a technical integration of a new codec is not possible, the three most popular methods for reducing bitrate are Per-Category, Per-Title, and Per-Frame Encoding optimization.

Per-Category Encoding optimization.

The Per-Category Encoding approach is least practical for premium movies, and TV shows since the range of encoding complexity within a category can vary significantly. Animated videos are typically easier to compress than video captured from a camera sensor, given the wide range of complexity. Animation techniques are highly diverse, from hand-drawn to 2D to 3D, and that makes it challenging to create an encoding ladder that works across animated content equally. 

Per-Category Encoding is the easiest of all the methods to implement, but also produces the lowest real bitrate reduction because of the variability of scenes. For example, a sports broadcast may include talking head in-studio shots along with fast action gameplay and slow-motion recaps, each requiring different bitrate values to preserve the quality level. 

Per-Title Encoding optimization.

Per-Title Encoding received a big boost when Netflix published a blog post explaining their encoding schema that creates a custom encoding ladder for each video file. The system performs a series of test encodes at different CRF levels and resolutions that are analyzed using the Video Multimethod Assessment Fusion (VMAF) quality metric. Netflix uses the scores to identify the best quality resolution at each applicable data rate. 

Though many video services have adopted their variation, Per-Title Encoding, or some variation of it can now be found in many video encoding workflows. It's a great way to rethink fixed ABR recipes that are the primary source of wasted bandwidth, or poor video quality. Per-Title Encoding only works when you have a smaller library as it requires extensive computing resources to run the hundreds of fractional encodes needed for each title. 

Per-Title Encoding helps to reduce bitrate but is limited in its ability since the rudimentary VBR rate-control bounds the encoder QP setting with no additional intelligence. 

Per-Frame Encoding optimization.

The weakness of a category or title based optimization method is that this approach cannot adapt to the specific needs of the video at the frame level. Only by steering the encoder decisions frame by frame is it possible to achieve the ultimate result of producing high quality with the least number of bits required. 

Beamr's CABR technology is the primary feature of the Beamr 4x and Beamr 5x encoding engines. CABR operates at the frame level to deliver the smallest possible size for each video frame while ensuring the highest overall quality of each frame within the video sequence. This approach avoids transient quality issues in other optimization techniques. The Beamr Quality Measure Analyzer has a higher correlation with subjective results than existing quality measures such as PSNR, and SSIM. CABR is protected by the majority of Beamr's 48 granted patents. 

To learn more about Beamr's Content-Adaptive Bitrate technology, you can hear Tamar Shoham, Head of Algorithms at Beamr, explain CABR here.

We must all play our part in preserving the integrity of the Internet.

Just as environmental sustainability is an essential initiative for companies who want to be good citizens of the world, in the COVID-19 world that we are living in, video sustainability is now an equally vital initiative. And this is likely to be unchanged in the future as the work from home and virtual meeting trends continue post COVID-19. Now is the time for the streaming video industry to play an active role in adopting more efficient codecs and content-adaptive bitrate technology so that we can ensure a great user experience without disrupting the Internet.