It's time to move beyond Apple TN2224

This post originally appeared in part in the Beamr blog.  You can read the original here: https://blog.beamr.com/blog/2016/03/17/tn2224-is-so-yesterday/

Apples TN2224 is useful for those seeking best practices for HLS implementations, especially HLS on Apple devices.  But if you are still building your recipes based on recommendations from TN2224, the state of the art for encoders has moved on.

Apple issued TN2224 in 2010 to help video distributors understand HTTP Live Streaming to iPhones and iPads, however, encoding advancements and methods to pack video quality into smaller files have improved by orders of magnitude.  This article focuses on new techniques for choosing optimum bit rates based on requirements of the content and its target resolution as opposed to the static recipes recommended by Apple.  At Beamr, we are coining the phrase, "content adaptive" as a way to quantify this methodology for selecting optimum encoding bit rates.

Network congestion courtesy of streaming video

Network capacity is under ever increasing pressure, as a result of the wholesale adoption of streaming video.  In response, T-Mobile introduced a highly controversial yet successful program called Binge-On and even gained the FCC’s blessing.  We previously published a post on Binge-On and the general cost of network expansion as a result of video traffic, which you may like to read.  Click here.

Whether consumers are replacing traditional pay TV services or other entertainment offerings with streaming video, or augmenting (expanding) their entertainment choices, it doesn’t matter.  The fact is, consumers are streaming video over WiFi, mobile, and fixed wireline networks - in short "everywhere".  Meaning video services must plan their bit rates very carefully. With every modern smartphone and tablet supporting 1080p video playback, streaming content distributors must consider device capabilities as well as the bandwidth that the network can deliver.

For an iPad connected to a home WiFi network, the ISP may be capable of maintaining a robust 50 Mbps and there should be little difficulty in playing back smoothly a high bitrate 1080p profile.  But once that iPad leaves the home, even though it may be connected to an LTE network, due to varying network and RF conditions, the 6 Mbps that the mobile network is theoretically capable of delivering, could become highly variable leading to a poor user experience, or compromised quality.  

Selecting ABR profile variants, the options are endless

Let’s examine the process of choosing more efficient ABR variants for the range of devices and networks we must address.  The first step to moving beyond fixed bit rate profiles, is to select variations that are most appropriate for your application.  You need to understand the devices that your customers will be viewing content on, including network type and performance characteristics.  For example, if you know most viewers are watching on mobile devices.  With a half and half split of WiFi and radio connectivity, you know the screen size is limited, likely no larger than 5.5 inches for a handheld phone or 10 inches for a tablet.  This makes higher pixel density less relevant to perceptual quality since the viewing distance puts the additional pixels beyond the range of human perception.  Since you will find WiFi and cellular radio congestion poses significant streaming reliability issues, it should be seriously considered if you must deliver the top one or two profile(s).  If the player negotiates speed with the network and finds the bit rate fits within the buffer model, there is a strong argument for not including these extra bits (pixels) to a device screen where the pixel size is beyond the range of human perception.

When it comes to user engagement, UX and general satisfaction, many services fail in one of the following ways:

1- Poor video quality.  The issue with poor video quality is triggered by the use of overly aggressive encoding bitrates.  The reason services pursue overly aggressive bit rates is the tension between marketing, who feels they must offer 1080p (after all 1080 is a larger number than 720), and the reality of data rates that do not support the needed bit rate to accurately encode the highest quality without artifacts. 

2- Low streaming quality.  The optics of viewing high-density screens such as retina displays, at higher resolutions such as 1080p, means much of the video that is encoded at the higher resolutions, is beyond human vision.  More bits are sent across the network than are needed or can even be seen, triggering excessive buffering due to congestion.   

Do not feel you must encode HD at 1080p - 720p may provide higher quality at the maximum targeted bit rate

On many mobile devices, encoding at 720p will yield better quality at lower bit rates.  The cause of poor streaming UX is easy to understand.  It is simply an issue of encoding at bit rates that are too high for the network.  The best example is right out of TN2224, where Apple recommends 1080p @ 8.5 Mbps.  

Though it is possible, on a rock solid LTE connection, to achieve sufficient throughput, the odds are pretty low of this happening in actual practice.  Even on WiFi, where interference is highly variable, 8.5 Mbps can be problematic.  Want proof?  Consider Netflix’s ISP Speed Index that shows even on Verizon FiOS, the average bit rate Netflix can stream reliably is 3.79 Mbps.

Content adaptive is the next frontier in encoding

Last December, Netflix announced they are re-encoding the entire library to gain a 20% reduction in bit rate.  For more details you'll want to read our blog post on the announcement here.  Needless to say, when the leader announces that they are re-encoding their library, using a content-based strategy for selecting bit rates, as opposed to fixed recipes, our ears perked up.  The Beamr approach is different, in that our solution adapts on a frame level, whereas Netflix is selecting a target bit rate based on scanning the entire file.  But the basic idea that we agree with Netflix on is that for a given a piece of content, fixed recipes will fail to deliver the optimum bit rate needed to balance video quality with file size.

Future posts will expand on this innovative approach and demonstrate why fixed recipes may soon be as antique as a 57 Chevy.  Thank you for reading.

I love feedback.  Please comment below and let me know how your company is moving beyond fixed recipes to a content adaptive approach.