What is H.264 video encoding? How does H.264 codec work?

Video technology has undergone a rapid evolution in the past couple of decades. Earlier, videos were made of large collections of still photos, and they used excessive bulky files to make them digital. But now, video encoding has brought a technological transition – compressing these files to consume less space. It has also been possible to stream video over the Internet, both real-time and on-demand.

One of the most popular encoding technologies is H.264 (AVC – Advanced Video Coding) which has been able to solve many quality issues with respect to video broadcasting. In today’s blog, let’s learn what H.264 video encoding is, how it works, and its advantages in detail.

What is video encoding?

Video encoding is the process of converting raw video files to digital files so that they are not saved as individual images but as fluid videos. In other words, it is the process of compressing and potentially changing an analog source to a digital one. Generally, a coder/decoder (codec) is used for video file compression.

Before discussing codec, first, let us look at why video encoding is required in embedded vision applications.

Importance of video encoding in embedded vision applications

In the early days, video files were all in raw formats, i.e., they were a collection of still photos. For a video recorded at 30 frames per second (fps), you must use 30 photos per second of footage. Therefore, 1800 images per minute are used for a video, resulting in a massive video file size. The only solution to overcome this issue is to compress these videos, but the quality will be lost through this process. Later, engineers developed video encoding techniques to compress these files without compromising the quality.

Some of the common objectives of video encoding are:

• Reducing file size
• Reducing buffering for streaming video
• Changing video resolution or aspect ratio
• Changing audio format or quality
• Converting obsolete files to modern formats
• Making a video compatible with a certain device like a computer, tablet, smartphone, smart TV, etc.

What are codecs?

Codec is a software or video encoder that encodes or decodes a digital data stream or signal. It compresses the raw video and audio files between analog and digital formats and makes them smaller. The block diagram of a codec is shown below.

Figure 1: Block diagram of codec

(Source –?Brid.TV)

A codec consists of two components:

• Encoder?– handles compression.
• Decoder?– decompresses the file and prepares it for viewing.

There are hundreds of different codecs such as:

• 264/AVC
• 265/HEVC
• AV1
• VP9
• 266/VVC

Among these codecs, H.264/AVC and H.265/HEVC codecs are commonly used in?embedded cameras. Since the scope of this article is limited to H.264, let us now look at it in detail in the next section.

What is H.264/AVC?

H.264 is also called Advanced Video Coding (AVC) or MPEG-4 Part 10. It is a video compression technology jointly developed by the International Telecommunications Union (as H.264) and the International Organization for Standardization/International Electrotechnical Commission Moving Picture Experts Group (as MPEG-4 Part 10, Advanced Video Coding, or AVC).

Nowadays, H.264 codec is most commonly used in video streaming. This codec is an industry standard for video compression that helps creators record, compress, and distribute their online videos. It delivers good video quality at lower bitrates compared to previous standards. Hence, it is widely used in cable TV broadcasting and Blu-ray disks.

As a video codec, H.264 is frequently produced in the MPEG-4 container format, which uses the .MP4 extension, as well as QuickTime (.MOV), Flash (.F4V), 3GP for mobile phones (.3GP), and the MPEG transport stream (.ts). Sometimes, an H.264 video is encoded with audio compressed with the Advanced Audio Coding (AAC) codec, an ISO/IEC standard (MPEG4 Part 3).

How does H.264/AVC work?

The H.264 video encoder performs prediction, transformation, and encoding processes to produce a compressed H.264 bitstream. It utilizes a block-oriented standard with motion competition to process frames of video content. The output will be macroblocks that consist of block sizes as large as 16×16 pixels.

Now, the H.264 video decoder performs complementary processes like decoding, inverse transform, and reconstruction to produce a decoded video sequence. It receives the compressed H. 264 bitstream, decodes each syntax element, and extracts the information like quantized transform coe?cients, prediction information, etc. Further, this information will be used to reverse the coding process and recreate a sequence of video images. The H.264 video coding and decoding process is shown below.

Figure 2: Process diagram of H.264

(Source –?Vcodex)

Advantages of H.264

• Lower bandwidth usage and higher resolution monitoring –?It provides high-quality transmission of full-motion video with lower bandwidth requirements and lower latency than traditional video standards like MPEG-2. H.264 uses an efficient codec that provides high-quality images and uses minimal bandwidth.
• Lower H.264 bitrate than other formats –?It has an 80% lower bitrate than Motion JPEG video. It is estimated that the bitrate savings can be 50% or more compared to MPEG-2. For example, H.264 can provide a better image quality at the same compressed bitrate. At a lower bitrate, it provides the same image quality.
• Reduced demand for video storage –?It reduces the size of digital video file content by 50% and uses less storage to store video compared to other standards that prove essential to allow easy video transmission through IP.
• Incredible video quality– It delivers clear, high-quality video content at a data rate of ?, which is half the size of the other video format.
• More efficient?– It is two times more efficient, and the file size is 3X times smaller than the MPEG-2 codecs – making this compression format more efficient. This codec results in low transmission bandwidth for video content.
• Suitable for slow-motion video content– It is extremely efficient for low-motion video codecs using megapixel cameras.

H.264 cameras developed by e-con Systems

e-con Systems, a leading OEM camera manufacturer with 18+ years of experience, offers a wide range of H.264 cameras:

• Hyperyon – 2MP Sony STARVIS IMX290 ultra low light USB camera
• e-CAM83_USB – 8MP HDR camera based on Sony IMX317

If you are interested in integrating H.264 cameras into your product, please write to us at [email protected]. Meanwhile, you can visit our?Camera Selector?to have a look at our complete portfolio of USB, MIPI, and GMSL cameras.