Navigating the World of IP Media Protocols
Reza Rahimi
APJ Regional Sales Strategy | Network Solution | AWS certified Cloud Architecture | Cyber Security
This article provides a comprehensive reference table for IP Media Protocols. The table outlines various IP transport protocols and industry standards. Given the extensive list of IP media protocols available—from foundational ones like RTP and UDP to more specialized streaming formats—this resource aims to demystify the complex landscape for professionals and enthusiasts alike in IP media streaming.
ST 2110
Release year: Released in 2017 and revised in 2022
SMPTE 2110 is a groundbreaking suite of standards developed by the Society of Motion Picture and Television Engineers (SMPTE). It revolutionises how digital media is transported by providing guidelines on transmitting media over IP networks. Instead of bundling everything into a single stream as in past broadcast standards, SMPTE ST 2110 distinctively outlines the delivery, synchronisation, and characterisation of individual essence streams, such as video, audio, and ancillary data. This enables seamless transmission over managed IP networks with varying speeds, ranging from 1 to 100 Gigabit Ethernet and beyond. This adaptation is particularly beneficial for real-time production, playout, and many other professional media applications, marking a significant stride forward in integrating broadcast technology with modern networking.
ST 2110-10
Release year: 2022
Additional info: ST 2110-10:2022
The ST 2110-10 standard outlines the framework for timing models in systems utilising RTP-based streams for different media essences. These streams are synchronised to a unified reference clock, establishing their temporal correlations. This standard is the cornerstone for understanding and implementing system timing. It offers fundamental guidelines that apply across all media essence streams in the suite.
RP 2110-11
Release year: TBD
Additional info: TBD
This recommendation has to do with timing planes.
ST 2110-20
Release year: Revised in 2022
Additional info: ST 2110-20:2022
The ST 2110-20 standard specifies transmission of uncompressed live video over IP networks utilising RTP-based transport techniques. An SDP-enabled signalling system manages essential image metadata to facilitate accurate reception and interpretation.
ST 2110-21
Release year: 2017
Additional info: ST 2110-21:2017, TV Technology
The SMPTE ST 2110-21 standard outlines the framework for traffic shaping and delivery timing for video RTP streams within the broader SMPTE ST 2110-10 architecture. This standard focuses explicitly on the timing model for these video RTP streams, measured when leaving the RTP sender. In addition to establishing the timing characteristics, ST 2110-21 also defines the SDP (Session Description Protocol) parameters the sender uses to signal the timing properties of such video streams. This ensures consistent and optimized delivery of video content across the network, enhancing interoperability and performance.
ST 2110-22
Release year: 2022
Additional info: ST 2110-22:2022
ST2110-22 specifies RTP-based constant bitrate compressed video transport.
RP 2110-23
Release year: 2019
Additional info: RP 2110-23:2019
RP 2110-23 defines an approach to dividing a high-bandwidth, singular video essence stream into multiple, lower-bandwidth SMPTE ST 2110-20 sub-streams and detailing how to correctly assemble and label these sub-streams, covering aspects such as SDP declarations, addressing standards, and RTP timestamp requirements. Lower-bandwidth streams can be generated using the 2SI and Square Division methods for UHD content or by reducing the video to streams with lower frame rates.
RP 2110-24
Release year: 2023
Additional info: RP 2110-24:2023
RP 2110-24 is a recommended practice for Standard Definition video over ST 2110 and specifies a relationship between the Sample Rows of SMPTE ST 2110-20 signals and the line numbering of SMPTE ST 125:2013.
RP 2110-25
Release year: 2023
Additional info: RP 2110-25:2023
The RP 2110-25 standard outlines the suggested terminology for taking measurements on SMPTE 2110 systems and the corresponding equations to ensure uniformity in executing and recording these measurements. This Recommended Practice also provides a variety of approaches for implementing ST 2110-21 buffer measurements. The characteristics and distinctions between these approaches are detailed, along with guidance on how to present the results in a way that helps users grasp their differences.
ST 2110-30
Release year: 2017
Additional info: ST 2110-30:2017
The ST2110-30 standard focuses on the live transmission of PCM digital audio streams over IP networks, using RTP as the underlying transport mechanism. It references AES67 to outline the specific protocols and methods used. To facilitate the correct receipt and interpretation of the audio streams, metadata is communicated through an SDP-based signalling approach. The standard does not cover non-PCM digital audio or compressed audio formats.
ST 2110-31
Release year: 2018
Additional info: ST 2110-31:2018
AES3 Transparent Transport - Describes the transport of AES3 audio, which may encompass non-PCM formats.
ST 2110-40
Release year: 2018
Additional info: ST 2110-40:2018, IP Showcase
SMPTE ST 2110-40 is a standard that governs the conveyance of ancillary data, often known as VANC, such as captions and timecodes, in real-time alongside main audio and video streams. This is done using RTP payload-based methods over IP networks and is synchronized using a common reference clock. This ensures that the additional elements like captions or timecodes are perfectly timed with the related digital video data streams.
ST 2110-43
Release year: 2021
Additional info: ST 2110-43:2021
Specifies the real-time, RTP-based transport of Timed Text Markup Language (TTML) for captions and subtitles in systems conforming to SMPTE ST 2110-10.
Used with ST 2110
ST 2059-2 PTP
Release year: 2015
Defines a specific profile for the Precision Time Protocol (PTP) set out in the IEEE 1588 standard. This profile ensures precise synchronization of audiovisual systems over IP networks. Within an ST 2110 system, various networked devices must operate in unison, relying on a shared reference clock for synchronization. Incorporating the SMPTE ST 2059-2 PTP profile ensures that all these devices are perfectly aligned in time, making seamless delivery and processing of real-time media over the network possible.
SDP
Release year: 1998
The Session Description Protocol (SDP), defined by IETF's RFC 4566, is a standardized format for multimedia sessions, particularly for media streaming and teleconferencing. SDP negotiates media types, formats, and properties between endpoints but doesn't transmit media itself. It facilitates information exchange primarily for media applications, including mobile networks. Within the context of SMPTE ST 2110, SDP describes individual media streams' parameters, such as media type, format, timing, and traffic characteristics. Each stream in ST 2110 has its own SDP document, accessible via the sending device's management interface, ensuring a shared understanding of stream details between sender and receiver.
ST 2022
Release year: 2007
Additional info: Wikipedia
SMPTE 2022 outlines how to transmit digital video, including MPEG-2 and serial digital interface formats (SDI), over IP networks. The standard has eight parts 1 to 8 listed below.
ST 2022 parts 1-4
ST 2022-1 and -2 cover low to midrange bandwidth
ST 2022-1
Release year: 2007
Additional info: ST 2022-1:2007
Forward Error Correction (FEC) for Real-Time Video/Audio Transport Over IP Networks.
ST 2022-2
Release year: 2007
Additional info: ST 2022-2:2007
Unidirectional Transport of Constant Bit Rate (CBR) MPEG-2 TS (Transport Streams) for compressed video transport over RTP on IP Networks.
ST 2022-3
Release year: 2019
Additional info: ST 2022-3:2019
Unidirectional Transport of Constant Bit Rate (CBR) MPEG-2 TS (Transport Streams) for compressed video transport over RTP on IP Networks.
ST 2022-4
Release year: 2011
Additional info: ST 2022-4:2011
Unidirectional Transport of Non-Piecewise Constant Variable Bit Rate (VBR) MPEG-2 TS over RTP on IP Networks.
ST 2022 parts 5-6
ST 2022-5 and -6 cover mid to high bandwidth - High Bit Rate Media (HBRMT)
ST 2022-5
Release year: 2013
Additional info: ST 2022-5:2013
Forward Error Correction for Transport of High Bit Rate Media Signals over IP Networks (HBRMT)
ST 2022-6
Release year: 2012
Additional info: ST 2022-6:2012
The SMPTE 2022-6 standard provides guidelines for transporting high-bitrate, uncompressed video, audio, and ancillary data over IP networks. This standard ensures interoperability between equipment from different vendors, particularly in live production and broadcasting environments. It specifies how to encapsulate these media streams into IP packets using the User Datagram Protocol (UDP), facilitating a seamless transition from traditional Serial Digital Interface (SDI) systems to modern IP-based infrastructures. While it doesn't prescribe the type of IP network or handle error correction, its focus on low-latency, real-time transmission has made it a cornerstone in the evolution of professional media workflows.
ST 2022 Protection and Timing
ST 2022-7
Release year: 2019
Additional info: ST 2022-7:2019
SMPTE 2022-7 provides a methodology for seamless protection switching, effectively allowing for redundant transmission paths. This scheme sends identical data streams over two separate, usually diverse, network paths. At the receiving end, the system selects the best packets from either of the two paths, allowing for seamless recovery from packet loss, delay, or failure in one of the paths. This provides a higher level of resilience and ensures uninterrupted service, making it a critical component for mission-critical broadcast and professional media applications where any loss or interruption is unacceptable.
ST 2022-8
Release year: 2019
Additional info: ST 2022-8:2019
SMPTE 2022-8 specifies how to use SMPTE ST 2022-6 streams in sync with the timing framework of SMPTE ST 2110-10. It includes the creation of a Synchronizing Timestamp in the ST 2022-8 receiver to harmonise the two standards. Additionally, the standard covers the Session Description Protocol (SDP) announcements for SMPTE ST 2022-5 Forward Error Correction (FEC) and SMPTE ST 2022-7 redundancy measures for these streams.
VSF
VSF technical recommendations are listed below.
VSF TR-01
Latest Release: 2018
Additional info: VSF TR-01
The VSF Technical Recommendation (TR) titled VSF TR-01:2018 provides guidelines for streaming JPEG 2000 Broadcast Profile video over IP networks using an MPEG-2 Transport Stream. This updated recommendation incorporates additional capabilities over its 2013 predecessor, including ultra-low latency encoding for professional video, support for higher video resolutions up to 4K and beyond, expanded colour space and mastering display metadata in line with Rec. ITU-T H.273 and SMPTE ST 2086:2018, and the resolution of interoperability issues concerning the JPEG 2000 elementary stream header by designating Annex S of Rec. ITU-T H.222.0 | ISO/IEC 13818-1 as the standard reference. The document aims to foster interoperability across equipment from different manufacturers.
领英推荐
VSF TR-02
Latest Release: 2015
Additional info: VSF TR-02
The TR-07 is a VSF Technical Recommendation detailing the use of JPEG XS compression for real-time video streaming over IP networks. Designed for its small size and speed, the TR-07 standardizes JPEG XS video, audio, and ancillary data transportation within an MPEG-2 Transport Stream. It also introduces an optional Forward Error Correction scheme for wide area network (WAN) applications. It seeks feedback on potential intellectual property issues.
VSF TR-03
Latest Release: 2015
Additional info: VSF TR-03
TR-03 is a Technical Recommendation for transmitting live professional media over IP, aiming to enhance video production flexibility. Unlike traditional methods that bundle video, audio, and ancillary data into the Serial Digital Interface (SDI) before IP encapsulation, TR-03 allows these elements to be transmitted as separate streams. This approach ensures efficient routing through a network and eases coordination with network-based registration and discovery of media capabilities.
VSF TR-04
Latest Release: 2015
Additional info: VSF TR-04
TR-04 is a Technical Recommendation that advances the system defined in VSF TR-03. It integrates SMPTE ST 2022-6 as a video payload, enabling a harmonious environment between SDI-based and IP-based in-studio networking equipment. By incorporating AES-67 audio signals and networked metadata, TR-04 ensures compatibility with professional audio gear while retaining the streamlined nature of SDI multiplex. This approach permits the separate forwarding of SDI multiplex, audio, and metadata streams through a network.
VSF TR-05
Latest Release: 2018
Additional info: VSF TR-05
TR-05 is a Technical Recommendation detailing essential formats for uncompressed video essence transport in line with SMPTE ST 2110-20:2017. These formats are defined using SDP parameters to ensure consistent IP packet stream specifications and enhance device interoperability. The document promotes using colloquial format names and Format Groups for streamlined documentation. It also requests readers to report any potential patent or intellectual property rights violations related to its guidelines.
VSF TR-06
Latest Release: 2020-2022
The VSF TR-06 series offers an open, interoperable protocol for reliable Internet streaming, addressing the market's fragmented landscape of proprietary solutions. It champions cross-vendor compatibility, with Part 1 detailing the core protocol and introducing an updated RTCP RTT Echo Request. Part 2 emphasizes the need for interoperability. Part 3 enriches the base protocol to cater to both low-latency media and legacy systems, incorporating features like packet loss recovery, content identification, and security. All parts also underscore the significance of intellectual property considerations.
VSF TR-07
Latest Release: 2022
Additional info: VSF TR-07
The TR-07 is a VSF Technical Recommendation detailing JPEG XS compression for real-time video streaming over IP networks. Designed for its small size and speed, the TR-07 standardizes JPEG XS video, audio, and ancillary data transportation within an MPEG-2 Transport Stream. It also introduces an optional Forward Error Correction scheme for wide area network (WAN) applications. It seeks feedback on potential intellectual property issues.
VSF TR-08
Latest Release: 2022
Additional info: VSF TR-08
The TR-08 is a VSF Technical Recommendation detailing using JPEG XS video streaming over SMPTE ST 2110-22, including transporting audio and ancillary data. JPEG XS, symbolizing "extra small" and "extra speed," facilitates cost-effective, high-quality, real-time video transmission over IP networks. Recipients of the document are urged to offer feedback and disclose any intellectual property concerns related to the Recommendation.
VSF TR-09
Latest Release: 2022
Additional info: Data Plane, Control Plane
SMPTE ST 2110 facilitates media production facilities to employ IP connectivity for both media flow and control, primarily focusing on intra-campus connections. However, with the rise in remote and multi-campus operations, there's a need for transport guidelines over Wide Area Networks (WANs). The Technical Recommendation is split into two sections: TR-09-1, which defines the data plane for secure media flow transport over WANs, and TR09-2, which describes the control plane, ensuring robust communications with security considerations.
VSF TR-10
Latest Release: 2023 DRAFT
Additional info: VSF TR-10
The VSF TR-10 suite of Technical Recommendations delineates the distinctions between the SMPTE ST 2110 standard and the Internet Protocol Media Experience (IPMX). IPMX was designed to champion open standards-based protocols for IP interoperability within the media, entertainment, and professional audio/video sectors. The VSF TR-10-0 document provides an overview of the organization and structure of the multiple Technical Recommendation documents that comprise the TR-10 specification. These documents range from TR-10-0 to TR-10-12 and include various facets of IPMX, such as "System Timing and Definitions," "Uncompressed Active Video," "PCM Digital Audio," "SMPTE ST 291-1 Ancillary Data," "HDCP Key Exchange Protocol," "Forward Error Correction," "NMOS Requirements," "Constant Bit-Rate Compressed Video," and "AES3 Transparent Transport," among others. Each of these documents is currently in its draft form.
IPTV (1990s)
I collected all the early IPTV protocols in this section.
RTP (Real-time Transport Protocol)
Release year: 1996
Additional info: Wikipedia, Mozilla Dev
Real-Time Transport Protocol (RTP) is a cornerstone for transmitting audio and video over IP networks. Designed to cater to the demands of streaming media, RTP finds extensive applications in telephony and video teleconferencing, including WebRTC and television services. Born out of a necessity for an efficient and adaptive protocol, RTP was developed in the mid-1990s by the Audio-Video Transport Working Group of the Internet Engineering Task Force (IETF). Initially outlined in 1996's RFC 1889, it was refined and updated in 2003's RFC 3550. Beyond mere media transmission, RTP plays a critical role in ensuring a smooth and optimized delivery by offering mechanisms for jitter compensation, packet loss detection, and addressing out-of-order deliveries – challenges particularly prevalent in UDP transmissions over IP networks. Its inherent design also facilitates data distribution to numerous destinations using IP multicast.
RTSP (Real-Time Streaming Protocol)
Release year: 1998
Additional info: Wikipedia, RFC 2326
A network protocol specifically tailored for controlling the delivery of multimedia content, like video and audio, across the internet. The framework enables functionalities such as play, pause, or record for media streams. Originating from a collaboration between RealNetworks, Netscape, and Columbia University during the 1990s, its development has been under the purview of the Internet Engineering Task Force (IETF). The most recent iteration, RTSP 2.0, was introduced in 2016, improving upon RTSP 1.0 but with limited backward compatibility.
RTCP (Real-time Transport Control Protocol)
Release year: 1996
Additional info: Wikipedia, GeeksForGeeks
The Real-Time Control Protocol (RTCP) is a companion protocol to the Real-time Transport Protocol (RTP) and is delineated in RFC 3550. While RTP is responsible for delivering and packaging multimedia data, RTCP does not transport media data. Instead, its chief role is to offer feedback about the Quality of Service (QoS) in media distribution. It accomplishes this by periodically transmitting statistics, such as transmitted packet counts, packet loss, packet delay variation, and round-trip delay time to participants of a multimedia streaming session. Applications can then leverage this feedback to adjust QoS parameters, like modulating flow or opting for a different codec.
MPEG TS (Transport Stream) over UDP
Release year: 1995
MPEG Transport Stream (MPEG-TS) over IP refers to transmitting digital container formats containing audio, video, and PSIP data over Internet Protocol networks. Initially designed for broadcast systems such as DVB, ATSC, and IPTV, MPEG-TS encapsulates packetized elementary streams with features like error correction and synchronization patterns for maintaining integrity. When transported over IP networks, this allows for the delivery of broadcast-quality content across platforms like IPTV or even online streaming, bridging the traditional broadcast and internet realms. Unlike the MPEG program stream tailored for media like DVDs, MPEG-TS over IP is adapted for varying network conditions, ensuring resilient delivery even over less stable networks.
Over Internet
This list contains the ARQ protocols used for the transport of video over unmanaged IP networks.
RIST (Reliable Internet Stream Transport)
Release year: 2018
Additional info: RIST Forum
RIST, or Reliable Internet Stream Transport, is an open-source protocol designed for dependable video transmission over challenging networks like the Internet. It balances low latency and high quality, making it invaluable for maintaining a seamless viewing experience. Spearheaded by the Video Services Forum's "RIST Activity Group," RIST's development aims to revolutionise video transmission. RIST is Engineered to ensure swift and faithful media stream transmission; even in the face of network losses,?libRIST?plays a crucial role in delivering the content that defines our digital interactions today.
SRT (Secure Reliable Transport)
Release year: 2013
Additional info: Haivision
SRT (Secure Reliable Transport) is an open-source video streaming protocol designed for dependable streaming over unreliable networks like the public internet. Using a retransmission method (ARQ) over UDP, SRT ensures secure, low-latency streaming despite packet loss and varying bandwidth. It's notably robust, handling up to 10% packet loss without compromising stream quality. SRT finds its niche in scenarios like remote live interviews and distributing events from headquarters to branch offices, particularly suited for challenging network conditions.
Zixi
Release year: 2007
Additional info: Zixi
Zixi is a video streaming protocol designed for secure, reliable, and scalable video streaming. It is content and network-aware, allowing it to adjust dynamically to both the type of content and the prevailing network conditions. Using advanced error correction techniques, Zixi ensures consistent video quality, even in challenging transmission scenarios or unstable networks. AWS Elemental MediaConnect, a recognized cloud service for video processing, fully supports the Zixi protocol for incoming and outgoing live video streams, highlighting its industry significance and dependability.
WebRTC (Web Real-Time Communication)
Release year: 2011
Additional info: WebRTC
WebRTC (Web Real-Time Communication) is an open-source technology that empowers web applications and websites to capture and transmit audio, video, and arbitrary data in real-time between browsers, eliminating intermediaries. Designed to integrate seamlessly with web browsers and mobile applications through a suite of application programming interfaces (APIs), WebRTC facilitates real-time communication (RTC) across diverse platforms. For those keen on delving into its intricacies, the Mozilla Developer Network (MDN) offers an in-depth guide detailing its fundamentals, setup processes, and effective media and data connection management.
RTMP (Real-Time Messaging Protocol)
Release year: 2002
Additional info: Wikipedia, GetStream.io
RTMP, or Real-Time Messaging Protocol, is a proprietary protocol developed by Macromedia (later acquired by Adobe) used primarily for transmitting audio, video, and other data between a server and a Flash player. It facilitates low-latency communication, making it a popular choice for live-streaming applications. As the internet has evolved, while newer protocols like HLS and DASH have gained popularity for video delivery, RTMP remains a common choice for ingesting live content to streaming platforms due to its real-time capabilities.
OTT Distribution
HLS (HTTP Live Streaming)
Release year: 2009
Additional info: Apple, Wikipedia, Cloudflare
HTTP Live Streaming (HLS) is a streaming protocol created by Apple Inc. in 2009 that utilizes adaptive bitrate technology to optimize video playback based on network conditions. HLS delivers content using the universally supported HTTP protocol by breaking videos into small HTTP files. This means it's compatible with virtually all internet-connected devices without specialized servers. One standout feature is its ability to adjust video quality on the fly in response to network speed, ensuring continuous playback even under varying conditions. Poor network conditions could potentially halt a video without this adaptive bitrate streaming.
MPEG-DASH
Release year: 2012
Additional info: Chiariglione, MDN, Wikipedia
Dynamic Adaptive Streaming over HTTP (DASH), often MPEG-DASH, is an international standard for adaptive bitrate streaming. It enables high-quality media streaming using regular HTTP web servers by dividing content into small segments at different bit rates. The client's device autonomously selects the optimal bit rate for each segment based on current network conditions, ensuring smooth playback without buffering interruptions. This means viewers experience fewer stalls and a consistent streaming quality tailored to their internet speed. The standard was revised in 2019 and once more in 2022.
Smooth Streaming
Release year: 2009
Additional info: Microsoft
Microsoft, specifically designed Smooth Streaming to deliver media files over HTTP. The essence of this technology is to optimize the viewer's experience by dynamically adjusting the video quality in response to the viewer's bandwidth and device capabilities. Its foundation on the PIFF (Protected Interoperable File Format) specification sets Smooth Streaming apart. This particular format is pivotal as it addresses the challenges of DRM (Digital Rights Management) interoperability, ensuring that digital media content is protected yet accessible across different platforms and devices.
Audio
AES67
Latest Release: 2015
Additional info: AES, Wikipedia
AES67 is a standard introduced by the Audio Engineering Society to ensure interoperability between different audio over IP (AoIP) and Ethernet (AoE) systems, such as RAVENNA, Livewire, Q-LAN, and Dante. Positioned at layer 3 of network protocols, it bridges the gap between competing networked audio systems. It supports synchronising audio streams using PTP and allows for 80 audio channels per stream.
Dante
Latest Release: 2006
Dante, developed in 2006 by Audinate, combines software, hardware, and network protocols to transmit uncompressed, multi-channel, low-latency digital audio over Ethernet networks using Layer 3 IP packets. Originating from a project at the National Information and Communication Technology Australia (NICTA) research centre, Audinate, after its establishment, expanded globally, and as of March 2021, it has licensed its technology to 350 companies, resulting in over 3000 Dante-integrated products.
Ravenna
Latest Release: 2010
Ravenna is a technology tailored for real-time audio and media data transmission over IP networks. It seamlessly integrates with most existing network infrastructures, leveraging standard networking technology. Ravenna's performance scales with the network, meeting broadcasters' low latency, signal transparency, and high reliability demands. It finds applications in broadcasting house signal distribution, venue setups, live events, outside broadcasting, and inter-studio connections across WAN links and production facilities.
Livewire+
Latest Release: 2003
Additional info: Telos Alliance
Livewire, developed by Axia Audio, a division of Telos Alliance, is an audio-over-IP system primarily for routing and distributing broadcast-quality audio within radio stations. Introduced in 2003, the original Livewire standard evolved into Livewire+, encompassing AES67 and Ravenna standards for broader equipment interoperability. Leveraging multicast networking, Livewire+ facilitates flexible anywhere-to-anywhere audio stream routing and employs standard IP and Ethernet over twisted pair cabling.
AV over IP | Media Networks | ProAV-Broadcast
8 个月This is great. Thank you Reza Rahimi.
Director Sales Engineering (Retired)
1 年Awesome!
Sales Director @ Net Insight | Live Media Networks and Transport
1 年What a fantastic summary thanks Reza!
Business Development Strategist at Techtel
1 年?