Streaming Technologies Delivering Multimedia into the Future May 2014
TABLE OF CONTENTS Abstract... 3 Abbreviations... 4 How it started?... 6 Technology Overview... 7 Streaming Challenges... 15 Solutions... 16 Conclusion... 18 References... 19 Author Info... 20 2013, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.
In today s era of connected world with increasing bandwidth and improving encoding standards multimedia streaming has become primary means of sharing multimedia content across the world. Abstract Multimedia as we know is a combination of both audio and visual content. There have been various ways of sharing and distributing digital multimedia content, like using tapes, CDs, DVDs, cable TV, over network (file download), and streaming. In today s connected era with increasing bandwidth and improving encoding standards multimedia streaming has become the primary means of sharing multimedia content across the world. With lots of choices available in terms of streaming technologies, each one has its pros and cons for various fields. No mechanism fits all kinds of applications like live or on-demand and areas like online video and video surveillance. 3
Abbreviations Sl. No. Acronyms (Page No.) Full form 1 ABR Adaptive Bitrate Streaming 2 AVC Advanced Video Coding 3 CD Compact Disc 4 CPU Central Processing Unit 5 DASH Dynamic Adaptive Streaming over HTTP 6 DASH-IF DASH Industry Forum 7 DVD Digital Versatile Disc 8 GOP Group of Pictures 9 GPAC GPAC Project on Advanced Content 10 HEVC/H.265 High Efficiency Video Coding 11 HLS HTTP Live Streaming 12 HTTP Hypertext Transfer Protocol 13 IEC International Electrotechnical Commission 14 IETF Internet Engineering Task Force 15 IIS Internet Information Services 16 ISO International Organization for Standardization 17 MMS Microsoft Media Services 18 MPEG Moving Picture Experts Group 19 NAT Network address translation 20 QOS Quality of Service 21 RTCP RTP Control Protocol 4
22 RTMP Real Time Messaging Protocol 23 RTP Real-time Transport Protocol 24 RTSP Real Time Streaming Protocol 25 SDP Session Description Protocol 26 TCP Transmission Control Protocol 27 UDP User Datagram Protocol 28 URL Uniform Resource Locator 29 VOD Video on Demand 30 XML Extensible Markup Language 5
How it started? Consumer grade computers in the late 1980's and early 1990's were able to play multimedia content. However, there were issues related to limitations of computer networks that prevented streaming as a technology to gain momentum. Anything happening close to streaming was downloading a multimedia file to a system and playing it locally. As the computer networks improved and internet became popular, it encouraged streaming to become an important method of delivering multimedia which gave rise to lots of streaming technologies over time like RTSP from IETF, Microsoft's Media Services (MMS), Adobe's Real Time Messaging Protocol(RTMP), and HTTP progressive download. Recently a set of methods known as adaptive streaming has come up that adds lots of advantages to HTTP based streaming technologies. 6
Technology Overview Streaming basically refers to a method of delivering multimedia, wherein content is delivered by a server (streaming server) and received and played by a client usually a media player. Now content could be available primarily in two forms, one is content generated on-the-fly from a live event like a soccer match which needs to be delivered in real time. And then there is recorded or stored content like a movie or recording of a previously occurred event. In the same way, as the type of content, there are two types of streaming live streaming for real time viewing and ondemand streaming for viewing stored content. Streaming is gaining more popularity day-by-day as a method of delivering multimedia, which has led to creation of multiple streaming protocols. Here we are giving a brief overview of some of the key streaming protocols in use today. RTSP/RTP Streaming Real Time Streaming Protocol (RTSP) is a protocol which establishes and controls multimedia sessions between client and server. Client issues commands like Play, Pause, Stop media streaming to media server. Actual multimedia content is not transmitted over RTSP. Most RTSP media servers use Real Time Transport Protocol (RTP) along with RTP Control Protocol (RTCP) for media transmission. RTP carries the media and RTCP provides feedback on Quality of Service (QoS). The details about the multimedia streams to be controlled are described by using presentation description. A presentation description contains information about one or more media streams within a presentation, such as the set of encodings, network addresses and information about the 7
content. Mostly Session Description Protocol is used for presentation description. Figure 1 RTSP streaming Figure 1 shows typical message flow in a media session establishment and termination using RTSP. Sources of data can include both live data feeds and stored multimedia clips. RTSP is similar to HTTP in syntax and operation. The default transport port for RTSP is 554. When a client want to access a media stream it would send a DESCRIBE request with an RTSP URL of the media stream to the RTSP server. Server would respond with a 200 OK response with session description usually in SDP format. Session description would provide media attributes (like codec) about media streams present. Typically a session would consist of an audio and video stream. Then client sends a SETUP request specifying how a 8
single media stream must be transported. The request contains the media stream URL and a transport specifier. This specifier typically includes a local port for receiving RTP data (audio or video), and another for RTCP data. The server reply usually confirms the chosen parameters, and fills in the missing parts, such as the server's chosen ports. Each media stream must be configured using SETUP before an aggregate play request may be sent. After receiving 200OK for SETUP requests client sends a PLAY request. On receiving PLAY request server would send 200 OK response and start playing media streams. A PAUSE request temporarily halts one or all media streams, so it can later be resumed with a PLAY request. When client wants to terminate the session it sends a TEARDOWN request. This request stops all media streams and frees all session related data on the server. RTP implementations typically use UDP as transport. There are a number of Open Source RTSP servers and client available. Some of the open source RTSP servers are VLC, FFSERVER, live555mediaserver, Darwin Streaming Server. Some open source RTSP clients are VLC client, live555 Media Framework. HTTP delivery based streaming There are various methods of streaming available today that rely on HTTP delivery as a method using a web server instead of a streaming server for streaming of multimedia content. Here is an overview about those methods. HTTP Progressive download Progressive download is nothing but simple HTTP download of video files to the user s computer. A media player that is capable of progressive download could start playing the video 9
file before the whole file is downloaded. This capability relies on the meta-data present in header of a video file, so once the header and a specific portion of video file is downloaded, player could start playing the file. Let s say we want to stream an.mp4 file using progressive download, then it requires moov atom data must be placed at the beginning of the file structure. This way required movie information is downloaded first, so that playback could start immediately. If the download speed is more than the bit-rate of the encoded video then video playback would appear as if it streaming the video. This method do not allow user to playback sections of a video that have not yet been downloaded. So, if a user tries to seek to a location in a file which has not yet been downloaded, player would download full file sequentially to that location and then only it would be able to play that section. This method could be used to stream only stored content and not for live streaming. Pseudo-streaming Pseudo streaming provides solution for a very important streaming feature i.e., SEEK, that is missing in case of progressive download. It provides the ability to seek in timeline to positions of a file that have not yet been downloaded. 10
Figure 2 Pseudo-streaming Figure 2 shows pseudo streaming mechanism. In this mechanism when the HTTP download of the video is started, the player reads the initial metadata and stores the information about offsets in the video where i-frames (keyframes) in the video start. These offsets basically provide seek-points in the video. If a user seeks to a particular location in timeline, the player locates the nearest key-frame offset and sends a request to web-server with this offset. Server responds with contents of the file starting from this offset. As this content starts from a key-frame, the player is able to play the content. This mechanism is also typically used for on-demand streaming only. 11
Adaptive Bitrate Streaming Adaptive Bitrate (ABR) streaming is a mechanism based on HTTP. In this method several instances with varying properties (e.g. bitrate, resolution) of a source file are created and provided to clients depending upon their available bandwidth and processing power. The clients monitor the available bandwidth and can change the stream to another instance of source file (with a different property like lower bitrate) to improve playback experience. A manifest file is used to identify multiple instances of a stream and their URLs. Figure 3 Adaptive Streaming 12
There are multiple variations of adaptive bitrate streaming by different organizations. MPEG-DASH Dynamic Adaptive Streaming over HTTP (DASH) is an adaptive streaming technology developed under MPEG and is also known as MPEG-DASH. It is published as an international standard ISO/IEC 23009-1:2012. In this mechanism a multimedia file is segmented into one or more segments known as Media Presentation. The description regarding the media presentation (URL, media properties like bitrates) is given in file called Media Presentation Description. It is an XML file that identifies various content alternatives and URLs. When a DASH plays a content it switches from one segment choice to another based on playback conditions like network bandwidth and processing capability. DASH is audio/video codec agnostic. DASH supports ISO Base Media File Format and MPEG-2 Transport streams as containers. It supports both on-demand and live video applications and services. It also provides features like encryption, DRM support and Ad-insertion. There are multiple open source implementations available: libdash DASH client library, multimedia framework GPAC provides mp4box for segmentation. HTTP Live Streaming HTTP Live Streaming (HLS) is an adaptive streaming communications protocol created by Apple to provide streaming to ios and other Apple devices. HTTP Live Streaming specification is an IETF Internet-Draft. HLS supports both live broadcasts and video on demand. It supports multiple alternate streams at different bit rates. Client player can switch between alternate streams based on network bandwidth. In HLS content is sent as segments of small files of duration typically about 10 seconds. There is a playlist, called index file that gives clients URLs of the segment files. In case of 13
live broadcasts the playlist file could be refreshed periodically to accommodate media segment files which are being generated constantly. HLS supports encryption as well as an optional feature. Microsoft Smooth Streaming Smooth Streaming is an IIS (a web server) Media Services extension that enables adaptive streaming of media to clients over HTTP. It dynamically detects local bandwidth and CPU conditions and switches the video quality of stream received by a client player. Basically, consumers with high bandwidth could experience HD quality streams while those with less bandwidth would receive lower quality streams. Smooth Streaming uses the MPEG-4 Part 14 (ISO/IEC 14496-12) file format as its disk (storage) and wire (transport) format. Its specification defines each chunk/gop as an MPEG-4 Movie Fragment, which is stored within a contiguous MP4 file. It allows for easy random access. One MP4 file is expected for each bit rate. When a client requests a particular time segment from a content file from the IIS Web server, the server dynamically finds the appropriate Movie Fragment box within the contiguous MP4 file and sends it over as a standalone file. This allows full caching of the content downstream. In other words, with Smooth Streaming, file chunks are created virtually upon client request, but the actual video is stored on disk as a single full-length file per encoded bit rate. This offers tremendous file-management benefits. 14
RTSP requires dedicated streaming server which has associated software costs. RTSP has issues with NAT traversal and also with firewall traversal. There are bandwidth waste issues as well with HTTP based streaming. Streaming Challenges All streaming technologies have some challenges as well. An RTSP server needs to maintain state by default in almost all cases. It requires dedicated streaming server which has associated software costs. If network congestion occurs, some packets of RSTP stream would be lost as it uses RTP for carrying media data, which is based on UDP. RTP is designed to carry real time payloads like live telecasts or voice calls, wherein it would not matter to lose few frames of video or voice data than to wait for retransmits. RTSP has issues with NAT traversal and also with firewall traversal. On the other hand HTTP delivery based streaming methods have their own set of shortcomings. HTTP progressive downloaded file playback although behaves like a file streamed using a streaming protocol like RTSP/RTP, there are notable differences between the two. In case of streaming (like RTSP) user could seek to any point in timeline of a video and begin to playback after few seconds of buffering. While progressive download based playback require file to be downloaded sequentially, which means user may only seek to a portion of video that has been downloaded and not beyond that. There are bandwidth waste issues as well with HTTP based streaming. HTTP content is delivered as fast as available bandwidth allows. If a user watching a video is having very high bandwidth data connection, then entire video would download quickly and if user stops watching after some time, then most of the downloaded content would be wasted. Also, HTTP based streaming supports unicast only and do not allow multicasting. 15
In live streaming prime requirement is to deliver audiovideo in real time. VoD require more reliability in terms of data transfer than low latency. DASH-HEVC combines the streaming advantages of MPEG-DASH with a high efficiency codec. Solutions Each streaming technology has some issues to become a common fit for all situations. So we have to view streaming technologies in terms of applications and solutions we are designing. Primarily we could categorize streaming solutions into live-video and video on demand. Let s first consider a live streaming solution. In live streaming the prime requirement is to deliver audio-video in real time. RTSP RTP combination fulfills this requirement very well. RTP which is usually based over UDP is well suited for streaming environment where there is some tolerance for packet loss/errors and it has low latency and less network overheads. These are the reasons that RTSP/RTP is highly used in surveillance products and solutions like in IP cameras, where a live feed is required. Also, as UDP is used as the transport, we could do multicasting or broadcasting of feeds which will save on bandwidth. RTSP/RTP suffers from issue during NAT traversal and firewall traversal, however there are ways to overcome these using RTSP/RTP aware application layer gateways, or tunneling RTSP/RTP through HTTP itself. Now coming to video on demand kind of solutions, they require more reliability in terms of data transfer than low latency. A user for example watching a movie on demand would prefer waiting for few seconds for buffering than to lose an important scene of movie. HTTP based delivery gives that reliability factor as it uses TCP for transport, among other advantages, that makes it obvious choice for Video on Demand (VOD) solutions. As is proved by the fact the world s largest VOD provider YouTube uses HTTP based streaming mechanisms. In case of HTTP-based streaming we need simple HTTP web server software, whose license and maintenance cost is very less, compared to a costly media server. HTTP-based streaming does not require any other firewall ports to be opened than the normal ports used by web browsers. 16
HTTP based streaming utilizes HTTP caching mechanisms on the web. Using caching mechanisms more data could be served from web-based caching servers instead of content originating servers, this decreases bandwidth cost for delivering the multimedia content. Since cached data is closer to the user, it improves quality of service. Among various HTTP streaming mechanisms, adaptive bitrate streaming methods provide users with the best viewing experience. As the client player automatically switches to an appropriate streaming content instance among many, based on any changes in user's network and playback conditions. All the HTTP adaptive streaming technologies are based on multiple instances of multimedia content/ files with different properties and a manifest file that defines streaming choices and their URLS. However, all are incompatible with each other. MPEG-DASH provides a standardized way to adaptive streaming reducing number of formats to be supported by moving away from multiple proprietary ABR solutions like HLS, Smooth streaming. It is very appealing for online video scenarios. It supports both the on-demand, and live applications and services. The leading streaming companies have got together to form an industry forum, DASH Industry Forum (DASH-IF) which is working towards refining interoperability of MPEG-DASH by providing profiles such as DASH-AVC/H264. Another concern in general is regarding the quality of streaming. Quality of streaming depends on the available bandwidth. There are two ways to approach this problem, one is to increase network bandwidth, and the other is to improve codec technologies. Network bandwidth is getting improved every day at its own pace. However, there is a significant improvement in codec technology with introduction of HEVC which provides 40-50% improvement over H.264. 17
DASH-IF has released guidelines for DASH-HEVC for review which combines the streaming advantages of MPEG-DASH with a high efficiency codec (H.265) to enable excellent quality of streaming experience. Conclusion In different applications and scenarios different streaming technologies could make sense. RTSP/RTP could provide better viewing experience in case of telecast of live ongoing event or video surveillance although with certain limitations like firewall traversal. HTTP based adaptive streaming could be a more economical option in terms of bandwidth and software used for video on demand as well as live applications in case of online video space. Among various adaptive streaming technologies, MPEG-DASH with lots of upcoming industry support and advancements such as DASH- HEVC, comes out as a very strong contender, for serving both live and on-demand video applications particularly in the online video universe in the future. 18
References 1. RTSP RFC 2326 http://www.ietf.org/rfc/rfc2326.txt 2. MPEG-DASH overview http://www.streamingmedia.com/articles/editorial/ What-Is-.../What-is-MPEG-DASH-79041.aspx 3. HTTP streaming http://www.streamingmedia.com/articles/readarticl e.aspx?articleid=65749&pagenum=2 4. Streaming media http://en.wikipedia.org/wiki/streaming_media 5. Multimedia streaming protocols http://www.streamingmedia.com/articles/editorial/ What-Is-.../What-Is-a-Streaming-Media-Protocol- 84496.aspx 6. HTTP Live streaming overview https://developer.apple.com/library/ios/documentati on/networkinginternet/conceptual/streamingmediag uide/introduction/introduction.html 7. Mp4box GPAC http://gpac.wp.mines-telecom.fr/mp4box/dash/ 8. DASH-IF http://dashif.org/ 9. Smooth Streaming http://www.iis.net/learn/media/on-demand-smoothstreaming/smooth-streaming-technical-overview 19
Author Info Aashish Kaushik works with the HCL ERS Practice team as a Senior Technical Lead. He has 10+ years of experience in software industry in the design and development of VoIP and multimedia streaming solutions. Currently he is involved in building HCL solutions in the Telecom and Video domains. 20
Hello, I'm from HCL's Engineering and R&D Services. We enable technology led organizations to go to market with innovative products and solutions. We partner with our customers in building world class products and creating associated solution delivery ecosystems to help bring market leadership. We develop engineering products, solutions and platforms across Aerospace and Defense, Automotive, Consumer Electronics, Software, Online, Industrial Manufacturing, Medical Devices, Networking & Telecom, Office Automation, Semiconductor and Servers & Storage for our customers. For more details contact: ers.info@hcl.com Follow us on twitter: http://twitter.com/hclers Our blog: http://www.hcltech.com/blogs/engineering-and-rd-services Visit our website: http://www.hcltech.com/engineering-rd-services