Video Streaming and Media Session Protocols 1
Streaming Stored Multimedia Stored media streaming File containing digitized audio / video Stored at source Transmitted to client Streaming Client playout begins before all data arrives Real-time constraint: data must arrive in time for playout 2
Quality of Service (QoS) Network parameters Data rate Error control Physical transmission speed in bps Bit error rate (BER) < 10-9 on fiber optic cable < 10-3 on wireless Packet loss rate Depends on congestion control policy End-to-end transmission time for one bit Delay High data rate lower transmission delay T trans Congestion / priority longer queuing delay T Q Datagram service longer processing delay T proc Dynamic variations in Delay variation Data rate / propagation delay (jitter) ΔT trans Queuing delay ΔT Q Processing delay ΔT proc 3
QoS and Network Policies Error Control Data Rate Transmission Speed Congestion Control Priority Control Access Delay Connection Type Delay Variation Delay Time 4
Streaming Types Live Multimedia Live event Interactivity Fast forward impossible Rewind / pause possible Stored Multimedia Recorded event Interactivity Pause / rewind / FF / relocate (move slider bar) Real-Time Interactive Multimedia IP telephony / video conference / multiplayer games Interactivity Conversation in real time Common features Device input from live input input from file Data must arrive in time to permit continuous play 5
Timing is Everything Streaming Client playing out early part of video Server sending later part of video Playback buffer Store data in client Playback (delay) tens of seconds Cumulative data network delay time Video transmitted by server Video received + played out at client 6
Streaming Multimedia: Client Buffering Best effort in TCP/UDP/IP No QoS guarantee transmission rate / error rate / delay No jitter control Jitter Variation in transmission delay Cumulative data constant bit rate video transmission client playout delay variable network delay client video reception constant bit rate video playout at client time 7
Streaming Multimedia UDP versus TCP UDP Requires separate control channels RTP (Real Time Transport Protocol RFC 1889) QoS (jitter) control / packet ordering RTSP (Real Time Streaming Protocol RFC 2326) Request / pause / resume / rewind / FF / relocate Server transmits at constant requested rate No congestion or flow control Buffer fill rate = constant rate - packet loss Small client playout delay to smooth jitter = 2-5 seconds TCP Enables use of HTTP Passes through firewalls Limited by TCP transmission rate variations TCP congestion control Larger client playout delay to smooth TCP delivery rate 8
RTSP Request Example Metafile <title>twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="pcmu/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="dvi4/16000/2" pt="90 DVI4/8000/1" src= "rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src= "rtsp://video.example.com/twister/video"> </group> </session> 9
RTSP Operation 10
RTSP Exchange Example Client: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=play Server: RTSP/1.0 200 1 OK Session 4231 Client: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0 Client: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 Client: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Server: 200 3 OK 11
HTTP Streaming HTTP over TCP No independent control channel Client issues standard HTTP GET request Used by YouTube and Netflix Server side HTTP server accesses stored video file Transfers over TCP is standard segment chunks Client side Standard TCP buffer feeds larger application buffer Smoothes TCP rate variation Permits prefetch fast download when possible Video File TCP Buffer TCP Buffer Application Buffer Player Server Client 12
Streaming Control Pause play TCP download continues unless Application buffer fills client TCP buffer fills RecvWind closes Reposition HTTP GET byte range header Overrides HTTP transaction Early termination Player cancels HTTP request Server stops HTTP response and clears buffer Efficiency issues Server buffered large volume never sent Waste of server resources Client buffer received many bytes never viewed Waste of Internet capacity 13
Content Distribution Network () Standard HTTP server Single massive data center Many hops from data center to client Higher probability of low throughput network link Popular video transmitted many times Waste of bandwidth and cost of bandwidth Single point of failure Dedicated Servers in wide geographical area Private operated within branches of large organization Google YouTube Outsource distribution to third party Akamai Netflix 14
Placement Enter Deep Bring Home Location of server clusters Near many ISPs Near a few large ISPs Hops to user Fewer More QoS Better Worse OAM Higher costs Lower costs Caching downloads video files from central data center on client demand Fills cache and streams to client simultaneously Clears cache on LRU basis 15
Operation User www.video.com 1 2 3 5 Local DNS 4 6 server video.com DNS DNS 1. User clicks on link to video at web page 2. User DNS query 3. Local DNS server queries video.com authoritative DNS server Query relayed to authoritative DNS server 4. Local DNS server queries authoritative DNS server Replies with IP address of server based on geography 5. Local DNS replies with server IP 6. User downloads video file from selected server 16
Voice over IP (VoIP) Digitized voice Sample analog voice round off samples bit stream Chunk = compressed sample (typically 20 msec) Chunk + header sent over UDP Best effort delivery UDP packet losses uncorrected Variable delay < 150 msec undetectable 150 400 msec tolerable Treating jitter Timestamp chunk in header Constant delay Play at timestamp + fixed delay Prefer longer initial delay between speakers over choppy voice Adaptive delay Lower initial delay when possible 17
VoIP Management Protocols Real Time Transport Protocol (RTP) Runs over UDP Provides limited transport control Packet ordering QoS (jitter) control Session Initiation Protocol (SIP) Call initiation Call management Add new media streams Change encoding during call Invite others Transfer / hold calls Call termination 18
RTP Header Fields Field Bits Function Flags Payload Type Sequence Number Timestamp Synchronization Source Identifier Extensions 9 7 16 32 32 Version, padding, etc. Voice encoding type 0 = PCM mu law, 64 kbps 3 = GSM, 13 kbps 7 = LPC, 2.4 kbps 14 = MPEG Audio 26 = Motion JPEG 31 = H.261 33 = MPEG2 video Sequence number of voice chunk Time of sampling of chunk Identifies voice stream Additional headers (optional) 19
Real Time Control Protocol (RTCP) Works with RTP Provides feedback mechanism Sender modify transmission parameters RTCP packets sent periodically By each participant to each participant Packet contains sender and/or receiver reports Number of packets sent Number of packets lost Interarrival jitter 20
SIP Call Set Up Alice: SIP invite with her port number, IP address, encoding (PCM mu law) mu Bob: 200 (OK) message with his port number, IP address, preferred encoding (GSM) Default SIP port = 5060 Bob and Alice use different voice encodings 21
SIP Negotiations Codec (coder / decoder) negotiation Bob has no PCM mu law encoder Reply to request 606 Not Acceptable Reply Lists available encoders Alice sends new INVITE message Rejecting a call Bob replies Busy Gone Payment required Forbidden 22
Example of SIP Invitation INVITE sip:bob@domain.com SIP/2.0 Via: SIP/2.0/UDP 167.180.112.24 From: sip:alice@hereway.com To: sip:bob@domain.com Call-ID: a2e3a@pigeon.hereway.com Content-Type: application/sdp Content-Length: 885 c=in IP4 167.180.112.24 m=audio 38060 RTP/AVP 0 Notes HTTP message syntax SDP = session description protocol Call-ID unique for every call 23
Name Translation and User Location Caller has callee's name or e-mail address Needs IP address of callee's current host User is mobile User obtains IP by DHCP User on multiple devices (PC, phone, tablet) SIP registrar server Bob starts SIP client Client sends SIP REGISTER message to registrar server REGISTER sip:domain.com SIP/2.0 Via: SIP/2.0/UDP 193.64.210.89 From: sip:bob@domain.com To: sip:bob@domain.com Expires: 3600 SIP proxy server Alice sends invite message to her proxy server Contains address sip:bob@domain.com Proxy routes SIP messages to Bob Bob responds through same proxies Proxy returns SIP response message to Alice with Bob s IP address 24
Proxy Example jim@umass.edu makes call to keith@upenn.edu 1. jim sends INVITE to umass SIP proxy 2. Proxy forwards request to upenn registrar server 3. upenn server returns redirect response: try keith@eurecom.fr 4. umass proxy sends INVITE to eurecom registrar 5. eurecom registrar forwards INVITE to 197.87.54.21 (keith's SIP client) 6. SIP response to eurecom registrar 7. SIP response to umass proxy SIP registrar 8. SIP response to jim upenn.edu 9. jim and keith in direct IP connection SIP proxy umass.edu 2 3 4 SIP registrar eurecom.fr 1 7 5 8 6 SIP client 217.123.56.89 9 SIP client 197.87.54.21 25
Service Classes TCP/IP Best effort service One-size fits all service model Service Classes Mark traffic by service class Network provides QoS on demand Extension of IP ToS bits H1 R1 Service type 1 R2 H3 H2 R1 output interface queue Service type 2 H4 26
Mixed FTP and Audio 1 Mbps IP phone and FTP on 1.5 Mbps link FTP data bursts can congest router audio loss Priority to audio Packet marking Router distinguishes packet different classes Router policy Prioritize packets by class R1 R2 1.5 Mbps link 27
QOS Guarantees Marking and policing at network edge Policing enforcing bandwidth allocations Fixed bandwidth allocation per service 1 Mbps to phone + 0.5 Mbps to FTP = 1.5 Mbps Inefficient for bursty traffic Priority queuing More efficient 1 Mbps phone R1 R2 1.5 Mbps link packet marking and policing 28
Priority Queuing Priority scheduling Send highest priority queued packet Token Bucket Enforces burst size and average rate Bucket hold < b tokens Tokens generated at rate r tokens/sec unless bucket full Over time T Number of packets sent rt + b 29
IETF Differentiated Services (Diffserv) Qualitative service classes Flow = sequence of packets associated with one service stream Profile = pre-negotiated data rate A and bucket size B Per-hop behavior (PHB) = observable performance Expedited Forwarding (EF) = guaranteed minimum rate Assured Forwarding (AF) = guaranteed min rate and max drop rate Edge router Entry router to Diffserv network Marks packets by flow Polices packets In-profile conforming to profile Out-profile not conforming to profile Core router Buffering and scheduling for forwarding Preference to in-profile packets 30
IETF Integrated Services (IntServ) Per session QOS guarantee Resource reservation Routers maintain state info Allocated resources + QoS requirements Resource Reservation Protocol (RSVP RFC 2205) Session set-up Request R-spec define requested QOS T-spec defines traffic characteristics Adequate resources grant new call setup request Else deny new call Guaranteed service Worst case traffic arrival = leaky-bucket-policed source Bound on delay Controlled load service QoS similar to same flow on unloaded network 31