Digital Audio. Audio Examples. Multimedia and Quality of Service. Introduction to Multimedia Networking amd QoS. QoS

Transcription

1 Introducton to Multmeda Networkng amd QoS Classfy multmeda applcatons Identfy the network servces the apps need Makng the best of best effort servce Streamng Stored Multmeda vs. Interactve Applcatons Adaptve Playback and Smart Error Recovery Algorthms Some Common Protocols: RTSP, RTP/RTCP, SIP QoS Interserv Dffserv RSVP Multmeda and Qualty of Servce QoS network provdes applcaton wth level of performance needed for applcaton to functon. Multmeda applcatons: network audo and vdeo ( contnuous meda ) 1 2 Dgtal Audo Samplng the analog sgnal Sample at some fxed rate Each sample s an arbtrary real number Quantzng each sample Round each sample to one of a fnte number of values Represent each sample n a fxed number of bts 4 bt representaton (values 0-15) Speech Audo Examples Samplng rate: 8000 samples/second Sample sze: 8 bts per sample Rate: 64 kbps Compact Dsc (CD) Samplng rate: 44,100 samples/second Sample sze: 16 bts per sample Rate: kbps for mono, Mbps for stereo 3 4

2 Why Audo Compresson Audo data requres too much bandwdth Speech: 64 kbps s too hgh for a dal-up modem user Stereo musc: Mbps exceeds most access rates Compresson to reduce the sze Remove redundancy Remove detals that human tend not to perceve Example audo formats Speech: GSM (13 kbps), G.729 (8 kbps), and G (6.4 and 5.3 kbps) Stereo musc: MPEG 1 layer 3 (MP3) at 96 kbps, 128 kbps, and 160 kbps A few words about audo compresson Analog sgnal sampled at constant rate telephone: 8,000 samples/sec CD musc: 44,100 samples/sec Each sample quantzed,.e., rounded e.g., 2 8 =256 possble quantzed values Each quantzed value represented by bts 8 bts for 256 values Example: 8,000 samples/sec, 256 quantzed values --> 64,000 bps Recever converts t back to analog sgnal: some qualty reducton Example rates CD: Mbps MP3: 96, 128, 160 kbps Internet telephony: kbps 5 6 Dgtal Vdeo Samplng the analog sgnal Sample at some fxed rate (e.g., 24 or 30 tmes per sec) Each sample s an mage Quantzng each sample Representng an mage as an array of pcture elements Each pxel s a mxture of colors (red, green, and blue) E.g., 24 bts, wth 8 bts per color The 2272 x 1704 hand The 320 x 240 hand 7 CSc5221: Multmeda 8

3 A few words about vdeo compresson Vdeo s sequence of mages dsplayed at constant rate e.g. 24 mages/sec Examples: MPEG 1 (CD-ROM) 1.5 Mbps MPEG2 (DVD) 3-6 Mbps Dgtal mage s array of MPEG4 (often used n pxels Internet, < 1 Mbps) Each pxel represented Research: by bts Layered (scalable) vdeo Redundancy adapt layers to avalable spatal bandwdth temporal Vdeo Compresson: Wthn an Image Image compresson Explot spatal redundancy (e.g., regons of same color) Explot aspects humans tend not to notce Common mage compresson formats Jont Pctures Expert Group (JPEG) Graphcal Interchange Format (GIF) Uncompressed: 167 KB Good qualty: 46 KB Poor qualty: 9 KB 9 10 Vdeo Compresson: Across Images Compresson across mages Explot temporal redundancy across mages Common vdeo compresson formats MPEG 1: CD-ROM qualty vdeo (1.5 Mbps) MPEG 2: hgh-qualty DVD vdeo (3-6 Mbps) Propretary protocols lke QuckTme and RealNetworks MM Networkng Applcatons Classes of MM applcatons: 1) Streamng stored audo and vdeo 2) Streamng lve audo and vdeo 3) Real-tme nteractve audo and vdeo Jtter s the varablty of packet delays wthn the same packet stream Fundamental characterstcs: Typcally delay senstve end-to-end delay delay jtter But loss tolerant: nfrequent losses cause mnor gltches Antthess of data, whch are loss ntolerant but delay tolerant

4 Applcaton Classes Applcaton Classes (more) Streamng Clents request audo/vdeo fles from servers and ppelne recepton over the network and dsplay Interactve: user can control operaton (smlar to VCR: pause, resume, fast forward, rewnd, etc.) Delay: from clent request untl dsplay start can be 1 to 10 seconds Undrectonal Real-Tme: smlar to exstng TV and rado statons, but delvery on the network Non-nteractve, just lsten/vew Interactve Real-Tme : Phone conversaton or vdeo conference More strngent delay requrement than Streamng and Undrectonal because of real-tme nature Vdeo: < 150 msec acceptable Audo: < 150 msec good, <400 msec acceptable Streamng Stored Multmeda Streamng Stored Multmeda: What s t? Streamng: meda stored at source transmtted to clent streamng: clent playout begns before all data has arrved tmng constrant for stll-to-be transmtted data: n tme for playout 1. vdeo recorded 2. vdeo sent network delay 3. vdeo receved, played out at clent streamng: at ths tme, clent playng out early part of vdeo, whle server stll sendng later part of vdeo tme 15 16

5 Streamng Stored Multmeda: Interactvty VCR-lke functonalty: clent can pause, rewnd, FF, push slder bar 10 sec ntal delay OK 1-2 sec untl command effect OK RTSP often used (more later) tmng constrant for stll-to-be transmtted data: n tme for playout Streamng Lve Multmeda Examples: Internet rado talk show Lve sportng event Streamng playback buffer playback can lag tens of seconds after transmsson stll have tmng constrant Interactvty fast forward mpossble rewnd, pause possble! Interactve, Real-Tme Multmeda applcatons: IP telephony, vdeo conference, dstrbuted nteractve worlds end-end delay requrements: audo: < 150 msec good, < 400 msec OK ncludes applcaton-level (packetzaton) and network delays hgher delays notceable, mpar nteractvty sesson ntalzaton how does callee advertse ts IP address, port number, encodng algorthms? Multmeda Over Today s Internet TCP/UDP/IP: best-effort servce no guarantees on delay, loss?????? But you sad multmeda apps requres QoS and level of performance to be?? effectve!?? Today s Internet multmeda applcatons use applcaton-level technques to mtgate (as best possble) effects of delay, loss? 19 20

6 Challenges TCP/UDP/IP sute provdes best-effort, no guarantees on expectaton or varance of packet delay Streamng applcatons delay of 5 to 10 seconds s typcal and has been acceptable, but performance deterorate f lnks are congested (transoceanc) Real-Tme Interactve requrements on delay and ts jtter have been satsfed by over-provsonng (provdng plenty of bandwdth), what wll happen when the load ncreases?... Challenges (more) Most router mplementatons use only Frst-Come-Frst-Serve (FCFS) packet processng and transmsson schedulng To mtgate mpact of best-effort protocols, we can: Use UDP to avod TCP and ts slow-start phase Buffer content at clent and control playback to remedy jtter Adapt compresson level to avalable bandwdth How should the Internet evolve to better support multmeda? Integrated servces phlosophy: Fundamental changes n Internet so that apps can reserve end-to-end bandwdth Requres new, complex software n hosts & routers Lassez-fare no major changes more bandwdth when needed content dstrbuton, applcaton-layer multcast applcaton layer Dfferentated servces phlosophy: Fewer changes to Internet nfrastructure, yet provde 1st and 2nd class servce. Wll dscuss QoS later! 23 Soluton Approaches n IP Networks Just add more bandwdth and enhance cachng capabltes (over-provsonng)! Need major change of the protocols : Incorporate resource reservaton (bandwdth, processng, bufferng), and new schedulng polces Set up servce level agreements wth applcatons, montor and enforce the agreements, charge accordngly Need moderate changes ( Dfferentated Servces ): Use two traffc classes for all packets and dfferentate servce accordngly Charge based on class of packets Network capacty s provded to ensure frst class packets ncur no sgnfcant delay at routers 24

7 Streamng Stored Multmeda Internet Multmeda: Smplest Approach Applcaton-level streamng technques for makng the best out of best effort servce: clent sde bufferng use of UDP versus TCP multple encodngs of multmeda Meda Player jtter removal decompresson error concealment graphcal user nterface w/ controls for nteractvty audo, vdeo not streamed: audo or vdeo stored n fle fles transferred as HTTP object receved n entrety at clent then passed to player no, ppelnng, long delays untl playout! Internet multmeda: Streamng Approach Streamng from a streamng server browser GETs metafle browser launches player, passng metafle player contacts server server streams audo/vdeo to player 27 Ths archtecture allows for non-http protocol between server and meda player Can also use UDP nstead of TCP. 28

8 User Control of Streamng Meda: RTSP HTTP Does not target multmeda content No commands for fast forward, etc. RTSP: RFC 2326 Clent-server applcaton layer protocol. For user to control dsplay: rewnd, fast forward, pause, resume, repostonng, etc What t doesn t do: does not defne how audo/vdeo s encapsulated for streamng over network does not restrct how streamed meda s transported; t can be transported over UDP or TCP does not specfy how the meda player buffers audo/vdeo RTSP: out of band control FTP uses an out-ofband control channel: A fle s transferred over one TCP connecton. Control nformaton (drectory changes, fle deleton, fle renamng, etc.) s sent over a separate TCP connecton. The out-of-band and nband channels use dfferent port numbers. RTSP messages are also sent out-of-band: RTSP control messages use dfferent port numbers than the meda stream: out-of-band. Port 554 The meda stream s consdered n-band RTSP Example Scenaro: metafle communcated to web browser browser launches player player sets up an RTSP control connecton, data connecton to streamng server 31 Metafle Example <ttle>twster</ttle> <sesson> <group language=en lpsync> <swtch> <track type=audo e="pcmu/8000/1" src = "rtsp://audo.example.com/twster/audo.en/lof"> <track type=audo e="dvi4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audo.example.com/twster/audo.en/hf"> </swtch> <track type="vdeo/jpeg" src="rtsp://vdeo.example.com/twster/vdeo"> </group> </sesson> 32

9 RTSP Operaton RTSP Exchange Example C: SETUP rtsp://audo.example.com/twster/audo RTSP/1.0 Transport: rtp/udp; compresson; port=3056; mode=play S: RTSP/ OK Sesson 4231 C: PLAY rtsp://audo.example.com/twster/audo.en/lof RTSP/1.0 Sesson: 4231 Range: npt=0- C: PAUSE rtsp://audo.example.com/twster/audo.en/lof RTSP/1.0 Sesson: 4231 Range: npt=37 C: TEARDOWN rtsp://audo.example.com/twster/audo.en/lof RTSP/1.0 Sesson: 4231 S: OK Streamng Multmeda: Clent Bufferng Streamng Multmeda: Clent Bufferng constant bt rate vdeo transmsson varable network delay clent playout delay clent vdeo recepton buffe ered vde eo constant bt rate vdeo playout at clent Clent-sde bufferng, playout delay compensate for network-added delay, delay jtter tme varable fll rate, x(t) buffered vdeo constant dran rate, d Clent-sde bufferng, playout delay compensate for network-added delay, delay jtter 35 36

10 Streamng Multmeda: UDP or TCP? UDP server sends at rate approprate for clent (oblvous to network congeston!) often send rate = encodng rate = constant rate then, fll rate = constant rate - packet loss short playout delay (2-5 seconds) to compensate for network delay jtter error recover: tme permttng TCP send at maxmum possble rate under TCP fll rate fluctuates due to TCP congeston control larger playout delay: smooth TCP delvery rate HTTP/TCP passes more easly through frewalls Streamng Multmeda: clent rate(s) 1.5 Mbps encodng 28.8 Kbps encodng Q: how to handle dfferent clent receve rate capabltes? 28.8 Kbps dalup 100Mbps Ethernet A: server stores, transmts multple copes of vdeo, encoded at dfferent rates Real-tme nteractve applcatons PC-2-PC phone nstant messagng servces are provdng ths, e.g. Yahoo messenger, googletalk Skype PC-2-phone Dalpad Net2phone Skype vdeoconference wth Webcams Gong to now look at a PC-2-PC Internet phone example n detal 39 Voce Over IP (VoIP) Delverng phone calls over IP Computer to computer Analog phone to/from computer Analog phone to analog phone Motvatons for VoIP Cost reducton Smplcty Advanced applcatons Web-enabled call centers Collaboratve whte boardng Do Not Dsturb, Locate Me, etc. Vocemal sent as e-mal 40

11 Tradtonal Telecom Infrastructure Corporate/Campus Prvate Branch Exchange 7043 External lne Telephone swtch Another swtch Corporate/Campus 7043 PBX VoIP Gateway VoIP Gateways External lne Another campus 8151 PBX VoIP Gateway Corporate/Campus LAN Internet LAN Internet LAN IP Phone Clent Skype Nklas Zennström and Janus Frs n 2003 Developed by KaZaA Instant Messenger (IM) wth voce support Based on peer-topeer (P2P) networkng technology Skype Network Archtecture Logn server s the only central server (consstng of multple machnes) Both ordnary host and super nodes are Skype clents Any node wth a publc IP address and havng suffcent resources can become a super node Skype mantans ther own super nodes 43 44

12 Challenges of Frewalls and NATs Frewalls Often block UDP traffc Usually allow hosts to ntate connectons on port 80 (HTTP) and 443 (HTTPS) NAT Cannot easly ntate traffc to a host behnd a NAT snce there s no unque address for the host Skype must deal wth these problems Dscovery: clent exchanges messages wth super node Traversal: sendng data through an ntermedate peer Interactve Multmeda: Internet Phone Introduce Internet Phone by way of an example speaker s audo: alternatng talk spurts, slent perods. 64 kbps durng talk spurt pkts generated only durng talk spurts 20 msec chunks at 8 Kbytes/sec: 160 bytes data applcaton-layer header added to each chunk. Chunk+header encapsulated nto UDP segment. applcaton sends UDP segment nto socket every 20 msec durng talkspurt Internet Phone: Packet Loss and Delay network loss: IP datagram lost due to network congeston (router buffer overflow) delay loss: IP datagram arrves too late for playout at recever delays: processng, queueng n network; end-system (sender, recever) delays typcal maxmum tolerable delay: 400 ms loss tolerance: dependng on voce encodng, losses concealed, packet loss rates between 1% and 10% can be tolerated. Delay Jtter constant bt rate transmsson varable network delay (jtter) clent playout delay clent recepton buffe ered dat ta constant bt rate playout at clent Consder the end-to-end delays of two consecutve packets: dfference can be more or less than 20 msec tme 47 48

13 Internet Phone: Fxed Playout Delay Recever attempts to playout each chunk exactly q msecs after chunk was generated. chunk has tme stamp t: play out chunk at t+q. chunk arrves after t+q: data arrves too late for playout, data lost Tradeoff for q: large q: less packet loss small q: better nteractve experence Fxed Playout Delay Sender generates packets every 20 msec durng talk spurt. Frst packet receved at tme r Frst playout schedule: begns at p Second playout schedule: begns at p packets packets generated packets receved loss playout schedule p - r playout schedule p' - r tme r 49 p p' 50 Adaptve Playout Delay, I Goal: mnmze playout delay, keepng late loss rate low Approach: adaptve playout delay adjustment: Estmate network delay, adjust playout delay at begnnng of each talk spurt. Slent perods compressed and elongated. Chunks stll played out every 20 msec durng talk spurt. t = tmestamp of the th packet r = the tme packet s receved by recever p the tme packet s played at recever r t = network delay for th packet d = estmate of average network delay after recevng th packet = Dynamc estmate of average delay at recever: d = ( 1 u) d 1 + u( r t ) where u s a fxed constant (e.g., u =.01). Adaptve playout delay II Also useful to estmate the average devaton of the delay, v : v = ( 1 u) v 1 + u r t d The estmates d and v are calculated for every receved packet, although they are only used at the begnnng of a talk spurt. For frst packet n talk spurt, playout tme s: p = t + d + Kv where K s a postve constant. Remanng packets n talkspurt are played out perodcally 51 52

14 Adaptve Playout, III Q: How does recever determne whether packet s frst n a talkspurt? If no loss, recever looks at successve tmestamps. dfference of successve stamps > 20 msec -->talk spurt begns. Wth loss possble, recever must look at both tme stamps and sequence numbers. dfference of successve stamps > 20 msec and sequence numbers wthout gaps --> talk spurt begns. Recovery from packet loss (1) forward error correcton (FEC): smple scheme for every group of n chunks create a redundant chunk by exclusve OR-ng the n orgnal chunks send out n+1 chunks, ncreasng the bandwdth by factor 1/n. can reconstruct the orgnal n chunks f there s at most one lost chunk from the n+1 chunks Playout delay needs to be fxed to the tme to receve all n+1 packets Tradeoff: ncrease n, less bandwdth waste ncrease n, longer playout delay ncrease n, hgher probablty that 2 or more chunks wll be lost Recovery from packet loss (2) Recovery from packet loss (3) 2nd FEC scheme pggyback lower qualty stream send lower resoluton audo stream as the redundant nformaton for example, nomnal stream PCM at 64 kbps and redundant stream GSM at 13 kbps. Whenever there s non-consecutve loss, the recever can conceal the loss. Can also append (n-1)st and (n-2)nd low-bt rate chunk Interleavng chunks are broken up nto smaller unts for example, 4 5 msec unts per chunk Packet contans small unts from dfferent chunks f packet s lost, stll have most of every chunk has no redundancy overhead but adds to playout delay 55 56

15 Summary: Internet Multmeda: bag of trcks use UDP to avod TCP congeston control (delays) for tme-senstve traffc clent-sde adaptve playout delay: to compensate for delay server sde matches stream bandwdth to avalable clent-to-server path bandwdth chose among pre-encoded stream rates dynamc server encodng rate error recovery (on top of UDP) FEC, nterleavng retransmssons, tme permttng conceal errors: repeat nearby data 57