Tema 0: Transmisión de Datos Multimedia Clases de aplicaciones multimedia Redes basadas en IP y QoS Computer Networking: A Top Down Approach Featuring the Internet, 3 rd edition. Jim Kurose, Keith Ross Addison-Wesley, July 2004. Transmisión de Datos Multimedia http://www.grc.upv.es/docencia/tdm Master IC 2007/2008
Definition of multimedia Hard to find a clear-cut definition What is multimedia? In general, multimedia is an integration of text, graphics, still and moving images, animation, sounds, and any other medium where every type of information can be represented, stored, transmitted and processed digitally Characteristics of multimedia Digital key concept Integration of multiple media type, usually including video or/and audio May be interactive or non-interactive 2
Text, Graphics, image, video, animation, sound, etc. Classifications of various media types Captured vs. synthesized media Various Media Types Captured media (natural) : information captured from the real world Example: still image, video, audio Synthesized media (artificial) : information synthesize by the computer Example: text, graphics, animation Discrete vs. continuous media Discrete media: space-based, media involve the space dimension only Text, Image, Graphics Continuous media: time-based, media involves both the space and the time dimension Video, Sound, Animation 3
Sound Image Continuous Video Classification of Media Type Animation Text Continuous Graphics Discrete Captured From real world Discrete Synthesized By computer 4
Plain text Unformatted Characters coded in binary form ASCII code All characters have the same style and font Rich text Formatted Contains format information besides codes for characters No predominant standards Characters of various size, shape and style, e.g. bold, colorful Text 5
6 Plain Text vs. Rich Text An example of Plain text Example of Rich text
Revisable document that retains structural information Consists of objects such as lines, curves, circles, etc Usually generated by graphic editor of computer programs 10 5 Graphics Example of graphics (FIG file) 0-5 -10 4 2 0-2 -4-4 -2 0 2 4 7
2D matrix consisting of pixels Pixel smallest element of resolution of the image One pixel is represented by a number of bits Pixel depth the number of bits available to code the pixel Have no structural information Two categories: scanned vs. synthesized still image Computer software Digital still image Synthesized image Images Camera Capture and A/D conversion Scanned image 8
Examples of images Binary image pixel depth 1 Gray-scale pixel depth 8 Color image pixel depth 24 Binary image Images (cont.) Gray-scale color image 9
Video vs. Animation Both images and graphics can be displayed as a succession of view which create an impression of movement Video moving images or moving pictures Captured or Synthesized Consists of a series of bitmap images Each image is called a frame Frame rate: the speed to playback the video (frame per second) Animation moving graphics Generated by computer program (animation authoring tools) Consists of a set of objects The movements of the objects are calculated and the view is updated at playback 1 0
1-D time-based signal 0. 2 0. 1 5 0. 1 0. 0 5 0-0. 0 5-0. 1-0. 1 5-0. 2 0 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 1 0 0 0 Speech vs. non-speech sound Speech supports spoken language and has a semantic content Non-speech does not convey semantics in general Natural vs. structured sound Natural sound Recorded/generated sound wave represented as digital signal Example: Audio in CD, WAV files Structured sound Synthesize sound in a symbolic way Example: MIDI file Sound 1 1
Local vs. networked multimedia Networked Multimedia Local: storage and presentation of multimedia information in standalone computers Sample applications: DVD Networked: involve transmission and distribution of multimedia information on the network Sample applications: videoconferencing, web video broadcasting, multimedia Email, etc. A scenario of multimedia networking Image server Video server Internet 1 2
Consideration of Networked Multimedia Requirements of multimedia applications on the network Typically delay sensitive end-to-end delay delay jitter: Jitter is the variability of packet delays within the same packet stream Quality requirement Satisfactory quality of media presentation Synchronization requirement Continuous requirement (no jerky video/audio) Can tolerant some degree of information loss 1 3
Technologies of Multimedia Networking Challenges of multimedia networking 1. Conflict between media size and bandwidth limit of the network 2. Conflict between the user requirement of multimedia application and the best-effort network 3. How to meet different requirements of different users? Media compression reduce the data volume Address the 1st challenge Image compression Video compression Audio compression Multimedia transmission technology Address the 2nd and 3rd challenges Protocols for real-time transmission Rate / congestion control Error control 1 4
Live media transmission system Multimedia Networking Systems Capture, compress, and transmit the media on the fly (example?) Send stored media across the network Media is pre-compressed and stored at the server. This system delivers the stored media to one or multiple receivers. (example?) Differences between the two systems For live media delivery: Real-time media capture, need hardware support Real-time compression speed is important Compression procedure can be adjusted based on network conditions For stored media delivery Offline compression better compression result is important Compression can not be adjusted during transmission 1 5
Streaming stored audio and video Streaming live audio and video Real-time interactive audio and video Classes of multimedia applications 1 6
Cumulative data 100% 1. video recorded t>0 2. video sent Streaming Stored Multimedia: What is it? network delay 3. video received, played out at client time streaming: at this time, client playing out early part of video, while server still sending later part of video 1 7
Streaming vs. Download of Stored Multimedia Content Download: Receive entire content before playback begins High start-up delay as media file can be large ~ 4GB for a 2 hour MPEG II movie Streaming: Play the media file while it is being received Reasonable start-up delays Reception Rate >= playback rate. Why? 1 8
Streaming Stored Multimedia: Interactivity VCR-like functionality: client can pause, rewind, FF, push slider bar 10 sec initial delay OK 1-2 sec until command effect OK RTSP often used (more later) timing constraint for still-to-be transmitted data: in time for playout 1 9
Cumulative data constant bit rate video transmission Streaming Multimedia: Client Buffering variable network delay client video reception buffered video constant bit rate video playout at client client playout delay time Client-side buffering, playout delay compensate for networkadded delay, delay jitter 2 0
Streaming Multimedia: Client Buffering Client-side buffering, playout delay compensate for network-added delay, delay jitter variable fill rate, x(t) constant drain rate, d buffered video 2 1
Interactive, Real-Time Multimedia applications: IP telephony, video conference, distributed interactive worlds end-end delay requirements: audio: < 150 msec good, < 400 msec OK includes application-level (packetization) and network delays higher delays noticeable, impair interactivity session initialization how does callee advertise its IP address, port number, encoding algorithms? 2 2
Internet multimedia: simplest approach audio or video stored in file files transferred as HTTP object received in entirety at client then passed to player audio, video not streamed: no, pipelining, long delays until playout! 2 3
Progressive Download browser GETs metafile browser launches player, passing metafile player contacts server server downloads audio/video to player 2 4
Streaming from a streaming server This architecture allows for non-http protocol between server and media player Can also use UDP instead of TCP. 2 5
TCP/UDP/IP: best-effort service no guarantees on delay, loss Multimedia Over Today s Internet But multimedia apps requires QoS and level of performance to be effective! Today s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss 2 6
UDP Streaming Multimedia: UDP or TCP? server sends at rate appropriate for client (oblivious to network congestion!) often send rate = encoding rate = constant rate then, fill rate = constant rate - packet loss short playout delay (2-5 seconds) to compensate for network delay jitter error recover: time permitting TCP send at maximum possible rate under TCP fill rate fluctuates due to TCP congestion control larger playout delay: smooth TCP delivery rate HTTP/TCP passes more easily through firewalls 2 7
Multimedia, Quality of Service: What is it? Multimedia applications: network audio and video ( continuous media ) QoS network provides application with level of performance needed for application to function. 2 8
Improving QOS in IP Networks Thus far: making the best of best effort Future: next generation Internet with QoS guarantees RSVP: signaling for resource reservations Differentiated Services: differential guarantees Integrated Services: firm guarantees simple model for sharing and congestion studies: 2 9
Principles for QOS Guarantees Example: 1Mbps IPphone, FTP share 1.5 Mbps link. bursts of FTP can congest router, cause audio loss want to give priority to audio over FTP 3 0 Principle 1 packet marking needed for router to distinguish between different classes; and new router policy to treat packets accordingly
Principles for QOS Guarantees (more) what if applications misbehave (audio sends higher than declared rate) policing: force source adherence to bandwidth allocations marking and policing at network edge: similar to ATM UNI (User Network Interface) Principle 2 3 1 provide protection (isolation) for one class from others
Principles for QOS Guarantees (more) Allocating fixed (non-sharable) bandwidth to flow: inefficient use of bandwidth if flows doesn t use its allocation Principle 3 3 2 While providing isolation, it is desirable to use resources as efficiently as possible
Principles for QOS Guarantees (more) Basic fact of life: can not support traffic demands beyond link capacity 3 3 Principle 4 Call Admission: flow declares its needs, network may block call (e.g., busy signal) if it cannot meet needs
3 4 Summary of QoS Principles