Introduction N. C. State University CSC557 Multimedia Computing and Networking Fall 2001 Lecture # 01
Class Website 2 http://courses.ncsu.edu/csc557/lec/001/
What is "Multimedia"? 3 Example: a Realvideo broadcast
13 1 Audio capture Video capture 2 Compre ssion Method 3 4 5 12 Audio playback Video playback 11 Decompre ssion Method 10 8 7 9 6
1 Camera and microphone convert analog optical and acoustic signals into analog electrical signals 2 3 4 5 6 7 8 9 10 11 12 13 Video capture and audio capture (sound) cards convert into (uncompressed) digital signals with a specific quality level Compression hardware+software compresses the digital representation Compressed digital information is combined into a single file and stored File is transmitted by ftp over a network to a streaming media server and stored. Media server retrieves files on request and delivers them to network. Client machine contacts streaming media server and requests media playback. Server agrees and client+server constantly monitor the quality of transmission and playback. Routers in network provide requested quality of service if possible. Client machine receives streaming data and uncompresses it. Client machine buffers audio + video for playback. Client machine sends digital data to video card and sound card, which convert digital data into analog electrical signals. Camera and microphone convert analog electrical signals back into analog optical and acoustic signals
Some Applications 6 Person-to-person communication Education and training Entertainment Games News Marketing, advertising, sales Assisting the handicapped Remote guidance and exploration Data analysis and presentation
Converging Industries 7 Alliances, mergers, and takeovers Some examples of industries Publishing Recording (music) Education Movies, video, TV Consumer electronics Computer software and hardware Networking equipment vendors Network service providers (telephone, internet, cable TV, broadcast TV, wireless,...)
Multimedia 8 Applications Content creation and storage Access and delivery Hardware Software
Converged Company #1: AOL Time Warner 11 35 billion $ in revenues Businesses Interactive Services and Properties (America Online) TV Networks Publishing (Time Inc.) Music (Warner Music Group) Films (Warner Bros., New Line Cinema) Cable Systems (Time Warner Cable) Statistics 130 million subscriptions (AOL, HBO, Time, TW Cable) 268 million magazine readers CNN reaches global audience of 1 billion AOL has 30 million members ICQ has 85 million registrants
AOL-T-W: Cable Systems 12 Cable TV delivery Digital video and music Video on demand Local cable news channels Roadrunner Internet service
AOL-T-W: Publishing 13 Time Inc. Time Warner Trade Publishing More than 60 magazines Several book publishers Audiobooks Direct marketing of CDs
AOL-T-W: Film 14 Warner Brothers Warner Brothers TV Castle Rock Entertainment Looney Tunes, Hanna-Barbera Warner Home Video Warner Brothers Studio Stores DC Comics
AOL-T-W: Music 15 Atlantic, Elektra, London, Warner Bros. Music distribution CD and DVD manufacturing
AOL-T-W: Networks 16 Turner Broadcasting, TBS, TNT, Cartoon Network, Turner Classic Movies... CNN Atlanta Braves, Atlanta Hawks, Atlanta Thrashers, Goodwill Games HBO, Cinemax, HBO productions WB television network, Kids WB
AOL-T-W: Interactive Services... 17 AOL, Compuserve Netscape.com, Netscape browser AOLTV (interactive TV) AOL Plus (multimedia content: video, audio, games) AOL mobile services (email, messaging) AOLbyphone Web services and info (Digital City, Mapquest, Moviefone) Messaging (AIM, ICQ) Winamp, Spinner (streaming media) Alliance with NTT DoCoMo mobile service
Converged Company #2: Sony 19 58 billion $ in revenue Electronics group 65% of revenues Audio stereos, recorders, personal stereos Video television sets, cameras, VCRs/VTRs, DVD-video players, tapes Computing and communications displays, PCs, PDAs, peripherals, media, receivers, telephones, car navigation Electronic components semiconductors, memory, LCDs, CRTs, batteries One example product: Airboard portable, wireless, convergence of TV and Internet
Sony: Other 35% 20 Games: game consoles and software (Playstation) Music: Columbia, Sony Film and TV: Columbia, Sony Insurance and finance
Course Objectives 21 Learn about Audio, image and video 1. perception 2. compression 3. processing Playback and control of audio and video on computers Transport of audio and video across the Internet
and Non-Objectives 22 But not about The use of commercial tools for creative manipulation of digital media (well, maybe a little bit) The legal / political / commercial aspects of using and protecting digital media Analog devices (cameras, speakers, ) User interfaces Graphics Speech recognition Advanced topics in signal and image processing
What We Study 23 Basic signal processing Common compression techniques Perception and processing of sound Audio and voice compression Music (MIDI) Perception and processing of images Image compression (JPEG) Video standards
( continued) 24 Video compression (MPEG) QoS issues, jitter buffers, playback Packet scheduling, dropping, and shaping in routers RSVP and Integrated Services Differentiated services (DiffServ) RTP, RTCP, RTSP SIP, MGCP, and VoIP MPLS support for QoS
Readings 25 Lecture notes and material on web (links provided) At the library, on reserve: Crowcroft: Internetworking Multimedia, TK5105.5.C78 1999 Huston: Internet Performance Survival Guide, TK5105.875.I57 H86 2000 Thomas: IPng and the TCIP/IP Protocols, TK5105.585.T46 1996 Keshav: An Engineering Approach to Computer Networking, TK5105.5.K48,1997 Stevens: Internetworking with TCP/IP, vol. III, TK5105.585.C6623 1996 Davie: MPLS Technology and Applications, TK5105.875.I57 D35 2000 Durham: Inside the Internet's Resource Reservation Protocol, TK5105.55.D87 1999 Hersent: IP Telephony, TK5105.8865.H47 2000 Collins: Carrier Grade Voice Over IP, TK5105.8865.C65 2001 Douskalis: Internet Telephony, TK5105.8865.D687 2000
( continued) 26 Gibson: Digital Compression for Multimedia, QA76.575.D535 1998 Rao: Techniques and Standards for Image, Video, and Audio Coding, TK5102.92.R365 1996 Haskell: Digital Video: An Introduction to MPEG-2, TK6680.5.H37 1997 Mitchell: MPEG Video Compression Standard, TK6680.M64 1997 Crane: A Simplified Approach to Image Processing, TA1637.C73 1997 McClellan: DSP First: A Multimedia Approach, TK5102.9.M388 1998 Goldberg: A Practical Handbook of Speech Coders, TK7882.S65. G66 2000 Barnwell: Speech Coding: A Computer Laboratory Textbook, TK7882.S65 B37, 1996 Gibson: Multimedia Communications, QA76.575.M76 2001 Fluckiger: Understanding Networked Multimedia, QA76.575.F58 1995 Buford: Multimedia Systems, QA76.575.M85 1994 Steinmetz: Multimedia: Computing, Communications, Applications, QA76.575.S73 1995 Braut: The Musician's Guide to MIDI, MT723.B73 1994 Possibly some articles handed out
Prerequisites 27 CSC/CPE/EE undergraduate course background Programming
Getting Help 28 Office hours Message board The TA
Exams, Homeworks 29 The calendar
What's Different About Multimedia? 30 Convergence of many technologies and markets
Why Should You Care About This Subject? 31 (your answer here)