MPEG-4 is a standardized digital video technology

MPEG-4 is a standardized digital video technology What is Digital Video? What are Digital Video Standards? What is MPEG-4? How MPEG-4 is the same as other digital video technologies. How MPEG-4 is different from other digital video technologies. Authoring MPEG-4. Distributing MPEG-4. Educational Uses of MPEG-4. MPEG-4 Resources

What is Digital Video? Working from micro to macro: encode/decode, intermediate vs. distribution, architectures, the user experience. ENCODE/DECODE A binary language describing a sequence of analog images that change over time. This can also but does not have to include audio information. A codec is the translation key used to encode (transform the moving pictures into a binary file) and decode (transform the binary file back into moving pictures) video/audio clips. Usually, codecs also utilize some kind of compression algorithm to eliminate redundant information. This helps decrease file sizes and data rates while maintaining acceptable image and sound quality.

TYPES OF CODECS Intermediate codecs...use minimal compression and maintain high quality; producing large files with high data rates. (spatial compression only) These video formats can be used for editing, graphics, motion effects, and other content creation tasks, but seldom are they appropriate for final delivery. Distribution codecs...use maximal compression and allow poorer quality; producing small files with low data rates. (both spatial & temporal compression) Real & Windows have been primarily for distribution, but that is changing. These video formats should only be used to deliver finished content, NOT for content creation tasks. Analog Tape DV Tape 4x CD-ROM Broadband 56K modem DIGITAL VIDEO

ARCHITECTURES A digital media architecture is a method for the creation, storage, and playback of digital media forms or types such as movies, audio files, animations, and interactive components. Current popular architectures include: MPEG-4 QuickTime RealMedia WinMedia For many years, QuickTime was the premiere authoring architecture (pun intended). All non-linear digital video editing/authoring software used the QuickTime architecture to handle video. Now, more and more applications use Real, WindowsMedia or their own proprietary architecture. DIGITAL VIDEO

END-USER EXPERIENCE 120x90 low Kbps no audio full frame, full motion, DTS surround 320x240 broadband mono PDA resources Meeting (with IT) Classroom Medical Student Corporation Boardmember Elementary Teacher An end-user experiences digital video as a movie that plays on a computer screen or some other electronic device; not a Television. The user will only see the Distribution version of the digital video file which may be stored on a video server, a web server, or a local disk such as a CD-ROM. For most of the architectures, the user needs to already have the "Player" software installed in order to view that flavor of digital video. "Player" software versions and features are not necessarily equal across Operating Systems/Platforms. Often, the profile of a typical end-user for a particular product is the deciding factor in determining which architecture is used for distribution. DIGITAL VIDEO

What are Digital Video Standards? A standard is defined by the Encyclopedia Britannica as: 3 : something established by authority, custom, or general consent as a model or example 4 : something set up and established by authority as a rule for the measure of quantity, weight, extent, value, or quality STANDARD applies to any definite rule, principle, or measure established by authority <standards of behavior> JPEG became the de-facto standard for still images on the web not because a government or corporation imposed it as a standard, but because of custom and general consent. And, it is important to consider the fact that the "baseline" JPEG technology can be implemented without payment of either royalty or license fees. In the US, there is a battle going on to be THE standard for HDTV. This battle is so heated because variations (different profiles) on an HDTV standard could become the standards for all devices; from computers to wireless phones. Broadcasting technology used to be entirely separate from digital video. Now, The company that can get their standard approved for use in DTV and HDTV could dominate the creation and distribution of digital video in ALL environments. In addition, popular perception of open standards is that the standardization process is "sluggish." The MPEG Industry Forum claims this is not true of MPEG-4.

!"#$%&$'()*+),+-$.)((/,0 123(/$-3456#7 -()*+),+-$8236+$9&*:3+/*9/; <*+$("/#$8236+$4),=/(-; >*-(/)+$&:$)$?&,6+ :,)9(3&*/+$3*(&$-4)66$9&45/(3*@$(/9"*&6&@#$:3/:+&4-7 -()*+),+-$9,/)(/$("/$:&2*+)(3&* :&,$?3+/-5,/)+$)+&5(3&*$&:$*/?$9&*-24/,$(/9"*&6&@3/-; A)=/$)$6&&=$),&2*+$#&2,$ 63B3*@$,&&4; <*#$C%$56)#-$3*$)*#$C%$56)#/,D )*#$(/6/B3-3&*$-()(3&*$9)*$8/$B3/?/+$ &*$)*#$8,)*+$&:$(/6/B3-3&*D )*#$%E%$56)#-$3*$)*#$%E%$56)#/,; A"3-$9&*:3+/*9/7 /)-/$ &:$2-/7 )*+$,)53+$9&*-24/,$)99/5()*9/$&:$*/?$4/+3)$(/9"*&6&@3/-$),/$)66$5&--386/ 8/9)2-/$&:$-()*+),+-;!"#$ %&'()* +,-.//0 1&'( %&2344$ 4$)2 5% 67 C&*-24/,$+/B39/-$8236($&*$-()*+),+-!"#$%#&%'()*+,(-$$./#"-.$ '()*+),+-$")B/$5)B/+$("/$?)#$:&,$(/9"*&6&@3/-$*/B/,$("&2@"($5&--386/; ')(/663(/$ (/6/B3-3&*$3-$8)-/+$&*$)$-()*+),+$+/B/6&5/+$3*$FGGHI.JKLMN; '()*+),+$:36/$:&,4)(- ")B/$4)+/$C%$)*+$.JO$56)#/,-$9&44&*56)9/$:&,$("/$56)#8)9=$&:$42-39;!"#$%#&%'(0&./-%+(+1.$.23 <-$-()*+),+-$),/$,)(3:3/+7 ("/$3*+2-(,#$9)*$:&92-$&*$"&?$(&$+/56&#$("/4$)($)$6&?/,$ 9&-($3*-(/)+$&:$&*$+/B/6&53*@$,/+2*+)*($(/9"*&6&@3/-; P&,$/Q)456/7 ("/$)+&5(3&*$&: ("/$.JKLMN$-()*+),+$")-$6&?/,/+$+/63B/,#$9&-(-$)*+$345,&B/+$R2)63(#$3*$%E%$)*+$ +3@3()6$-)(/663(/$(/9"*&6&@3/-;!"#$%#&%'(0&./-%+(14.-1+ '()*+),+-$/*)86/$("/$8236+/,-$&:$4/+3)$*/(?&,=-$(&$-/6/9($5,&+29(-$:,&4$)$*248/,$ &:$B/*+&,-$)*+$3*(/@,)(/$("/4$3*(&$)$-3*@6/7-9)6)86/$-#-(/4; C&45/(3(3&*$8/(?//* B/*+&,-$?366$,/-26($3*$)$8,&)+/,$-/($&:$5,&+29(-$(&$9"&&-/$:,&47 B),#3*@$3*$9&-(7 5/,:&,4)*9/7 )*+$:/)(2,/-; A"/$<556/$'()*+),+$&:$>**&B)(3&*$ <556/$")-$)$6&*@$"3-(&,#$&:$2-3*@$-()*+),+-$(&$-255&,($("/$)+&5(3&*$&:$3**&B)(3B/ STANDARDS (/9"*&6&@3/-7 )*+$.JKLMH$3-$*&$/Q9/5(3&*; P,&4$S'T$)*+$T62/(&&("$9&**/9(3&*-$:&, 5/,35"/,)6-7 (&$<3,J&,($UVWN;FFX$?3,/6/--$*/(?&,=3*@7 (&$P3,/!3,/$U>KKK$FOGHX$"3@"M-5//+ +3@3()6$B3+/&$(,)*-:/,7 <556/$")-$9")453&*/+$(/9"*&6&@#$-()*+),+-$(")($")B/$8//*

There isn't a shortage of possible digital video standards. By definition, a codec must be standard; otherwise, it wouldn't work. In fact, we have even more standards than we have architectures... There is a shortage of authority, custom, and/or general consent on digital video standards. There probably won't be any decision made until DTV standard(s) are resolved. So we have even more standards than we have architectures... QuickTime DV (NTSC/PAL) Sorenson MPEG-4 Video Apple H.263 MP3 RealMedia RealVideo 8 or 9 RealVideo G2 Envivio for MPEG-4 RealAudioCook RealAudioATRC WindowsMedia WM Video 7, 8, & 9 AVI Indeo MPEG-4 V3 MPEG-4 MPEG-4 Video Profiles: Simple - Advanced H.264 AAC STANDARDS

What is MPEG-4? MPEG-4 was designed to provide standardized ways to: 1. represent units of aural, visual or audiovisual content, called media objects. These media objects can be of natural or synthetic origin; this means they could be recorded with a camera or microphone, or generated with a computer; 2. describe the composition of these objects to create compound media objects that form audiovisual scenes; 3. multiplex and synchronize the data associated with media objects, so that they can be transported over network channels providing a QoS appropriate for the nature of the specific media objects; and 4. interact with the audiovisual scene generated at the receiver s end. media objects interactivity sync layer MPEG-4 scene description scalable data transport 56K - 2Mbps HDTV DSS PDA LAN MPEG-4 technology was developed around the concept of providing a container (or 'wrapper') for the digital delivery of ALL audiovisual media. Audio and video were not the primary focus, but included in that larger concept.

A1*#-$)($1('$8-+#: 2($&)'$*#,1%#*$56$("#$;<=>$M;-A1%L$<18(/+#$=K4#+('$>+-/4N$8-..1((##3 ("#$&-+91%L$L+-/4$&1("1%$("#$2%(#+%)(1-%)0$G+L)%1O)(1-%$,-+$P()%*)+*1O)( M2PGN$(")($'4#81,1#*$("#$&1*#06$)*-4(#*3 =..6$Q&)+*R&1%%1%L$'()%*)+*'$9%-&% ;<=>?C$)%*$;<=>?H: ;<=>?@$1'$("#$+#'/0($-,$)%$1%(#+%)(1-%)0$#,,-+($1%A-0A1%L$"/ -,$+#'#)+8"#+'$)%*$#%L1%##+': ;<=>?@3 &"-'#$,-+.)0$*#'1L%)(1-%$1'$2PGS2=D$C@@T,1%)01O#*$1%$G8(-5#+$CTTV$)%*$5#8).#$)%$1%(#+%)(1-%)0$'()%*)+*$1%$#)+06$CTTT:!"#$8-.4-%#%('$-,$;<=>?@ ABOUT

How MPEG-4 is the same as other digital video technologies. DIGITAL DELIVERY ENCODE/ DECODE COMPRESSION SCRIPTING FOR MULTIMEDIA MPEG-4 QuickTime Real WinMedia DIGITAL DELIVERY: Ability to transfer files using RTSP/HTTP/FTP, removable disk, LAN, etc. ENCODE/DECODE: Using the basic codec methodology to create digital video. COMPRESSION: Discarding redundant information to decrease file size & maintain perceptual quality. SCRIPTING FOR MULTIMEDIA: Either included in architecture or add-on using scripting language, like SMIL.

for broad industry support. Proprietary solutions can only succeed if they are adopted by large market segments, which has not happened with existing technologies. The table below gives a comparison of MPEG-4 against most commonly used multimedia formats on the Internet today. MPEG-4 Windows Media Real Flash Audio/Video Standards based; multi- Proprietary Proprietary, but Proprietary + Codec vendor support. supports automatic proprietary Real and download of MPEG-4 QuickTime formats. plug-in. Interactivity Highly interactive. Limited Yes, via SMIL. Highly interactive. Digital Rights Interfaces to proprietary Microsoft DRM Content access No Management DRM. More interoperable control DRM under development in MPEG-4 and MPEG-21 Real-time stream control Yes Yes Yes No Synchronization Audio, video and all Tight synchronization Tight synchronization No synchronization other objects can be between audio and between audio and between scene and tightly synchronized with video video streams high accuracy Broadcast Yes, including interactive A/V only Scene must be No capable features unicast Object model Video/audio and rich Audio/Video only Video/audio and Video/audio and mixed support 2D/3D mixed media, mixed media through media through synthetic graphics. DRM SMIL based protocol. proprietary protocol. on separate streams. No streaming of mixed media. Graphic Objects Yes No No Yes Transport Support exists for HTTP, HTTP, UDP, HTTP, RTP/RTSP, HTTP UDP, RTP/RTSP, MPEG- RTP/RTSP, mobile mobile 2TS, mobile PC, Set Top Box, Wireless Yes Yes Yes No SIMILARITIES

How MPEG-4 is different from other digital video technologies. UNDERLYING TECHNOLOGY MPEG-4 builds on the proven success of three fields: Digital television; Interactive graphics applications (synthetic content); Interactive multimedia (World Wide Web, distribution of and access to content) MPEG-4 provides the standardized technological elements enabling the integration of the production, distribution and content across paradigms of the three fields. Object-oriented, container architecture. only QuickTime is similar since MPEG-4 is based on the same container concept. MPEG-4 playback functionality can be added in most software players, natively supported or as a plug-in if nothing else. A JAVA applet can be used to play back MPEG-4 embedded in HTML. There's one available from a research group at IBM. Parameters for synthetic modeling of facial/body movement and speech in addition to natural recorded audiovisual. Text and synthetic objects are rendered locally with their own coding, rather than forcing them into pre-rendered pixel bitmaps or waveforms. With current Television technology, you can add graphic and textual elements to the final presentation, but you cannot delete them. The MPEG-4 paradigm is just the opposite. It is dynamic, where other video technology is static. Version 2 builds on Version 1 of MPEG-4. The Systems layer of Version 2 is backward compatible with Version 1. In the area of Audio and Visual, Version 2 will add Profiles to Version 1. The work on MPEG-4 does not stop after MPEG-4; more functionality will be added, albeit in particular, well-defined areas. The same principle applies, and new tools will find their way in the standard in the form of new Profiles. This means that existing systems will always remain compliant, because Profiles will not be changed in retrospect.

INTERACTIVITY MPEG-4 The Media Standard MPEG-4 and SMIL, SVG SMIL is the Synchronized Multimedia Integration Language xiii of the W3C and SVG is W3C s Scalable Vector Graphics specification xiv. A comparison of MPEG-4 and SMIL+SVG capabilities follows below. MPEG-4 provides a rich multimedia experience, in which interactivity, streaming, and various mixed media, including graphics objects, are combined seamlessly. SMIL and SVG, as currently proposed for use by 3GPP, provide somewhat similar functionality, with notable differences, as SMIL is more declarative in nature and MPEG-4 is more procedural. The comparison mostly concerns MPEG-4 BIFS (the Binary Format for Scenes) and the Object Descriptor framework of MPEG-4, which takes care of the synchronization between the different objects. Requirement MPEG-4 SMIL+SVG Spatial and temporal composition of text, graphics, images and streamed media (audio and visual streams) Flexible synchronization models of different objects (co-start, co-end, ) Broadcast-grade synchronization of all objects on a rigid timeline (e.g. A and V) Very simple to very complex composition. 2D and 3D profiles Yes Yes Only 2D composition Yes No Streaming scene description Yes No Compression of scene description Yes No Dynamic scenes (add/ remove objects, etc) Yes No Streamed animation of scene components Yes No Broadcast capable Yes No DRM tightly coupled with scene (e.g. can protect streams independently) Yes No MPEG-4 s Textual Format: XMT Originally, MPEG-4 only contained a binary DIFFERENCES scene description language. Later, it became clear that it would be helpful to add a textual representation as well, in the form of XMT, the extensible MPEG-4 Textual format. XMT is an XML-based language, like SMIL. MPEG has been careful to build XMT as compatible with SMIL as possible, to aid interoperability in

PLAY SCALABILITY In order to allow effective implementations of the standard, subsets of the MPEG-4 Systems, Visual, and Audio tool sets have been identified, that can be used for specific applications. These subsets, called Profiles, limit the tool set a decoder has to implement. For each of these Profiles, one or more Levels have been set, restricting the computational complexity. Profiles and Levels are pre-defined for automatic delivery to multiple networks/devices from HD televisions to mobile phones. FOR EXAMPLE: MPEG-4 Advanced Audio Coding with SBR (Spectral Bandwidth Replication) can deliver high quality stereo audio at a mere 48kbit/s rather than 64kbit/s without SBR. SBR is an additional extension to AAC and is both forward and backward compatible: an existing MPEG-4 AAC decoder can decode the extended SBR signal (without the SBR enhancement) and a decoder with SBR can understand a signal that does not use the SBR extension. Client-side render of vector-based objects (Flash, VRML), text, and graphics. That allows each element to be coded using the particular technique that's ideal for that element type as well as providing high quality at low bandwidths. Can have scaleable, moveable, synthetic 2D and 3D objects with their own associated metadata; face and body animations; VRML objects; JAVA 'MPEGlets'; Digital Rights Management; and on and on. DIFFERENCES

Authoring MPEG-4. PROBLEMS AUTHORING IN MPEG-4 There are still very few developer tools - if any - that have an easy, WYSIWYG interface. MPEG-4 has so many features and layers of complexity that you almost need to have a software engineering degree to understand it all. There are still very few developer tools - if any - that have an easy, WYSIWYG interface. There are many ways in which an.mp4 file can be created, from QuickTime export to compiling an XML file, and all are different. There are still very few developer tools - if any - that have an easy, WYSIWYG interface. ADVANTAGES AUTHORING IN MPEG-4 There are many ways in which an.mp4 file can be created, from QuickTime export to compiling an XML file, and all are different. MPEG-4 offers features that otherwise would only be available by combining separate media objects, each one authored in its own environment, into one final product. Interaction, dynamic rendering, JAVA, interaction, dynamic rendering, JAVA, interaction, dynamic rendering,...

Distributing MPEG-4 PROBLEMS DISTRIBUTING DIGITAL VIDEO End users on computers in a lab, service bureau, school, or office where they do not have administrator privileges and cannot install any new software won't be able to view the video. End users are behind firewalls where they cannot receive RTSP streams or the IT dept. has blocked the necessary ports. Authors must create once, then encode multiple times for a range of applications; and then transport/package in different ways to deliver to a range of devices. Authors must include 'system requirements', technical documentation, licensed installers for proprietary software, and so on for packaged products. Authors cannot anticipate EVERY possible environment and/or scenario in which their product will be used, so the final product may have usability issues once it's out in the world. Digital video is static, not dynamic. Once it has been created, it is frozen in place. The 'prototype::test::revise::distribute' model is more difficult than it is with other media. To make changes, an author has to go back to the beginning and start over.

content and services from low bandwidths to high-definition quality across broadcast, broadband, wireless and packaged Is MPEG-4 media. easier to distribute? MPEG-4 is an open toolbox to build bitstreams and decoders for all multimedia content. MPEG-4 provides Only a standardized if it is widely framework adopted as for a standard many other across forms the of television, media including text, pictures, animation, computer, 2D handheld, 3D objects wireless, which and entertainment can be presented industries. interactive and personalized media experiences. To support the diversity of the future content market MPEG- The future is video and multimedia on many different devices, with very 4 offers a many variety totally of different so-called uses. profiles, tool sets from the toolbox, useful for specific applications, e.g. in audio-video coding, simple visual or advanced simple visual profile. Users While the internet is growing exponentially, and streaming media and need only implement video on demand the profiles are poised that support to be large the applications, functionality the required. future is also about wireless connectivity on phone and PDA or slate type devices. Creation Contribution YES, BUT Distribution Consumption TV video Production Playout Storage terrestrial PC audio interaction Encode Packaged satellite DVD 2D images animation MPEG-4 server wireless CellPhone PDA 3D images BIFS IPMP DRM Cable Game Console text Retail Radio Figure 1: The MPEG-4 ecosystem liberates multimedia for delivery across any network to any user of any device DISTRIBUTION 19/11/2002 3

Educational Uses of MPEG-4. EXAMPLES MPEG-4: IBM Technologies player & research MPEG-4 Video Gallery IDEAS

MPEG-4 Resources MPEG Industry Forum http://www.m4if.org/ MPEG Home Page (Moving Pictures Experts Group) http://www.chiariglione.org/mpeg/ MPEG-4, Overview of the Standard http://www.chiariglione.org/mpeg/standards/mpeg-4/mpeg-4.htm MPEG-4, Standard for Digital Media, from Apple (with valuable links) http://www.apple.com/mpeg4 MPEGIF Tutorials http://www.m4if.org/tutorials.php WEB LINKS Digital Video for the Web http://stream.uen.org/medsol/digvid/index.html BOOKS MPEG Handbook, by Touradj Ebrahimi, Fernando Pereira Publisher: Focal Press ISBN: 0240516567 The MPEG-4 Book, by Touradj Ebrahimi, Fernando Pereira Publisher: Pearson Education ISBN: 0130616214 MPEG-4 Jump-Start, by Aaron E. Walsh, Mikael Bourges-Sevenier Publisher: Prentice Hall ISBN: 0130600369 Understanding MPEG-4: Technology and Business Insights, (not released yet) Publisher: Focal Press ISBN: 0240805941