Hardware. Multimedia computers. Embedded devices. Mobile Phones. Multimedia PC (MPC) Current equipment

Transcription

1 Hardware Multimedia computers Video and graphics Audio Telephone, video conference, and networks CD and DVD USB and FireWire Processors Video for Windows, DirectX, and ActiveMovie Petri Vuorimaa 1 Multimedia computers Most computers support multimedia They are usually based on bus architecture Processor Memory Hard disk CD-ROM Bus Video Audio Ports USB MIDI Network Petri Vuorimaa 2 Embedded devices Mobile Phones Embedded devices do not have a common bus Set-top-box Tuner and demodulator Conditional access Demux Video decoder Audio decoder WLAN. Bluetooth. Display Camera Input RAM Java OS. ASIC Multimedia HW accel.. DSP MCU SRAM Protocols ASIC Systems Logic DSP MCU SRAM GPS Antenna Analog Circuits Memory Microprocessor Modem Mem. Card Flash Flash Petri Vuorimaa 3 Petri Vuorimaa 4 Current equipment Properties: 1) multimedia play, edit, and storage 2) data transfer via telephone and computer networks Limitations: 1) limited number of concurrent media (performance) 2) synchronization (done with hardware) 3) interactivity Multimedia PC (MPC) Common hardware manufacturer standard Defines basic features for hardware devices Application developer can easily specify required hardware level Current version 3 Petri Vuorimaa 5 Petri Vuorimaa 6 1

2 Video and graphics Video and graphics (cont.) Color code 8 or n bits 256 or 2 n colors R G B.. VGA x 480 (max resolution) colors SVGA x bit. color Video pictures are displayed via graphics memory MPC3: + 30 frames/s x 240 resolution + 15 bit/pixel MPEG1 compression Display Petri Vuorimaa 7 Petri Vuorimaa 8 Audio Most computers have SoundBlaster compatible sound card or circuit PC AUDIO PCM CODEC MIDI Port Audio in/out CD analog in in out through stereo mic CD-DA Petri Vuorimaa 9 MPC3 audio requirements 1) 8 or 16 bit samples; 8, , 16, tai 44.1 khz frequency 2) Pulse Code Modulation (PCM) codec 3) 16 bit stereo takes max. 13 % of processing power with 44.1 khz frequency and 7 % with khz frequency 4) CD-ROM drive with digital audio 5) two speakers (120 Hz khz) 6) subwoofer ( Hz) 7) MIDI port Petri Vuorimaa 10 Telephone Computer-Telephony Integration (CTI) 1) Call control: + dialing + call transfer + call centers, etc. 2) Telephone applications: + personal information +fax + answering machine, etc. Petri Vuorimaa 11 Windows programming interfaces 1) Telephony Application Programming Interface (TAPI) 2) Telephone Service Programming Interface (TSPI) 3) Messaging Application Programming Interface (MAPI) 4) Speech Application Programming Interface (SAPI) 5) Media Control Interface (MCI) MPC3 requires that the system is TAPI2.0 compliant Petri Vuorimaa 12 2

3 Video conference Video conference and group work Many vendor specific solutions Standards: 1) ISDN H.320: video H.261; audio G.711, G.722, G.728; data T.120 2) Telephone (POTS) H.324: video H.261, H.263, audio G.723; data T.120 3) LAN and Internet H.323: video H.261, H.263, audio G.711, G.722, G.723, G.728, data T.120 Video conference (cont.) Bandwidth 1) ISDN: 128 kbps (n x 64 kbps) 2) Telephone: 56 kbps 3) LAN and Internet: unlimited bandwidth Usually video conference requires special hardware support Petri Vuorimaa 13 Petri Vuorimaa 14 Video conference hardware TV and radio Graphics, video & audio Graphics, video & audio Audio codec Video codec Data codec Audio codec Video codec Data codec Network card Network card Television and radio can be integrated with special cards + also USB devices Blanking interval (TV) can be used for sending additional information (Intercast in USA) E.g., www pages, hyperlinks, program information Also, digital TV receiver cards exists Petri Vuorimaa 15 Petri Vuorimaa 16 Network card CD Network connection requires a special card or circuit Typical speed 4, 10, 16, 25, or 100 Mbps Physical media can be either coaxial cable, twisted pair, or optical fiber Also wireless alternatives: radio, infrared The network connection is used via network protocol stack (e.g., TCP/IP) Petri Vuorimaa 17 CD-ROM is well suited for multimedia distribution Many standards CD-DA (red book): digital audio, 150 kbps, 44.1 ksamples/s, 682 MB CD-ROM (yellow book): 1) text & data, 2) audio & video data CD-I (green book): text, graphics, audio, and video repeat (multimedia and games) Petri Vuorimaa 18 3

4 CD (cont.) DVI: Indeo video (72 min) CD-XA (extended yellow book): improves audio and video MSCDEX: MS-DOS operating system extensions PhotoCD: max 100 compressed pictures Video CD (white book): MPEG1 video (74 min) CD-R (orange book): write once CD Petri Vuorimaa 19 MPC3 CD requirements 1) CD-DA 2) CD-ROM 3) CD-XA 4) CD-I 5) Video CD 6) CD-R 7) Photo CD 8) Enhanced Music CD Transfer rate has to be at least 550 kbytes/s Petri Vuorimaa 20 DVD DVD have bigger storage capacity than CD 1) smaller pit distance (0.4 microns vs microns) 2) smaller track distance (0.74 microns vs. 1.6 microns) 3) smaller laser wave length ( nanometers vs. 780 nanometers) 4) two sides 5) two layers DVD capacity DVD capacity: 1) one side and one layer: 4.7 GBytes 2) two sides and one layer: 9.4 GBytes 3) two sides and two layers : 17 GBytes DVD drives can be compatible with CD records Video uses MPEG-2 compression and audio Dolby AC-3 Petri Vuorimaa 21 Petri Vuorimaa 22 DVD formats Physical layer + DVD-ROM, DVD-RAM, DVD-R, DVD-RW Logical layer + Universal Disc Format (UDF) Applications + DVD-Video, DVD-Audio, DVD-Professional DVD encryption Part of content is encrypted (saves processor time) DVD player uses hardware for decoding The encryption key is on the DVD disk + only license holders know how to locate it TV signal is scrambled, so it can not be copied DVD-ROM drive gives encryption key via secure bus (Content Scrambling System - CSS) Water stamps can be used for limited copying Petri Vuorimaa 23 Petri Vuorimaa 24 4

5 Does it work? Petri Vuorimaa 26 USB Universal Serial Bus (USB) can be used for connecting different kinds of devices USB advantages: 1) hardware settings (e.g., interrupts, DMA, addresses, etc.) do not have to be defined 2) no need to open computer 3) no separate power wires USB devices 1) keyboard, mouse, printer, modem, scanner, CD- ROM, etc. 2) telephone, ISDN, etc. 3) joystick, game controllers, data gloves, head displays, etc. 4) video cameras, digital cameras, etc. 5) TV and radio receivers Petri Vuorimaa 27 Petri Vuorimaa 28 USB features USB speed is 1.5 Mbps and 12 Mbps USB has four wires Vbus D+ D- GND Bits are defined as voltage differences Maximum distance is 5 meters Devices are connected into star topology (HUB) Networks can be hierarchical Vbus D+ D- GND USB supports 1) Isochronous video and audio transfer 2) Real-time interrupts 3) Large data transfers with best effort method 4) Control information transfer Petri Vuorimaa 29 Petri Vuorimaa 30 5

6 USB 2.0 Transfer capacity max 480 Mbps (40 times) Compatible with USB 1.1 Speed is dropped automatically if required Allows the use of new hardware: + video conferencing (better image quality) + printers & skanners (speed & resolution) + CDROM & DVD (more data) FireWire As USB, but larger transfer rate: , , and Mbps Over 1 Gbps in development devices and bridges One cable max distance 4.5 m Two devices max distance 16 cables (i.e., 72 m) Petri Vuorimaa 31 Petri Vuorimaa 32 FireWire configuration Device groups are linked via bridges FireWire cable Cable has two data wires and two power wires Petri Vuorimaa 33 Petri Vuorimaa 34 FireWire advantages 1) Scales well 2) Flexible topology 3) Devices can be attached to a hot bus 4) Easy to use (no settings) 5) Designed for consumer devices (cheap) FireWire situation Limited amount of devices FireWire cards available for computers Some workstations and PC have FireWire as standard feature Some video cameras, etc. A standard feature in future digital television settop-boxes Petri Vuorimaa 35 Petri Vuorimaa 36 6

7 Processors Multimedia processing requires performance Problem can be solved with: 1) special hardware (MPEG decompression, etc.) 2) special processors (signal processors, etc.) 3) extensions of standard processors (MMX, etc.) MMX MMX is Intel Pentium and Pentium Pro multimedia extension Based on Single Instruction Multiple Data (SIMD) architecture One instruction control several data units at the same time E.g., 64 bit register has eight 8 bit data units or four 16 bit data units Petri Vuorimaa 37 Petri Vuorimaa 38 Example: parallel multiply MMX in practice I(3) I(2) I(1) I(0) I(3) I(2) I(1) I(0) I(3) I(2) I(1) I(0) Multiply operation Same operation is repeated for several data units Video compression, image processing, computer graphics, modems, and digital signal processing For example, MPEG-1 video decompression is 1.5 times faster than without MMX instruction set The algorithms have to be recoded Petri Vuorimaa 39 Petri Vuorimaa 40 New processor architectures K. Diefendorff, P. K. Dubey: How Multimedia Workloads Will Change Processor Design, IEEE Computer, Sept. 1997, pp ) real-time response 2) continuous media 3) significant amount of low level parallelism 4) significant amount of high level parallelism 5) instructions are mainly inside loops 6) wideband memory 7) broadband communications 8) different data formats Petri Vuorimaa 41 V-IRAM C. E. Kozyrakis et al., Scalable Processors in the Billion-Transistor Era: IRAM, IEEE Computer, Sept. 1997, pp Intelligent DRAM instead of Cache memory Fast serial buses Vector processing Super scalar processing Petri Vuorimaa 42 7

8 PC software Multimedia application Video for Windows MCI (Multimedia Control Interface) Audio TAPI TSPI Video Conference Devices are controlled via Application Programming Interfaces (APIs) Some are standards Some are manufacturers own definitions Petri Vuorimaa 44 Video for Windows Microsoft solution for video compression, decompression, and play Codecs: 1) Microsoft Video 1 2) Intel Indeo R3.1 & R3.2 3) Radius Cinepak 4) Microsoft Runlength Encoding (RLE) VFW advantages Compression and decompression can be done with same software Video is stored into a file (.AVI) with audio MCI = Media Control Interface VFW = Video for Windows AVI = Audio Video Interleave ICM = Installable Compression Manager Petri Vuorimaa 45 Petri Vuorimaa 46 Media Control Interface (MCI) DirectX Low level API 1) DirectDraw: 2D & 3D video and animation 2) Direct3D: real-time 3D graphics 3) DirectSound: audio 4) DirectPlay: multiplayer games via Internet 5) DirectInput: game controllers Petri Vuorimaa 47 Petri Vuorimaa 48 8

9 DirectX usage First via VFW and MCI DirectX usage (cont.) Later ActiveMovie services are used Petri Vuorimaa 49 Petri Vuorimaa 50 Active Movie Active Movie example ActiveMovie architecture is based on filters E.g., MPEG filters Filters can be controlled according to available processing power (i.e., orchestration) Petri Vuorimaa 51 Petri Vuorimaa 52 9