Multimedia applications - PDF Free Download

applications László Kálmán 1 Csaba Oravecz 1 Péter Szigetvári 2 1 Research Institute for Linguistics Hungarian Academy of Sciences 2 Department of English Linguistics Eötvös Loránd University Lecture 9. / 18 Nov, 2009

Outline 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Introduction abstract this lecture tells you about

Introduction abstract this lecture tells you about some popular multimedia devices;

Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files

Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files and the applications to manipulate them;

Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files and the applications to manipulate them; serving and getting media content through the web.

Introduction Outline Sample exam question 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Introduction Sample exam question Sample exam question Sample exam question

Introduction Sample exam question Sample exam question Sample exam question VoIP is... A a video file format B a computer game C an Internet goblin D an Internet protocol to carry voice signal

Introduction Outline 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Introduction uses multiple forms of information content and information processing (text, audio, graphics, animation, video, interactivity)

Introduction uses multiple forms of information content and information processing (text, audio, graphics, animation, video, interactivity) also refers to the use of electronic media to store and experience multimedia content

devices Outline Internal devices 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

devices Display adapter (a.k.a. video/graphics card) Internal devices Figure: Video card generates and outputs images to a display GPU (graphics processing unit): a microprocessor dedicated to manipulating and rendering (3D) graphics video memory (128MB-2GB): separate onboard memory, (video RAM (VRAM)) to store the display image, textures, other elements. VRAM typically runs at higher speeds than desktop RAM.

devices Sound card Internal devices Figure: Sound Blaster X-Fi Platinum sound card in general motherboards have built-in sound functionality but for high quality audio separate card is needed music composition, editing video or audio, presentation, entertainment (games) full-duplex: can record and play digitized sound simultaneously

devices TV tuner card Internal devices Figure: Hauppauge WinTV allows television signals to be received by a computer can also function as video capture cards, allowing recording television programs onto hard disk hybrids are capable of analog (the present) and digital (DVB the future) reception designed for the radio frequencies and video formats used in each country some have hardware MPEG encoders

devices TV tuner card Internal devices Figure: Watching television on the desktop

devices Sample non-exam question Internal devices What is the difference between a video card with a TV output and a TV tuner card...

devices Sample non-exam question Internal devices What is the difference between a video card with a TV output and a TV tuner card... The former allows you to make the TV a computer display, the latter the computer display a televison.

devices Outline External devices 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

devices DVD drives External devices DVD-ROM DVD±RW DL: (plus/dash!!!) rewritable, dual layer, compatible with DVD+RW and DVD-RW format capacity: up to 8.5GB DVD+RW supports random write access (data can be added and removed without erasing the whole disc and starting over) Blu-ray Disc: ultra-high-density optical disc, uses shorter wavelength (405 nm) blue-violet laser (DVDs use a 650-nm-wavelength red laser); 25-50GB (single/dual)

devices Regional lockout External devices Regional lockout Programming practice, code, chip, or physical barrier used to prevent the playing of media designed for a device from the country where it is marketed on the version of the same device marketed in another country. (video games) DVD: DVD drives come from the factory with RPC (Regional Playback Control) RPC-1 (older drives): player software has the responsibility of enforcing the region control RPC-2 (newer drives): control is enforced by the drive s firmware In practice, many DVD players allow playback of any disc, or can be modified to do so.

Figure: DVD regions

devices Portable multimedia players External devices Figure: 5th generation ipod a hard disk or flash memory based electronic device capable of storing and playing files in one or more media formats dedicated proprietary software, non-transferable data can function as mass storage devices with some standard file system to store data files Digital Rights Management (DRM): prohibits transfer of music from one type of device to another

Digitizing media content Outline AD conversion 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form.

Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form. Sampling discretization: signal space is split into chunks and each chunk is replaced with its representative signal quantization: the representative signal values are approximated by some value from a finite set

Digitizing media content Outline From sound to computer data 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Digitizing media content Sound processing From sound to computer data After sampling, audio signals are represented by one of a fixed number of values pulse-code modulation (PCM)

Digitizing media content Sound processing From sound to computer data After sampling, audio signals are represented by one of a fixed number of values pulse-code modulation (PCM) PCM A digital representation of an analog signal in a series of symbols in binary code. Figure: 4-bit PCM of a signal

Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio)

Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels 44100 16 2 = 1411.2 kbit/s dither: small amount of deliberately-added noise to the signal before digitization to avoid noticeable jumps in low level signals

Digitizing media content Important concepts I. From sound to computer data Audio codec (s/audio/video/gi) A computer program/algorithm that compresses/decompresses digital audio data according to a given audio file format or streaming audio format. Most codecs are implemented as libraries which interface to one or more multimedia players. Container format A computer file format that can contain various types of data, compressed by means of standardized codecs. The container file is used to identify and interleave the different data types (audio, video, subtitles, chapters, meta-data, synchronization info).

Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio

Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF))

Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems Lossless compressed formats/codecs (lossless) Windows Media Audio (WMA) (.wma): proprietary Microsoft; WMA lossless codec

Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs

Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3)

Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images)

Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders...

Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface

Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card)

Digitizing media content Outline From images to computer data 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels.

Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution number of pixels in an image single number (see eg. digital cameras) 6.2 megapixel product (displays) 1024x768

Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal.

Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally 25 30 frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD MPEG-4 Part-2: used for internet, broadcast, storage media; improved quality a Moving Picture Experts Group

Digitizing media content Feedback slide I. From images to computer data Sample exam question

Digitizing media content Feedback slide I. From images to computer data Sample exam question Which of the following is true?

Digitizing media content Feedback slide I. From images to computer data Sample exam question Which of the following is true? A With digital video, lossless compression is typical. B AVI, MOV, Ogg are codecs whereas Vorbis, WAV are container formats. C AVI, MOV, Ogg are container formats whereas Vorbis, Sorenson are codecs. D A video card with a TV output allows you to watch television on the computer.

Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed)

Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files Encapsulated Postscript (.eps): a PostScript file minimally with a BoundingBox comment, describing the rectangle containing the image Graphics Interchange Format (.gif): 8-bit-per-pixel bitmap image format using a palette of up to 256 distinct colors; lossless but unsuitable for images with continuous color; multiple images in one file animations Portable Network Graphics (.png): bitmap image format with lossless data compression (patent-free successor of GIF); no animations, incompatible with early IE

Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Figure: More and more lossy...

Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group Figure: More and more lossy...

Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg Figure: More and more lossy...

Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg compression method and (commonly) file format, too but other formats can carry jpeg (eg. TIFF) Figure: More and more lossy...

Digitizing media content Common image file formats From images to computer data Figure: More and more lossy... Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg compression method and (commonly) file format, too but other formats can carry jpeg (eg. TIFF) most used for storing and transmitting photographs on the Web

world The of color

The world of color Outline Color models 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

The world of color Color models Color models Color model A model describing the way colors can be represented as tuples of numbers (typically three or four values or color components). It is associated with a precise description of how the components should be interpreted color space.

The world of color RGB color model Color models

The world of color RGB color model Color models Red, green, blue

The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness)

The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science as percentages, from 0% (minimum) to 100% (maximum) (full intensity red is???)

The world of color Truecolor 24 bit representation Color models yellow (255,255,0) green (0,255,0) cyan (0,255,255) red (255,0,0) blue (0,0,255) red (255,0,0) magenta (255,0,255)

The world of color CMYK color model Color models Cyan, magenta, yellow, key

The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed. magenta + yellow = red magenta + cyan = blue cyan + yellow = green the use of black: produce pure black (without mixing three colors, which is not perfect and expensive)

The world of color Outline Image models 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

The world of color Raster graphics image Image models Bitmap Figure: RGB bitmap

The world of color Raster graphics image Image models Bitmap data file or structure representing a rectangular grid of pixels (points of color) Figure: RGB bitmap

The world of color Raster graphics image Image models Bitmap data file or structure representing a rectangular grid of pixels (points of color) color of each pixel is individually defined Figure: RGB bitmap

The world of color Raster graphics image Image models Figure: RGB bitmap Bitmap data file or structure representing a rectangular grid of pixels (points of color) color of each pixel is individually defined corresponds bit for bit with an image displayed on eg. screen characterized by the width and height of the image in pixels and the number of bits per pixel

The world of color Vector graphics Image models Figure: Vector scales well

The world of color Vector graphics Image models image is represented through the use of geometrical primitives such as points, lines, curves, and polygons Figure: Vector scales well

The world of color Vector graphics Image models image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display Figure: Vector scales well

The world of color Vector graphics Image models Figure: Vector scales well image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display minimal amount of information results in small file size moving, scaling, rotating, filling etc. doesn t degrade the quality of a drawing

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization.

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling

Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution) compression (lossy/lossless) codecs

Recap Recap Images

Recap Recap Images representation

Recap Recap Images representation raster

Recap Recap Images representation raster vector

Recap Recap Images representation raster vector storage/file format lossless lossy compressed uncompressed

Images Outline Digital image editing 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors selection: selecting part(s) of an image, so applying a change selectively without affecting the entire picture; complex means (eg. edge detection, color-based, lasso) layers: transparent sheets stacked on top of each other, each capable of being individually edited

Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image

Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image

Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture

Images Outline Digital photography 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.?

Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB) Exif Exchangeable image file format specification for the image file format used by digital cameras TIFF for the highest quality format and JPEG as a space-saving but lower quality format; (RAW: proprietary, minimally processed format, not standardized, plugins needed for image editors to handle these files)

Images Exif data Digital photography Figure: Exif data in Nautilus file manager