Multimedia applications

Transcription

1 applications László Kálmán 1 Csaba Oravecz 1 Péter Szigetvári 2 1 Research Institute for Linguistics Hungarian Academy of Sciences 2 Department of English Linguistics Eötvös Loránd University Lecture 9. / 18 Nov, 2009

2 Outline 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

3 Introduction abstract this lecture tells you about

4 Introduction abstract this lecture tells you about some popular multimedia devices;

5 Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files

6 Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files and the applications to manipulate them;

7 Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files and the applications to manipulate them; serving and getting media content through the web.

8 Introduction abstract this lecture tells you about some popular multimedia devices; basic properties of different multimedia files and the applications to manipulate them; serving and getting media content through the web. Sodipodi.

9 Introduction Outline Sample exam question 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

10 Introduction Sample exam question Sample exam question Sample exam question

11 Introduction Sample exam question Sample exam question Sample exam question VoIP is... A a video file format B a computer game C an Internet goblin D an Internet protocol to carry voice signal

12 Introduction Sample exam question Sample exam question Sample exam question VoIP is... A a video file format B a computer game C an Internet goblin D an Internet protocol to carry voice signal

13 Introduction Outline 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

14 Introduction uses multiple forms of information content and information processing (text, audio, graphics, animation, video, interactivity)

15 Introduction uses multiple forms of information content and information processing (text, audio, graphics, animation, video, interactivity) also refers to the use of electronic media to store and experience multimedia content

16 devices Outline Internal devices 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

17 devices Display adapter (a.k.a. video/graphics card) Internal devices Figure: Video card generates and outputs images to a display GPU (graphics processing unit): a microprocessor dedicated to manipulating and rendering (3D) graphics video memory (128MB-2GB): separate onboard memory, (video RAM (VRAM)) to store the display image, textures, other elements. VRAM typically runs at higher speeds than desktop RAM.

18 devices Sound card Internal devices Figure: Sound Blaster X-Fi Platinum sound card in general motherboards have built-in sound functionality but for high quality audio separate card is needed music composition, editing video or audio, presentation, entertainment (games) full-duplex: can record and play digitized sound simultaneously

19 devices TV tuner card Internal devices Figure: Hauppauge WinTV allows television signals to be received by a computer can also function as video capture cards, allowing recording television programs onto hard disk hybrids are capable of analog (the present) and digital (DVB the future) reception

20 devices TV tuner card Internal devices Figure: Hauppauge WinTV allows television signals to be received by a computer can also function as video capture cards, allowing recording television programs onto hard disk hybrids are capable of analog (the present) and digital (DVB the future) reception designed for the radio frequencies and video formats used in each country some have hardware MPEG encoders

21 devices TV tuner card Internal devices Figure: Hauppauge WinTV allows television signals to be received by a computer can also function as video capture cards, allowing recording television programs onto hard disk hybrids are capable of analog (the present) and digital (DVB the future) reception designed for the radio frequencies and video formats used in each country some have hardware MPEG encoders (a tuner displaying an HDTV image on a computer monitor is typically much cheaper than a high-definition television system... )

22 devices TV tuner card Internal devices Figure: Watching television on the desktop

23 devices Sample non-exam question Internal devices What is the difference between a video card with a TV output and a TV tuner card...

24 devices Sample non-exam question Internal devices What is the difference between a video card with a TV output and a TV tuner card... The former allows you to make the TV a computer display, the latter the computer display a televison.

25 devices Outline External devices 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

26 devices DVD drives External devices DVD-ROM DVD±RW DL: (plus/dash!!!) rewritable, dual layer, compatible with DVD+RW and DVD-RW format capacity: up to 8.5GB DVD+RW supports random write access (data can be added and removed without erasing the whole disc and starting over) Blu-ray Disc: ultra-high-density optical disc, uses shorter wavelength (405 nm) blue-violet laser (DVDs use a 650-nm-wavelength red laser); 25-50GB (single/dual)

27 devices DVD drives External devices DVD-ROM DVD±RW DL: (plus/dash!!!) rewritable, dual layer, compatible with DVD+RW and DVD-RW format capacity: up to 8.5GB DVD+RW supports random write access (data can be added and removed without erasing the whole disc and starting over) Blu-ray Disc: ultra-high-density optical disc, uses shorter wavelength (405 nm) blue-violet laser (DVDs use a 650-nm-wavelength red laser); 25-50GB (single/dual) DVD drives speed: up to 16x writing (21MB/s), 40x reading logo format!

28 devices Regional lockout External devices Regional lockout Programming practice, code, chip, or physical barrier used to prevent the playing of media designed for a device from the country where it is marketed on the version of the same device marketed in another country.

29 devices Regional lockout External devices Regional lockout Programming practice, code, chip, or physical barrier used to prevent the playing of media designed for a device from the country where it is marketed on the version of the same device marketed in another country. (video games) DVD: DVD drives come from the factory with RPC (Regional Playback Control) RPC-1 (older drives): player software has the responsibility of enforcing the region control RPC-2 (newer drives): control is enforced by the drive s firmware In practice, many DVD players allow playback of any disc, or can be modified to do so.

30 Figure: DVD regions

31 devices Portable multimedia players External devices Figure: 5th generation ipod a hard disk or flash memory based electronic device capable of storing and playing files in one or more media formats dedicated proprietary software, non-transferable data can function as mass storage devices with some standard file system to store data files Digital Rights Management (DRM): prohibits transfer of music from one type of device to another

32 Digitizing media content Outline AD conversion 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

33 Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form.

34 Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form. Sampling discretization: signal space is split into chunks and each chunk is replaced with its representative signal

35 Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form. Sampling discretization: signal space is split into chunks and each chunk is replaced with its representative signal quantization: the representative signal values are approximated by some value from a finite set

36 Digitizing media content Analog to digital conversion AD conversion Aim Convert analog real-world signals to discrete digital form. Sampling discretization: signal space is split into chunks and each chunk is replaced with its representative signal quantization: the representative signal values are approximated by some value from a finite set the higher the sampling frequency, the higher the fidelity

37 Digitizing media content Outline From sound to computer data 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

38 Digitizing media content Sound processing From sound to computer data After sampling, audio signals are represented by one of a fixed number of values pulse-code modulation (PCM)

39 Digitizing media content Sound processing From sound to computer data After sampling, audio signals are represented by one of a fixed number of values pulse-code modulation (PCM) PCM A digital representation of an analog signal in a series of symbols in binary code. Figure: 4-bit PCM of a signal

40 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio)

41 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD)

42 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels

43 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels = kbit/s

44 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels = kbit/s dither: small amount of deliberately-added noise to the signal before digitization to avoid noticeable jumps in low level signals

45 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels = kbit/s dither: small amount of deliberately-added noise to the signal before digitization to avoid noticeable jumps in low level signals The one major uncompressed audio format: PCM...

46 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels = kbit/s dither: small amount of deliberately-added noise to the signal before digitization to avoid noticeable jumps in low level signals The one major uncompressed audio format: PCM... is adequate for storing and archiving an original recording but

47 Digitizing media content Sound processing From sound to computer data sampling rate: from 8 (telephony) to 44.1k samples/s (CD) or 48k samples/s (professional audio) quantization: from 8 bits (telephony) to 16 bits (CD) bitrate=sampling rate quantization num.o.channels = kbit/s dither: small amount of deliberately-added noise to the signal before digitization to avoid noticeable jumps in low level signals The one major uncompressed audio format: PCM... is adequate for storing and archiving an original recording but needs large disk space

48 Digitizing media content Important concepts I. From sound to computer data Audio codec (s/audio/video/gi) A computer program/algorithm that compresses/decompresses digital audio data according to a given audio file format or streaming audio format. Most codecs are implemented as libraries which interface to one or more multimedia players.

49 Digitizing media content Important concepts I. From sound to computer data Audio codec (s/audio/video/gi) A computer program/algorithm that compresses/decompresses digital audio data according to a given audio file format or streaming audio format. Most codecs are implemented as libraries which interface to one or more multimedia players. Container format A computer file format that can contain various types of data, compressed by means of standardized codecs. The container file is used to identify and interleave the different data types (audio, video, subtitles, chapters, meta-data, synchronization info).

50 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio

51 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF))

52 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple

53 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems

54 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems Lossless compressed formats/codecs

55 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems Lossless compressed formats/codecs (lossless) Windows Media Audio (WMA) (.wma): proprietary Microsoft; WMA lossless codec

56 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems Lossless compressed formats/codecs (lossless) Windows Media Audio (WMA) (.wma): proprietary Microsoft; WMA lossless codec Free Lossless Audio Codec (.flac) compression rate: 30-50%

57 Digitizing media content Audio file formats From sound to computer data Uncompressed container formats for PCM encoded audio WAV(E) (.wav) Microsoft (from the Resource Interchange File Format (RIFF)) Audio Interchange File Format (AIFF) (.aiff) Apple AU (.au) Sun Microsystems Lossless compressed formats/codecs (lossless) Windows Media Audio (WMA) (.wma): proprietary Microsoft; WMA lossless codec Free Lossless Audio Codec (.flac) compression rate: 30-50% Apple Lossless: stored within an MP4 container (.m4a); compression rate: 40-60%

58 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs

59 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3)

60 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images)

61 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate)

62 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders...

63 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders... Ogg format with Vorbis codec

64 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders... Ogg format with Vorbis codec open source and patent-free

65 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders... Ogg format with Vorbis codec open source and patent-free cross-platform

66 Digitizing media content Audio file formats From sound to computer data Lossy compressed formats/codecs MP3: (MPEG-1 Audio Layer 3) discards components less audible to human hearing ( jpeg images) quality is heavily dependent on the choice of encoder and encoding parameters (eg. bit rate) has several limitations, licensing and patent problems but too popular to be replaced by better encoders... Ogg format with Vorbis codec open source and patent-free cross-platform AAC (Advanced Audio Coding, default in Apple itunes); WMA

67 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface

68 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other

69 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about:

70 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play

71 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning

72 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo

73 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card)

74 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card) file format: Standard MIDI File (.smf,.mid)

75 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card) file format: Standard MIDI File (.smf,.mid) created using computer-based sequencing software

76 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card) file format: Standard MIDI File (.smf,.mid) created using computer-based sequencing software

77 Digitizing media content MIDI From sound to computer data Musical Instrument Digital Interface communications protocol that enables electronic musical instruments, computers and other equipment to communicate, control and synchronize with each other does not transmit audio but only digital data about: the pitch and intensity of musical notes to play control signals for parameters of volume, vibrato and panning clock signals to set the tempo in computers it is sound cards that are MIDI-compatible and capable of creating realistic instrument sounds (quality sound quality card) file format: Standard MIDI File (.smf,.mid) created using computer-based sequencing software List_of_MIDI_editors_and_sequencers

78 Digitizing media content Outline From images to computer data 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

79 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels.

80 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color

81 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information

82 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution

83 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution number of pixels in an image

84 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution number of pixels in an image single number (see eg. digital cameras) 6.2 megapixel

85 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution number of pixels in an image single number (see eg. digital cameras) 6.2 megapixel product (displays) 1024x768

86 Digitizing media content Image processing From images to computer data Digital image A representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. color brightness information Resolution number of pixels in an image single number (see eg. digital cameras) 6.2 megapixel product (displays) 1024x768 Color depth Number of bits used to represent the color of a single pixel (8 48 bpp). (cf. Highcolor 16bpp; Truecolor 24bpp)

87 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal.

88 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time

89 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard

90 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 a Moving Picture Experts Group

91 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD a Moving Picture Experts Group

92 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD MPEG-4 Part-2: used for internet, broadcast, storage media; improved quality a Moving Picture Experts Group

93 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD MPEG-4 Part-2: used for internet, broadcast, storage media; improved quality MPEG-4 Part-10: (Advanced Video Coding (AVC)) state of the art standardized compression technology a Moving Picture Experts Group

94 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD MPEG-4 Part-2: used for internet, broadcast, storage media; improved quality MPEG-4 Part-10: (Advanced Video Coding (AVC)) state of the art standardized compression technology DivX, Xvid, 3ivx: different implementations of MPEG-4 Part-2 a Moving Picture Experts Group

95 Digitizing media content Image processing From images to computer data Digital video Digital (binary) representation of the video signal. composed of still images; frame: a set of all pixels that correspond to a single point in time normally frames per second, depending on standard Video compression (lossy) codecs MPEG a -1 Part-2: used for Video CDs, sometimes for online video; quality VHS; normally up to 1150 kbit/s and 352x288 MPEG-2 Part-2: used on DVD, SVCD, in most digital video broadcasting and cable distribution systems; 720x576 and 10 Mbit/s for DVD MPEG-4 Part-2: used for internet, broadcast, storage media; improved quality MPEG-4 Part-10: (Advanced Video Coding (AVC)) state of the art standardized compression technology DivX, Xvid, 3ivx: different implementations of MPEG-4 Part-2 Theora: open source and patent-free competitor to MPEG-4 a Moving Picture Experts Group

96 Digitizing media content Feedback slide I. From images to computer data Sample exam question

97 Digitizing media content Feedback slide I. From images to computer data Sample exam question Which of the following is true?

98 Digitizing media content Feedback slide I. From images to computer data Sample exam question Which of the following is true? A With digital video, lossless compression is typical. B AVI, MOV, Ogg are codecs whereas Vorbis, WAV are container formats. C AVI, MOV, Ogg are container formats whereas Vorbis, Sorenson are codecs. D A video card with a TV output allows you to watch television on the computer.

99 Digitizing media content Feedback slide I. From images to computer data Sample exam question Which of the following is true? A With digital video, lossless compression is typical. B AVI, MOV, Ogg are codecs whereas Vorbis, WAV are container formats. C AVI, MOV, Ogg are container formats whereas Vorbis, Sorenson are codecs. D A video card with a TV output allows you to watch television on the computer.

100 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed)

101 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files

102 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files Encapsulated Postscript (.eps): a PostScript file minimally with a BoundingBox comment, describing the rectangle containing the image

103 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files Encapsulated Postscript (.eps): a PostScript file minimally with a BoundingBox comment, describing the rectangle containing the image Graphics Interchange Format (.gif): 8-bit-per-pixel bitmap image format using a palette of up to 256 distinct colors; lossless but unsuitable for images with continuous color; multiple images in one file animations

104 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files Encapsulated Postscript (.eps): a PostScript file minimally with a BoundingBox comment, describing the rectangle containing the image Graphics Interchange Format (.gif): 8-bit-per-pixel bitmap image format using a palette of up to 256 distinct colors; lossless but unsuitable for images with continuous color; multiple images in one file animations Portable Network Graphics (.png): bitmap image format with lossless data compression (patent-free successor of GIF); no animations, incompatible with early IE

105 Digitizing media content Common image file formats From images to computer data Lossless (compressed/uncompressed) Windows Bitmap (.bmp): a bitmapped graphics format used internally by the Microsoft Windows graphics subsystem; uncompressed (but compresses losslessly well) large files Encapsulated Postscript (.eps): a PostScript file minimally with a BoundingBox comment, describing the rectangle containing the image Graphics Interchange Format (.gif): 8-bit-per-pixel bitmap image format using a palette of up to 256 distinct colors; lossless but unsuitable for images with continuous color; multiple images in one file animations Portable Network Graphics (.png): bitmap image format with lossless data compression (patent-free successor of GIF); no animations, incompatible with early IE Tagged Image File Format (.tif(f)): widely supported lossless format; can handle multiple images and data in a single file

106 Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Figure: More and more lossy...

107 Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group Figure: More and more lossy...

108 Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg Figure: More and more lossy...

109 Digitizing media content Common image file formats From images to computer data Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg compression method and (commonly) file format, too but other formats can carry jpeg (eg. TIFF) Figure: More and more lossy...

110 Digitizing media content Common image file formats From images to computer data Figure: More and more lossy... Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg compression method and (commonly) file format, too but other formats can carry jpeg (eg. TIFF) most used for storing and transmitting photographs on the Web

111 Digitizing media content Common image file formats From images to computer data Figure: More and more lossy... Lossy compressed JPEG Joint Photographic Experts Group.jpg,.jpeg compression method and (commonly) file format, too but other formats can carry jpeg (eg. TIFF) most used for storing and transmitting photographs on the Web JPEG2000: much improved successor of jpeg but not in common use yet

112 world The of color

113 The world of color Outline Color models 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

114 The world of color Color models Color models Color model A model describing the way colors can be represented as tuples of numbers (typically three or four values or color components). It is associated with a precise description of how the components should be interpreted color space.

115 The world of color RGB color model Color models

116 The world of color RGB color model Color models Red, green, blue

117 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness)

118 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors):

119 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science

120 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science as percentages, from 0% (minimum) to 100% (maximum) (full intensity red is???)

121 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science as percentages, from 0% (minimum) to 100% (maximum) (full intensity red is 100%, 0%, 0%) in the range 0 to 255 (full intensity red is???)

122 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science as percentages, from 0% (minimum) to 100% (maximum) (full intensity red is 100%, 0%, 0%) in the range 0 to 255 (full intensity red is 255,0,0) same in hexadecimal (full intensity red is???);

123 The world of color RGB color model Color models Red, green, blue uses additive color mixing, describes what kind of light needs to be emitted to produce a given color (out of darkness) representations (color vectors): in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red is 1.0, 0.0, 0.0); used in color science as percentages, from 0% (minimum) to 100% (maximum) (full intensity red is 100%, 0%, 0%) in the range 0 to 255 (full intensity red is 255,0,0) same in hexadecimal (full intensity red is #ff,#00,#00); contracted to #ff0000 if used as web colors

124 The world of color Truecolor 24 bit representation Color models yellow (255,255,0) green (0,255,0) cyan (0,255,255) red (255,0,0) blue (0,0,255) red (255,0,0) magenta (255,0,255)

125 The world of color CMYK color model Color models Cyan, magenta, yellow, key

126 The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed.

127 The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed. magenta + yellow = red magenta + cyan = blue cyan + yellow = green

128 The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed. magenta + yellow = red magenta + cyan = blue cyan + yellow = green the use of black: produce pure black (without mixing three colors, which is not perfect and expensive)

129 The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed. magenta + yellow = red magenta + cyan = blue cyan + yellow = green the use of black: produce pure black (without mixing three colors, which is not perfect and expensive) no complete equivalance between RGB and CMYK

130 The world of color CMYK color model Color models Cyan, magenta, yellow, key uses subtractive colors in terms of light absorption. The colors that are seen are from the part of light that is not absorbed. magenta + yellow = red magenta + cyan = blue cyan + yellow = green the use of black: produce pure black (without mixing three colors, which is not perfect and expensive) no complete equivalance between RGB and CMYK representation: vector with components

131 The world of color Outline Image models 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

132 The world of color Raster graphics image Image models Bitmap Figure: RGB bitmap

133 The world of color Raster graphics image Image models Bitmap data file or structure representing a rectangular grid of pixels (points of color) Figure: RGB bitmap

134 The world of color Raster graphics image Image models Bitmap data file or structure representing a rectangular grid of pixels (points of color) color of each pixel is individually defined Figure: RGB bitmap

135 The world of color Raster graphics image Image models Bitmap data file or structure representing a rectangular grid of pixels (points of color) color of each pixel is individually defined corresponds bit for bit with an image displayed on eg. screen Figure: RGB bitmap

136 The world of color Raster graphics image Image models Figure: RGB bitmap Bitmap data file or structure representing a rectangular grid of pixels (points of color) color of each pixel is individually defined corresponds bit for bit with an image displayed on eg. screen characterized by the width and height of the image in pixels and the number of bits per pixel

137 The world of color Vector graphics Image models Figure: Vector scales well

138 The world of color Vector graphics Image models image is represented through the use of geometrical primitives such as points, lines, curves, and polygons Figure: Vector scales well

139 The world of color Vector graphics Image models image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display Figure: Vector scales well

140 The world of color Vector graphics Image models image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display minimal amount of information results in small file size Figure: Vector scales well

141 The world of color Vector graphics Image models Figure: Vector scales well image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display minimal amount of information results in small file size moving, scaling, rotating, filling etc. doesn t degrade the quality of a drawing

142 The world of color Vector graphics Image models Figure: Vector scales well image is represented through the use of geometrical primitives such as points, lines, curves, and polygons mainly used today in the context of two-dimensional computer graphics to create an image on a raster display minimal amount of information results in small file size moving, scaling, rotating, filling etc. doesn t degrade the quality of a drawing indefinite zoom in (cf. image b (vector) and image c (raster))

143 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed

144 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization.

145 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video

146 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data

147 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling

148 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution)

149 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution) compression (lossy/lossless) codecs

150 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution) compression (lossy/lossless) codecs storage container files/formats

151 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution) compression (lossy/lossless) codecs storage container files/formats Colors

152 Recap Recap Whatever the source of various signals, for computers a binary (bitwise) representation is needed digitization. Sound, video continuous (analog) signal binary data sampling quantization (resolution) compression (lossy/lossless) codecs storage container files/formats Colors binary representation of the color spectrum: RGB, CMYK

153 Recap Recap Images

154 Recap Recap Images representation

155 Recap Recap Images representation raster

156 Recap Recap Images representation raster vector

157 Recap Recap Images representation raster vector storage/file format lossless lossy compressed uncompressed

158 Images Outline Digital image editing 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

159 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors).

160 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors

161 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors selection: selecting part(s) of an image, so applying a change selectively without affecting the entire picture; complex means (eg. edge detection, color-based, lasso)

162 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors selection: selecting part(s) of an image, so applying a change selectively without affecting the entire picture; complex means (eg. edge detection, color-based, lasso) layers: transparent sheets stacked on top of each other, each capable of being individually edited

163 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors selection: selecting part(s) of an image, so applying a change selectively without affecting the entire picture; complex means (eg. edge detection, color-based, lasso) layers: transparent sheets stacked on top of each other, each capable of being individually edited size alteration: scaling of images

164 Images Digital image editing Digital image editing Editing The process of altering digital images (for home use, typically digital photographs) by means of specialised software programs (graphics editors). Main features of editors selection: selecting part(s) of an image, so applying a change selectively without affecting the entire picture; complex means (eg. edge detection, color-based, lasso) layers: transparent sheets stacked on top of each other, each capable of being individually edited size alteration: scaling of images cropping: creating a new image by removing a desired rectangular portion from the image being cropped

165 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image

166 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image

167 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image

168 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color

169 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror

170 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture

171 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture special effects

172 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture special effects contrast change and brightening: improving an underexposed image

173 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture special effects contrast change and brightening: improving an underexposed image sharpening and softening images (eliminating red-eye effect)

174 Images Digital image editing Digital image editing Main features of editors noise: adding or removing noise in an image cloning: removing unwanted elements from the image selective color change: changing the color of specific items in an image image gradient: gradual blend of color orientation: rotation and mirror merging: merging one or more individual images into a single picture special effects contrast change and brightening: improving an underexposed image sharpening and softening images (eliminating red-eye effect) color adjustments?

175 Images Outline Digital photography 1 Introduction Sample exam question 2 devices Internal devices External devices 3 Digitizing media content AD conversion From sound to computer data From images to computer data 4 The world of color Color models Image models 5 Recap 6 Images Digital image editing Digital photography 7 on the Internet Transmitting content Sharing media content 8 applications Video players Audio players Image editors Software galore

176 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.?

177 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB)

178 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB) Exif Exchangeable image file format

179 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB) Exif Exchangeable image file format specification for the image file format used by digital cameras

180 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB) Exif Exchangeable image file format specification for the image file format used by digital cameras TIFF for the highest quality format and JPEG as a space-saving but lower quality format; (RAW: proprietary, minimally processed format, not standardized, plugins needed for image editors to handle these files)

181 Images Digital cameras Digital photography Connectivity USB port, as USB mass storage device. FireWire, Bluetooth or Wifi also possible.? Storage Memory cards: CompactFlash, Memory stick, SD card (up to 8GB) Exif Exchangeable image file format specification for the image file format used by digital cameras TIFF for the highest quality format and JPEG as a space-saving but lower quality format; (RAW: proprietary, minimally processed format, not standardized, plugins needed for image editors to handle these files) metadata tags in images: date and time; camera settings; thumbnail; descriptions and copyright information (normally viewable in graphical file managers?)

182 Images Exif data Digital photography Figure: Exif data in Nautilus file manager