Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig
|
|
- Andrew Wilkerson
- 6 years ago
- Views:
Transcription
1 Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig
2 6 Audio Retrieval 6 Audio Retrieval 6.1 Basics of Audio Data 6.2 Audio Information in Databases 6.3 Audio Retrieval Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 2
3 6.1 Basics of Audio Data Information transfer through sound Audio (Latin, "I hear") Three different types of data: Music Spoken text Noise Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 3
4 6.1 Basics Auditory perception through pressure fluctuations in the air Eardrum vibrates synchronously Ear bones amplify, direct Auditory hair cells in the ear cochlea, are stimulated by the vibrations Neurons produce electrical impulses Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 4
5 6.1 Basics 3D model of the human ear Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 5
6 6.1 Basics Our brain only interprets two major properties of sound: Pitch Volume Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 6
7 6.1 Basics Quantitative performance of the sound wave Amplitude as volume Logarithmic perception (tenfold increase in amplitude doubles the perceived loudness) Frequency as pitch Number of periods per unit time is known as frequency (measured in hertz) Hearing range between 20 Hz and 20 khz Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 7
8 6.1 Basics Audio signals are time-dependent (overlapping) waveforms Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 8
9 6.1 Basics Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 9
10 6.1 Basics Constructive and destructive interference Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 10
11 6.1 Basics Audio examples Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 11
12 6.1 Sound Creation Musical instruments are for vibration exciter arranged E.g., string-, blowing-, percussion Acoustics depends on the vibration generator E.g., strings-, air, membrane instruments Synthetic creation needs an oscillator The oscillator generates voltage oscillations Speaker transmit the voltage changes on a membrane Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 12
13 6.1 Sound Creation Influence of the oscillator Higher voltage Higher frequency (Moog, 1964) Amplifier influences the amplitude thus the volume ADSR (attack-decay-sustain-release) envelope influences the loudness of a sound in time Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 13
14 6.1 Sound Creation Moog 901B (1964) Modular Moog Synthesizer (1967) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 14
15 6.1 Sound Creation Emerson, Lake & Palmer: The Great Gates of Kiev (1974) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 15
16 6.1 ADSR envelope For the production of a single sound synthesizer through four phases are typical: Attack: speed and strength of the signal rise Decay: lowering the level Sustain: actual pitch Release: end of the signal Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 16
17 6.1 Digitalization of Audio Data Transformation of the continuous sound wave into a discrete representation Sampling: save at regular intervals, the current amplitude value of vibration Clearly, we have to reconstruct audio signals from these values Amplitude Time Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 17
18 6.1 Sampling Basic characteristics Sampling rate: how many times per unit time is the value of the continuous signal tapped? Resolution: which accuracy are the values recorded with? Often, a resolution of 16 bits is used (2 16 different amplitude values) The sampling rate is application dependent: Audio CD: Hz Phone: 8000 Hz Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 18
19 6.1 Sampling Rate The clear recoverability of the oscillation is very important The higher the sampling frequency, the more values must be saved Minimum sampling frequency? Sampling theorem (Nyquist, 1928) Sampling rate must be at least twice as large as the highest frequency occurring in the signal Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 19
20 6.1 Sampling Rate Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 20
21 6.1 Sampling Rate Phone: 8000 Hz DVD audio: 96,000 Hz or 192,000 Hz Audio CD:44,100 measurements per second for two stereo channels with 16 bits per measurement results in kb/s ca 10 MB/min, i.e., 635 MB/h Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 21
22 6.1 Audio Formats For space reasons, is digital data usually stored in compressed form Known uncompressed formats: AIFF: *. aif (Apple Inter opportunity File Format) Wave: *. wav (Windows) IRCAM: *. snd (Institut de Recherche et Coordination Acoustique / Musique) AU: *. au (Sun audio) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 22
23 6.1 Compression Data reduction, usually with lossy algorithms There are also lossless compression methods, but they generally do not compress very much Lossless Audio (LA) Apple Lossless WavPack... Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 23
24 6.1 Compression Lossy compression algorithms typically are based on simple transformations Modified Discrete Cosine Transformation (MDCT) or wavelets Encoding: transformation of the waveform in frequency sequences (sampling) Decoding: Reconstruction of waveform from these values Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 24
25 6.1 Compression Change of the data without changing the subjective perception Omit very high/low frequencies Save superimposed frequencies with less precision Use of other effects according to psychoacoustic model, e.g., low tones before/after very loud sounds and frequency changes at a minimum distance are impossible to hear... Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 25
26 6.1 Compression MPEG-1 Audio Layers I, II and III (MP3) CD quality at bit rates of 128 kb/s Coarse approach to MP3 Channel coupling of the stereo signal by using the difference Cut off inaudible frequencies and all frequencies > 2 sampling rate (sampling theorem) Eliminate redundancy by considering the psychoacoustic effects Compress data using Huffman coding Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 26
27 6.1 Compression AAC (Advanced Audio Coding) Industry-driven improvement of the MP3 format (supported by the MPEG) Used in TV-/radio broadcasts, Apple itunes Store,... Better quality at same file size Support for multi-channel audio Supports 48 main sound channels with up to 96 khz sampling rate, 16 low-frequency channels (limited to 120 Hz) and 15 data flows Ogg Vorbis, Real Audio, WMA 9,... Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 27
28 6.1 Compression Lossless compression Lossy compression Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 28
29 6.1 Compression Lossless compression, test procedure Rip N audio CDs with different music types Encode the resulting file Measure speed and compression rate Decode the encoded version Measure speed and compression rate Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 29
30 6.1 Compression Lossless compression, results Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 30
31 6.1 Compression Lossy compression, test procedure Rip N audio CDs with different music types Compress with different methods Measure the quality of the compressed sound file Use Mean Opinion Score (MOS) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 31
32 6.1 Compression Lossy compression, results Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 32
33 6.1 Compression Lossy compression, results Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 33
34 6.1 The MIDI Format Communication protocol For transmission, recording and playing musical control information between digital instruments and the PC Developed since 1983 by Sequential Circuits and Roland Statements are not sounds, but commands that can be used e.g., by sound cards Some commands: Note-on, note-off, key velocity, pitch, type of instrument Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 34
35 6.1 The MIDI Format 10 minutes music are about 200 KB of MIDI data Significant savings compared to sampling, but no original sound Data are inputted to the PC via keyboard and outputted via synthesizer Sequencer for caching data and changes Audio score can be automatically generated Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 35
36 6.2 Audio Information in Databases Audio data Music, CDs Sound effects, Earcons Audio data represent most of information transfer Storage of historical speeches Recordings of conversations, phone calls or negotiations Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 36
37 6.2 Special Applications Three main applications of audio signals in the context of databases Identification of audio signals (audio as query) Classification and search of similar signals (matching of audio) Phonetic synchronization Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 37
38 6.2 Identification of Audio Signals Find the title, etc. for this music piece Monitoring of audio streams Control of broadcasting of advertisements on radio Copyright Control (GEMA) (Remote) diagnosis based on noise Audio on Demand Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 38
39 6.2 Classification and Matching Find perceptionally similar audio signals E.g., similar pieces of music, the same quotation,... Recommendation E.g., bands with similar music Genre classification (rock, classical,...) E.g., in audio libraries Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 39
40 6.2 Synchronization Synchronization of audio streams Language text, notes audio,... Retrieval of text from or to speech Find specific points in a speech Verbal query to text documents Following of audio scores in concerts, etc. Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 40
41 6.2 State of the Art Identification The simplest of the three problems, in recent years, successful research Classification and Matching Often still manual annotations Automatic classification only works roughly, on small collections Matching is still largely unresolved Synchronization Meanwhile, tolerable error rates (language text) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 41
42 6.2 Persistent Storage (Compressed) audio files are stored in the database as (smart) BLOBs Additionally, are metadata and feature vectors stored for the realization of the search functionality Language: transcription as text Music: musical notation or MIDI Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 42
43 6.3 Audio Retrieval Search in audio data: metadata describe the audio file Semantically: difficult to generate title, artist, speaker, keywords,... File information: can be automatically generated e.g., time/place of recording, filename, file size,... Widely used, e.g., music exchange markets, online shops,... Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 43
44 6.3 Metadata-based Search Manual indexing is extremely labor intensive and expensive Information is often incomplete, partial and subjective (e.g., genre classification) No possibility to Query by Example ( "Sounds like...") Search only with SQL, approximate string search, etc. Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 44
45 6.3 Content-based Search Using content of audio files Compare measure vs., measure Not very promising, inefficient Differences in sampling rate and resolution Sounds can be differentiated by certain characteristics Low Level Features Logical Features Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 45
46 6.3 Low Level Features Acoustic features Same basic idea as in image databases Description of signal information by means of characteristic features In contrast to image information we don t use a single feature vector, but a time-dependent vector function Time-point of the acoustic characteristics, rather than being contained in the audio file Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 46
47 6.3 Low Level Features Typical Low Level Features Mean amplitude, loudness Frequency distribution Pitch Brightness Bandwidth Measured in the... Time domain (amplitude versus time)... Frequency domain (intensity versus frequency) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 47
48 6.3 Features in the Time Domain Amplitude Pressure fluctuations around the zero point Silence is equivalent to 0 amplitude Average energy Characterizes the volume of the signal with N as the total number of measurements and x n as n th measurement Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 48
49 6.3 Features in the Time Domain Zero-Crossing Rate Frequency of the sign change in the signal with sgn as a sign function (signum) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 49
50 6.3 Features in the Time Domain Silence Ratio Proportion of values that belong to a period of complete silence We must first establish: The amplitude value below which a pitch is considered to be silence The minimum number of consecutive readings that need to be silent, to form a silence period Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 50
51 6.3 Features in the Frequency Domain Fourier transform of the signal Decomposition into frequency components with coefficients (Fourier coefficients) Representation of frequency spectrum of the signal Size of the coefficients of the frequency (represents the amount of energy per frequency) Usually measured in decibels (db) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 51
52 6.3 Frequency Spectrum "Ahhh" sound and Fourier spectrum Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 52
53 6.3 Features in the Frequency Domain Bandwidth Interval of occurring frequencies Difference between the largest and smallest frequency in the spectrum (we consider minimum frequency, only frequencies above the silence threshold) Can also be used for classification, e.g., bandwidth in music is higher than for voice Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 53
54 6.3 Features in the Frequency Domain Power Distribution Can be read directly from the spectrum Distinction of frequencies with high/low energy Calculation of frequency bands with high/low energy Centroid as the center of the spectral energy distribution (brightness) Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 54
55 6.3 Features in the Frequency Domain Harmony The lowest of all the "noisy" frequencies is called the fundamental frequency Harmony of a signal increases when the dominant components in the spectrum are multiples of the fundamental frequency E.g., standard pitch A, as the fundamental frequency (440 Hz) produced on a violin creates harmonic oscillations at 880 Hz, 1320 Hz, 1760 Hz, etc. Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 55
56 6.3 Harmony Harmonic oscillations Frequency spectrum of a sound played on an instrument Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 56
57 6.3 Features in the Frequency Domain Pitch Detectable only for periodic sounds Can be approximated by means of the Fourier spectrum The value is calculated from the frequencies and amplitudes of the peaks Related to the fundamental frequency, which is often used as an approximation Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 57
58 Next lecture Classification and Retrieval of Audio Low level Audio Features Difference Limen Pitch Detection Multimedia Databases Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 58
Digital Media. Daniel Fuller ITEC 2110
Digital Media Daniel Fuller ITEC 2110 Daily Question: Digital Audio What values contribute to the file size of a digital audio file? Email answer to DFullerDailyQuestion@gmail.com Subject Line: ITEC2110-09
More informationCHAPTER 6 Audio compression in practice
CHAPTER 6 Audio compression in practice In earlier chapters we have seen that digital sound is simply an array of numbers, where each number is a measure of the air pressure at a particular time. This
More information2.4 Audio Compression
2.4 Audio Compression 2.4.1 Pulse Code Modulation Audio signals are analog waves. The acoustic perception is determined by the frequency (pitch) and the amplitude (loudness). For storage, processing and
More information5: Music Compression. Music Coding. Mark Handley
5: Music Compression Mark Handley Music Coding LPC-based codecs model the sound source to achieve good compression. Works well for voice. Terrible for music. What if you can t model the source? Model the
More informationPrinciples of Audio Coding
Principles of Audio Coding Topics today Introduction VOCODERS Psychoacoustics Equal-Loudness Curve Frequency Masking Temporal Masking (CSIT 410) 2 Introduction Speech compression algorithm focuses on exploiting
More informationPerceptual Coding. Lossless vs. lossy compression Perceptual models Selecting info to eliminate Quantization and entropy encoding
Perceptual Coding Lossless vs. lossy compression Perceptual models Selecting info to eliminate Quantization and entropy encoding Part II wrap up 6.082 Fall 2006 Perceptual Coding, Slide 1 Lossless vs.
More information08 Sound. Multimedia Systems. Nature of Sound, Store Audio, Sound Editing, MIDI
Multimedia Systems 08 Sound Nature of Sound, Store Audio, Sound Editing, MIDI Imran Ihsan Assistant Professor, Department of Computer Science Air University, Islamabad, Pakistan www.imranihsan.com Lectures
More informationMultimedia Databases. 9 Video Retrieval. 9.1 Hidden Markov Model. 9.1 Hidden Markov Model. 9.1 Evaluation. 9.1 HMM Example 12/18/2009
9 Video Retrieval Multimedia Databases 9 Video Retrieval 9.1 Hidden Markov Models (continued from last lecture) 9.2 Introduction into Video Retrieval Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme
More informationAudio-coding standards
Audio-coding standards The goal is to provide CD-quality audio over telecommunications networks. Almost all CD audio coders are based on the so-called psychoacoustic model of the human auditory system.
More informationMultimedia Databases. Wolf-Tilo Balke Younès Ghammad Institut für Informationssysteme Technische Universität Braunschweig
Multimedia Databases Wolf-Tilo Balke Younès Ghammad Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Previous Lecture Audio Retrieval - Query by Humming
More informationSkill Area 214: Use a Multimedia Software. Software Application (SWA)
Skill Area 214: Use a Multimedia Application (SWA) Skill Area 214: Use a Multimedia 214.4 Produce Audio Files What is digital audio? Audio is another meaning for sound. Digital audio refers to a digital
More informationITNP80: Multimedia! Sound-II!
Sound compression (I) Compression of sound data requires different techniques from those for graphical data Requirements are less stringent than for video data rate for CD-quality audio is much less than
More informationAudio-coding standards
Audio-coding standards The goal is to provide CD-quality audio over telecommunications networks. Almost all CD audio coders are based on the so-called psychoacoustic model of the human auditory system.
More informationPerceptual coding. A psychoacoustic model is used to identify those signals that are influenced by both these effects.
Perceptual coding Both LPC and CELP are used primarily for telephony applications and hence the compression of a speech signal. Perceptual encoders, however, have been designed for the compression of general
More informationCHAPTER 10: SOUND AND VIDEO EDITING
CHAPTER 10: SOUND AND VIDEO EDITING What should you know 1. Edit a sound clip to meet the requirements of its intended application and audience a. trim a sound clip to remove unwanted material b. join
More informationEE482: Digital Signal Processing Applications
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 13 Audio Signal Processing 14/04/01 http://www.ee.unlv.edu/~b1morris/ee482/
More informationBoth LPC and CELP are used primarily for telephony applications and hence the compression of a speech signal.
Perceptual coding Both LPC and CELP are used primarily for telephony applications and hence the compression of a speech signal. Perceptual encoders, however, have been designed for the compression of general
More informationCompressed Audio Demystified by Hendrik Gideonse and Connor Smith. All Rights Reserved.
Compressed Audio Demystified Why Music Producers Need to Care About Compressed Audio Files Download Sales Up CD Sales Down High-Definition hasn t caught on yet Consumers don t seem to care about high fidelity
More informationAudio Fundamentals, Compression Techniques & Standards. Hamid R. Rabiee Mostafa Salehi, Fatemeh Dabiran, Hoda Ayatollahi Spring 2011
Audio Fundamentals, Compression Techniques & Standards Hamid R. Rabiee Mostafa Salehi, Fatemeh Dabiran, Hoda Ayatollahi Spring 2011 Outlines Audio Fundamentals Sampling, digitization, quantization μ-law
More informationMultimedia Databases
Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Previous Lecture Audio Retrieval - Low Level Audio
More informationDigital Audio Basics
CSC 170 Introduction to Computers and Their Applications Lecture #2 Digital Audio Basics Digital Audio Basics Digital audio is music, speech, and other sounds represented in binary format for use in digital
More informationELL 788 Computational Perception & Cognition July November 2015
ELL 788 Computational Perception & Cognition July November 2015 Module 11 Audio Engineering: Perceptual coding Coding and decoding Signal (analog) Encoder Code (Digital) Code (Digital) Decoder Signal (analog)
More informationLecture #3: Digital Music and Sound
Lecture #3: Digital Music and Sound CS106E Spring 2018, Young In this lecture we take a look at how computers represent music and sound. One very important concept we ll come across when studying digital
More informationDigital Audio. Amplitude Analogue signal
Digital Audio The sounds that we hear are air molecules vibrating in a wave pattern. These sound waves are measured by our ear drums and processed in our brain. As computers are digital machines, sound
More informationChapter 14 MPEG Audio Compression
Chapter 14 MPEG Audio Compression 14.1 Psychoacoustics 14.2 MPEG Audio 14.3 Other Commercial Audio Codecs 14.4 The Future: MPEG-7 and MPEG-21 14.5 Further Exploration 1 Li & Drew c Prentice Hall 2003 14.1
More informationChapter 5.5 Audio Programming
Chapter 5.5 Audio Programming Audio Programming Audio in games is more important than ever before 2 Programming Basic Audio Most gaming hardware has similar capabilities (on similar platforms) Mostly programming
More informationMpeg 1 layer 3 (mp3) general overview
Mpeg 1 layer 3 (mp3) general overview 1 Digital Audio! CD Audio:! 16 bit encoding! 2 Channels (Stereo)! 44.1 khz sampling rate 2 * 44.1 khz * 16 bits = 1.41 Mb/s + Overhead (synchronization, error correction,
More informationFundamental of Digital Media Design. Introduction to Audio
Fundamental of Digital Media Design Introduction to Audio by Noraniza Samat Faculty of Computer Systems & Software Engineering noraniza@ump.edu.my OER Fundamental of Digital Media Design by Noraniza Samat
More informationOptical Storage Technology. MPEG Data Compression
Optical Storage Technology MPEG Data Compression MPEG-1 1 Audio Standard Moving Pictures Expert Group (MPEG) was formed in 1988 to devise compression techniques for audio and video. It first devised the
More information3 Sound / Audio. CS 5513 Multimedia Systems Spring 2009 LECTURE. Imran Ihsan Principal Design Consultant
LECTURE 3 Sound / Audio CS 5513 Multimedia Systems Spring 2009 Imran Ihsan Principal Design Consultant OPUSVII www.opuseven.com Faculty of Engineering & Applied Sciences 1. The Nature of Sound Sound is
More informationAudio Coding and MP3
Audio Coding and MP3 contributions by: Torbjørn Ekman What is Sound? Sound waves: 20Hz - 20kHz Speed: 331.3 m/s (air) Wavelength: 165 cm - 1.65 cm 1 Analogue audio frequencies: 20Hz - 20kHz mono: x(t)
More informationIntroducing Audio Signal Processing & Audio Coding. Dr Michael Mason Senior Manager, CE Technology Dolby Australia Pty Ltd
Introducing Audio Signal Processing & Audio Coding Dr Michael Mason Senior Manager, CE Technology Dolby Australia Pty Ltd Overview Audio Signal Processing Applications @ Dolby Audio Signal Processing Basics
More informationMultimedia Communications. Audio coding
Multimedia Communications Audio coding Introduction Lossy compression schemes can be based on source model (e.g., speech compression) or user model (audio coding) Unlike speech, audio signals can be generated
More informationLecture 16 Perceptual Audio Coding
EECS 225D Audio Signal Processing in Humans and Machines Lecture 16 Perceptual Audio Coding 2012-3-14 Professor Nelson Morgan today s lecture by John Lazzaro www.icsi.berkeley.edu/eecs225d/spr12/ Hero
More informationIntroducing Audio Signal Processing & Audio Coding. Dr Michael Mason Snr Staff Eng., Team Lead (Applied Research) Dolby Australia Pty Ltd
Introducing Audio Signal Processing & Audio Coding Dr Michael Mason Snr Staff Eng., Team Lead (Applied Research) Dolby Australia Pty Ltd Introducing Audio Signal Processing & Audio Coding 2013 Dolby Laboratories,
More informationUNDERSTANDING MUSIC & VIDEO FORMATS
ComputerFixed.co.uk Page: 1 Email: info@computerfixed.co.uk UNDERSTANDING MUSIC & VIDEO FORMATS Are you confused with all the different music and video formats available? Do you know the difference between
More informationMultimedia Databases. 8 Audio Retrieval. 8.1 Music Retrieval. 8.1 Statistical Features. 8.1 Music Retrieval. 8.1 Music Retrieval 12/11/2009
8 Audio Retrieval Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 8 Audio Retrieval 8.1 Query by Humming
More informationData Representation. Reminders. Sound What is sound? Interpreting bits to give them meaning. Part 4: Media - Sound, Video, Compression
Data Representation Interpreting bits to give them meaning Part 4: Media -, Video, Compression Notes for CSC 100 - The Beauty and Joy of Computing The University of North Carolina at Greensboro Reminders
More informationFILE CONVERSION AFTERMATH: ANALYSIS OF AUDIO FILE STRUCTURE FORMAT
FILE CONVERSION AFTERMATH: ANALYSIS OF AUDIO FILE STRUCTURE FORMAT Abstract JENNIFER L. SANTOS 1 JASMIN D. NIGUIDULA Technological innovation has brought a massive leap in data processing. As information
More informationFigure 1. Generic Encoder. Window. Spectral Analysis. Psychoacoustic Model. Quantize. Pack Data into Frames. Additional Coding.
Introduction to Digital Audio Compression B. Cavagnolo and J. Bier Berkeley Design Technology, Inc. 2107 Dwight Way, Second Floor Berkeley, CA 94704 (510) 665-1600 info@bdti.com http://www.bdti.com INTRODUCTION
More informationDSP. Presented to the IEEE Central Texas Consultants Network by Sergio Liberman
DSP The Technology Presented to the IEEE Central Texas Consultants Network by Sergio Liberman Abstract The multimedia products that we enjoy today share a common technology backbone: Digital Signal Processing
More informationWorkshop W14 - Audio Gets Smart: Semantic Audio Analysis & Metadata Standards
Workshop W14 - Audio Gets Smart: Semantic Audio Analysis & Metadata Standards Jürgen Herre for Integrated Circuits (FhG-IIS) Erlangen, Germany Jürgen Herre, hrr@iis.fhg.de Page 1 Overview Extracting meaning
More informationSqueeze Play: The State of Ady0 Cmprshn. Scott Selfon Senior Development Lead Xbox Advanced Technology Group Microsoft
Squeeze Play: The State of Ady0 Cmprshn Scott Selfon Senior Development Lead Xbox Advanced Technology Group Microsoft Agenda Why compress? The tools at present Measuring success A glimpse of the future
More informationMultimedia Database Systems. Retrieval by Content
Multimedia Database Systems Retrieval by Content MIR Motivation Large volumes of data world-wide are not only based on text: Satellite images (oil spill), deep space images (NASA) Medical images (X-rays,
More informationData Compression. Audio compression
1 Data Compression Audio compression Outline Basics of Digital Audio 2 Introduction What is sound? Signal-to-Noise Ratio (SNR) Digitization Filtering Sampling and Nyquist Theorem Quantization Synthetic
More informationDigital Recording and Playback
Digital Recording and Playback Digital recording is discrete a sound is stored as a set of discrete values that correspond to the amplitude of the analog wave at particular times Source: http://www.cycling74.com/docs/max5/tutorials/msp-tut/mspdigitalaudio.html
More informationPrinciples of MPEG audio compression
Principles of MPEG audio compression Principy komprese hudebního signálu metodou MPEG Petr Kubíček Abstract The article describes briefly audio data compression. Focus of the article is a MPEG standard,
More informationCISC 7610 Lecture 3 Multimedia data and data formats
CISC 7610 Lecture 3 Multimedia data and data formats Topics: Perceptual limits of multimedia data JPEG encoding of images MPEG encoding of audio MPEG and H.264 encoding of video Multimedia data: Perceptual
More informationExport Audio Mixdown
26 Introduction The function in Cubase Essential allows you to mix down audio from the program to a file on your hard disk. You always mix down an output bus. For example, if you have set up a stereo mix
More informationLossy compression. CSCI 470: Web Science Keith Vertanen
Lossy compression CSCI 470: Web Science Keith Vertanen Digital audio Overview Sampling rate Quan5za5on MPEG audio layer 3 (MP3) JPEG s5ll images Color space conversion, downsampling Discrete Cosine Transform
More informationParametric Coding of High-Quality Audio
Parametric Coding of High-Quality Audio Prof. Dr. Gerald Schuller Fraunhofer IDMT & Ilmenau Technical University Ilmenau, Germany 1 Waveform vs Parametric Waveform Filter-bank approach Mainly exploits
More informationFor Mac and iphone. James McCartney Core Audio Engineer. Eric Allamanche Core Audio Engineer
For Mac and iphone James McCartney Core Audio Engineer Eric Allamanche Core Audio Engineer 2 3 James McCartney Core Audio Engineer 4 Topics About audio representation formats Converting audio Processing
More informationAUDIOVISUAL COMMUNICATION
AUDIOVISUAL COMMUNICATION Laboratory Session: Audio Processing and Coding The objective of this lab session is to get the students familiar with audio processing and coding, notably psychoacoustic analysis
More informationCapturing and Editing Digital Audio *
Digital Media The material in this handout is excerpted from Digital Media Curriculum Primer a work written by Dr. Yue-Ling Wong (ylwong@wfu.edu), Department of Computer Science and Department of Art,
More informationAUDIOVISUAL COMMUNICATION
AUDIOVISUAL COMMUNICATION Laboratory Session: Audio Processing and Coding The objective of this lab session is to get the students familiar with audio processing and coding, notably psychoacoustic analysis
More informationCSCD 443/533 Advanced Networks Fall 2017
CSCD 443/533 Advanced Networks Fall 2017 Lecture 18 Compression of Video and Audio 1 Topics Compression technology Motivation Human attributes make it possible Audio Compression Video Compression Performance
More informationTransporting audio-video. over the Internet
Transporting audio-video over the Internet Key requirements Bit rate requirements Audio requirements Video requirements Delay requirements Jitter Inter-media synchronization On compression... TCP, UDP
More informationMultimedia Databases
Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 0 Organizational Issues Lecture 07.04.2011 14.07.2011
More informationAppendix 4. Audio coding algorithms
Appendix 4. Audio coding algorithms 1 Introduction The main application of audio compression systems is to obtain compact digital representations of high-quality (CD-quality) wideband audio signals. Typically
More informationMPEG-1. Overview of MPEG-1 1 Standard. Introduction to perceptual and entropy codings
MPEG-1 Overview of MPEG-1 1 Standard Introduction to perceptual and entropy codings Contents History Psychoacoustics and perceptual coding Entropy coding MPEG-1 Layer I/II Layer III (MP3) Comparison and
More informationCompression; Error detection & correction
Compression; Error detection & correction compression: squeeze out redundancy to use less memory or use less network bandwidth encode the same information in fewer bits some bits carry no information some
More informationChapter X Sampler Instrument
Chapter X Sampler Instrument A sampler is a synthesizer that generates sound by playing recorded sounds rather than calculated waveforms. The wave file player instrument described in an earlier chapter
More informationCOS 116 The Computational Universe Laboratory 4: Digital Sound and Music
COS 116 The Computational Universe Laboratory 4: Digital Sound and Music In this lab you will learn about digital representations of sound and music, especially focusing on the role played by frequency
More informationMPEG-7 Audio: Tools for Semantic Audio Description and Processing
MPEG-7 Audio: Tools for Semantic Audio Description and Processing Jürgen Herre for Integrated Circuits (FhG-IIS) Erlangen, Germany Jürgen Herre, hrr@iis.fhg.de Page 1 Overview Why semantic description
More informationWhat is multimedia? Multimedia. Continuous media. Most common media types. Continuous media processing. Interactivity. What is multimedia?
Multimedia What is multimedia? Media types +Text + Graphics + Audio +Image +Video Interchange formats What is multimedia? Multimedia = many media User interaction = interactivity Script = time 1 2 Most
More informationElementary Computing CSC 100. M. Cheng, Computer Science
Elementary Computing CSC 100 1 Graphics & Media Scalable Outline & Bit- mapped Fonts Binary Number Representation & Text Pixels, Colors and Resolution Sound & Digital Audio Film & Digital Video Data Compression
More informationFundamentals of Perceptual Audio Encoding. Craig Lewiston HST.723 Lab II 3/23/06
Fundamentals of Perceptual Audio Encoding Craig Lewiston HST.723 Lab II 3/23/06 Goals of Lab Introduction to fundamental principles of digital audio & perceptual audio encoding Learn the basics of psychoacoustic
More informationICT 514 Multimedia Systems Topic 4: Audio
ICT 514 Multimedia Systems Topic 4: Audio Ref: Chapman N. & Chapman J., Digital Multimedia 2 nd Edition Chapter 9 Lance Fung master of science in information technology Overview Objectives of this lecture
More informationAUDIO. Henning Schulzrinne Dept. of Computer Science Columbia University Spring 2015
AUDIO Henning Schulzrinne Dept. of Computer Science Columbia University Spring 2015 Key objectives How do humans generate and process sound? How does digital sound work? How fast do I have to sample audio?
More informationMultimedia Databases. 0. Organizational Issues. 0. Organizational Issues. 0. Organizational Issues. 0. Organizational Issues. 1.
0. Organizational Issues Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Lecture 22.10.2009 04.02.2010
More informationAudio Compression Using Decibel chirp Wavelet in Psycho- Acoustic Model
Audio Compression Using Decibel chirp Wavelet in Psycho- Acoustic Model 1 M. Chinna Rao M.Tech,(Ph.D) Research scholar, JNTUK,kakinada chinnarao.mortha@gmail.com 2 Dr. A.V.S.N. Murthy Professor of Mathematics,
More informationMultimedia. What is multimedia? Media types. Interchange formats. + Text +Graphics +Audio +Image +Video. Petri Vuorimaa 1
Multimedia What is multimedia? Media types + Text +Graphics +Audio +Image +Video Interchange formats Petri Vuorimaa 1 What is multimedia? Multimedia = many media User interaction = interactivity Script
More information9/8/2016. Characteristics of multimedia Various media types
Chapter 1 Introduction to Multimedia Networking CLO1: Define fundamentals of multimedia networking Upon completion of this chapter students should be able to define: 1- Multimedia 2- Multimedia types and
More informationCS 074 The Digital World. Digital Audio
CS 074 The Digital World Digital Audio 1 Digital Audio Waves Hearing Analog Recording of Waves Pulse Code Modulation and Digital Recording CDs, Wave Files, MP3s Editing Wave Files with BinEd 2 Waves A
More informationCOS 116 The Computational Universe Laboratory 4: Digital Sound and Music
COS 116 The Computational Universe Laboratory 4: Digital Sound and Music In this lab you will learn about digital representations of sound and music, especially focusing on the role played by frequency
More informationMPEG-4 Version 2 Audio Workshop: HILN - Parametric Audio Coding
MPEG-4 Version 2 Audio Workshop: HILN - Parametric Audio Coding Heiko Purnhagen Laboratorium für Informationstechnologie University of Hannover, Germany Outline Introduction What is "Parametric Audio Coding"?
More information_APP B_549_10/31/06. Appendix B. Producing for Multimedia and the Web
1-59863-307-4_APP B_549_10/31/06 Appendix B Producing for Multimedia and the Web In addition to enabling regular music production, SONAR includes a number of features to help you create music for multimedia
More informationMPEG-4 Structured Audio Systems
MPEG-4 Structured Audio Systems Mihir Anandpara The University of Texas at Austin anandpar@ece.utexas.edu 1 Abstract The MPEG-4 standard has been proposed to provide high quality audio and video content
More informationModule 9 AUDIO CODING. Version 2 ECE IIT, Kharagpur
Module 9 AUDIO CODING Lesson 29 Transform and Filter banks Instructional Objectives At the end of this lesson, the students should be able to: 1. Define the three layers of MPEG-1 audio coding. 2. Define
More informationNew Results in Low Bit Rate Speech Coding and Bandwidth Extension
Audio Engineering Society Convention Paper Presented at the 121st Convention 2006 October 5 8 San Francisco, CA, USA This convention paper has been reproduced from the author's advance manuscript, without
More informationA Digital Audio Primer
Conversion of Sound Wave to Analog Signal A Digital Audio Primer Many people don t care about the technology behind their stereo system. As long as it sounds good and they can press a button and listen
More informationCS 4455: Video Game Design & Implementation
CS 4455: Video Game Design & Implementation March 31, 2006: Audio (Insert Disclaimer Here) Overview Today s Lecture What I m talking about now Audio Theory Digitizing Sound Game Implementation High Level
More informationEUROPEAN COMPUTER DRIVING LICENCE. Multimedia Audio Editing. Syllabus
EUROPEAN COMPUTER DRIVING LICENCE Multimedia Audio Editing Syllabus Purpose This document details the syllabus for ECDL Multimedia Module 1 Audio Editing. The syllabus describes, through learning outcomes,
More informationCompression Part 2 Lossy Image Compression (JPEG) Norm Zeck
Compression Part 2 Lossy Image Compression (JPEG) General Compression Design Elements 2 Application Application Model Encoder Model Decoder Compression Decompression Models observe that the sensors (image
More informationSpeech-Coding Techniques. Chapter 3
Speech-Coding Techniques Chapter 3 Introduction Efficient speech-coding techniques Advantages for VoIP Digital streams of ones and zeros The lower the bandwidth, the lower the quality RTP payload types
More informationAUDIO MEDIA CHAPTER Background
CHAPTER 3 AUDIO MEDIA 3.1 Background It is important to understand how the various audio software is distributed in order to plan for its use. Today, there are so many audio media formats that sorting
More informationAUDIO COMPRESSION USING WAVELET TRANSFORM
AUDIO COMPRESSION USING WAVELET TRANSFORM Swapnil T. Dumbre Department of electronics, Amrutvahini College of Engineering,Sangamner,India Sheetal S. Gundal Department of electronics, Amrutvahini College
More informationQuick Guide to Getting Started with:
Quick Guide to Getting Started with: 1.0 Introduction -- What is Audacity Audacity is free, open source software for recording and editing sounds. It is a program that manipulates digital audio waveforms.
More informationData Compression. Media Signal Processing, Presentation 2. Presented By: Jahanzeb Farooq Michael Osadebey
Data Compression Media Signal Processing, Presentation 2 Presented By: Jahanzeb Farooq Michael Osadebey What is Data Compression? Definition -Reducing the amount of data required to represent a source
More informationImage and Video Coding I: Fundamentals
Image and Video Coding I: Fundamentals Thomas Wiegand Technische Universität Berlin T. Wiegand (TU Berlin) Image and Video Coding Organization Vorlesung: Donnerstag 10:15-11:45 Raum EN-368 Material: http://www.ic.tu-berlin.de/menue/studium_und_lehre/
More informationMPEG-4 aacplus - Audio coding for today s digital media world
MPEG-4 aacplus - Audio coding for today s digital media world Whitepaper by: Gerald Moser, Coding Technologies November 2005-1 - 1. Introduction Delivering high quality digital broadcast content to consumers
More informationNetworking Applications
Networking Dr. Ayman A. Abdel-Hamid College of Computing and Information Technology Arab Academy for Science & Technology and Maritime Transport Multimedia Multimedia 1 Outline Audio and Video Services
More informationParametric Coding of Spatial Audio
Parametric Coding of Spatial Audio Ph.D. Thesis Christof Faller, September 24, 2004 Thesis advisor: Prof. Martin Vetterli Audiovisual Communications Laboratory, EPFL Lausanne Parametric Coding of Spatial
More informationAAMS Auto Audio Mastering System V3 Manual
AAMS Auto Audio Mastering System V3 Manual As a musician or technician working on music sound material, you need the best sound possible when releasing material to the public. How do you know when audio
More informationProfessor Laurence S. Dooley. School of Computing and Communications Milton Keynes, UK
Professor Laurence S. Dooley School of Computing and Communications Milton Keynes, UK How many bits required? 2.4Mbytes 84Kbytes 9.8Kbytes 50Kbytes Data Information Data and information are NOT the same!
More informationCHAPTER 2 - DIGITAL DATA REPRESENTATION AND NUMBERING SYSTEMS
CHAPTER 2 - DIGITAL DATA REPRESENTATION AND NUMBERING SYSTEMS INTRODUCTION Digital computers use sequences of binary digits (bits) to represent numbers, letters, special symbols, music, pictures, and videos.
More informationProceedings of Meetings on Acoustics
Proceedings of Meetings on Acoustics Volume 19, 213 http://acousticalsociety.org/ ICA 213 Montreal Montreal, Canada 2-7 June 213 Engineering Acoustics Session 2pEAb: Controlling Sound Quality 2pEAb1. Subjective
More informationMultimedia Data and Its Encoding
Lecture 13 Multimedia Data and Its Encoding M. Adnan Quaium Assistant Professor Department of Electrical and Electronic Engineering Ahsanullah University of Science and Technology Room 4A07 Email adnan.eee@aust.edu
More information6MPEG-4 audio coding tools
6MPEG-4 audio coding 6.1. Introduction to MPEG-4 audio MPEG-4 audio [58] is currently one of the most prevalent audio coding standards. It combines many different types of audio coding into one integrated
More information2.1 Transcoding audio files
2.1 Transcoding audio files 2.1.1 Introduction to Transcoding One of the basic tasks you can perform on an audio track is to convert it into another format. This process known as Transcoding, is the direct
More information