Multimedia Systems Speech I Mahdi Amiri September 2015 Sharif University of Technology

Transcription

1 Course Presentation Multimedia Systems Speech I Mahdi Amiri September 215 Sharif University of Technology

2 Sound Sound is a sequence of waves of pressure which propagates through compressible media such as air or water. Digital representation of an analog signal Sampling Quantization Parameters: Sampling Rate (Samples per Second) Quantization Levels (Bits per Sample) This is a form of coding too: Basics Pulse-code modulation (PCM) Page 1

3 Pulse-code Modulation (PCM) Why Call it PCM? 4-bit PCM Page 2

4 Audio: Sampling & Quantization How to choose proper Sampling Rate: 8 Khz? Quantization Level: 8 bit/sample? Low Sampling rate: Aliasing, Low qualityin reconstruction High Sampling rate: Data redundancy, High storage, High processing power consumption Low Quantization Levels: Large Q. noise, low SNR. High Quantization levels: More bits required, High storage, High processing power consumption Bit per Second for 8 Hz 8 bit PCM 64 kbit/s Bit per Second (bit/s or bps) Page 3

5 Supplementary Audio, Sampling Aliasing due to low sampling Properly sampled image of brick wall. Spatial aliasing in the form of a Moiré pattern. Two different sinusoids that fit the same set of samples. Page 4

6 Audio, Sampling Sampling Rate Human Hearing Frequency Range 2 Hz to 2 khz Most people will find that their hearing is most sensitive around 1-4 khz and that it is less sensitive at high and low frequencies. Play with Audacity tone generator to test your hearing audacity.sourceforge.net Page 5

7 Audio, Sampling Test your own hearing range 3 Sec. tones with different frequencies. 1 Hz 2 Hz 3 Hz 4 Hz 5 Hz 75 Hz 1 Hz 2 Hz 5 Hz 1 KHz 2 KHz 4 KHz 8 KHz 12 KHz 14 KHz 15 KHz 16 KHz 18 KHz Page 6

8 Audio, Sampling Hearing Range Ferret Porpoise Page 7

9 Frequency Allocations Radio Frequency Bands Also see "Communications Regulatory Authority of The I.R of Iran" 9 KHz thr. 3 GHz AM Radio 535 KHz thr. 1.6 MHz FM Radio 88 MHz thr. 18 MHz TV Various bands from 54 MHz thr. 7 MHz United States radio spectrum frequency allocations chart as of 211. GSM (Global System for Mobile Communications) Mostly 9 MHz and 18 MHZ Page 8

10 Modulation AM, FM, PM Signal (or message): Carrier: Amplitude Modulation (AM) conveys information over a carrier wave (the transmitted signal) by varying the amplitude (strength) of the carrier in relation to the information being sent ( carrier's frequency remains constant). Frequency Modulation (FM) works by varying carrier's instantaneous frequency. A signal may be carried by an AM or FM radio wave. Phase Modulation (PM) represents information as variations in the instantaneous phase of a carrier wave. PM Example and PM Page 9

11 Audio, Sampling Sampling Rate Human Vocal Range Normal: 8 Hz to 11 Hz Guinness Book of Records Female: Georgia Brown Eight octaves G2 ( Hz) thr. G1 ( Hz) Play and see! Low freq. voice of Tim Storms, as you may can t hear it. Male: Tim Storms Ten octaves (.7973 Hz thr Hz) Octave: In music, an octave is the interval between one musical pitch and another with half or double its frequency. Page 1

12 Audio, Sampling 8, Hz - Telephone, adequate for human speech. 11,25 Hz lower quality PCM (one quarter the sampling rate of audio CDs). 22,5 Hz Radio. Common Sampling Rates 32, Hz - minidv digital video camcorder, DAT (LP mode). 44,1 Hz -Audio CD, also most commonly used with MPEG-1 audio (VCD, SVCD, MP3) (Originally chosen by Sony, 1979). 48, Hz -Digital sound used for minidv, digital TV, DVD, DAT, films and professional audio. 96, or 192, Hz -DVD-Audio, some LPCM DVD tracks, BD-ROM (Blu-ray Disc) audio tracks, and HD-DVD (High-Definition DVD) audio tracks MHz-Super Audio CD (SACD), 1-bit sigma-delta modulation process known as Direct Stream Digital (DSD), co-developed by Sony and Philips MHz -Double-Rate DSD, 1-bit Direct Stream Digital at 2x the rate of the SACD. Used in some professional DSD recorders (128 * 441 Hz). DXD -24-bit sampled at khz, suited for editing, eq. with MHz 1-bit DSD Page 11

13 Audio, Sampling Why 44.1 khz? The rate was chosen following debate between manufacturers, notably Sony and Philips, and its implementation by Sony, yielding a de facto standard. The technical reasoning behind the rate being chosen is as follows: Firstly, because the hearing range of human ears is roughly 2 Hz to 2, Hz, and via the Nyquist Shannon sampling theoremthe sampling frequency must be greater than twice the maximum frequency one wishes to reproduce, the sampling rate therefore had to be greater than 4 khz. In addition to this, signals must be low-pass filtered before sampling, otherwise aliasing occurs, and, while an ideal low-pass filter would perfectly pass frequencies below 2 khz (without attenuating them) and perfectly cut off frequencies above 2 khz, in practice a transition band is necessary, where frequencies are partly attenuated. The wider this transition band is, the easier and more economical it is to make an anti-aliasing filter. The 44.1 khz sampling frequency allows for a 2.5 khz transition band. Page 12 Ref.: en.wikipedia.org/wiki/44,1_hz

14 Pulse-code Modulation (PCM) The trademark name used by Sony and Philips. Uses pulse-density modulation encoding Direct Stream Digital Page 13

15 Audio, Quantization Quantization, Audio and Images Page 14

16 Audio, Quantization Simple and popular Midtread Odd number of reconstruction levels (N) (quantizing levels) Here N = 9 Uniform Quantizer, Midtread Page 15

17 Audio, Quantization Simple and popular Midrise Even number of reconstruction levels Here N = 8 Uniform Quantizer, Midrise Quantization error for bounded input Page 16

18 Audio, Quantization Quantization Levels, SNR in db Want to prevent human ear fatigue by minimizing quantization noise Signal-to-Noise Ratio (SNR) = 6.2*B db SNR is approximately 6 db per bit. 16-bit => 96 db Above 36 db is required Horizontal axis: Power ratio in linear scale Vertical axis: Power ratio in db Page 17

19 Audio, Quantization db in SNR not to be confused with db in SPL SPL: A logarithmic measure of the effective sound pressure of a sound relative to a reference value. db in Sound Pressure Level (SPL) The commonly used reference sound pressure: 2 micropascals (µpa) (roughly the sound of a mosquito flying 3 m away) Sound SPL (Pa) SPL(dB) Jet engine at 3 m Passenger car at 1 m Pa 6 8 Normal conversation at 1 m Pa 4 6 Calm breathing Pa 1 en.wikipedia.org/wiki/sound_pressure Sound level meter Quantifying sound pressure level in db Page 18

20 Audio, Quantization The human ear responds more to frequencies between 5 Hz and 8 khz and is less sensitive to very low-pitch or high-pitch noises. The frequency weightings used in sound level meters are often related to the response of the human ear, to ensure that the meter is measuring pretty much what you actually hear. dba in Sound Pressure Level (SPL) A-Weighted frequency response dbz: means no weighting at all Page 19

21 Supplementary Audio, Quantization VSLM Virtual Sound Level Meter (VSLM) The MATLAB development of a virtual sound level meter for analyzing calibrated sound files. Ref.: Page 2

22 .8 Audio, Quantization Text: A lathe is a big tool. Grab every dish of sugar. Quantization Levels, SNR in db Click to play original sound x db db 5 db 6 db 1 db 2 db 7 db 8 db 3 db 4 db Sample output of SpeechNoise_T3.m (Play noisy speech with different SNR values) 9 db Page 21

23 Audio, Quantization [ n] = ˆ[ n] [ n] e x x m [ ] X < x n < X [ ] < e n 2 2 [ ] m 2X = B 2 Assumption: e n is uniform over (, ] 2 2 The probability density function of e[n] m 6 db per bit rule of thump Average power of a process or signal: + ( x ) ( ) x 2 2 p x dx x µ = σ µ x: Mean ( ) SNR B db p x : Probability density function 2 2 ( ) = 1 log 1 ( σx σe) : Variance 1 X m e = ( e e) p( e) de= e de = = 2B 2 2 σ µ 2B 2 2 db( ) = 1log1( 2 3σx Xm) 2 2 ( 2log12) B 1log1( 3σ x Xm) SNR B = + µ = ( ) ( ) ( 2 2 = log ) 1 3σ SNR B B X db x m e 1 p e = Page 22

24 Audio, Quantization 6 db per bit rule of thump ( ) ( 2 2 = log ) 1 3σ SNR B B X db x m Example 1: SNR for Uniform Quantization of Uniformly-Distributed Input X 2 2 x m 3 σ = SNR ( B) = 6.2 Example 2: SNR for Uniform Quantization of Sinusoidal Input X 2 2 x m 2 σ = SNR ( B) = 6.2B db db B Example 3: SNR for Uniform Quantization of Gaussian Input 2 2 x Xm 16 σ = SNR ( B) = 6.2B 7.27 db Page 23

25 Audio, Bit-rate Good to Know The average person cannot tell the difference between a bitrateabove 192 kbit/s and the original CD/WAV. Even if your headphones seal really well around your ears, they will probably only give you about 2 to 25 db insulation from the external sound Noise level for 192 kbps audio is under -125 db and certainly inaudible Meaning of this db: Noise power after coding and decoding over original signal power in logarithmic scale. Page 24 2 ~ 25 db insulation

26 Audio, Quantization What is a Histogram? Histogram: To roughly assess the probability distribution of a given variable by depicting the frequencies of observations occurring in certain ranges of values. Sound Signal Histogram Histogram of the set {1,2,2,3,3,3,3,4,4,5,6} The shape of the graph gives us an idea of how the numbers in the set are distributed. Histogram of the set {3, 11, 12, 19, 22, 23, 24, 25, 27, 29, 35, 36, 37, 45, 49} with a binwidth of 1 Wonder how the histogram of a typical sound signal will look like? Is it uniform? Data Range (bin) Frequency Page 25

27 Audio, Quantization Original sound 1 See SpeechHistT1.m Typical Speech Signal Waveform x Page

28 2 1.5 Audio, Quantization x bins Typical Speech Signal Histogram x bins figure; hist( x, 256 ); axis([-1 1 -inf inf]) bins bins Page 27

29 aq_partition = [ ]; aq_codebook = [ ]; [aindex, aquantsa ] = quantiz( acura, aq_partition, aq_codebook ); Audio, Quantization 1.5 Original (177:182) Uniform Quantizer, Midrise Original sound See SpeechQuantizationT1.m bit Quantized Quantized signal Original signal Quantized signal Page

30 aq_partition = [ ]; aq_codebook = [ ]; Audio, Quantization 1.5 Original (177:182) Uniform Quantizer, Midtread Original sound See SpeechQuantizationT2.m bit Quantized Quantized signal Original signal Quantized signal Page

31 Audio, Quantization Nonuniform Quantization Typical speech signal and its histogram (a) Uniform and (b) non-uniform quantization Q(x) and quantization error q(x) Page 3

32 Audio, Quantization 1.5 aq_partition = [ ]; aq_codebook = [ ]; Original (177:182) Uniform Quantizer, Midtread Original sound * Deliberately repeated slide See SpeechQuantizationT2.m bit Quantized Quantized signal Original signal Quantized signal Page

33 Audio, Quantization 1.5 Original (177:182) aq_partition = [ ]; aq_codebook = [ ]; Non-Uniform Quantizer Original sound Play and compare quantized speech in MATLAB See SpeechQuantizationT3.m bit Quantized Quantized signal Original signal Quantized signal Page

34 .8.6 Audio, Quantization Click to play original sound x 1 Text: A lathe is a big tool. Grab every dish of sugar. 4 Quantization Levels, SNR in db UniQ_3_bit_MidRise_snr_2.971_dB.au UniQ_3_bit_MidTread_snr_14.878_dB.au UniQ_4_bit_MidRise_snr_ _dB.au UniQ_4_bit_MidTread_snr_25.756_dB.au UniQ_5_bit_MidRise_snr_ _dB.au UniQ_5_bit_MidTread_snr_36.167_dB.au NonUniQ_3_bit_snr_ _dB.au Sample output of SpeechQuantizationT1.m thr. SpeechQuantizationT3.m (Uniform and Nonuniform Speech Quantization) Page 33

35 Audio, Quantization u-law, a-law Nonuniform quantizers: Difficult to make, Expensive. Solution: Companding Uniform Q. Expanding Page 34

36 Audio, Quantization u-law, a-law Page 35

37 Audio, Quantization u-law, a-law u-law North America and Japan a-law Europe Page 36

38 Homework Original Sound Speech Quantization Companding parameter (µ) Compander Page 37 Quantization bit No. Uniform Quantizer Dequantizer Expander µ-law encoded sound SNR Calculation Plot and Play MATLAB code or GUI implementation (Take a look at Speech noise test MATLAB codes to have sample input signal and to find out more about how to plot and play the sounds. Make a plot and show how SNR changes with different values for Mu and B. - +

39 Human Auditory System How ear works: Ear structure Page 38

40 Human Auditory System How ear works: Filter banks Inside the cochlea. Play How the ear works animation. Page 39 Filter bank frequencies on the cochlea.

41 Multimedia Systems Speech I Thank You Next Session: Speech II FIND OUT MORE AT Page 4