Appendix 4. Audio coding algorithms

Transcription

1 Appendix 4. Audio coding algorithms 1 Introduction The main application of audio compression systems is to obtain compact digital representations of high-quality (CD-quality) wideband audio signals. Typically audio signals recorded on CDs and digital audio tapes are sampled at 44.1 or 48 khz and each sample is represented by a 16-bit integer, that is, the uncompressed two-channel stereo CD-quality audio requires 2 41(48) 16=1.41(1.54) b/s for transmission. Unlike speech compression systems the audio codecs process sounds generated by arbitrary sources and they cannot exploit specific features of the input signals. However, almost all modern audio codecs are based on a model of the human auditory system. The key idea behind the so-called perceptual coding is to remove the parts of the input signal which the human cannot perceive. The imperceptible information removed by the perceptual coder is called the irrelevancy. Since, similarly to speech signals, audio signals can be interpreted as outputs of sources with memory then perceptual coders remove both irrelevancy and redundancy in order to provide the lowest bit rate possible for a given quality. An important part of perceptual audio coders is the psychoacoustic model of the human hearing. This model is used in order to estimate the amount of quantization noise that is inaudible. In the next section we consider physical phenomena which are exploited by the psychoacoustic model. 2 Basics of perceptual coding The frequency range which audio coders deal with is roughly from 20 Hz to 20 khz. The human auditory system has remarkable detection capability with a range (called dynamic range) of 120 db from quietest to loudest sounds. However, digitized CD-quality audio signals have a dynamic range of about 96 1

2 db because they use a 16-bit sample representation. The absolute threshold of hearing T q (f) characterizes the amount of energy in a pure tone signal such that it can be detected by a listener in a noiseless environment. This absolute threshold is computed as T q (f) = 3.64(f/1000) e 0.6(f/ ) (f/1000) 4 (1) where f denotes a frequency sample [?]. The threshold is measured as a Sound Pressure Level (SPL), normalized by a minimum value, and expressed in db. The frequency dependence of this threshold is shown in Fig. 1. Using the absolute threshold of hearing in order to estimate the amount of inaudible quantization noise is only the first step in constructing the psychoacoustic model. It turned out that the threshold of hearing at a given frequency depends on the input signal. The reason of that dependency is that louder input sounds can mask or hide weaker ones. The psychoacoustic model takes into account this masking property that occurs when a strong audio signal in the time or frequency domain makes neighboring weaker audio signals imperceptible. The coder analyzes the audio signal and computes the amount of available noise masking as a function of frequency. In other words, the noise detection threshold is a modified version of the absolute threshold T q (f), with its shape determined by the input signal at any given time. In order to estimate this threshold we have first to consider the concept of critical bands. Our inner ear (cochlea) can be interpreted as a bank of highly overlapping nonlinear (level-dependent) bandpass filters. Different areas of the cochlea, each containing a set of neural receptors correspond to different frequency bands called critical bands. The bandwidths of the cochlea filters (critical bandwidths) increase with increasing frequency. They are less than 100 Hz for the lowest audible frequencies and more than 4 khz at the highest frequencies. It is said that the human auditory system has a limited, frequency-dependent resolution. The critical bandwidth is a function of frequency that quantifies the cochlea filter passbands. ore than 60 years ago, due to experiments of H. Fletcher, it was discovered that a critical band is a range of frequencies over which the masking signal-to-noise ratio (SNR) remains more or less constant. For example, H. Fletcher found that a tone at 700 Hz can mask narrow-band noise of less energy in the vicinity of approximately ±70 Hz. If the noise is outside the range 2

3 Sound pressure level (SPL), db f Hz Figure 1: The absolute threshold of hearing in a quiet environment Hz, then to be inaudible it has to have much less energy than the tone. Thus, the critical bandwidth is 140 Hz. The phenomenon of masking has a simple explanation. A strong tone or noise masker present in a given critical band blocks detection of a weaker signal since it creates a strong excitation of the corresponding area of the cochlea. Fig. 2 illustrates masking property. A second observation about masking is that noise and tone have different masking properties. Let B denote a critical-band number and E T and E N be the tone and noise masker energy levels, respectively, for this critical band then the following empirical rules have been observed: and T H N = E T ( B) db (2) T H T = E N K db (3) where T H N and T H T are the noise masking thresholds for tone-maskingnoise and the tone masking threshold for noise-masking-tone types of masking, respectively. The parameter K takes values in the range of 3-6 db. In other words, for the tone-masking-noise case, if the energy of noise (in db) is 3

4 40 Sound pressure level (SPL), db asked sound asker Inaudible sound asking treshold Audible sound Threshold in quiet f, Hz Figure 2: asking of weaker signals by strong tone masker less than T H N, then the noise is imperceptible. For the noise-masking-tone case, if the energy of tone (in db) is less than T H T, then the tone is imperceptible. The problem is that speech and audio signals are neither pure tones nor pure noise but rather a mixture of both. The degree to which a signal, within a critical band, appears more or less tone-like (or noise-like) determines its masking properties. Usually in perceptual coding masking signals are classified as tone or noise and then the thresholds (2) and (3) are computed. The masking property of a given masker in the critical band often spreads to the neighboring critical bands and affects detection thresholds in these critical bands. This effect, also known as the spread of masking, or inter-band masking is modeled in coding applications by a spreading function. Since typically each subband of an audio signal to be encoded contains a collection of maskers of tone and noise types, then the individual masking thresholds computed for each of the maskers and modified by applying a spread function are then combined for each subband into a global masking threshold. The noise spectrum shaped according to the computed global masking threshold is often called Just Noticeable Distortion (JND). The absolute threshold of hearing T q (f) is also considered when shaping the noise 4

5 Cochlea Filter odeling Threshold Estimation Absolute Threshold asking thresholds for subbands Signal-to-mask ratio Tonality estimate Short Term Signal Spectrum Tonality Calculation Estimated Psychoacoustic Threshold Figure 3: Typical psychoacoustic model for audio signal spectra. Let T qmin denote the minimal value of the absolute threshold of hearing in a given subband, then in this subband max {JND, T qmin } is used as the permissible distortion threshold. Exactly how the human auditory system works is still a topic of active research. For the purpose of constructing perceptual audio coders all the considered phenomena are described by formulas and exploited to reduce the bit rate of the perceptual audio coder. Fig. 3 is a block diagram of a typical psychoacoustic model of hearing. The input signal in this diagram is actually the short-term signal spectrum. The cochlea filter model computes the short-term cochlea energy model, i.e., a distribution of the energy along the cochlea. This information is used in two ways. The short-term signal spectrum and the cochlea energy for a given subband are used to compute the tonality of the signal in this subband. Tonality is needed because tones mask noise better than noise masks tones. The cochlea energy plus the tonality information allow us to compute the threshold of audibility for the signal in the subband. This is a spectral estimate indicating what noise level will be audible as a function of frequency. As long as the quantization noise is less than the threshold of audibility at all frequencies, the quantization noise will be imperceptible. Fig. 4 is a diagram for an entire perceptual audio coder. The psychoacoustic model represents one of its blocks. The input signal of the coder is a discrete-time audio signal with sampling rate 32, 44.1, or 48 khz. Each sample is represented by a 16-bit integer value. The input audio stream is split into frames and for each frame transform coefficients are computed. The audio coder is based on transform 5

6 PC audio input Coding filter bank (computing transform coefficients) Bit allocation, Quantization and coding Bitstream formatting Encoded bitstream Psychoacoustic model Encoded bitstream Bitstream unpacking Transform coefficient reconstruction Inverse transform Decoded PC audio Figure 4: Generic perceptual audio coder and decoder coding technique. From the theoretical point of view any kind of transform such as the DFT, the DCT or the so-called odified DCT (DCT), or the transform based on a filter bank can be used in the first block of the coder. In audio standards the audio stream passes through a filter bank that divides the input into multiple frequency subbands. This special type of transform coding is called subband coding [?]. oreover, as will be shown below, some particular cases of subband coding can be implemented as the DCT. The distinguishing feature of all transforms used in audio coding is that they are overlapped transforms. Using nonoverlapped transforms as, for example, in video coding would lead to audible clicks with block frequency. The input audio stream simultaneously passes through a psychoacoustic model that determines the ratio of the signal energy to the masking threshold (SR) for each subband. The quantization and coding block uses the SRs to decide how to allocate the total available number of bits over the subbands to minimize the audibility of the quantization noise. The transform coefficients are then quantized according to the chosen bit allocation based on the perceptual thresholds. Usually uniform scalar quantization is used. The outputs of the quantizer are then further compressed using lossless cod- 6

7 ing, most often a Huffman coder. In recent standards, context arithmetic coding is used instead of Huffman coding. The obtained bitstream contains encoded transform coefficients and side information (bit allocation). The decoder decodes this bitstream, restores the quantized subband values, and reconstructs the audio signal from the subband values. 3 Overview of audio standards The most important standards in this domain were developed by the ISO otion Pictures Experts Group (ISO-PEG) subgroup on high-quality audio. Although perfectly suitable for audio-only applications, PEG Audio is actually one part of a three-part compression standard that also includes video and systems. The original PEG Audio standard was created for mono sound systems and had three layers, each providing greater compression ratio. PEG-2 was created to provide stereo and multichannel audio capability. The PEG-2 Advanced Audio Coder (PEG-AAC) duplicates the performance of PEG-2 at half the bit rate. Recently the AC technique was further enhanced and included to the standard PEG-4 Audio as two efficient coding modes: High Efficiency AAC (HE AAC) and HE-AAC v2. The original PEG-coder is sometimes now referred to as PEG-1. It contains three independent layers of compression. This provides a wide range of trade-offs between code complexity and compressed audio quality. Layer I has the lowest complexity and suits best bit rates above 128 kb/s per channel. For example, Philips Digital Compact Cassette (DCC) developed in 1992 used layer I compression at 192 kb/s per channel. Layer II has an intermediate complexity and bit rates around 128 kb/s per channel. The main application for this layer is audio coding for Digital Audio Broadcasting (DAB) and for Digital Video Brodcasting. It uses a more complicated quantization procedure than layer I. Layer III is the most complex but it provides good audio quality for bit rates around 64 kb/s per channel. It can be used for transmission audio over ISDN channels and storing audio on CD. The widely used P3 player contains a decoder of audio signals compressed according the layer III of the PEG Audio standard. The specific features of this layer are: hybrid transform coding based on the DCT and filter bank coding, a more complicated psychoacoustic model, and the so-called echo-supression system. The variable-length coding of the transform coefficients is used at this layer. 7

8 h ( ) 0 v ( n ) y ( ) 0 0 z 0( n ) g ( ) 0 h1( n ) v1( n ) y1( n ) z1( n ) g1( n ) x( n ) xˆ( n ) h v y 1( n) 1( n) z 1( n) 1( n) g 1( n) Figure 5: -band analysis and synthesis filter banks The digital transform implemented as a polyphase filter bank is common for layers I and II of PEG Audio compression algorithm. This filter bank divides the audio signal into 32 frequency subbands with equal bandwidths. Each subband is 750 Hz wide. Surely, the equal widths of the subbands do not accurately reflect the human auditory system frequency-dependent behavior. Since at lower frequencies critical bandwidth is about 100 Hz, the low-frequency subbands cover more than one critical band. As a result, the critical band with the least noise masking determines the number of bits (number of quantization levels) for the entire subband. At high frequencies, where critical bandwidths are about 4 khz, one critical band covers a few subbands but this circumstance is less critical in the sense of bit allocation. Notice that the polyphase filter bank and its inverse are not always lossless transforms. Even without quantization, the inverse transform sometimes cannot perfectly recover the original signal. However, the error introduced by using the filter bank is negligible and inaudible. Adjacent filter bands in this filter bank have a major frequency overlap. A signal at a single frequency can affect two adjacent filter bank outputs. At each step of the coding procedure = 32 samples of the audio signal enter the -band filter bank as shown in Fig. 5. These samples together with the L = 480 previous samples form an input frame of length N = L + = = 512. Using these N samples the filter bank computes transform coefficients. The output of the ith, i = 0, 1,..., 1, filter is 8

9 computed as v i (n) = N 1 m=0 x(n m)h i (m) (4) where x(n) are the input samples. After decimation by a factor we obtain y i (n) = v i (n) = N 1 m=0 x(n m)h i (m) where h i (n) is the pulse response of the ith analysis filter. In the decoder the quantized transform coefficients ŷ i are upsampled by a factor in order to form the intermediate sequences z i (n) = { ŷi (n/), n = 0,, 2,... 0, otherwise. The obtained values z i (n) are then filtered by the synthesis filter bank where g i (n) = h i (N 1 n), i = 0, 1,..., 1 is the pulse response of the ith synthesis filter. The ISO-PEG coding uses a cosine modulated filter bank. The pulse responses of the analysis and synthesis filters are cosine modulated versions of the pulse response of a lowpass prototype filter, that is, ( ) π(i + 1/2)(m 16) h i (m) = h(m) cos 32 where { w(m), if m/64 is odd h(m) = w(m), otherwise denotes the pulse response of the lowpass prototype filter, w(m), m = 0, 1,..., 511, is the transform window. The pulse response h(n) is multiplied by a cosine term to shift the lowpass pulse response h(n) to the appropriate frequency band. These filters have normalized by f s center frequencies (2i + 1)π/64, i = 0, 1,..., 31, where f s is the sampling frequency. Although the presented form of the filter bank is rather convenient for analysis, it is not efficient from an implementation point of view. A direct implementation of (4) requires =16384 multiplications and =16352 additions to compute the 32 filter outputs. Taking into account 9

10 the periodicity of the cosine function we can obtain the equivalent, but computationally more efficient, representation 63 v i (n) = (i, k) k=0 7 w(k + 64j)x(n (k + 64j)) (5) j=0 where w(k) is one of the 512 coefficients of the transform window, ( ) π(i + 1/2)(k 16) (i, k) = cos 32 i = 0,..., 31, k = 0,..., 63. In order to compute the 32 filter outputs according to (5) we perform only =2560 multiplications and =2464 additions or roughly 80 multiplications and additions per output. The cosine modulated filter bank used by the PEG standard does not correspond to any invertible transform. However, it can be shown that there exist generalized cosine modulated filter banks which provide perfect reconstruction of the input signal. Such filter banks can be implemented more efficiently by reducing the convolution to the DCT which can be implemented using the FFT. The layer III of PEG-1 standard uses the DCT based on DCT-IV. The input of the transform block are blocks of samples x t = (x(n + t), x(n + t + 1),..., x(n + t + 1)) of length each. Together with the (L 1) previous blocks they form a new block s t of length N = L. First, this block is component-wise multiplied by the analysis window w a (n), n = 0, 1,..., N 1. Then by multiplying the obtained vector v t of length N by matrix A of size N (transform matrix), specified below, we obtain the vector y t of transform coefficients: y t = v t A. To perform the inverse transform, first, the vector of transform coefficients y t of length is multiplied by matrix A T : ŝ t = y t A T. Then the obtained vector ŝ t of length N is component-wise multiplied by the synthesis window w s (n), n = 0, 1,..., N 1. The resulting vector ˆv t is 10

11 shifted by positions with respect to the previous block ˆv t 1 and is added to the output sequence of the inverse transform block. Each block ˆx at the output of the decoder is the sum of L shifted blocks of length N. The blocks are overlapped in (L 1) samples. The overlapped analysis and overlapadd synthesis are shown in Fig. 6. The transform matrix A of size N can be written in the form A = Y T where matrix T of size is the matrix of the DCT-IV transform with entries (( 2 t kn = cos k + 1 ) ( n + 1 ) ) π 2 2 k, n = 0, 1,..., 1, matrix Y describes the preprocessing step. It has the following form Y = [Y 0 Y 1 Y 0 Y 1 Y 0 Y 1... ] T where submatrices Y 0 and Y 1 of size in turn consist of submatrices of size /2 /2 : ( ) 0 0 Y 0 = I J ( ) J I Y 1 = 0 0 where I is the identity matrix of size /2 /2 and J is of the same size but the contra-identity matrix which has ones along the second diagonal and its other entries are zero. To complete the description of the modified discrete cosine transform it is necessary to specify the window functions w a (n) and w s (n). The analysis and synthesis windows can be chosen to be equal, w a (n) = w s (n) = w(n). The requirement of the perfect reconstruction can be written as L 2s 1 l=0 w(n + l)w(n + l + 2s) = { 1, s = 0 0, s 0 n = 0, 1,..., /2 1. Often the DCT with N = 2 is used, that is, each block is overlapped with the adjacent block in half of the length. This corresponds to the parameter L = 2. In this case (6) takes the form (6) w 2 (n) + w 2 (n + ) = 1 (7) 11

12 N = L w a DCT w a DCT w a DCT N = L IDCT w s IDCT w s IDCT w s Figure 6: odified DCT 12

13 n = 0,..., /2 1. The example of the window which satisfies (7) is ( ( π w(n) = sin n + 1 )) 2N 2 n = 0,..., N 1. All perceptual codecs based on transforms suffer from a specific artifact called pre-echo. This artifact occurs when, at the end of the transform block containing a low-energy region, a powerful signal with fast changes begins. Such a phenomenon is called sound attack. We compute the average SR for the block. The decoder spreads the quantization error over the entire block. This quantization noise is easy to hear during the low-energy part of the signal. As a result we obtain in the reconstructed block an unmasked distortion in the low-energy region which precedes the sound attack in time. In order to suppress the pre-echo, the PEG standard uses adaptive switching of the transform block length. The block length is larger for stationary parts of the signal and is reduced if the encoder detects a transient region. Layer III of the PEG standard improves performances of layers I and II by using the hybrid filter bank and adaptive switching of the transform block length for pre-echo suppression. The hybrid filter bank combines a 32-band polyphase filter bank with an adaptive DCT. In order to improve the frequency resolution, each of the 32 subbands is subjected to an 18-point DCT. Using this transform improves the frequency resolution from 750 Hz to Hz. However, introducing the second step of transform increases the length of the transform block. To improve the pre-echo suppression the 18-point DCT is switched to a 6-point DCT when a sound attack is detected. For coding stereo signals, layers I and II of the PEG standard use the Intensity Stereo method. The basic idea for intensity stereo is that for some high-frequency subbands, instead of separate transmitting signals of the left and right channels, only the sum of signals from these two channels is transmitted together with scale factors for each of the channels. Layer III supports intensity stereo coding but also uses a more efficient method of stereo coding called iddle-side (S)-stereo. The encoder uses the sum of the signals in the left and right channels (middle) and the difference between them (side) information. The psychoacoustic model uses a separate, independent, time-to-frequency transform instead of the polyphase filter bank because it needs finer frequency 13

14 resolution for accurate calculation of the masking thresholds. There are two psychoacoustic models in PEG-audio; the so-called PA-I and PA-II. The PA-I is less complex than the PA-II and has more compromises to simplify the calculations. Each model works for any of the layers of compression. However, the PA-I is recommended for layers I and II while the PA-II is recommended for layer III. The PA-I uses a 512-point FFT for layer I and a 1024-point FFT for the layer II. PA-II uses a 1024-point FFT for all layers. 14