A HIGH PERFORMANCE, LOW LATENCY, LOW POWER AUDIO PROCESSING SYSTEM FOR WIDEBAND SPEECH OVER WIRELESS LINKS

Transcription

1 A HIGH PERFORMANCE, LOW LATENCY, LOW POWER AUDIO PROCESSING SYSTEM FOR WIDEBAND SPEECH OVER WIRELESS LINKS Etienne Cornu 1, Aain Dufaux 2, and David Hermann 1 1 AMI Semiconductor Canada, 611 Kumpf Drive, Unit 200, Wateroo, Ontario, Canada N2V 1K8 2 AMI Semiconductor Switzerand, Champs-Montants 12a, 207 Marin, Switzerand phone: +1 (519) , fax: +1 (519) , emai: etienne_cornu@amis.com web: ABSTRACT In this paper we present an audio processing system optimized for bi-directiona wideband speech processing over wireess ins. The system s architecture features a DSP core, a WOLA fiterban coprocessor and an I/O processor. We show how speech encoding, decoding and enhancement agorithms can be depoyed simutaneousy on this patform. The resuting system may incude a combination of features that are usuay not found when standard codecs are depoyed on standard DSPs in wireess headsets. The agorithms can be combined through efficient re-use of the fiterban coprocessor for oversamped and criticay samped subband processing. Such a system may incude features such as ow atency, utra-ow power consumption, resiience to bacground noises and music and the possibiity to add forward error correction for gracefu degradation in difficut RF environments. 1. INTRODUCTION As emerging gaming and VoIP appications support wideband speech (up to 7 Hz audio bandwidth), speech processing techniques used for norma teecommunication are no onger sufficient to provide the required quaity. End users expect noticeaby increased quaity from wideband speech communications. The use of wideband speech in wireess speech transmission aso requires the use of a speciaized audio codec since many wireess speech communication ins are ony designed for narrowband audio transmission. For exampe, in Buetooth, speech is typicay samped at 8 Hz using 16 bits and compressed to 6 bits/sec. In order to maintain ow power consumption and tae advantage of the existing transport protocos at 6 bits/sec, more compex data compression and transmission techniques must be used for wideband speech. Wideband speech is typicay samped at 16-bits and 16 Hz, therefore a minimum -to-1 compression ratio for the speech is required. A number of codecs are abe to provide this compression ratio for wideband speech. However, compression ratio is not the ony factor in choosing a codec for wireess transmission of wideband speech. Other issues to consider are robustness to pacet oss, ow atency in encoding and decoding and ow compexity for ow power consumption. The codec must aso be robust in noise of a types, and shoud minimize distortion when coding music since these are both common in gaming appications that feature musica soundtracs and oud audio effects. It is aso important to consider how the codec wi integrate with speech quaity enhancement agorithms such as noise reduction and dynamic range compression. Thus, a compete wideband speech processing system wi require a robust codec which can be easiy integrated with additiona agorithms in order to provide improved sound quaity in noisy environments. The codec shoud aso be robust in the presence of ossy wireess transmissions, by using features such as variabe bit-rates or side channes with smaer bit aocations to account for isoated pacet drops. Athough existing wideband speech codecs (ITU-T G [1], G [2], for exampe) usuay provide good performance, they may not be suitabe for ow atency, ow power appications because of reativey ong deays, poor speech coding performance in certain noisy conditions, excess distortion when coding music or high computationa resource requirements. In this paper we describe an audio processing system whose architecture is optimized for ow-atency, ow-power audio processing that can encode, decode and improve the quaity of wideband speech in wireess appications. Codecs and other speech processing agorithms can be depoyed in the subband domain, which has many advantages. For exampe, subband-based codecs can provide genera coding abiity ie other transform-domain audio codecs, but can sti be optimized for speech s since speech is sti the primary of interest. The subband approach aso provides a usefu framewor for frequency-domain noise reduction and muti-band dynamic range compression [3]. In this appication, the use of a weighted overap-add (WOLA) fiterban coprocessor in an audio processing DSP provides a means to impement this subband processing whie introducing very ow atency and achieving utra-ow power consumption []. The content of this paper is organized as foows. Section 2 describes the architecture of the audio processing system and its integration in wireess devices. In section 3 we present the techniques required to be abe to use the fiterban for subband encoding/decoding as we as other subband speech enhancement processing. In section we describe how bi-directiona encoding, decoding and processing are depoyed on the audio processing system. In

2 section 5 we present some characteristics of the agorithms depoyed on the audio processing system, and finay in section 6 we present a summary and describe possibe future wor. 2. AUDIO PROCESSING SYSTEM The audio processing system used in this appication is designed for utra-ow power appications such as hearing aids and wireess headsets []. The digita part of the audio processing system consists of three major components: a weighted overap-add (WOLA) fiterban coprocessor, a 16- bit fixed-point DSP core, and an input-output processor (IOP). These three components run in parae and communicate through shared memory and interrupts. The parae operation of these components aows for the impementation of compex processing agorithms with ow system coc rates and ow resource usage. The system is particuary efficient for subband processing since the configurabe WOLA fiterban coprocessor efficienty computes the anaysis and synthesis fiterban operations whie eaving the DSP core free to perform the other agorithm cacuations. The WOLA fiterban coprocessor aso features a vector mutipy function that can be used for appying gains to the incoming and outgoing s in the frequency domain without intervention of the DSP core. Mic Input Input Stage WOLA Fiterban Coprocessor GPIO User IOP Shared Memory PCM Wireess Chip Output Stage 16-bit DSP Seria Port Contro Figure 1 Audio processing system architecture Speaer Output The fow of the incoming and outgoing audio sampes is managed by the IOP. It coordinates the data fow between the FIFO memory and the A/D and D/A converters without intervention from the DSP core. The DSP core does not begin processing of the sampes unti a specific number of sampes have been stored in the FIFO. The IOP raises a periodic interrupt (at a rate caed the tic rate) to the DSP core that a new boc of R sampes has been written to the FIFO, and that a new boc of sampes must be ready in the FIFO to be read for output. In many wide-band speech appications, R is typicay 16, resuting in a tic rate of 1 miisecond at a samping frequency of 16 Hz. In addition to the three processing units that operate in parae, the audio processing system aso features a PCM interface for communication with a wireess chip, a seria port for contro interfacing and a number of genera-purpose input/output (GPIO) pins for other interfacing as shown in Figure 1. The FIFO memory and the fiterban coprocessor are designed so that they can process two streams in parae. These streams may represent a unidirectiona stereo or, as in this appication, a bi-directiona mono stream. In appications running on the audio processing system, most of the actua processing of these two streams occurs in the subband-domain data produced by the fiterban coprocessor. 3. SUBBAND PROCESSING Subband processing is a usefu methodoogy for both speech enhancement and audio coding. Cassica speech enhancement agorithms can be mapped to an oversamped subband processing scheme in order to produce high-fideity outputs. This oversamping aows greater freedom in the anaysis and synthesis fiter design, as we as aowing these fiters to be shorter in ength which utimatey reduces the group deay through the fiterban. Typicay, compex-moduated (i.e., -based) fiterbans are used. Subband coding is aso a popuar method for owresource audio coding such as that used in the Buetooth Advanced Audio Distribution Profie (A2DP) [5]. In order to minimize the amount of data to be quantized, the fiterbans used in these audio coding appications generay provide rea-vaued, criticay-samped s in the subbands. In other words, the channe fiters are rea and the subbands are maximay decimated. Typicay, cosine-moduated fiterbans are used. The WOLA fiterban coprocessor readiy provides the oversamped subband s necessary for high-quaity speech enhancement agorithms, as it reaizes an oversamped, N-channe compex-moduated fiterban, (hereafter caed a fiterban), which provides compex data in its subbands. For audio coding appications requiring a cosine moduated fiterban, rea-vaued, criticay samped s can be obtained by transforming the N channes of the WOLA fiterban into the M=N/2 channes of a corresponding cosine-moduated fiterban. This abiity to re-use the WOLA fiterban architecture for both compex, oversamped processing and rea-vaued, criticay samped processing aows the integration of subband coding and subband speech enhancement agorithms. A simiar compex to rea-vaued fiterban conversion was originay expained in [6] in the context of the Buetooth Subband codec. It is further deveoped here into a form independent of the number of channes (M) and the ength of the prototype fiter (L). This is important for the design of ow-deay codecs, where the ength of the prototype fiter can be optimized by taing advantage of different fiterban configurations and improved prototype window design. The transformation from the subbands of the fiterban to the subbands of the cosine-moduated fiterban can be derived by writing and comparing the z-domain expressions for the -th subband in both fiterbans. In the foowing, x(n) is the input to the fiterban, h (n) is

3 the fiter for channe, R is the decimation factor in the fiterban (which aways equas M for a criticay samped cosine-moduated fiterban), and (m) is the R-times decimated subband centred at frequency ω = (+0.5)π/M. Note that m is the decimated time index in the subbands. Figure 2 shows the fiterban anaysis for the -th channe (=0,, N-1) [11]. ( m) x ( n) h ( n) R x ( m) Figure 2 fiterban anaysis In the case of the fiterban, the fiter impuse response h (n) taes the form (1), with h(n) the ow-pass prototype fiter. Then the z-transform (2) of ( ) m can be evauated, invoving the transfer function of the downsamping boc. (z) and H(z) are the respective z- transforms of x(n), the input, and h(n), the prototype fiter impuse response. For now, et us discard the subsequent compex moduation eading to the fina output subband (m). 2π 1 L h ( n) = h( n) expj + n N 2 2 L 1 2π 1 2π 1 jω 2 R R R R jω ( z) = e z e H z e e R (2) = 0 In the case of the cosine-moduated fiterban, the anaysis operations are the same as in Figure 2, but the fina compex moduation is absent. Furthermore, the fiter impuse response h (n) taes a different form (3). The z- transform of the output subband ( ) for a particuar channe (=0,, M-1). j ω Rm (1) m is expressed in (), π 1 L π h ( n) = 2 h( n) cos + n + ( 1) M 2 2 (3) π L 1 2π 1 2π 1 j ( 1) jω 2 R R R R jω ( z) = e e z e H z e e +... R = 0 π L 1 2π 1 2π 1 j ( 1) jω 2 R R R R jω + e e z e H z e e R () = 0 e Assuming the same prototype h(n), the subband expressions (2) and () can now be compared. A carefu observation shows that the M subbands of the cosine-moduated fiterban can be obtained by performing the foowing transformation to the channes of the fiterban. In this expression, the range of the channe index is =0,, M-1: 2 M 1 j ( 1) ( 1) ( ) π COS j ( ) π = + 2M 1 ( ) m e m e m π j ( 1) 2 = Reae ( m) Now, taing into account the compex-moduation in jω Rm m = e m, which Figure 2, this shows that ( ) ( ) eads to the fina transformation (6), for =0,, M-1: π j Rm ( 1) COS ω + ( m) = 2 Rea ( m) e This expression can be appied to the output of the anaysis operation performed by the WOLA fiterban coprocessor, to convert it into a cosine-moduated fiterban. At synthesis, an opposite transformation can be simiary derived by comparing the z-transforms of the timedomain re-synthesized s for both fiterbans. Such a deveopment shows that if the cosine-fiterban channes are modified according to equation (7), a subsequent fiterban synthesis wi rebuid the same output as a cosine fiterban synthesis. For =0,, M-1 : π j ( 1) _ Rm I COS ω + ( ) ( ) (5) (6) m = m e (7) Hence, transformation (7) must be appied before the synthesis operation performed by the WOLA fiterban coprocessor. I _ ( ) m becomes the input to the WOLA fiterban synthesis. Within the context of our appication, it is important to note that these expressions ony invove phase shift operations. The amount of phase shift in every band varies cycicay with the boc index m. Hence, for cacuation efficiency, the shift vaues can be stored in a tabe and appied as compex gains, an operation that can be performed efficienty by the WOLA fiterban coprocessor. The coprocessor aso taes advantage of the fact that the computation of the fiterban for two channes (one for noise reduction and one for coding, for exampe) can be simpified by we-nown methods for computing the transform of two rea-vaued sequences using a singe compex [7]. The sampes for each channe are stored in the FIFOs of the DSP system in intereaved format and the same prototype fiter is appied to each sequence. The resuting intereaved sequence can be viewed as a set of compexvaued pairs, where the rea part corresponds the first channe and the imaginary portion corresponds to the second channe. A singe compex-vaued is then used to produce the fiterban anaysis resuts for two channes, after a suitabe separation step as described in [7]. This saves memory and computation when compared to performing two independent fiterban anaysis operations. An anaogous

4 procedure can be used to efficienty synthesize two channes that use the same fiterban configuration. Microphone Time (1 tic). CODEC AND ALGORITHM INTEGRATION The audio processing system s architecture and its abiity to process two channes in parae aows subband-based codecs and agorithms to be integrated in an efficient way. When the audio processing system is depoyed in a wireess headset, these agorithms may typicay incude noise reduction on the transmit channe and dynamic range compression on the receive channe. Processing of these two channes requires the execution of a number of operations on the WOLA fiterban coprocessor. For the transmit channe, these operations incude: a first transformation of the timedomain microphone into the frequency domain for processing by the noise reduction agorithm, the appication of gains cacuated by the agorithm, transformation bac into the time-domain, a second transformation into the frequency domain for the encoder and finay the appication of gains to perform the phase shift operation described in the previous section. The is then ready to be processed by the encoder. On the receive channe, the operations performed by the WOLA fiterban coprocessor incude: a first gain appication to perform the inverse phase shift, a second gain appication for voume contro, and finay a transformation of the bac into the time domain where it can be payed on the oudspeaer. Figure 3 iustrates how these WOLA fiterban coprocessor operations are performed for every set of R=16 sampes and executed in parae with processing performed on the DSP core. As iustrated in Figure 3, the microphone stored in the input FIFO by the IOP first goes through a set of transformations that resuts in noise-reduced time-domain in the output FIFO. This is copied bac to the input FIFO and transformed again into the frequency domain, this time for processing by the wideband speech encoder in a criticay samped manner. After the fiterban conversion (phase shift) and the actua encoding and compression are performed, frames of are then sent to the RF chip via the PCM interface. The incoming, in the form of frames of encoded speech, reaches the audio processing system via the PCM interface. The wideband speech decoder extracts the subband data from these frames and stores them so that the WOLA fiterban coprocessor can transform the fiterban into a compex moduated fiterban, appy a gain (set by the user using voume contro buttons on the headset), transform the into the time domain and pace it in the output FIFO, where it wi be eventuay payed bac on the speaer. As a resut of this architecture, the outgoing is processed in two tics, one for noise reduction and one for encoding. A singe tic is needed for processing the incoming. Anaysis 1 Anaysis 2 noise-reduced WBS decoding PCM in Gain cacuation for noise reduction Gains 1 Fiterban conversion Gains 2 Fiterban conversion. Gains 1 Gains 2 oudspeaer WBS encoding Synthesis 1 Synthesis 2 oudspeaer Executed by DSP core Executed by coprocessor Figure 3 Signa fow and agorithm scheduing 5. SYSTEM EVALUATION Copy to input FIFO PCM out Speaer The audio processing system and the integration framewor shown in Figure 3 end themseves to a range of subbandbased codecs and speech processing agorithms. The foowing sections describe sampe modues that can be integrated into the framewor and depoyed in Buetooth wireess headsets. a. Codec. Codec functionaity required for wireess transmission of wideband speech can be impemented by coding each subband using a simpe adaptive PCM quantization scheme. One such scheme is a modified version of the oneword memory adaptive quantization scheme of [8]. The adaptation occurs in each subband using quantizer step sizes and adaptation mutipiers that are designed for processing speech s. This quantization scheme provides reasonabe subband coding performance for speech s with a minimum of computationa resources. For exampe, a typica impementation of a codec may produce a nomina bitstream of 8 bps and encode the frequency range from Hz. Informa tests were done to evauate the quaity of this wide-band speech codec. Using standard objective speech quaity measurements, it achieves an average predicted MOS score of 3.6. Additiona tests with a wide variety of bacground noises showed that the codec coud aso hande non-speech environmenta sounds without serious degradation. When the nomina codec bitrate is ony 8 bps, there is additiona bandwidth in a typica 6 bps in to perform forward error correction that can mitigate the effect of ost or damaged audio pacets in poor RF environments. Good resuts can be obtained by incuding seectivey encoded sections of the spectrum for this purpose.

5 b. Noise reduction. An exampe of a noise reduction agorithm that can be depoyed on the system is a ow-resource subband noise reduction agorithm based on the MMSE spectra noise reduction rue of Ephraim & Maah [9]. This agorithm has been tested in rea-ife acoustic situations and shown to produce up to 16 db of SNR improvement in both white and convention type noise [10]. c. Dynamic Range Compression. Gain adjustment agorithms incuding muti-band dynamic range compression or even simpe voume contro can be impemented using the WOLA fiterban coprocessor by appying gains to the subband s before synthesis [3]. The encoding and decoding agorithms described above require approximatey. MIPS in tota on the DSP core. This is about an order of magnitude ess than G As previousy described, a the fiterban processing occurs on the WOLA fiterban coprocessor. This fiterban coprocessor provides another 5 MIPS worth of equivaent processing power. The noise reduction agorithm from above requires approximatey 5 MIPS on the DSP core. Gain appication is performed on the WOLA fiterban coprocessor, so the combined set of agorithms can be impemented on the DSP system with a system coc ess than 10 MHz. The tota power consumption for this configuration is ess than 6 mw at a 1.8 Vot power suppy. In a typica Buetooth setting, 30-byte data pacets are transmitted every miiseconds. Using the framewor described in Section, boc processing for noise reduction and encoding taes 2 miiseconds. An additiona 6 miiseconds must aso be taen into consideration for the fitering that taes pace as part of the fiterban anaysis. Boc deay for decoding is 1 miisecond and the deay for the fitering associated with synthesis is 6 miiseconds, resuting in a tota atency through the audio processing system of 19 miiseconds, which is about haf of the ow compexity codec G If a forward error correction scheme is depoyed, additiona deays woud normay be incurred. This additiona deay is nominay the time ength of one additiona pacet ( ms). 6. CONCLUSIONS AND FUTURE WORK We have presented an audio processing system and an integration framewor for the processing of wideband speech in wireess headsets. Depoyment of an encoder, decoder and other subband based agorithms resuts in a system with ow atency and ow power consumption that provides good quaity compared with existing soutions. The system proposed by this paper incudes noise reduction offering up to 16 db of SNR improvement and a simpe subband audio codec with an average predicted MOS score of 3.6. These agorithms can be depoyed with a 10 MHz system coc and requires ess than 6 mw of power consumption with a 1.8 V suppy. Future wor wi incude integrating other agorithms such as echo canceation in the framewor in order to aow a wide variety of mechanica designs. This wi aso incude decreasing the tota atency by improving the fiter design of the fiterban. REFERENCES [1] ITU-T Recommendation G.722.1, Coding at 2 and 32 bit/s for hands-free operation in systems with ow frame oss. [2] ITU-T Recommendation G (2002) Wideband coding of speech at around 16 bit/s using Adaptive Muti-Rate Wideband (AMR-WB). [3] T. Schneider, R. Brennan, A Mutichanne Compression Strategy For A Digita Hearing Aid, Proc. ICASSP [] R. Brennan and T. Schneider, Fiterban Structure and Method for Fitering and Separating an Information Signa into Different Bands, Particuary for Audio Signa in Hearing Aids, United States Patent 6,236,731. WO 98/7313. Apri 16, [5] Advanced Audio Distribution Profie. Buetooth Audio Video Woring Group [6] D. Hermann et. a., Low-Power Impementation of the Buetooth Subband Audio codec, Proc. ICASSP 200. Montrea, Canada, May 200 [7] H. Sorenson et. a., "Rea-Vaued Fast Fourier Transform Agorithms", IEEE Transactions on Acoustics, Speech and Signa Processing, Vo. 35, No. 6, June [8] N. S. Jayant and P. No, Digita Coding of Waveforms, Prentice Ha, 198. [9] Y. Ephraim, D. Maah, Speech Enhancement Using a Minimum Mean-Square Log-Spectra Ampitude Estimator, IEEE Transactions on Acoustics, Speech and Signa Processing, ASSP-33 (2), pp [10] K. Tam and E. Cornu, System architecture for audio processing in headsets, Goba Signa Processing Expo (GSPx) Proceedings 2005, Santa Cara, USA, Oct [11] N. J. Fiege, Mutirate Digita Signa Processing, Wiey, 199