Pei-Yin Tsai, Tien-Ming Wang, and Alvin Su SCREAM Lab National Cheng Kung University Tainan, Taiwan

Transcription

1 Proc. of he 13 h In. Conference on Digial Audio Effecs (DAFx-1), Graz, Ausria, Sepember 6-1, 21 GPU-BASED SPECTRAL MODEL SYNTHESIS FOR REAL-TIME SOUND RENDERING Pei-Yin Tsai, Tien-Ming Wang, and Alvin Su SCREAM Lab Naional Cheng Kung Universiy Tainan, Taiwan vivian7684@gmail.com ABSTRACT The imbre of an insrumen is usually represened by sinusoids plus noise. Specral modeling synhesis (SMS) is an audio synhesis echnique which can creae musical imbre and give conrol over he frequency and ampliude. Addiive synhesis and LPC synhesis are usually applied for synhesizing sinusoids and residuals, respecively. However, i akes fairly large compuing power while implemening he algorihms. The purpose of his paper is o presen GPU-based echniques of implemening SMS for real-ime audio processing by using parallelism and programmabiliy in graphics pipeline. The performance is compared o CPU-based implemenaions. 1. INTRODUCTION In recen years, audio synhesizer is a fundamenal componen in mos of mulimedia sysems. The sofware version of synhesizer presens significan advanages such as grea flexibiliy for poring. However, some algorihms are compuaional inensive when synhesizing los of channels is required. I is limied by he compuaional capaciy of CPU. Even hough curren CPUs are so powerful ha hey can handle mos common audio processing asks, i is hard for hem o accomplish such real-ime missions. Therefore, parallel archiecures are likely o improve he efficiency. On he oher hand, modern video cards have presened heoreical hroughpu capabiliies ha highly exceed mos CPUs. Graphics hardware companies have recenly developed echnologies such as CUDA (NVIDIA) [1] and CTM (AMD) [2] which are oriened oward general-purpose processing in order o reflec a demand for offloading compue-inensive processes o GPUs. Unlike CPUs, however, GPUs have parallel many-core archiecure, and each core is capable of handling housands of hreads simulaneously. In CUDA, a GPU belongs o one absrac objec named grid. The grid consiss of a number of absrac objecs named blocks. The minimum process uni called hread is capable of execuing user-specified kernel funcion in parallel. The flexibiliy is obviously aken ino accoun while user can specify he number of hreads and blocks for a cerain applicaion. Some researchers presened he performance of video cards by implemening several common algorihms such as marix muliplicaion [3, 4], fas Fourier ransform [5, 6], Vierbi algorihm [7], and digial filer design [8, 9]. Moreover, a number of echniques abou audio processing such as sound spaializaion [1, 11] and modal synhesis [12] were proposed by using GPUbased implemenaion. The parallel programming model for prevalen algorihms is he mos imporan opic discussed by above lieraures. In his paper, echniques for implemening specral modeling synhesis (SMS) [13] are presened for realime audio processing by using parallelism and programmabiliy in graphics pipeline. The paper is organized as follows. The inroducion of specral modeling represenaion is given in Secion 2. The deail implemenaion of specral modeling synhesis on GPUs is described in Secion 3. Secion 4 presens GPU performances comparing o CPU. Finally, Secion 5 concludes wih a brief summary. 2. DESCRIPTION OF SPECTRAL MODELING REPRESENTATION As menioned in [13], sounds are modeled as sable sinusoids (deerminisic componen) plus noise (residual componen). The deerminisic componen is assumed o be represened by harmonics, s h, while he residual componen is represened by a filered noise, s n. Therefore, he inpu sound can be represened by sinusoidal model formulaion as, where p h n s s s P Ap cos p( ) sn p1 (1) A, and () are he insananeous ampliude and p h phase of p harmonic, respecively. P () denoes he number of harmonics included in he harmonic par. If he esimaion he fundamenal frequency is achieved, equaion 1 can be replaced by, where p he iniial phase of P s Ap cos p p sn p1 (2) h p harmonic. By using ime-domain addiive synhesis, he deerminisic componen is able o be generaed wih neglecing phase informaion. Therefore, he fundamenal frequency and he ampliude of each harmonic are recorded a each ime uni which is se as 1 milliseconds. On he oher hand, he synhesis of he sochasic componen can be undersood as he generaion of a noise signal ha has he specral envelopes of he sochasic represenaion. A low-order LPC filer can compleely characerize he residual by encoding is ampliude and specral feaures. The predicion model can be represened as where P sˆ n a s n i, (3) n i n i1 sˆn n is he prediced noise signal, sn n i observed noise samples, and he previous a i he predicor coefficiens. By DAFX-1

2 Proc. of he 13 h In. Conference on Digial Audio Effecs (DAFx-1), Graz, Ausria, Sepember 6-1, 21 aking he whie noise as he exciaion, we can generae he sochasic componen by an all-pole filer wih he coefficiens a. i 3. SPECTRAL MODELING SYNTHESIS ON GPUS Several algorihms are presened on GPUs in his secion. In he simulaion, hundreds of insrumens are synhesized using SMS. Each of hem comprises 5 parials. The sampling rae is 441 Hz. Furhermore, for real-ime purpose, applicaion programs manage concurrency hrough sreams. A sream is a sequence of commands ha execue in order. On he oher hand, differen sreams may execue heir commands ou of order concurrenly or wih respec o one anoher. The lengh of a sream here is defined as one ime uni (1ms). Figure 2: The memory model of he firs kernel funcion in Algorihm Deerminisic par Two algorihms are implemened for he generaion of sinusoids o es he speedup. Le here be N insrumens o be synhesized. The oher mehod is designed for residual par. Algorihm 1 Here one sream represens a frame of synheic audio samples. The synheic sound is sereo; sinusoids of N/2 insrumens are synhesized in each channel. Based on CUDA s srucure, we se 2 blocks per grid, each of hem deals wih one channel. Then 441 hreads are consruced for each block because he lengh of a sream is one ime uni (1 ms). A sraighforward implemenaion using C is shown in Appendix A. Figure 1 shows he memory model of his algorihm. Figure 3: The memory model of he second kernel funcion in Algorihm 2. Algorihm 2 Figure 1: The memory model of Algorihm 1. In Algorihm 1, hreads deal wih (N/2)*5 sinusoids o generae an audio sample simulaneously. In his case, Algorihm 1 is re-designed wih wo kernel funcions by using more GPU resources o shoren he execuion ime. The firs one allocaes an N/2-by-882 memory space in order o sore he samples of one sream. There are 441 blocks being included in one grid and N hreads in one block. Every hread calculaes he value of one insrumen. Figure 2 shows he memory model of he firs kernel funcion. The second kernel funcion is designed for calculaing he samples. We use 2 blocks o consruc one grid for sereo. Each block conains 441 hreads which sum up he values of he N/2 insrumens a each ime insan. Figure 3 shows he memory model of he second kernel funcion. The kernel funcions are shown in Appendix B Residuals In his par, we apply an all-pole filer o generae residuals. In a recursive filer, however, he value of an oupu sample depends on he values of previous ones. Such values may no be available in parallel compuaion archiecure. Two kernel funcions are presened in his sage. The firs one allocaes a 2-D memory wih 441*n-by-N, where n is he number of ime unis in a granulariy which sands for he laency of updaing parameers when synhesizing sounds. Then one block is se in a grid and N hreads in a block. Each hread is in charge of generaing residual samples of some insrumen in a cerain granulariy. The difference beween deerminisic par and residual par is ha he samples of one synhesized insrumen canno be compued in parallel in residual par because he dependency exiss in such a predicion model. Figure 4 shows he memory model of he firs kernel funcion. The second kernel funcion adds he residuals o he sinusoids. The parameers are se as n block per grid, 441 hreads per block. Each block here is responsible for one ime uni. Each hread in a cerain block is going o add he residual values o he corresponding sinusoidal samples. Figure 5 shows he memory model of he second kernel funcion. The kernel funcions are shown in Appendix C. DAFX-2

3 Time (second) Proc. of he 13 h In. Conference on Digial Audio Effecs (DAFx-1), Graz, Ausria, Sepember 6-1, 21 Machine 1 (Noebook) Machine 2 (PC) OS Windows 7 Windows XP CPU Inel Core TM Duo Mobile Processor T83 (2.4 GHz) Inel Core TM Quad Processor Q66 (2.4 GHz) GPU GeForce 95M GS Tesla C16 CUDA Capabiliy revision number Figure 4: The memory model of he firs kernel funcion of residual par. Number of muliprocessors 4 3 Number of cores Clock rae.95 GHz 1.3 GHz Table 1: The specificaions of wo machines. Granulariy NB(sec) PC(sec) (sec) CPU Alg 1 Alg 2 CPU Alg 1 Alg Table 2: The execuion ime of SMS wih differen granulariies. (N=1) Figure 5: The memory model of he second kernel funcion of residual par. 4. RESULTS AND DISCUSSION The environmen of he SMS CUDA implemenaion is presened in Table 1. Table 2 shows he execuion ime of SMS in he experimen. These GPUs are able o achieve he ask (synhesizing 5 sinusoids + residual for each sample) in real ime when alernaive GPU implemenaions are used. CPU implemenaions fail in all cases. Though Inel Q66 CPU is more powerful hen T83, he CPU execuion ime of personal compuer is sill quie similar o noebook. One reason is ha he esing program execued on CPU is implemened on single hread, which is independen of he number of CPU cores. Figure 6 shows he line chars of Table 2. Table 3 shows ha Algorihm 2 has he mos significan speed up. Granulariy NB(sec) PC(sec) (sec) Alg 1 Alg 2 Alg 1 Alg Table 3: The speed up of GPUs comparing o CPUs. (N=1) Execuion Time (GPU) Par 1 NB algorihm 1 NB algorihm 2 PC algorihm 1 PC algorihm Granulariy (second) DAFX-3

4 Time (millisecond) Time (second) Proc. of he 13 h In. Conference on Digial Audio Effecs (DAFx-1), Graz, Ausria, Sepember 6-1, (a) Execuion Time (GPU) Par 2 NB algorihm 1 NB algorihm 2 PC algorihm 1 applicaions ha require real-ime processing wih a huge number of arge sources. 6. APPENDIX Appendix A. Kernel funcion of algorihm PC algorihm Granulariy (second) (b) Figure 6: The line chars of able 1. Granulariy is presened by wo pars depending on (a) smaller and (b) larger han.5 second Execuion ime (Telsa) wih 1ms granulariy Non-real-ime Real-ime Number of insrumens esla C1 wih algorihm Figure 7: The execuion ime of synhesizing differen numbers of insrumens on Tesla C16 by using Algorihm 2. The performance limiaion of Tesla is also esed by increasing he number of synheic insrumens. The execuion ime of implemening Algorihm 2 by synhesizing differen number of insrumens is shown in Figure 7. I is execued on Tesla and he granulariy is 1 ms. The execuion ime is growing inuiively and almos linearly wih he number of insrumens. When he number is larger han 17, he compuing power will be so huge ha Algorihm 2 may no be performed in real ime. 5. CONCLUSION In his paper, we presen efficien implemenaions of SMS on GPUs. The alernaive soluions are presened and discussed based on CUDA archiecure, and paricularly designed for addiive synhesis. In addiion, we compare he mehods on wo differen hardware plaforms. This may enable new sound rendering 17 global void sinusoidal_mehod1( floa* freq, // poiner o frequency array in global memory floa* ampliude, // poiner o ampliude array in global memory shor* samples, // poiner o sample array in global memory in sreamno, // he sream index floa imeoffse // he ime offse of curren granulariy ) { floa = (floa)(hreadidx.x) / samplerae // samplerae : 441Hz + (floa)sreamno*unitime + imeoffse ; // UniTime :.1s floa sum = ; // blockidx.x: for lef channel, 1 for righ channel. if ( blockidx.x == ) { // block compues he insrumen index 49 for lef channel // d_ninsrumen : number of insrumens. (N=1) for ( in i = ; i < d_ninsrumen/2 ; i ++) { // each insrumen has 5 parials // d_ nparials: number of parials of one insrumen. for ( in k = ; k < d_nparials ; k ++) { // exparials : he look-up able of normalized parial energy sum += ampliude [ i ] * ex2d( exparials, i*d_nparials + (k-1), sreamno ) * sinf(2 * Pi * freq[i] * k * ) ; samples [ hreadidx.x*2 ] = (shor) sum ; else{ // block 1 compue he insrumen index 5 99 for righ channel for ( in i = d_ninsrumen/2 ; i < d_ninsrumen ; i ++) { for ( in k = 1 ; k <= d_nparials ; k ++) { sum += ampliude [ i ] * ex2d( exparials, i*d_nparials + (k-1), sreamno ) * sinf(2 * Pi * freq[i] * k * ) ; samples[ hreadidx.x*2 +1 ] = (shor) sum ; Appendix B. Kernel funcion of algorihm 2 /********** firs kernel **********/ global void sinusoidal_mehod2_1 ( floa* freq, // poiner o frequency array in global memory floa* ampliude, // poiner o ampliude array in global memory shor* samples, // poiner o sample array in global memory in sreamno, // he sream index floa imeoffse // he ime offse of curren granulariy floa = (floa)(hreadidx.x) / samplerae // samplerae : 441Hz + (floa)sreamno*unitime + imeoffse ; // UniTime :.1s floa sum = ; // The frequency and ampliude in one block is he same. shared floa f; shared floa a; If ( hreadidx.x == ) { f = freq [ blockidx.x ]; a = ampliude [ blockidx.x ]; synchreads(); // one insrumen has 5 parials for ( in k = 1 ; k <= d_nparials ; k ++) { // exparials : he look-up able of normalized parial energy sum += a * ex2d( exparials, blockidx.x*d_nparials + (k-1), sreamno ) * sinf(2 * Pi * f * k * ) ; // assign sum regiser o emporary soring marix in global memory DAFX-4

5 Proc. of he 13 h In. Conference on Digial Audio Effecs (DAFx-1), Graz, Ausria, Sepember 6-1, 21 sored_marix[ blockidx.x * blockdim.x + hreadidx.x ] = sum; /********** second kernel **********/ global void sinusoidal_mehod2_2 ( shor* samples floa sum = ; // blockidx.x: for lef channel, 1 for righ channel. if ( blockidx.x == ) { // block compue he insrumen index 49 for lef channel for ( in i = ; i < d_ninsrumen/2 ; i ++) sum += sored_marix[ hreadidx.x + i * blockdim.x ]; // assign sum regiser o sample array in global memory samples[ hreadidx.x*2 ] = (shor) sum; else { // block 1 compue he insrumen index 5 99 for righ channel // d_ninsrumen : number of insrumens. (N=1) for ( in i = d_ninsrumen/2 ; i < d_ninsrumen ; i ++) sum += sored_marix[ hreadidx.x + i*blockdim.x ]; // assign sum regiser o sample array in global memory samples[ hreadidx.x*2 +1 ] = (shor) sum; Appendix C. Kernel funcion of residual par /********** firs kernel **********/ global void noise_1( in* noise // poiner o noise array in global memory in index = ; for( in i = ; i < fsize ; i++ ) { // fsize: he frame size (= granulariy size) // index : he index of emporary marix in global memory // FILTER_ORDER : he order of LPC filer (5) index = (i+ FILTER_ORDER)*blockDim.x + hreadidx.x ; // (i%n) is equivalen o (i&(n-1)) if he second number is a power of wo. // WHITE_NOISE_LENGTH : 124 noise[ index ] = ex1dfech( exwhienoise, i&(white_noise_length-1) ); for ( in k = 1 ; k <= FILTER_ORDER ; k++ ) //d_a : he array for soring he preceding noise samples noise[ index ] -= (in)( d_a[ k-1 ] * (floa)noise[ index - k * blockdim.x ] ); /********** second kernel **********/ global void noise_2( floa* d_g, // poiner o filer gains in global memory shor* samples, // poiner o synheic sample array in global memory in* noise // poiner o noise array in global memory in index = blockidx.x * blockdim.x + hreadidx.x ; in noisel =, noiser = ; in halfinsrumennum = N_INSTRUMENT/2; // halfinsrumennum : 1/2 = 5 for( in i = ; i < halfinsrumennum; i++ ) { // lef channel noisel += (in)((floa) noise[ (index+filter_order)*n_instrument + i ] * d_g [ i ] / (floa)halfinsrumennum ) ; // d_g : filer gain // righ channel noiser += (in)((floa) noise[ (index+filter_order)*n_instrument + i + halfinsrumennum ] * d_g [ i + halfinsrumennum ] / (floa)halfinsrumennum ) ; // runcaion of lef channel if( noisel > SHRT_MAX) noisel = SHRT_MAX; else if ( noisel < SHRT_MIN) noisel = SHRT_MIN; //add noise o he correspond sample value samples[ index*2 ] += (shor) noisel; 7. REFERENCES [1] NVIDIA. "NVIDIA GPU Compuing Developer Home Page," hp://developer.nvidia.com/objec/gpucompuing.hml. [2] J. Hensley, AMD CTM overview, in Inernaional Conference on Compuer Graphics and Ineracive Techniques, San Diego, California, 27, pp. 7. [3] K. Faahalian, J. Sugerman, and P. Hanrahan, Undersanding he efficiency of GPU algorihms for marix-marix muliplicaion, in Proceedings of he ACM SIGGRAPH /EUROGRAPHICS conference on Graphics hardware, Grenoble, France, 24, pp [4] E. Larsen, and D. McAlliser, Fas marix muliplies using graphics hardware, in Proceedings of he 21 ACM/IEEE conference on Supercompuing, Denver, Colorado, 21, pp [5] K. Moreland, and E. Angel, The FFT on a GPU, in Proceedings of he ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, San Diego, California, USA, 23, pp [6] T. Jansen, B. von Rymon-Lipinski, N. Hanssen e al., "Fourier volume rendering on he GPU using a spli-sream- FFT," in Proceedings of he Vision, Modeling, and Visualizaion Conference, Sanford, California, USA, 24, pp [7] D. Horn, M. Houson, and P. Hanrahan, ClawHMMER: A Sreaming HMMer-Search Implemenaio, in Proceedings of he 25 ACM/IEEE conference on Supercompuing, Seale, Washinon, 25, pp. 11. [8] A. Smirnov, and T. Chiueh, "An Implemenaion of a FIR Filer on a GPU," Tech. rep., Experimenal Compuer Sysems Lab, Sony Brook Universiy, 25. hp://www. ecsl. cs. sunysb. edu/fir, 25. [9] F. Trebien, and M. Oliveira, Realisic real-ime sound resynhesis and processing for ineracive virual worlds, The Visual Compuer, vol. 25, no. 5, pp , 29. [1] N. Tsingos, E. Gallo, and G. Dreakis, Breaking he 64 spaialized sources barrier, Gamasura Audio Resource Guide 23, 23. [11] E. Gallo, and N. Tsingos, "Efficien 3D audio processing wih he GPU," in ACM Workshop on General Purpose Compuing on Graphics Processors, Los Angeles, California, 24. [12] Q. Zhang, L. Ye, and Z. Pan, Physically-based sound synhesis on GPUs, Lecure noes in compuer science, vol. 3711, pp. 328, 25. [13] X. Serra, and J. Smih III, Specral modeling synhesis: A sound analysis/synhesis sysem based on a deerminisic plus sochasic decomposiion, Compuer Music Journal, vol. 14, no. 4, pp , 199. // runcaion of righ channel if( noiser > SHRT_MAX) noiser = SHRT_MAX; else if ( noiser < SHRT_MIN) noiser = SHRT_MIN; //add noise o he correspond sample value samples[ index*2+1 ] += (shor)noiser ; DAFX-5