Progressive Filtering Using Multiresolution Histograms for Query by Humming System

Transcription

1 Progressve Flterng Usng Multresoluton Hstograms for Query by Hummng System Trsladev C. Nagav 1 and Nagappa U. Bhajantr 2 1 Department of Computer Scence and Engneerng S.J.College of Engneerng Mysore, Karnataka, Inda tnagav@yahoo.com 2 Department of Computer Scence and Engneerng Government Engneerng College Chamarajanagar, Karnataka, Inda bhajan3nu@gmal.com Abstract. The rsng avalablty of dgtal musc stpulates effectve categorzaton and retreval methods. Real world scenaros are characterzed by mammoth musc collectons through pertnent and non-pertnent songs wth reference to the user nput. The prmary goal of the research work s to counter balance the perlous mpact of non-relevant songs through Progressve Flterng (PF) for Query by Hummng (QBH) system. PF s a technque of problem solvng through reduced space. Ths paper presents the concept of PF and ts effcent desgn based on Mult-Resoluton Hstograms (MRH) to accomplsh searchng n manfolds. Intally the entre musc database s searched to obtan hgh recall rate and narrowed search space. Later steps accomplsh slow search n the reduced perphery and acheve addtonal accuracy. Expermentaton on large musc database usng recursve programmng substantates the potental of the method. The outcome of proposed strategy glmpses that MRH effectvely locate the patterns. Dstances of MRH at lower level are the lower bounds of the dstances at hgher level, whch guarantees evason of false dsmssals durng PF. In due course, proposed method helps to strke a balance between effcency and effectveness. The system s scalable for large musc retreval systems and also data drven for performance optmzaton as an added advantage. Keywords: Progressve Flterng, Multresoluton Hstograms, Multresoluton Analyss and Query by Hummng. 1 Introducton Content based onlne musc enablng systems are beng developed and revamped n order to keep up wth expectatons of search and browse functonalty. These approaches as a group descrbe the Musc Informaton Retreval (MIR) systems and have been the area under exhaustve research. The ratonale of MIR research s to develop new theory and technques for processng and searchng musc databases by ts content. The QBH s a specal branch of MIR and also a popular content based musc retreval method where the user enters a search query by hummng.

2 Most of the research works on QBH[2][3][11][13][10] are based on the musc processng and focused on components lke melody extracton, representaton, smlarty measurement, sze of databases, query and search algorthms. The strong lterature supports the symbolc representaton for melody n the form of zero-cross detecton, energy, Modfed Dscrete Cosne Transform (MDCT) [5], ptch contour [10], rhythm [18] and quantzed ptch change descrptor [11]. Also there s a remarkable amount of research work [6] [7] [12] n the broader areas of smlarty measurement wth reference to musc patterns. Most of the approaches proposed n the lterature are not suted for real-world applcatons of musc retreval from a large musc database. Perhaps, s due to ether undue complexty n computaton whch leads to longer response tme or performance degradaton; subsequently leadng to erroneous retreval results. Strkng a balance between computaton and performance s the ultmate goal for such retreval systems. As a result there are a few speedng up [14] [19] [20] mechansms proposed for QBH. Qute extensve lterature[2][3][11][13][5][10][6][18][7][12] s avalable on QBH system, but there s no sgnfcant amount of lterature[21][9][16][1][8] towards desgnng flterng procedures. Authors [9] have projected a mathematcal analyss for a two-stage Query by Sngng\Hummng (QBSH) system, whch s the frst applcaton of PF to QBSH. In another work authors [17] proposed the concept of teratve deepenng Dynamc Tme Warpng (DTW), whch s a specal form of PF for speedng up DTW. Improvement n the form of mult phase PF for QBSH wthout much desgn analyss s presented n [19] [16]. Research work [17], proposes a smplfed verson of PF wth a constant computaton tme wth respect to survval rates for each comparson stage. However, most of the proposed methods stll portray the defct n metculous nvestgaton, effcency and effectveness. Therefore, n ths paper we have proposed to apply PF usng MRH approach for QBH system to accomplsh the mproved retreval accuracy. Real-world applcatons of musc retreval symbolze huge amount of non relevant songs wth reference to user queres causng nput mbalance problem. We expect that these two technques are most applcable to mtgate the effect of nput mbalance. The exhaustve expermentaton substantates the potental of proposed method to construct an effectve musc retreval system based on hummng nput. In ths paper, as explaned above, we have motvated to use PF as a flterng procedure. So, the next secton gves a bref vew of PF used for search space reducton. In secton 3 we have made dlgent dscusson on MRH framework for pattern matchng n musc retreval systems. Whle secton 4 elaborates the detals on smlarty measure stratagem for QBH. In secton 5, expermental results are presented and dscussed. The last secton enumerates the concluson. 2 Progressve Flterng (PF) The nspraton behnd PF s to apply a seres of comparsons, n whch each comparson wll select a smaller set that s lkely to contan the target of the nput query. The process s repeated untl fnal output contans lst of songs wth

3 approprate length, say 10 or 20. PF on QBH s performed by applyng multple stages of comparsons between a query and the songs n the database, usng an ncreasngly more complcated recognton mechansm to the decreasng canddate pool. So that the correct song wll reman n the fnal canddate pool wth a maxmum probablty. Intutvely, the ntal few stages are quck and mpure such that the most unlkely songs n the database are elmnated. On the other hand, the last few stages are more sophstcated and tme consumng such that the most lkely songs are dentfed [9]. After each stage of PF, the number of survvng canddates n the canddate pool of the database becomes smaller, and the recognton technque turns nto refned and effectual. The fnal output s the survvng canddate songs at the last stage. The multstage representaton of PF s shown n Fg. 1, where there are m stages, correspondng to dfferent comparson methods wth varyng complexty. Fg. 1. Multstage Representaton of Progressve Flterng For stage, the nput s the query and n -1 survvng songs from the prevous stage. The output of stage s a reduced set of canddate songs of sze n =n -1 s for the succeedng stage +1. In other words, each stage performs a flterng process that reduces the number of the canddate songs by a factor of the survval rate s. Each stage s characterzed by ts capablty to select the most lkely song canddates as the nput to the succeedng stage. For a gven stage, ths capablty can be represented by ts recognton rate, whch s defned as the probablty that the target song of a gven query s retaned n the output song lst of ths stage. Intutvely, the recognton rate s a functon of the survval rate.

4 3 Mult-Resoluton Hstograms (MRH) 3.1 Essence Over the past few years Mult-Resoluton Analyss (MRA) s recevng major attenton by researchers n the doman of computer graphcs, geometrc modelng, sgnal analyss and vsualzaton. It s a most mportant approach for profcently representng sgnals at many levels of detal wth numerous advantages lke compresson, dfferent layers of detals dsplay and progressve transmsson [4]. The term mult-resoluton s used n dverse perspectve such as mult-resoluton based wavelets, subdvsons, herarches and mult-grds. Hstograms provde a very effectve means of data reducton and depct many attrbutes of the data lke locaton, spread, and symmetry. It s also possble to decompose musc sgnal and buld hstograms on the underlyng cumulatve data dstrbutons. Hstograms gve better approxmaton for cumulatve data dstrbutons wth less space usage. However, hstograms provde a comprehensve analyss of the data dstrbuton by excludng sequence detals of values. MRH depcton s proposed for enhanced dscrmnaton of musc data based on ther poston fne ponts to assst effectual QBH system. The musc sgnal s recursvely decomposed and cumulatve hstograms are bult. Together all these cumulatve hstograms of a musc sgnal are remarked as MRH. The selecton of number of levels l s drectly proportonal to precson. Early phase cumulatve hstograms exhbt lesser amount of musc nformaton than later phases. These early phase MRH are used to provde quck approxmate answers to musc retreval queres n the begnnng. Later phase of searchng wth next level MRH gves us better estmates. In ths paper, a MRH based representaton s proposed to approxmate musc sgnal that s nvarant to shftng and scalng. The MRH representaton detects exstence of a pattern along wth shape matchng. In the early phases of searchng musc sgnals wth specfc specfed pattern are retreved, then search contnues for shape matchng yeldng result set of musc sgnals that are of nterest to the user. The herarchcal MRH framework s shown n Fg. 2. The symbol HR ndcates the hstogram representaton at level. Fg. 2. Mult-Resoluton Hstogram Representaton

5 3.2 Connotaton of Mathematcs Hstogram functon h counts the number of samples that fall nto each of the dsjont sets known as bns. Thus, f n s the total number of samples and t s the total number of bns, the hstogram functon h s defned as followng: t n h (1) 1 A cumulatve hstogram functon counts the cumulatve number of samples n all of the bns up to the specfed bn. In partcular, the cumulatve hstogram functon hc of a hstogram functon h j s specfed as: hc j1 h (2) Cumulatve frequency dstrbutons authorze users to approxmate frequences over numerous bns. There s no standard value for number of bns, and dfferent number of bns exhbt dfferent features of the samples. Based on the data dstrbuton and the objectve of the analyss, dfferent bn wdths are chosen. The numbers of bns t are calculated from a recommended bn wdth w as: max( S) mn( S) t w (3) where S=samples to be hstogrammed. Also the equal szed bn wdths are found by dvdng the range wth the number of bns t. Prmary objectve of our research s to develop search crtera usng smlarty based queres over one dmensonal musc sgnal. Such musc sgnal S s defned as a sequence of values: S=[s 1,s 2,...s N] (4) where N, the number of samples n S and s s a vector of values that was sampled at tmestamp t. Gven a musc sgnal database D={S,S,...S } (5) 1 2 M and a query Q, the am s to fnd all the musc sgnals n D that contan the specfed query Q as well as hstogram shape smlar to that of Q. MRH are constructed by dvdng the range [mn D,max D ] of musc database D nto t non-overlappng equal sze sub-regons, dentfed as hstogram bns. Hstogram H s s computed by countng the number of data values h ( t) that are located n each hstogram bn. s 1 2 t j H =[h,h,...h ] (6) A cumulatve MRH s a mappng that counts the cumulatve number of observatons n all of the bns up to the specfed bn. That s, the cumulatve hstogram HC s of a hstogram H s s defned as: HC s t h (7) 1 MRH at hgher levels have enhanced dscrmnaton power; however, the computaton of MRH Dstance (MRHD) at hgher scales s more expensve than those at lower

6 levels. So the number of levels trade-off should be establshed to balance complexty and precson. 3.3 Proposed Strategy MRH constructon system for database D s depcted n Fg. 2 and steps are shown n the followng algorthm 1. Algorthm 1: Procedure to Construct Mult-Resoluton Hstograms for Musc Database I n p u t : a musc database D, number of levels l and the number of hstogram bns t Out put: a hstogram data set H D 1. level l=0 2. repeat S of database D do 3. for each 4. dvde the S nto 2 l non overlappng equal segments S l,l and S r,l 5. locate max D and mn D values of the D 6. dvde the range [mn D,max D] nto t non-overlappng equal sze bns h l,l and h r,l 7. for each S l,l and Sr,l of D do 8. for each data pont l,l and sr,l S and S 9. for each bn hl,land hr,l do h s h then 10. f,,, s of l,l r,l respectvely do l lowerlmt l l l uppperlmt 11. h l,l h l,l 1; 12. end f h s h then 13. f,,, r lowerlmt r l r uppperlmt 14. h r,l h r,l 1; 15. end f 16. end for 17. end for 18. end for 19. nsert generated 20. end for 21. l=l+1 //ncrease level by 1 / / 22. untl (l=user specfed levels) 23. return the result data set H D H and H to the result data set H D S l,l S r,l

7 4 Smlarty Measure Stratagem for Query By Hummng 4.1 Mult-Resoluton Hstograms Dstance (MRHD) Measure In order to recognze the query pattern n the musc database, we have attempted to develop a smlarty functon whch separately consders sgnal frequency as well as postonal nformaton. Gven a song S of musc database D and hummng query Q, feature vectors H by means of the MRHD measure: where S f extracted from song MRH are matched wth query MRH t S Q S f Q f S Q (8) 1 2 MRHD( H, H ) mn( H, H ) s a Eucldean Dstance functon. 2 d( H, H ) t S Q s q 2 (9) 0 d( H, H ) ( h h ) H Q f 4.2 Database Prunng Usng Threshold MRHD for whole musc database s calculated usng equaton 8 and 9. The average of the MRHD consdered as the upper lmt and 0 as the lower lmt of threshold as shown n equaton 10 and 11: and T upperlmt 1 = ( ) M M MRHD (10) 1 T lowerlmt =0 (11) where M=no of songs n the database. Unlkely songs are quckly elmnated by comparng MRHD values of database songs wth threshold range. The song whose threshold s not n the range wll be elmnated from the pruned database. In other words, f the followng condton s not satsfed such song may be purged: T MRHD T (12) lowerlmt S upperlmt Ths procedure s carred out at dfferent hstogram resoluton level to form PF. The database prunng rate analyss s depcted n Fg Results and Dscussons The relatve performance of the proposed QBH method demonstrates several nterestng trends and ths secton s dedcated to evaluate the proposed approach. Substantaton of feasblty of the proposed crtera s done through expermentaton. In the sequel, three seres of experments were conducted wth correspondng target

8 and query corpus by varyng the number of hstogram bns from 100 to 1000 and hstogram resoluton level from 1 to 5. Fnally, comprehensve dscussons of performances are portrayed n terms of error rate, database prunng, Mean Recprocal Rank (MRR), Mean of Accuracy (MoA) and Top X Ht Rate. 5.1 Target Corpus We are proposng a novel QBH system exclusvely for Indan musc songs, so the corpus chosen for ths study conssts of 1000 Indan Kannada devotonal monophonc MP3 songs. Ths collecton s prepared from 39 subjects ncludng songs from 22 males and 17 female sngers. The correspondng tranng set ncludes a subset of 100, 200, 500 and 1000 songs for dfferent experments. MP3 songs contan convoluted melody nformaton and even nose. Thus preprocessng s appled on the MP3 songs database to extract nformaton needed by the system. In musc, human vocal part always plays an mportant role n representng melody rather than ts background musc therefore t s desred to segregate both [15]. 5.2 Query Corpus For system evaluaton, we employ a monophonc query corpus contanng total 200 sample queres from ten partcpants. Each partcpant was asked to hum begnnng of the target song two or three tmes each. The partcpants were selected from varety of muscal backgrounds lke wth and wthout consderable muscal tranng. Also they were nstructed to hum each query as naturally as possble usng the lyrcs of the target corpus. 5.3 Error Rate Analyss

9 Fg. 3. Error Rate Analyss Usng the query and target corpus descrbed above, the error rate s computed for the QBH system mplementatons presented n sectons 2 through 4. Fg. 3 dsplays the error rate for fve hstogram resoluton levels. The target database number of hstogram bns s represented along the horzontal axs and the error rate along the vertcal axs. As expected, drect comparson of error rates wth ncreasng hstogram bn numbers, yelds the better performance, ths mprovement dmnshes as the number of bns decrease. Through promnent observaton t was found that fne gran level musc sgnal approxmaton s possble wth hgher number of hstogram bns, whch yelds better performance. However, error rate ncreases wth the decrease n the number of hstogram bns. 5.4 Database Prunng Rate Analyss Fg. 4. Database Prunng Rate Analyss Fg. 4 dsplays the prunng rate analyss for QBH system across dfferent szed target databases wth fve hstogram resoluton levels. The target database's number of bns are represented along the horzontal axs and the prunng rate along the vertcal axs. In ths fgure, the prunng rate for hstogram resoluton level 1,2,3,4 and 5 are shown wth a lne, dashed lne, small dashed lne, dash-dot lne and dash-dot-dot lne respectvely. Indeed, for ncreasng number of hstogram bns and hstogram resoluton levels prunng rate s approxmately 55% as shown n Fg. 4. The frst hstogram resoluton level representaton yelds the most robust performance of prunng around 55%. For the target database wth ncreasng number of hstogram bns the best prunng rate s n the range 55.21% to 39.35% across dfferent hstogram resoluton levels. That s, the hstogram representaton wth hgher number of hstogram bns yelds good prunng rate, however t s computatonally domneerng.

10 5.5 Performance Analyss Fg. 5. Performance Analyss Many dfferent measures for evaluatng the performance of QBH systems have been proposed [14] [9] [8]. The measures requre a collecton of tranng and testng samples for each test scenaro and parameter combnatons. The Mean Recprocal Rank (MRR) s defned as: n 1 1 MRR n (13) rank ( t ) 1 MRR s a metrc for estmatng any system that generates lst of potental responses to a query. Recprocal rank of a query outcome s the multplcatve nverse of the rank of the frst accurate response. That s, the MRR s estmated as the average of the recprocal ranks of outcomes for a sample of queres. The recprocal value of the MRR refers to the harmonc mean of the ranks. In other words frequency of the system estmatng one of the frst ranks s calculated through MRR [15]. We obtaned MRR n the range 16.41% to 21.34% for dfferent hstogram resoluton levels. The proposed strategy reveals that the MRR ncreases wth ncrease n hstogram resoluton level as portrayed n Fg. 5. In other words, frequency of occupyng top fve ranks ncreases as hstogram resoluton level ncreases. Smlarly for each test scenaro and parameter combnaton the Mean of Accuracy (MoA) s defned as: n 1 n rank( t ) MoA n (14) 1 n 1 It demonstrates the average rank at whch the target was found for each query. We obtaned MoA n the range 68.84% to 83.21% wth hstogram resoluton levels one to fve. From Fg. 5, t s found that the MoA decreases wth ncrease n hstogram

11 resoluton level. Ths ndcates average rank of the retreved song decreases wth hgher hstogram resoluton levels. The Top X Ht Rate s defned as percentage of successful queres and t can be shown mathematcally as: Top( X ) #{ rank ( ): rank ( ) X}/ N (15) where X symbolze top most songs and N ndcates total number of songs. The mpact of Top X Ht Rate for dfferent hstogram resoluton level s portrayed n Fg. 5. The top X Ht Rate vared from 65.78% to 78.90% for dfferent hstogram resoluton levels. From the Fg. 5, X value 10 was found to be the best, at whch system obtaned retreval accuracy n the range 65.78% to wth ncreasng hstogram resoluton level. Comparng fgures 3, 4 and 5, the MRH based representatons emprcally yeld relatvely better performance n terms of MRR, MoA and Top X Ht Rate. 6 Concluson In ths work, we have attempted to explot advantages of MRA technque to progressvely reduce search space for QBH applcatons. In these knds of applcatons, ntal result set conssts of songs that have some specfc patterns; subsequent steps perform relatvely slow search n the small space to retreve all songs whose hstogram shape matches wth query. MRH analyss s employed as database flterng procedure to support teratve search n the database to produce effectve musc retrevals. The results obtaned from exhaustve expermentaton are encouragng. Exhaustve exploraton of the possblty of combnng equal area bn hstogram and MRA s to be consdered as part of further nvestgaton. References 1. A.Adds, G.Armano, and E.Vargu:. Usng the progressve flterng approach to deal wth nput mbalance n large-scale taxonomes. In Proc. of In Large-Scale Herarchcal Classfcaton Workshop of ECIR, (2010). 2. A.Ghas, J.Logan, D.Chamberln, and B.C.Smth:. Query by hummng-muscal nformaton retreval n an audo database. In Proc. ACM Multmeda, pages , (1995). 3. A.K.Trpathy, N.Chhatre, N.Surendranath, and M.Kals:. Query by hummng system. Internatonal Journal of Recent Trends n Engneerng, 2(5): , (2009). 4. Georges-Perre Bonneau, Gershon Elber, Stefane Hahmann, and Basle Sauvage:. Multresoluton analyss. In Lela De Floran and Mchele Spagnuolo, edtors, Shape Analyss and Structurng, Mathematcs+Vsualzaton, chapter 3, pages Sprnger, Jan (2008). 5. C.C.Lu and P.J.Tsa:. Content-based retreval of mp3 musc objects. ACM, pages , (2001).

12 6. C.Francu and C.G.Nevll-Mannng:. Dstance metrcs and ndexng strateges for a dgtal lbrary of popular musc. In Proc. of IEEE Internatonal Conference on Multmeda and Expo, (2000). 7. J.S.R.Jang and H.R.Lee:. Herarchcal flterng method for content based musc retreval va acoustc nput. In Proc. of the 9th ACM Multmeda Conference, Ottawa, ON, Canada, pages , (2001). 8. J.S.R.Jang and H.R.Lee:. An ntal study on progressve flterng based on dynamc programmng for query by sngng/hummng. In Proc. of 7th IEEE Pacfc-Rm Conf. Multmeda, Zhejang, Chna, pages , November (2006). 9. J.S.R.Jang and H.R.Lee:. A general framework of progressve flterng and ts applcaton to query by sngng/hummng. IEEE Transactons on Audo, Speech and Language Processng, 16(2), Feb (2008). 10. J.S.R.Jang and M.Y.Gao:. A query-by-sngng system based on dynamc programmng. In Proc. of Internatonal Workshop on Intellgent System Resolutons (8 th Bellman Contnuum), Hsnchu, Tawan, R.O.C., pages 85-89, (2000). 11. L.Fu and X.Y.Xue:. A new effcent approach to query by hummng. Internatonal Computer Musc Conference, ICMC, Mam, USA, (2004). 12. L.Lu, H.You, and H.J.Zhang:. A new approach to query by hummng n musc retreval. In Proc. of IEEE Internatonal Conference on Multmeda and Expo (ICME), pages , (2001). 13. M.A.Raju, B.Sundaram, and P.Rao:. Tansen: a query-by-hummng based musc retreval system. In Proc. of the Natonal Conference on Communcatons (NCC), (2003). 14. N.Adams, D.Marquez, and G.Wake_eld:. Iteratve deepenng for melody algnment and retreval. In Proc. of Internatonal Symp. Musc Inf. Retreval (ISMIR), pages , (2005). 15. Trsladev C. Nagav and Nagappa U. Bhajantr:. Perceptve analyss of query by sngng system through query excerpton. In Proc. of the 2nd Internatonal CCSEIT-2012, Avnashlngam Unversty, Combatore, Inda., October (2012). Forthcomng. 16. N.H.Adams, M.A.Bartsch, J.B.Shfrn, and G.H.Wake_eld:. Tme seres algnment for musc nformaton retreval. In Proc. of 5th ISMIR, pages , (2004). 17. S.Chu, E.Keogh, D.Hart, and M.Pazzan:. Iteratve deepenng dynamc tme warpng for tme seres. In Proc. of 2nd SIAM Internatonal Conference on Data Mnng, CD-ROM, (2002). 18. W.Jeon and C.Ma:. Effcent search of musc ptch contours usng wavelet transforms and segmented dynamc tme warpng. In Proc. of IEEE Internatonal Conference on Acoustcs, Speech and Sgnal Processng (ICASSP), pages , (2011). 19. X.Wu and M.L:. A top-down approach to melody match n ptch contour for query by hummng. In Proc. of 5th Internatonal Symposum on Chnese Spoken Language Processng, Sngapore, (2006). 20. Y.Zhu and D.Shasha:. Warpng ndexes wth envelope transforms for query by hummng. In Proc. of SIGMOD, San Dego, CA., (2003). 21. Z.Wang and B.Zhang:. Quotent space model of herarchcal query-by-hummng system. In Proc. of IEEE Internatonal Conference on Granular Computng, 2: , (2005).