A Flexible Hierarchical Classificatio Algorithm for Cotet Based Image Retrieval Qiao Liu, Jiagfeg Che, ad Hui Zhag Abstract he goal of paper is to describe a flexible hierarchical classificatio algorithm ad a ew image similarity computig model based o mixig several image features for promotig the performace ad efficiecy of speed for cotet-based image retrieval. With a experimetal compariso of a large umber of differet represetative poit selectio approach, we are tryig to seek for a method of uiform divisio of image space, evetually desig a ovel approach elighteig by high-dimesioal idexig ad social etworkig, that itroduces the directivity to image classificatio that is used to explai the covergece of images to edge poits of the high-dimesio feature space i this paper. Meawhile we fid the laws of parameter settig of this algorithm through experimets ad these laws acquires satisfied effects i differet dataset. I additio to that algorithm, we also fid some features assemblig with reasoable formula to represet images better i color, texture ad shape. Experimetal results based o a database of about 50,000 perso images demostrate improved performace, as compare with other combiatios i our descriptor set cosistig of several geeral features metioed below. Idex erms CBIR, hierarchical classificatio, feature combiatio. I. INRUCIN With the cotiuous developmet of the search egie techology, people are o loger satisfied with simple text retrieval. Multimedia Retrieval has become a idispesable part of the search field, especially image. Most of the search egies have put text-based image retrieval ito their services, eve a few outstadig leaders of them just like google have commercialized cotet-based image retrieval successfully. But although google ad other experimetal image search projects of the famous uiversity ad research istitutios have made great progress i this field, either performace or speed has a huge room for improvemet. he mai reasos for this dilemma ivolves two aspects: (i) lack of a ucomplicated but accurate computig model ad (ii) the high computatio complexity. Accordig to this situatio, research iterest i this field focuses o solvig those problems. I a typical cotet-based image retrieval system, the visual cotets of images i the dataset are coverted to multi-dimesioal feature vectors. he distace betwee two feature vectors computig by correspodig method is cosidered as the similarity of two origial images, ad we select the op N miimum distace images as output from calculatig results betwee the feature vectors of the query image ad target images i the dataset [1]-[5]. he images that exist i a high-dimesioal space formed various sizes clusters based o the distace betwee them. he approach proposed by this paper arrows greatly the scope of calculatio through selectig uiformly represetative poits usig desity ad directivity of image space i cluster. he paper is orgaized as follows. Sectio 2 describes related work. I Sectio 3, we elaborate our hierarchical classificatio algorithm ad i which the role of desity ad directivity of image space. Sectio 4 shows experimetal performace ad sectio 5 cotais coclusios. II. RELAE WRK I the CBIR (cotet-based image retrieval) field, most researches focused o descriptor extractig [2], [8], [9] ad relatioship aalysis betwee images[5], [7]. After decades of developmet, the descriptors that ca be grouped ito the (i)color represetatio, (iii)texture represetatio[2], ad (iii) shape represetatio[8] cotais dozes of differet kids ad the models geerated by the combiatio of these characteristics also have show a tred of diversificatio. Relatioship aalysis has bee made o the impact of image retrieval o mergig iterests amog differet fields of study, such as multimedia (MM), machie learig (ML), iformatio retrieval (IR), computer visio (CV), ad huma-computer iteractio (HCI) [10]. Because of ievitably eed to fid the optimal results by usig the traverse method i the retrieval framework, researchers have attempted to select the subset of image database as small ad represetative as possible for reducig the amout of computatio, ivolvig maily classificatio[7] ad clusterig[5], [11]. But clusterig is ot effective for query i several high-desity classes. I additio to that, high-dimesioal idexig as a effective method to quickly fid the target i the high-dimesioal space has also bee applied to this area, ad distace based methods of that, such as SR-tree [4] ad VP-tree[12], are suitable for kids of complex ad chageable image descriptor vectors. A research achievemet o social etworkig iterpersoal patters also brigs elightemet to other similar researches. For example, the pheomeo that people always have bias towards a specific topic to express their opiios could be used for simulatig the relatioship betwee images [3]. Mauscript received September 4, 2012; revised November 18, 2012. he authors are with the epartmet of Computer Sciece ad Egieerig, Beijig Uiversity of Aeroautics ad Astroautics, Beijig, CHN (e-mail: lq@ lsde.buaa.edu.c). III. IMAGE SIMILARIY CMPUING WIH CMBINE FEAURES I this sectio we give a overview of the features tested 237
ad try to make the selectio of feature combiatios as represetative ad at the state-of-the-art as possible. he experimetal descriptors that come from geeral features[6] ad lire (A Cotet Based Image Retrieval Library ) iclude followig kids : (I). Scalable color (II). Color layout (III). Edge Histogram (IV). CE (V). FCH (VI). JC (VII). amura (VIII). Gabor (IX). Simple Color Histogram. performace is show i Fig. 2c. Fig. 2a. he query image. Group a Group b Fig. 1. he group two of test case. We coducted to fid the feature combiatio from descriptors above through reasoable experimets to satisfy the requiremet of computig resource ad accuracy. he best aswer should cosist of two or three kids of descriptors so that the idexig geerated will ot be too large. Ad the combiatio also should be able to weake the impact of sigle feature caused by slight chage of image backgroud ad shape. he experimet extract 50,000 perso images accordig to their text labels from 1,170,000 geeral-purpose images i our image database to make up experimetal dataset. Meawhile we selected two groups of perso images from that icludig :(i). same portrait but differet backgroud ad (ii) same backgroud but differet portrait is show i Fig. 1. he combiatio could be regarded as oe of cadidates if the result of experimets o two group all satisfy that the op 10 results retrieved from dataset usig it iclude oe image while other image is query i the same group. hrough buildig idexigs cosistig of differet kids of descriptors ad aalyzig searchig result, there exists four kids of combiatios meetig the above requiremets: Edge Histogram + JC + Simple Color Histogram Edge Histogram + CE + Simple Color Histogram Edge Histogram + JC Edge Histogram + CE Because of overlappig betwee differet combiatios ad idexig size (for example, the size of the feature vector of JC is almost three times more tha CE but has less improvemet), the last oe, edge histogram ad CE i good proportio, is selected as the combiatio of first step of our computig model ad the performace is show i Fig. 2b while the query is show i Fig. 2a. hrough cotiuously experimets, the further improvemet method is proposed that makig use of two other features (FCH ad Color Layout) to sort the results geerated by the first step. the basis of less impactig to better results, a formula that sum up the old distace old computed by the first step ad ew distace ew computed by ew feature i accordace with a suitable proportio is ld ld = * + * Last ld he imum istace i the above formula meas the maximum value that may appear i the first step, ew New Fig. 2b. he result of computig after first step. Fig. 2c. he result of computig after secod step. IV. HIERARCHICAL CLASSIFICAIN ALGRIHM BASE N ENSIY AN IRECIVIY Accordig to the distributio situatio of images preseted i the high-dimesioal space, we hope to select the represetative images with appropriate quatity ad relatively uiform distace by layer-by-layer recursio. he distace of two picture Pi ad Pj is deoted by (Pi, Pj). Ad the most similar image i dataset H with image Q is called cadidate(q, H).he specific steps of our algorithm are represeted as follows : Step 1. Iitialize the classificatio tree Step 2. Select the N least similar images(lsi) P1, P2,..., P as L1 i the whole dataset as first-level child odes of, the meaig of least similar images are explaied below. Ad each image Pi selected represets oe class Ci. Step 3. Each picture Pm of class represeted by root (the whole dataset) is classified ito differet class Ci represeted by first-level child odes Pi of if Pi = cadidate(pm, L1) Step 4. Every child ode follow the example of root repeat Step 2 ad Step 3 util the umber of images of class is less tha Lmax or the level of Node is larger tha the maximum level Kmax Step 5. Collect every leaf ode of to make up represetative poits set S Step 6. Each picture is classified ito classes represeted by each poits of S usig the same method as above. he least similar images(lsi) based o the rule of directivity of iter-image relatioship i the Step 1 ca be 238
mathematically illustrated as follows. Give a set represetative of images selected S with distace matrix W is let S is P P P S P { (, ) I } i j i j P P S P P R { " (, ) > } i j i j k Rk meas the mi-distace betwee LSIs of K level of classificatio tree. Noetheless three questios still remai: (i) how to select these the least similar images (LSI)? (ii) how to set parameters metioed above? ad (iii) how to retrieve images usig these class? For the first questio, a iterative approach is itroduced as follow steps : Step 1. Get ay oe image A from dataset H, fid the image P1 = cadidate(a, H).. Step 2. Get subset H1 of origial dataset H cotaiig every images that has distace with P1 more tha Rk Step 3. Add P1 i S ad Select P2 = cadidate(p1, H1) Step 4. Repeat Step 2 ad Step 3 util the size of S is larger tha N For the secod questio, the basic parameters metioed above are described i able I. he size ad desity of dataset is maily cosidered i these variables settig. For istace, the larger amout of dataset always matches the rise of Kmax ad Lmax, because more images eed to classify more times but every class have better ivolved more to avoid the reductio of the accuracy. he chagig tred i geeral is cocluded i able II. S i this paper cosistig of 1,170,000 geeral-purposed images ad draw 100,000 items from it i radom as subset B to discover the laws of parameter settig of our classificatio algorithm. he feature combiatio metioed i Sectio 3 is used as the similarity measure for computig the similarity betwee the query ad target images i the database. ur CBIR system has bee deployed o a ELL Ispiro 660 desktop with 4GB memory ad Quad-Core 3.1GHz CPU, usig the method of readig idexig ito memory ad multi-threaded parallel computatio to speed up its computig. Parameter N R k L K max ABLE I: HE ESCRIPIN F PARAMEERS escriptio he umber of LSI i every layer he miimum distace betwee ay two LSIs i kth layer he miimum size of class allowed to classify he maximum umber of layer Fig. 3. he amout of computatio ad precisio rate whe N is 5. ABLE II: HE CHANGING REN F PARAMEERS RELYING N HE SIZE AN ENSIY F AASE Chages Size esity N Rk Lmax Kmax 1 2 3 4 For the third questio, we traverse the represetative poits set S at first ad obtaied the M earest poits, the searchig amog images belog to the class represeted by these poits ad gettig the fial results. I additio to cotet based image retrieval, this kid of classificatio algorithm could be used for selectig subset i high-dimesio space. V. EXPERIMEN AN RESULS Cosiderig the performace of this algorithm i real eviromet ad the characteristic of this algorithm that is ot limited by the similarity model, we coducted experimets with images of NIPIC database called dataset A Fig. 4. he amout of computatio ad precisio rate whe N is 10. Although parameter settig has strog relatioship with specific dataset, we expect to fid a regular patter for settig i detail with experimets ad geeral tred summed up above. Rk is impacted by similarity computatio model ad 239
the mai ifluece of Kmax is the size of dataset. So our experimets focus o the role of N ad Lmax i the classificatio. We select three represetative test cases from 30 query with differet complexity of color ad shape. he experimets o the image subset B is divided ito two groups. N is set to 5 ad 10 i two groups ad the umber of earest classes selected as computig set is five. hrough comparig the amout of computatio ad precisio rate i the differet value of Lmax, we choose ideal poit of both show i Fig. 3 ad Fig. 4. he statistics of experimets show that the first group has better performaces but larger computatio due to the larger umber of images of every class. But the precisio rate of the secod group is ear the first group with less calculatio whe Lmax exceeds the threshold of 2000. We surmise that the ifluece of N will dimiish but the rise of amout of computatio is ot obvious whe Lmax is larger tha a threshold value because icreasig Lmax solves the problem of excessive partitioig caused by N. his coclusio has bee verified i the dataset A. Cosiderig the practicality of our system, we hope to decrease the amout of calculatio i a acceptable accuracy, so we choose a group of parameters that has more coductive to the former. Ad the fial results of classificatio of two dataset based o the reasoable parameters settig is show i able III. ABLE Ⅲ: HE SIUAIN F PARAMEER SEING: HERE RK SUCCESSIVELY INCLUES HE ISANCES BEWEEN LSIS IN EVERY LAYER FRM R LEAF AN 15 HA ENES HE NUMBER F IMAGES IN 15 LARGES CLASSES IS HE UPPER LIMI F AMUN F CMPUAIN time of idex buildig of our method is the sum of two parts: lsi ad classificatio. Here lsi is the time to search LSIs ad classificatio. is the time to classify images.herefore the total time is: idex = lsi + classifica tio ( ) ( ) N = + Here is the umber of images of dataset, N ad Kmax has metioed i able 1. he time complexity of most of other algorithms icludig HC ad SVM is 2 ( ) >> ( ) + ( ) N It's ot hard to see that our methods is better suited for massive images tha HC ad SVM. Query A Query B Query C Fig. 5. hree query i test case. Before classifyig K max K max ataset A B he umber of images 1,170,000 100,000 N 5 10 400,350,300,250, 400,350,300, Rk 200,170,150,130 250,200,170 Lmax 2000 2000 After classifyig Fig. 6. he result of query A. K 8 6 he umber of class 2951 204 15 113432 46205 I our system, we extract the 15 earest classes as computig set. For each query, we select the top 100 results but oly show the top 12 i the paper due to the space limitatio. We tested it with comparig the chage of calculatio amout ad accuracy of results before ad after usig this algorithm ad select three query images metioed above. he statistics of these test cases is show i able 4 ad actual performace i Fig. 6, 7 ad 8 while the query images are show i Fig. 5. Although the umber of images i 15 largest class is 113432 for the dataset A, oly about 20000 images i 30 query will be computed as usual because high-desity classes always do't get together. Compare to other mai methods with similarity purpose, such as HC (hierarchical clusterig)[11] ad SVM[13], our hierarchical classificatio method has a great advatage i idex buildig. Accordig to algorithm metioed above, the Before classifyig After classifyig Fig. 7. he result of query A. It is ot difficult to discover the accuracy of search results after classifyig iflueced by the complexity of query images ad target images i test database. Query A is composed by much various colors ad complex shape 240
compared to query B ad query C followig the accuracy less tha two other query. Ad the solutio of this problem has become the focus of our ext step. Before classifyig After classifyig Fig. 8. he result of query A. ABLE Ⅳ : HE QUERY Q SHWN ABVE, AN HE AMUN F CMPUAIN BEFRE AN AFER USING HIS ALGRIHM MB AN MA, AN HE PRECISIN RAI F RESULS PR IN 100 RESUL IMAGES q mb ma pr A 1170551 20689 60% B 1170551 23496 100% C 1170551 18200 81% VI. CNCLUSIN I this paper, we proposed a algorithm for cotet based image retrieval by hierarchical classificatio that is used to reduce amout of calculatio i ergodic search. ur systems makes use of the directivity of images i high-dimesio space ad select the least similar images (LSI) i every subset as the classificatio ceter poits. Meawhile high-dimesio idexig is applied to solve the problem that the differet desity of image space eeds to match differet umber of class. We experimeted with a practical used database cosistig of approximately 1,170,000 images ad its subset cosistig 100,000 images. I our experimets, we used a feature combiatio that we desig for decreasig the impact of slight chage of backgroud ad shape of images as the similarity measure for computig the similarity of images i the database with a query image. Compare to the results without classificatio, we foud that the proposed algorithm give better efficiecy for a acceptable retrieval accuracy rage. ACKNWLEGMEN F. A. Author is grateful for support from the subject" Recogitio ad evelopmet red Study based o Network Commuity Hot Issue "of NLSE, BUAA. REFERENCE [1] Y. Rui,. S. Huag, ad S. F. Chag, Image retrieval: Curret techiques, promisig directios, ad ope issues, Joural of Visual Commuicatio ad Image Represetatio, vol. 10, o.1, pp. 39 62, 1999. [2] S. A. Chatzichristofis ad Y. S. Boutalis, CE: Color ad edge directivity descriptor, a compact descriptor for image idexig ad retrieval, i Proc. of ICVS'08, Satorii, Greece, vol. 5008, pp. 312 322, 2008. [3] P. H. C. Guerra, A. Veloso, W. Meira, ad V. Almeida, From bias to opiio: A trasfer-learig approach to real-time setimet aalysis, i Proc. of K'11, pp. 150-158, New York, USA, 2011. [4] N. Katayama ad S. Satoh, he SR-tree: a structure for high-dimesioal earest eighbor queries, i Proc. of SIGM' 97, pp. 369-380, New York, USA, 2009. [5] S. M. Zakariya, R. Ali, ad N. Ahmad, Usupervised cotet based image retrieval by combiig visual features of a image with a threshold, i Proc. of ICC'10, pp. 204-209, Bhubaeswar, Idia, 2010. [6]. eselaers,. Keysers, ad H. Ney, Features for image retrieval: A experimetal compariso, Joural Iformatio Retrieval, vol. 11 o. 2, pp. 77-107, 2008. [7] A. Vailaya, M. A.. Figueiredo, A. K. Jai, ad H.-J. Zhag, Image classificatio for cotet-based idexig, IEEE ras. Image Processig, vol. 10, o. 1, pp. 117 130, 2001. [8] C. S. Wo,. K. Park, S. J. Park, Effciet use of MPEG-7 edge histogram descriptor, ERI Joural, vol. 24, o. 1, pp. 23 30, Feb. 2002. [9] M. Bober, MPEG-7 visual shape descriptors, Joural IEEE rasactios o Circuits ad Systems for Video echology, vol. 11, o. 6, pp. 716-719, 2001. [10] R. atta,. Joshi, J. Li, ad J. Z. Wag, Image retrieval: Ideas, iflueces, ad treds of the ew age, Joural ACM Computig Surveys, vol. 40, o. 2, 2008. [11] S. Krishamachari ad M. A. Mottaleb, Hierarchical clusterig algorithm for fast image retrieval, i Proc. SPIE Cof. Storage ad Retrieval for Image ad Video atabases VII, Sa Jose, CA, pp. 427 435, Jauary, 1999. [12] I. Markov, VP-tree: Cotet-based image idexig, i Proc. of IJCNN 2004. [13] K. K. Seo, A applicatio of oe-class support vector machies i cotet-based image retrieval, Expert System wit. System. Qiao Liu received the E.E. degree i software egieerig, from Nakai Uiversity, iaji, Chia, i 2011, respectively. Now he is a graduate studet with State Key Laboratory of Software evelopmet Eviromet, the epartmet of Computer Sciece ad Egieerig, Beijig Uiversity of Aeroautics ad Astroautics, Beijig, Chia. His research iterests are i the fields of iformatio retrieval ad istributio Jiagfeg Che received the E.E. degree i Flight Vehicle esig ad Applied Mechaics, ad the Ph.. degrees i computer sciece ad techology, all from Beijig Uiversity of Aeroautics ad Astroautics (BUAA), Beijig, Chia, i 1998, 2008, respectively. He held a visitig positio with the epartmet of Computig Sciece, the Uiversity of Illiois ad the Uiversity of Washigto. Now he is a Assistat Professor with State Key Laboratory of Software evelopmet Eviromet, the epartmet of Computer Sciece ad Egieerig, BUAA. His scietific iterests are i the fields of iformatio retrieval ad computer etwork. Hui Zhag received the E. E., M.S. ad Ph.. degrees i computer sciece ad techology, all from Beijig Uiversity of Aeroautics ad Astroautics (BUAA), Beijig, Chia, i 1989, 1991 ad 1994, respectively. Sice 1994, he has bee with the epartmet of Computer Sciece ad Egieerig, BUAA. Sice 1998, he has bee deputy director of Network Ceter of BUAA. Now he is the deputy director of State Key Laboratory of Software evelopmet Eviromet, the epartmet of Computer Sciece ad Egieerig, BUAA. His scietific iterests are i the fields of computer etwork, Iteret iformatio retrieval ad mass data miig. 241