A DIVISIVE HIERARCHICAL CLUSTERING- BASED METHOD FOR INDEXING IMAGE INFORMATION

Transcription

1 A DIVISIVE HIERARCHICAL CLUSTERING- BASED METHOD FOR INDEXING IMAGE INFORMATION ABSTRACT Najva Izadpanah Department of Computer Engineering, Ilamic Azad Univerity, Qazvin Branch, Qazvin, Iran In mot practical application of image retrieval, high-dimenional feature vector are required, but current multi-dimenional indexing tructure loe their efficiency with growth of dimenion. Our goal i to propoe a diviive hierarchical clutering-baed multi-dimenional indexing tructure which i efficient in high-dimenional feature pace. A projection puruit method ha been ued for finding a component of the data, which data' projection onto it maximize the approximation of negentropy for preparing eential information in order to partitioning of the data pace. Variou tet and experimental reult on high-dimenional dataet indicate the performance of propoed method in comparion with other. KEYWORDS Content-baed image retrieval, Multi-dimenional indexing tructure, Hierarchical diviive clutering, Projection puruit method 1. INTRODUCTION Due to the advance in hardware technology and increae in the production of digital image in variou application, it i neceary to develop efficient technique for toring and retrieval of uch image [1, 2, 3]. One of the current technique for image retrieval i content-baed image retrieval. In contentbaed image retrieval approach, viual feature uch a colour feature, texture feature, hape feature and local feature are automatically extracted from the image object and organized a feature vector. Then at earch phae, after electing the query image by uer, retrieval engine retrieve the mot imilar image to the query image by performing imilarity comparion between query feature vector and all the feature vector in databae [1, 4]. One of the dicued iue in content-baed image retrieval reearch area i to ue efficient method to accelerate earch operation in image retrieval databae. For thi purpoe, multidimenional indexing tructure have been ued for organization of multi-dimenional image feature and then earching image databae ha been done via multi-dimenional indexing tructure for having appropriate peed in retrieval proce [5, 6, 7]. Up to now many multi-dimenional indexing tructure have been propoed [8, 9]. But due to large volume of image in image databae and high dimenionality of image feature vector, propoing an efficient multi-dimenional indexing tructure capable of managing thee large feature vector i very arduou and important. Increae of the dimenionality trongly aggravate the quality of the multi-dimenional indexing tructure. Uually thee tructure exhaut their poibilitie until dimenion around 15[6]. However, it i hown that in high-dimenional data pace, multi-dimenional indexing tructure tend to perform wore than the equential canning of the databae. Thi fact i due to the cure of dimenionality phenomena [7]. Hence new approache have been recently propoed for indexing DOI : /ipij

2 of multi-dimenional object epecially for high-dimenional pace. In [9] a claification of high-dimenional indexing tructure ha been propoed. Many clutering algorithm are propoed by now, but only few reearch tudie are conducted to propoe clutering method in the purpoe of applying them to indexing of image information [10]. Reearcher have claified clutering method in four total clae of partitioning algorithm, hierarchical algorithm, grid-baed algorithm and denity-baed algorithm [11]. Hierarchical algorithm ha developed in two clae of hierarchical diviive algorithm including PDDP[12] and hierarchical Agglomerative algorithm including BIRCH[13] and CURE[14] for partitioning the data pace in a hierarchical tructure. The goal of thi reearch tudy i to propoe a diviive hierarchical clutering-baed multidimenional indexing tructure in order to manage high-dimenional image feature vector which alo prevent overlapping in it tructure. The ret of the paper i organized a follow: In ection 2, we decribe circumtance of the operation of retrieval ytem according to our propoed indexing tructure. In ection 3, propoed multi-dimenional indexing tructure (NO-NGP-tree1 i decribed. In ection 4, implementation and experiment on our propoed method are explained. And Section 5 include concluion and feature work. 2. CIRCUMSTANCE OF THE OPERATION OF RETRIEVAL SYSTEM ACCORDING TO OUR PROPOSED INDEXING STRUCTURE Like image retrieval ytem architecture propoed in [16], image retrieval baed on our propoed indexing tructure conit of four main component including Feature extraction, Dataet, Retrieval and Multi-dimenional indexing cheme. Figure.1 how the image retrieval baic architecture and our propoed indexing cheme i hown with green color on it. Figure.1 Image retrieval baic architecture and our propoed indexing cheme place in it 1 No Overlapping-Non Gauian Projection baed-tree 14

3 Feature vector are extracted from each image in the image databae and aved in the feature databae. When uer ak their query via Query interface, image query i converted to a multidimenional feature vector via Query Proceing. In Multi-dimenional indexing cheme, in an offline phae(building multi-dimenional indexing tructure phae, every feature vector in feature databae are organized in a hierarchical diviive clutering-baed tructure that we name it NO-NGP-tree, and then in imilarity earch phae, query feature vector i taken and via a recurive imilarity earch algorithm the NO-NGP-tree i earched and k-nearet neighbor to the query feature vector end to the Query interface for repreenting to uer. In thi reearch, reult of the Building multi-dimenional indexing tructure phae, a it i hown in Figure. 2, i an un-balanced binary tree that in highet level or root conit of all multidimenional feature vector. Figure.2 NO-NGP-tree tructure In NO-NGP-tree, node with two child are directory node and node with no child if have le data point than Minpt are outlier and otherwie are leave node. In imilarity earch phae we ued the nearet-neighbor earch propoed in [17]. 3. PROPOSED MULTI-DIMENSIONAL INDEXING STRUCTURE NO- NGP-TREE In image retrieval ytem baed on our propoed indexing tructure, each image i repreented a a multi-dimenional feature vector and all feature vector are aved in the feature databae. Therefore feature databae could be conidered a matrix M 0 (n, m that conit of feature vector in it row. Where n i dimenion of feature vector and m i number of feature vector in feature databae. In order to build the NO-NGP-tree, M 0 ha partitioned recurively via an iterative algorithm. In each iteration a not complete binary tructure ha contructed. The iterative procedure continue until k leaf node are created. Finally a binary tructure which i hown in Figure. 2 i built. Peudo code of NO-NGP-tree i hown in Figure. 3. Figure. 3 NO-NGP-tree Peudo code 15

4 In Figure. 4 the proce of building propoed multi-dimenional indexing tructure i hown. Proce of building propoed multi-dimenional indexing tructure conit of four main phae: Pre-partitioning, Partitioning, bounding and Build tree tructure. Thi procedure iteratively continue until k leaf node are created. Since data node of each leave could be conidered a a matrix, in each iteration, all leaf matrixe of tree in iteration ith that i defined by et M i i taken. M i ha been defined a in (1. (1 M i ={M i-1 - (M i-1 } (MR i-1 (ML i-1 - outlier i -1 Where, M i-1 i the et of leave matrixe at i-1th iteration, (MR i-1 and (ML i-1 are right ub-cluter and left ub-cluter that i elected at i-1th iteration, and outlier i-1 i an outlier at i-1th iteration. Alo (I i j ha been defined a in(2. (2 (I i j ={F j,(a i j,idx1,idx2,cp1,cp2} Where, F j i a vector conit of projection of jth leaf on to the (a i j, (a i j i the meaningful non-gauian component of jth leaf in M i, IDX1, IDX2 are projection ub-cluter, and CP1,CP2 are centroid of projection ub-cluter which all will explain later in thi paper. Figure. 4 Proce of building NO-NGP-tree In the firt iteration (i=1, i i the counter for number of iteration, M 0 i given a input. In Pre-partitioning phae, required information for partition phae (I i i prepared. Then in partitioning phae, next cluter for farther partitioning will be elected and partitioned into the right ub-cluter((mr i and left ub-cluter((ml i. Partitioning i done baed on projection of datanode of elected cluter on the meaningful non-gauian component obtained from pre-partitioning phae ((a i. Then (a i will be ent to the bounding phae. In the bounding phae, right and left ub-cluter will be bounded with minimum bounding rectangle that directed according to the (a i, therefore there i no overlap between them. Reult of bounding phae are (BR i,(bl j that are bounding information about left and rightub- cluter of elected cluter in ith iteration. Then in tree tructure building phae, two right and left ub-cluter will be verified and if number of datanode in each of them i le than Minpt,it will be marked a outlier,otherwie it will be marked a leaf node. In both cae (BRi,(BL j will be aved. Then TS i that pecifie the binary tree in ith iteration in addition to LNO i will be reulted in thi phae. It hould be aid that LNO i i the number of leaf node in ith iteration. 3.1 Pre-Partitioning In Pre-partitioning phae, required information for partition phae (I i i prepared. In thi phae, the M i conit of leaf matrixe in i-1 iteration i taken and then for each member of M i that 16

5 are leaf node, in three phae, (I i j will be obtained. In thi reearch, (I i j for j=1 to LNO i are defined a I i,where j i the counter for number of leave in M i, and (M i j i the jth leaf in M i. Then I i and M i will be ent to partitioning phae. A the pre-partitioning peudo code how in Figure. 5, pre-partitioning conit of following three phae that iterate for j=1to LNO i : Find meaningful non-gauian component: In thi phae the meaningful nono-guian component will be extracted from (M i j where thi component that i pecified with(a i j directed according to a direction that how the clutered tructure of data ditribution. Projection: In thi phae, data node In (M i j will be projected on to the (a i j. 2-mean clutering: In thi phae, 2-mean clutering will be applied on projection, then two centroid (IDX1, IDX2 and two cluter (CP1, CP2 will be obtained that we pecified thee two ub-cluter with projection ub-cluter. Figure. 5 Pre-partitioning Peudo code Find meaningful non-gauian component For building propoed multi-dimenional indexing tructure, in each iteration elected leaf matrix will be partitioned in to two ub-cluter at partitioning phae. For thi purpoe, data node of elected leaf have been projected on to one component extracted from that leaf, and then at partitioning phae have been decompoed in to two ub-cluter from a data node among projected data node. Idea of projecting data on to a component from data i not a new idea and it ha already been ued in hierarchical diviive algorithm uch a PDDP. In PDDP, the data et i divided into two ub cluter. In uch a way that, it project all the data point on to firt principal component. After the algorithm ha plit the initial dataet into two ub cluter, it i going to recure thi procedure for one of the two ub cluter. Thi partitioning trategy create a binary tree whoe leave are the final cluter of the data et. In thi reearch, in order to propoe a hierarchical diviive clutering method, a Projection Puruit(PP method ha been ued for finding a meaningful non-gauian component of the data which projection of data onto maximize the approximation of negentropy, therefore data projection on thi component could contain deirable information for plitting data into two ubcluter[12]. In fact, in PDDP, PCA 2 [18, 19] linear tranformation i done and dataet i projected on firt principal component. Deirability of the component(direction with maximum non-gauian attribute than principal component(direction i hown in Figure. 6. In Figure. 6, firt principal component which i hown with green color preent the clutering tructure of the data ditribution, but non-guaian component that i hown with red color preent the eparation between cluter then it how the clutering tructure of data ditribution. 2 Principal Component Analyi 17

6 Figure. 6 Deirability of non-gauian component than principal component. The main idea of Projection Puruit method i to extract meaningful component from dataet with local optimization of a deire Objective function (Projection index [20]. Pat tudie indicate that Gauian ditribution how le meaning at data ditribution and deire component (direction have maximum ditance from Gauian direction. Local optimization of an Objective function i alo done in ICA 3 which i one of the Projection Puruit method. In [21] uing tranformation of neural network learning rule to fixed point iteration, FatICA algorithm ha been propoed. FatICA i originally propoed for finding independent component but in fact, it i a Projection Puruit method that we can ue to find meaningful non-gauian component. FatICA algorithm find each component individually. Therefore we can ue it for finding only one component. Objective function in thi algorithm could be each objective function which i ued by reearcher in proce of finding projection puruit. In thi reearch, we ue one verion of FatICA algorithm that approximation of negentropy i ued in a objective function. Negentropy i one of the mot important meaure of non-gauianity. Gauian variable have mot entropy among variable with ame variance. Therefore, negentropy could be conidered a one of the non- gauianity meaure. Data ditribution with more gauianity and therefore with more entropy i more non-tructured and more unpredictable. Negentropy J i defined a in (3 [20]. (3 J ( y = H ( y - H ( y gau Where, y gau i Gauian variable that ha ame variance with variance of y, and H i entropy. Since it i difficult to calculate negentropy, approximation of negentropy practically ha been ued a Objective function in [20]. Thi approximation defined in (4. (4 J ( y [E{G(y}- E{ G ( V }] 2 Where, E i expectation function, V i a Gauian variable with zero mean, and G i defined in (5. (5 G ( u = tanh( cu Where 1 c 2 i a contant that i uually et a 1. In thi reearch, FatICA with firt principal component a initial weight vector i ued to extract a meaningful non-gauian component and the firt component found by the algorithm i ued for projecting of data onto, becaue projection of data onto it maximize the approximation 3 Independent Component Analyi 18

7 of negentropy and therefore ince thi component i directed according to the direction of data with mot non-gauianity attribute, projection of data onto it could include intereting information for partitioning the dataet into two ub-cluter Projection In projection phae, data matrix (M i j (M 0 in firt iteration will be projected on to (a i j. projection on (a i j i characterize in (6. T (6 F( x = (a ( x j b i j b Where, x b i bth feature data point(image feature vector in (M i j, and F j (x b i a vector whoe each component i a calar that how the projection value of x b (b=1 to nodeize on to (a i j, nodeize i the number of data point in (M i j. Data Projection on to the meaningful non-gauian component could conit of following information: Location with le denity are appreciating location for partitioning of data point of related cluter from them. Location with more denity are cloe to the ub-cluter centroid of related cluter. In Figure. 7 an example of not balanced and not eparated cluter ha been hown. In Figure. 7, firt principal component i hown by green color and the meaningful non-gauian component i hown by red color. We can ee that the red one i aligned to the direction with more irregularity. Then projection of data on the red one could how the location with more and le denity Clutering of data projection Figure.7 an example of not balanced and not eparated cluter In thi phae, a clutering algorithm uch a k-mean [22] will be applied on the data projection F j. A the reult two projection ub-cluter (IDX1, IDX2 and their centroid (CP1, CP2 will be obtained. Data projection onto (a i j could how the location with more and le denity, But how we can find thee location. In thi reearch, in order to gue thee location, k-mean clutering with k=2 ha been applied on projection. So two obtained centroid are conidered a location with more denity, and the mean of thee two centroid i conidered a the location with le denity. In data ditribution, with balanced and eparated natural cluter, mean of two centroid of projection ub-cluter i probably near the real centroid of the data but in data ditribution with not balanced and not eparated natural cluter, the mean of two projection ub-cluter centroid i hown the location with le denity that centroid of the data. 19

8 Figure. 8 how centroid of the dataet hown in Figure. 7, and the mean of projection ubcluter centroid. A it i hown in Figure. 8, location of mean of the projection ub-cluter centroid that located on the meaningful non-gauian component ha le denity than the location of the real data centroid. Figure. 8 comparing location of mean of projection ub-cluter centroid with the location of real data centroid 3.2 Partitioning In partitioning phae, I i and M i i taken. M i include leave of tree tructure contructed until ith iteration. Figure. 9 how partitioning peudo code. Figure. 9 Pre-partitioning Peudo code A the partitioning peudo code how in Figure. 9, partitioning conit of following two phae: Cluter election: In thi phae, one of the leaf node cluter in ith iteration will be elected for further partitioning. In thi reearch the node with more clutered tructure will be elected. Split: In thi phae, the elected cluter will be plit into two ub-cluter Cluter election In thi phae, the cluter which hould be plit into two ub-cluter i elected from cluter related to the leaf node. For thi purpoe Mi and Ii ha been taken and then according to thi information one of the cluter related to leaf node will be elected. In thi reearch, information obtained from projection of data of each cluter on to the (a i j ha been ued and a cluter election meaure i propoed. Cluter election i one of the main challenge of diviive clutering method. In ome pat tudie (for exapmle ee [12], the cluter with the mot catter value ha been elected for further partitioning. Scatter value i defined a in (7. 20

9 (7 Signal & Image Proceing : An International Journal (SIPIJ Vol.6, No.1, February 2015 Scattvalue = 1 N N i= 1 _ x i w 2 Where, Scatvalue i catter value, N i the number of data point in related cluter, and w i the centroid of the cluter. But a it i hown in [23], catter value i not a good meaure for meauring the clutering tructure of a cluter. Propoed cluter election meaure i defined in (8. (8 c= 1,2 c elvalue= CP1 CP2 max ( diameter( IDX Where, CP1 and CP2 are projection ubcluter centroid obtained from the conidered leaf node, and diameter (IDXc i the diameter of cth(c=1,2 projection ub-cluter diameter that i defined in (9. (9 diameter(idx = max( F p min( F p Where, IDX i the conidered projection ub-cluter, and F p i pth data projection. Longer ditance between centroid of two projection ub-cluter with le cluter diameter indicated the more clutering tructure of the data. In other word, the cluter with more value of thi meaure ha more clutering tructure. In Figure. 10, a imple example with it dataponit, their projection on to the meaningful non-gauian component, and centroid of projection ub-cluter i hown. Figure. 10 an example of data projection on to the meaningful non-gauian component in twodimenional pace. a cluter with untructured ditribution, elvaue=1.5. b cluter with tructured ditribution, elvalue=2.8. Figure. 10.a how a cluter with untructured ditributation, which ha the elvalue equal to 1.5, and Fig.10.b preent a cluter with tructured ditribution and the elvalue equal to

10 3.2.2 Split Signal & Image Proceing : An International Journal (SIPIJ Vol.6, No.1, February 2015 In thi phae, the matrix related to the elected leaf in ith iteration((m i and it meaningful non-gauian component ((a i, a well a the vector conit of projection data of elected leaf (F are taken. Then it i plit into two right and left ub-cluter from the mean of centroid of projection ub-cluter by the hyper-plane orthogonal to the (a i. (a i i alo aving for right and left ub-cluter. In ection 3.1.3, we have dicued about why it i better to plit the cluter by the hyper plane paing from the mean of centroid of projection ub-cluter. Figure. 11-a, how the eparating hyper-plane paing from real (origin centroid of the data ditribution wa hown in Figure. 7. A it i demontrated in Figure. 11 becaue the data ditribution conit not balanced and not eparated cluter, real centroid of the data i not a good choice a location that eparating hyper plane paing from, but mean of the centroid of projection ub-cluter i more uitable. Figure. 11-b how the eparating hyper-plane paing from mean of the centroid of projection ub-cluter for the data ditribution hown in Figure. 7. A it i hown in Figure. 11, by hyper-plane orthogonal to the meaningful non-gauian component paing from mean of the centroid of projection ub-cluter, the data i partitioned into two tight ub-cluter, and finally after bounding it i reulted to the tighter minimum bounding rectangle. A the reult of plitting by hyper-plane orthogonal to the meaningful non-gauian component paing from mean of the centroid, data point in elected cluter that their projection are in the right ide of the hyper-plane will be aigned to the right ub-cluter that attributed by (MR i, and data point that their projection are in the left ide of the hyper-plane will be aigned to the left ub-cluter that attributed by (ML i. In fact, for each x b, if reult of (10 i poitive, then x b will be aigned to right ub-cluter and, ele if it i not poitive then x b will be aigned to the left ub-cluter. T (10 F ( xb ( ai cmean Where, x b i bth data point, F (x b i the projection of bth data point, and Cmean i the mean of centroid of projection ub-cluter. 22

11 Figure. 11 Split by eparating hyper-plane. a hyper-plane orthogonal to the meaningful non-gauian component paing from mean of the centroid of projection ub-cluter.b hyper-plane orthogonal to the meaningful non-gauian component paing from the real data centroid. 3.3 Bounding A the proce of building NO-NGP-tree i how in Figure. 4, in each iteration, after partitioning phae, right and left ub-cluter obtained from partitioning phae and alo (a i will be ent to the bounding phae, and in bounding phae, right and left ub-cluter will be bounded with minimum bounding rectangle which are directed according to (a i. Bounding peudo code i hown in Figure. 12. Figurer. 12 Bounding peudo code Bounding phae conit of two following phae: Change-reference mark: in thi phae, reference mark i changed according to the (a i direction. Minimum bounding rectangle contruction: in thi phae, minimum bounding rectangle contructed in the new reference mark Change-reference mark A it ha been mentioned before, to prevent from bounding overlapping, both right and left ub-cluter are bounded with minimum bounding rectangle which are directed according to (a i therefore bounding rectangle of two nearby node do not have any overlap. Changing reference mark and directed bounding rectangle according to the new reference mark in order to prevent from overlapping covering node area i not a new idea. In [17], changing reference mark i done in NOHIS-tree for preventing from node overlapping. In thi phae, right and left ub-cluter in new reference mark obtained uing (11 and (12. T (11 ( NR i = (MR i 2V. V.(MR i 23

12 T (12 (NL i = (ML i 2V. V.(ML i Where, (NR i, (NL i are matrixe related to the right and left ub-cluter in new reference _ (( ai e1 mark, and V = _ where e 1 = (1,0,...,0. ( a e i Minimum bounding rectangle contruction In thi phae right and left ub-cluter in new reference mark are taken and bounded with minimum bounding rectangle (MBR. Then information related to the right and left MBR which are defined with (BR i and (BL i will be aved. In thi reearch, for contruction of MBRS the method that it i ued in [17] ha been applied. It i hown in Figure. 13 that how changing reference mark and contructing minimum bounding rectangle according to the new reference mark lead to minimum bounding rectangle of nearby node with no overlap between them. Figure. 13 MBR contruction. a MBR contruction according to real reference mark. b MBR contruction according to the new reference mark. c data plitting by the heyper-plane paing through real data centroid and MBR contructing according to the new reference mark 24

13 A we can ee in Figure. 13-b contruction of MBR according to the new reference mark prevent overlapping between node. In Figure. 13-a and b data i plit by the hyper-plane orthogonal to the meaningful non-gauian paing through mean of centroid of projection ubcluter. But in Figure. 13- c data i plit by the hyper-plane orthogonal to the meaningful nongauian paing through real data centroid, and MBR are contructed according to the new reference mark. A it i hown in Figure. 13- c that there i no overlapping between bounding area of the node, but a plitting i done by the hyper-plane paing through the real data centroid, it i lead to not completely eparated ubcluter and therefore the total volume of the bounding rectangle in Figure. 13-c i larger than total volume of bounding rectangle in Figure. 13-b. 3.4 Building tree tructure In thi phae, in each iteration of contructing the propoed index tructure, right and left ub-cluter and alo information related to the right and left MBR are added to the tree tructure TS i. Building tree tructure peudo code i hown in Figure. 14. A we can ee in Building tree tructure peudo code, if the number of data node in each ub-cluter wa le than Minpt, it will be marked a outlier, otherwie it will be marked a leaf node. Figure. 14. Building tree tructure peudo code. 4. IMPLEMENTATION AND EXPERIMENTS We have reported implementation and experiment reult of our propoed method and in thi ection. 4.1 Implementation In thi ection, we have covered detail about implementation uch a evaluation meaure, tet method, dataet and implementation environment Evaluation meaure In thi reearch, Recall and Repone time are conidered a evaluation meaure of our propoed method. Quality of final evaluated cluter i evaluated with Recall meaure. Recall i conidered a ratio of the number of data point related to the query point that are retrieved to the total number of related data point to the query point. Recall i defined in (13 [10, 24, 25]. α β (13 Re call = α Where, α i et of related data point and β i et of related retrieved data point. Alo in thi reearch the retrieval peed via propoed indexing tructure i evaluated with Repone time meaure. Repone time i the time that i expended to earch the databae via 25

14 indexing tructure for finding k-nearet data point in the databae to the query point. Repone time i defined in (14 [10, 17, 24]. (14 Repone time = the time that final reult exit from imilarity earch phae-the time that query point enter to the imilarity earch phae Le value for repone time how better performance of indexing tructure Tet method All tet in thi reearch are done with Cro-validation method. For thi purpoe, each tet i done 10 time and each time, 20 feature vector from dataet are conidered a tet et, and for each of them all the dataet i canned and 20-nearet neighbor for them will be found and inerted in the related data point et (α. Then indexing tructure will be contructed with the ret of feature vector in the dataet. Then we ue tet et for querie and mean reult of 20 querie will be obtained and finally mean reult for 10 tet will be reported [24, 23] Dataet To evaluate and tet our propoed method, we have ued a databae including 1,193,647 feature vector extracted from color image. Feature vector are local feature baed on point of interet derived from 4,996 color image which are ued in previou tudie for evaluating indexing tructure performance [17]. In thi reearch, we have ued four databae of different dimenion including 50,000 feature vector from decribed dataet. Firt databae include 50,000 feature vector of dimenion 25. Second databae include 50,000 feature vector of dimenion 40. Third databae include 50,000 feature vector of dimenion 60 and fourth databae include 50,000 feature vector of dimenion Implementation environment In thi reearch, for implementing indexing method in our experiment, all algorithm are implemented in MATLAB Algorithm run on a PC with Intel CPU.4GHZ, 4GB of RAM. 4.2 Experiment In thi ection, we have covered detail about our experiment on our propoed indexing method Prerequite parameter adjutment In thi ection, we have done experiment for adjuting prerequiite parameter of our propoed indexing cheme. A it i decribed in peudo code of ection 3, we have conidered two prerequiite parameter k and Minpt. Parameter k i final number of cluter (final leaf and outlier. In fact parameter k define the number of leaf node plu outlier node after building of NO-NGPtree. Alo in our propoed cheme, minimum number that i conidered for number of data node in each leaf node i defined with parameter Minpt. In our experiment, Minpt characterized a percent of average number of final cluter data member. For example 15 mean 15 percent of average number of final cluter data member. Average number of final cluter data member are defined in (15. (15 26

15 Average number of final cluter data member= total number of data node /k Where, k i final number of cluter. For adjuting two parameter k and Minpt, ome experiment are done and the average Repone time i reported. Thee experiment that their reult are hown in continue are done with different value for Minpt and fixed value for k in each for databae of dimenion 25, 40, 60 and 80. Reult of average Repone time obtained from experiment on NO-NGP-tree baed on the econd are hown in Tab. 1. Table.1 Average Repone time for different value of Minpt and k on databae with different dimenion Average Repone( time on different dimenional databae k= Average Repone( time on different dimenional databae k= Average Repone( time on different dimenional databae k= Average Repone( time on different dimenional databae k= Minpt 25-d databae 40-d databae 60-d databae 80-d databae 25-d databae 40-d databae 60-d databae 80-d databae 25-d databae 40-d databae 60-d databae 80-d databae 25-d databae 40-d databae 60-d databae 80-d databae Reult in Tab.1 how that with growing of dimenion of data, average Repone time i increaed for different value of Minpt. Alo it i conidered from reult in Tab.1 that with variation of value for Minpt, different reult have been obtained. Therefore, for analyzing the reult more accurately, in Figure. 15 with waiver of number of dimenion, hitogram of average Repone time on 25-d databae for different value of k i hown. Figure. 15 Hitogram of average Repone time on 25-d databae for different value of k Figure. 15 demontrate that with adjutment done on prerequiite parameter, when Minpt i 25 and k i 600, bet reult are obtained. Alo it i oberved that when Minpt i adjuted on greater value uch a 65 or le value uch a 5, average repone time i increaed. Alo when 27

16 Minpt i adjuted on 15 to 25, better reult have obtained. In fact if a minimum value conidered for number of leaf node data member (Minpt in each iteration i very large, ince cluter election meaure ued in our cheme tend to elect maller cluter, mall cluter will be plit continuouly. Thi lead to increae in height of the unbalanced indexing tructure and therefore to decreae in peed of earch via propoed indexing tructure. Alo adjuting Minpt on very mall value lead to not fine detection of real outlier and jut cluter with very mall number of data node are marked a outlier, o that not incorrect partitioning i done and then it lead to decreae in the quality of final cluter a well a increae in Repone time. Alo it i hown in Figure. 15 that Repone time for different value of k and fixed value of Minpt i different. Thi i becaue if the value of k i near to the real number of natural cluter in dataet, quality of the final cluter i better, and ince it lead to earch in fewer cluter in our cheme it lead to decreae of Repone time through our propoed indexing tructure Experiment for evaluating quality of final earched cluter In thi ection, reult of experiment for evaluating quality of final earched cluter with Recall meaure in compare to following cheme are reported: NGP-tree: thi tructure wa obtained through elimination of changing reference mark phae from the proce of contructing the propoed indexing tructure NO-NGP-tree. PDDP-tree: thi tructure wa obtained by adding MBR clutering method PDDP. NOHIS-tree: a clutering baed indexing method [17]. It i obervable that in four conidered indexing method, in imilarity earch phae, we have ued imilarity earch algorithm that wa propoed in [17]. Experiment on NO-NGP-tree and NGP-tree in thi ection are done with adjuting Minpt on 25 and different value for k. The purpoe of running the experiment with different value of parameter k i to how the effect of adjuting thi parameter in a comparion of imilar indexing cheme. Figure. 16 how average Recall with repect to number of earched final cluter on 25-d and 80-d databae where k i 600,800 and 1000, and Minpt i 25. Via NO-NGP-tree, earching the databae after earching average 14 final cluter i topped. But via NOHIS-tree after earching average 20 final cluter i topped. A it i demontrated in Figure. 16, with increaing the number of earched cluter, average Recall i increaed, and after earching about 14 cluter via NO-NGP-tree Recall turn to 1. Alo it i demontrated in Figure. 16 that for PDDP-tree and NGP-tree in whoe tructure there are ome overlap between node, average Recall i increaed lightly, and it i topped for NGPtree after earching about 257 cluter and for PDDP-tree after 273 cluter. Slower increae in the average Recall for tructure with ome overlap i actually becaue of not accurate calculation of MINDIST (ditance between MBR and query vector [17]. Alo if MBR are tighter, the calculation of MINDIST will be more accurate. Figure. 16 indicate that NO-NGPtree ha better reult than NOHIS-tree becaue of optimum partitioning during building indexing tructure which lead to tighter MBR. A we can ee in Figure. 16, average Recall on 80-d databae for four method increae lower than 25-d databae. Alo with increaing the value of prerequiite parameter k, average Recall for NO-NGP-tree and NGP-tree ha le increaing, becaue when k ha more ditance from real number of natural cluter in dataet ditribution, quality of final cluter will be decreaed. Thi decreae in the performance how itelf on 80-d databae in uch a way that after earching 12 cluter, average Recall for NOHIS-tree i higher than average Recall for NO-NGPtree. Reaon of thi problem i decreae in performance of the algorithm ued in pre-partitioning phae for finding a meaningful non-gauian component from data on high dimenional databae. 28

17 29

18 Figure. 16 average Recall repect to number of final earched cluter on 25-d and 80-d databae where k i 600, 800 and 1000, and Minpt i Experiment for evaluating earch peed via propoed indexing cheme In thi ection, reult obtained from our experiment for evaluating earch peed via our propoed indexing tructure with Repone time evaluation meaure in compare to NOHIS-tree, NGP-tree, PDDP-tree and equential-can i reported. Experiment in thi ection are done by adjuting prerequiite parameter for NO-NGP-tree and NGP-tree on bet cae obtained from adjuting parameter ection. Therefore in all experiment of thi ection, Minpt i adjuted on 25 and k i adjuted on 600. Alo, in imilarity earch phae, we have ued imilarity earch algorithm that wa propoed in [17]. Figure. 17 how average Repone time for the comparion of NO-NGP-tree, NOHIS-tree, NGP-tree and PDDP-tree on 25-d, 40-d, 60-d and 80-d databae. Figure. 17 average Repone time for comparing NO-NGP-tree, NOHIS-tree, NGP-tree and PDDP-tree on 25-d,40-d, 60-d and 80-d databae A it i demontrated in Figure. 17, our propoed indexing cheme NO-NGP-tree that i hown in black color i ha the le repone time than NOHIS-tree, NGP-tree and PDDP-tree for different dimenion. Alo it i demontrated that Repone time i growing with increae in dimenion. Becaue in thee indexing cheme, increae in dimenion of databae lead to increae in the volume of minimum bounding rectangle (MBR of node. Therefore, average ditance of query vector from MBR will increae, and then it lead to decreae in accuracy of calculating MINDIST ditance. Thi lead to more navigation of tree tructure and alo more final cluter that ha been earched during earch proce then it lead to growing Repone time. 30

19 Alo Figure. 17 indicate that NGP-tree and PDDP-tree in whoe tructure, overlapping between node i not guaranteed, have more Repone time than NO-NGP-tree and NOHIS-tree. Reaon of thi problem i the increae in number of navigation of tree tructure becaue of thee overlap. In thi reearch, we claimed that reaon of decreae in Repone time via propoed NO-NGPtree in compare to NOHIS-tree i optimum partitioning and tighter ub-cluter, which lead to decreae in the total volume of MBR in NO-NGP-tree tructure. Figure. 18 how average Repone time for comparion of NO-NGP-tree and Sequentialcan on 25-d, 40-d, 60-d and 80-d databae. Figure. 18 average Repone time for comparing NO-NGP-tree and Sequential-can on 25-d, 40-d, 60-d and 80-d databae We compared our indexing cheme with equential-can becaue mot conventional indexing cheme loe their efficiency in high-dimenional pace o that equential canning of dataet how better reult [6, 7]. A it i demontrated in Figure. 18, NO-NGP-tree improve repone time with large ditance than equential-can. 5. RESULTS AND FUTURE WORKS In mot application of image retrieval, image are decribed with high-dimenional feature vector but reearch tudie on indexing tructure have hown that thee tructure loe their efficiency with increae in dimenion. In thi reearch we have propoed an indexing tructure for managing high-dimenion image feature vector uing pattern of hierarchical diviive clutering method. In ection 4, prerequiite parameter have adjuted on different value and our propoed cheme i evaluated and reult are reported. Alo we have run ome experiment for evaluating final cluter quality which can alo affect on earch efficiency. Finally by adjuting prerequiite parameter on bet cae we have run ome other experiment for evaluating peed of earch in the databae via our propoed indexing tructure. Some more invetigation in thi field which might be helpful in improvement of our indexing cheme in feature include: 1-Uing other objective function intead of approximation of negentropy in proce of finding a meaningful non-gauian component and compare their reult. 2-Our indexing tructure ha an unbalanced binary tructure. Therefore plitting the elected cluter to everal ub-cluter intead of only two ub-cluter in each iteration might improve our indexing cheme performance. 3-Propoe a method to etimate prerequiite parameter k value uing election model method. 31

20 REFERENCES [1] Ritendra Datta, Dhiraj Johi, Jiali, and Jame Z. Wang. Image Retrieval: Idea, Influence, and Trend of the New Age, The Pennylvania State Univerity, ACM Tranaction on Computing Survey, [2] Shanmugapriya, N., and R. Nalluamy, A New Content baed Image Retrieval Sytem GMM and Relevance Feedback, Journal of Computer Science,VOL.10,NO.2, [3] CHRISTIANBÖHM, STEFAN BERCHTOLD, DANIEL A. KEIM, earching in high-dimenional pace-index tructure for Improving the performance of Multimedia Databae,ACM Computing urvey,vol.33,no.3, ,2001. [4] M.S.LEW, N.SEBE, C.DJERABA, and R. JAIN, Content-baed Multimedia Information Retrieval: State of the Art and Challenge, ACM Tranaction on Multimedia Computing, Communication, and Application, [5] V. Gaede and O. Günther, Multi-dimenional Acce Method, ACM Computing Survey, Vol. 30, No. 2, [6] Kraimir Markov, Kraimira Ivanova, Ilia Mitov and Stefan Karatanev, ADVANCE OF THE ACCESS METHODS, International Journal of Information Technologie and Knowledge, Vol.2, [7] Kauhik Chakrabarti, Sharad Mehrotra, The hybrid tree:an tructure for high dimenional feature pace,ieee international confererance on data engineering,1999. [8] M-R.keyvanpour, N.izadpanah, Analytical Claification of Multimedia Index Structure by Uing a Partitioning Method-Baed Framework, The International Journal of Multimedia & It Application (IJMA Vol.3, No.1, February [9] M-R.Keyvanpour,N.izadpanah, Claification and evaluation of High-dimenional Image Indexing Structure,The International journal of Machine learning and Copmuting,vol.2,No.3,2012. [10] C. Li and E. Chang and, H. Garcia-Molina and G. Wiederhold, Clutering for approximate imilarity earch in high-dimenional pace, IEEE Tranaction on Knowledge and Data Engineering, Vol.14,No.4,pp , [11] Hongli Xu, De Xu, and Enai Lin. An Applicable Hierarchical Clutering Algorithm for Content- Baed Image Retrieval.Springer-Verlag Berlin Heidelberg, [12] DANIEL BOLEY, Principal Direction Diviive Partitioning.Data Mining and Knowledge Dicovery 2, (1998. [13] Tian Zhang, Raghu Ramakrihnan, and Miron Livny. BIRCH: An Efficient Data Clutering Method for VeryLarge Databae. In Proceeding of the 1996 ACM SIGMOD International Conference on Management ofdata, Montreal, Canada, pp , [14] Guha S, Ratogi R, Shim K. CURE: an efficient clutering algorithm for large databae, Proc. of the ACM SIGMOD Int l Conf. on Management of Data. Seattle: ACM Pre, pp , [15] Aapo Hyvärinen and Erkki Oja, Independent Component Analyi: Algorithm and Application. Neural Network, Vol.13,No.(4-5,pp , [16] Y. Rui and T. S. Huang, Image Retrieval: current technique,promiing, Direction,and Open Iue. Journal of Viual communication and Image repreentation Vol.10, Iue. 1, pp.39-62,1999. [17] M. Taileb, S. Lamrou,and S. Touati, Non-overlapping Hierarchical Index Structure for imilarity earch International Journal of Computer Science, Vol. 3,No. 1, [18] J.E. Jackon. A Uer' Guide to Principal Component. New York: John Wiley and Son,Inc, [19] I.T. Jolliffe. Principal Component Analyi. Second edition, New York: Springer-Verlag New York, Inc, [20] Aapo Hyvärinen and Erkki Oja, Independent Component Analyi: Algorithm and Application. Neural Network, Vol.13,No.(4-5,pp , [21] Hyvärinen, A. and Oja, E., A fat fixed-point algorithm for independent component analyi. NeuralComputation, Vol.9,NO.7,pp: , [22] J. MacQueen, Some Method for Claification and Analyi of Multivariate Obervation. In Proc. 5th BerkeleySymp. Mat. Statit, Prob., 1, pp , [23] Sergio M. Savarei, Daniel Boley, Sergio Bittanti, Giovanna Gazzaniga. Cluter Selection in Diviive Clutering Algorithm. In Proceeding of SDM',2002. [24] Dantong Yu and Aidong Zhang, CluterTree: Integration of Cluter Repreentation and Nearet- NeighborSearch for Large Data Set with High Dimenion, IEEE tranaction on Knowledge and data Engineering, VOL. 15, NO. 3, MAY/JUNE [25] T.M. Mitchell, Machine Learning. McGraw-Hill,