Latent Visual Context Analysis for Image Re-ranking

Transcription

1 Latet Visual Cotext Aalysis for Image Re-rakig Wegag Zhou 1, Qi Tia 2, Liju Yag 3, Houqiag Li 1 Dept. of EEIS, Uiversity of Sciece ad Techology of Chia 1, Hefei, P.R. Chia Dept. of Computer Sciece, Texas State Uiversity at Sa Atoio 2, Texas, TX Microsoft Research Asia 3, Beijig, P.R.Chia zhwg@mail.ustc.edu.c 1, qitia@cs.utsa.edu 2, lijuy@microsoft.com 3, lihq@ustc.edu.c 1 ABSTRACT Curretly Web image search is mostly implemeted as text retrieval based o the textual iformatio extracted from the Web page associated with the image. Sice the text i the Web page may ot match with the image cotet, image search re-rakig is preferable to refie the text-based search results. I this paper, we propose a ovel scheme of latet visual cotext aalysis (LVCA) for image re-rakig. The latet visual cotext is explored i both latet sematic cotext ad visual lik graphs. We argue that the image sigificace is determied by its cotaied visual word cotext, which is aalyzed through Latet Sematic Aalysis (LSA) ad visual word lik graph. With the visual word cotext iformatio, the image cotext is explored by aalysis of image lik graph ad the sigificace value for each image ca be iferred by VisualRak. I both visual word lik graph ad image lik graph, latet-layer will be icorporated to effectively discover the visual cotext. We validate our approach o textquery based search results retured by Google Image. Experimetal results show improvemet of both accuracy ad efficiecy of our method over the state-of-the-art VisualRak algorithm. to exploit. Recetly, a ovel image idetificatio based commercial search egie Tieye [1] has bee attractig more ad more attetio with real-time respose for a billio scale database. But it is desiged oly for the duplicated image search. For text-based image search egie, with the igorace of visual cotet iformatio, the retured image search results may be oisy ad cluttered with irrelevat images. For istace, with Google Image, the text query America Gothic Paitig returs both good ad poor results, as show i Fig. 1(a). Sice users are more cocered with the relevace performace of top-raked images to the query cocept, if there is a post-processig stage such that good results are raked i the top, while poor results are moved to the rear, such as Fig. 1(b), it will be much more satisfactory. Categories ad Subject Descriptors H.4 [Iformatio Systems Applicatios]: Miscellaeous Geeral Terms Algorithms, Maagemet, Desig, Theory (a) Keywords Visual cotext, image re-rakig, visual lik graph, Latet Sematic Aalysis. 1. INTRODUCTION Image retrieval has become a popular service i may search egies, such as Big, Google ad Yahoo!. However, most of them are maily based o textual iformatio. This is partly due to the fact that text-based search techiques are successful ad mature while image cotet iformatio is difficult or expesive Permissio to make digital or hard copies of all or part of this work for persoal or classroom use is grated without fee provided that copies are ot made or distributed for profit or commercial advatage ad that copies bear this otice ad the full citatio o the first page. To copy otherwise, or republish, to post o servers or to redistribute to lists, requires prior specific permissio ad/or a fee. CIVR 10, July 5 7, 2010, Xi a, Chia. Copyright 2010 ACM /10/07... $ (b) Figure 1: A toy re-rakig example: top- ad bottom- 20 results from the first 500 images retured by Google Image via query America Gothic Paitig (a) before ad (b) after re-rakig. To address the above weakess of text-based image search, image re-rakig has become a active research topic i multimedia commuity. The goal of image re-rakig is to refie

2 the text-based image search results accordig to their visual cotet cosistecy, such that relevat image are moved to the top while irrelevat images sik to the bottom, resultig i better relevace. This is both ecessary ad of sigificace. Text-based image search egie respods promptly ad returs iitial search results. The, the visual cotet cosistecy ca be ehaced with image re-rakig ad better results will be retured to users. Figure 2: The framework of our Latet Visual Cotext Aalysis for image re-rakig. I this paper, we propose a framework of latet visual cotext aalysis (LVCA) for image re-rakig, as illustrated i Fig. 2. We argue that the image sigificace is determied by its cotaied visual word cotext ad explore the image cotext through discoverig visual word cotext. For image re-rakig, a key problem is how to measure the visual similarity. I this paper, two kids of visual cotext similarity, i.e., visual word similarity ad image similarity, are addressed. We propose to costruct visual cotext graphs for visual word ad image, respectively. I visual cotext graph, the ode deotes visual word (or image), ad the edge that liks two odes are weighted by their similarity. The visual similarity is formed by latet sematic aalysis ad visual lik aalysis. To explore the implicit sematic cotext, we itroduce Latet Sematic Aalysis [4] to discover the implicit sematic similarity for pair-wise visual words. To aalyze the visual lik cotext, a four-layer visual word lik graph ad a three-layer image lik graph are costructed, respectively. I the visual word lik graph, MSER [3] is used as a latet layer to impose weak local geometric costraits for visual words. I the image lik graph, visual word works as a latet layer ad the image liks are weighted by the visual word sigificace. After obtaiig the visual graphs for visual word ad images, similar to [8], PageRak [9] is used to explore the visual sigificace for visual words ad images, respectively. PageRak is the iitial idea of Google search egie for web-page rakig. Fially, all images are re-raked accordig to sigificace values of the latet visual cotext. As a summary, the mai cotributios of this paper iclude: (1) Propose a ovel framework to re-rak images by explorig latet visual cotext i both latet sematic ad visual lik aspects. The implicit sematic structure is aalyzed with Latet Sematic Aalysis. (2) Propose to costruct visual lik graphs to aalyze the lik cotext iformatio for visual word ad image, respectively. A four-layer visual word lik graph ad a three-layer image lik graph are built, respectively. I Visual word lik graph, the local geometric costraits are implicitly imposed i a latet layer with MSER regio [3]. The rest of the paper is orgaized as follows. Sectio 2 reviews the related works. Sectio 3 ad 4 discuss image rak ad visual word cotext learig, respectively. Sectio 5 presets experimetal results to evaluate our approach. Fially, coclusio ad discussio are give i Sectio RELATED WORK Our work is related to several research topics, icludig visual re-rakig, topic model, lik graph aalysis ad MSER detectio. The related literature is briefly reviewed below. I literature, there are may works about visual re-rakig based o differet schemes, such as clusterig-based [15][21], classificatio-based [19] ad the graph-based [8][16][17][18][20][22][24]. Geerally, a assumptio is made that visually similar images should be clustered close to each other. I all visual re-rakig methods, a essetial problem is how to measure the visual similarity. Curretly, the similarity is maily estimated based o image low-level features: global features [17][18][24], such as color momets ad Gabor feature, ad local features [8][20][22][24], such as SIFT (Scale Ivariat Feature Trasform) [2]. Global features work well for cases such as atural scee images, while local features do good job i rigid caoical object images. As the state-of-the-art approach, VisualRak [8] builds a image graph ad ituitively determies the pair-wise image similarity by the umber of shared SIFT features ad computes the image rak value directly through a iterative procedure similar to PageRak [9]. I fact, there is a uderlyig assumptio i VisualRak that all matched local features i image are equally importat. I fact, give a image set retured by text-based image search egie, some local features are expected to be more discrimiative tha others. Therefore, it is preferred to give these discrimiative features with a larger weight. I our approach, we ivestigate the visual word sigificace by aalyzig both latet sematics cotext ad visual word lik cotext, to ifer image sigificace. Our work is related to Bag of Visual Words (BoW) model [12] [13] [14]. BoW is origially derived from atural laguage processig ad popularly applied i image ad visio domais. Geerally, BoW geerates a codebook by clusterig image local features ad takes each cluster ceter as a visual word. The each image ca be compactly represeted with a histogram of the visual word occurreces. Based o BoW, may topic models, such as Latet Sematic Aalysis (LSA) [4], Probabilistic Latet Sematic Aalysis (plsa) [5] ad Latet Dirichlet Aalysis (LDA) [6] ca be applied to aalyze the topics withi images. As geerative data models, plsa ad LDA are based o statistical foudatio ad work with the umber of latet topics determied beforehad. LSA, istead, is based o Sigular Value Decompositio (SVD) to explore the higher-order sematic structure i a implicit maer without the latet topic umber. I this paper, LSA is adopted to explore the uderlyig implicit sematic cotext i cojuctio with visual words ad to geerate visual word similarity i latet sematic sese.

3 Our work is motivated by [10], where the webpage is segmeted ito differet uique blocks ad the webpage lik aalysis is coverted to multi-layer lik graph aalysis, so as to better explore the sematic topics of the web page. As for our problem, we regard the cotext relatioship betwee image ad visual word as visual hyperliks ad costruct visual word lik graph ad image lik graph, respectively. The, the visual sigificace discovery for visual words ad images is fulfilled by aalyzig these visual graphs. Our work is also related to [11], as both make use of MSER regio [3] to impose local geometric costraits. The differece is that, [11] exploits MSER to budle local features to improve the discrimiative power of visual word, while i our approach MSER is adopted as a itermediate layer to formulate the visual hyperlik cotext amog visual words. As a latet layer i our visual word hyperlik graph, MSER regio is used such that local features will be related with oly those sharig the same MSER regio i the image. 3. IMAGE RANK ANALYSIS 3.1 Image Rak As a aalogy to PageRak, VisualRak aalyze the visual lik structures amog images ad rak images accordig to images sigificace obtaied with Rakdom Walk [8]. The VisualRak iteratio equatio is defied as follows, S d U S ( 1 d) p, where 1 p (1) 1 where S the image sigificace vector, U is the columormalized pair-wise image similarity matrix. I VisualRak, the most importat issue is how to defie the pair-wise image similarity U. Istead of formulatig it as the umber of shared SIFT features betwee images [8], we defie it by image lik graph aalysis, as discussed i the ext subsectio. We argue that the image pair-wise similarity is determied by the sigificace of visual words. Ad visual word sigificace will be obtaied by latet visual cotext learig, which will be discussed i Sectio Image Lik Graph Geerally, images are related through itermediate medium of visual words, which work as visual hyperliks. Ad, differet visual words will vote differet weight to the image that cotais them, accordig to their sigificace. These cotext relatioships ca be represeted with a graph of three-layer, as illustrated i Fig. 3. Istead of direct trasitio, a image first trasmits to a visual word cotaied i it, ad the to aother image that shares the same visual word. Visual word plays a role of latet layer i the image lik graph. Based o the above discussio, the image trasitio probability ca be defied as follows, P 1 ( I j Ii ) P( I j Vk ) P( Vk Ii ) f ( Rk ) (2) Ni VkI j, Ii Figure 3: A illustratio of three layers betwee two odes i the image lik graph. where P ( I j Vk ) is defied as the iverse image frequecy of visual word V k for image I j, P( V k Ii ) deotes the ormalized term frequecy of visual work Vk i image I i, Rk is the sigificace value of visual word Vk which will be discussed i details i Sectio 4, f () is a o-decreasig fuctio ad N i is a ormalizatio factor such that the sum of trasitio probability for the i th image to ay other image is oe. I Eq. (2), f ( Rk ) weights visual lik that passes through visual word Vk ad works like a prior item. Therefore, images with may sigificat visual words will ejoy high probability to be propagated to. There are may choices for f (), such x as f ( x) e, f ( x) x. It should be oted that the image trasitio probability matrix is asymmetric. I fact, the symmetry property is ot required. For istace, cosider the case i which oe image is a cropped versio of aother image. It is obvious that the alterate coditioal probability is uequal. I Eq. (2), the image trasitio probability is obtaied. However, it does ot ecessarily defie the image pair-wise similarity, sice the more features regardless of importace a image cotais, the larger probability it is propagated from other images. Therefore, a regularizatio term should be icluded for pealizig images with too may features. We formulate the image visual similarity as follows, W ( j i j i, P( I I ) ( I ) (3) where I ) is the regularizatio term for the j-th image, defied as ( j 1 ( I j ) (4) N( I N j ) where N I ) deotes the umber of features i the j-th image, ( j N is the average local feature umber per image, is a costat. I our experimet, = 10. I Eq. (4), the term N works as a residue to prevet over-weightig those images with too few local features.

4 Cosequetly, U i Eq. (1) is defied as the columormalized versio of the traspositio ofw. 4. VISUAL WORDS CONTEXT LEARNING I this sectio, we will discuss how to discover the sigificace of visual words with visual cotext aalysis. Visual words are distiguished from each other via Radom Walk [8] [9]. Before Visual word rak, it is required to defie the pair-wise visual word similarity, which is formulated from latet cotext learig of both latet sematic cotext ad visual lik graphs. 4.1 Visual Word Similarity Decompositio Similar to VisualRak [8], the sigificace of visual word R is iteratively defied as follows, R d W R ( 1 d) p, where 1 p (5) 1 where W is the pair-wise visual word similarity which will be explaied by the followig Eq. (6), p is a distractig vector for radom walk behavior, ad d is a costat dampig factor. Usually, d 0. 8 is chose. The iteratio of Eq. (5) is cosidered coverged whe the chage of R is small eough or a maximal iteratio umber, such as 100, is achieved. Before re-rakig, it is ecessary to explore the pair-wise relatioships for visual words, which is equivalet to costruct a graph where the ode deotes visual word, ad the edge likig two odes is weighted by their similarity. Now the problem is how to defie the pair-wise image similarity. Visual similarity is related with huma psychological cogitio, which is a very complex process for simulatio. I this paper, we approach it from two aspects. The first oe is related with latet sematics, ad the secod oe is about visual lik graph. We propose to formulate the similarity defiitio for visual word pair (i, as follows, where W Ws s g ( i, W ( i, (1 ) W ( i, (6) is latet sematics related, Wg is visual lik graph related, ad is a weightig factor with rage 0 1. I the followig subsectios, we will explai the formulatio of the above two decomposed similarity compoets for visual words. Oce W is obtaied, W i Eq. (5) is defied as the colum-ormalized versio of the traspositio of W 4.2 Latet Sematic Similarity Aalysis Sice our cadidate images are retured from text query retrieval, it is reasoable to assume that there exist some topics amog these images. The latet sematic cotext uderlyig the visual word-image relatioship ca be explored by meas of sematic model. Geerally, image topic is difficult to be represeted explicitly. Also, this is ot ecessary for our case. LSA, origially proposed for text idexig ad retrieval ad proved to be powerful for discoverig the implicit higher-order cotext i the associatio of terms with documets [4], ca serve for our task. Accordig to LSA, give a visual word-image matrix M0 with size m, each colum of which is a ormalized histogram of visual word occurrece i the correspodig image, it ca be decomposed ito the product of three other matrixes by sigular value decompositio (SVD) as follows, T M0 T0 S0 D0 (7) where T 0 ad D 0 are colum-orthoormalized matrices ad S 0 is a diagoal matrix with all diagoal elemets positive ad i decreasig order. The sizes of T 0, S 0 ad D0 are m k, k k ad k, respectively. The beauty of SVD is that it provides a simple strategy for optimal approximate fit usig smaller matrices. To maitai the real data structure ad at the same time igore the samplig error or uimportat details, oly the top t ( t k ) largest diagoal elemets i S 0 are kept while the remaiig smaller oes are set to zero. This is equivalet to delete the zero rows ad colums of S 0 to obtai a compact matrix S ad delete the correspodig colums of T 0 ad D0 to yield T ad D, respectively. Cosequetly, a reduced matrix M ca be defied as follows, T M T S D (8) which is the rak- t model with the best possible least squares-fit to M 0 [4]. Geometrically, the rows of the reduced matrices, i.e., T a D, ca be regarded as coordiates of poits represetig the images ad visual words i a t -dimesioal space. The amout of dimesio reductio of S 0, i.e., the value of t, is a sigificat issue ad usually determied by operatioal criterio. If t is too large, M will be sesitive to oise. O the other had, if t is too small, the latet structure may ot be kept. Therefore, some trade-off should be made. I our implemetatio, we empirically fid best results with t mi( i S0 ( i, i) S0 (0,0) / 30). Assume row-ormalizig M yields M. The the dot product R betwee two row vectors of M reflects the extet to which two terms have a similar patter of occurrece across the set of images R [4]. Therefore, the pair-wise row vector distace of Y ca be defied as, R R R T M ( M (9) U ) Each etry i U is a cosie distace betwee two ormalized vectors, with value ragig from -1 ad 1. We keep those similar visual word pairs ad discard those dissimilar oes. I implemetatio, pair-wise visual word similarity i the sese of latet sematic cotext is defied as follows, W s ( i, max( U( i,, 0) (10)

5 4.3 Visual Word Lik Graph Visual word is a kid of visual cocept atom ad itrisically related through image, which works as visual cocept carrier. Usually, visual cocept is composed of a set of visual atoms uder some geometric costraits. Therefore, it is ecessary to icorporate the local geometric relatioship amog visual words to aalyze the lik cotext betwee visual words. MSER regio ca serve for this task. Usually, the MSER detector geerates a relatively small umber of regios per image with high repeatability. Such MSER regio itrisically imposes local geometric costraits for its cotaied visual words. Cosequetly, a four-layer visual word graph is costructed. As illustrated i Fig. 4, there are two itermediate layers, i.e., image layer ad MSER layer. Visual words do t trasit to each other directly. Istead, a visual word Vi first trasit to a image that cotaisv i, the further to the MSER regio i the image, ad fially to aother visual word V j that shares the same MSER regio. The ituitio behid is that if a user is viewig a image, he or she is most likely attracted by some local features, ad other local features withi the eighborhood may also be of iterest. Such trasitio behavior aturally reflects the co-occurrece cotext of visual words. Figure 4: A illustratio of four layers betwee two odes i the visual word lik graph. The red ellipses i layer 3 deote the detected MSER regios. Based o the visual word graph, we essetially defie a propagatio matrix W o the edges of the graph from i terms of probability as follows, N N k P( V j Vi ) P( V j M k, t ) P( M k, t Ik ) P( Ik Vi ) (11) k1 t1 where N deotes the total umber of images to be raked, N deotes the umber of MSER regios i the k -th image, k M, deotes the set of visual words i the t -th MSER regio i k t the k -th image, P( V j M k,t ) is defied as the ormalized termfrequecy of the visual word V j i k t defied as the ormalized MSER-frequecy of M, ; P M k I ) is (, t k M k, t i image I k. P I k V ) is defied as the iverse image frequecy of visual ( i word Vi for image I k. Further, we simply defie the visual lik similarity for visual words as follows, g ( j i W i, P( V V ) (12) 5. EXPERIMENTS To validate the effectiveess of our Latet Visual Cotext Aalysis (LVCA) scheme, we coducted experimets with images collected from the Web. Thirty famous ladmark related text queries are gathered ad 500 full size images for each query are dowloaded through Google Image. Typical queries iclude Coliseum, Licol Memorial, Eiffel Tower, Statue of Liberty, ad so o. Each image is labeled with groud truth accordig to its relevace to the correspodig text query o four levels, i.e., Excellet, Good, Fair, Irrelevat. Ladmark queries are selected for several cosideratios. First of all, they are popular i Web queries. Secod, they cotai caoical objects ad fit themselves well to the type of local features used i our study. For each image, we extract the widely used SIFT features, with a stadard implemetatio. The DOG detectors are used for key poit detectio ad a 128-dimesioal orietatio histogram is extracted to describe the local patch aroud the key poits. Before feature extractio, images are scaled to have a maximum axis size of 400. From our study, the average SIFT feature umber for a sigle image is 680. For SIFT quatizatio, a hierarchical visual vocabulary tree [13] with 4 levels ad 10 braches for each oleaf ode is adopted. With 1 millio samples out of 10 millio SIFT features from a idepedet image dataset (radom crawled by image URLs) for clusterig, a visual codebook of 10 thousad visual words is obtaied. To evaluate the performace, we adopt Normalized Discouted Cumulative Gai (NDCG) [23] [24], which is widely used i iformatio retrieval evaluatio ivolvig more tha two relevace levels. Give a rakig list, the NDCG score at positio is defied as, NDCG r( i) 2 Z (13) log(1 i) i1 where r(i) is the relevace score of i-th image i the rakig list, Z is the ormalizatio factor which is chose such that NDCG@ for the perfect rakig list is 1. Cosiderig that users are more cocered about the top raked images, we therefore oly take ito accout those images rakig top 100. For our algorithm, there is oe parameter to be determied. The re-rakig performace for differet values of is illustrated i Fig. 5. It ca be observed that latet sematic similarity ad visual word hyperlik similarity complemet each other ad a trade-off is achieved whe 0. 3.

6 = 0.0 = 0.1 = 0.2 = 0.3 = 0.4 = 0.6 = 0.8 = Text VisualRak LVCA Figure 5: Performace (NDCG) for differet values of. Best performace is obtaied with Figure 7: Performace compariso of re-rakig results betwee VisualRak ad LVCA. The baselie is text-based search results No visual word weightig No MSER latet layer Stadard LVCA Figure 6: Performace compariso with ad without visual word weightig ad MSER layer. The stadard LVCA icludes both visual word weightig ad MSER latet layer. Oe of the key issues of our approach is to weight the image graph with visual word sigificace value. To demostrate the ecessity of the visual word weight for image graph, we set f ( x) 1 i Eq. (2) ad keep the other compoets uchaged. The we compare the results with ad without visual word sigificace, as show i Fig. 6. It ca be observed that, without weightig for the visual word layer, the performace will suffer from dramatic decrease. To demostrate the ecessity of MSER layer for our visual word graph, we replace the origial four-layer visual word graph with a three-layer graph, igorig the MSER layer, ad formulate the visual word trasitio probability as follows, M P( V V ) P( V I ) P( I V ) (14) j i k1 where P ( V j Ik ) deotes the term frequecy of V j i image ad P ( I k Vi ) is the same as previous defiitio. Ad other compoets of the framework stay the same. The, the altered versio is evaluated with the same ladmark dataset ad the results are compared with that of the stadard LVCA approach, as illustrated i Fig. 6, from which we ca observe that with the latet MSER layer for LVCA, improved performace ca be achieved. j k k i I k Figure 8: Average time cost compariso betwee VisualRak ad our LVCA approach for re-rakig 500 full size images per query. Feature extractio time is ot icluded. Fig. 7 illustrates the compariso betwee VisualRak ad LVCA, with the text search results as the baselie. It ca be observed that LVCA yields better re-rakig results tha VisualRak. I additio, 5 examples of re-rakigs results are also give. I Fig. 9, the top 30 retured results of both VisualRak ad our approach are show for each query, icludig Chiche Itza, Petra i Jorda, Tower Bridge, Sydey Opera House ad Temple of Heave. It ca be observed from these istaces that, besides the improved relevace, the image cosistecy to query is also greatly ehaced for our approach. Our experimets are performed o a server with 2.0G Hz CPU ad 16G memory. Without cosiderig the time for feature extractio, it takes a average of 3.4 miutes to re-rak 500 full size images per query with our algorithm, while for VisualRak the average time cost is miutes per query, about two orders of magitude more tha our approach, as illustrated i Fig. 8. I fact, the expesive computig of VisualRak is caused by the image pair-wise similarity computatio. As for our approach, the most time-cosumig part is the SIFT quatizatio ad SVD decompositio of a sparse matrix for LSA. 6. CONCLUSION AND DISCUSSION We propose a ovel framework of latet visual cotext aalysis (LVCA) for image re-rakig. We explore the visual cotext structure by aalyzig visual lik graphs ad Latet Sematic

7 Aalysis. Experimet with ladmark Web image dataset demostrates the superiority of the proposed approach over the state-of-the-art VisualRak [8]. I this paper, we select the local feature, ad MSER regio for image represetatio. Therefore, it is well suited for re-rakig image set of rigid caoical objects, such as ladmarks. It also works well for most product images, which will be tested i the ext work. However, for cases such as atural scee images, where the image cotet caot be satisfactorily characterized by local features, our approach may fail to work. I this work, Sigular Value Decompositio (SVD) is adopted whe performig LSA. Due to the computatioal complexity, it is ot applicable to re-rak a large amout of images, say, over oe millio. However, i practice, it is reasoable to assume the top text-search results are i good quality. Therefore, it will be sufficiet to re-rak oly top, such as 500, images, with acceptable computatioal time cost. I the future, we will icorporate global feature ito our framework to deal with cases where local features are isufficiet to work. Besides, the iitial image rakig order iformatio, which is igored i our approach, will also be explored. Also, discoverig more represetative regioal visual word set with MSER or some other regioal costraits is the ext research directio. Moreover, as for our three-layer image visual lik graph, ituitively, it is more preferred to geerate a middle-layer image graph, similar to the visual word lik graph, with visual word set i MSER regio as a latet layer. However, this ivolves extractig represetative MSER visual word sets by clusterig potetial samples of these sets. Due to the imperfect repeatability of local features ad quatizatio error iduced by BoW, the potetial variats of similar MSER visual word sets are umerous. I our ext work, we will try to summarize item-set patter for MSER visual word set ad itroduce a latet MSER layer. Further, although image re-rakig ca improve the relevace of top-raked images to the text query, image redudacy, such as duplicated or very similar images as show i Fig. 1(b), is still prevalet. If caoical images, a subset of photos that best summarize a photo collectio, ca be selected so as to compress the image redudacy, the the user experiece of image exploratio will be greatly improved. That is ot our focus i this paper ad will be explored i our future work. 7. ACKNOWLEDGEMENTS The work of Wegag Zhou ad Houqiag Li is supported by NSFC uder cotract No ad Program for New Cetury Excellet Talets i Uiversity (NCET). 8. REFERENCES [1] [2] D. Lowe. Distictive image features from scale-ivariat key poits. IJCV, 60(2):91-110, [3] J. Matas, O. Chum, M. Urba, ad T. Pajdla. Robust wide baselie stereo from maximally stable extremal regios. I Proc. BMVC, [4] S. Deerwester, S. T. Dumais, R. Harshma. Idexig by latet sematic aalysis. Joural of the America Society for Iformatio Sciece, 41(6), [5] T. Hofma. Probabilistic latet sematic idexig. I ACM SIGIR, [6] D. M. Blei, A. Y. Ng, M. I. Jorda ad J. Lafferty. Latet dirichlet allocatio. Joural of Machie Learig Research, [7] B. J. Frey ad D. Dueck. Clusterig by passig messages betwee data poits. I Sciece, vol. 315, pp , [8] Y. Jig ad S. Baluja. VisualRak: applyig PageRak to large-scale image search. IEEE Tras. o PAMI, 30: , [9] S. Bri ad L. Page. The aatomy of a large-scale hypertextual(web) search egie. I The Seveth Iteratioal World Wide Web Coferece, [10] D. Cai, X. He, J. We, ad W. Ma. Block-level lik aalysis. I ACM SIGIR, [11] Z. Wu, Q. Ke, M. Isard, J. Su. Budlig features for largescale partial-duplicate web image search. I Proc. CVPR, [12] J. Sivic ad A. Zisserma. Video Google: A text retrieval approach to object matchig i videos. I Proc. ICCV, [13] D. Nister ad H. Steweius. Scalable recogitio with a vocabulary tree. I Proc. CVPR, pages , [14] Xiao Zhag, Zhiwei Li, Lei Zhag, Wei-Yig Ma, Heug- Yeug Shum. Efficiet Idexig for Large Scale Visual Search. I Proc. ICCV, [15] W. H. Hsu, L. S. Keedy, ad S. F. Chag. Video search rerakig via iformatio bottleeck priciple. I ACM Multimedia, pp , [16] J. Liu, W. Lai, X. Hua, Y. Huag ad S. Li. Video search rerakig via multi-graph propagatio. I ACM Multimedia, pp , [17] X. Tia, L. Yag, J. Wag, Y. Yag, X. Wu, ad X. Hua. Bayesia video search rerakig. I ACM Multimedia, [18] H. Zitoui, S. Sevil, D. Ozka, ad P. Duygulu. Re-rakig of web image search results usig a graph algorithm. I ICPR, pages 1 4, [19] R. Ya, A. G. Hauptma ad R. Ji. Multimedia search with pseudo-relevace feedback. I CIVR, [20] W. H. Hsu, L. S. Keedy ad S. F. Chag. Video search rerakig through radom walk over documet-level cotext graph. I ACM Multimedia, pp , [21] N. Be-Haim, B. Babeko, ad S. Belogie. Improvig webbased image search via cotet based clusterig. I SLAM, pp , [22] S. Zhag, Q. Tia, G. Hua, ad Q. Huag ad S. Li. Descriptive visual words ad visual phrase for Image applicatios. I ACM Multimedia, [23] K. Jarveli ad J. Kekalaie. IR evaluatio methods for retrievig highly relevat documets. I ACM SIGIR, 2000.

8 [24] L. Wag, L. Yag, X. Tia. Query Aware Visual Similarity Propagatio for Image Search Rerakig. I ACM Multimedia, 2009 (f) (a) (g) (b) (h) (c) (i) (d) (e) ( Figure 9: Top 30 re-rakig results of VisualRak (a) (c) (e) (g) (i) ad LVCA (b) (d) (f) (h) ( for image set obtaied from Google Image with text query Chiche Itza, Petra i Jorda, Tower Bridge, Sydey Opera House ad Temple of Heave, respectively.