Transcription

1 3 Conference on Inforation Sciences and Systes, The Johns Hopkins University, March, 3 Sensitivity Characteristics of Cross-Correlation Distance Metric and Model Function F. Porikli Mitsubishi Electric Research Laboratories 55 Central Avenue Murray Hill, New Jersey 797 e-ail: fatih@erl.edu Abstract We present a 3-fold etric and a transfer function to evaluate the siilarity of two finite length sequences. We analyze the sensitivity characteristics of the proposed etrics for Gaussian shape functions. Our ethod is based on cross-correlation atrix analysis and extrapolation of a iniu cost path using dynaic prograing. Unlike the existing sequential (bin-by-bin) and non-sequential (crossbin) approaches that copute a single scalar as a result of the easureent, we calculate the as well as deterine how two sequences are correlated with each other in ters of a non-paraetric transfer function. We shown that the proposed etrics provide better discriination than conventional etrics do. Furtherore, we show that we can reduce our etric to any one of sequential etrics with suitable siplification. I. Introduction Distance between two sequences is one of the ost coon easures used in coputer algoriths for sequence analysis. Fro iage retrieval in ultiedia databases to coparison of aino acid sequences for DNA pattern recognition, it is used in various areas. However, past studies have shown that ost etrics are neither robust to sall shape deforations of sequences nor nonlinear shifts on the indexing axis. Furtherore, there is no etric that can copute the and evaluate the alignent of sequences in ters a transfer function at the sae tie. A finite-length sequence, h, is a vector [h[];:::;h[m]] in which each bin h[] is the value of the vector at the index nuber. In case h represent an iage histogra, h[] contains the nuber of pixels corresponding to the color range of in the iage I where M is the total nuber of the bins. In order words, it is a apping fro the set of color vectors to the set of positive real nubers R +. In this paper, we assue that bins are identical i.e. sapling frequency of the indexing axis is constant i P i = j j, and the sequences are M noralized such that h[] =. = II. Cross-Correlation Distance (CCD) We define a cross-correlation atrix C between two sequences as the set of positive real nubers that represent the bin-wise s. Let h [] and h [] be two sequences with =;:::;M and =;:::;N i.e. the lengths are not necessarily sae. The cross-correlation atrix is C M N = h Ω h = c c ::: c N c : : : : : c M ::: c MN where each eleent is a positive real nuber, and () c n = d(h [];h [n]) () where d( ) is a nor which satisfies the triangleinequality. As a atter of fact, this definition stands for the dissiilarity of sequences instead of their correlation. The correlation can be easily established by defining c n = d(h [] h [n]). Theore The su of the diagonal eleents of C represents the bin-by-bin with given nor d( ) if the sequences have equal nuber of bins M = N. For exaple, by choosing the nor as L, the su of the diagonals becoes the agnitude between a pair of sequences c = jh [] h []j = d L(h ;h ): (3) Let p : f( ;n ); :::; ( i;n i); :::; ( I;n I)g represents a iniu cost path (defined in the next section) fro the c to c MN in the atrix C, i.e. the su of the atrix eleents on the connected path p gives the iniu score aong all possible routes. The total length of the path cannot be ore than the su of the lengths of the sequences p M + N» I» M + N () We define a cost function for the path as g(p i)=c i ;n i where p i denotes the path eleent ( i;n i). We define a apping i! j fro the path indices to the projection onto the diagonal of the atrix C, and an associated transfer function f(j) that gives the fro the diagonal with respect to the projection j. The transfer function is a apping fro the atrix indices to real nubers ( i;n i) t! f(j) (5) where j =;:::;J and J = p M + N. Depending on the shape of the path, these appings ay not be one-to-one.

2 Figure : The figure shows the relation between the iniu cost path and f(j). Figure : Each vertex represents a atrix index cobination and each edge is the corresponding atrix eleent for that index. Fro Fig., the angle between the diagonal and the current path index is = tan M N tan i n i Without loss of generality, we ay assue M = N, i.e. tan ( M N ) = ß. Then, the agnitude of the projection j is () j = jp ij cos (7) = p i + n i cos ß arctan i n i = () i + ni p (9) Thus, the transfer function f(j) becoes f(j) = j +( i + n i ) () = i + n i + ini () The f(j) is negativeif i <n i. The apping t in equation 5 is decoposed into two functions t ( i)=n i and t n(n i)= i such that they give the iniu cost path as a function of sequence index. Their derivatives with respect to both indices represent the aount of warping of the ( i) = t ( i) t ( i ) n(n i) = t n(n i) t n(n i ) (3) It is straightforward to derive the following properties f(j) = ) i = n i () f(j) > ) i >n i (5) f(j) < ) i <n i = ( i)=@t n(n i) > ( i) <@t n(n i) < ( i) >@t n(n i) (9) where the derivative of f(j) with respect to j is liited in range ß Definition The cross-correlation (CCD) is the total cost along the transfer function (CCF) d CC(h ;h )= IX i= jg( i;n i)j () An alternative definition of the above etric weights the transfer function with the current cost d CC(h ;h )= JX j= jf(j)jg(( i;n i)) () The can be easured as the length of the iniu cost path as well d CC(h ;h )=J: () III. Dynaic Prograing Dynaic prograing is an approach developed to solve sequential, or ulti-stage, decision probles []. Basically, what dynaic prograing approach does is that it solves aulti-variable proble by solving a series of single variable probles. The essence of dynaic prograing is Richard Bellan's Principle of Optiality. This principle is intuitive: fro any point on an optial trajectory, the reaining trajectory is optial for the corresponding proble initiated at that point. Given two sequences, the question is what is the best alignent of their shapes and how can the alignent be deterined? We reduce the coparison of two sequences to finding the iniu cost path in a directed weighted graph. A iniu cost path fro a vertex to another vertex in a directed graph is a path that has the sallest total edge-weights aong all paths fro the sae source vertex to the sae destination vertex. Let v beavertex and e be an edge between the vertices of a directed weighted graph. We associate a cost to each edge!(e). We want to find the iniu cost path by oving fro an origin vertex v to a destination vertex v S. The cost of a path p(v ;v S)=fv ; ::; v Sg is the su of its constituent edges Ω(p(v ;v S)) = SX s!(v s) (3) Suppose we already know the costs Ω(v ;v Λ) fro v to every other vertex. Let's say v Λ is the last vertex the path goes through before v S. Then, the overall path ust be fored by concatenating a path fro v to v Λ, i.e. p(v ;v Λ), with the edge e(v Λ;v S). Further, the path p(v ;v Λ) ust itself be a iniu cost path since otherwise concatenating the iniu cost path with edge e(v Λ;v S) would decrease the cost of the overall path. Another observation is that Ω(v ;v Λ) ust be equal or less than Ω(v ;v S), since Ω(v ;v S)=Ω(v ;v Λ)+!(v Λ;v S) and we are assuing all edges

3 have non-negative costs, i.e.!(v Λ;v S). Therefore if we only know the correct value of Ω(v ;v Λ)we can find a iniu cost path. We odified Dijkstra's algorith is odified to find the shortest paths between one source vertex and all the other vertices which are the destinations. To find all iniu cost paths between all pairs of vertices we need to apply it to each of the vertices as a source vertex. Let Q be the set of active vertices whose iniu cost paths fro v have already been deterined, and ~p(v) isaback pointer vector that shows the neighboring iniu cost vertex of v. Then the iterative procedure is given as. Set u = v Q = fu g, Ω(u ) =, ~p(v ) = v, and!(v) = for v = u.. For each u i Q: ifv is a connected to u i, assign!(v) ψ inf!(u i); Ω(u i)+!(v)g. If!(v) is changed, assign ~p(v) =u i and update Q ψ Q [ v. 3. Reove u i fro Q.. If Q = ; go to step. Then the iniu cost path p(v ;v s) = fv ; :::; v Sg is obtained by tracing back pointers by starting fro the destination vertex v S as v s = ~p(v s). The algorith takes tie O(S ). As shown in figure, the graph that is converted fro the cross-correlation atrix is directed such that a vertex v n has directional edges to vertices v +;n;v ;n+;v +;n+ only. Therefore, we do not allow overlaps of the bin indices, and eliinate cyclic paths. The dynaic prograing can be applied to obtain the partial atches between two sequences. To find the best atch for the part [ a; :::; b ] of the first sequence in the second sequence, we odify the initial conditions such that the initial vertex is iteratively assigned to ( a;n ), where n =; ::; N, and the target vertex is chosen as ( b ;n ) where n = M N a; ::; N. The above process is repeated for every cobination and the iniu cost path is chosen. IV. Case Study: Illuination Copensation We tested the proposed transfer function to recover distorted color histogras. The intensity values of an input iage Fig.3-a is distorted non-linearly by hand to obtain its over-exposed version Fig.3-b. We extracted histogras (Fig.3- c, upper graphs) of the input and over-exposed iages. We coputed the cross-correlation atrix using these histogras. As the kernel, we used the L nor. Then, we found the iniu cost path within the cross-correlation atrix (Fig.3-d) by starting fro the lower-right end of the atrix and tracking towards to upper-left corner as explained in the dynaic prograing section. We reapped the intensity values of the over-exposed iage using transfer function that is obtained by projecting the iniu cost path (Fig.3-c, lower graph) on the ain diagonal as explained in the crosscorrelation section. Note that, the distortion is not linear, and it is not paraetric either. The result of the copensation is given in Fig.3-e. We observed that the transfer function atches with the non-linear distortion characteristics. As visible in the histogra graph (Fig.3-c), the nonparaetric transfer function successfully copensated for the non-linear distortions by taking the advantage of the non-paraetric transfer function (d) (c) Figure 3: A saple iage, its over-exposed version. (c) The upper graph shows the histogras of the input iage (black), over-exposed iage (blue), and the copensated iage (red). The lower graph is the transfer function. (d) The cross-correlation atrix and the iniu cost path (yellow). Higher red values indicate saller s. (e) The copensated iage. We repeated the sae analysis using several other color/gray-level iages. We observed that the corrected iages visually are uch siilar to the originals after the copensation. Their color histogras are accurately aligned as well. The iproveent is substantial even though histogra operations are invariant to spatial transforations, and thus have only liited ipact. We coputed the of two histogras such that the score allows the aount of non-linear, non-paraetric but sequential alignent of the two histogras. Note that, no other etric can give such a copensated. (e) V. Sensitivity Analysis for Gaussian Functions We analyzed the sensitivity characteristics of the proposed etric for Gaussian shaped sequences. We generated a reference Gaussian sequence with zero and unit N (; ), and copared it with a set (Set-) of Gaussians se-

4 quences N (k; ) where k : ; ::;, i.e. their s are sae but the s are different, as plotted in Fig.5-a. We also tested another set (Set-) of zero Gaussians with different s N (; k), k : ; ::; (Fig.5-b). We coputed s between the original Gaussian and the Set- for the etrics that are described in the Appendix and also the corresponding CDD s using both the total cost and the total length nors as defined in equations and. We presented these results in Fig. 5-c. As visible, the total cost nor is shift invariant. Then, we coputed s for the Set-. The corresponding graph is given in Fig.5-d. We observed that the Kologorov-Sirnov, Lorentzian and Intersection s have alost identical responses, and the Bhattacharyya and Kullback-Leibler s have siilar results for the Set-. For sae- shifted- case (Set-), the Lorentzian and agnitude s have siilar responses. As visible, one nor of the CCD etric (total cost) can identify the sae shape sequences while another nor (total length) can effectively detect the differences for the Gaussian shape functions. An ideal etric is supposed to have linear response for linearly varying s and s of the input sequences in our case. Most of the above etrics satisfies this constraint. The graphs obtained using the CCD show that it is linearly proportional to the linear changes of the input sequence. The graphs justifies that the proposed etrics are sensitive to the changes of the and values for Gaussian shape functions. We also evaluated each etric described in the Appendix and our CCD etrics for varying and values as given in Fig.. We observed that the CCD etrics are very sensitive to the shape changes of the input sequences. Even the value of the sequences are diverges, our etrics accurately identify deviations. On the other hand, ost other conventional etrics lose their sensitivity and becoe inversely proportional to the changes in case of severe shifts. VI. Discussion on Distance Measures A ajor drawback of the bin-by-bin easures (Minkowski, Intersection, Lorentzian, Chi, Bhattacharyya, etc.) is that they account only for the correspondence between bins with the sae index, and do not use inforation across bins. A shift of the bin index ay result in larger s although the two sequences otherwise have the sae values. For iage histogras, quantization is yet another consideration; a slight change in lighting conditions ay result in a corresponding shift in the color sequence, causing these etrics to isjudge siilarity copletely. Contribution of the epty bins is also iportant. Weighted versions of the Minkowski etric ay underestiate s because they tend to accentuate the siilarity between color sequences presenting any nonepty bins. Furtherore, not all sequences have the sae nuber of bins. Yet, the bin size ay not be identical within the sae sequence either. The bin-by-bin easures do not allow atching different size sequences, while the CCD does. The Hausdorff provides the best echanis to handle partial atches, as well as the sequence intersection, the quadratic, the EMD, and the CCD. Since the K- L divergence evaluates only the relative between the given sequences by using one of the as a reference, it is not syetric, thus it does not qualify as a etric. For ost of L (c) K L (e) K S (g) Infinity (i) L (d) Bhattacharyya (f) Intersection (h) Figure : The CCD, which is coputed using the total length definition, between N (; ) and N (μ; ff )'s where μ =::;ff =::, and the CCD s that are coputed using total cost definition. (c) The agnitude s, (d) the Euclidean s, (e) the Kullback- Leibler s, (f) the Bhattacharyya s (g) the Kologorov-Sirnov s, (h) the intersection s, (i) the Minkowski s for L, and (j) the χ s. Except the CCD, ost etrics lose sensitivity and becoe inversely proportional to the changes in case of severe shifts..... Chi (j)

5 L L Inf Lorent. Inters. Bhatt. Chi KL KS CCD CCD L L Inf Lorent. Inters. Bhatt. Chi KL KS CCD CCD Figure 5: Set of Gaussian shape functions (Set-) with different values, with different s (Set-). (c) The graphs of the noralized s between the original Gaussian N (; ) and other Gaussian sequences in Set-. The cross-correlation s are coputed by Eq. (blue) and Eq. (red). The horizontal axis is the value. (d) The graph of the noralized s coputed for Set-. The horizontal axis is the. the easures, the triangle inequality, which is iportant for iage retrieval, holds only for specific cases. Only the CCD has the ability to find a non-linear, non-paraeterized odel of the color warping between the sequences. This property is especially iportant in prediction of lighting changes. The quadratic requires an abiguous co atrix that states the perceptive relation between the color bins. The choice of the co etric effects the qualification of the quadratic as a etric. The Hausdorff does not qualify as a etric, and it overestiates the siilarity of two sequences if there is a partial atch. Not all easures can be extended to the ulti-diensional sequences, e.g. the Kologrov-Sirnov statistic. Coputational coplexity ofthe cross-bin easures are higher than the bin-by-bin easures. In cases of coputing the where the nuber of bins is large, or sequences are ulti-diensional, the EMD, the Hausdorff, and the quadratic for becoe infeasible. Although cross-bin atching is possible for EMD, the Hausdorff, and the quadratic, these ethods do not have any echanis to preserve the ordering of the color bins. Obviously, changing the order of the color bins ay significantly deteriorate the accuracy of the iage since a sequence is already a arginal. The CCD, on the other hand, preserves the order of bins while atching. None of the easures has the ability to recover a apping function that transfers one sequence to other except the CCD. VII. Conclusion In this contribution, we investigated the sensitivity properties of our cross-correlation atrix and dynaic prograing based etric for Gaussian shape functions. We showed that the proposed etric is sensitive to the and variations of the input sequences. Our etric evaluates the siilarity of two finite length sequences and deterines a non-paraetric transfer function that accurately aligns the input sequences. The ay be coputed using one of the three proposed definitions which are the total cost on the iniu cost path, the length of the iniu cost path, and the total area under the transfer function. The total cost nor is invariant to changes, and it easures the shape divergence of the sequences. The length and total area nors react the shape isatches as well as they detect the changes. The transfer function copensates for the non-linear warping of the sequences. This is an additional functionality which is crucial in histogra atching applications. Our etrics also provide better discriination than conventional etrics, and allow the atches of the epty bins which can not be done by the ost other bin-by-bin etrics. Furtherore, our etric can reduce to any one of the bin-by-bin etrics by suitable siplification. References

6 [] A. Bhattacharyya, On a easure of divergence between two statistical populations defined by their probability distributions", Bull. Calcutta Math. Soc., Vol. 35, 999, 93 [] M. Black and A. Rangarajan, On the unification of line processes, outlier rejection and robust statistics with application in early vision", Journal Coputer Vision, Vol 9, 57-9, 99. [3] J. Hafner, H. S. Sawhney, W. Equitz, M. Flickner, and W. Niblack. Efficient color histogra indexing for quadratic for functions," IEEE. Transactions on Pattern Analysis and Machine Intelligence, July 995. [] R. Keeney and H. Raiffa, Decisions with ultiple objectives", Wiley, 97. [5] S. Kullback, Inforation theory and statistics", Dover, 9. [] Y. Rubner, J. Puzicha, C. Toasi, and J. M. Buhann. Epirical evaluation of dissiilarity easures for color and texture". Coputer Vision and Iage Understanding, Vol, 5-3,. [7] M. Swain and D. Ballard, Color Indexing", International Journal of Coputer Vision, Vol. 7, pp.3, 99. Appendix The Minkowski [7] is a generalized for of coon spatial nors such as the agnitude L, the Euclidean L, and the axiu L. It is defined as d Lp (h;h) = ψ! =p jh[] h[]j p = () The higher order nors (p >) exponentially weight the absolute difference, thus they are ore sensitive to the isatches. The Lorentzian [] is frequently used in robust estiators to iniize the effect of the outliers. It is defined as d R (h;h) = log ( + jh[] h[]j) (5) = Usually, a scaling factor is used to noralize the absolute difference ter. The sequence intersection is defined by the area of the overlap between two sequences P M = in (h [];h[]) d (h;h) = in PM = h []; P M n= h [n] () The Bhattacharyya [] is a separability easure between two Gaussian distributions. We adapted it for sequence coparison as d B (h;h) = ln Xp h()h() (7) The χ weights inversely the squared differences between color bins by the expected frequency, and tends to equalize the contributions of rare and frequent color values to the etric structure of the space d χ (h;h) = = (h[] h[]) h[]+h[] () The Kullback-Leibler (K-L) is perhaps the ost frequently used to evaluate the between two sequences of rando variables that have the sae Markovian dependence order [5] because of its geoetrical iportance. d KL (h;h) = X h[] log h [] h[] (9) However, the K-L is non-additive and nonsyetric, besides it requires identical bins. The quadratic [3] is given by d Q (h;h) = h[; n]a n h[; n] (3) = n= where the coefficient h[; n] = jh[] h[n]j, and the co atrix eleent a n is based on the perceptual siilarity of the colors and n, which is expressed as a n = h [; n] ax h (3) When a ground that atches perceptual dissiilarity is available for single features, incorporating this inforation results in perceptually ore ingful dissiilarity easures for distributions of features. The Earth Movers (EMD) is defined as d E (h;h) = P P n d(h []h[n])f P P n n f n (3) where f n stands for the flow between h[] and h[n] that iniizes an overall cost function. Given two distributions, one can be seen as piles of earth in feature space, the other as a collection of holes in that sae space []. The between two color distributions is defined as the iniu aount of work needed to transfor one color distribution into the other. The Hausdorff coputes the degree of isatch between two sequences as the axiu between the colors d H (h;h) = ax ax n h ax in in n jh[] h[n]j ; jh [] h[n]j i (33) The Kologorov-Sirnov statistic deterines the greatest between two cuulative distributions. This statistic can be expressed in ters of the significance level of an observed value of the statistic, giving the probability for the null hypothesis that both data sets are drawn fro the sae distribution. Despite these advantages, the K-S test has several iportant liitations: It only applies to continuous distributions. It tends to be ore sensitive near the center of the distribution than at the tails. We define this statistic by the cuulative sequences as d KS (H;H) = ax jh [] H[]j (3)