Matching Pursuit Filter Design

Transcription

1 Matching Pursuit Filter Design P Jonathon Phillips * US Army Research Laboratory Ft. Belvoir, VA Abstract A method has been devised of using localized information to detect objects with varying signatures without prior segmentation. The detection is performed by a new class of nonlinear filters called matching pursuit filters, which are trained on multiple examples of the object of interest. Matching pursuit filters are designed through a generalization of the matching pursuit algorithm that allows for the simultaneous decomposition of multiple images. The matching pursuit algorithm decomposes a signal into an adapted linear combination of wavelets. There are two implementations of the matching pursuit filter design algorithm. The first implementation detects objects by correlating an image with a kernel designed by the decomposition of an observation of the object of interest. The second method directly compares the coefjcients of the decomposition of the training set with coefjcients produced by the decomposition of an observed image. The algorithm has been used for detecting features on human faces, identifying faces and searching for man-made objects in infrared imagery. 1 Introduction This paper addresses the problem of using localized information at different scales to detect objects with varying signatures without prior segmentation. Multiple observations are used to find the structure underlying the variable signature. The localized information is represented by wavelets that are chosen by simultaneous decompositions of multiple observations of the object of interest. The decomposition can be tuned to look for the similarities in the observations or the differences. A decomposition tuned to the similarities is used to design a matching pursuit filter that is trained to detect the object of interest without prior *Author's address: US Army Research Lab, AMSRL SS SK Phillips, Burbeck Rd Ste 430, Ft Belvoir, VA jphillip@nvl.army.mil. segmentation. Because multiple observations are used, the filter is sufficiently robust that it is not sensitive to the variations in the signature of the object. The decision to forego the segmentation step is dictated by the applications to which the algorithm is applied. In the two applications of interest, facial feature extraction and detecting man-made objects in infrared imagery, the development of techniques to produce a reliable segmentation is difficult and the resulting algorithms are computationally expensive. The signature variations in these two applications are due to two different causes. In the case of facial features, for example noses or eyes, each nose or eye is slightly different, but has the same overall structure as all noses or eyes. In infrared images the physical object, usually manmade, is the same, but the signature of the object varies greatly over time. Face recognition is an application where the filters are designed to find the differences between the images. All faces have the same underlying structure, but there are enough differences so that a human can distinguish between them. In face identification the filter design algorithm searches for the features that differentiate faces[5]. This paper draws from two strands of previous work: work in wavelets for generating adaptive wavelet representations of signals [l],[6], [4], and work in pattern recognition that uses Gabor jets for identifying objects [3], [7]. To generate a best-adaptive wavelet packet, Coifman and Wickerhauser[l] use an entropy-based criterion. Their entropy criterion globally adapts the wavelets to the signal. Ramchandran and Vetterli[6] find adaptive wavelets using a framework that includes both distortion and rate in the basis selection criteria. Mallat and Zhang[4] use a greedy algorithm to express a signal as a linear combination of wavelets. They use this expansion for time-frequency analysis and to perform chirp detection. For pattern recognition, Lades et a1[3] use Gabor jets placed on a grid as features. For recognition, the Gabor jets from a reference image are compared to the Gabor jets of an observed image. The comparison is through elastic graph matching of the two girds of /94 $04.00 Q 1994 IEEE 57

2 Gabor jets. This method has been demonstrated for human face recognition. Zhou[7] uses Gabor jets to detect objects in infrared imagery. The next section presents notational conventions and a brief review of the Mallat and Zhong's matching pursuit algorithm[4]. Section 3 presents the matching pursuit filter design algorithm, and section 4 presents the results of applying the algorithm to detecting facial features, identifying faces and searching for man-made objects in infrared imagery. 2 Preliminaries In this paper all functions are in either L2(%) or L2(%'). Let f and g be two functions in L2; then their inner product is and the L2 norm off is (fld = J flidx, Let g, E 2) be a family of wavelets parametrized by y E r such that 11 gr [I= 1 and f is a function in L2(%). The family of wavelets 2) is referred to as a dictionary, and the elements of V are referred to as atoms. The dictionary is complete if it spans L2(%). The family need not be orthogonal. The problem is finding a best-adapted countable subset of 2) such that f = c aig,,. Mallat and Zhong[4] use a greedy heuristic to iteratively construct a best-adapted decomposition of a function f. The algorithm works by choosing at each iteration the wavelet in the dictionary V that has maximal projections onto the residue of f. Let T = ~up,~rl(f,g,)l, which is the upper bound for the projection of f onto the atoms of the dictionary V. Since r is infinite, the upper bound may not be achieved. To avoid this problem, yo must be chosen such thak l(.flgyo)l 2 csup,,r I(f,gr)l, where 0 < 1. The best-adapted decomposition is selected by the following greedy strategy. Let Rof = f; then gy, is chosen such that where l(~zf1g,jl L CSUP I(R",s,)l, Ri+lf = Rif - (Rif, g,)g,* for i 2 1. In practice, r is finite, c is set equal to 1, and sup becomes max. 3 Matching Pursuit Filter Design In matching pursuit filter design algorithm, a single observation can be used to design the filter (sect 3.1), or a generalization of the algorithm can be used to designing the filter from multiple observations (sect 3.2). 3.1 Single Template Matching pursuit filters can be designed based on a single observation. Two methods are possible: one method is correlation-based, and the other compares the wavelet coefficients between the example or template and the image to detect the object. In the single template case, the design of the filter is based on one example of the object in an image. Let 2*(x1, Q) be an image that contains an example of the object of interest. Let T be the region of support in 2* of the object of interest. In practice, T is usually a rectangular region, and it is assumed that T contains the origin. In the matching pursuit algorithm, to make clear that the function of interest is 1' restricted to T, let the template t(x1,q) = ~*(Z~,X~)IT. The region of support T of the object determines the atoms that are in the dictionary D(T), which is parametrized by y E r(t). When an image or signal is being analyzed, a complete dictionary is used and the matching pursuit algorithm is iterated until the residue term is sufficiently small. For pattern recognition, an incomplete dictionary is used and the matching pursuit algorithm is executed for a limited number of iterations. The dictionary excludes high-frequency atoms so that the filter is invariant to illumination level and reduces the effects of high frequency noise. Low-frequency atoms are excluded for computational considerations and to insure that atoms used in the decomposition do not encode information outside T. The construction of the matching pursuit filter is divided into three stages. The first stage computes a matching pursuit decomposition of the template t. The second stage constructs the filter h, that will be used in recognizing the object of interest; two ways are presented to construct h,, each of which leads to a different on-line implementation that scans an image for the object. The the third stage is the implementation of the on-line search algorithm. When there is one example of the object, the algorithm that decomposes the template t using the dictionary D(T) is an extension of the one-dimensional case. The extension to two dimensions requires that the maximization occurs over a higher dimensional set of parameters. Let Rot = t; then iterations proceed 58

3 as follows: and ^/a = a% max,~r(t)i(r'tt,gy)i R"'t = Rit - (Ritl g,,)g,,. The value of the projection of Rit onto gr, is denoted by ai = (Ritlg,,). The matching pursuit filter h, is constructed with the first n terms of the above decomposition. The number of projections is chosen so that enough information about the object is captured and other occurrences of the object are recognized, while at the same time regions of the image that do not contain the object are rejected. If n is too large, the filter will encode minor details that do not generalize to all instances of the object, and the filter will not be able to recognize all instances of the object. If n is too small, the filter will not contain sufficient detail of the object to differentiate it from other objects present in the image. The first method of constructing a matching pursuit filter is designed to locate the object of interest using correlation. Using the region T of Z* as a filter, correlating it with a new image Z, and searching for maxima is a basic method of searching for an object. One of the drawbacks of this approach is the computational time associated with computing the correlation function. A filter constructed from the first n atoms of the matching pursuit decomposition greatly reduces the computational expense of computing the correlation function. Because of the method used to extract the first n atoms, the important structural information about t is encoded in the first n terms of the decomposition. The first n terms are used to construct a filter h, as a sum of these atoms weighted by the set {ao,...,an-l}. Thus Because h, is a sum of atoms, the computation of Z correlated with h, reduces to correlating the image Z with the first n atoms and summing the results by the appropriate weight. The second method for designing matching pursuit filters compares the coefficients of the decomposition of the templates and the image, with care taken in defining the decomposition of the image. Since matching pursuit filters are not translation invariant, the location of the object is important. In this method, the coefficients generated at one location are compared with the coefficients for the decomposition of the template. Let 4211, uz) = (RiZ, ST,(. + u1,. + uz)) be the projection of the image onto the ith atom when the algorithm is searching for the object located at (~1,142) and ai = (Rit, g,,). Given the two vectors of coefficients, one can ascertain the presence of the object of interest by computing the weighted L2 or Loo norms or the angle between the two vectors. For computational considerations ai(u1, uz) can be approximated by bi(u1,uz) = (2,g,,(.+ul,.+uz)) and ai by pi = (t, g,,). In practice this approximation works. 3.2 Multiple Templates The previous subsection the design of matching pursuit filters based on one template or example. In many cases, however, one template or example does not capture all the variations necessary to detect the object of interest in all circumstances. If only one example is used, the designed filter is tuned to that specific example and the filter cannot recognize other examples of the object of interest. This section addresses the case of designing filters based on m templates or examples. The constructed filter is designed to work using the projection method. When the filter is designed based on multiple templates, the m observations of the object are assumed to occur in the set of images {Zy,..., Zk} with one observation per image. As before, the origin is contained in T, D(T) is the dictionary of atoms, and r(t) is the corresponding parameter set. Each image produces a template t,(xl, 22) = Z;"(xl, Q)IT. All m templates are used to design the matching filter algorithm. For a single template, detection of an object is made by comparison of the ai's and the a(u1, u2)'s. Let (ao,..., an-l) be referred to as a coefficient vector. For multiple templates, the filter design algorithm generates N coefficient vectors. For object detection, the number of coefficient vectors is small, and the vectors are cluster points. For identification, each person or object is represented by a coefficient vector. The original matching pursuit algorithm generates the g,,'s by finding the argument that maximizes (Rit,g,), where t is a single function or template. Applying the matching pursuit technique to pattern recognition requires that the filter design algorithm not only generate a set of atoms from multiple images, but also sets of coefficients that are used for feature detection or identification. The atoms of the decomposition and the sets of coefficients are selected by a 59

4 pair of functions, the choice of which is based on the purpose of the filter. The atoms of the decomposition are selected by the function C,, and the coefficients are generated by the function C;,. The function C;, determines N sets of coefficients. The number N is determined by the application to be solved. The ith atom and sets of coefficients are calculated by iterations of C, and C;,. Both C, and C;t, are functions of the residue images and the coefficients previously determined by C;,. Let a! be the coefficient from the ith iteration from coefficient vector IC then A! = (a;,..., a!-.l), or the first i coefficients for coefficient vector IC. Let Ai = Uk A!, i 2 0, all coefficients through the ith iteration, and A-1 = 0. Each iteration of the filter design algorithm first generates the ith atom using C, and then calculates the af's with C;,. The ith time-frequency atom is selected by gr, = argmaxcy(ritl,..., Ritm, Ai-I), and the residual images are updated by Risltl = Ritl - (Ritl, gr,)g,, The coefficients for the ith iteration are chosen by = C;,(Ritl,...,Ritm,hi-l; k). There are numerous possibilities for the choice of C,: the mean of the absolute values of {(Ri-'te, gr)}y=l or the minimum of the absolute values of the same set. The function C;, is usually selected to be a clustering algorithm. The goal of choosing C, is to find a function that produces coefficients that are in agreement with the goals of the algorithm and the method used to detect the object of interest. Another aspect of designing a matching pursuit filter is the distance measure used to determine whether two sets of coefficients represent the same object. The method of detection is the same as in the single template case, except that the coefficient vectors from the image are compared with all the coefficient vectors associated with the filter. 4 Results The matching pursuit filter design algorithm has been demonstrated on three applications. The first application detects features in human faces, the second identifies faces and the third locates man-made objects in infrared imagery. These applications were chosen because the signatures of objects in the classes of objects interest vary. Matching pursuit filters were constructed based on two different types of dictionaries. One dictionary consisted of Gabor wavelets. For computational ease, only the odd and even phase Gabor wavelets were used. The second dictionary consisted of separable steerable filters[2]. The steerable fikers used were the second partials of the Gaussian density and their Hilbert transforms. The separable steerable filters were chosen because they are computationally less expensive than nonseparable filters. Both dictionaries were incomplete. Figure 1 shows the results of taking a nose and executing the matching pursuit algorithm using the incomplete dictionaries. Figure la is the original nose; figure lb is the nose reconstructed from the first 75 terms of the steerable filter dictionary. The matching pursuit filter was applied to recognizing facial features, in this case by locating noses on faces. The filter was designed based on four noses; one of the noses was reconstructed from the first 30 terms with the steerable filter dictionary (see Figure 2). The filters were then used to detect noses in images not in the training set. The second applications of the matching pursuit filter algorithm was to face identification. The details of the algorithm are given elsewhere[5]. On a database of 172 images, the algorithm achieved 97% correct identification. In the third application, the algorithm was used to detect man-made objects in infrared imagery. The training set consisted of two images, and the filter was constructed from the first 30 terms with the Gabor dictionary. The filters were able to detect the man-made object in images that were not in the training set. 5 Conclusion This paper presents a new class of filters called matching pursuit filters, which are designed to detect objects with varying signatures without prior segmentation. They are trained to detect objects because their design is based on the simultaneous decomposition of the training examples. This allows the filters to find the structure underlying the variable signatures. The filters were applied to three difficult problems: locating facial features, identifying faces and finding man-made objects in infrared imagery. There are a number of possible avenues for future research. The first is to incorporate prior information about the object of interest into the filter design. This includes using information obtained from models of the objects. 60

5 This would be particularly useful in recognizing manmade objects. Another extension is to modify the algorithm to simultaneously search for more than one object. Acknowledgements The author gratefully thanks his thesis adviser Yehuda Vardi for his time, guidance, support and many helpful conversations. Prof. Vardi s National Science Grants DMS and DMS are acknowledged. The author is a PhD candidate in the Operations Research program at RUTCOR, Rutgers University, and would like to thank their support for a nontraditional operations research thesis. Portions of the research in this paper use the FERET database of la steerable filters, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 13, No 9, [31 M. Lades, J. Vorbriiggen, J. Buhmann, J. Lang, C. von der Malsburg, R, Wurtz, and w. Konen, Distortion invariant object recognition in the dynamic link architecture, IEEE Transactions on Computer, Vol 42, [71 S. Mallat and Z. Zhang, Matching pursuit with time-frequency dictionaries, New York University Technical Report 619, October P. J. Phillips, Matching pursuit filters for face identification, SPIE Vol. 2277, San Diego, CA, K. Ramchandran and M. Vetterli, Best wavelet packet bases in a rate-distortion sense, IEEE Transactions on Image Processing, Vol 2, No 2, Y. T. Zhou, Unsupervised target detection in a single IR image frame, SPIE Vol 1567, San Diego, CA, lb Figure 1: Reconstruction of a nose using the matching pursuit decompostion. la: original image of the nose. lb: reconstruction Using the steerable filter dictionary. Figure 2: Reconstruction of a nose using a matching pursuit filter trained on 4 noses 61