Robust Contour Matching via the Order Preserving Assignment Problem

Transcription

1 Robust Contour Matching via the Order Preserving Assignment Problem Clayton Scott, Student Member, IEEE, and Robert Nowak, Member, IEEE Abstract We consider the problem of matching shapes defined by a single, closed contour. Shape matching is accomplished by solving for corresponding points on the two shapes, and using the correspondences to estimate an aligning transformation. In this framework, point corresondences are usually determined by solving the assignment problem for a cost matrix derived from the two shapes. We propose a modification of the standard assignment problem in which the correspondences are required to preserve the ordering of the points as defined by the shapes contours. This provides a significant gain in the structural integrity of the resulting alignment. Further robustness with respect to shape irregularity is achieved by requiring only a fraction of feature points to be matched. We present an efficient solution to this constrained assignment problem by means of an algorithm rooted in dynamic programming. Our experiments demonstrate that preserving the order of feature points leads to significant improvements in performance relative to the standard assignment problem. EDICS Category: 2-ANAL C. Scott is with the Department of Electrical and Computer Engineering, Rice University, 6100 S. Main, Houston, TX 77005; Vox: ; Fax: ; cscott@rice.edu. R. Nowak is with the Department of Electrical and Computer Engineering, University of Wisconsin, 1415 Engineering Drive, Madison, WI 53706; Vox: ; Fax: ; nowak@engr.wisc.edu. This work was supported by the National Science Foundation, grant nos. CCR and ANI , the Office of Naval Research, grant no. N , the Army Research Office, grant no. DAAD , and the State of Texas ATP, grant no

2 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST (a) (b) (c) (d) Fig. 1. (a) Two shapes in the form of binary images. (b) Representations of those shapes by points sampled from the boundary. (c) An order preserving matching. (d) A non-order preserving matching. Robust Contour Matching via the Order Preserving Assignment Problem I. INTRODUCTION Image registration is the problem of finding a spatial transformation mapping an object in one image to a similar object in another image. When this problem is solved using shape information from the two objects, it is referred to as shape matching. When the shape information is represented by points extracted from edges, corners, or other descriptive features, it is referred to as feature-based, or point-based shape matching. For example, Figure 1 (a) and (b) depicts two shapes and their representation by points sampled from their boundaries. One approach to point-based shape matching proceeds as follows: First, extract a set of feature points from each object by running an edge detector over each image and sampling from the edges. Second, determine pairs of corresponding points in the two feature sets. Third, use the correspondence information to find an optimal aligning transformation. The second and third steps can be repeated iteratively. This approach to point-based shape matching has been shown to work well [1] [3]. The first stage of the process described above is straightforward: one selects an edge detector, uses it to extract edges from the images, and samples points on those edges in a uniform way. The third stage is typically solved by finding the least-squares optimal aligning transformation from some class of

3 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST transformations (e.g., rigid, affine, thin-plate splines) that reflects the variability of the object. The second stage, determining point correspondences, is more involved. In one approach, each point is assigned a local shape descriptor that characterizes the shape near that point. A cost matrix is built by measuring the similarity between shape descriptors of pairs of points (one on each point set). Finally, the point correspondences are given by the solution to the assignment problem (AP) for the constructed cost matrix. Typically, only a fraction of points are assigned counterparts on the other shape, to allow for outliers and irregularities. In this paper we consider the special case where the shape of the object of interest is defined by a single, closed contour. Then the points in the feature sets can be arranged in a cyclic order. When determining the optimal assignment, it intuitively makes sense to constrain the assignment so that the ordering of the points (inherited from the contour) is preserved. Assignments that violate this ordering are physically implausible. Figure 1 (c) and (d) show point correspondences that do and do not preserve the ordering of the boundary points, respectively. Our contribution is the formulation and solution of a new criterion that enforces the cyclic constraint while achieving robustness to outliers. We formulate the cyclic order preserving assignment problem (COPAP), and show how the solution to COPAP is given in terms of a simpler problem, which we call the linear order preserving assignment problem (LOPAP). LOPAP is a dynamic program, and can be solved recursively in cubic time. This leads to a quartic time solution to COPAP. The algorithm of LOPAP can be viewed as the Viterbi algorithm on a certain trellis contructed from the cost matrix. Fast approximate algorithms based on approximations of the Viterbi algorithm are possible. Experiments show improved perfomance of COPAP versus the standard assignment problem. In our experiments, we use the shape context as our shape descriptor [1]. A. Related Work The shape matching problem has been considered in various contexts, and a wide range of methodologies have been applied to it. The order-preserving assignment problem developed in this paper is most closely related to three areas of past work. The first area follows our general setup, but does not assume the feature points are ordered in any way; see [1], [2] and the references therein. These approaches are based on assigning to each feature point a local shape descriptor that captures the spatial relationships with other feature points. The method of [2] uses labeled distance sets to capture information about the spatial interrelations of features. The notion of a shape context, based on polar histograms of feature points, is put forward in [1] as a robust

4 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST method for capturing spatial relationships. The labeled distance sets and shape contexts lead naturally to a formulation of the feature correspondence task as an assignment (or bipartite graph matching) problem. The order-preserving assignment problem that we propose in this paper can be interpreted as a constrained version of the assignment problem. The second area concerns what can be described as contour matching or curve matching methods [4] [10]. These methods are based on extracting features or shape descriptors from contours and then matching the features to gauge the degree of similiarity between the two shapes. The reference approaches differ by the shape descriptor used to characterize each curve and by the matching algorithm used to determine point correspondences. Typically a dynamic programming algorithm is used to preserve the ordering of features along the contour. Where our work departs from these is in our formulation and solution of the point correspondence problem. While other approaches seek correspondences mimimizing an edit distance or some other dissimilarity measures computed using all the data, our dissimilarity achieves robustness to outliers by simultaneously selecting a subset of points to match and finding the optimal matching within that subset. A third related area includes structural graph matching methods, e.g., [11], [12] and the references therein. A key aspect of these methods is that an effort is made to preserve the spatial interrelationships of feature points in the matching process. This can be accomplished using graphical models in conjunction with expectation-maximization or belief propagation algorithms. Our approach is based on a linear graphical model (along the contour) and spatial structure (cyclic ordering) is enforced. Thus, the problem and algorithm in this paper is a special case of structural graph matching which happens to admit an optimal solution in polynomial-time. The general cases [11], [12] require iterative algorithms and convergence to an optimal solution is not guaranteed. II. POINT-BASED SHAPE MATCHING In this section we introduce notation and formulate the problem. Let denote points on the first object, and let denote points on the second object. Observe that the point sets may have different sizes. A shape descriptor may be thought of as a mapping from the image domain to a high-dimensional vector that encodes shape information of points in the feature set. The cost matrix is defined by where is a function that measures the similarity between shape descriptors.

5 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST The cost matrix is used to determine the optimal way to match points in the two feature sets. The maximal size of such a matching is. However it is frequently desireable to find a smaller matching of size. This is essential if one wishes to account for outliers, or points on one shape that have no natural counterpart on the other shape. Let denote the set of all subsets of with cardinality. For, and, denote by the set of all one-to-one, onto mappings. Then the optimal (non-order preserving) matching between the two point sets is defined to be the solution to (1) When, this problem is known as the assignment problem, or the weighted bipartite matching problem. It is well studied, and several algorithms exist that produce exact solutions, most (including the fastest) having cubic running time [13]. When! or ", one extends the cost matrix to an # " # matrix by adding zeros, where # $ %. This effectively adds dummy points to each point set. The extended cost matrix is then input to the assignment problem, and the resulting assignment, when restricted to correspondences between non-dummy points, provides a solution to the problem in (1). III. THE CIRCULAR ORDER PRESERVING ASSIGNMENT PROBLEM In this paper we consider a slight modification of the scenario just described. In particular, we assume that the points in each point set are arranged along a single, closed contour. Further assume that each point set is labelled, starting at 1, in a clockwise manner. (We could also label the points counterclockwise, as long as both point sets have the same orientation.) The location of the first point does not matter. With this a priori information about the problem, it makes sense to constrain the matching to preserve the ordering of the contour. We thus define the cyclic order preserving assignment problem (COPAP) as follows: Let and. Denote by & the subset of consisting of those matchings that preserve the cyclic ordering of the point sets. In other words, if ' ', with ' ( ( ( ', and &, then for some ) * % '+, we have ( ( ( ' ' ( ( ( '+. Then COPAP is defined to be - (2) The search space for COPAP is smaller than for AP, although that does not necessarily imply that it can be solved more quickly. Indeed, because of the special structure of the assignment problem, it

6 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST can be formulated as a linear program, and linear programming techniques are used to solve it. We are unaware of any such reduction that may be applied to COPAP. Note that the search space for COPAP is still exponentially large. There are elements of & for any, and so there are possible configurations to search. For a typical problem, say ** and *, this number is on the order of *. Thus a brute force exhaustive search in infeasible. In the following sections we explain an intelligent way to conduct the search. IV. COPAP REDUCES TO LOPAP COPAP can be solved in terms of LOPAP, the linear order preserving assignment problem. LOPAP is defined as follows: Let and, and let denote the element of & that maps the smallest element of to the smallest element of, the second smallest element of to the second smallest element of, and so on. We say that is linear order preserving. Then LOPAP is the problem Note that depends on and, although this is not reflected in the notation. We now argue that with appropriate re-indexing of the feature points on the second contour, the solutions to COPAP and LOPAP coincide. Suppose, and (3) & comprise the solution to COPAP, with ' '. Then there exists ) * % such that '+, ( ( ( ' ' ( ( ( '+. Now let us introduce re-indexed feature points +, where the indices are taken modulo. Since the cyclic order of the contour is preserved under this re-indexing, the solution to COPAP on and is, and, where % ) (modulo ), and ' ' % ) (modulo ). By our choice of ), is linear order preserving. Since the search space for LOPAP is a subset of the search space for COPAP, we conclude that, and comprise the solution to LOPAP. In summary, for some cyclic shift of the feature points on the second contour, COPAP and LOPAP have the same solution. Therefore, to solve COPAP, it suffices to solve LOPAP for all such shifts (of which there are ), and take the solution with smallest cost. As we will see in the next section, LOPAP may be solved in cubic time, leading to a quartic time solution to COPAP.

7 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST V. SOLUTION TO LOPAP We provide an alternate characterization of LOPAP that aids in visualizing the algorithm. Consider an " grid as depicted in Figure 2 (a). A black dot at square ' denotes that point ' on the first contour is matched to point on the second contour. We refer to a path of length ) as a collection of ) black dots that progress down and to the right, as in the figure. Paths are denoted by. Let each square on the grid be associated with the cost at the corresponding position in the cost matrix. The cost of a path, denoted, is the sum of the over points in the path. Observe that paths are in one-to-one correspondence with linear order preserving matchings. Thus, LOPAP entails finding the path of length having minimum cost. The last point in a path, i.e., the point that is furthest down and to the right, is called the terminal point of the path. Let + denote the path with the smallest cost among all paths of length ) that terminate at point '. (Note that + is undefined if ) ' or ).) Then the solution of LOPAP is, where ' arg min Thus, we can solve LOPAP if we can determine for each possible ' and. Fortunately, + may be computed recursively. The base of the recursion is simple: for ' and, is the point '. For ), + is determined as follows: Let ' ' denote the predecessor to in the path +. Observe that ' arg min + Then + is simply + with the point ' appended. In the above formulation, determining ' requires a search over all points above and to the left of ', which in general requires considering different options. However, it is possible to limit the scope of the search to 3 options, thus giving rise to a much faster algorithm. The reader not interested in these details may skip the next paragraph. We know that + terminates at point ' on the grid, by definition. There are three possibilities for ' ' ' : (a) % and % ' ; (b) % ' and % ' ; and (c) ' % and %. It is not important that cases (b) and (c) are not mutually exclusive. The value of + is determined by these three cases as follows: (a) + + ; (b) + + % ; and (c) + + %. We can determine which case holds by finding the smallest of these three expessions (which are based on previously computed quantities). If (a) holds, then ' ' % %. If (b) holds, then ' is the predecessor of ' % in +. If (c) holds, then '

8 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST (a) (b) Fig. 2. Grid representation of LOPAP with,, and. (a) The four dots represent one possible linear order preserving matching. (b) Notice the bold boxes. The -th box contains all squares that could possibly be the -th point in a length path. Each box is. The shaded squares could not possibly be part of a length path. is the predecessor of ' % in +. Note that if both (b) and (c) hold, the same value of ' is returned in both cases. A. Computational Complexity In this section we count the number of operations required by the above algorithm. The complexity is dominated by the computation of + for ', and ). At first glance, there appear to be such quanities to compute. However, + is defined only for certain combinations of ' and ). For example, the last point in any path of length can be no fewer than % rows from the top of the grid, and no fewer than % columns from the left side of the grid. Thus, we need only compute for ' and and ). In general, we need to compute + for ) ' % $ ) % $ ). These regions are represented by the boxes in Figure 2 (b). This gives a reduction from to % $ % $ in the number of updates of +. Each such update takes a fixed amount of processing, so the overall runtime of LOPAP is % $ % $. This reduction does not, in general, affect the order of the computational complexity, although it does significantly decrease the leading constant. Assuming and are the same order of magnitude, and is a fixed percentage of or, say for some *, then the complexity is. From Section IV, this implies an algorithm for COPAP. In practice, with ** and, COPAP is solved in about second on a 425 MHz Pentium II processor. This compares with about 10 ms to solve the standard assignment problem, using the algorithm of Jonker and Volgenant [14]. Various approximate algorithms that significantly speed up

9 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST (a) (b) Fig. 3. Trellis representation of LOPAP with and : (a) Enumeration of grid. (b) Trellis showing all possible length 3 paths. LOPAP seeks the path through this trellis with smallest cost. COPAP are discussed in Section VI below. B. Trellis Formulation Our algorithm for LOPAP may be viewed as the Viterbi algorithm on a certain trellis. Figure 3 shows a simple example. Each node in the trellis corresponds to a square on the grid, and hence to a particular pair of points on the two shapes. In particular, the nodes in the )-th column of the trellis correspond precisely with those squares ' in the grid representation such that ) ' % $ ) and ) % $ ). These are the same points described by the bold boxes in Figure 2 (b), although the dimensions of the problem are different there. Suppose ' and ' are the squares corresponding to nodes in columns ) % and ) of the trellis. Then there is a link between those two nodes if and only iff ' ' and. A path through the trellis thus corresponds to a path on the grid, and to a linear order preserving assignment. With this setup, LOPAP may be viewed as finding the shortest (or least expensive) path through the trellis, where the cost of the path is the cost of the corresponding assignment. The Viterbi algorithm is a well known dynamic programming procedure for determining shortest paths through trellises [15]. A brief inspection reveals that our algorithm coincides with the Viterbi algorithm on this trellis. C. Adaptive Matching Size It is possible to let the data determine the size of the matching. One selects a lower bound on the matching size and the most appropriate matching size is achieved as follows. For each, solve for the length solution to COPAP. Because of shared computations, this procedure requires about as much time as one call to COPAP. For each matching, points not part of the matching are assigned to dummy points with assignment cost (a parameter set by the user). may be regarded as a threshold parameter for outlier detection [1]. The total cost of each matching is

10 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST then computed as the cost minimizing COPAP plus times the number of unmatched points. Finally, the matching size minimizing this quantity is selected. VI. APPROXIMATE ALGORITHMS Perhaps the most obvious approximation to COPAP is to not perform LOPAP at every possible shift of the points on the second contour, but at a coarser spacing. In other words, cyclicly shift the second contour at multiples of some, and perform LOPAP at each such shift. Then, take the optimal shift from the coarse search, and conduct a more refined search in that vicinity. We have implemented this multiresolution approach and found that it works quite well in practice. Other approximation may arise from approximations of LOPAP. For example, there are several wellknown approximations to the Viterbi algorithm for decoding used in digital communication or speech processing. There is, however, a significant difference between those problems and our own: In decoding applications, the height of the trellis is much smaller than the length of the trellis. For LOPAP, the opposite is true. There are in general nodes per column of the trellis, but only columns. For this reason, some of the approximations to the Viterbi algorithm require modification, or must be scrapped altogether and new approximations devised. This is a topic of current research. VII. EXPERIMENTS We present two experiments based on the collection of 25 shapes shown in Fig. 4, obtained from the Brown computer vision group web site [16]. Each row of the figure is considered to be a separate class. In both experiments, each shape in the database is matched to every other shape, and a measure of similarity is assigned. For each shape, the top three most similar shapes are identified. The quality of shape matching is evaluated by counting, for each test shape, the number of first, second, and third ranked matches that belong to the same class. We uniformly sample 100 points from the contours of each shape, and enforce matchings of size. For each feature point, we compute its shape context as a local shape descriptor. The cost matrix is formed by computing the chi-squared distance between pairs of shape contexts on the two contours, as in [1]. The similarity between shapes is defined to be the overall cost of the optimal assignment. This experiment was performed in [1] to demonstrate rotationally invariant shape matching with shape contexts. The number of correct first, second, and third ranked matches achieved was 25/25, 24/25, and 22/25 (we have verified this result). We repeat this experiment, using COPAP instead to compute the

11 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST Fig. 4. Shapes from the Brown database. optimal assignment. Our results are 25/25, 24/25, 23/25. The same experiment was also presented in [5], [16], where the results were 23/25, 21/25, 20/24 and 25/25, 21/25, 19/25, respectively. This first experiment indicates that COPAP offers a modest improvement over matching with the standard assignment problem. In the second experiment, however, the benefits of preserving the cyclic ordering are considerably greater. The second experiment repeats the first, but uses a different measure of similarity that better captures the geometric quality of the correspondences. In particular, we solve for the optimal assignment between two shapes, using either AP or COPAP, and estimate a geometric transformation based on the matching. Our measure of similarity is the average squared error between transformed points on the first shape and their corresponding points on the second. The transformation we choose is the least squares optimal rigid body transformation (variable translation and rotation). Using COPAP to determine point correspondences, the numbers are 25/25, 25/25, 21/25. Using AP, the numbers are 21/25, 19/25, 14/25. Nearly all of the errors when using AP arise from classes that require rotational invariance (e.g., tools and airplanes). See for example the matching produced by COPAP and AP in Figure 5. We offer two possible explainations for this gap in performance. First, rotation invariant shape matching with shape contexts requires the estimation of tangents to the contours, which can be difficult for contours with corners. Second, several of the shapes (e.g., tools and airplanes) possess symmetry, or near-symmetry. In other words, some parts of a shape (e.g., the nose of the airplane) look like other parts (e.g., the wing), as far as the shape context (or other shape descriptor) is concerned. This leads to the behavior in Figure 5 (b), were AP assigns points on the nose of one airplane to points on the nose

12 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST (a) (b) Fig. 5. Aligning two airplanes: (a) Matching and registration produced by COPAP. (b) Matching and registration produced by the standard assignment problem. Unlike COPAP, neighboring points are not required to match neighboring points. and wing of the other. In contrast, such a matching is impossible with COPAP. Neigboring points are forced to match neighboring points, thus helping to overcome problems arising from estimating tangents (in the case of shape contexts) and near-symmetries. As a final note, we remark that all of the third rank errors involved the pattern of the F-14 Tomcat (the rightmost plane in Figure 4). As illustrated in Figure 5 (a), this is not due to a poor registration, but simply to the width of the aircraft. With a more flexible transformation model, we might expect to avoid this problem. VIII. SUMMARY AND CONCLUSIONS We introduce a method for determining point correspondences for points sampled from a closed contour. Correspondences are derived from the solution to a constrained assignment problem that enforces the correspondences to preserve the ordering of the contour. This approach eliminates physically implausible matchings and improves the quality of transformation estimates based on those correspondences. The exact solution to the constrained assignment problem is provided by a quartic time dynamic programming. The algorithm relies on computing the shortest path through a certain trellis, and as such can be seen as an application of the Viterbi algorithm. Multiresolution search strageties offer a considerable speed-up. Our experiments reveal the effectiveness of our approach. Using the shape context as our shape descriptor, we demonstrate improved performance versus non-order preserving matchings, especially when the goal is to find an aligning transformation between two shapes, or when shapes possess symmetries. Although in this paper we have focused on two-dimensional shape analysis, both COPAP and LOPAP may find application in other areas such as text recognition or DNA string matching. Finally, MATLAB code (with fast C-MEX subroutines for COPAP) is available at cscott/opap.html.

13 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST REFERENCES [1] S. Belongie, J. Malik, and J. Puzicha, Shape matching and object recognition using shape contexts, IEEE Trans. Patt. Anal. Mach. Intell., vol. 24, no. 4, pp , [2] C. Grigorescu and N. Petkov, Distance sets for shape filters and shape recognition, Institute of Mathematics and Computing Science, University of Groningen, Tech. Rep. IWI , [3] H. Chui, Non-rigid point matching: Algorithms, extensions and applications, Ph.D. dissertation, Yale University, May [4] L. J. Latecki and R. Lakämper, Contour-based shape similarity, in Proc. of Int. Conf. on Visual Information Systems, vol. LNCS 1614, Amsterdam, June, 1999, pp [5] Y. Gdalyahu and D. Weinshall, Flexible syntactic matching of curves and its application to automatic hierarchical classification of silhouettes, IEEE Trans. Patt. Anal. Mach. Intell., vol. 21, pp , [6] F. Mokhtarian, S. Abbasi, and J. Kittler, Efficient and robust retrieval by shape content through curvature scale space, in Image Databases and Multi-media search, A. W. M. Smeulders and R. Jain, Eds. World Scientific, [7] J. Gregor and M. Thomason, Dynamic programming alignment of sequences representing cyclic patterns, IEEE Trans. Patt. Anal. Mach. Intell., vol. 15, no. 2, pp , [8] D. Tell and S. Carlsson, Combining appearance and topology for wide baseline matchine, in Computer Vision - ECCV 2002 : 7th European Conference on Computer Vision, A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, Eds. Heidelberg: Springer-Verlag, [9] T. Sebastian, P. Klein, and B. Kimia, On aligning curves, IEEE Trans. Patt. Anal. Mach. Intell., vol. 25, no. 1, pp , [10] E. Milios and E. Petrakis, Shape retrieval based on dynamic programming, IEEE Trans. Image Processing, vol. 9, no. 1, pp , [11] B. Luo and E. R. Hancock, Structural graph matching using the EM algorithm and singular value decomposition, IEEE Trans. Patt. Anal. Mach. Intell., vol. 23, pp , [12] J. M. Coughlan and S. J. Ferreira, Finding deformable shapes using loopy belief propagation, in Proc. European Conference on Computer Vision, LNCS Springer-Verlag, 2002, pp [13] M. Dell Amico and P. Toth, Algorithms and codes for dense assignment problems: The state of the art, Discrete Applied Mathematics, vol. 100, pp , [14] R. Jonker and A. Volgenant, A shortest augmenting path algorithm for dense and sparse linear assignment problems, Computing, vol. 38, pp , [15] D. Forney, The Viterbi algorithm, Proc. IEEE, vol. 61, no. 3, pp , [16] D. Sharvit, J. Chan, H. Tek, and B. Kimia, Symmetry-based indexing of image databases, J. Visual Communication and Image Representation, vol. 9, no. 4, pp , 1998.

14 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, AUGUST List of figures and captions. Figure 1 (a) Two shapes in the form of binary images. (b) Representations of those shapes by points sampled from the boundary. (c) An order preserving matching. (d) A non-order preserving matching. Figure 2 Grid representation of LOPAP with,, and. (a) The four dots represent one possible linear order preserving matching. (b) Notice the bold boxes. The )-th box contains all squares that could possibly be the )-th point in a length path. Each box is % $ " % $ ". The shaded squares could not possibly be part of a length path. Figure 3 Trellis representation of LOPAP with and : (a) Enumeration of grid. (b) Trellis showing all possible length 3 paths. LOPAP seeks the path through this trellis with smallest cost. Figure 4 Shapes from the Brown database Figure 5 Aligning two airplanes: (a) Matching and registration produced by COPAP. (b) Matching and registration produced by the standard assignment problem. Unlike COPAP, neighboring points are not required to match neighboring points.