Learning Two-Dimensional Shapes using Wavelet Local Extrema

Size: px

Start display at page:

Download "Learning Two-Dimensional Shapes using Wavelet Local Extrema"

Theodore Maximillian Elliott
5 years ago
Views:

1 Learning Two-Dimensional Shapes using Wavelet Local Extrema tyuichi NAKAMURA t Institute of Information Sciences and Electronics University of Tsukuba Tsukuba City, 305, JAPAN STsu t omu Y 0 SHID A $ Communications Policy Bureau Ministry of Posts and Telecommunications Kasumigaseki, Chiyoda-ku, To kyo, , JAPAN Abstract The baszc zdea of the method for shape learnang as presented. Each shape as haerarchzcally characterazed wath sangularataes (local extrema) of wavelet transform of ats tangentaal oraentataon functaon. By takang the common structure of the sangulamtaes of two OT more shapes, the common characterzstzcs of the anput shapes are obtaaned. Through thas operatzon, not only the rough outlane of a shape but also several amportant characterastzcs of shapes are acquared. 1 Introduction Generally, a shape has a great amount of information. Often, we have to handle not only rough outline of a shape, but also detailed corners or 1D texture or irregular structure which have quite important information of a shape. Learning shapes, therefore, is a quite difficult problem. One of the difficulties arises from representation methods of a shape. Many of the early methods which analyze shapes axe based on approximation using polygons or other analytic curves. Because these methods destroy the detailed structures of an input shape, they can not be powerful tools for shape acquisition. On the contrary, there are methods based on multiscaling (especially, scale-space filtering method) in which behaviors of a signal over a wide range of spatial scale are analyzed. Most of all, scale-space filtering method has great advantages on the point that wide range of changes of curvature, that correspond to from rough outline of a shape to small projections, can be represented (e.g.[ab86, MM86, US90]). This method, however, has disadvantages on the completeness of the obtained data. It is proved that the fingerprint of zero-crossing points does not uniquely characterize the original function. In other words, while the convexity or concavity of frag- ments between inflection points is held, the quantity (or a detailed shape) of fragments is not preserved. Although some researches concentrated on the reconstruction problem [HM89, WenSO], enough results have not been achieved. For this purpose, we propose a new framework to deal with shape learning problem. Previously, we proposed the method for shape description by wavelet local extrema (that is local extrema extracted from wavelet transform of an input shape)[nsn93]. In this method, the detailed structure of a shape is fully characterized by wavelet local extrema. Several quite important characteristics of a shape can be quantized by the behavior of local extrema. This implies that these characteristics of shapes can be learned by using these extrema. Moreover, it is proved by Mallat [MZ92] that a close approximation of the original signal can be reconstructed from wavelet local extrema using alternate projections. This enables us to visualize (reconstruct) the acquired structures of shapes, which can be models or templates. In the followings, we will present the relationships between characteristics of shapes and wavelet local extrema, the basic ideaof this method, and the experiments of shape acquisition. 2 Wavelet Local Extrema and Characteristics of Shapes In our method, local extrema are extracted from wavelet transform. This is based on Mallat s signal characterization method, in which very close approximation of the original signal can be reconstructed from the local extrema. 2.1 Wavelet Transform and Wavelet Local Extrema Let baszc wavelet be $(t). Wavelet Function can be obtained by dilating the basic wavelet by a scaling factor S. The wavelet transform of f(t) is computed by convolving the signal with a dilated wavelet. Wdt) = f (t) * $s(t) (2) /94 $ IEEE 48

2 H H d f - WawI*1 Tmulorm... wsvdm Tnrulom...* Prolrtbn Op.mtor ~ 2 ) Figure 1: Wavelet Transform and Signal Reconstruction from Wavelet Local Extrema (a) Shape Contour - I I I I The wavelet transform that we use can be organized similar to quadratic mirror filter [EG77]. First, let us define a smoothing function 4(t) which satisfies the following formula. This formula means that each wavelet function is an independent component obtained by subtracting one smoothing function from the successive smoothing function. S2nf(~) is a smoothed signal by using &p(t)(= &(&)). It can be denoted as follows: %"f(w) = mi244 (4) We assume that the description at (j + 1)th level can be extracted from the description of jth level. For this purpose, we define low-pass filter fi(w). &+lf(w) = B(2jW)S2J(W) (5) Wavelet transform is considered as the mutually <independent component of the above description. To get wavelet transform we introduce high-pass filter G which satisfies = G(w)~(w). kv2r+lf(w) = G(2ju)S2J(w) (6) We can get wavelet transform by calculating W23f(t) changing j from 1 to n ( ma scale) according to the above formulae. While and G(w) are under constrained, the following set of filters satisfy the above conditions for any integer n (In the experiments, we set n = 0). W H(w) = ei"/2(cos( -))2n+l 2 G(w> = 4ieiw/2 sin(z) 2 (7) Fig. 1 shows a brief overview of our wavelet transform. The local extrema of wavelet transform can be extracted from each wavelet transform by simple operations (differentiation and extraction of zero-cross points). Mallat developed the method to reconstruct the close approximation of the original signal from these extrema. The method uses alternative convex projections between h(t) (a function which has the same local extremum values as those of original f(t)) and g(t) (a function ob- (b) 4(s) (c) Wavelet Local Extrema Figure 2: Example of Wavelet Local Extrema tained by inverse wavelet transform of a function which has finite energy). His experiments showed that iterations from 20 to 50 times are enough to get a very close approximation of the original signal. The details are skipped for lack of space (see [MZ92] for details). This method requires all wavelet local extrema and the smoothed function S2nf(~) at the largest scale. It can be shown, however, that if 2" is large enough, we can regard Spf(t) as a direct current component (that is, mean value of a signal). This implies that we only have to record one value or one extremum for Sznf(~). Thus we can reconstruct the original signal by using only wavelet local extrema on the condition that the number of levels is large enough. 2.2 Shape characteristics and Wavelet Local Extrema Generally, the contour of a two-dimensional shape can be expressed as one dimensional signal. For practical reasons, we expressed a shape by the tangential orientation 8(s) of each contour point, where s is the arc length from the starting point on the contour. This signal is transformed, then local extrema are extracted. These extrema hold complete information of the input shape. An example is shown in Fig. 2. The contour of China is shown in (a), and its tangential function (8(s)) in (b). The tangential function is wavelet transformed, then, local extrema are extracted. In Fig. 2(c), an example of a more simple shape is given, in which we can see that an extremum corresponds to a corner at each scale; positive extrema correspond to convex corners, and negative extrema to concave corners. The behaviors of extrema show the quite important characteristics of a curve. The detailed relationships between the characteristics of a shape and extrema are as follows: Extremum Position: Position of a corner at each scale. In Fig. 2(c), corners (a--f) generate extrema 49

3 CI position angle sharpness compound structure Figure 3: The basic idea of corner interpolation in the graph below. Extremum Value: Changing angle of a corner '. In Fig. 2(c), a corner with an acute angle (b) makes large extremum value, while a corner with obtuse angle (a) makes small extremum value. Value Changes across scales: Shape of a corner. If the value increases as the scale decreases, the discontinuity of a corner is large. In Fig. 2(c), a sharp corner (f) makes a series of extrema that the value increases as the scale decreases, while a dull corner (d) makes those that value decreases. This phenomenon can be explained by the relationship between Lipschitz exponents and the decay of wavelet transform [MH92]. Position Shift across scales: A combination of corners. If the position shifts as scale changes, this corner should be considered as a portion of a compound corner. Combination of adjacent extrema: A combination of extrema expresses the compoud structure of a corner. 2.3 Learning Shape Characteristics Using these relationships, we can consider several pseudo axes of corner characteristics which are position, angle, and sharpness (singularity or discontinuity). Although it is still difficult to directly compose a corner which has desired characteristics, interpolation and extrapolation of their characteristics are usually easy to acquire. The basic idea is shown in Fig. 3. For example, when the positions of the extrema are interpolated, the positions of the corners are interpolated. Although there is no proof that a real shape exits which have extrema with these positions and values, a similar shape (which has extrema with close positions and close values) is almost always obtained. The reason is the stability of Mallat's reconstruction method which uses convex projection. The flow of shape learning process in our method is as follows: 'This is only a rough relationship. Strictly speaking, the value of envelope should be considered at the same time. Wavelet transform and extrema extraction Finding correspondences among extrema (a) hierarchical organization within a shape (b) hierarchical matching between shapes Generation of new extrema and reconstruction to a shape. In the following sections, 2 and 3 are described. 3 Finding Correspondences by Shape Matching To find correspondences of corners across shapes, we need shape matching. First, relationships among extrema within a shape are searched. The operation is quite similar to the process of getting fingerprints for the scale-space expression. Then, matching between shapes is performed. Since we have already shown the method for this operation in [NSN93], a brief overview is given in the followings. Note that we do not use positive minima and negative maxima for simplicity because they are not necessary for the shape reconstruction (However, they are important information as connecting points between convex corners or between concave corners). 3.1 Getting Hierarchy Dynamic Programming is used to get the correspondences between extrema across scales. Although the basic idea is similar to [US90], the details are different because the nodes used in this research are wavelet local extrema, First, let us denote two sets of nodes (hereafter we call data structure for an extremum node) at two consecutive scales as follows: (9) where pi" is a i-th node (extrema) at k-th level, and Pk is a set of extrema at k-th level. Each of them includes all nodes sorted according to their position at their scale. This matching can be considered as an operation to find mapping function J which minimizes the following formula: where, s(i, J(i)) is a score of matching between two nodes. For this matching, we placed a few constraints on the correspondences across scales. The penalty values (the costs of paths) of DP matching is determined according to the following conditions: 1. pf" and p;(i) have the same sign. 50

4 (a) Chinal (c) China2 (b) Hierarchy within Chinal (d) Hierarchy within China2 Nodes express the Wavelet Extrema for each shape. The extrema at the same scale are horizontally aligned, while those at different scales are vertically aligned. An arc expresses the relationship between a parent and child. Figure 4: Example of two shapes 2. The positions of two nodes are close enough to each other. 3. The extremum value and the of two nodes are close to each other. In this step, matching is recursively performed from the largest scale (P" and P-') to the smallest scale (P2 and P'). An example is shown in Fig. 4, in which extrema from two shapes are hierarchically organized. 3.2 Shape Matching by DP method We also applied DP matching to this problem. The obtained hierarchy of wavelet local extrema within a shape is used for this purpose. The type, value, and position of extrema can be considered as conditions for matching. As mentioned in the previous section, this process is also performed from coarse to fine resolution. Let us denote the two sets of nodes at k-th scale as follows: P1h ={pl;li=o,l,...,n'} (11) Pa'" = (p2;jj = 0,1,..., N ~ ) (12) The objective of shape matching is to find J which minimizes the following formula: N, -1 where s(i,j) is the penalty value of the correspondence between pl: and p2;. The penalty values of this matching are determined according to the following conditions: scale 27 scale 25 scale 23 scale 2' Figure 5: Matching Result 1. Maxima must match maxima, and minima must match minima. 2. The extremum values should be close to each other. 3. The distances from the previous extremum (or previously matched extremum) should be close to each other. 4. The match of parents nodes at larger scale are used as constraints. Assume that plt have a hierarchical correspondence to plr+', and p2fc have a hierarchical correspondence to p2&+'. If p2&+' does not match plq'l, plt has less possibility of correspondence to p2;. The result of matching at each scale is shown in Fig. 5. Extrema are placed on the contour, and corresponding extrema are related by arcs (Each shape is reconstructed by using wavelet transform at its scale). 4 Learning Shape 4.1 Learning Corner Characteristics If the correspondences of corners are correctly determined, we can get interpolation of a corner by simply interpolating extrema values along scales. The operation can be simply denoted as follows: U, = k.?i1 + (1- k) * U2 (14) where?i is either the value or the position of a extremum. For example, zli means the extremum value of shapel. In this formula, 0 L. k L. 1 means interpolation, otherwise extrapolation. A simple example is shown in Fig. 6. In this example, each of two original shapes (originall and original2) has a concave corner with different characteristics near the center of each shape. The shape in (d) is obtained when k = 0.5. By averaging extrema which have correspondences, we can get quite good approximation for both shapes. For (c) and (e), we set k = 1.5 and k = -0.5 respectively. The obtained shapes show that their characteristics are well interpolated or extrapolated. In our experiments, interporation works fairly well, while extrapolation sometimes gives quite distorted results2. 'There is no guarantee that a shape exists which has such characteristics as the extrapolations of those of input shapes! 51

5 (a) Acquired shape (b) Extrapolation (c) Simple average Figure 7: Acquired Shapes (c) extrapolation (d) interpolation (e) extrapolation (over 1) (over 2) Figure 6: Interpolation of Corners If we consider three or more shapes as input, the notions of interpolation and extrapolation get inappropriate. In this case, we can think of averaging and emphasizing differences from the average. ij=- cy=^ vi n 2rne, = k4+(1 -k).va (15) where V means the average of extrema taken from input shapes. 4.2 Learning Whole Shape Given a shape matching result, we have the extrema correspondences for an entire shape. We can consider these corresponding extrema as the common structure of two shapes. Learning the whole shape structure, therefore, is achieved by applying the method in the previous section to all the pair of corresponding extrema. In other words, the interpolation of whole shape can be obtained by interpolating all pairs of corresponding extrema. An example is shown in Fig. 7. The shape reconstructed from all the interpolated extrema is shown. It shows that the rough outline, which is common in two shapes, is correctly obtained. In addition to that, the characteristics of the corners in the original shapes are preserved well in the acquired shape. (However, the upper part of the reconstructed shape is slightly distorted because the shape matching is not completely succeeded for that part). For comparison, the obtained shape by taking simple average of xy-coordinates of two shapes is shown as (c). Although the rough outline is also preserved in this shape, we can see a lot of small projections which belongs to only one of the shapes, and upper part of the shape is distorted. An example of extrapolation is also shown in Fig. 7. We can see the differences between two original shapes are emphasized by the ext,rapolation. An example for three or more shapes is shown in Fig. 8. Using the shape matching results between Fishl and each of other shapes of fish (Fishl vs. Fish2, Fishl vs. Fish3,...), we obtained the average by the formula Fishl Fish2 Fish3 Fish4 Fish5 Acquired Shape Figure 8: Example of Shape Acquisition from five shapes (15). In this example, the characteristics of dorsal fins and pectral fins are acquired quite well, since all of the shapes of fish have simlar structures around them. Ventral or anal fin is distorted because input shapes do not have common structure around it. 5 Summary This paper proposed a new framework for shape learning using wavelet local extrema. In this method, not only the rough outline of a shape but also several characteristics of a shape are preserved. The basic idea of this method is simple enough to be applied to many kinds of shape acquisition. Therefore, it can be a new basis for bidirectional process of shapes; analysis, learning, and generation. References [AB861 [EG77] [H h4 891 [MH92] [MM861 [MZ92] [NSN93] [US90] [Wen 9 01 H. Asada and M. Brady. Curvature Primal Sketch. IEEE PAMI, Vol. 8, No. 1, pp. 2-14, Jan D. Esteban and C. Galand. Application of quadrature mirror filters to split band voice coding schemes. Proc. Int. Conj. ASSP, pp , R. Hummel and R. Moniot. Reconstructions from Zero Crossings in Scale Space. IEEE ASSP, Vol. 37, No. 12, pp , dec S. Mallat and W. Hwang. Singularity Detection and Processing with Wavelets. IEEE Inform. Theory, Vol. 38, No. 2, pp , March F. Mokhtarian and A. Mackworth. Scale-Based Description and Recognition of Planar Curves and Two-Dimensional Shapes. IEEE PAMI, Vol. 8, No. 1, pp , Jan S. Mallat and S. Zhong. Characterization of Signals from Multiscale Edges. IEEE PAMI, Vol. 14, No. 7, pp , jul Y. Nakamura. K. Satoda, and M. Nagao. Shape description and matching using wavelet extrema. Proc. Asian Conference on Computer Vasion, pp , N. Ueda and S. Suzuki. Automatic shape model acquisition using multiscale segment matching. Proc. foth ICPR, Vol. 1, pp , J. Weng. A Theory of Image Matching. Proc. Yrd ICCV, pp ,

A Modified Spline Interpolation Method for Function Reconstruction from Its Zero-Crossings

Scientific Papers, University of Latvia, 2010. Vol. 756 Computer Science and Information Technologies 207 220 P. A Modified Spline Interpolation Method for Function Reconstruction from Its Zero-Crossings