Multi-Orientation Estimation: Selectivity and Localization

Transcription

1 in: M. van Ginkel, P.W. Verbeek, and L.J. van Vliet, Multi-orientation estimation: Selectivity and localization, in: H.E. Bal, H. Corporaal, P.P. Jonker, J.F.M. Tonino (eds.), ASCI 97, Proc. 3rd Annual Conference of the Advanced School for Computing and Imaging (Heijen, NL, June 2-4), ASCI, Delft, 1997, Multi-Orientation Estimation: Selectivity and Localization M. van Ginkel, P.W. Verbeek, L.J. van Vliet Pattern Recognition Group Faculty of Applied Physics, Delft University of Technology, Lorentzweg 1, 2628 CJ Delft, The Netherlands, Keywords: lines, edges, orientation, teture Abstract Filtering of an image with rotated versions of an orientation selective filter yields a set of images which can be stacked to form an orientation space. Orientation space provides a means of analyzing overlapping and touching anisotropic tetures. A set of rotated k th order directional derivatives yields a discrete orientation space, which allows interpolation. Net we apply a deconvolution scheme that results in improved orientation selectivity. This scheme allows decomposition of noisy multi-orientation patterns. 1 Introduction Images are composed of various types of structures, such as lines, edges and tetures. These in turn can be characterized by various properties, the most important being intensity, scale and orientation. In this paper we focus on improved orientation selectivity for multi-orientation estimation. An image can often locally be modeled as a weighted sum of translation invariant patterns or (paintbrush) strokes. Each stroke has a onedimensional intensity profile and a typical orientation: the profile orientation across the stroke, see figure 1. We define directions as angles in the interval (0, 2π), thus making a distinction between an arrow pointing to the left or right. When we refer to orientation (0, π) we make no such distinction. For many applications a reliable estimator for a single locally dominant orientation suffices, Kass & Witkin [4], Haglund [3] and Van Vliet & Verbeek [9]. There are a number of applications for Figure 1: An oriented pattern which an estimate of the local orientation will not be sufficient. Tetures can often be characterized by a number of overlapping patterns with a different orientation. At boundaries between two single orientation regions, an estimator for a locally dominant orientation will give the wrong answer. In such cases a more advanced analysis is required. For another important property, scale, an etensive framework has been created over the last decade, initiated by Witkin [12] and Koenderink [6]. In the scale space paradigm, scale is eplicitly dealt with by embedding an image in a new image with one etra dimension, the scale dimension. This image is generated from the original by iteratively applying an isotropic (e.g. Gaussian) filter. In order to deal with anisotropy an image or scale space can be replaced by a stack of directionally filtered versions of the original. The resulting orientation space or orientation+scale space is periodic along the orientation ais with a period of π. Orientation space and scale space differ in two aspects; the original image is part of scale space, but not of orientation space; scale space has a hierarchical structure, whereas orientation space does not. As shown in Van Vliet and Verbeek [10], such an orientation space can also be used to per- 99

2 form a segmentation of overlapping objects. The main objective of this paper is to establish how to generate an orientation space representation of an image with a high orientation selectivity. Interesting related work on orientation estimation and junction classification has been done by Andersson [1] and Michaelis [7]. 2 Constructing an orientation space Orientation space is created by applying rotated versions of some orientation selective filter to the image. Initially the only constraint on the choice of filter is that it allows proper sampling of orientation space. The filters we use are all two dimensional and operate in the (, y) plane. Yet, most of the ensuing (Fourier) analysis takes place in the orientation dimension and will be onedimensional. 2.1 Directional derivatives We will consider the possibility of using a Gaussian directional derivative of order k as orientation selective filter. In section 2.3 we will use the results of this section to derive a filter with a better orientation selectivity. The Gaussian is only used for regularization of the derivative operators. Therefore the σ, which specifies the width of the Gaussian, is not of interest in what follows. The Gaussian derivative filter of order k in the direction is given by: G (k) (, y, ) = (cos + sin y )k ep( 2 + y 2 2σ 2 ) (1) Net consider an image I(, y) that consists of a sum of strokes f i, with corresponding orientations θ i. Thus I(, y) can locally be described by: I(, y) = i f i ( cos θ i + y sin θ i ) (2) The response of the k th order directional Gaussian derivative to the stroke f i when = θ i, is denoted by f k,ma i. The response of the directional derivative to the image I(, y) can then be written as: I (k) (, y, ) = G (k) (, y, ) (,y) I(, y) = i cos k ( θ i )f k,ma i (, y) (3) Where (,y) denotes convolution in the (, y) plane. This can be rewritten as: I (k) (, y, ) = i cos k () f k,ma i (, y)δ( θ i ) (4) Convolution in the -dimension is denoted by (). The last equation has a simple interpretation. In orientation space each stroke gives an impulse in the -dimension convolved with the following kernel: a (k) () = cos k (5) A property of this kernel is that the orientation selectivity increases with increasing order k. This means that a specific selectivity can be chosen for a particular application. A less desirable property is that not only the orientation selectivity changes with the order k, but the radial frequency sensitivity as well. Therefore we will follow Knutsson [5] and decompose the filter into an angular and a radial part. The Fourier transform of a directional Gaussian derivative using polar coordinates ω and θ is given by: G (k) (ω, θ, ) = (jω) k ep( 1 2 σ2 ω 2 ) cos k ( θ) (6) Although separable, both the angular response, cos k ( θ), and the radial response depend on k. Since we wish to vary only the orientation selectivity of the filter, we will keep the radial response fied at k = 2 by multiplying G (k) by ω k+2. This is a rather arbitrary choice, but will suffice for the purpose of this paper. The actual choice of radial function depends on the application. The Fourier transform of the resulting filter A is as follows: Ã (k) (ω, θ, ) = ω k+2 G(k) (ω, θ, ) = j k ω 2 ep( 1 2 σ2 ω 2 ) cos k ( θ) (7) 2.2 Sampling the orientation space In practice only a finite number of filters can be used. Therefore the orientation space must be sampled. In this section we derive how many filters are needed for a given order k to allow reconstruction. Given a certain filter, Freeman & Adelson [2] and Perona [8] address the problem of creating a set of filters that allows interpolation and minimizes the error between this set and the filter response obtained by rotation of the original filter. They also present the filter constraints required to allow interpolation. The following analysis is essentially identical to the derivations in the references given above and shows that our filter allows interpolation. The Fourier coeffients of cos() are zero, ecept for ω = ±1. Repeated convolution (k 1 times) of 100

3 the Fourier series representation of cos by itself, yields the Fourier coefficients ã (k) (ω ) of a (k) (), which are then given by Pascal s triangle: ã (k) (ω ) = ( ) 1 k k k odd, ω odd k (k + ω ) k even, ω even k(8) 0 elsewhere The coefficients are depicted in figure 2a for order 10. The filter is bandlimited and the coefficients that correspond to frequencies higher than kω are zero. The Nyquist sampling theorem states that we need 2k + 1 samples on the interval (0, 2π) to allow reconstruction, or equivalently 2k + 1 filters. The number of filters needed can be reduced to k + 1, by noting the following symmetry: a (k) ( + π) = ( 1) k a (k) () (9) 2.3 Improving the angular resolution The kernel a (k) () can be well approimated by a Gaussian shape with a variance that decreases as 1 k. Therefore the peak width decreases 1 as k. Since the kernels are periodic the notion of peak width only makes sense on the interval ( π 2, π 2 ). The 1/ k behaviour is disappointing, because one would wish that doubling the number of filters would halve the peak width. We will now show how this can be achieved. Equation 4 shows that each stroke gives an impulse convolved with the kernel a (k) (). To improve the orientation selectivity we can simply deconvolve the resulting signal, because the blurring kernel is eactly known. As equation 8 shows, some Fourier series coefficients are zero, meaning that only a partial deconvolution can be performed. For order k the amplitude of the Fourier coefficients can be flattened (deconvolved) to: 1 k odd, ω odd k b(k) (ω ) = k+1 k even, ω even k 0 elsewhere (10) In figure 2b the coefficients b (k) (ω ) are depicted for k = 10. The resulting response in orientation space is given by: b (k) () = sin((k + 1)) sin (11) In figure 2c the kernels a (k) () and b (k) () are depicted for k = 10. It is clear that the peak width is indeed reduced, but at the cost of adding some side lobes. These lobes will cause problems if the amplitude of the different strokes differs too much. The first zerocrossing of b (k) () lies at π/(k + 1). This shows that the peak width indeed decreases approimately as 1/k. The response along the ais in orientation space to a stroke is equivalent to the angular response of the filter that is used. So instead of performing the deconvolution, it is also possible to use a filter that uses equation 11 as its angular response. The Fourier transform of the filter created by replacing the angular part of equation 7 by b (k) () is given by: B (k) (ω, θ, ) = j k ω 2 ep( 1 2 σ2 ω 2 sin((k + 1)( θ)) ) sin( θ) (12) In figure 3 the A and B filters are shown in the frequency domain as well as in the spatial domain. 2.4 Measuring the rms orientation intensity The filters A and B do not measure the rms orientation intensity, but respond to either locally symmetric or antisymmetric structures. For teture analysis we do not wish to discriminate between the two. The rms intensity is measured by the following procedure; First the filter result is squared, yielding an estimate of the local orientation energy, but with an image containing high frequency signals superimposed on it. By adding the square of its quadrature counterpart [5] the high frequency signals will cancel. Creating a quadrature counterpart for filter B results in a filter that is discontinuous for even k and nondifferentiable for odd k. Therefore it is not possible to use a quadrature filter approach. Instead the image will be smoothed by a Gaussian to remove the high frequency signals. The square root is taken of the smoothed image to yield the final estimate of the rms orientation intensity. Any interpolation must take place before the squaring operation, otherwise the necessary conditions to allow interpolation will no longer hold. All these operations take place in the (, y) planes of orientation space. 3 Single Peak detection In order to perform an analysis in orientation space, it is necessary to detect peaks in the dimension at every (, y) position. We assume that the peaks are non-overlapping and in the eperiments we have ensured this by choosing a sufficiently high order k. The shape of the peaks is 101

4 ω ω -0.2 π π (a) (b) 2 (c) 2 Figure 2: a.) Fourier coefficients ã (10) (ω ). The numbers show relative amplitude. b.) Fourier coefficients b(10) (ω ). c.) The kernels a (10) (), dashed, and b (10) (), solid line (a) A (9) (, y) (b) Ã(9) (ω, ω y) (c) B (9) (, y) (d) B (9) (ω, ω y) Figure 3: Spatial and frequency domain versions of the filters A and B known and given by b k (). Since we are not interested in the side lobes of this function, a simpler Gaussian shaped function will be used in the fit procedure. Because a fit has to be performed for every (, y) position, traditional iterative methods such as Levenberg-Marquardt require too much computation time. In this section we describe a fast alternative for the fit procedure. The σ of the Gaussian can be determined in advance since the width of the peaks is known. The Gaussian was fitted to a peak for order 9 yielding σ = radians. The fitted Gaussian coincides almost perfectly with the main lobe. Since the peak width decreases approimately as 1/(k + 1), the following relation gives the σ for any k: σ = (radians) (13) k + 1 To estimate the location and amplitude of the Gaussian the following error measure will be minimized: f Ag(ψ) 2 (14) Where f is the orientation response ( dimension), g a Gaussian positioned at ψ with amplitude factor A and the σ as given by equation 13. The inner product between functions m and n is defined as: (a) Figure 4: a.) Signal and inner product between shifted copies of the template and the signal (Gaussian filtering). b.) Signal and the detected peak. m() n() = (b) m()n()d (15) The integral is performed over one period of. Equation 14 can be written as: f 2 + Ag 2 2Af g (16) Only the last term is a function of ψ. Minimizing equation 14 with respect to ψ is equivalent to finding ma ψ (f g). Due to the symmetry of the Gaussian, convolving f with the Gaussian kernel yields f g for every value of ψ. The position of the maimum of the resulting signal is the ψ that maimizes f g. A typical f is shown in figure 4a. In the same figure f g is shown for every ψ. The net step is to fit the A parameter. Setting the deritivative of equation 16 with respect to A to zero and solving the resulting equation yields: 102

5 y orientation space y C slice CD slice AB A original image D (a) y A B C B (b) D (c) k=19 (d) (e) (f) (g) (h) (i) Figure 5: a) Schematic representation of orientation space. An, slice as indicated is used in (d-f). b) A noise free image containing two superimposed patterns. The intersection of the slice with the (, y) plane is indicated by line AB. The intersection with a second slice is indicated by line CD. c) Noisy version of (b). d) The slice indicated in (a) for the rms orientation intensity measured by filter A in image (b). e) Same as (d), but using filter B. f) Same as (e), but applied to image (c). (g-i) Same as (d-f) but using the other slice as indicated by line CD. Note the structures net to (f) indicating where in the slice each pattern gives a response. A = f g g 2 (17) In figure 4b the peak as detected by the algorithm, Ag, is shown plotted over the original signal f. To improve the fit it is necessary to remove the background offset before using the algorithm as discussed above. Since each parameter is optimized individually, there is no guarantee that a global optimum will be found. Maimizing f g will first yield the peaks with the largest volume. Such a peak is either a true peak or may consist of several overlapping peaks (This does not happen in the test images). Therefore a quality measure is needed to assess whether our assumption of isolated peaks holds. We will use the following quality measure: Q = f W g f W (18) Where f W is a windowed version of f with the window s center at the position of the detected peak. This is to prevent other peaks from influencing the quality measure. The window should be about as wide as the Gaussian, for eample 4σ. Q lies in the interval [ 1, 1], with 1 indicating a 103 high quality. By subtracting the peak that was found from f, a second peak can now be detected by repeating the procedure. Since Gaussian filtering can be implemented very efficiently [13], the entire procedure is very fast. 4 Eperiments In figure 5 we show the results of applying the filters proposed in this paper to an image consisting of two overlapping patterns. Each of the patterns has been created by calculating the Euclidean distance to some point in the image and computing the sine of the result. The amplitude of each individual pattern is one. The second test image was created by adding Gaussian noise with a variance of one to the first test image (SNR=6dB). We have computed the orientation space for both images using both filter A and filter B for k = 19. We show two slices of the orientation space for the following combinations: noise free image and filter A, noise free image and filter B, noisy image and filter B. The slices show the rms orientation intensity, using σ = 5 for the smoothing. The resulting slices show that the orientation selectivity is indeed improved. It is also clear that the scheme is quite robust with respect to noise. Furthermore, close eamination of the slices shows that there is indeed sub-piel infor-

6 (a) (b) (c) (d) Figure 6: a) Three touching patterns: left 51 degrees with respect to ais, middle 36, right 21. Image size 224 by 224 piels, diameter of circle 128 piels. Gaussian noise with variance 0.2 was added to the image. b), slice of orientation space halfway along the y ais. Parameters used: k = 27, σ filter = 1.5, σ smoothing = 5. c) Magnitude of second detected peak divided by magnitude of first detected peak for each, y position. d) Detected boundaries superimposed on (a). mation present along the orientation ais, and it is therefore possible to interpolate in (the original) orientation space to get improved accuracy. The net eperiment concerns touching orientation fields. A test image was created containing a circular region in the middle and two regions around it separated by a second vertical boundary in the middle of the image. The circular region has an orientation of 36 degrees with respect to the ais, the left region 51 degrees, and the right region 21 degrees. The amplitude of the patterns is again one, and Gaussian noise with a variance of 0.2 (SNR=13dB) was added. Using the method described in section 3, the dominant peak in orientation space was found for each, y position. In figure 6c the magnitude of the second peak divided by the magnitude of the first is shown. When two orientation fields are present, as at the boundaries of the regions, the two peaks found will have approimately the same magnitude and hence the quotient will approach one. Indeed, the quotient is maimal at the boundaries as the results in figure 6c show. Using a watershed [11] based technique, the ridges in figure 6c are detected and shown superimposed on the original image in figure 6d. They coincide almost perfectly with the actual boundaries. Since the parameters were chosen to ensure that there are no overlapping peaks in orientation space, it was not necessary to use the quality measure as defined in section 3. The location of the maima is not determined with subpiel accuracy in the current implementation. Subtracting a slightly misaligned peak results in larger residual flanks. The orientation of the center region in figure 6a corresponds eactly with a piel position, whereas the other patterns do not. As can be seen in figure 6c, the residual peaks are indeed larger for the other two regions. The location of the boundaries found in the second eample is very accurate. However, as the difference in angle between the different regions decreases, a higher order filter will be required to distinguish between the regions. But, as the 104

7 narrow wide [2] W.T. Freeman and E.H. Adelson, The Design and Use of Steerable filters, IEEE transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 9, September 1991, pp [3] L. Haglund, Adaptive Multidimensional Filtering, PhD thesis, Linköping University, Sweden, Figure 7: Localization depends on orientation with respect to the boundary filter becomes narrower in the frequency domain, the filter will become wider in the spatial domain. This will have a corresponding adverse effect on localization. Due to the shape of the filter as shown in figure 3c, localization depends on the orientation of the boundary with respect to the oriented pattern, see figure 7. Fortunately, the situation where the pattern(s) and the boundary have a similar orientation (allowing good localization) occurs most frequently in real world images. 5 Conclusions We have shown that the peak width of the response in orientation space can be made to depend linearly on the number of filters k. This occurs at the cost of introducing some side lobes. Simple eperiments with this scheme show that it works well for noisy images. The method used to detect the boundaries is still rather crude and more advanced techniques to analyze orientation space must be developed. Real orientation fields may not only differ in orientation, but also in amplitude and frequency content. Furthermore the interplay between filter order, frequency content, attainable orientation selectivity and localization and its consequences must be investigated. Acknowledgements This work was partially supported by the Rolling Grants program 94RG12 of the Netherlands Organization for Fundamental Research of Matter (FOM) and by the Royal Dutch Academy of Sciences (KNAW). References [1] M. Andersson, Controllable Multidimensional Filters and Models in Low Level Computer Vision, PhD thesis, Linköping University, Sweden, [4] M. Kass and A. Witkin, Analyzing Oriented Patterns, Computer Vision, Graphics and Image Processing, vol. 37, 1987, pp [5] H. Knutsson, Filtering and Reconstruction in Image Processing, PhD thesis, Linköping University, Sweden, [6] J.J. Koenderink, The Structure of Images, Biological Cybernetics, vol. 50, 1984, pp [7] M. Michaelis and G. Sommer, Junction Classification by multiple orientation detection, in: Jan-Olof Eklundh (ed.) ECCV 94, Third European Conference on Computer Vision, Stockholm, Sweden, May 1994, pp [8] P. Perona, Deformable Kernels for Early Vision, IEEE transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 5, May 1995, pp [9] L.J. van Vliet and P.W. Verbeek, Estimators for Orientation and Anisotropy in Digitized Images, in: J. van Katwijk, J.J. Gerbrands, M.R. van Steen, J.F.M. Tonino (eds.), ASCI 95, Proc. First Annual Conference of the Advanced School for Computing and Imaging (Heijen, NL, May 16-18), ASCI, Delft, 1995, pp [10] L.J. van Vliet and P.W. Verbeek, Segmentation of overlapping objects, in: L.J. van Vliet, I.T. Young (eds.), Abstracts of the ASCI Imaging Workshop 1995, Venray, The Netherlands, October 1995, pp [11] B.J.H. Verwer, L.J. van Vliet and P.W. Verbeek, Binary and Grey-value skeletons: Metrics and Algorithms, International Journal of Pattern Recognition and Artificial Intelligence, vol. 7, no. 5, 1993, pp [12] A. Witkin, Scale space filtering, Proc. Int. Joint Conf. on Artif. Intell., Karlsruhe, Germany, 1983, pp [13] I.T. Young, L.J. van Vliet, Recursive implementation of the Gaussian filter, Signal Processing, vol. 44, no. 2, 1995,