A Novel Keypoint Detection in Wavelet Pyramid Space

Transcription

1 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 A Novel Keypoint Detection in Wavelet Pyramid Space Pattern Recognition Intelligent System Laboratory, Beijing University of Posts telecommunications, Beijing, China {gaoyanyan, zhhg, guojun}@bupt.edu.cn doi: /jdcta.vol5.issue6.8 Abstract Keypoint detection is important for object recognition, image retrieval, mosaicing etc., has attracted ample research. In this paper, we propose a novel wavelet-based detector () based on the previous researches on keypoint detection. is performed in wavelet pyramid space, it extracts the local extrema of the energy map computed by intra-scale coefficient product (ISCP) as the cidate keypoint, then discards some points by Hessian matrix. In the experiments, the novel detector was compared with Harris detector detector by the evaluation of repeatability, it achieved better performance for some scenes in the database provided by Mikolajcyzk Schmid, such as wall, trees, graffiti. 1. Introduction Keywords: Keypoint Detector,, the Intra-scale Coefficient Product Keypoints usually mean the corners, blobs in an image, they are important in many current solutions to computer vision challenges, such as object detection recognition, content-based image retrieval, mosaicing, motion tracking, stereo matching, etc. Aiming to gather useful features, keypoints detection has long been researched various detectors have been proposed so far. Moravec s corner detector [1], Harris detector [] Smallest Univalue Segment Assimilating (SUSAN) [3] are three corner detectors proposed in early. Although those three detectors process image in different ways, none of them is scale-invariant. To solve this problem, Lindeberg [4] introduced the scale selection theory, which allows detecting keypoints with their own characteristic scale. Meanwhile, Hessian detector for blob is proposed. Based on the theory, Mikolajczyk Schmid proposed Harris/Hessian-Laplace detector with scale-invariant Harris/Hessian-affine detector [5] with affine-invariant. In addition, intensity-based region [6] Maximally Stable Extremal Region (MSER) [7] are also affine-invariant for detecting regions. However, most of detectors described so far are computationally expensive. To improve this problem, Lowe proposed Scale Invariant feature Transform () [8] which detects the keypoints by Difference-of-Gaussian (DoG) as the approximate of the Laplacian of Gaussians (LoG). For high efficiency, Bay et al. proposed Speed-Up Robust Feature (SURF) [9] based on Hessian-matrix, which performed the convolution of the integral image box-type filter in parallel to obtain Gaussian scale space. Although SURF are efficient, they are lack of affine-invariant. Wavelet theory provides a powerful framework to decompose images into different scales orientations, which is coherent with human perception. Wavelet transform has successfully been exploited in image compression, denoising texture analysis [10], also been introduced into image classification [11] keypoint detection [1, 13]. However, previous work has not exploited the scale of wavelet transform for image feature-invariant, leaving a study space for us. In this paper, we detect the keypoints in wavelet scale space which is constructed by the Dual Tree Complex Wavelet Transform (DTCWT) [14], our goal is to obtain the keypoint with multi-scale coordinates. This paper is organized as following. Section describes some widely used corner blob detectors. Section 3 details the Novel Wavelet-Based Detector () after the detector proposed by Loupias [1]. Section 4 gives an overview of the experimental setup, then shows the detailed results of the experiments followed by the conclusion in Section

2 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011. Related work.1. Corner detectors Corner detection is a kind of common operator in image processing computer vision. In the real world, Corners mean the corners of object, the intersections T-junction of street so on. Corners in the D image are the points with high curvature, many corner detectors have been proposed in the literature. In this section, we describe some corner detectors: Harris, Harris-Laplace, Harris- Affine. Harris Detector It is based on the second moment matrix, which is often used for feature detection for describing local image structures. The matrix of a point x is computed: L ( ) ( ) x D LxLy D M DG( )* I LL x y( D) Ly( D) (1) L L where x y are respectively derivatives in the x y direction calculated after smoothing the image I by a Gaussian of size D (derivation scale). G( I ) is a Gaussian function of width I (integration scale). In practice, we compute the measurement: R det( M) trace ( M) where is empirical constant, [0.04,0.06] trace( M ) is the trace of M. For each pixel x of image,, det( M) () is the determinant of M R, R t R 0, R t (3) where t is the threshold. And then we can extract the local extrema as the keypoint. In Ref. [15], it exploited the Harris detector for image match on the case of unknown epipolar geometry unavailable epipolar constrain in a single scene. Harris-Laplace detector Although Harris detector is invariant with intensity rotation, it fails to satisfactorily deal with scaling. For being more suitable for human visual characteristics, Harris- Laplace detector is developed by Mikolajczyk Schmid based on Harris detector scale selection theory [5]. The detection can be performed by two steps: (1) Construct the scale space detect the corners by Harris detector at each scale as the cidate points; () Select the point with a characteristic scale which is the extremum of the Laplacian over different scales. Harris-Affine detector It is invariant with affine transformation, such as viewpoint large-scale changes. Given a set of initial points extracted at their characteristic scales by Harris-Laplace detector, the iterative estimation of elliptical affine regions allows obtaining affine invariant corners. The steps details: (1) Detect the initial keypoints using Harris-Laplace detector; () Estimate the affine shape with the second moment matrix; (3) Normalize the shape into a circle; (4) Re-detect the new location scale in the normalized images. (5) Return step if the eigenvalues of the second moment matrix for the new point are not equal

3 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June Blob detector Blob detection is another aspect of computer vision, which is the special case of region detection. Blob is referred as the region which intensity color are different with its surrounding. Blob detectors are based on the scale space theory rely on the differential method such as Laplacian of Gaussian (LoG), difference of Gaussian (DoG) determinant of Hessian [4]. Hessian Detector The second * matrix issued from the Taylor expansion of the image intensity function I(x) is the Hessian matrix: Lxx ( x, y; D ) Lxy ( x, y; D ) H Lxy ( x, y; D ) Lyy ( x, y; D ) (4) where L L xx, xy L yy are second-order Gaussian smoothed image derivatives. These encode the shape information by describing how the normal to an isosurface changes. The detection is based on det( H ) LxxLyy Lxy the determinant of Hessian matrix computed by, which reflects the local information of image structure. And the local extrema of det( H ) is extracted as the keypoint. DoG detector As demonstrated in [8], LoG can be approximated with DoG at reduced computational complexity. The detail scheme is described as: (1) Construct the scale space by smoothing the image several times with a Gaussian convolution mask; () Compute the difference of the smooth images pairwise to obtain a set of DoG response map; (3) Find the extrema of one map compared with its two neighbor scales by non-maximal suppression; (4) Discard some unstable points. Hessian-Laplace/Affine The Hessian-Laplace Hessian-affine detectors are similar in spirit as their Harris-Laplace Hessian-affine, expect that they start from the determinant of Hessian matrix rather than Harris corners. And they also been proposed by Mikolajczyk Schmid in Ref. [5] 3. Wavelet based keypoint detection In [1], the Discrete Wavelet Transform is used to perform the multi resolution analysis for the image. The image f is studied at one scale j, j N W f. The wavelet coefficients j are obtained as the convolution of the image with the wavelet function dilated at the scale j. Since wavelets with compact support are used, for each coefficient, the region of support in the proceeding of decomposition stage can be determined. For each coefficient of the coarsest level, it keeps a track of wavelet coefficients from coarse level to fine level, by finding the highest coefficient in the support region of next finer level. The sum of all traced wavelet coefficients is calculated as the saliency value for the final target pixel. The saliency value describes the intensity change of image signal over the different scales. The point with high saliency value is keypoint. The tracing procedure for a very small wavelet transformed image (8 8 pixels) is depicted schematically in Figure 1, a more detailed description of the original algorithm is in [1]

4 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 W W W 3 1 Figure 1. Tracing of wavelet coefficients with 3 scales Figure. Directional wavelets their magnitudes of D DT-CWT Keypoint detected by DWT only has the information of localization without the scale. For obtained the scale invariant, in this paper we exploit DTCWT to detect the keypoint. The main advantage of the complex wavelet is that it has improved properties in terms of shift sensitivity, directionality phase information. DTCWT overcomes the shortcomings of DWT which is lack of shift invariance the directional selectivity [14]. In Figure 1, we can see DTCWT has 6 wavelets which are oriented 6 directions ( 15, 45, 75 ). DTCWT has been exploited in image processing, such as in Ref. [16], it exploits DTCWT to extract the texture for image retrieval, in Ref. [17] it used DTCWT for image restoration. In this paper, we exploit the Intra-Scale Coefficients Product (ISCP) [18] to detect the keypoints. The coefficients can be denoted as a set C corresponding to the response in 6 directions: i C e, e,..., e 6 i 1 1 i 6 where i is presented the magnitude of the complex response, i (i=1, 6)is the corresponding. At one scale, we can get the coefficient product by s EC ( ) In our experiments, we choose 1, 1/6. The DTCWT decomposition of an w h image results in a dyadic decomposition into s 1,..., m s s scales. At each scale, we can get ISCP as (4) to obtain the energy map E s which size is w/ h/. After getting E s, we can detect the keypoints as following: (1) Find the extrema in a local neighborhood in E s. p is cidate keypoint, if E s (p) E s (p ), where p' N( p ), N( p) is the neighborhood of p with size of 3 3 in this paper. Note that, we do not consider the location of Es ( p) 0, the scale is s ; Es ( p) 0.1max( E s ) () Discard the low contrast points: If, the point is discarded, 0.1 is an empirical value; (3) Eliminating the edge responses by exploiting Hessian matrix of cidate keypoint s location, only need to check: 6 b1 b (5) (6) trace ( H) det( H ) r 1 r (7) where H is Hessian matrix, trace( H ) is the trace of H, det( H ) is the determine of H :

5 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 E E E xx xy H xy Eyy, trace( ) H Exx E yy det( H ) E, xxeyy Exy (8) And we set r 10. If the current point is satisfied Formula (5), keep the current point as the keypoint, return the coordinates of this point with current scale s. When we obtain the multiscale coordinates by above steps, we can project the coordinates onto the corresponding location in original image by multiplied the factor s. Figure 3 shows the detection results computed by (SiftDemo4, downloaded from In Figure 3, we can see that missed some corner points which attract our attention. 4. Experiments Figure 3. Detection results by our detector (left) (right). The small square represents the location of detected points. We evaluate our method on the database provided by Mikolajczyk Schmid in Ref. [5], which is downloaded from the website linked to Figure 4. shows some examples in this database. This database consists of real images with different geometric photometric transformations, including viewpoint change, image blur, illumination change, JPEG compression, rotation scale change, also gives the ground truth matches through estimated homographic matrix H from the query image to the other reference image with changing. In detail, each scene has 6 images to show different extent of variance in some way, only 3 samples for each scene are shown in Figure 4. In this paper, the repeatability [19] is exploited to measure the detector. (a) bark: rotation + scale (b) boat: rotation + scale (c) bikes: blur (d)trees: blur (e) graffiti: viewpoint (f) wall: viewpoint (g) cars: illumination (h) ubc: JPEG compression Figure 4. Some images in the Oxford image database

6 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 X P O P i I O xo x i I i O o H oi Figure 5. The points xo xi are the projection of 3D points at image Io I i : x o P 1 X, xi PX i, where Po Pi are the projection matrices. A detected point xo is repeated if a point was detected in the neighborhood of x i, the size of neighborhood is defined by. In case of planar scenes the points xo xi are related by the homographic matrix H oi. Repeatability criterion is proposed in [19], which is defined as the number of points repeated R between two images with respect to the total number of detected points. The repeatability ( ) i for the image Ii is defined as: O i # correspondences R ( ) i min( n, n ) o i (9) where no ni are the number of points detected in the common part of original image Io Ii respectively. #correspondences is the correspondent number of the two related images, computed by: correspondences xˆ, xˆ dist( H xˆ, xˆ ) o i oi o i (10) where is the threshold determining the size of neighborhood (see Figure 5), x ˆo points in the common parts of the two image I o Ii ˆ respectively. { x o } ˆ { x i } parts of the two images, defined as: xˆi are the are the common { xˆ } { x H x I } i i io i o { xˆ } { x H x I } o o oi o i (11) ˆ where { x o } { x i } are the points detected in Io image Io to I i, inversely Hio is from Ii to o I i, 1 I, H H Hoi is the homographic matrices from io oi. Figure 6 is the comparison results of repeatability of 3 detection algorithms: (Harris), (DoG), (the detector proposed in section 3.). The X-axis represents different reference images in the same scene with query image, Y-axis is repeatability value. In this experiment, we set =

7 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 (a) bark (b) bike (c) boats (d) cars (e) graffiti (f) trees (g) ubc (h) wall Figure 6. Repeatability of 3 different detectors:, (DoG), (Harris) ( =4). In Figure 6 we can see that, for two scenes of graffiti wall with the viewpoint changing, tree with the blur, the repeatability of is better than others. For the scene of ubc with different compressing ratios, there is no big change of the repeatability which is keeping around 70% using, but the others are declined, especially declining precipitously; for bark with rotation scale variances, is better than at the first two pairs of images, but the last is not better than others; for boat also with rotation scale variances, is at the disadvantage, but all of the repeatability is about above 65%; for bike with blur cars with light changes, also did not get better performance obviously, but the tendency of variation is slowly. From the results, we can get the performance of has advantage in repeatability for texture image (such as wall, trees) because the wavelet coefficients of high frequency can show image details better. For images with scale rotation variance it can achieves similar performance with Harris detector. 5. Conclusion In this paper, we proposed to detect the keypoints by DTCWT, which extracts the extrema in energy map at each scale of wavelet transform, discards some low energy points by threshold Hessian matrix. The experiment results show that, for texture image, the proposed method has higher repeatability, for images with scale rotation variance, its performance is comparable with Harris detector. In future work, we could consider exploiting the information of wavelet coefficients to describe the local point, then perform the experiments in image retrieval, image match, or mosaicing etc. 6. Acknowledgements This paper was partially sponsored by the grants from National High-tech 863 Project of China under Grant No. 007 AA01Z417, the Fundamental Research Funds for the Central Universities,

8 International Journal of Digital Content Technology its Applications. Volume 5, Number 6, June 011 Project of China (B08004) Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry. 7. References [1] H. Moravec, Towards automatic visual obstacle avoidance, International Joint Conference on Artificial Intelligence, p. 584, Aug [] C. Harris M. Stephens, A combined corner edge detector, Alvey Vision Conference, pp , [3] S.M. Smith J.M. Brady, Susan-a new approach to low level image processing, International Journal of Computer Vision, vol. 3, no. 1, pp , [4] T. Lindeberg, Feature detection with automatic scale selection, International Journal of Computer Vision, vol. 30, no., pp , [5] K. Mikolajczyk C. Schmid, Scale & affine invariant interest point detectors, International Journal of Computer Vision, vol. 60, no. 1, pp , 004. [6] T. Tuytelaars L. Van Gool, Matching widely separated views based on affine invariant regions, International Journal of Computer Vision, vol. 59, no. 1, pp.61-85, August 004. [7] J. Matas, O. Chum, M. Urba, T. Pajdla, Robust wide baseline stereo from maximally stable extremal regions, British Machine Vision Conference, pp , 00. [8] D.G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no., pp , 004. [9] H. Bay, T. Tuytelaars, L. Van Gool, Surf: Speed-up robust feature, European Conference of Computer Vision, pp , 006. [10] Yong Xu, Xiong Yang, Haibin Ling, Hui Ji, A newtexture descriptor using multifractal analysis in multi-orientation wavelet pyramid, IEEE Conference on Computer Vision Pattern Recognition, pp , 010. [11] A. Teynor H. Burkhardt, Wavelet-based salient points with scale information for classification, International Conference on Pattern Recognition, pp. 1-5, 008. [1] E. Loupias, N. Sebe, S. Bres, J.-M. Jolion, Wavelet-based salient points for image retrieval, International Conference on Image Processing, vol., pp , 00. [13] W. Ayadi A. Benazza-Benyahia, Wavelet based statistical detection of salient points by the exploitation of the interscale redundancies, IEEE International Conference on Image Processing, pp , Nov.009. [14] N.G. Kingsbury, Complex wavelets for shift invariant analysis filtering of signals, Journal of Applied Computational Harmonic Analysis, vol. 10, no. 3, pp , May 001. [15] Yue Zhao, Xiaohong Shang, Hongqiang Ding, Matching Approach Based on Cross-Correlation Affine Transformation, International Journal of Digital Content Technology its Applications, vol. 5, no., pp , 011. [16] Jianhua Wu, Zhaorong Wei, Youli Chang, Color Texture Feature For Content Based Image Retrieval, International Journal of Digital Content Technology its Applications, vol. 4, no. 3, pp , 010. [17] Yuanjiang LI, Yuehua LI, Jianqiao Wang, "Passive Millimeter-wave Image Restoration Based on Symmetric Inverse Gaussian Model", International Journal of Digital Content Technology its Applications, vol. 5, no. 3, pp , 011. [18] J. Fauqueur, N.G. Kingsbury, R. Anderson, Multiscale keypoint detection using the dual tree complex wavelet transform, IEEE International Conference on Image Processing, pp , 006. [19] C. Schmid, R. Mohr, C. Bauckhage, Evaluation of interest point detectors, International Journal of Computer Vision, vol. 37, no., pp , Jun