P-SURF: A Robust Local Image Descriptor *

Transcription

1 JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 27, 2-25 (2) : A Robust Local Image Descriptor * CONGXIN LIU, JIE YANG AND HAI HUANG + Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai, 224 P.R. China + Department of Computer Science and Engineering Zhejiang Sci-Tech University Hangzhou, 3 P.R. China -like representations are considered as being most resistant to common deformations, although their computational burden is heavy for low-computation applications such as mobile image retrieval. H. Bay et al. proposed an efficient implementation of called SURF. Although this descriptor has been able to represent the nature of some underlying image patterns, it is not enough to represent more complicated ones. Also, the proposed high-dimensional alternative to SURF indeed improves the distinctive character of the descriptor, while it appears to be less robust. In this paper, an enhanced version of SURF is proposed. Specifically, it consists of two components: the feature representation for independent intensity changes and the coupling description for these intensity changes. To this end, phase space is introduced to model the relationships between the intensity changes and several statistic metrics quantizing these relationships are also proposed to meet practical demands. The feature matching experiments demonstrate that our method achieves a favorable performance close to that of and faster construction-speed. We also present results showing that the use of the enhanced SURF representation in a mobile image retrieval application results in a comparable performance to. Keywords: SURF,, local image descriptor, image retrieval, image matching. INTRODUCTION Extracting and matching distinctive local image features between images is a fundamental problem in many applications. Many approaches have been presented to describe local image patterns in the literature. Popular descriptors include differential invariants [], steerable filters [2], complex filters [3], moment invariants [4], spin image [5], (Scale-Invariant Feature Transform) [6], and Shape Context [7]. The detailed performance evaluations of these descriptors were presented in [8, 9], where it was shown that the high-dimensional representations based on histograms of localized gradient orientations such as outperform other descriptors by a certain margin in matching images of both planar surfaces and 3D objects. Various refinements have been proposed in the literature to improve the gradient orientation based descriptors. For example, Ke and Sukthankar developed PCA- [] that represent the surface of an image patch by the principal components of the normalized gradient patch. The computational burden of PCA- is comparable to since the process of forming the normalized gradient patch involves many interpolation operations, which are time consuming. In addition, Received January 2, 2; revised August 7, 2 & January, 2; accepted January 3, 2. Communicated by Tong-Yee Lee. * This work was supported by the National Nature Science Foundation of China (No. 6752) and Projects of International Cooperation between Ministry of Science and Technology (No. 29DFA287). 2

2 22 CONGXIN LIU, JIE YANG AND HAI HUANG applying PCA also slows down the feature computation. GLOH (Gradient Location Orientation Histogram) [8] modified the representation by using alternative spatial sampling strategy and PCA for dimensionality reduction. 28-dimensional GLOH was proved to more distinctive than, but it needs more computational demands. Several low-cost descriptors have also been reported in the literature [-3]. H. Bay et al. proposed a descriptor, called SURF, which is an efficient implementation of by applying the integral image for faster computation []. It contains both feature detection and feature description. As can be observed from [], SURF is able to represent the nature of some underlying image patterns, although it cannot distinguish the following two image patterns as shown in Fig. 3 (a). This can be attributed to the fact that SURF describes the intensity changes of local image patterns in two orthogonal orientations independently; this makes it impossible to obtain enough structural information of the patterns. Moreover, to increase the distinctiveness of SURF, [] also proposed a highdimensional alternative to it. This extension is indeed more distinctive than SURF, while it appears to be less robust. Since the extension is only a finer subdivision scheme to the low-dimensional one, it is more sensitive to small errors in feature localization and small variations in the shapes of features. Furthermore, the finer subdivision scheme just encodes a little more structural information of local image patterns as compared with the low-dimensional one; therefore, the proposed alternative scheme can be further improved in discriminative power. Another two low-cost descriptors, namely CS-LBP (simplified center-symmetric local binary patterns) [2] and Contrast Context Histogram [3], both exploited the contrast between pixel values to reduce computational cost. The former one needs to set a contrast threshold in the case of flat regions, which increases the sensitivity of the descriptor to the parameter. The latter lacks the description of the correlation between adjacent pixels in its feature representation. Motivated by these findings, we still focus on such kinds of algorithms and explore an improved version of SURF. In this paper, phase space is introduced to improve the performance of SURF. Phase space, in which all possible states of a system are represented, is introduced from physics. In a phase space, every degree of freedom or parameter of the system is represented as an axis of a multidimensional space. If an image surface is regarded as a system and its pixel location (x, y) is considered as a vector variable, then the triples (pixel intensity, horizontal gradient, and vertical gradient) can represent the state of an image at each sample position. Due to brightness changes, intensity value cannot serve as a state component in the context of image matching. Therefore we constructed a reduced phase space using the tuples (horizontal gradient, and vertical gradient) to describe the state of each pixel. Essentially, each region in phase space corresponds to a kind of relationship between the gradients. Besides, the densely sampled gradients are used to represent the intensity changes at each sample location instead of using the sparsely sampled Haar wavelet responses. This can be attributed to the following three reasons. First, like Haar wavelet responses, gradients can also reflect intensity changes. Second, the use of gradients can simplify the calculation of Haar wavelet responses and facilitate the implement of the descriptor in practice. Third, the computational complexity of two approaches is comparable on the normalized patches [4]. Compared to [6], our method (: phase-space based SURF) is more efficient since it neither computes gradient orientation nor applies any interpolation to the feature representation while yet preserving consider-

3 A ROBUST LOCAL IMAGE DESCRIPTOR 23 able distinctiveness. The remainder of this paper is organized as follows: in section 2, the details of the proposed local image descriptor are presented. In section 3, we provide detailed experimental results on feature matching experiments and also in the context of a mobile image retrieval application. Section 4 concludes the paper. 2. THE PHASE-SPACE BASED SURF DESCRIPTOR 2. Primary Representation can be summarized in the following steps: () Given an image patch, compute both the horizontal and vertical gradient maps. The horizontal gradient, vertical gradient and corresponding image gradient magnitude are first computed at each sample point over the image patches [4]. To guarantee invariance to orientation change, the coordinates of the descriptor are rotated relatively to the assigned dominant orientation. Both the new image coordinates and the gradient maps at each sample point can be obtained from the old ones by a linear transformation. The new gradient maps are illustrated with small arrows at each sample location in the middle-right of Fig.. haar x haar y haar x haar y SURF-64 haar y > haar x haar x haary haarx x haar x > haary y haarx haary y SURF-28 haar x haar y Process o Process dx v dy dx dy Independent component Gradients for the 4th sub-region dy dy v 2 Region Region dx dx Region 8 Haar responses for the 4th sub-region Gaussian kernel A phase space partition scheme Coupling component 4*4 sub-regions SURF representation over an image patch representation Fig.. The difference between and SURF. Region 8 (2) Building the descriptor based on a concrete phase-space partition scheme. In the new coordinate system, the input image patch is first divided into 4 4 subregions shown in the middle of Fig.. Such sub-regions inspired by [6] can capture important spatial information. (a) The representation for the independent intensity changes (similar as SURF). In each sub-region, the horizontal gradient, denoted as dx, and the vertical gradient, denoted as dy, are summed up to form the first two entries of the description vector. Given the polarity of the intensity changes, the absolute values of the gradients are also accumulated. Thus, a 4-dimensional vector v = { dx, dy, dx, dy } is obtained, which serves as the first set of entries of the proposed descriptor for each sub-region.

4 24 CONGXIN LIU, JIE YANG AND HAI HUANG Furthermore, to reduce the impact from misregistration errors, dx and dy are weighted by a Gaussian function with the deviation equal to half of the width of the image patch shown by the overlaid red circle in the middle of Fig.. (b) The coupling description for the intensity changes. Next, we use phase space to model the relationship between dx and dy. Specifically, the intensity space over each sub-region is first transformed into phase space, and then a specific subdivision scheme is applied to the space. A naive scheme to partition the phase space is illustrated in Fig., where the whole phase space is evenly divided into eight regions. Each region of the phase space represents a kind of relationship between dx and dy. In order to describe these regions, three statistic metrics, namely gradient norm: sqrt ((dx) 2 + (dy) 2 ), count norm:, and fluctuation norm: dx dy /( dx + dy ), are introduced. Each phase-space region is quantized by the sum of the metrics. These sums are illustrated by the green line segments on the lower-right of Fig.. In practice, which mea- sure is adopted depends on the actual needs. Generally, a good performance can be obtained by using the gradient norm. However, when quick computation is more important, count norm may be a good choice. Under rotation and scale change, fluctuation norm illustrates a good performance. In this way, another 8-dimensional vector v 2 is obtained. It can be observed that what v 2 tries to reveal is the underlying structure information of local image patterns. Therefore, v and v 2 are mutually complementary feature representations. This is different from the proposed high-dimensional extension to SURF [], where the extension is achieved by splitting up the old one based on the signs of haar x and haar y as shown on the left side of Fig.. Thus, our improved descriptor is clearly more distinctive than it. Likewise, in order to lessen the impact from the feature localization errors and shape errors of features, the gradient norm (or count norm, fluctuation norm) at each sample point is assigned a weight by a Gaussian kernel shown by the overlaid red circle in the middle of Fig.. The farther a sample point is away from the center point of the patch, the smaller weight it obtains to weight the statistic norms. Combining v and v 2, we obtain a combined vector v = {v, v 2 } for each sub-region. The descriptor for the image patch is achieved by concatenating these vectors on 4 4 sub-regions in the given order as shown in the middle of Fig.. The dimension of the descriptor depends on the specific phase space partition. is similar to in that they are both the histograms (or combined histograms) of spatially localized gradient; thereby they are both robust to significant shift ingradient positions. As shown in Fig., a gradient sample, shifting randomly over 5 5 set of sample locations, makes almost the same contribution to the histogram of the 4th sub-region. Furthermore, is less sensitive to noise than, as shown in the example of Fig. 2. This is attributed to the fact that integrates the gradient samples within the sub-regions and the phase-space regions, while only relies on the integration of the gradient samples within adjacent orientations. In terms of discriminative power, performs well, as shown in Fig. 3. One can imagine that the combination of the two intensity patterns and the other ones [] will result in a distinctive descriptor. Therefore, our good performance in non-geometrical scenes is not surprising in the following experiments.

5 A ROBUST LOCAL IMAGE DESCRIPTOR 25 Cl ean v v Noi sy SI FT dx dy dx dy dx dy dx dy SURF64 P- SURF28 Regi on Regi on Due to projective distortions or noise, a gradient sample v is transformed into v. If not beyond the range of the region, v makes the same contribution to as v. Therefore is more robust to the variations of gradient directions than the locally operating but less robust than SURF64. Note that the image patches are from []. dx dy dx dy dx dy dx dy Fig. 2. The robustness comparison between, SURF, and. d.4 SURF 2 d 2 SURF (-) 3 d d d d d d (a) Intensity pattern with fluctuations in two (b) Intensity pattern with frequencies in one orientations (4 pixels). orientation (8 pixels). Fig. 3. The distinctiveness comparison between, SURF, and on the underlying intensity patterns with significant fluctuations. 2.2 Refined Representations Some important issues about in practical applications should be further considered. First, how to lessen the boundary effect? () Dominant orientation alignment. The region layout scheme illustrated in Fig. usually causes significant boundary effect when the horizontal direction has been aligned with the dominant orientation. This is mainly attributed to the fact that the gradients close to the horizontal direction often commute between the region and the region 8 due to the initial misregistration errors. To address the problem, a refined region partition scheme is proposed in Fig. 4 (c), where the central axis of region is aligned with the horizontal direction and the dominant orientation is assumed in accordance with it. As a result, the gradients around the dominant orientation are almost partitioned into an identical region (region ), which tends to lessen boundary effect and increase the robustness of the descriptor; (2) Overlapping boundaries of phasespace regions. This measure enables the adjacent regions to share the gradients near the boundaries, which can further lessen boundary effect. The proposed overlapping schemes are illustrated in Figs. 4 (a)-(c). Second, how to reduce memory consumption and further increase matching speed? For embedded devices, lower memory usage and higher computational efficiency are critical to practical applications; therefore it is also necessary to find low-dimensional description schemes. Figs. 4 (a) and (b) show two alternative low-dimensional counterparts to the 8-region scheme, respectively. Third, some tricks may be taken for the convenience of computation. For the 4-re-

6 26 CONGXIN LIU, JIE YANG AND HAI HUANG dy dy dy dy=2.4dx Region2 dy=dx Region2 Region2 dy=dx dy=dx dx dx dx Region3 Region Region Region dy=-dx dy=-dx dy=-dx Region4 dy=-2.4dx (a) 4-region partition. (b) 6-region partition. (c) 8-region partition. Fig. 4. Refined representations. gion partition scheme, an offset-angle of π/4 radians is first subtracted from the dominant orientation and then the descriptor coordinates are rotated to align with this new dominant orientation. Next, the histogram entry of each gradient sample can be determined by judging the configurations of the signs of the gradient maps at each sample location. 2.3 Computational Complexity In this section, we discuss the computational burden of and. has to compute gradient (magnitude and orientation) at each sample location. This process involves time-consuming inverse tangent computation. circumvents and simplifies the problem by comparing the size of horizontal gradient and vertical gradient at each sample position or judging configurations of their signs to obtain the corresponding bin in the histogram. Moreover, to avoid boundary effects, carries out trilinear interpolation to smoothen the histogram, while the representation doesn t apply any interpolation scheme. Based on the above reasons, is computationally faster than. 3. Experimental Setup 3. EXPERIMENT EVALUATION In this section, is evaluated according to the evaluation metrics in [8]. The standard dataset, which comes from [8, 4], contains eight image sequences. Various deformations have been applied to these images, like viewpoint change, scale and rotation change, light change, blur, and JPEG compression. The descriptors in [8] were built on 4 4 normalized regions [4] as being transformed from affine invariant regions. These regions are extracted by Harris-affine and Hessian-affine feature detectors [5]. The dominant orientations of these image patches are defined as the directions of the smoothed gradients over the patches. A Gaussian weighting function with the standard variance equal to half of the width of the image patches is used to assign a weight to the (dx, dy) at each sample point. Additionally, to evaluate the performance of on the patches invariant to scale change, DOG [6] is used to conduct the experiments of Recall versus IT and mobile image retrieval. Recall versus IT (Increasing transformations) is defined as follows. Recall is the ratio of the number of correct matches found to that of total matches found by the descrip-

7 A ROBUST LOCAL IMAGE DESCRIPTOR 27 tor. The curves are obtained by varying the test images to match the first images in the sequences. Due to using DOG as detector, the metric x a Hx b 2 < t [5] is applied to the determination of the number of the correct match for the convenience of calculation. H is the homography between the image pairs; x a and x b is a tentative corresponding point pair; t represents the distance threshold. -like approaches obtained the best performance [8] and are still identified as being most resistant to common image deformations until today [2, 3, 6]. Hence, we compare to, SURF in this paper. Since the concept of scale is not distinct over the normalized patches, SURF was simplified by using its densely sampled alternative instead. The simplified SURF is called A-SURF64 in the following experiments. We believe the similar relative performance will re-occur in its original way of implementation []. A-SURF28 was achieved by following the approach proposed in []. The distance threshold t is set to be 3 (pixel); Overlapping regions are within the range of. All the test programs were performed on a LAPTOP of AMD.9GHz, 2GM. 3.2 The Experimental Results of Feature Matching 3.2. Scheme selection The scheme for our experiments is selected from the proposed ones in Fig. 4 by comparing their performance on Hessian-Affine regions. The results are shown in Fig. 5, where the 4-region scheme performs relatively better than the other two on the image pairs. Similar results are also obtained in the other scenes. Moreover, 28 outperforms 64 (it is the second part of 28) in all the cases. Due to the space limit, the other experimental results aren t shown here. In the following experiments, the 4-region scheme is used to compare with due to its relatively leading performance and more reasonable dimensions Bark Leuven precision precision (a) Scale + rotation. (b) Illumination change. Fig. 5. Performance comparison of under different parameter configurations on Hessian- Affine regions Comparative experiments on boundary effect In this section, a series of comparative experiments were conducted to verify the ef-

8 28 CONGXIN LIU, JIE YANG AND HAI HUANG.9 Bike HarAff&PSURF(N) HesAff&PSURF(N) HarAff&PSURF(D).9 Bark HarAff&PSURF(N) HesAff&PSURF(N) HarAff&PSURF(D).8.7 HesAff&PSURF(D) HarAff&PSURF(DO) HesAff&PSURF(DO).8.7 HesAff&PSURF(D) HarAff&PSURF(DO) HesAff&PSURF(DO) precision (a) Blur (st-4th). Leuven precision (b) Scale + rotation (st-4th). Ubc HarAff&PSURF(N) HesAff&PSURF(N) HarAff&PSURF(D) HesAff&PSURF(D) HarAff&PSURF(DO) HesAff&PSURF(DO) precision.2. HarAff&PSURF(N) HesAff&PSURF(N) HarAff&PSURF(D) HesAff&PSURF(D) HarAff&PSURF(DO) HesAff&PSURF(DO) precision (c) Illumination change (st-4th). (d) JPEG compression (st-4th). Fig. 6. The results of reducing boundary effect for (a) Blur; (b) Scale + rotation; (c) Illumination change; (d) JPEG compression. fectiveness of the measures proposed in section 2.2 in avoiding boundary effect. The experimental results are shown in Fig. 6, where (N) represents without dominant orientation alignment, (D) with dominant orientation alignment, and P- SURF (DO) with dominant orientation alignment and boundary overlapping. From the plots in Fig. 6, it can be observed that with dominant orientation alignment outperforms (N) significantly in robustness and distinctiveness. In the experiment of Recall versus IT, when dominant orientation alignment is used, the number of correct matching point pairs increased by around 5%. Therefore, dominant orientation alignment is an effective measure for to reduce boundary effect. Moreover, boundary overlapping can also improve s performance in most cases, but the performance gain obtained from it is less than that introduced by the dominant orientation alignment. By the way, radially-weighted scheme was also employed to lessen boundary effect. The experimental results demonstrated that this measure improves the robustness of P- SURF, but reduces the discriminative capability of the descriptor. Concerning the overall performance, we abandoned this measure Feature matching experiment Figs. 7 (a)-(f) show the experimental results of on the different image pairs

9 A ROBUST LOCAL IMAGE DESCRIPTOR Wall HarAff& HarAff& HarAff&A-SURF28 HarAff&A-SURF64 HesAff&.9.8 Bark HarAff& HarAff&PSURF28 HarAff&A-SURF28 HarAff&A-SURF64 HesAff&.7 HesAff& HesAff&A-SURF28.7 HesAff&PSURF28 HesAff&A-SURF28 HesAff&A-SURF64 HesAff&A-SURF precision (a) Viewpoint (st-5th). HarAff& HarAff&PSURF28 HarAff&A-SURF28 HarAff&A-SURF64 HesAff& HesAff&PSURF28 HesAff&A-SURF28 HesAff&A-SURF64 Bike.2.8 -precision (b) Scale + rotation (st-4th). Tree HarAff& HarAff& HarAff&A-SURF28 HarAff&A-SURF64 HesAff& HesAff& HesAff&A-SURF28 HesAff&A-SURF precision (c) Blur (st-4th). Leuven.2.8 -precision (d) Blur (st-4th). Ubc HarAff& HarAff&PSURF28 HarAff&A-SURF28 HarAff&A-SURF64 HesAff& HesAff&PSURF28 HesAff&A-SURF28 HesAff&A-SURF precision.2. HarAff& HarAff&PSURF28 HarAff&A-SURF28 HarAff&A-SURF64 HesAff& HesAff&PSURF28 HesAff&A-SURF28 HesAff&A-SURF precision (e) Illumination change (st-4th). (f) JPEG compression (st-4th). Fig. 7. (a) Recall versus -precision curve for viewpoint change; (b) Scale + rotation; (c, d) Blur; (e) Illumination change; (f) JPEG compression. from the standard dataset. In Fig. 7 (a), we show the comparative result of the four methods, namely,, A-SURF64, and A-SURF28, under affine transformation, while Fig. 7 (b) shows the result under scale change and rotation. Moreover, Figs. 7 (c) and (d) show the performance comparison of the methods under significant amount of image blur in the structured scene and textured scene, respectively. Finally, the comparative results under illumination change and JPEG compression are shown in Figs. 7 (e) and (f). In all of these plots, Harris-Affine and Hessian-Affine were used as detectors.

10 2 CONGXIN LIU, JIE YANG AND HAI HUANG As can be seen from these plots, obtains comparable or better results compared to in the non-geometric transformation scenes and lower scores in the geometric transformation scenes. This is because that s smoothed gradient orientation histogram was carefully designed for handling misregistration errors caused by viewpoint or other changes. However, both the local out-performance and the minor performance difference confirm the robustness of our method under such geometric transformations. Moreover, we can also observe that is superior to A-SURF64 and A-SURF28 on all the image pairs. This demonstrates that our method indeed results in a significant improvement in performance. In addition, as expected, A-SURF28 performs better than A-SURF64 in most cases, but it is closely followed by the latter in many cases. In the cases of Leuven and Ubc, it is even outperformed by the latter. Thus, A-SURF28, which was proposed as the extension of SURF, is more sensitive to noise as compared to the other three algorithms in this experiment. It is also observed that the performance gap between and is larger for Harris Affine regions than for Hessian Affine regions. This could be attributed to that the higher localization accuracy of Hessian Affine detector favors our method more compared to as the latter can handle more localization errors. Figs. 8 (a)-(h) show that the overall quality of correspondences found by and A-SURF28 is close to that of, while A-SURF64 obtains a slightly lower score in this experiment. In some cases, even surpasses in ratio, which is Graffiti A-SURF28 A-SURF Wall viewpoint change (a) Viewpoint (structured scene) Boat A-SURF28 A-SURF64.2. A-SURF28 A-SURF viewpiont change (b) Viewpoint (textured scene). Bark scale change.2. A-SURF28 A-SURF scale change (c) Scale + rotation (structured scene). (d) Scale + rotation (textured scene). Fig. 8. (a, b) Recall versus IT curve for viewpoint change; (c, d) Scale + rotation.

11 A ROBUST LOCAL IMAGE DESCRIPTOR Bike Tree A-SURF28 A-SURF ASURF28 ASURF Increasing blur (e) Blur (structured scene). Leuven Increasing blur (f) Blur (textured scene). Ubc A-SURF28 A-SURF Decreasing light.2. A-SURF28 A-SURF JPEG Compress % (g) Illumination change. (h) JPEG compression. Fig. 8. (Cont d) (e, f) Blur; (g) Illumination change; (h) JPEG compression. shown in plots (d), (e), (g) and (h). These experimental results are in accordance with the experimental results shown in Fig. 7. Compared to Recall versus -precision curve, Recall versus IT curve can more directly reflect the performance of the descriptors in the practical applications such as image retrieval and 3D reconstruction. To sum up, the above experiments have demonstrated that has a similar performance to. Compared to SURF, has obtained a significant improvement in performance almost in all the cases. 3.3 Computational Cost Comparison Table compares with and SURF in terms of running time. From Table, it can be observed that 28 is more than three times faster than in the descriptor construction while slightly slower than A-SURF64 and A-SURF28. Note that the time values are the average of 5 times of tests, which are calculated from 758 local image regions extracted by Harris-Affine detector in the first image of Graffiti set. Table. Comparison of the average time consumption. A-SURF64 A-SURF28 28 Detection Descriptor 28s 28s 27s.698s

12 22 CONGXIN LIU, JIE YANG AND HAI HUANG 3.4 The Experimental Results for a Mobile Image Retrieval In this experiment, first, 5, images of different scenes have been scanned from the Magazine Business Week to create a reference image database for the later retrieval. Then, 2, test images are captured from the same scenes using different mobile phones by different people. Some of them are showed in Fig. 9. As can be seen, significant image degradations have occurred in these test images, like non-linear illumination change, blurring, and noise contamination. Furthermore, viewpoint change and scale change further increase the degradations. Reference image database (from a scanner) Test image database (from mobile phones) Fig. 9. Example images for the mobile image retrieval. Table 2. The practical results of image retrieval. A-SURF64 A-SURF28 28 Cluster algorithm AKM [8] AKM AKM AKM Cluster centers 5, 5, 5, 5, Accuracy rate 94.5% 95.9% 96.6% 96.2% Image retrieval was performed based on the bag of keypoints [7]. The final retrieval results are illustrated in Table 2, which shows that 28 offers a comparable performance to. The explanations for the retrieval results are shown as follows. First, the main reason resides in that the descriptors created by and both have the similar discriminative capability. In the case of non-geometrical transformation scenes, even obtains higher quality of matching point pairs than. Second, due to much noise contained in the test images, robust descriptors usually perform better in these cases. has more robustness than and therefore it obtains a better score in this experiment. Third, the approximate computations in clustering and encoding significantly reduce discriminative requirements for descriptors. As a result, A-SURF64 and A-SURF 28 also obtain a good performance in this experiment. The distinctiveness of a descriptor, which is very crucial to find exact correspondences among images, is less important in image retrieval, whose purpose is to find the most similar image as a whole.

13 A ROBUST LOCAL IMAGE DESCRIPTOR 23 Finally, the saliency detection algorithm [9] contributes much to the high retrieval rates illustrated in Table 2. This algorithm helps to obtain high informational regions while filtering out much background noise. As a result, 85% of the feature points are removed and a 2%-3% performance increase occurs. 4. CONCLUSION In this paper, we improve the SURF representation by introducing phase space to capture more structure information of local image patterns. In phase space, one region represents a kind of relationship between intensity changes. By building histograms on such regions, these relationships can be quantized. Three popular schemes to partition phase space are proposed and two corresponding measures, namely dominant orientation alignment and overlapping boundaries, are also introduced to lessen the boundary effect. Moreover, in order to compute more conveniently, the densely sampled gradients are used to represent intensity changes instead of the sparsely sampled Haar wavelet responses used by SURF. Experimental results demonstrate that a significant improvement in performance compared to SURF and its high-dimensional extension has been achieved. Compared to the state-of-the-art method, illustrates a favorable performance and a significant reduction in computational demands. Also, in the case of the mobile image retrieval experiment, our method yields an accuracy rate close to that of. Thus, P- SURF is an appropriate trade-off between performance and computational burden. We believe that it holds a great promise for many applications in the Computer Vision community, especially in the cases where low computational requirements are necessary. In the future, we will apply to large-scale mobile image retrieval. REFERENCES. J. Koenderink and A. J. van Doorn, Representation of local geometry in the visual system, Biological Cybernetics, Vol. 55, 987, pp W. Freeman and E. Adelson, The design and use of steerable filters, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 3, 99, pp F. Schaffalitzky and A. Zisserman, Multi-view matching for unordered image sets, in Proceedings of European Conference on Computer Vision, 22, pp L. J. V. Gool, T. Moons, and D. Ungureanu, Affine/photometric invariants for planar intensity patterns, in Proceedings of European Conference on Computer Vision, 996, pp S. Lazebnik, C. Schmid, and J. Ponce, Sparse texture representation using affineinvariant neighborhoods, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 23, pp D. G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, Vol. 6, 24, pp S. Belongie, J. Malik, and J. Puzicha, Shape matching and object recognition using shape contexts, IEEE Transactions on Pattern Analysis and Machine Intelligence,

14 24 CONGXIN LIU, JIE YANG AND HAI HUANG Vol. 24, 22, pp K. Mikolajczyk and C. Schmid, A performance evaluation of local descriptors, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, 25, pp P. Moreels and P. Perona, Evaluation of features detectors and descriptors based on 3D objects, in Proceedings of IEEE International Conference on Computer Vision, Vol., 25, pp Y. Ke and R. Sukthankar, PCA-: A more distinctive representation for local image descriptors, in Proceedings of IEEE International Conference on Computer Vision, 24, pp H. Bay, T. Tuytelaars, and L. V. Gool, SURF: speeded up robust features, in Proceedings of European Conference on Computer Vision, Vol., 26, pp M. Heikkilä, M. Pietikäinen, and C. Schmid, Description of interest regions with local binary patterns, Pattern Recognition, Vol. 42, 29, pp C. R. Huanga, C. S. Chena, and P. C. Chung, Contrast context histogram An efficient discriminating local descriptor for object recognition and image matching, Pattern Recognition, Vol. 42, 28, pp K. Mikolajczyk and C. Schmid, Scale and affine invariant interest point detectors, International Journal of Computer Vision, Vol. 6, 24, pp Z. Chen and S. K. Sun, A Zernike moment phase-based descriptor for local image representation and matching, IEEE Transactions on Image Processing, Vol. 9, 2, pp J. Sivic and A. Zisserman, Video google: A text retrieval approach to object matching in videos, in Proceedings of IEEE International Conference on Computer Vision, 23, pp J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, Object retrieval with large vocabularies and fast spatial matching, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 27, pp X. Hou and L. Zhang, Saliency detection: A spectral residual approach, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 27, pp. -8. Congxin Liu ( 刘 ) received a B.S. degree from Wuhan University of Hydraulic and Electrical Engineering (Yi Chang), China, in 997, a M.S. degree from Three Gorges University, China, in 24. He is currently a Ph.D. student in Shanghai Jiao Tong University. His research interests include local invariant feature and image matching.

15 A ROBUST LOCAL IMAGE DESCRIPTOR 25 Jie Yang ( ) received a Ph.D. degree in Computer Science from the University of Hamburg, Germany in 994. Dr. Yang is now the Professor of the Institute of Image Processing and Pattern Recognition in Shanghai Jiao Tong University. He has taken charge of many research projects and published more than 2 journal papers. His major research interests are image retrieval, object detection and recognition, data mining, and medical image processing. Hai Huang ( ) received his Ph.D. degree in Department of Computer Science and Engineering of Shanghai Jiao Tong University, China in 26. Currently, he is an Assistant Professor in Zhejiang Sci-Tech University, Hangzhou, China, 2. His research interests include cryptograhy, information security and digital watermarking.