An Evaluation of Volumetric Interest Points

Transcription

1 An Evaluation of Volumetric Interest Points Tsz-Ho YU Oliver WOODFORD Roberto CIPOLLA Machine Intelligence Lab Department of Engineering, University of Cambridge

2 About this project We conducted the first performance evaluation of volumetric interest point detectors. Evaluation of different volumetric features: quantitative analysis qualitative analysis An initial guide for selecting a suitable feature for applications. Volumetric Interest Points First Performance Evaluation New Evaluation Metric Selection Guide

3 Motivations In the world of Computer Vision, an object is the combination of: Texture (2D appearance) Shape (3D structure) An object can be represented by: Advantages of shape over texture features Viewpoint independent Robust to illumination changes Robust to occlusions

4 Motivations Existing applications: Mainly texture Some applications start to use 3D shapes as well Why are there more shape-based approaches?? Earlier shape-based approaches were limited by availability of data. More 3D shape data are available nowadays: Cheaper acquisition hardwares Better reconstruction techniques Mesh ToF Camera Stereo / SfM Videos Medical Imaging Kinect TM

5 Motivations Feature detection is the initial step for many C.V. applications: Volumetric Interest Points Feature selection has great effects on final performance. A systematic evaluation is therefore important However, evaluation of volumetric interest points is an unexplored area Evaluation Objectives: GIVEN Detection Recognition Registration Which interest point has the best performance? What are their characteristics? Application suitability

6 Contributions The first performance evaluation of volumetric interest points A baseline comparison for evaluating new volumetric interest points We enables researchers and developers to easily select a suitable detector for their application. Contributions A new metric to represent the quality of a interest point detector Typical repeatability does not address accuracy. Our metric combines both repeatability and localization accuracy.

7 Methodology Six Interest Point Detector Candidates: They have been applied in existing applications. Categorized to three types: Corner detection Harris, FAST Region-based blob detection MSER Derivative-based blob detection Hessian, SURF, DoG FAST Harris Corner Detection Hessian SURF MSER Region-based Blob Betection DoG Derivative-based Blob Betection

8 Methodology Evaluation Dataset Mesh Synthetic meshes No occlusion Synthetic transformation Known groundtruth Stereo From multi-view stereo Occlusions Uneven sampling density MRI Simulated MRI scans Occlusions MESH dataset : Volumetric conversion Meshes are converted to volumes using kernel density estimation. Mesh Point Cloud Vol. Data

9 Results: Quantitative Analysis NOISE ROTATION SCALE R area score under transformations: noise, scale, rotation. MSER has the best overall performance, followed by Hessian and DoG Corner-detection approaches (Harris and FAST) are less robust against transformations.

10 Results: Quantitative Analysis Mesh Stereo MRI Stereo has a considerably lower performance due to Occlusion and Noise of point clouds captured from stereo systems. R area score with different datasets Similar rankings are observed in the three datasets (i.e. MSER > DoG = Hessian > Others) Please refer to poster / paper for detailed evaluation results.

11 Results: Qualitative Analysis We have also evaluated the interest points on their qualitative properties: Types of features (e.g. edge/blob/corners, scale of features) Qualitative Properties Number of interest point detected Position of features Shapes of features Please refer to poster / paper for further details. Other Considerations Efficiency (e.g. MSER is accurate but it can be slow for 3D data, FAST is more efficient but it is less repeatable.) Feature positions (Overlapping? Concentrated or evenly distributed?)

12 Conclusions The first performance evaluation of volumetric interest point detectors. In terms of R area score only, MSER is the best, followed by Hessian and DoG. Our work provides a guidance on the choice of detectors and explores the differences between texture-based and volumeteric interest points. Efficiency and types of features,are also important considerations.

13 THE END

14 References [1] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool, A comparison of affine region detectors, IJCV [2] C. Schmid, R. Mohr, and C. Bauckhage, Evaluation of interest point detectors, IJCV, 2000 [3] G. Flitton, T. Breckon, and N. M.Bouallagu, Object recognition using 3D sift in complex CT volumes, BMVC [4] J. Knopp, M. Prasad, G. Willems, R. Timofte, and L. Van Gool, Hough transform and 3D SURF for robust three dimensional classification, ECCV 2010 [5] M. Donoser and H. Bischof, 3D segmentation by maximally stable volumes (MSVs), ICPR [6] T. H. Yu, T. K. Kim, and R. Cipolla, Real-time action recognition by spatiotemporal semantic and structural forest, BMVC 2010 [7] E. Rosten, R. Porter, and T. Drummond, Faster and better: A machine learning approach to corner detection, TPAMI [8] A. Bronstein, M. Bronstein, and R. Kimmel, Numerical Geometry of Non-Rigid Shapes, 1st ed. Springer, 2008.

15 Hidden slides

16 Results: Qualitative Analysis Corner detection (e.g. Harris, FAST): Smaller corner features. More robust in low sampling density than other interest points. Derivative-based Blob Detection (e.g. SURF, Hessian, DoG): Better performance than corner-based Sensitive to sampling quality and density. Feature shapes are limited to sphere/ellipsoids. Region-based Blob Detection (e.g. MSER): Best overall performance Overlapping features of different scales. Free-form salient regions. Other Considerations Efficiency (e.g. MSER is accurate but it takes a long time to compute for 3D volumetric data, FAST is more efficient but it is less repeatable) Feature positions (surface / edge / corner / centres of blobs)?

17 Repeatability Methodology Major performance metrics: Repeatability Ratio of matching interest points within certain spatial displacements Accuracy (matching distance threshold) is a free parameter The R area score is proposed in this paper Repeatability-accuracy graph: Repeatability of an interest point under different accuracy requirements Definition of R area score: the normalized area under R-A curve Repeatability vs. Accuracy Common Measurements Repeatability Accuracy 100% 0 D Matching Threshold R area Score: = Normalized Area Under Curve Area of blue region = Area of shaded region

18 Related Work Applications Similar to their texture-based counterparts, volumetric local features has a broad range of applications in computer vision. Detection Retrieval Evaluation Methodology Major measurement: repeatability ratio (and its variants) Area of ROC curve Entropy measure Recognition Volumetric Interest Points Registration Flitton et al. [3] Detection Knopp et & al. Recognition [4] Donoser Recognition and &Registration Bischof [5] Bronstein Segmentation et al. [8] Retrieval

19 Local VS Global Shape Techniques Earlier research on shape-based representations have relied on a global feature framework. The shape of an object is characterized by a single descriptor. Global approaches Simple Low complexity Sensitive to shape deformations, occlusions, sampling noise and incomplete shapes. Local approaches More sophisicated Higher complexity, yet more powerful More robust in terms of deformations, occlusions, etc.

20 Is MSER really that good? Advantages of using MSER: High repeatability High robustness to noise (because region remains unchanged when ) Relatively accurate Small number of interest point detected Important issues to consider: Multiple features (with different shapes and scales) are detected at the same position. Hence, given the same number of features, the feature space (in other words, diversity) described by MSER is smaller than other detectors. The performance of MSER drops rapidly under heavy occlusions or incomplete sampling (missing parts / sparsely sampled data).

21 Why Volumes but not Mesh Interest point detector? MESH dataset Manually drawn meshes Surface estimation / triangulation is required for real data. Mesh is hollow, no data inside the shape. Volumetric data is more generic Can be converted from various types of data Multimodal data: videos, point clouds, meshes, 3d scanner