Improving Vision-based Topological Localization by Combining Local and Global Image Features

Transcription

1 Improving Vision-based Topological Localization by Combining Local and Global Image Features Shuai Yang and Han Wang Nanyang Technological University, Singapore

2 Introduction Self-localization is crucial for autonomous systems. Current solutions: GPS. Signal may be unavailable at times. Accuracy is also limited. 3D sensors (laser). Can be costly. Camera. Visual odometry. Visual SLAM. Appearance-based localization.

3 Introduction Appearance-based techniques use image features and a database of previously collected images with known poses. Given a query image, the position can be recovered by matching this query image against database images. Sunderhauf and Protzel, IROS, 2011 Image feature types for place representation: Localimage features (e.g., SIFT and SURF). Extracting local features consists of describing visual information from a patch or neighborhood (M M pixels) around a point of interest in the place item. Can have hundreds of features for a given image. 3

4 Introduction Appearance-based techniques use image features and a database of previously collected images with known poses. Given a query image, the position can be recovered by matching this query image against database images. Sunderhauf and Protzel, IROS, 2011 Image feature types for place representation: Global image features (e.g., GIST) incorporate statistics of the overall distribution of the visual information in the scene. Describe the entire image with a single feature vector. 4

5 Introduction Several local image feature based localization systems have been proposed [FAB-MAP, Zamir 13, Badino 11, Valgren 10, etc.] Features are local, so robust to occlusion, illumination and clutter. More discriminative. Many features can be generated for even small objects. More time consuming. Extraction, description and matching these local features is more time-consuming. Global information lost, e.g., spatial relationships. Spatial relationships between image features are important in the sense that they provide a kind of linkage information between independent image features.

6 Introduction Global image feature based systems [SeqSLAM, Murillo 13, etc] High efficiency, compact representation, suitable for large dataset. Global structure covered. Low invariance to viewpoint changes. The robot does not always follow exactly the same path on its second (live) traversal, but observes the known scenes from a different angle. Occlusion and illumination sensitive. Different feature subsets offer complementary information. Intuitively, a hybrid approach that use both global and local image features to represent locations should perform better.

7 Proposal In this work, we propose an approach that localize the mobile robot who follows a previously taken route or almost the same route. Main difference: Both global and local image features are used. Do not simply combine global feature vector with local feature vector, but consider them independently. A Bayesian framework is used to estimate the probability of the robot position as the robot moves and new observations are acquired.

8 Local and Global Features Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image Local feature: Exactlocal feature representation: accurate but has high memory and computational requirements. Quantized local feature representation: visual bag-of-word model is used. Schematic for representing an image with Bow model.

9 Local and Global Features Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image Local feature: Term frequency-inverted document frequency (TF-IDF) weighting scheme is used in our bag-of-word model. Schematic for representing an image with Bow model.

10 Local and Global Features Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image Global feature: Gist-based representation: Model the shape of a scene(dominant spatial structure). Given an input image, the gist feature is computed by convolving the image with the an oriented filter (Gabor filter) at several different orientations and scales In this work we restrict to this combination Although similar ideas can be applied to other modalities.

11 Bayesian Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image Previous Posterior Likelihood State Transition

12 Bayesian Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image

13 Bayesian Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image

14 Bayesian Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image

15 Bayesian Framework Input Query Images Compute Local for Detected Interest Points Bayesian Tracking Filter (spatial-temporal smoothing) Select Image with Highest Probability Compute Global for The Entire Image Final State Estimation: The estimated location of the mobile robot at every time step will be that with the largest probability within the set of possible locations.

16 Experiments Sequences collected by our mobile robot were used. Video captured from a Ladybug camera. The route contains a variety of environments (e.g., buildings, trees, openspace, etc). Two sequences captured in different weather conditions (one as dataset, cloudy. another one as query, sunny). DGPS as ground truth. Robot and path Challenging images

17 Experiments

18 Experiments

19 Conclusion Proposed a mobile robot localization system using both local and global image features. Bayesian framework is used to consider spatial-temporal connections. Experimental results show the improvements of the combination.

20 Questions?