1 CHAPTER 1 INTRODUCTION Table of Contents Page No. 1 INTRODUCTION 1.1 Problem overview 2 1.2 Research objective 3 1.3 Thesis outline 7
2 1. INTRODUCTION 1.1 PROBLEM OVERVIEW The process of mapping and aligning corresponding points among all images taken from the same scene at different time intervals, angle of view and/or sensors is called image registration. Once feature correspondences are established between images and the geometric alignment has been performed. In order to represent several views of scene into a wide angle image (i.e., both geometrically and photometrically consistent), this process is known as image mosaicking. For a long time, image registration [1,2,3,4] and mosaicking are two leading research themes in the image processing. This image registration is very useful in medical applications, remote sensing applications and object recognition, etc. The image registration is always a challenging task when there is large degree of variability of the input data. This change in visual information data occurred due to the image distortions during capturing process. When the multiple images captured at different orientations may cause the geometrical distortions such as scaling, rotations, projective transformations. The captured images may also distort when photometric changes due to different acquisition modalities and lighting conditions. Figure 1.1 is an example of an
3 image pairs that have been registered using the algorithms that will be described and analyzed in the next chapters. Large amount of effort is made to construct efficient algorithms for solving different aspects of image registration and mosaicking problem. This thesis is focused on overcoming the obstacles and several open questions that need to be answered. In the next section we will discuss the motivations that lead us to tackle some of these obstacles and to answer some of these questions. 1.2 RESEARCH OBJECTIVE The research object is to study an image registration system that must be able to provide accurate and realistic results and self asses the quality of its output with minimal human intervention and reduced computational resources. In this method there are two possibilities to implement in real scenarios. They are 1. Automation 2. Manual In manual process the human operators are the key players to select required features. In an automatic registration, a RANdom Sample And Consensus (RANSAC) algorithm generally used to select and matching corresponding feature points between the images. To improve the efficiency of this process, a good feature point detector is used to detect invariable/stable feature pairs in the images.
4 It is observed that, all cameras may not have enough field of view to capture the exact scene in single shot. Typically compact camera has a Field Of View (FOV) of 50x35 degrees, whereas the human visual system has a FOV of around 200x135 degrees. The mosaic/panorama image enhances the field of view in computer vision and while capturing these images, it suffers from certain limitations. These limitations will degrade the system performance due to the image blurring problems. This problem occurs when both camera and object move in the same direction. Our research objective is to analyze the limitations of cameras in different circumstances. This problem can be overcome by using image mosaic system. Image mosaic is an effective process with minimum distortions from the original image. In this process we combine two or more different images and form the new image with different angles with little distortion. The resultant image will give better clarity when compared to individual images (multiple images captured at different angles). These mosaic implementations play an active role in different research areas like image processing, video processing and computer graphics. In the image mosaic system, image registration technique is used to generate mosaic image. We will focus our attention in this dissertation on feature-based approaches. The system first extracts a set of features from the images that are to be registered. Once the features are extracted, each feature is assigned with a distinctive label
5 to establish tentative matches. These matches are further used to estimate geometric model to describe the transformation between the images. Finally images are fused together to produce a coherent mosaic. Fig. 1.1: Some examples of image pairs that have been registered and mosaicked using the methods that will be described in following chapters. First row: pair of Ellora caves images. Second row: outdoor 3D scene image pairs. Third row: a tiger image pair capture with a consumer camera.
6 The basic models used in the registration flow are shown in figure 1.2. Functions of each block are as follows. Fig. 1.2: Image registration flow. Pre-processing: It is an improvement of the image data that suppresses unwanted distortions or enhances some important image features for further processing. Feature detection: Salient and distinctive objects (closedboundary regions, edges, contours, line intersections, corners, etc.) are manually or, preferably, automatically detected. For further processing, these features can be represented by their point representatives (centres of gravity, line endings, distinctive points), which are called Control Points (CP) in the literature. Feature matching: In this step, the correspondence between the features detected in the sensed image and those detected in
7 the reference image is established. Various feature descriptors and similarity measures along with spatial relationships among the features are used for that purpose. Transform model estimation: The parameters of the mapping functions are computed by means of the established feature correspondence. The type and parameters of the so-called mapping functions, aligning the sensed image with the reference image, are estimated. Image resampling and transformation: The sensed image is transformed by means of the mapping functions. Image values in non-integer coordinates are computed by the appropriate interpolation technique. 1.3 THESIS OUTLINE This thesis contains 8 chapters. Following the introduction, the rest of the thesis is organized as follows: Chapter 2 explains an overview of image features like, edges, corners, and blobs. Image registration system includes classification of image registration as an area based and feature based methods. Finally the image mosaicing can be classified into direct methods and feature based methods. Chapter 3 explores an overview of different feature point detectors such a Kitchen-Rosenfeld, Harris, Kanade-Lucas-Tomasi (KLT), Smallest Univalue Segment Assimilating Nucleus (SUSAN), Features from
8 Accelerated Segment Test (FAST), and proposed method using steerable filters. Chapter 4 gives an overview of RANSAC algorithm. This algorithm used to classify the interest points as an inliers and outliers. Chapter 5 expresses how Homography can be used to estimate the transformation model between the corresponding point-to-point maps between the images. It also provides the estimation of image geometry. Chapter 6 shows the experimental result analysis of proposed method and other four Harris, KLT, SUSAN and FAST corner detectors performs are measured in terms of Corner Consistency Number (CCN), Accuracy, Repeatability and Matching scores. Chapter 7 presents application of proposed corner detector as image matching, image registration and mosaic system. Chapter 8 conveys conclusion and future works.