Object Recognition using Visual Codebook

Transcription

1 Object Recognition using Visual Codebook Abstract: Object recognition is an important task in image processing and computer vision. This paper proposes shape context, color histogram and completed local binary pattern (CLBP) approaches and compare their results for recognizing objects in real word images. With a simple glance of an object, humans are able to tell its identity or category despite of the appearance variation but it is a challenge for machine vision systems. The commercial systems for object recognition depends on correlation based template matching but this technique is not feasible in presence of rotation, scale, illumination and variable pose. Hence it is desirable to use low level features extracted from images. The main goal is to recognize test image using visual codebook. Visual codebook is impressive technique because we are simply modeling the statistical distributions of low-level image features into a fixed-length vector in histogram space to which standard classifiers can be directly applied. Here, recognition is done by Nearest Neighbor Classifier. An experimental result on ETH-80 database of 100 objects with various poses confirms the effectiveness of given approaches. Keywords: Object Recognition, Visual Codebook, Shape Context, Completed Local Binary Pattern (CLBP), Color Histogram 1. INTRODUCTION Ms. Rupali D. Rane 1, Prof. Bharati K. Khadse 2, Prof. S. R. Suralkar 3 Object recognition is a fundamental technology that is necessary to perceive objects and to establish the spatial relationship among detected objects. In short, Object recognition is concerned with determining the identity of an object being observed in the image from a set of known labels. Object recognition is one of the most fascinating abilities that humans easily possess since childhood. With a simple glance of an object, humans are able to tell its identity or category despite of the appearance variation due to change in pose, illumination, texture, deformation, and under occlusion [1]. Furthermore, humans can easily generalize from observing a set of objects to recognizing objects that have never been seen before. For example, kids are able to generalize the concept of chair" or cup" after seeing just a few examples. But recognizing object classes in realworld images is a long standing goal in computer vision. Conceptually, this is challenging due to large appearance variations of object instances belonging to the same class. Additionally, distortions from background clutter, scale, and viewpoint variations can render appearances of even the same object instance to be vastly different. Further challenges arise from interclass similarity in which instances from different classes can appear very similar. Hence there is necessity of object recognition systems. In general, an object recognition system contains two basic stages: feature extraction and feature matching [2]. For feature extraction, the shape context, Completed Local Binary Pattern (CLBP) and Color histogram is widely used due to its scale, rotation and illumination invariance. During the feature matching stage, each extracted feature in the video frame requires a search of its nearest neighbors among a great number of object features stored in a visual codebook. 1.1 Texture Review: Texture is an important property that represents the surface and structure of an Image. It is a repetition of an element or pattern on a surface. Tuceryan and Jain (1993) classified texture analysis methods into four categories: statistical, geometrical, model-based and signal processing [3]. The most traditional way to analyze spatial distribution of gray levels of an image is by statistical analysis. It can be further classified into firstorder, second-order and higher-order statistics depend on pixels. Geometrical methods analyze textures by texture elements or primitives. This analysis is made considering the geometrical properties of the primitives, like size, shape, area, and length. Model-based methods rely on construction of generative model that can be used to synthesize and describe texture. Signal processing methods characterizes textures by applying filters over the image. Both spatial domain and frequency domain filters can be used. This method analyzes the frequency content of the image. Early methods use statistical analysis to characterize the stochastic properties of spatial distribution of gray levels [4]. Recently, Ojala [5] has proposed local binary pattern (LBP) that provides a simple and efficient approach to gray scale and rotation invariant texture classification. The LBP method extracts rotation invariant texture features from a local region by thresholding the gray values of the P-neighborhood pixels relative to the corresponding value of the central pixel, which is computationally efficient and robust to monotonic illumination variation. Although LBP provides a theoretically simple, yet efficient approach to texture Volume 2, Issue 3 May June 2013 Page 328

2 classification, it has some limitations. Firstly, it shows poor performance in the presence of random noise. Secondly, LBP method only considers the sign of the difference between two gray values and thus discards the magnitude of the difference which is very important texture information. Hafiane [6] has proposed median binary pattern (MBP) that provides robustness against noise as texture primitives are obtained by thresholding a 3 3 neighborhood against the local median. More recently, Guo [7] has proposed LBP variance (LBPV) to characterize the local contrast information into the onedimensional LBP histogram. 1.2 Shape Review: Shape is used to describe image content. Shape can be defined as characteristic surface configuration of an object [8]. There are two methods for shape representation and description: contour-based methods and region-based methods [9]. Contour based shape representation only uses the outer boundary/contour of the shape. Regionbased shape representation uses the entire shape region by describing the considered region using its internal characteristics; i.e. the pixels contained in that region. this are further divided into structural approaches and global approaches. This further division is based on whether the shape is represented as a whole or represented by segments/sections. According to literature survey of shape matching, there are two approaches [10]: (1) feature-based methods and (2) brightness-based methods Feature-based approaches use spatial arrangements of extracted features such as edges or junctions. It is having been directly matched using dynamic programming e.g. [11]. Since silhouettes are used as shape descriptors for general objects, other approaches treat the shape as a set of points in the 2D image, extracted using, say, an edge detector. Amit and Geman [12] recognize objects with spatial arrangement of key point. Brightness-based methods work with the intensity of the pixels and in that image itself act like feature descriptor. One can use brightness information in one of two frameworks. First category includes the methods that find correspondences using grayscale values. Yuille[13] presents a very flexible approach in that invariance to certain kinds of transformations can be built into the measure of model similarity, but it suffers from the need for human-designed templates and the sensitivity to initialization when searching via gradient descent. Lades [14] use elastic graph matching, an approach that involves both geometry and photometric features in the form of local descriptors based on Gaussian derivative jets. The second category includes those methods Build classifiers without explicitly finding correspondences. Such approach relies on a learning algorithm having enough examples to acquire the appropriate invariance. Murase and Nayar applied these ideas to 3D object recognition system [15]. 1.3 Color Review: The most intuitive information that can be extracted from images for comparison is the color characteristics of an image. Literature presents three main approaches for color analysis. The global approach considers the image color information globally. As no Partitioning or preprocessing stage is applied to the image during features extraction. The fixed-size regions approach divides an image into cells of fixed size and extracts color information from each cell separately. The descriptors from this approach encode more spatial information, at the cost of usually generating larger feature vectors. The segmentation-based approach divides an image in regions that may differ in size and quantity from one image to another. A number of algorithms have been developed since the late 1980s that use color information extracted from images for retrievals [16]. A most basic form of color retrieval involves specifying color values that can be searched for in images from a database. Indeed, Google s image organization and editing software, Picasa 3.0, allows users to use an experimental tool to search for certain colors in images. Even this basic method presents challenges in implementation due to the different manners in which computers and human see colors. Computers represent all visible colors with a combination of some set of base color components, generally Red, Green and Blue (RGB). Thus, images perceived by a computer to contain a large component of red may not necessarily appear reddish as perceived by a human eye. Indeed, Picasa s experimental tool suffers from this and returns certain unintuitive results. Other image retrieval methodologies rely on specifying more precisely the nature of the color that is to be retrieved. These methodologies range from improving upon the nature of the color query to associating spatial conditions to the color information. For example, when searching for country flags, one could specify a search for different regions of color and their positions relative to each other [16]. 2 OBJECT RECOGNITION SYSTEM A Block Diagram of proposed object recognition system is shown in Figure1. Figure 1 Block Diagram of Proposed Object Recognition System Volume 2, Issue 3 May June 2013 Page 329

3 During input stage, images are given to region descriptor which processes images for computing the selected features such as shape, color, texture to represent an image. This is called as indexation. Indexation assigns set of identifying descriptors or indices to each image which will be used by system in classifying phase to retrieve relevant images and reject others. These indices are stored in visual codebook, ideally are designed for efficient retrieval. Different features (color, shape, texture.) express different aspects of image contents. Visual codebook contains visual words produced by descriptors. The key role of a visual codebook is to provide a way to map the low-level features into a fixedlength vector in histogram space to which standard classifiers can be directly applied. The quality of the codebook model is determined by discriminative power of visual codebook, whereas the size of the codebook controls the complexity of the model. Its size determines how well a histogram approximates the true distribution of local descriptors in an image or video. When an image query is posed, its selected features are computed using the same procedures applied to input images. Image retrieval is then performed by a Nearest Neighbor classifier, which compares the features of the query image with those of the stored images in the codebook. The recognition or retrieval takes place according to selected metric/feature. 3 SHAPE CONTEXT The shape context descriptor describes the coarse distribution of the rest of the shape with respect to a given point on the shape. For shape matching of any two objects following steps are considered: 1) The basic idea is to pick n points on the contours of known shape and unknown shape using canny edge detector. 2) Compute the shape context of each point found in step 1. For each point p i. on the shape consider n-1 vectors by connecting it to other points. The set of all these vectors represent rich description of shape. For the point p i, the coarse histogram of the relative coordinates of the remaining n 1 points, This histogram is the shape context of p i. 3) Match each point from the known shape to a point on an unknown shape. 4) Consider, a point on the first shape and a point on the second shape. Let, ) denote the cost of matching these two points. Where, ) = (k) denote K-bin normalized histogram at resp. 5) Finding the matching that minimizes total cost Now, one-to-one matching p i that matches each point p i on shape 1 and on shape 2 that minimizes the total cost of matching, H(π) = is needed. This can be done in O ( time using the Hungarian method [17], although there are more efficient algorithms. For robust handling, one can add "dummy" nodes that have a constant but reasonably large cost of matching to the cost matrix. This would cause the matching algorithm to match outliers to a "dummy" if there is no real match. 6) Use Nearest Neighbor classifier to identify unknown Object. 4 COMPLETED LOCAL BINARY PATTERN Figure 2 Central pixel, its P circularly and evenly spaced neighbors with radius R. Referring to Figure 2, given a central pixel and its P circularly and evenly spaced neighbors,(p=0,1,..,p- 1), we can simply calculate the difference between and as.the local difference vector [,,.., ]characterizes the image local structure at. Because the central gray level is removed, [,,.., ]is robust to illumination changes and they are more efficient than the original image in pattern matching. can be further decomposed into two components., And = sign ( ), = Where = Where is the sign of, is the magnitude of. With Eq. of is transformed into a sign vector [,,.., ]and a magnitude vector [,,.., ]. We call the eq. of as the Local Difference Sign magnitude Transform (LDSMT). Figure 3 shows an example. (a) is the original 3x3 local structure with central pixel being 24. The difference vector (b) is [3, 12, -9, -16, -14, -11, 54, 72]. After LDSMT, the sign vector (c) is [1, 1, -1, -1, -1, -1, 1, 1] and the magnitude vector (d) is [3, 12, 9, 16, 14, 11, 54, 72]. It is clearly seen that the original LBP uses only the sign vector to code the local pattern, as an 8-bit string ( -1 is coded as 0 ). Volume 2, Issue 3 May June 2013 Page 330

4 pixels for each quantized bin. In color histogram discrimination power depends on no. of bins. A color histogram H for a given image is defined as a vector H H[0],H[1],...,H[i],...H[N] Where, i represent the color in color histogram and H[i] represent the number of pixels of RGB color i in the image, and N is the number of bins used in color histogram. Figure 3 (a) 3x3sample block; (b) the local differences (c) the sign and (d)magnitude components Sign components contain more value of the local difference vector information than magnitude components. Meanwhile, the magnitude components in the proper using of textures can also provide identification information. Further studies have shown that center gray level contains some useful texture information. If putting the center, sign and magnitude components together constitute an integrated component of the LBP algorithm, it will be very effective in extracting image texture features. Figure 4 shows CLBP framework to describe the texture features of the original image. Ultimately, it establishes the histogram of CLBP and uses nearest neighbor classifier, for texture classification. The magnitude components can use a similar sign components coding strategy to transform the model into what we need. The CLBP_M operator was defined the following:, t(x, c) Where, c is a threshold to be determined adaptively. 5 COLOR HISTOGRAM Figure 4 Framework of CLBP The color histogram is a representation of the distribution of colors in an image. Color histogram do not relate spatial information with pixels of given color and are invariant to rotation, occlusion, changes in camera view point and translation of objects in the image. It is mostly used to compare images. Since any pixel in the RGB image can be described by three components in a certain color space ( red, green, and blue components in RGB space). Histogram is the distribution of the number of 6. EXPERIMENTAL RESULTS The proposed technique has been applied on ETH-80 database of 100 objects with variable poses. An object image of size is considered for evaluation. By edge detection and segmentation of given test image obtain a clear silhouette boundary of the object in the image. Figure 5 shows test image and extracted image of particular object. Then this boundary points are sampled to n points that are used for calculating shape context. Figure 5 Region description image Table 1 show the results and comparison of three methods proposed in this paper with Nearest Neighbor Classifier. The result confirms that shape context descriptor is superior than completed local binary pattern (CLBP) and color histogram descriptors in terms of % accuracy. Table 1 Result of ETH-80 Database ETH-80 Database Methods Test % Accuracy Images Shape Context Color Histogram CLBP CONCLUSION This framework was motivated by desire to compare shape, color and texture descriptors for Object recognition. This work implements Shape Context, Completed Local Binary Pattern and Color Histogram approaches to categorize different classes of objects. Shape Context gives 93% accuracy on ETH-80 database which is better than other approaches. The results obtained in this work can be improved further by using better feature extractors and by using the combination of Volume 2, Issue 3 May June 2013 Page 331

5 shape and color descriptor or shape and texture descriptor. References [1] Mikolajczyk,K., Leibe, B., & Schiele, B., Multiple object class detection with a generative model. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp , [2] Yu-Chi Su, Keng-Yen Huang, A 52 Mw Full Hd 160-Degree Object Viewpoint Recognition Soc With Visual Vocabulary Processor For Wearable Vision Applications IEEE Journal Of Solid-State Circuits, Vol. 47, No. 4, Pp ,April [3] A. Materka, M. Strzelecki, Texture Analysis Methods A Review, Brussels [4] U.S.N. Raju, A.S. Kumar, B. Mahesh and B.E. Reddy, Texture Classification with High Order Local Pattern Descriptor: Local Derivative Pattern, Global Journal of Computer Science and Technology, vol. 10, issue 8, pp , [5] T. Ojala, M. Pietikainen and T. Maenpaa, Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp , [6] A. Hafiane, G. Seetharaman, and B. Zavidovique, Median binary pattern for textures classification, Image Analysis and Recognition, pp , [7] Z. Guo, L. Zhang and D. Zhang, Rotation invariant texture classification using LBP variance (LBPV) with global matching, Pattern Recognition, pp ,2010 [8] S. Belongie, J. Malik, and J. Puzicha, Matching Shapes Proc. Eighth Int'l Conf. Computer Vision, pp , July [9] Dengsheng Zhang, Guojun Lu, Review of shape representation and description techniques, Pattern Recognition,2004 [10] R. C. Veltkamp and M. Hagedoorn, State of the art in shape matching, Technical Report UU-CS , Utrecht, [11] Y. Gdalyahu and D. Weinshall., Flexible syntactic matching of curves and its application to automatic hierarchical classification of silhouettes, IEEE Trans. PAMI, 21(12): , [12] Amit, Geman, and Wilder., Joint induction of shape features and tree classifiers, IEEE Trans. Pp , November [13]A. Yuille, Deformable Templates for Face Recognition, J. Cognitive Neuroscience, vol. 3, no. 1, pp , [14] M. Lades, C. Vorbuggen, J. Buhmann, J. Lange, Distortion Invariant Object Recognition in the Dynamic Link Architecture,º IEEE Trans. Computers, vol. 42, no. 3, pp , Mar [15] H. Murase and S. Nayar, Visual Learning and Recognition of 3-D Objects from appearance, Computer Vision, vol. 14, no. 1, pp. 5-24, Jan [16] Smith, John R, and Shi-Fu Chang. Tools and Techniques for Color Retrieval, [17] C. Papadimitriou and K. Stieglitz, Combinatorial Optimization- Algorithms and Complexity, 1982 [18] Digital Image Processing by Gonzalez Woods and Eddins. Volume 2, Issue 3 May June 2013 Page 332