A Robust Feature Descriptor: Signed LBP

Size: px

Start display at page:

Download "A Robust Feature Descriptor: Signed LBP"

Claud Ramsey
5 years ago
Views:

1 36 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 A Robust Feature Descriptor: Signed LBP Chu-Sing Yang, Yung-Hsian Yang * Department of Electrical Engineering, National Cheng Kung University, No., University Road, Tainan City 70, Taiwan. csyang@mail.ee.ncku.edu.tw * Corresponding Author: Tel.: ; q @mail.ncku.edu.tw Abstract - We improve a texture descriptor, Local Binary Feature (LBP), called Signed Local Binary Pattern (SLBP) which is more robust in rotation and scale. In this paper we will introduce how to obtain more information in LBP, which with signed bit, and make feature more stable with mean of local area instead of single pixel s intensity. Finally, to reach more robust in scale by difference smooth factor and implement by Integral Images to reduce computation cost. A pixel can perform different texture information in each scale, thus we select meaningful edge type in smallest scale. And signed bit is adding by current center area is greater or less then whole neighbor area. Then we implement cell and block concept from Histogram of Gradient to test the character recognition. In result part we prove SLBP have more robust than LBP in rotation, scale by texture image and natural scene image. The last part is testing the performance of recognition rate in IIIT5K database. Keywords: Local Binary Pattern, Texture Descriptor, Integral Image, Histogram. Introduction Recently, number of digital image increased very fast by street camera, portable camera device, cell-phone, and even google street view with some flaws such like low resolution, blurred with hand shake, or low luminance. These flaws result it s hard to extract interest image object from image database, but the information in images is usually useful in many field such like surveillance system, image search engine, auto license plate recognition (ALPR), text recognition, or human and face detection etc., which can help our life to be more convenience. And this information is too large to analysis by human resource. Thus the image object recognition will play an important position in intelligence system in feature. By Bag-of-Words [] model is more popular in object recognition at current computer vision. Its performance is based on descriptor s robustness to find the separate visual words and statistic these word into an image object. Therefore, stronger descriptor had a prominent part in this model. In general, descriptor have three parts to overcome: luminance change, rotation, and scale. In original Local Binary Pattern, [2], has been proven capability in gray scale and rotation invariant in texture recognition. Based on LBP, we improve the LBP kernel with sign bit and remove redundancy information in original LBP bring to more texture information in pixel level. Then we search the scale to find the significant edge type at current pixel. Like HOG [5] we build two level histogram information with cell and block to statistic edge information into high dimension to recognition text object. In the remaining part, related work will explain original LBP, Integral image, histogram and relative paper s technology we sited. Chapter 3 will expound our method in detail, and experimental results in chapter 4 with compared between original LBP and SLBP in rotation, scale, gray scale and text recognition with IIIT5K [6] database. 2 Related work In visual word and bag of word model were most popular in image object recognition and matching, how to descript the visual word efficiently and high accuracy become more important to achieve robust and invariant at many kinds. Image descriptor can classify into three part roughly, color descriptor, texture descriptor, and intensity descriptor. Color descriptor were out of favor because unexpected luminance will make information lost in color domain, thus major methods trend to texture base and intensity base method. 2. Histogram of Gradient Histogram of Gradient (HOG) is robust descriptor to static eadges. HOG count the edge s gradient and magnitude by an area called cell and histogram into angle bins, which number of bin can adjust by case. Then construct these cells into overlapped block as feature. Each block were normalized to overcome the unbalanced intensity. HOG feature size will be decide by number of angle s bin, cell size, block size, block overlap, and image size. In our experiment accuracy of HOG in proportion to feature number. And HOG have high recognition rate in text recognition. Although HOG can extract more detail of edge features, but it still have some drawbacks to be overcome, such like how to choose edge operator, smooth operator size [7], descriptor blocks (C-HOG), or use image pyramid to achieve scale invariant, which make high computation cost. 2.2 Haar-like Feature and Integral Image In viola and Jones s [8] work propose a Haar-like feature to detect the different intensity in between rectangles. For example in face detection, nose and cheek have significant

2 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 37 intensity difference. And then implement by Integral Image (II), which is a fast way to calculate sum of a rectangle by four operator after II was construct. Integral image was propose by Frank Crow at 984 and widely used in invariant feature extraction [9]. II can define as equation. The value of means the sum of image from lefttop to point, thus to calculate a sum of Specific area sum by. D C Sum of all Image Figure. Schematic to show Integral Images B A Instead of the scale problem the method have a large feature table in learning image. And then they select the most significant features to filter out the none-face subimages by using cascade Adaboost [0] classifier, which combined by many weak classifier into a strong classifier to achieve detection speed. The error rate will decrease when strong classifier is increase. 2.3 Region base methods The bottle neck of Region base match is how to extract complete regions after view angle and luminance change. MSER [] is an efficiency method to extract the region object, in Per-Eric s work performs the performance will effect by blur. In Chen s [2] work propose a edge enhancement method to divide MSER Regions by edge detection result. In 2.4 Rotation invariant feature and descriptor BRISK [3] proposed an idea similar to DAISY [4], which is pre-calculate smooth image with pyramid kernel to reduce redundant computing cost, and also use the binary feature to fit the interest points. Then the trend start to find the pairs from coarse-to-fine from the bigger Gaussian kernel at periphery and smaller kernel when closer to center to extract more detail. With this trend, inspired us to develop multi-scale in Local binary pattern. ORB (Oriented FAST and Rotated BRIEF) [5] is a fast robust local feature detector and It is based on the FAST [6] keypoint detector and the visual descriptor BRIEF. Its aim is to provide a fast and efficient alternative to SIFT. 2.5 Local Binary Pattern (LBP) LBP feature is compare each pixel with neighbor pixels to obtain a circle binary feature shown as figure 2, and equation shown as equation 2. Where is gray value at current pixel and is gray value of its neighbors and binary feature express by compare with every with equation 3. Notice that P is pair number and R is the radius from central pixel. Features present the relationship between central pixel and neighbor and express by binary feature with 36 kinds of combination. In rotation invariant version of LBP was achieved by circular bitwise right shift (ROR). Then in Ojala s [4] work, mark U as transition between 0 and. By Ojala s [4] experiment, LBP feature have significant meaning when so they only count the first P+ combination and group remind feature into same category as P+2. Mark that the features with is rare and can t express local texture well or we can called it is disorganized. Thus the number of combination of LBP feature is reduced to P+2, where is 0 bins in. And author also present stronger descriptor by extent P and R 3 Main Method In this section we will introduce Signed Local Binary Pattern (SLBP), which have better performance keep the same feature bin. We will point out the difference between SLBP and original LBP step by step. 3. LBP Base on Integral Image In our proposed method, feature is computing base on Integral Image to speed up. So at first, we defined the as Integral Image and is mean value of a sum of area as equation = (000)2 ROR (000) Figure 2. Schematic diagram of

38 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 scale and keep zero when. By integral image, the computation cost of each scale is same. Schematic diagram is shown as Figure 4.

Thus first LBP feature can present as equation 5. Briefly, means average value in block instead of to build LBP in first scale and means original image.

3 38 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 scale and keep zero when. By integral image, the computation cost of each scale is same. Schematic diagram is shown as Figure 4. Parameter s is scale for smooth factor as with s = [, 2 N] to present each scale. The final scale can be adjust by current situation.we mark the as original image. Thus first LBP feature can present as equation 5. Briefly, means average value in block instead of to build LBP in first scale and means original image. (5) s This idea was presented by MLBP [7], and we also agree with this approach is more robust than original LBP. Thus we also use Integral Image to obtain first scale LBP by compare with mean value of block. Then we also use the ROR operator to achieve rotation invariant. 3.2 Scale invariant As mention before the feature is meaningless when, these pixels maybe were electronic noise at current scale. For Example, shows in Figure 3, we extract the LBP feature at Figure 4. First and third images show current pixel is in edge. Second and fourth images show current pixel is out of edge. Then the LBP feature can extract from each scale with difference meaning. Without loss of generality, we select the first none-zero value in scale space as LPB feature to avoid too much blur to interfere correct edge information. So the LPB with scale invariant ( ) feature can express as equation , s= , s= And in our experiment, scale s were set to 3 can achieved every pixel have significant edge type. Figure 3. Image shows the difference scale can extract different feature yellow box with (0000) 2 after ROR and this information is meaningless when we histogram this into a bin in P+2. But in next scale, blue box, the pixel can extract the useful feature as (000) 2 after ROR process. So in our algorithm, we extract P+ feature bins from ( ) 2 to () 2 at current 3.3 Signed Bit Last step is to decide the sign bit, which can descript the center information from LBP. Since we apply the propose method from MLBP, which is use the mean value to instead of center intensity to achieve more stable. It s makes the intensity of center area can be an additional information for current pixels. Thus we modified the LBP equation into equation 8. Figure 5. All edge type in SLPB, which from black spot to white spot.

Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 39 Figure 6. Schematic diagram for integral image apply to each scale.

It mark the current pixels is in edge or out of edge. In our experiments use sign bit to instead P+2( ) type can improve significant recognition rate in text recognition.

So after features extracted in pixel level, we use the concept in Histogram of gradient to divide pixel into cell; and overlapped cells into block to build high dimension feature vector.

4 Result and experiment In generally, we will test the rotation, scale and luminance first in this section.

We use three images, which is Brick Wall, Carpet, and Boat to test invariant and compare between and original. And we believe the result will be much better when implement higher P and R in SLBP.

Rotation Invariant Test At first, we test rotation invariant by rotated image with 360 degree and 0 bins in both LBP and SLBP.

4 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 39 Figure 6. Schematic diagram for integral image apply to each scale. After add sign bit, the SLBP feature bin will resume into 0 bins, but this sign bit can express the edge type more specifically. The main physical meaning is showing as Figure 5. It mark the current pixels is in edge or out of edge. In our experiments use sign bit to instead P+2( ) type can improve significant recognition rate in text recognition. And total combination of edge type shows as Figure 6. The goal of the SLBP, we proposed, is to extract the useful edge information form text. So after features extracted in pixel level, we use the concept in Histogram of gradient to divide pixel into cell; and overlapped cells into block to build high dimension feature vector. After this step we expect similar recognition rate in text, but reverse is true, the performance of SLBP or LBP for text recognition is disappointed; we will show the result in next section. 4 Result and experiment In generally, we will test the rotation, scale and luminance first in this section. And the second part is to apply in character recognition by IIIT5K with simple method: k-nearest neighbor (KNN). We use three images, which is Brick Wall, Carpet, and Boat to test invariant and compare between and original. And we believe the result will be much better when implement higher P and R in SLBP. But we remind high order SLPB in our feature work because complete SLBP need more detailed reflections and we will implement on CUDA architecture to achieve real time feature descriptor extraction. 4. Rotation Invariant Test At first, we test rotation invariant by rotated image with 360 degree and 0 bins in both LBP and SLBP. First row in figure 7 shows original images and second row shows rotated images respectively. Results shows as Figure 8, 9, and 0 in squared error with none rotated images. And we can observe obviously four peaks, which is caused by discrete digital image and without Gaussian weight. This phenomenon can eliminate by high order P or weighted by Gaussian. The result shows the squared error. Form the seriese of results, SLBP has proved have lower error than the original LBP. We used exist original LBP function build in Matlab and set the parameters same as SLBP to obtain fair comparison result. In rotation test, SLBP have lower square error in each image. It is because we use the mean value instead of single pixel and search the scale space to find meaningful edge at each pixel. Figure 8. Square error in 360 degree rotation with Brick Wall. Figure 9. Square error in 360 degree rotation with Carpet. Figure 7. First row shows test images, and second row shows rotated test

5 320 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 Figure 0. Square error in 360 degree rotation with Carpet 4.2 Scale Invariant Test In scale test, we scaled image from scale factor 0.5 to.5 and also calculate squared error as Figure, 2 and 3. Result shows SLBP is more robust in scale than original LBP and squared error is less than 0.5 when images size was scaled by 0.5. Searching scale space to find correct edge type make the SLBP have more scale invariant. Figure 3. Square error in scale test with Boat. Figure 4. Square error in gray scale test with Brick Wall. Figure. Square error in scale test with Brick Wall. Figure 5. Square error in gray scale test with Carpet. Figure 2. Square error in scale test with Carpet. Figure 6. Square error in gray scale test with Boat.

6 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV' Luminance Invariant Test And in luminance test, we didn t actually take pictures in each luminance. So we used alternative solution, which scale the gray value of original images from 0.2 to to simulate different luminance. The results show in figure 4, 5, and 6. As our expected, LBP and SLBP have similar performance in gray scale. But LBP s gray scale invariant has been proved in [4] s work, thus compression does not make much sense. 4.4 Character Recognition Test We test the IIIT5K character database Last, Each character in IIIT5K training database was extracted and normalized these images size into pixel 2 to build KNN model. Which KNN classifier is set K = 63 for 0-9, a-z, up-case, and none-character. Parameters in LBP and SLBP is setup by, and 2 cell size, which is 3 and 9, to descript text texture. And in test part, we also crop the image from test database then normalize the image size as test dataset. Although KNN is a simple classifier, but it can test descriptor s performance quickly and easy to implement. The result shows as Table. In this paper we only compare original LBP and SLBP by simple method, so recognition rate is low as our expected without any particular algorithm. On the other hand, we thought LBP feature is not enough capability in text recognition. The same method in HOG performed much better than LBP and SLBP. But we will keep improve LBP feature to fit text recognition. In same situation, HOG with 4 cell and 2 overlap block s recognition rate is 84.4%. This result allows us to doubt LBP is a good repetitive texture descriptor, but not a good solution in character recognition. Table. Compare the Character Recognition Performance with LBP, SLBP, and blocks SLBP Descriptor Number of bins Recognition rate with cell % with % with % cell with % cell with % cell, 4 blcoks with cell, 4 blcoks % 5 Conclusions In this paper, we improve SLBP which is more robust than original LBP in rotation, scale and similar performance in gray scale. SLBP remove redundancy edge type and fetch more useful information by scale up in scale space and add sign bit to make pixels have correct edge type and more information. Although we only compared with and but result can be foreseeable with significant improvement when SLBP implement in higher P and R. In computation time, SLBP cost N times than original LBP when N scale space, in this paper N = 3. But still keep small time complex by LBP originally is a fast and simple algorithm. This paper is additional reward when we research for more robust feature and descriptor in text extraction and recognition in real scene. Thus, to complete full SLBP in different P and R and apply it into text extraction is our feature work. 6 References [] J. S. Zisserman, «Efficient visual search of videos cast as text retrieval,» IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, p , February [2] D. He et L. Wang, «Texture Unit, Texture Spectrum, And Texture Analysis,» Geoscience and Remote Sensing, IEEE Transactions on, pp , 990. [3] L. Wang et D. He, «Texture Classification Using Texture Spectrum,» Pattern Recognition, pp , 990. [4] T. Ojala, l. Pietikainen et T. Maenpaa, «Multiresolution Gray Scale and Rotation Invariant Texture Classification With Local Binary Patterns,» IEEE Transactions on Pattern Analysis and Machine Intelligence, pp , July [5] N. Dalal et B. Triggs, «Histograms of Oriented Gradients for Human Detection,» IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p , June [6] A. Mishra, K. Alahari et C. Jawahar, «Scene text recognitionition using higher order language priors,» chez Proceedings of the British Machine Vision Conference, [7] N. He, J. Cao et L. Song, «Scale Space Histogram of Oriented Gradients for Human Detection,» chez International Symposium on Information Science and Engieering, [8] P. Viola et M. Jones, «Rapid Object Detection using a Boosted Cascade of Simple Features,» chez COMPUTER VISION AND PATTERN RECOGNITION 200, 200. [9] H. Bay, A. Ess, T. Tuytelaars et L. V. Gool, «Speeded Up Robust Features,» Computer Vision and Image Understanding, p , June [0] Y. Freund et R. E. Schapire., «A decision-theoretic generalization of on-line learning and an application to boosting,» chez Computational Learning Theory: Eurocolt, Springer-Verlag, 995. [] P.-E. Forssen et D. Lowe, «Shape Descriptors for Maximally Stable Extremal Regions,» chez ICCV, [2] H. Chen, S. S. Tsai, G. Schroth, D. M. Chen, R.

7 322 Int'l Conf. IP, Comp. Vision, and Pattern Recognition IPCV'6 Grzeszczuk et B. Girod, «Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions,» chez 8th IEEE International Conference on Image Processing, Brussels, 20. [3] S. Leutenegger, M. Chli et R. Y. Siegwart, «BRISK: Binary Robust Invariant Scalable Keypoints,» chez ICCV, 20. [4] V. L. a. P. F. E. Tola, «Daisy: An efficient dense descriptor applied to wide-baseline stereo,» Pattern Analysis and Machine Intelligence, IEEE Transactions on, pp , [5] E. Rublee, V. Rabaud, K. Konolige et G. Bradski, «ORB: an efficient alternative to SIFT of SURF,» IEEE International Conference on Computer Vision, pp , 20. [6] E. Rosten et T. Drummond, «Machine learning for highspeed speed corner detection,» chez In European Conference on Computer Vision, [7] B. O Connor et K. Roy, «Facial Recognition using Modified Local Binary Pattern and Random Forest,» International Journal of Artificial Intelligence & Applications (IJAIA), pp , November 203.

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image SURF CSED441:Introduction to Computer Vision (2015S) Lecture6: SURF and HOG Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Speed Up Robust Features (SURF) Simplified version of SIFT Faster computation but