Detection and Recognition of Text from Image using Contrast and Edge Enhanced MSER Segmentation and OCR

Transcription

1 Detection and Recognition of Text from Image using Contrast and Edge Enhanced MSER Segmentation and OCR Asit Kumar M.Tech Scholar Department of Electronic & Communication OIST, Bhopal Abstract Text detection and recognition in traffic scene images or natural images has applications in computer vision systems like registration number plate detection, automatic traffic sign detection, image retrieval and help for visually impaired people. Scene text, however, has complicated background, blur image, partly occluded text, variations in font-styles, image noise and ranging illumination. Hence scene text recognition could be a difficult computer vision problem. This work addresses the matter of dictionary driven end-to-end scene text recognition, which is divided into a text detection drawback and a text recognition drawback. For such reasons, an enhanced algorithm is proposed in which image is preprocessed before detection phase. This is done using noise removal technique, i.e. Lucy- Richardson algorithm. After noise removal, text region detection phase starts with contrast enhanced edge enhanced MSER region detection technique is used there after morphological segmentation is used to segment text region in the image. After detection phase recognition phase starts in which text candidates are filtered using geometric filtration using properties such as aspect ratio, eccentricity, solidicity, etc. Then Bounding box technique is used to identify letter candidates and form word out of them. Finally, Optical Character Recognition (OCR) tool is used to extract text out of image. The system presented outperforms state of the art methods on the dataset of the traffic text sign data that were obtained from Jaguar Land Rover Research. The results, in the text-detection phase (precision = 0.96 and Fmeasure = 0.93) and text recognition/extraction phase (precision = 0.94 and Fmeasure = 0.95) tasks, shows the enhanced result as compared to the existing techniques. Keywords Text Detection, Lucy-Richardson algorithm, Contrast and Edge Enhanced Maximally stable extremal region (MSER), Traffic Text Recognition, Morphological Segmentation, Bounding Box, Optical Character Recognition I. INTRODUCTION Visual text is one in all the foremost necessary strategies of communication utilized by human beings and is wide utilized in our everyday life. Hence, interpreting this textual data is of great significance. Human beings inherently get the ability to find and acknowledge the textual content in their surroundings whereas it's a challenging problem for computer systems. The field of text detection focuses on detecting text embedded in images and videos with the help of computer systems[1]. Researchers have made significant progress in detecting the text from images of machine printed documents; on the other hand detecting the text in natural scenes is still a new topic for research. Text detection in natural scenes is difficult because of Sumit Gupta Professor Department of Electronic & Communication OIST, Bhopal Sumitmanit25@gmail.com the variation in fonts, backgrounds, lighting conditions and textures occurring in these images. Also, the location of text in natural scenes is highly randomized. Detecting the embedded text information in natural scene images is of great significance for image understanding, content-based image retrieval and navigation.on account of these challenges, text detection in unconstrained images is considered as far from solved [2-4, 8]. Methods for scene text localization and recognition aim to find all areas in an image that would be considered as text by human, mark boundaries of the areas and output a sequence of characters associated with its content. They allow for real-world images processing and extracting the content of every detected text area into a digital text layout that can be additionally executed by a computer. Scene text localization and recognition (also known as text localization and recognition in real-world images, nature scene OCR or text-in-the-wild problem) is an open problem, unlike printed document recognition where state-of-the-art systems are able to recognize correctly more than 99% of characters. Factors contributing to the complexity of the problem include: nonuniform background, the need for compensation of perspective effects (for documents, rotation or rotation and scaling is sufficient); real-world texts are often short snippets written in different fonts and languages; text alignment does not follow strict rules of printed documents; many words are proper names which prevents an effective use of a dictionary [3, 9]. Applications of text localization and recognition in real-world images range from automatic annotation of image databases based on their textual content (e.g. Flickr or Google Images), assisting the visually impaired to reading labels on businesses in map applications (e.g. Google Street View). Text detection and recognition in traffic images is, however, a challenging, unsolved computer vision problem. Scene text has complex background, image blur, partially occluded text, variations in font-styles changeable illumination and image noise as illustrated in Figure 1. Commercial systems do not work in these setting. 1

2 Figure 1 Traffic Image Containing Text This paper is arranged as follows. Section II reviews related work of text detection and recognition process while section III describes the proposed methodology. Section IV gives result analysis of proposed methodology. Section V concludes with the performance of proposed methodology as well as shows the enhancement that could be possible in future works. II. RELATED WORK Intelligent decision support system (IDSS) has been proposed which analyzes online news before its publication. It predicts if an article will become popular. Online news popularity is measured by considering communication between web and social networks with factors like number of shares, likes and comments. The popularity of candidate articles is first estimated and changes in unpopular news are suggested in optimization module [1]. Existing text-detection methods can be divided into region based and texture based methods. Region based methods rely on image segmentation. Pixels are grouped to CCs which are character candidates. These candidates are further grouped to candidate words and textlines based on geometric features. Texture based methods distinguish text from non-text based on local features and machine learning techniques. Fischler et al. planned a replacement model known as Random Sample consensus (RANSAC) [1]. It is a fitting model. It's capable of smoothing knowledge that contain a major proportion of errors. Here a set of the original data is taken first. this is often known as hypothetic inliners. Then a model is fitted into this hypothetic inliners. The remaining knowledge are tested during this model. The point that matches these models are a region of the consensus data set. This is often however RANSAC algorithmic rule is functioning. Chen et al. [4] propose a text detection method using MSER. The outlines of MSER are improved by edges detection techniques such as canny edge detection. This makes MSER less responsive to blur images. Based on geometric cues these candidate character regions are then grouped to words and textlines. Neumann et al.[7] propose ERs for segmenting regions. ERs are extracted on the gradient images, HSI and RGB to recover regions for character candidate. As an alternative of using heuristics as Epshtein et al. [3] for labeling text, an AdaBoost classifier based on geometric features is used. Text-CCs are then grouped to words. In [11] Zheng et al. proposed a completely unique image operator is projected to observe and find text in scene images. to attain a high recall of character detection, extremal regions are detected as character candidates. 2 classifiers are trained to spot characters, and a algorithmic native search algorithm is projected to extract characters that are incorrectly known by the classifiers. An efficient pruning technique, which mixes component trees and recognition results, is projected to prune continuation elements. A cascaded technique combines text line entropy with a Convolutional Neural Network model. It's wont to verify text candidates that reduce the quantity of nontext regions. The projected technique is taking a look at on 3 public datasets, i.e. ICDAR2011 dataset, ICDAR2013 dataset and ICDAR2015 dataset. Wang et al. [6] propose HOG features with a Random Ferns classifier to detect and classify text in an end-to-end setting. The multiclass-detector is trained on letters. Non-maxima of the detector results are concealed. The remaining letters are then combined in a Pictorial Structure framework, where letters are parts of words. For each word in a dictionary, the most plausible character responses are found in the image. Detected words are then rescored based on geometric information and non-maxima suppression is done to remove overlapping wordresponses. In [10] Greenhalgh et al. proposed a unique system for the automated detection and recognition of text in traffic signs. Scene structure is employed to describe search regions at intervals the image, surrounded by traffic sign candidates are then found. Maximally Stable Extremal Regions (MSERs) and hue, saturation, and worth color thresholding are used to locate a large range of candidates, that are then reduced by applying constraints supported temporal and structural data. A recognition stage interprets the text contained at intervals detected candidate regions. Individual text characters are detected as MSERs and are classified into lines, before being interpreted using optical character recognition (OCR). Recognition accuracy is immensely improved through the temporal fusion of text results across consecutive frames. III. PROPOSED METHODOLOGY A novel Connecting Character based text recognition and extraction algorithm is designed which uses Maximally Stable Extremely Regions (MSER) for test candidate recognition and extraction from traffic signs. Despite their auspicious 2

3 properties, MSER has been conveyed to be delicate towards blurred Image. To allow for detecting small letters in images of limited resolution or blurred Image, the complimentary properties of Lucy-Richardson Algorithm and canny edge Algorithm is used. Further geometric filtering and pairing is applied to efficiently obtain more reliable results. Finally, texts are clustered into lines and additional checks are performed to eliminate false positives. The proposed algorithm, illustrated in Figure 2, is divided into two basic steps i.e. text area detection and text recognition. The overall flow of the proposed algorithm is divided into two stages i.e. Text Recognition and Text Extraction as described below. The detection stage exploits knowledge of the structure of the scene, i.e., the size and location of the road in the frame, to determine the regions in the scene that it should search for traffic text signs Once a potential traffic sign has been located, the next stage of the algorithm attempts to recognize text within the region. In this step firstly load the traffic image in which we have to detect text. Before preceding towards next step first of all the algorithm crop that portion of image that contains text and further the text can be rotated in plane, if required. Step 1: Noise Removal and De-blurring Image Due to imperfections in the image capturing procedure, on the other hand, the recorded image invariably represents a corrupted version of the original image. The degradation results in blurring of image, which affects identification and retrieval of the essential information in the image frames. It can be the result of relative motion between the camera and the original image frame, by an out of focus of optical system, atmospheric disturbances and deviation in the optical system. Noise introduced by the medium through which the image is created can also cause degradation. The degradation phenomenon of the acquired images results severe costeffective loss. Consequently, restoring the corrupted images is an urgent task in order to expand uses of the images. In this step the proposed algorithm uses Lucy-Richardson Algorithm is used for noise removal and de-blurring the blurred image. Step 2: Contrast Adjustment and Conversion RGB image to Binary Image Image enhancement techniques are used to improve an image, where "improve" is sometimes defined objectively (e.g., increase the signal-to-noise ratio), and sometimes subjectively (e.g., formulate definite features easier to see by modifying the colors or intensity value). Intensity adjustment is an image enhancement technique that maps an image's intensity values to a new range. In this step, contrast or brightness level of the input image is enhanced. Further in this step RGB Image is converted into gray scale Image. The rgb2gray method function transforms RGB images to grayscale by removing the information of hue and saturation at the same time as retaining the luminance. Step 3: Edge Enhancement In this step, Canny edge detection algorithm is used for image edge detection. Text Area Detection Phase Input Image Noise Removal De-blurring Image Contrast Enhancement & Binary Conversion Edge Detection and Enhancement MSER region detection Text Recognition Phase Geometric Filtering Morphological Segmentation Character Connecting Text line Formation Word Separation Figure 2 Flow chart of proposed Algorithm Step 4: MSER region detection Since the intensity distinction of text to its background is often important and an even intensity or color inside each text are often assumed, MSER could be a natural choice for text detection. Step 5: Morphological Segmentation After detection of edge enhanced MSER region, text recognition phase is started. In text recognition phase first of all we perform the morphological segmentation over edge enhanced MSER region. 3

4 Connecting International Journal of Innovative Engineering Research (E-ISSN: X) Step 6: Geometric Filtering and Character With the extraction of segmented region, geometric filtration phase starts. In this phase segmented MSER region is then filtered on the basis of features such as aspect ratio, eccentricity, solidicity, etc. And thus filtered candidates are further connected using bounding box technique. Step 7: Text line formation and Word separation Subsequent in this stage of the algorithm locates lines of text among the detected candidate regions. This allows the total number of CCs to be reduced, removing non-character CCs and therefore raising the probabilities for higher accuracy. As a final step, text lines are split into individual words by classifying, by OCR, the inter letter distances into two classes: the character spacing and the word spacing. IV. EXPERIMENTAL PARAMETERS AND RESULT ANALYSIS In order to evaluate the performance of proposed algorithm scheme, the proposed algorithm is simulated in following configuration: Pentium Core I5-2430M 2.40 GHz 4GB RAM 64-bit Operating System MATLAB Platform Image Processing Toolbox Computer Vision Toolbox The traffic text sign data that is used in proposed work were obtained from Jaguar Land Rover Research, and these are available to other researchers at These data were captured with a camera, for which the full calibration parameters. To evaluate the performance of the proposed system following parameters such as Precision, Recall and Fmeasure are used. Precision = TP/ (TP+FP) Recall = TP/(TP+FN) Fmeasure = 2* (Precision* Recall) / (Precision + Recall) Where, True Positive (TP) = Correctly detected text in image False Positive (FP) = Text incorrectly identified in images False Negatives (FN) = Text that are failed to be detected in image. The results of detected phase and recognition phase of proposed methodology are evaluated on 8 different image frames and the result analysis of these frames are illustrated in Table 1. To evaluate the performance of the detection and recognition stage on the parameters such as Precision, Recall, and Fmeasure are compared with existing algorithms are illustrated in Table 2 and 3 as well as in Graph 1 and 2. Table 2 comparative Analysis for Text Detection Stage METHOD PRECISION RECALL F_MEASURE Reina et al. [12] Gonzalez et al. [13] Greenhalgh et al. [10] Proposed method Graph 1 Comparative analysis of different text detection techniques Table 3 Comparative Analysis for Text Recognition Stage METHOD PRECISION RECALL F_MEASURE Standard Tesseract OCR OCR with shape correction OCR with temporal fusion Proposed method Graph 2 Comparative analysis of different text recognition techniques 4

5 Frame No. Table 1 Result analysis of proposed method using different image frames Contrast-Edge-Enhanced MSER Input Image Region Extracted Text 0 Cargo 8. Aviation Services 1 LEFT TURN ON GREEN ARROW ONLY 2 Commercial Districts 3 CUVMBERELAAAND HWY Liverpool Canberra Manchester city centre. Didsbury - Q I Ring Rd (W 8. N) Liverpool (M 62) Bolton (M 61) Leeds (M 62) Ring Road All other traffic 514 Keresley Radford B4098 V. CONCLUSION Text extraction from natural scene images is a challenging problem due to the variations in color, font size, text alignment, illumination etc. And it is a technique to identify and isolate the desired text from the images. The proposed algorithm represented a new methodology for text detection 5

6 from traffic image by introducing contrast enhanced edge enhanced MSER region based text detection and recognition system. The existing approaches deal with the same are lacking in accuracy. The proposed contrast enhanced edge enhanced Maximally Stable Extremal Region (EMSER) algorithm works with morphological segmentation to identify the shape of the text objects. After morphological operation geometric filtrations based on eccentricity, aspect ratio, solidicity, etc is performed this will find connected component more accurately. Finally, the proposed method is integrated with bounding box technique to combine connected components into words and extract text using OCR. The newly proposed method not only reported higher precision, Recall and F measure values but also reduced the execution time. Both the detection and recognition stages of the system were validated through comparative analysis, achieving the Fmeasure of 0.93 for detection and 0.95 for recognition. Moreover, we still have to study and understand how the tracking information can help build a better user interface for assistive devices. Observing the patterns of movement and context in the surroundings is crucial for deciding when and how to read text back to the user, enabling a more useful interaction experience. We plan to develop these ideas as part of our future work. REFERENCES [1] M. Fischler and R. Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM, vol. 24, no. 6, pp , Jun [2] X. Chen and A. L. Yuille, Detecting and Reading Text in Natural Scenes, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition., volume 2, pages , [3] B. Epshtein, E. Ofek, and Y.Wexler. Detecting Text in Natural Scenes with Stroke Width Transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages IEEE, [4] H. Chen, S. S. Tsai, G. Schroth, D. M. Chen, R. Grzeszczuk, and B. Girod., Robust Text Detection in Natural Images with Edge-Enhanced Maximally Stable Extremal Regions, In International Conference on Image Processing, pages IEEE, [5] Yi-Feng Pan, Xinwen Hou, and Cheng-Lin Liu, A Hybrid Approach to Detect and Localize Texts in Natural Scene Images, IEEE Transactions on Image Processing, 20(3): , [6] K. Wang, B. Babenko, and S. Belongie, End-to-end scene text recognition, In Proceedings of the IEEE International Conference on Computer Vision, pages , Barcelona, Spain, 2011 [7] L. Neumann and J. Matas, Real-Time Scene Text Localization and Recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages IEEE Computer Society, [8] J. Greenhalgh and M. Mirmehdi, Real-time detection and recognition of road traffic signs, IEEE Trans. Intell. Transp. Syst., vol. 13, no. 4, pp , Dec [9] A. González, L. Bergasa, and J. Yebes, Text detection and recognition on traffic panels from street-level imagery using visual appearance, IEEE Trans. Intell. Transp. Syst., vol. 15, no. 1, pp , Feb [10] Jack Greenhalgh and Majid Mirmehdi, Recognizing Text- Based Traffic Signs, IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 3, pp , June [11] Yang Zheng, Qing Lia, Jie Liu, Heping Liua, Gen Lib, Shuwu Zhang, A cascaded method for text detection in natural scene images, Elsevier [12] A. Reina, R. Sastre, S. Arroyo, and P. Jiménez, Adaptive traffic road sign panels text extraction, in Proc. WSEAS ICSPRA, 2006, pp [13] A. Gonzalez, L. M. Bergasa, J. J. Yebes, and J. Almazan, Text recognition on traffic panels from street-level imagery, in Proc. IVS, Jun. 2012, pp