Expert Systems with Applications

Transcription

1 Expert Systems with Appliations 39 (2012) Contents lists available at SiVerse SieneDiret Expert Systems with Appliations journal homepage: An automated vision system for ontainer-ode reognition Wei Wu a, Zheng Liu b,, Mo Chen a, Xiaomin Yang a, Xiaohai He a a Shool of Eletronis and Information Engineering, Sihuan University, Chengdu , China b Shool of Information Tehnology and Engineering, University of Ottawa, Ottawa, ON, Canada K1A 0R6 artile info abstrat Keywords: Computer vision Text-line loation Charater isolation Charater segmentation Charater reognition Support vetor mahine Automati ontainer-ode reognition is of great importane to the modern ontainer management system. Similar tehniques have been proposed for vehile liense plate reognition in past deades. Compared with liense plate reognition, automati ontainer-ode reognition faes more hallenges due to the severity of nonuniform illumination and invalidation of olor information. In this paper, a omputer vision based ontainer-ode reognition tehnique is proposed. The system onsists of three funtion modules, namely loation, isolation, and harater reognition. In loation module, we propose a text-line region loation algorithm, whih takes into aount the harateristis of single harater as well as the spatial relationship between suessive haraters. This module loates the text-line regions by using a horizontal high-pass filter and sanline analysis. To resolve nonuniform illumination, a twostep proedure is applied to segment ontainer-ode haraters, and a projetion proess is adopted to isolate haraters in the isolation module. In harater reognition module, the harater reognition is ahieved by lassifying the extrated features, whih represent the harater image, with trained support vetor mahines (SVMs). The experimental results demonstrate the effiieny and effetiveness of the proposed tehnique for pratial usage. Ó 2011 Elsevier Ltd. All rights reserved. 1. Introdution In reent years, ship transportation industry beomes more and more important with the development of internationalization. Meanwhile the number of ontainers transported has inreased with this trend. Reognizing ontainer identity ode (also known as ontainer ode) beomes an essential for ontainer management. The traditional manual method to reognize ontainer ode has lots of shortomings, inluding slow speed, high error rate, et. So an automati tehnique to identify the ontainer is desired. Comparing with other tehniques, suh as wireless sensor network and radio frequeny identifiation (RFID), a omputer vision based automati ontainer-ode reognition (ACCR) is a vital tehnique to manage ontainers effiiently. The ACCR system desribed in this paper is to reognize ISO-6346 ode on ontainers (ISO- 6346, 2010). Aording to ISO standards, the ISO-6346 ode onsists of three parts: four apital letters, six digits, and one hek digit. There may be extra haraters beside eleven ISO haraters on the ontainer; however, these eleven ISO haraters defined by ISO standard are onsidered as a unique ode to identify different ontainers. The ACCR system onsists of three main modules: (1) loating text-line regions (loation), (2) isolating ontainer-ode haraters Corresponding author. address: zheng.liu@ieee.org (Z. Liu). (isolation), and (3) reognizing ontainer-ode haraters (harater reognition). In the loation module, text-line regions are identified based on the features of ontainer ode. In the isolation module, ontainer-ode haraters are segmented from the bakgrounds. In the third module, the extrated harater images are reognized and transferred into atual haraters. The flowhart of the ACCR proedure is given in Fig. 1. The ISO standard only defines the ode types on the ontainer. Colors of the haraters and bakgrounds, font types and sizes, and ontainer-ode positions vary from ontainer to ontainer. This introdues hallenges to the ACCR appliation. Examples of typial ontainer images are given in Fig. 2. The harateristis of the ontainer summarized below: 1. Container ode may our in different positions, and may have different olors, sizes, and font types. 2. The ontrast between ontainer-ode haraters and bakgrounds is sharp, whih leads to strong vertial edges for eah harater. 3. The alignment modes of ontainer-ode haraters are varied. For instane, ontainer-ode haraters may be aligned horizontally in one row to multi-row or vertially in one olumn. 4. For horizontal alignment mode, spaes between two suessive haraters are small. And there are always several haraters in eah text-line region. An example of text-line regions is shown in Fig /$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi: /j.eswa

2 W. Wu et al. / Expert Systems with Appliations 39 (2012) Container image Loating text-line regions (loation) Isolating ontainerode haraters (isolation) Reognizing ontainerode haraters (harater reognition) Container ID ode Fig. 1. Flowhart of automati ontainer-ode reognition. Fig. 2. Examples of ontainer-ode images. text-line region text-line region text-line region A 1 A 2 A 3 Fig. 3. The text-line regions and its proess order. Sine the horizontal alignment mode is a dominating mode, this paper mainly desribes the method applied to the horizontal alignment mode. The proess an be easily adapted to vertial alignment mode by rotating the ontainer image 180. Only few parameters need to be adjusted. Although ontainer-ode reognition is similar to vehile liense plate reognition (for example, both inlude three same funtion modules: loation, isolation, and harater reognition), ontainer-ode reognition is faing more hallenges ompared with liense plate reognition. Color (Deb, Kang, & Jo, 2009) and plate edges (Duan, Du, Phuo, & Hoang, 2005), whih are very useful for loating liense plate region, annot provide useful information for ontainer-ode loation. Moreover, ontainer-ode haraters are more easily affeted by outdoor illumination onditions when ompared with liense plate haraters. Espeially, surfae urvature introdues severe illumination hange for ontainer-ode haraters. Due to suh differenes, methods for liense plate reognition may be not appliable to the ontainer-ode reognition. In this paper, a omputer vision based ontainer-ode reognition tehnique is proposed. This tehnique onsists of three funtion modules: loation, isolation, and harater reognition. A sanline-based text-line region loation algorithm is implemented in the loation module, whih identifies the text-line regions by using a horizontal high-pass filter and sanline analysis. In isolation module, a two-step segmenting operation is first applied to text-line regions. Then, a projetion proess is implemented to isolate haraters from these regions. In the harater reognition module, SVMs are trained to lassify the isolated haraters with extrated features. Experiments are arried out with 1214 ontainer images. The rest of this paper is organized as follows. Related works is overviewed in Setion 2. An algorithm to loate text-line region is desribed in Setion 3. Setions 4 and 5 are devoted to the isolation and harater reognition respetively. Experimental results are presented in Setion 6. This paper is summarized in Setion Related works Loating text-line regions on a ontainer is a hallenge beause their positions, olors of haraters and bakgrounds, and font types, sizes may vary from one to another. Kim, Kim, and Woo (2007) proposed using an adaptive resonane theory (ART-2) based quantization method to identify text-line regions with features of haraters suh as olor, size, and ratio of height to width. However, this method may fail when the ontainer image suffers from nonuniform illumination. Moreover, the ART-2 based quantization is omputation intensive. Similar to the methods in (Abolghasemi & Ahmadyfard, 2009; Huang, Chang, Chen, & Sandnes, 2008; Huang, Chen, Chang, & Sandnes, 2009), He, Liu, Ma, and Li (2005) proposed a method, where image edges were extrated and projeted horizontally with a Gaussian smoothing filter. The position of the loal maximums and loal minimums of the smoothed histogram were found. From eah loal maximum, the top and bottom position of eah text-line region an be obtained. However, this method is a still lak of robustness for ontainer images with noises and harater likenesses. The purpose of harater isolation is to isolate all eleven ISO haraters from bakgrounds. Charater isolation methods an be roughly lassified into two lasses, i.e. global optimization-based method and segmentation-based method. In global optimizationbased methods, the goal is to obtain a ombined result of harater spatial arrangement and single harater reognition result rather

3 2844 W. Wu et al. / Expert Systems with Appliations 39 (2012) than only to obtain good reognition results for eah harater. In Fran and Hlava (2005) and Fan and Fan (2009), a hidden Markov hain was used to formulate the dynami segmentation of haraters, and the segmentation problem was expressed as the maximum a posteriori estimation from a set of admissible segmentations. However, the global optimization-based method is impratial for real-time implementation. In segmentation-based methods, text-line or plate region is segmented first. Then, vertial and horizontal projetions, onneted omponent analysis (Li, Zeng, & Lin, 2006; Mahini, Kasaei, Dorri, & Dorri, 2006; Martin, Garia, & Alba, 2002), or ontour analysis (Anagnostopoulos, Anagnostopoulos, Psoroulas, Loumos, & Kayafas, 2008) are applied to obtain the position of eah harater and other binary objet measurements suh as height, width, and area may be used to eliminate noises. Sine segmentation with one global threshold annot always ahieve good results with nonuniform illumination, loal segmentation method beomes a better hoie in this ase. The segmentation methods proposed in Niblak (1986), Sauvola and Pietikinen (2000), Nakagawa and Rosenfeld (1979) are adopted in Coetzee, Botha, and Weber (1998), Anagnostopoulos, Anagnostopoulos, Loumos, and Kayafas (2006), Chang, Chen, Chung, and Chen (2004) respetively. Although these segmentation methods are well applied to vehile liense plate images, they may fail for ontainer images enountering nonuniform illumination. In Naito, Tsukada, Yamada, Kozuka, and Yamamoto (2000), an image is first divided into m n fixed-size bloks, and then a threshold is hosen for eah blok. However, the fixed-size blok is not an optimal hoie for segmentation. Meanwhile, it is hard to determine the size of bloks. To reognize segmented harater images, numerous methods from template mathing and neural network to SVM have been investigated. The template mathing tehnique is suitable to reognize haraters with non-deformable and fixed-size fonts. However, ontainer-ode haraters do not meet suh requirements. Self-organized neural network (Chang et al., 2004), probabilisti neural networks (Anagnostopoulos et al., 2006), and bak-propagation neural network (Jiao, Ye, & Huang, 2009), HMM (hidden Markov model ) (Duan et al., 2005), and SVM (Dong, Krzyzak, & Suen, 2005; Shanthi & Duraiswamy, 2010) are well adopted methods in reognition. It has demonstrated that SVM outperforms other reognition methods in previous study (Tapia & Rojas, 2005). 3. Sanline-based text-line region loation Text-line region loation is the first module in the whole reognition system. Its purpose is to distinguish text-line regions from the bakgrounds. Poor image quality, disturbanes from other haraters, and the refletion from the ontainer surfae make it a hallenge to loate text-line regions aurately and effiiently. As the olor information is not useful for this appliation, a olor ontainer image is first onverted to a graysale image with the following equation: L ¼ 0:299R þ 0:587G þ 0:114B where the R/G/B represent the red/green/blue omponent of the olor image. Aording to ontainer-ode harateristi (2) and (4), luminane values hange sharply at a line rossing a text-line region; furthermore, suh hanges in the text-line region are muh more frequent and signifiant than in other non-text-line regions. Fig. 4 gives an example of luminane hanges in a ontainer image. Fig. 4(b) shows the ross-setion line at position 1 while Fig. 4() illustrates the luminane hange at position 2. Based on above observation, a text-line region an be identified by these sharp ð1þ hanges and the hange frequeny. The proposed loation proedure is shown in Fig Generating edge image Sine ontainer-ode haraters have strong vertial textures, a region with vertial textures will be identified and extrated as a harater region. A horizontal high-pass filter is applied to detet the vertial edges. Suh operation an be expressed as: I vg ¼ I H where I and I vg are the original and vertial edge image respetively. Operation stands for onvolution and H is a horizontal high-pass filter, whih is expressed as: 1 ½ 1;...; 1; k; 1;...; 1Š n fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl} n where n is the length of H, and k = n 1. We hose n = 13 in our system. Fig. 6(b) shows the results of filtering two original ontainer images, whih are shown in Fig. 6(a). After filtering operation, every pixel in I vg above a threshold T is onsidered as a salient edge pixel. As the threshold is higher in brighter regions, an adaptive thresholding operation based on the mean luminane value in the filter window is implemented. The pixel at (x,y) of the binary edge image I vb is set to either 0 (nonedge point) or 1 (edge point) aording to: I vb ðx; yþ ¼ 0; I vgðx; yþ < bm V ðx; yþ ð4þ 1; I vg ðx; yþ > bm V ðx; yþ where M V is the mean image of I obtained by an average filter. M V (x,y) refers to a pixel value of M V, whose oordinate is (x,y). And b is a oeffiient, whih is set as 0.1 by trial and error. The binary edge images are shown in Fig. 6() Removing the non-harater edges Besides the harater edges, there are many non-harater edges in I vb, suh as random noise edges, refletion edges, and bakground edges. These non-harater edges may interfere with the ontainer-ode loation. In order to suppress these non-harater edges, we obtain eah edge s height and position by applying onneted omponent analysis, firstly. Then, we remove two types of suppressed non-harater edges, whih are defined as follows: Any edge shorter than the minimum height of ontainer-ode harater C h min ; Any edge longer than the maximum height of ontainer-ode harater C h max on ondition that the distane between the edge and any harater edge is larger than the maximum distane between two suessive haraters C dis. Suppressed edges of the first type are mainly aused by stains and random noises while the seond type is usually bakground edges or refletion edges, whih are generated by the rugged ontainer surfae or by the nonuniform illumination. The ondition in the seond type is to prevent the harater edges, whih adhere to the refletion edges, to be onsidered as non-harater edges. Fig. 7 shows two-type suppressed edges to be removed. After removing these edges, most of the non-harater edges are eliminated while harater edges are preserved. The binary edge images after removing non-harater edges are shown in Fig. 6(d). ð2þ ð3þ

4 W. Wu et al. / Expert Systems with Appliations 39 (2012) (a) (b) () Fig. 4. Example of luminane hanges (a) the luminane image; (b) ross-setion line at position 1; and () ross-setion line at position 2. Input ontainer image Graying olor im age Generating edge image Obtaining vertial edge im age Binarizing vertial edge image Text-line regions Analyzing sanline Removing the edge of non-harater Fig. 5. Flowhart of the text-line loation Analyzing sanline The proedure to analyze sanline for loating text-line regions is illustrated in Fig. 8. Suppose that a jump is haraterized by a hange of the luminane value from 1 to 0 or from 0 to 1. Herein, we define a new term sanline for loating text-line regions. A sanline is defined as a line at a row of I vb, whih starts with a jump and ends with another jump. A group of neighboring sanlines an omprise of a text-line region. Fig. 9 shows a sanline and its jumps. The jumps in one sanline must satisfy the following onditions: The distanes of any two suessive jumps in a sanline are smaller than C dis. There are at least N pl 2 ontinual jumps in a sanline. N pl denotes the minimum number of haraters in a text-line region. As there are at least three haraters in a text-line region, we set N pl = 3 in our system. To loate text-line regions, we first san image I vb from left to right for eah row to find sanlines. Then, we onnet the neighboring sanlines together to omprise of text-line regions. The relationship of neighboring sanlines an be expressed as: Con S m i ; S n j¼i1 ¼ 1; Sm i ðendþ > S n j ðstartþ and Sm i ðstartþ < S n j ðendþ 0; otherwise where S m i denotes the mth sanline in ith row; S m i ðstartþ and S m i ðendþ refer to horizontal start and end position of S m i respetively. If S m i and S m i are onneted to eah other, Con S m i ; S n j returns one; otherwise it returns zero. One the relationship of sanlines is identified, sanlinegroup regions an be loated by onneted omponent analysis. However, not all of these sanline-group regions are text-line regions. Sine a text-line region is determined by its haraters edges, the height of text-line region is approximately equal to the height of its orresponding haraters. Therefore, any region, whose height is shorter than C h min or longer than Ch max is removed. The final loation results are shown in Fig. 6(e). 4. Container-ode harater isolation The isolation module will isolate all the eleven ISO ontainerode haraters from the text-line regions. The isolation proedure is illustrated in Fig. 10. Sine ontainer ode may be aligned in several text-line regions, we need to identify these text-line regions ð5þ

5 2846 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 6. Some example results for loation proedure (a) original ontainer images; (b) edge images I vg ; () binary edge images I vb ; (d) binary edge images after removing nonharater edges; and (e) final results of text-line region loation. based on the harateristis of the alignment modes. As the ISO haraters are printed from left to right and from top to bottom, we proess the text-line regions in the same order. Fig. 3 illustrates suh an order. Suppose urrent text-line region A is fed into this module and eah harater in this region is isolated. If all the eleven ISO haraters are not isolated, we move to the text-line region A neighbor adjaent to A until all the ISO haraters are isolated. Speifially, the relationship between A neighbor and A should satisfy either of the two onditions: ( jt t n j < ea height l n r < D h and ð6þ jb b n j < ea height or t n b < D v and 8 < jl l n j < ea height : A height A height < ea height neighbor where t, b, l, r and t n, b n, l n, r n refer to the top, bottom, and left and right position of A and A neighbor respetively. D h and Dv are the maximum distane between two text-line regions in the horizontal and vertial diretion. A height respetively. We set D h ¼ D v ¼ A height ð7þ and A height are the height of A neighbor and A neighbor =2 aording to the priori knowledge of alignment mode. e is a small plus onstant, whih is tolerane for noise. And we set e = 0.1 in our implementation. Eq. (6) represents the horizontal relationship of two text-line regions

6 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 9. A sanline and its jumps. Type one suppressed non-harater edges Type two suppressed non-harater edges Fig. 7. Type one and two suppressed edges to be removed. while Eq. (7) is based on the vertial relationship. Eqs. (6) and (7) are illustrated in Fig. 11(a) and (b) respetively. As harater sizes, and the distanes between suessive harater are roughly same in one ontainer image, the standard varianes of them are used to valid these isolated ontainer-ode haraters. If the summation of the standard varianes is smaller than a threshold, these isolated haraters are validated. Otherwise, we ontinue the isolation proess. Usually, the hek digit of ontainer ode is within its bounding retangle (see Fig. 12(a)). To reognize the hek digit, this retangle needs to be removed. Detailed desriptions on the hek digit retangle elimination and the harater isolation from the text-line regions are provided in the following subsetions Isolating haraters from a text-line region To isolate the haraters, the text-line regions are firstly segmented. Then, vertial and horizontal projetion is applied to get the position of eah harater. This operations illustrated in Fig Segmentation Sine ontainer images are aptured in the outdoor environment, they are subjet to nonuniform illumination, refletions, and shadows. This makes it diffiult to separate haraters from bakgrounds. Many segmentation approahes, whih have been suessfully applied to liense plate reognition, are not suitable to ontainer-ode haraters. Examples of a nonuniform illumination are given in Fig. 2(d) (f). To deal with these problems, we propose a two-step approah to segment haraters from the bakgrounds. Although the whole text-line region is subjet to nonuniform illumination, loal regions an still be onsidered in an ideal ondition of uniform illumination. In the first step, the text-line region an be roughly divided into three area types, namely non-harater area, harater area, and refletion area. The three area types in an original graysale image are illustrated in Fig. 14(a). In the seond step, a ombined segmentation strategy is applied to these areas. Dividing. Beause harater edges are stable and not sensitive to nonuniform illumination, the text-line region is divided into different areas with the edges. To redue the omputational load, the vertial version of filter H is applied to I vg (A) i.e. text-line region A in I vg to generate the edge image I vhg (A). I vhg (A) onsisting of both vertial edges and horizontal edges an be expressed as: I vhg ðaþ ¼I vg ðaþh T ¼ IðAÞH H T where H T denotes the vertial version of filter H. After obtaining I vhg (A), the same adaptive threshold used in loation module is applied to generate the binary edge image I vhb (A). Fig. 15(a) and (b) show the original image I(A) and its binary edge image I vhg (A) respetively. Similar to the loation module, we remove the nonharater edges aording to edges height and width (any edge, whose height is shorter than C h min ; or any edge, whose height is longer than C h max and width is shorter than Aheight =2, is removed). The binary edge image after removing non-harater edges is shown in Fig. 15(). Subsequently, the vertial projetion histogram of I vhb (A) is alulated as: Aheight h A ðiþ ¼ X j¼1 ð8þ I vhb ðaþði; jþ; i ¼ 1; 2;...; A width ð9þ where A height and A width represent the height and width of the textline region A respetively. Fig. 15(d) shows the vertially projeted results. Sine usually refletions appear vertially and ross textline region (see Fig. 2(e)), the h A (i) in these areas are almost equal to A height. Therefore, the areas with h A (i) larger than the threshold T 1 =(1 e)a height are identified as refletion areas. Sine non-harater areas have few edges, h A (i) of non-harater area are below a threshold T 2, where T 2 < T 1 and T 2 = ea height. A harater area is defined as an area with its h A (i) falling in [T 2,T 1 ]. The illustration of a text-line region with different areas and their orresponding thresholds is shown in Fig. 14(b). And Fig. 15(e) shows the different areas after dividing. Segmenting. After dividing the text-line region into different areas, a ombined segment strategy is applied. We only need to deal with the areas ontaining haraters, i.e. harater areas and refletion areas. For harater areas, we assume that the harater and the bakground have two normal distributions with similar varianes. Therefore, Ostu s segmentation method is applied to eah harater areas (Otsu, 1979). For refletion areas, we use a threshold defined as T 3 =(1 e)v max to segment every refletion area. v max is the mean value of the largest k luminane values in the refletion area. Here, we set k equal to 5% of pixels of the refletion area. Fig. 15(f) shows the final segmentation results. The omparison of different segmentation methods is shown in Fig. 16. We an observe that our approah is better than other state-of-art methods. Eah harater is learly separated from the Binary edge image Removing the Sanning to obtain sanlines Analyzing onneted sanlines to obtain regions unsatisfied regions Text-line regions Fig. 8. The proedure for sanline analysis.

7 2848 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 10. Isolation framework. bakgrounds, and there are few noises in our results when ompared with others Projetion After segmentation, projetion method (Shi, Zhao, & Shen, 2005; Wang, Ni, Li, & Chen, 2004) or onneted omponent analysis (Li et al., 2006; Mahini et al., 2006; Martin et al., 2002) is adopted to isolate the haraters from the text-line region. In our implementation, a projetion method is used for omputational simpliity. To identify the horizontal onfine of a harater, vertial projeting is arried out to obtain a histogram, whose minimum values allow us to divide the text-line region into onfined harater areas. If two adjaent areas are lose enough, they will be merged into one. Similarly, the top and bottom onfine of eah harater an be loated by using horizontal projetion. With the above proess, eah harater an be aurately loated. image from left to right at eah row as shown in Fig. 12(b). Then, vertial projetion histogram of the hange-point map I p h is alulated, whih an be expressed as: Aheight h Ahek ðiþ ¼ X hek I p hði; jþ j¼1 i ¼ 1; 2;...; Awidth hek ð10þ where A height hek and Awidth hek are the width and height of the hek-digit image. A vertial projetion example is shown in Fig. 12(). Sine the method in Kumano et al. (2004) does not work well in all situations, we hoose the first position k in h Ahek ðiþ bigger than T 4, where there is T 4 ¼ ea height hek. After obtaining k, the left edge of the image is made to be zero from 0 to k. Performing a similar operation on the right, upper, and bottom sides of the hek-digit image, we an obtain a hek digit image without the bounding retangle (see Fig. 12(d)) Removing the retangle of the hek digit To remove the retangle from the hek-digit image, we use a similar method as desribed in Kumano et al. (2004). Take the left edge of the retangle as an example. The horizontal position of first hange from 1 to 0 an be obtained by sanning hek-digit 5. Charater reognition The harater reognition module is to onvert isolated harater images into haraters. We first extrat the features whih represent the harater image. Edge densities generated by Sobel operator in harater image pathes are used as the features in this

8 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 11. The harateristis of the alignment modes: (a) the horizontal relationship; and (b) the vertial relationship of two text-line regions. Fig. 12. Example of removing a retangle from hek-digit image: (a) original hek-digit image; (b) the hange point map (hange points are shown in white olor); () the histogram of hanging points; and (d) the final result.

9 2850 W. Wu et al. / Expert Systems with Appliations 39 (2012) Segmentation Candidate textline Region Inferring binary edge image Dividing Removing nonharater edges Dividing into different parts Projeting Segmenting out different parts Segmenting Isolated haraters Isolation Isolating haraters by projetion method Fig. 13. The text-line segmentation proess. (a) Charater areas Refletion Non-harater areas areas C y ¼ I ð12þ (b) Charater areas Refletion areas paper. Then, these features are fed into SVMs for lassifiation. Sine there are two types of ontainer-ode haraters, i.e. apital letter and digit, we build two SVM based reognition models for harater reognition, one for apital letters and the other for digits. The flowhart of the proposed harater reognition is shown in Fig Feature extration Non-harater areas Fig. 14. Illustration of text-line region with non-harater areas, harater areas, refletion areas, and orresponding thresholds: (a) original graysale image of a text-line region and (b) histogram of the vertial projetion of the binary edge image derived from (a). Sobel operator is first applied to extrat vertial edge map C x and horizontal edge map C y from a harater image I. C y and C x an be obtained as follows: C x ¼ I ð11þ T 1 T 2 Fig. 18 gives an example of extrated edge maps. To emphasize the harateristis of eah harater, twenty-five loal pathes are defined as the shadow regions in Fig. 19. InFig. 19(a), there are 4 4 = 16 pathes, whih evenly divide the harater image. In Fig. 19(b), there are 3 3 = 9 pathes, whih overlap those pathes defined in Fig. 19(a). We an divide C x and C y into 25 pathes respetively. For eah path, the edge density is alulated as a feature. Suppose that F l x and Fl y are the edge densities of C x and C y in zone l and an be expressed as: F l x ¼ P ði;jþ2zðlþ C xði; jþ Z w Z h F l y ¼ P ði;jþ2zðlþ C yði; jþ Z w Z h ð13þ ð14þ where (i,j) 2 Z(l) refers to the position of i, j belonging to zone l; Z w, Z h is the width and height of the zone respetively. Thus, eah harater image has 50 features fed into the SVM lassifier Classifiation A SVM lassifier is generally a supervised binary lassifier based on the statistial theory of Cortes and Vapnik (1995) and Burges (1998), where experimental data and strutural behavior are taken into aount for better generalization apability based on the priniple of strutural risk minimization (SRM). Given a training data set (x i,y i ); i =1,...,l, where x i 2 R n is samples and y i 2 {1, 1} are labels; l refers to the number of samples. The SVM lassifies an input z through the funtion: f ðzþ ¼ Xl i¼1 b ¼ y r Xl a i y i Kðz; x i Þ b i¼1 a i y i Kðx r ; x i Þ r ¼ 1; 2;...; l ð15þ ð16þ

10 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 15. Two segmentation examples: (a) original ontainer-ode images; (b) binary edge images; () denoised binary edge images; (d) vertially projeted results; (e) different areas; and (f) final segmentation results. Fig. 16. Comparison of segmentation: (a) original images; (b) results of Ostu s method (Otsu, 1979); () results of Niblak s method (Niblak, 1986); (d) results of logial level tehnique (LLT) (Kamel & Zhao, 1993); (e) results of Sauvola s method (Sauvola & Pietikinen, 2000); and (f) our results. Feature Extration Charater image Obtaining vertial and horizontal edge Dividing edge maps into pathes Calulating the densities of edge Classifiation Output apital letter harater SVM (Capital letter reognition) Y Is alphabet N Output digit harater SVM (Digit reognition) Fig. 17. Flowhart of harater reognition.

11 2852 W. Wu et al. / Expert Systems with Appliations 39 (2012) Table 1 Performane evaluation of the proposed tehnique. Module Daytime (%) Night (%) Auray (%) Loation Isolation Charater reognition Overall performane where oeffiient a i are non-zero only for the subset of the input data alled support vetors; K(x, y) is a kernel funtion, whih satisfies the Merer onditions. The most ommonly used kernels, inluding radial basis, polynomial, and sigmoid. In this study, the radial basis funtion (RBF) kernel is used, whih an be express as: Kðx; yþ ¼exp k x y 2r 2 k2! ð17þ When training the models, samples were olleted by hand for eah harater. Over 300 samples were marked for eah harater to ensure the reognition performane. 6. Experimental results (a) (b) () Fig. 18. An example of edge maps (a) original binary harater image; (b) horizontal edge map; and () vertial edge map. The suess of the ontainer-ode reognition is defined as a proess that is able to extrat the text-line regions, isolate all the eleven ISO haraters, and reognize all these haraters orretly. The input to the reognition system is an image with ontainer ode, and the output is a series of haraters derived from the image. This setion presents the experimental results of the proposed ontainer-ode reognition system, inluding loation, isolation, reognition, and overall reognition results. In order to evaluate our method, 1214 ontainer images ( pixels) obtained from real port environments were used in the experiment, among whih 376 and 838 ontainer images were aptured during night and daytime respetively. Both the brightness and ontrast of the images hange rapidly. And the ontainer odes in the images are of varied olors and sizes, and are aligned and loated differently. The reognition results are given in Table 1, whih inludes loation, isolation, reognition, and the overall auray rate. The auray rate for eah module is defined as: Number of orretly proessed samples Auray rate ¼ Number of all test samples 100% ð18þ The overall auray rate is the produt of auray rates of the three modules. The proposed algorithms an aurately loate the text-line regions for varied positions, olors, illumination onditions, alignment modes, and harater sizes. Text-line region loation auray rate is as high as 97.94%. Examples for loation during daytime and night are given in Figs. 20 and 21 respetively. Multiple missing haraters and severe noise ontamination are the major reasons for inorretly loating ontainer ode. Some examples of failure in loation module are shown in Fig. 22. Sine faded and seriously nonuniform illumination introdues diffiulties to the isolation module, 30 images at daytime and 21 images at night were not suessfully isolated. Thus, the isolation module ahieved an auray rate of 95.71%. Examples of isolation results are given in Fig. 23. And some examples of the failure in isolation module are shown in Fig. 24. Due to harater deformity in some ases, 25 Fig. 19. Loal pathes for feature extration.

12 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 20. Loation examples of variety of ontainers at daytime. Fig. 21. Loation examples of variety of ontainers at night. harater images were not reognized orretly, whih led to a harater reognition rate of 97.80%. The overall reognition rate is 91.68%, for the all 1214 images. A omparison of He s method in He et al. (2005) with ours for loation and isolation is given in Table 2. Sine the details of harater reognition were not desribed in He s paper, the omparison of harater reognition and overall performane is impossible. We adopted four different segmentation algorithms (Kamel & Zhao, 1993; Niblak, 1986; Otsu, 1979; Sauvola & Pietikinen, 2000) in the implementation of He s isolation method. Our proposed method outperforms He s loation and isolation algorithms, whih relies only on vertial edges of haraters. In ontrast, our loation algorithm not only makes use of vertial edges of haraters but also exploits the spatial relationships between suessive haraters. In addition, the proposed segmentation approah an deal with the nonuniform illumination and refletion on the ontainers very well.

13 2854 W. Wu et al. / Expert Systems with Appliations 39 (2012) Fig. 22. Some failure examples of loation. Fig. 23. Examples of isolation results. 7. Summary Fig. 24. Some examples of failure in isolation. Table 2 Performane omparisons. Method Loation Isolation (%) (%) He s method (He et al., 2005) (Ostu s method) (Niblak s method) logial level tehnique (LLT) (Sauvola s method) Proposed method The time omplexity of the proposed tehnique is linear with the number of pixels of proessed image. In the loation module, image graying, edge extration, non-harater edge elimination, and sanline analysis, all have a time omplexity of O(s), where s = W H, and W and H define the width and height of the ontainer image. For ontainer-ode harater isolation, proessing unit is text-line region, whih is only a small part of the ontainer image. Furthermore, all the operations of generating edge image, eliminating noises, and projeting in a text-line region have a time omplexity O(a), where a s is the number of pixels of the textline regions. For harater reognition, the feature extration proedure has no normalization, whih results in that this proedure performs very fast. In the implementation of SVM, the time omplexity of training algorithms is muh higher than that of testing phase of the SVM. However, the SVM training proess an be performed in advane. The SVM testing phase only requires O(MN s ) operations, where N s refers to the number of support vetors. And N s is usually quite smaller than S, whih refers to the number of samples while M is the dimension of the feature vetor. The average time for proessing a ontainer image is less than 110 ms on a personal omputer (Pentium-IV 2.4G with 1G RAM). In this paper, we present a ontainer-ode reognition tehnique based on omputer vision. There are two major ontributions from this tehnique. The first one is the sanline-based algorithm to extrat text-line regions, whih ombines the vertial harater edges and the spatial relationship between suessive haraters. The proposed algorithm is suitable to loate ontainer odes of different olors, sizes, and alignment modes. The seond ontribution is a two-step segmentation approah in the isolation module to takle the severity of nonuniform illumination. The experiments with ontainer images aptured from the real port environment demonstrate the effetiveness of the proposed tehnique. Future work will fous on reognizing ontainer ode from multiple images aptured from the same ontainer with one or multiple ameras during its moving. The ontainer images are aptured at the different positions or different time. Thus, the haraters on the ontainer may be lear in one image but not in another one. The authors believe that integrating or fusing these images for the same ontainer an improve the performane of automati

14 W. Wu et al. / Expert Systems with Appliations 39 (2012) ontainer-ode reognition. This remains a topi for our future development. Referenes Abolghasemi, V., & Ahmadyfard, A. (2009). An edge-based olor-aided method for liense plate detetion. Image and Vision Computing, 27(8), Anagnostopoulos, C., Anagnostopoulos, I., Loumos, V., & Kayafas, E. (2006). A liense plate-reognition algorithm for intelligent transportation system appliation. IEEE Transation on Intelligent Transportation Systems, 7(3), Anagnostopoulos, C.-N. E., Anagnostopoulos, I. E., Psoroulas, I. D., Loumos, V., & Kayafas, E. (2008). Liense plate reognition from still images and video sequenes: A survey. IEEE Transations on Intelligent Transportation Systems, 9(3), Burges, C. J. C. (1998). A tutorial on support vetor mahines for pattern reognition. Data Mining and Knowledge Disovery, 2(2), Chang, S., Chen, L., Chung, Y., & Chen, S. (2004). Automati liense plate reognition. IEEE Transation on Intelligent Transportation Systems, 5(1), Coetzee, C., Botha, C., & Weber, D. (1998). P based number plate reognition system. In IEEE international symposium on industrial eletronis (pp ). Cortes, C., & Vapnik, V. (1995). Support-vetor networks. Mahine Learning, 20, Deb, K., Kang, S.-J., & Jo, K.-H. (2009). Statistial harateristis in HSI olor model and position histogram based vehile liense plate detetion. Intelligent Servie Robotis, 2(3), Dong, J.-X., Krzyzak, A., & Suen, C. Y. (2005). An improved handwritten hinese harater reognition system using support vetor mahine. Pattern Reognition Letters, 26, Duan, T. D., Du, T. L. H., Phuo, T. V., & Hoang, N. V. (2005). Building an automati vehile liense plate reognition system. In Proeedings of international onferene on omputer siene (pp ). Fan, X., & Fan, G. L. (2009). Graphial models for joint segmentation and reognition of liense plate haraters. IEEE Signal Proessing Letters, 16(1), Fran, V., & Hlava, V. (2005). Liense plate harater segmentation using hidden markov hains. In Leture notes in omputer siene, Vienna (Vol. 3663, pp ). He, Z., Liu, J., Ma, H., & Li, P. (2005). A new automati extration method of ontainer identity odes. IEEE Transation on Intelligent Transportation Systems, 6(1), Huang, Y.-P., Chang, T.-W., Chen, Y.-R., & Sandnes, F. E. (2008). A bak propagation based real-time liense plate reognition system. International Journal of Pattern Reognition and Artifiial Intelligene, 22(2), Huang, Y.-P., Chen, C.-H., Chang, Y.-T., & Sandnes, F. E. (2009). An intelligent strategy for heking the annual inspetion status of motoryles based on liense plate reognition. Expert Systems with Appliations, 36(Issue 5), ISO-6346, retrieved in URL < Jiao, J., Ye, Q., & Huang, Q. (2009). A onfigurable method for multi-style liense plate reognition. Pattern Reognition, 42(3), Kamel, M., & Zhao, A. (1993). Extration of binary harater/graphis images from graysale doument images. Graphial Models and Image Proessing, 55(3), Kim, K. B., Kim, M., & Woo, Y. W. (2007). Reognition of shipping ontainer identifiers using art2-based quantization and a refined rbf network. In Leture notes in omputer siene (Vol. 4432, pp ). Kumano, S., Miyamoto, K., et al. (2004). Development of a ontainer identifiation mark reognition system. Eletronis and Communiations in Japan-part 2, 87(12), Li, G., Zeng, R., & Lin, L. (2006). Researh on vehile liense plate loation based on neural networks. In Proeedings of the first international onferene on innovative omputing, information and ontrol, Beijing, China (Vol. 3, pp ). Mahini, H., Kasaei, S., Dorri, F., & Dorri, F. (2006). An effiient features based liense plate loalization method. In International onferene on pattern reognition, Hong Kong (Vol. 2, pp ). Martin, F., Garia, M., & Alba, J. L. (2002). New methods for automati reading of vlps (vehile liense plates). In The IASTED international onferene on signal proessing, pattern reognition, and appliations. Naito, T., Tsukada, T., Yamada, K., Kozuka, K., & Yamamoto, S. (2000). Robust lienseplate reognition method for passing vehiles under outside environment. IEEE Transation on Vehiular Tehnology, 49(6), Nakagawa, Y., & Rosenfeld, A. (1979). Some experiments on variable thresholding. Pattern Reognition, 11(3), Niblak, W. (1986). An introdution to digital image proessing. Englewood Cliffs, NJ: Prentie-Hall. Otsu, N. (1979). A threshold seletion method from gray-level histograms. IEEE Transations on Systems, Man and Cybernetis, 9(1), Sauvola, J., & Pietikinen, M. (2000). Adaptive doument image binarization. Pattern Reognition, 33(2), Shanthi, N., & Duraiswamy, K. (2010). A novel svm-based handwritten tamil harater reognition system. Pattern Analysis & Appliations, 13(2), Shi, X., Zhao, W., & Shen, Y. (2005). Automati liense plate reognition system based on olor image proessing. In O. Gervasi, et al. (Ed.), Leture notes on omputer siene, Springer (Vol. 3483, pp ). Tapia, E., & Rojas, R. (2005). Reognition of on-line handwritten mathematial expressions in the e-halk system An extension. In Proeedings of the eighth international onferene on doument analysis and reognition (pp ). Washington, DC, USA: IEEE Computer Soiety. Wang, T. H., Ni, F. C., Li, K. T., & Chen, Y. P. (2004). Robust liense plate reognition based on dynami projetion warping. In IEEE international onferene on networking, sensing and ontrol (pp ).