A Computer Vision System for Automated Container Code Recognition

Transcription

1 A Computer Vson System for Automated Contaner Code Recognton Hsn-Chen Chen, Chh-Ka Chen, Fu-Yu Hsu, Yu-San Ln, Yu-Te Wu, Yung-Nen Sun * Abstract Contaner code examnaton s an essental step n the contaner flow management. To date, ths step s mostly acheved by human vsual nspectons, whch are however tme-consumng and error-prone. We hence propose a new computer vson system for automated contaner code recognton. The proposed system conssts of model constructon and code recognton stages. In the model constructon stage, we frst ncorporate a locally thresholdng method wth pror knowledge of code character geometry to segment the code characters, ncludng Englsh characters A-Z and numerc characters 0-9, from a tranng set of contaner mages. Wth the segmentaton results of each character, we subsequently construct ts Egen-feature model usng the prncpal component analyss (PCA). In the recognton stage, the code characters are frstly segmented from the gven contaner mage. Each segmented character s then recognzed by fndng the best matched Egen-feature model that mantans the mnmal PCA reconstructon error of the character appearance. Experments showed that the proposed method acheved the code recognton wth a hgh recognton rate and lttle recognton tme for each mage automatcally. Overall, our proposed system has great potental for mprovng the effcency of contaner termnals as well as enhancng the contaner management. Index Terms code recognton, locally thresholdng, character geometry, Egen-feature model, prncpal component analyss C I. INTRODUCTION ONTAINER code examnaton s a crtcal step n the procedure of contaner securty and flow management. Manuscrpt receved December 28, Ths work was supported under contract CSIST-0839-V201 (99) from the Chung-Shan Insttute of Scence and Technology, Tawan, R.O.C. and utlzed the shared facltes supported by the Program of Top 100 Unverstes Advancement, Mnstry of Educaton, Tawan, R.O.C.. H.C. Chen s wth Department of Computer Scence and Informaton (e-mal: wale1212@gmal.com). C.K. Chen s wth Department of Computer Scence and Informaton (e-mal: heartthrob.ka@gmal.com). F.Y. Hsu s wth Department of Computer Scence and Informaton (e-mal: cash.barca@gmal.com). Y.S. Ln s wth Arborne Electronc Secton, Electronc System Research Dvson, Chung-Shan Insttute of Scence and Technology, Taoyuan, Tawan, R.O.C. (e-mal: dostoevosky@yahoo.com.tw). Y.T. Wu s wth Department of Bomedcal Imagng and Radologcal Scences, Natonal Yang-Mng Unversty, Tape, Tawan, R.O.C. (e-mal: ytwu@ym.edu.tw). Y.N. Sun s wth Department of Computer Scence and Informaton (correspondng author to provde phone: ext ; fax: ; e-mal: ynsun@mal.ncku.edu.tw). Nowadays, human vsual nspecton, whch s laborous and error-prone, remans the most common approach n port gates for checkng the nformaton of contaners. Rapdly and relably examnng the contaner code, whch s helpful for ncreasng the economc effcency of contaner termnals, hence becomes mportant. For ths purpose, our research ams to develop a computer vson system for automated contaner code recognton. The proposed computer vson system contans two techncal parts, whch are code segmentaton and recognton. Up to present, there have been several knds of methods developed for mage segmentaton, and a popular one s the thresholdng-based method [1] [5]. Otsu s thresholdng method [1] s a typcal one of global thresholdng. It dvdes the mage nto foreground and background regons based on the optmal computaton wth ntensty hstogram of the entre mage. Bascally, the Otsu s method s easy to mplement and effcent n computaton. However, t may not be suted for the mages whose ntensty hstogram s not bmodal. On the other hand, locally thresholdng methods estmate a threshold value for every pxel n the mage based on the ntensty dstrbuton of ts neghborhood. Therefore, they are less constraned by the global ntensty dstrbuton of the mage. Pradhan et al. [3] proposed a locally thresholdng method wth adaptve wndow to segment objects from the mages under uneven lghtng condton. Moreover, the locally thresholdng technques were employed n the applcatons of medcal mage analyss for segmentng tssues of nterest, such as the works of Zhang et al. [4] and Peng et al. [5]. Overall, the thresholdng-based methods segment objects manly based on the ntensty value. Hence, rrelevant regons wth ntensty values smlar to the target objects tend to be ncluded n the thresholded results. To resolve ths problem, features other than ntensty value, should be taken nto account n the segmentaton protocols. For example, Sankaran et al. [6] ncorporated the geometry nformaton of cells, ncludng area and shape, nto the segmentaton process and obtaned satsfactory segmentaton results. As to the character recognton part, the correlaton coeffcent s commonly used to evaluate the smlarty of grey-level dstrbuton between two segmented character regons [7]. However, the correlaton coeffcent s senstve to the pose varaton between the characters. To overcome ths problem, Wakahara et al. [8] proposed a character recognton algorthm based on the non-rgd regstraton. Unfortunately, ther method suffers a lot from computatonal cost. The computaton problem and recognton rate can be sgnfcantly mproved by carefully selectng the mage space of character recognton. Moreover, the egen space s a

2 wdely adopted feature space for character recognton. Park et al. [9] and Manjunath et al. [10] recognzed characters based on the Egen-feature space and acheved good results. These lteratures manly focused on character recognton, and however, less dscussed the detals of segmentaton. In ths paper, character segmentaton and recognton are both addressed for automated contaner code recognton. The features of proposed method are descrbed below. Frst, we ncorporate a locally thresholdng method wth pror knowledge of character geometry to segment the characters from contaner mages automatcally. Second, the Egen-feature space, whch can better accommodate the noses and deformaton n character appearances, s thus adopted to recognze the segmented code characters. Thrd, we employ the generaton rule of code characters n the recognton protocol to reduce the ms-recognton rate. Expermental results showed that our proposed system could acheve promsng results wth a hgh recognton rate and low computatonal tme. (a) II. OVERVIEW OF THE PROPOSED METHOD The proposed method conssts of model constructon and code recognton stages. In the model constructon stage, we frstly segment the code characters from a tranng set of contaner mages by ncorporatng a locally thresholdng method wth pror knowledge of code character geometry. We subsequently construct an Egen-feature model for each code character based on ts segmentaton results on the tranng mages. Gven a contaner mage n the code recognton stage, we segment the code characters usng the above-mentoned segmentaton protocol. Each segmented character s then recognzed by fndng the best matched Egen-feature model that mantans the mnmal reconstructon error of character appearance. The detals of the proposed method are descrbed n the followng sectons. (b) (c) III. MODEL CONSTRUCTION A. Locally Thresholdng Segmentaton Three ntensty propertes of contaner mage are used to segment the contaner code characters. Frst, the code characters are wth smlar ntenstes to each other. Second, the ntensty contrast of characters wth respect to ther surroundngs s large. Thrd, there s possble uneven sunlght llumnaton n the mage. Based on these observatons, we hence select the locally thresholdng method [11] to segment the contaner code characters. The locally thresholdng method s desgned based on the ntensty dstrbuton of the neghborhood of every ndvdual pxel n the mage. At frst, we transform the nput mage from RGB color to grayscale. Then, we slde a fxed-szed wndow over the mage. For each pxel at the center of the wndow, we calculate the average ntensty from the neghborhood pxels covered by the wndow. The examned pxel s classfed as the foreground (wth zero ntensty n Fg. 1(c) and Fg. 1(d)) f ts ntensty s lower than c percent of the average ntensty, and s assgned as the background otherwse. After ths step, we can obtan the segmented (d) Fg. 1. Locally thresholdng segmentaton: (a) and (b) the orgnal contaner mages; (c) and (d) the segmentaton results of (a) and (b) respectvely. foreground denoted as FG. We also employed the ntegral mage technque to speed up the threshold computaton. The length of the sde of the sldng wndow was set to 1/30 of the mage wdth, and c was assgned as 80. The aforementoned thresholdng crteron s only sutable for the contaner mage, n whch the code character regons are darker than the surroundng regons. In some cases, the ntenstes of code characters, however, may be hgher than those of surroundng regons. An erroneous segmentaton may thus occur such that the code characters are msclassfed nto the background. Therefore, a confrmaton strategy s desgned to compensate ths lmtaton. At frst, we apply the Otsu s method [1] to the orgnal grayscale mage and obtan two clusters of pxels denoted as C1 and C2. Snce the contaner occupes most parts of the mage, t s supposed to be ncluded n the larger szed cluster (assumed to be C1). On the other hand, as the code characters and markers are darker or brghter than the surroundngs n ntensty (.e. hgher n

3 ntensty contrast) and smaller n sze n the contaner mage, the code characters are contaned n the other cluster C2. We subsequently calculate the average ntensty on the orgnal grayscale mage for C1 and C2 respectvely. If the dfference between average ntenstes of FG and C1 s smaller than the one between FG and C2, an error of local segmentaton s detected. We then change the crteron of the local thresholdng and process the mage agan. A pxel s classfed nto the foreground f ts ntensty s hgher than the average ntensty of ts neghborhood, and nto the background otherwse. Based on the confrmaton process, we effcently obtan acceptable segmentaton results of the code characters n ether black or whte, as shown n Fg 1. (a) B. Character Extracton Based on Geometry Features The thresholdng result however contans a number of non-code character regons. As the code characters are supposed to be wth certan geometry propertes, e.g., wth the sze n a certan range, we then estmate for each regon several geometry parameters, ncludng area, crcumference, wdth, and heght. If the values of parameters of a regon are out of the emprcally determned ranges, t s fltered out, and otherwse s preserved. After that, those rrelevant regons such as the levers on the contaner door can be easly removed, as shown n Fg. 2(a). Next, t s further consdered that the code characters are located close together. We thus desgn a dstance-based groupng algorthm to pck out the code character regons: Step 1. Step 2. Step 3. Step 4. Step 5. Step 6. Label all the regons as FALSE and set ther groupng states to UNGROUPED. Label an arbtrary FALSE regon as TRUE. If there s any FALSE regon satsfyng the followng two rules: (1) The dstance between ts center and the centers of any TRUE & UNGROUPRED regons s smaller than one-hundred pxels, (2) The dfference of x- or y-axs coordnates between ts center and the center of any TRUE & UNGROUPRED regons s smaller than eght pxels, then the examned regon s labeled as TRUE. Go to step 3 untl there are no more TRUE regons found. Group these TRUE & UNGROUPRED regons together, and set ther groupng state to GROUPED. Go to step 2 untl there are no more groups found. After the groupng process we can obtan several groups as shown n Fg. 2(b). The code character regons can be dentfed by the largest szed group ndcated by the red rectangle, and each ndvdual character can also be readly extracted (see Fg. 2(c)). C. Egen-feature Models Usng the proposed segmentaton method, we can automatcally extract the character regons from the tranng (b) (c) Fg. 2. Character extracton from Fg. 1(c): (a) the flterng result by geometry features; (b) the groupng result by character dstance; (c) the character extracton result. mages. For each Englsh and numerc character, we construct an Egen-feature model based on the PCA to characterze ts appearance varaton among dfferent tranng mages. In the model constructon process, ts bnary segmentaton results are frst algned to each other to elmnate the pose dfferences. The algnment results then serve as the tranng samples, denoted as X = (x 1, x 2,, x,, x m-1, x m ), where m s the number of tranng samples. Each tranng sample x s represented by a one-dmensonal vector (x 1, x 2,, x j,, x n-1, x n ) T, where x j s the ntensty value of the j-th pxel n the tranng sample and n s the number of pxels. Next, we average the m samples to obtan the mean vector x, and then calculate the covarance matrx C mplyng the varance between each tranng sample and the mean vector. 1 m T And C s gven by ( x x)( x x). We subsequently m 1 solve the egenvalues (v 1, v 2,, v,, v n-1, v n ) and ther correspondng egenvectors (u 1, u 2,, u,, u n-1, u n ) from the covarance matrx. Gven the egenvalues n descendng order, the dmenson of the egen features s reduced by preservng the frst t components, subject to ( t v 1 n 1 v ) At last, the resultng Egen-feature model, characterzed wth the mean vector and egenvectors, can be utlzed to dentfy the character appearance. By applyng ths constructon process to all the characters ncludng 0-9 and A-Z, we can consequently obtan a set of Egen-feature models and apply them to the subsequent recognton stage.

4 IV. CODE RECOGNITION In the recognton stage, we frst segment the code characters from a gven mage. Each unknown character s then compared to all Egen-feature models, and s recognzed by fndng the best matched model whch mantans the mnmal PCA reconstructon error of the character appearance. The detals of character recognton algorthm are delneated below. Step 1. Input an unknown character C unknown (.e., a one-dmensonal ntensty vector) and the set of Egen-feature models. Step 2. Select an Egen-feature model and project the character vector C unknown from the mage space u1 onto the model space, u 2 y. [ Cunknown x] ut Step 3. Reconstruct the character appearance through a u1 back projecton process, u 2 xˆ. y x ut Step 4. Calculate the reconstructon error, Cunknown xˆ. Step 5. Record the value of the error, and then go to step 2 untl all the Egen-feature models are examned. Step 6. Fnd the character wth the mnmal reconstructon error as the recognton result of C unknown. An example of the character recognton s demonstrated n Fg. 3. The unknown character s shown n Fg. 3(a), and the reconstructed appearances usng the Egen-feature models of 2, 3, and 7 are dsplayed n Fgs. 3(b)-(d), respectvely. It s observed that usng the correspondng Egen-feature model to reconstruct the character appearance can lead to the smallest dstorton. After all the characters are recognzed, we can obtan the recognton result of the entre contaner code. On the other hand, to ncrease the recognton rate of the proposed system, we further ncorporate the nformaton of contaner code generaton rule nto the recognton process based on the standard of ISO 6346 [12]. The generaton rule provdes the nformaton that a character s supposed to be matched to ether numerc or Englsh characters, as shown n Table I. For example, the frst three characters of the contaner code represent the owner d n Englsh. Hence, only Englsh characters are taken as feasble solutons for ther recognton. Consequently, the ms-recognton between the numerc and Englsh characters wth smlar appearance, e.g., 1 and I, can be easly avoded. V. RESULTS AND DISCUSSION The followng experments conssted of three parts ncludng segmentaton accuracy, recognton accuracy and computatonal performance evaluatons. The valdaton work ncluded 94 contaner mages and was performed on a T (a) (b) (c) (d) Fg. 3. Example of the character recognton: (a) the unknown character; (b)-(d) the reconstructed appearances usng three dfferent Egen-feature models. TABLE I CODE GENERATION RULE FOR THE PROPOSED RECOGNITION ALGORITHM 1 st Row Code characters Specfc meanng Character type The frst three characters Owner d Englsh character The fourth character Category dentfer Englsh character The last one character Check dgt Numerc character The other characters Seral number Numerc character 2 nd Row Code characters Specfc meanng Character type The frst two characters Sze code Numerc character The second character Type code Englsh character The last character Type code Numerc character desktop PC wth 3.0GHz Intel Core 2 Duo E8400 processor, wndows XP Professonal, and 2GB memory. A. Segmentaton Accuracy Evaluaton To evaluate the segmentaton accuracy, the proposed method was appled to the 94 valdaton mages. A successful segmentaton n an mage has to satsfy that every character (totally 15 characters n a contaner code) s ncluded n the segmentaton result. The success rate n these segmentatons was 88.3 % (83/94), ndcatng a satsfactory accuracy of the proposed segmentaton method. B. Recognton Accuracy Evaluaton In ths experment, the recognton accuracy wth respect to ndvdual character and entre code was valdated. At frst, we counted the number of total occurrences of each character n the 83 successfully segmented mages. For each ndvdual character, ts recognton success rate (RSR) was gven by the proporton of the number of correct recogntons to the total occurrences, as lsted n Table II. The average RSR of all the characters was %. Moreover, t was observed that the RSRs of two characters, B and N, were much lower than the others. Ths may be because ther sample szes are very small. If more testng mages are ncluded, the RSRs are supposed to ncrease. Moreover, we also valdated the recognton accuracy wth respect to the entre contaner code. We calculated the number of mages, n whch the code wth 15 characters was correctly recognzed. 73 successful recogntons were obtaned among these 83 mages, that s, % recognton rate. If more data can be ncluded n the future experments, the recognton rate s expected to be further mproved. C. Computatonal Performance Evaluaton In ths experment the computatonal performance of the proposed system was also evaluated. For each of the 83 mages, we measured the computatonal tme n code segmentaton and recognton, and obtaned an average computatonal tme less than 1 second. Overall, the proposed system can acheve the segmentaton and recognton of contaner code effcently.

5 VI. CONCLUSION In ths paper we have proposed a computer vson system for automated contaner code recognton. We employed the locally thresholdng method and character geometry features to automatcally segment the contaner code from the gven mage. In the segmentaton process we further desgned an ntensty-based strategy to compensate the error of locally thresholdng segmentaton. In addton, the Egen-feature models were utlzed n the code recognton process for accommodatng mage noses and deformaton n character appearances. And, the generaton rule of contaner code was taken nto account to avod ms-recogntons between Englsh and numerc characters wth smlar appearance. The expermental results showed that our proposed method can automatcally acheve code recognton wth a hgh average RSR. In the future research, more contaner mages wll be ncluded to fne-tune the models n ths system for mprovng the recognton rate. Moreover, the recognton of markers n contaner mage (e.g., trangular or rectangular sgns) wll also be conducted for constructng a more complete examnaton system. TABLE II RECOGNITION SUCCESS RATE (RSR) OF INDIVIDUAL CHARACTER Character Number of Number of correct RSR occurrences recogntons % % % % % % % % % % A % B % C % D % E % F % G % H % I % J 0 0 None K % L % M % N % O % P % Q 0 0 None R % S % T % U % V 0 0 None W % X % Y % Z % Average RSR % [2] Gonzalez, R.C., and Woods, R.E., Dgtal Image Processng. 3rd ed, Pearson Ed. New Jersey, USA, 2010, pp [3] Pradhan, S.S., and Nanda, P.K., Adaptve thresholdng based mage segmentaton wth uneven 11 lghtng condton, IEEE Regon 10 Colloquum and the thrd ICIIS 2008, pp [4] Zhang, J., Yan, C.H., Chu, C.K., and Ong, S.H., Fast segmentaton of bone n CT mages usng 3D adaptve thresholdng, Computers n Bology and Medcne, 40, 2010, pp [5] Peng, J.Y., and Hsu, C.N., Adaptve local thresholdng for fluorescence cell mcrographs, Techncal Rep. No. TR-IIS , Nov. 11, [6] Sankaran, P., & Asar, V.K., Adaptve thresholdng based cell segmentaton for cell-destructon actvty verfcaton, n 2006 IEEE Conf. AIPR, pp. 14. [7] Ozbay, S., and Erceleb, E., Automatc vehcle dentfcaton by plate recognton, Proceedngs of World Academy of Scence, Engneerng and Technology, 2005, pp [8] Wakahara, T., and Odaka, K., Adaptve normalzaton of handwrtten characters usng global/local affne transformaton, IEEE Trans. Pattern Analyss and Machne Intellgence, vol. 20, no. 12, pp , Dec [9] Park HS, Km SY, Lee SW, Gray-scale handwrtten character recognton based on prncpal features. SPIE vol. 3027, pp , [10] Manjunath AVN, Hemantha KG, and Noushath S., Two-dmensonal matrx prncpal component analyss useful for character recognton, ICIA, pp , [11] Bradley, D., and Roth, G., Adaptve thresholdng usng the ntegral mage, Journal of Graphcs, GPU, & Game Tools, vol. 12, no. 2, pp , [12] Contaner Handbook: Cargo loss preventon nformaton from German marne nsurers, ch [Onlne]. Avalable: tra_03_04_00.html REFERENCES [1] Otsu, N., A threshold selecton method from gray-level hstograms, IEEE Trans. Systems, Man and Cybernetcs, vol. 9, no. 1, pp , Jan