A Neural Network Based Bank Cheque Recognition system for Malaysian Cheques Ahmad Ridhwan Wahap 1 Marzuki Khalid 1 Abd. Rahim Ahmad 3 Rubiyah Yusof 1 1 Centre for Artificial Intelligence and Robotics, Universiti Teknologi Malaysia, Jalan Semarak, 54100 Kuala Lumpur Tel : +603-26913710 / 26154816 / 26154892 Fax : +603-26970815, Email: marzuki@utmkl.utm.my 2 Dept. of Computer Science and IT, College of Engineering, Universiti Tenaga Nasional, 43009 Kajang Tel: +603-89287245, Fax: +603-89263506, E-mail: abdrahim@uniten.edu.my Abstract Automated bank cheque processing is becoming an important technology due to the massive amount of cheques to be processed daily by banks. This paper presents the results from a research to design and develop a bank cheque recognition system for Malaysian banks. The system concentrates on recognizing the courtesy amount and date only. The system consists of 3 modules: detection module, extraction module and recognition module. The first involves locating the required image and the second extracting the required images. The third, the recognition module recognizes the numerical characters by first segmenting the images into individual characters and then recognizing it using a back propagation MLP neural network. The feature extraction technique used were a combination of geometrical and topological feature analysis and an improved moment invariant. The digit recognition module was trained with 6000 training data created from some sample bank cheques and the test data consists of 1000 digits taken from another sample of bank cheques. Results from the digit recognition module were satisfactory. However, recognition rates on actual bank cheques were quite unsatisfactory. We report the actual results and provide the reasons for such cases. Keywords: Automation, Neural Network, Bank Cheque Processing. Introduction There have not been many research done in Malaysia for automating cheque processing by banks. The Centre for Artificial Intelligence and Robotics (CAIRO) of Universiti Teknologi Malaysia have conducted some research towards the design of automated bank cheque processing system. Bank cheque processing is a challenging area for researchers working in document analysis and handwriting recognition. The challenge lies in the fact that bank cheques processing involve recognition of a number of components that are rather complex [1]. The backgrounds of the cheque and the pre-printed information on the bank cheques are further issues that need to be handled. In addition to that, Malaysian cheques allow the use of both Bahasa Melayu and English. Ready made systems from Europe, America or other countries are not suitable since the styles of handwriting of Malaysians differ from the actual target user. With over a million cheques processed daily, designing a Malaysian based bank cheque recognition system would be the best option. This paper reports the result of the design and development of the earlier proposed recognition system. However, it only concentrates on recognizing the courtesy amount and date only. Another research at CAIRO deals with the design of the recognition system for the legal amount. [8] The system developed consists of 3 modules: detection module, extraction module and recognition module. The first involves locating the required images (courtesy amount) and the second extract the required images. The third, the recognition module recognizes the numerical characters by first segmenting the images into individual characters and then recognize it using a back propagation MLP neural network. The feature extraction technique used is a combination of geometrical and topological feature analysis (GF46) and improved moment invariant (IMI). The algorithm for detection, extraction, segmentation and, recognition of the courtesy amount and date are described in this paper. This paper has been organised as follows. The next section presents some physical features of Malaysian bank cheques that have profound impact in the design of the system. Section 3 discusses the overall system design. The approach and methods for the images extraction, feature extraction and digit recognition are discussed in section 4. Section 5 presents some experimental results. Finally, section 6 concludes the paper.
Analysis of Malaysian Bank Cheques Malaysian banks use their own cheque template, which vary from one another in terms of their layout, background and format. Figure 1 below shows 2 typical examples: inside the rectangle line, and the date is normally written above the guideline. General locations of the courtesy amount and the date obtained from measurements of different cheques from the image database are also used as a guide. The following are some points that summarises the Malaysian cheques layout: courtesy amount location is always on the right side of the cheque courtesy amount location is also approximately at the centre of the cheque s height date is always located above the courtesy amount type A image A type B Figure 1 - Typical Cheque layout. The two cheques above differ in the layout of rectangles, guidelines and background. Due to that, different detection algorithms are required to detect the required courtesy amount. For this research, layout of type B is used to design the detection algorithms as most of the banks uses this layout though the position and style of the rectangles and the guidelines vary slightly from one bank to another. An appropriate detection strategy is used to overcome the difficulty in handling the variation. Figure 2 shows three different types of rectangle layout used for courtesy amount in Malaysian bank cheques. They can be summarised as follows: rectangle line (see image A and C) rectangle line separated horizontally (image B) rectangle box (image D) The research focussed on cheques of the types in images A, B, and C only. There are also three types of guidelines used in Malaysian bank cheques: single straight line (image C and D) two straight lines (image B) dotted line (image A) In the research, rectangle line and the guideline were used as the marker or cue to detect the location of the courtesy amount and date. The courtesy amount is normally written image B image C image D Figure 2 - Rectangle Layout By using the above three points, a strategy to detect the two subjects was formulated. Two search regions where the courtesy amount and the date reside were determined. The objective of using search region is to help the detection of the rectangle and the guideline by focusing the detection in specific area of the whole cheque.
Overall System Design. System Overview The Courtesy Amount and Date Recognition System (CADRS) consists of three main modules; detection module, extraction module, and recognition module. Each module is comprised of several sub-modules that perform specific tasks to complete the objectives of that module. This section discusses the process flow in the CADRS. The following section discusses the operation of each module and the algorithm used in detail. Cheque Image Detection Module CA and date images. The images are binary images where the foreground is set to black and the background is set to white. The CA and date images are then fed into the recognition module for recognition. This module implements several algorithms for the object joining, digit string segmentation, slant estimation and correction, feature extraction, and classification algorithms. The result of the recognition module is the representation of the CA and date in ASCII code. Approach and Methods Image Detection Module The Detection module consists of three sub-modules: the skew detection and correction. the origin and dimension estimation. the rectangle and guideline detection. Extraction Module Input Image Recognition Module Result Figure 3 - Process flow of overall system. Figure 3 shows the process flow in the system. The input image is processed in the detection module to get two sets of co-ordinates for the bounding rectangle and the guideline. The locations of the two markers are then used to clip out the courtesy amount (CA) image and the date image from the skew-corrected image. The result of the detection module is the CA image and the date image with the exact location of the guideline and the co-ordinates of the top-most edge of the rectangle. The guideline location is used in the line removal algorithm to remove the guideline from the date image. The top most edge of the rectangle is used as a control parameter in determining the search region for the date. It is possible that the search region for the date overlap a portion of the rectangle. Therefore, the control parameter will ensure that the search region for date will never overlaps the rectangle as in real cheques where the guideline is always above the rectangle. The extraction module cleans up the rectangle and the date image from noises such as the inner part of the bounding rectangle for the CA image and the guideline for the date image. The output of the extraction module is the noiseless CA Image and Top Edge Location Skew Detection and Correction Origin and Dimension Estimation Search Region Determination Rectangle and Guideline Detection Skew-Corrected Image Search Region Images Date Image and Guideline Location Figure 4 - Process flow in detection module. Figure 4 shows the process flow in the detection module. The input image may not properly aligned. The skew detection and correction sub-module will detect the skewed angle of the input image. Then the input image will be rotated in the opposite direction using the skewed angle detected earlier. The resultant image is then processed by the origin and dimension estimation sub-module to detect the four edges of the cheque image. The left and top edges give the offset of
the origin of the cheque with respect to the origin of the skew-corrected image in the horizontal and vertical axes respectively. The top left corner of the skew-corrected image is the origin of the skew-corrected image. The difference between the left and the right edges give the width of the cheque, whereas the difference between the top and the bottom edges give the height of the cheque. The origin`s offset and the dimension of the cheque are important in determining the search region of the CA and the date. The search region determination and the marker detection operations are performed twice. The operations will work on the rectangle followed by the guideline. The rectangular guide and guideline detection sub-module will detect the location of the two markers within the two search regions. The module works by scanning the search region left to right and top to bottom to find the horizontal lines in the regions. In rectangular guide detection, another scanning is performed to detect the two vertical lines in between the two horizontal lines. The CA image and the date image are extracted from the skew-corrected image based on the location of the rectangle and the guideline. Image Extraction Module Extraction module is a noise removal module where the input image is cleaned in four steps. The input image undergoes spatial filtering and binarization operations in the first cleaning step. In spatial filtering, the image is filtered using a median filter and then sharpened followed by smoothing to enhance the image. After performing spatial filtering, the image will be binarized using Otsu algorithm [3]. In the second step, the binary image is then processed using line removal algorithm to remove the preprinted lines such as the rectangular line for the rectangle image and the guideline for the date image. The resultant image then undergoes blob analysis operation to label every blob in the image and obtain the basic information of the blobs in the image. Examples of the basic information are area, height, width and position. The basic information obtained above is used in the subsequent operation for cleaning purposes. The area of the blob is used in the size filtering operation in the third cleaning step. The blobs that are smaller than the predefined ones are removed from the image. The predefined areas of the blobs must be carefully selected in order to avoid removing important information such as decimal point. In the fourth cleaning step, the image undergoes boundary-filtering operation. The blobs that are located at the border of the image are excluded in the next processing. The border blob is defined as the blob that has center point in horizontal and vertical axes located in the border region. Figure 5 shows the process flow in the extraction module, for the extraction operation of the CA and date images. CA/Date Image Spatial Filtering and Binarization Border/Line Removal Size Filtering Boundary Filtering Noise Free CA/Date Image Figure 5 - Process flow in Image extraction module. Feature Extraction and Recognition The Recognition module is the last module in the system that processes the CA/date image to get the results. The Recognition module consists of three main sub-modules that perform their own specific tasks. Figure 6 shows the process flow in the recognition module. CA/Date Image Blob Joining Blob Analysis Digit String Recognition Result Figure 6 - Process flow in recognition module. The Blob joining Sub-Module processes the input image to join broken objects in the image. The output of this sub-module is also an image. Blob analysis sub-module processes the input image to get individual blobs from the image. The blob basic information such as area, height, width, dimensions and location of each blob is extracted and stored for further use. The blobs are then sorted from left to right before proceeding to the next sub-module. The output of this sub-module is a string of blobs.
The Digit String Recognition Sub-Module processes the blob string from the Blob Analysis Sub-Module to recognise the digits. The blobs are fed one at a time starting from the left-most blob, to the sub-module for recognition. The process flow in the digit string recognition sub-module is shown in Figure 7. Blob String Load i th blob Identify for comma, dash and dot Yes Another blob? No END Performance of the Detection Module Several hundred samples of Malaysian bank cheques are used in this experiment. The cheques are generated from the more than 400 samples of original cheques. The orientation of the original cheques is varied to create skewed cheques for use in the experiment. The detection module detects the location of the rectangle and the guideline in the cheques. The experiments done have shown that the detection module successfully detects all the rectangles and the guidelines in all the samples. Figure 8 shows some examples of the CA and the date that has been detected by the detection algorithm. Identified? No Digit Recognition Yes Save Result Digit recognized? No Digit string segmentation Yes Figure 7- Process flow in digit string recognition sub-module. The blobs strings are sequentially fed into the digit recognition sub-module one at a time. Firstly, the blob is analysed for comma, dash and dot. If detected, the result is saved and another blob is loaded for analysis. If not, the blob is fed into a digit recogniser. A digit recogniser consists of two operations, which are the feature extraction operation followed by a classification operation by MLP neural network. If the blob is recognised, the result is saved and another blob is loaded if available. If the blob is not recognised, the blob is fed into digit string segmentation algorithm to be segmented and recognised gain. Figure 8 - Some examples of the detected CA and date. Performance of the Extraction Module The task of the extraction module is to remove the markers and other noises such as the border noises and the small blobs that scatter in the region of interest. Results from the experiment have shown that the extraction module has successfully remove all noises mentioned. The line removal algorithm is able to remove the rectangle and the guideline in all image samples used in the experiment. However, some of the noises are unable to be deleted due to their size and location. Figure 9 shows some examples of the CA and date that have been extracted by the extraction module. Experimental Results and Analysis The Determination of the Search Region Parameters Initially the search region parameters are calculated by manually measuring the location of the rectangle and the guideline for some samples using the imaging software. These parameters are then used in the automatic detection of the rectangle and the guideline using the detection algorithm. In both cases, the global search region parameters are set slightly below the calculated values to ensure the coverage of the search region. Figure 9 - Some examples of the extracted CA and date. MLP Training This section discusses the performance of the feature extraction methods for the unconstrained digit recognition system. The feature extraction method is the combination of GF46 and IMI. The training samples consist of 7000 characters where 6000 characters are used as the training
samples and the remaining 1000 characters are used as the testing samples. The database consists of 14 character sets that are digit 0 to 9, R, M, RM and slash. The characters are obtained from 15 persons who are asked to write the CA and the date in a form. Each person has to write 54 CAs and 24 dates. Individual characters are then extracted from the form to build the database. Figure 6.3 shows the samples of the characters used in the experiment. Table 2 - Test Set Characters Occurrence 0 86 1 86 2 86 3 70 4 70 5 70 6 70 7 70 8 70 9 86 R 50 M 50 RM 50 Slash 86 Total = 1000 Slash Figure 10 - Samples of characters used. There are several types of slash that exist in the database. Figure 10 shows three types of slash. The first slash is almost like digit 1 except the size of the slash is bigger than digit 1. However, due to the fact that characters are written without any constraint, the size of the slash sometimes equal to the size of the digit 1 or even smaller. This causes problem to the recognition module to distinguish the slash and the digit 1. Table 1 and Table 2 show the occurrence of each character in the training and the test sets respectively. Table 1- Training Set Characters Occurrence 0 604 1 584 2 521 3 505 4 430 5 435 6 430 7 459 8 430 9 558 R 150 M 150 RM 150 Slash 594 Total = 6000 The experimental result of the training of the MLP using the GF46+IMI is summarized in Table 3 below. Table 3 - MLP training information and results Feature extraction method GF46 + IMI No. of input neurons 55 No. of hidden neurons 55 No. of output neurons 14 Learning rate, η 0.001 Momentum, α 0.08 Minimum error reach 0.1 Training epoch 15057 Recognition rate(%) 98.60 Testing the Cursive Handwritten Digit Segmentation Algorithm One hundred and fifty connected digits are used in this experiment. The samples are obtained from the same source as the MLP s training samples. Figure 6.4 shows the examples of the connected digits used in this experiment. Figure 11 - connected digits samples.
The result of the experiment gives 108 correct answers out of 150 connected digit samples or 72% of the input samples. There are several causes of the incorrect answers in the experiment. From the results, it is observed that, most of the problems are caused by the failure of the recognition module to properly recognise the digits. This will certainly give incorrect recognition of the input digits. For example, the segmentation algorithm has correctly segmented the input image but the recognition module fails to recognise the slices correctly. Some errors also occur during the slicing mechanism in the segmentation algorithm. The recognition module incorrectly classifies the initial slice of the slicing mechanism. Further slicing might lead to the correct answer but the initial answer is still regarded as the final answer and the change of the recognition output caused the slicing mechanism to stop. Sometimes, the slice also converges to other digit during the slicing mechanism. In some cases, the slice could not be recognised until the end of slicing mechanism or the recognition module recognises the connected digits as a single digit. The small number of the training samples used that is only 6000 samples might also cause the failure of the recognition module. Apart from the failure of the recognition module, the segmentation module also contributes to the misclassification of the input samples. The segmentation algorithm depends mainly on the vertical transition histogram of the connected digit. For this reason, it is possible that the segmentation algorithm gives the wrong segmentation points. For example, the digit 0 that is broken at one part of it will cause the segmentation algorithm concludes that that part is a link. Digit 4 also gives problems to the segmentation algorithm because sometimes it looks like two digits that are connected with a link. However, from the observation, the performance of the segmentation algorithm is considerably good. Analysis of the Overall System The CADRS has been tested on several Malaysian bank cheques to investigate its performance. The CADRS is successful in term of detecting and extracting the CA and the date from the cheques. However, in term of recognition capability, the CADRS performance is not satisfying. In most cases, the CADRS could not recognize all the digits and other characters composing the CA and the date. At least one of the digits or characters is wrong while the others are perfectly recognized. In certain cases, the CA or the date image contains noises that cause the CADRS to give a wrong answer. From the observations during the experiment, there are six main reasons that cause the failure of the CADRS. Those causes of failure are the recognition failure, the segmentation failure, the extraction failure, blurred image, dark image and noisy image. The first and the second causes have been discussed in the previous section. Blurred image causes the important information such as the digit, dot, slash or dash is removed or broken during extraction process. Dark image produces significant noises and some of the noises are overlapped with the CA and the date. Binarizing the dark image causes some of the background objects to be included in the output image and hence cause the recognition module to give incorrect answers. Noisy images such as images with stamping and other significant noises that are overlapped or located near to the CA and the date also contribute to the failure of the CADRS to recognise the CA and the date. The reason is the same as the dark image. Extraction algorithm also sometimes fails to produce proper image for further processing. The broken foreground objects, significant noises and loss of important information are examples of the failure in the extraction module. The binarization algorithm might not be able to cover the wide range of image types. The brightness and the contrast of the input images are different from each other. Therefore, the input images would be sometimes outside the range of the capability of the binarization algorithm. This will produces output images that suffer the previously mention problems. Conclusion In this paper, the design and development of the courtesy amount and date of Malaysian bank cheques was reported. The system has successfully implemented the detection and extraction module of the system but the recognition results were not very satisfactory. Possible causes of failure have been discussed to point out improvements that can be made and pitfalls that should be avoided in future work. References [1] M. N. Sulaiman and M. Khalid. 2000. Grouping Single Slice Mechanisms to Segment and Recognize Off-line Cursive Handwritten Courtesy Amounts of Malaysian Bank Cheques, CAIRO report. [2] J. Ramesh, K. Rangachar, Schunk B. G. 1995. Machine Vision, New York: McGraw Hill. [3] N. Otsu. 1979. A Threshold Selection Method from Gray-Level Histograms, IEEE Transactions on Systems, Man, and Cybernatics, Vol. 9, No. 1, pp 62-66. [4] G. Dimauro, S. Impedovo, G. Pirlo and A. Sazlo, 1997. Automatic Bankcheck Processing: A New Engineered System, Automatic Bankcheck Processing pp5-42, World Scientific Publ. Co. [5] W. N. Lim, 2000. Design of an Automated Data Entry System for Hand-Filled Forms, Universiti Teknologi Malaysia: Master Thesis. [6] S. T. Welstead, 1994. Neural Networks and Fuzzy Logic Applications in C/C++, New York: John Wiley & Sons, Inc. [7] K. S. Yap, 1999. Design of an Intelligent Vehicle License Plate Recognition System, Universiti Teknologi Malaysia: Master Thesis. [8] M. N. Sulaiman and M. Khalid, 2001. An Offline Segmentation Technique for the Processing of the Courtesy Amount of Malaysian Bank Cheques, ELEKTRIKA, Journal of Electrical Engineering. Vol. 4, Num. 2, pp 1-13.