Automatic Mail Sorting For Hand Written Postcodes

Transcription

1 Automatic Mail Sorting For Hand Written Postcodes Naeem Abbas, M Tahir Qadri, M Tayyab, Shoaib Mughal, and M Aamir Khan Electronic Engineering Department, Sir Syed University of Engineering and Technology, Main University Road, Karachi-75300, Pakistan. Abstract drabbas.naeem@gmail.com, mtahirq@hotmail.com, tayyab.engr1@gmail.com, mughalshoaib@hotmail.com, aamirkhan_129@hotmail.com Most of the postal systems in many areas are still manually operated for mail sorting and processing. It has been observed that the manual sorting and processing has many disadvantages such as human errors, more processing time and manpower required. In this paper, we intended to propose Automatic Mail Sorting (AMS) system which would eliminate human errors, reduce the mail sorting time and requires minimum man power. We proposed fully automated processing technique involving Optical Character Recognition (OCR) approach. The emergence of OCR has provided a solution for automatic postal address reading. In this technique, a camera is used to capture the images of postal envelops and then the OCR method is applied for the recognition of postal address. The important contribution in this paper is to compare different OCR techniques in order to effectively recognize the hand written postcodes. Keywords: Automatic mail sorting, postal automation, optical character recognition, image processing. 1 INTRODUCTION The postal system is important in the development of modern mail transportation. A brief history on the development of postal services is quoted from [1]. The broad development of mechanization in postal operations was not applied until the mid 1950 s [2]. The translation from mechanization to automation of the U.S Postal Services (USPS) started in 1982, when the first optical character reader was installed in Los Angles. The introduction of computers revolutionized the postal industry and since then the pace of change has accelerated dramatically [1]. Postal services are going to remain an integral part of the infrastructure for any economy. For example, recent growth in e-commerce has caused an increase in international and domestic posts. To maintain the role of mail as one of most efficient means of business communication, postal services need frequently improvements in their organizational and technological infrastructure for mail processing and delivery [3]. In the beginning, we observe that manual sorting cannot meet the requirements of modern post, not only because of its time-consuming labor, but also because of its costly manpower [4-5]. So it is required that the manual processing or sorting of the mails must be transformed into fully automated processing methods. The method we are adapting for this system is Automated Mail Sorting by using Image Processing technique involving Optical Character Recognition (OCR) respectively [4]. The emergence of this method using OCR has provided a solution for automatic postal address reading [4]. In this method, a camera is used to capture images of mail items and then OCR are applied for the recognition of postal address. Automatic mail sorting is the most popular method of automatic addressing reading in the automation of Postal systems. Traditional automatic mail sorting systems have focused on the identification of the postcode on the envelope, since the correct recognition of the postcode determines the city and state of the destination address. However, recent research shows that the recognition of the actual postal address can improve the accuracy of automatic sorting because the information contained in the postal address can be used to verify whether the postcode has been correctly recognized [4]. For examples, the names of the city and street in the postal address can be matched to those indicated by the postcode. In another technique [5] automatic mail sorting machine, mail feeder, image scanner, mail stacker, real-time control system, postcode and address recognition ( i.e OCR) are basic modules. The function of the mail feeder module is to feed the letters to machine one by one, which generates a mail stream in the transport ( conveyer) belt. As a mail passes by the CCD (charged

2 coupled device) camera in the image scanner module, its image is captured and sent to the postcode and the address recognition unit (OCR). The real-time control system traces each one in the mail stream, and controls the mail to its corresponding stacker according to the result of the postcode and address recognition unit. In another technique [6] attempt has been made to recognize postal letter sorting system based on PIN code. The development of exact barcode to each PIN code is introduced for sorting the postal letters automatically. The letters travel up to barcode scanner, if the PIN code is written on the envelope, equivalent barcodes are developed and printed on the each of the postal letters in the bottom line by the barcode printer, if PIN codes are not written on the envelope, by comparing a lookup table of place and PIN code, equivalent barcode to PIN code will be printed on the backside of the postal letters. Then the automatic letter sorting systems is employed to separate the letters using the barcodes. The conveyer belt having on/off states enables the letter to travel up to the exact location of the drop box. This barcode method yields 99.5% accuracy. Another technique [7] describes the analysis of Automatic Sorting of handwritten mail involving address segmentation into words and characters, Optical Character Recognition (OCR) of postcode characters and feature extraction to match handwritten words from the database. In this technique investigating automatic location and identification of the postcode characters, segmentation of handwritten addresses into words and lines, extraction of features from the handwritten words and verification of the address using the postcode and extracted features from a database of postcodes and addresses. In this paper [8] Automated Postal System was proposed which would reduce the mail sorting time ruling out the human errors. Automatic Mail Processor (AMP) scans a mail and interprets the fields of the destination address such as PIN code, City name, Locality name and the Street name. The interpreted address is converted into Delivery Point Code (DPC) which is a 12 digit number. The DPC is printed on to the mail in the form of a barcode which can be read by a machine. By converting the destination address into a barcode, all of the future sorting processes can be achieved by using a mechanical machine sorter, which can sort the mails according to the barcode present on them. 2 SYSTEM MODEL The overall Automatic Mail Sorting (AMS) system can be divided into four parts. The first part consists of the envelop detection by using image acquisition using MATLAB. The second part is the postcode extraction by orientation of an envelope and the selection of the target area where the address and post code could be present by using image cropping technique. The third part is the post code recognition which is performed by OCR algorithms. Finally, the last part is the mail sorting in which we are sorting the mail area wise. The overall block diagram of the proposed system is illustrated below. Mail Sorting OCR Algorithms for Post Code Recognition Envelope Detection & Image Acquisition Post Code Extraction 2.1 ENVELOPE DETECTION AND IMAGE ACQUISITION Fig 1: Block diagram of the proposed model The first step of the proposed model is to detect an envelope and process the image which is captured from the camera by using MATLAB software. To achieve this the IR sensors are used to detect an envelope and the stationary camera in order to capture the image. In the first stage the

3 system work in a way that the envelopes are coming on the conveyer belt, when the envelope enter into the range of IR sensors, the IR sensors detect an envelope and system generate interrupt signal. Due to interrupt signal the conveyer belt is stopped and immediately camera capture the image of an envelope. This image is further processed by converting the image from RGB (Redpost code. Fig 2 Green-Blue) to greyscale in order to further process the image for extraction of the shows the image of an envelope which is captured from the stationary camera. Fig 2: Image of an Envelope The next section explains the procedure of extraction of the postcode. 2.2 EXTRACTION OF POSTCODE This section describes image cropping and extraction of the postcode. First we capture the image (envelope containing address and postcode) with the help of the camera. The next step is to select the desired area and exclude the unnecessary information by image cropping. The purpose of image cropping is to identify the region where the actual information present and crop the desired information. The complete process is shown in the figure3. (a) (b) (c) Fig. 3: (a) Extraction of Address, (b) Black background, (c) Extraction of Postcode The next part explains the procedure of postcode recognition. 2.3 POSTCODE EXTRACTION USING OCR ALGORITHM After extracting the postcode area, the third step is the extract the handwritten postcode numbers by using Optical Character Recognition (OCR) algorithm. In OCR algorithm each digit of extracting postcode is correlated with the predefined database. This database is derived from the original MNIST database available at [9]. In this database there are 10 (0-9) human written numeric

4 digits and each digit has 100 samples with different styles and patterns so total samples in this database is The database is illustrated in fig 4. Fig. 4: Human Written Numeric Database Figure 4 shows total 100 samples, in which each digit has 10 samples with different styles and patterns but practically we are using 1000 samples. In this paper OCR algorithm is implemented by using four different techniques to recognize the human written postcode s numeric digits. The first technique is simple correlation, the second technique is mean image correlation, the third technique is mean filter based correlation and the fourth technique is principal component analysis (PCA). A. SIMPLE CORRELATION In simple correlation technique, first we took the image of the first human written numeric digit of the extracted postcode which can be any numeric digit called test image. Then correlated this test image with the predefined human written numeric database to recognize the human written postcode s numeric digits and then add all the correlated results of this test image with the numeric database of 1000 samples. By addition of all individual correlation results, the digit is recognized which matches more accurately with the digit in the database and this process continues till the last digit of the extracted postcode. There are two ways to execute a simple correlation technique. The first is separate correlation and the second is mixed correlation. In separate correlation technique the first test image (first digit of the extracted post code) correlated with the 1000 samples of the predefined database at a time and then add all the correlated results for recognition but in the mixed correlation technique the first test image (first digit of the extracted postcode correlated any 10 samples of the predefined database like 901:910 then again the same test image correlated with the next 10 samples of the predefined database which is 911:920, this process continue till 1000 samples and then add all the correlated results for recognition. Accuracy is very important for the correct recognition and it is determined by the addition of the correlated results because after addition recognize the digit who more accurately match. For example the test image digit is 6 and after correlation and the addition of the correlated results if the accuracy of the test image 6 to 6 in the database is more than any other digit of the database then it recognize 6 but if the accuracy of the test image 6 is more with any other digit of the database (e.g. 8) than the 6 in the database then it recognize 8 which is wrong recognition so the correct recognition of the digit is highly depended on the accuracy of the system. Figure 5 shows the accuracy of the simple correlation technique.

5 100 Simple Correlation Accuracy (%) Separate Correlation Mixed Correlation Digits (1-9 & 0) Fig 5: Accuracy of simple correlation technique Figure 4 shows the simulation results of both techniques of simple correlation. The blue graph shows the system s accuracy of separate correlation and the red graph shows the system s accuracy of mixed correlation. On the x-axis defined the digits which are starting from 1 and continue to 9 and last digit is 0 which is represented as 10 in above figure. On the y-axis defined the accuracy of each digit. In order to analyze the accuracy of both techniques of simple correlation we draw a table from the fig 5 which is illustrated below. Table1: Analyzation of simple correlation (separate and mixed) Separate Correlation Mixed Correlation Digits Accurac y Table 1 clearly shows a comparison between the accuracy of separate correlation and mixed correlation. In separate correlation digits 1, 6 & 0 achieved more than 90% accuracy, digit 9 achieved 90% and digit 8 achieved 80% accuracy which shows good results as compared to the mixed correlation. Digits 2, 7 & 8 have achieved 67%, 67% & 78% accuracy respectively in separate correlation which are close to the mixed correlation. But only two digits 4 & 5 which have 56% and 27% accuracy respectively in separate correlation shows poor accuracy as compared to the mixed correlation. Therefore the accuracy of the separate correlation technique is better than the mixed correlation technique because in separate correlation techniques few digits achieved 90% and more than 90% accuracy but in mixed correlation technique the accuracy of all digits are vary between 70%-80%. But the overall (average) accuracy of both techniques are same. B. MEAN IMAGE CORRELATION This is the another proposed technique to recognize the numbers in postcode. This technique is an extension of the simple correlation technique to further improve the system accuracy. In this technique first the database is modified by taking the mean of 100 samples of each digit (0-9) of the database. After calculating their mean, the new modified database is reduced from 100 samples of each digit to 1 sample of each digit because in this new database each digit has only one sample which is the mean of 100 samples so the total samples in new database is 10 (0-9). After creating the new database the second step is to correlate the test digit of the extracted postcode with the each mean digit of the new database one by one and after correlation on the basis of results we recognized the digit which more accurately matches with the mean digit in the new database and this process continue till the last digit of the extracted postcode. Here there is no

6 need to add the correlated results because in new database each digit has only one sample. For example the test image digit is 6, after correlation with each mean digit of the new database, if the accuracy of the test image 6 to mean digit 6 in the new database is more than any other mean digit of the new database then it recognizes 6 but if the accuracy of the test image 6 is more with any other mean digit of the new database (e.g. 8) than the 6 in the new database then it recognize 8 which is wrong recognition. There are two main advantages of this technique. The first advantage is the simulation time which is significantly reduced as compared to the simple correlation technique because of the reduction in the database size. The second main advantage is the improvement in accuracy of the system compared with the simple correlation technique which is illustrated in figure Mean Image Correlation Accuracy (%) Digits (1-9 & 0) Fig 6: Accuracy of mean image correlation technique Figure 6 shows the simulation results of the mean image correlation technique. In order to analyze the above results properly draw a table which is illustrated below. Table 2: Analyzation of Simple Correlation (Separate and Mixed) Mean Image Correlation Digits Accuracy Table 2 shows the clear picture of the simulation results of the mean image correlation technique. The comparison of table 1 and table 2 clearly shows that the accuracy of digit 2, 4 & 5 are significantly improved as compared to the separate correlation technique and achieved 77%, 68% & 70% accuracy respectively. Even though the accuracy of few digits is decreased in mean image correlation as compared to the separate correlation but that is not significant and is tolerable. If compared the mean image correlation with the mixed correlation it is cleared that the mean image correlation technique is better than the mixed correlation technique because in the mean image correlation technique few digits achieved more than 90% accuracy but in mixed correlation technique the accuracy of all digits are vary between 70%-80%. But the overall (average) accuracy of the mean image correlation technique is better than the both techniques of simple correlation technique so on the basis of average we can say that the mean image correlation technique is better than the simple correlation technique.

7 C. MEAN FILTER BASE CORRELATION It is another technique to recognize the postcode. This technique is the combination of two previous techniques i.e.: simple correlation technique and mean image correlation technique. The mean filter base technique has resolved an important issue which occurs when recognize the human written numeric digit. The issue is during the recognition of the human written numeric digit it is a chance that test digit of the extracted post code match with more than one digit of the database (accuracy is same with more than one digit of the database) which creates the confusion and due to this correct ad accurate recognition is not possible. In order to resolve this issue this technique is introduced. Actually the mean filter base correlation technique is a further extension of the mean image correlation technique. In this technique we further processed the results of the previous technique in a way that after the correlation of the test digit with the new database which is created in the previous technique instead to recognize the digit on the basis of correlated results select the top three highly accurate digits from the results and then these top three digits correlated again with the old database which contain 1000 samples. After the correlation we added the correlated results to recognize the digit same as done in the simple correlation technique. For example the test image digit is 6, after correlation with each mean digit of the new database, instead of recognizing the digit 6 select the top three digits of the result whose more accurately match with 6, e.g. 8, 9 and 10 are the digits whose accuracy high with the 6 in the results. Then these three digits correlate with the old database which contains 100 samples of each digit. And after correlation add the correlated results to recognize the digit same as done in the simple correlation technique. Figure 6 shows the accuracy of the mean filter base correlation technique. 100 Mean Filter Base Correlatio Accuracy (%) Digits (1-9 & 0) Fig 7: Accuracy of Mean Filter Base Correlation Technique Figure 7 shows the simulation results of the mean image correlation technique. In order to analyze the above results properly we drew a table which is illustrated below. Table 3: Analyzation of mean filter based correlation technique Mean Filter Base Correlation Digits Accuracy Table 3 shows that the accuracy of the mean filter base technique is same as the separate correlation technique except the accuracy of digit 8 which is little bit decrease here as compared to the separate correlation technique. And the only difference is the simulation time of mean filter base

8 technique is less than the separate correlation technique. D. PRINCIPAL COMPONENTS ANALYSIS (PCA) It is another technique used for postcode recognition. This technique is different from all the above defined techniques, in which we extracted the features and represented the data which is widely used for pattern recognition [10]. The main idea of PCA is to identify the patterns and expressing the data by highlighting their differences and similarities [11]. The main function of the PCA technique is to describe the data economically by reducing the large dimension data into smaller dimensional data in the form of features (patterns) [12]. Once the pattern is found, the data can be compressed without much loss of information by reducing the number of dimensions that is the main advantage of the PCA technique [11]. The PCA technique is quite famous for face recognition and image compression. Here the idea of PCA technique is used for recognition of human written numeric postcode by determining the pattern of each numeric digit of postcode. There are two phases of PCA technique; the training phase and the recognition phase. The first phase of PCA is the training phase in which we train the system. In the training phase we kept the same database which we used in previous techniques and determined the features (patterns) of each digit of the database and stored in the system. The first step is to calculate the mean for each image of the digit of the database and then subtracted the mean for each of the data dimensions for normalizing the image of the digit. We called this resultant data as mean adjusted data. The second step is to calculate the covariance matrix. Mean value tells the middle point and the covariance tells the spreading of data from the middle point. The third step is to calculate the eigenvectors and eigen values from the covariance matrix. The eigenvectors tell us useful information about our data and all the eigenvectors are orthogonal to each other [11]. The eigenvalues are associated with the eigenvectors. They are scalar values and uses to scale the eigenvectors [11]. The fourth step is to selecting the principle component of the data set and form the feature vector. There are more than one eigenvector of each data set so first sort the eigenvectors and eigenvalues in descending order and select the eigenvector with the highest eigenvalue which is a principle component. Then construct the feature vector by taking all the eigenvectors or some of the selected eigenvectors that we want to keep in our data from the list of eigenvectors and forming a matrix with these eigenvectors in columns. In the fifth and the final step is to develop the new data set by transpose the feature vector and the mean adjusted data and then multiply both transposed feature vector and transposed mean adjusted data. This resultant data we called final/new data. This final/new data is actually the features (patterns) of each digit of the database which we called projected images. The second phase of the PCA is the Recognition phase in which the system recognizes the postcode. The first step is to calculate the features of the test image digit(human written postcode s numeric digit) by using the same process which is defined in the training phase and the same mean value is used which is calculated in the training phase. In the second step get the weights of the input image with respect to our eigenvectors by multiplying the transposed eigenvectors with the mean adjusted test data, and this resultant data we called called project input image. The third step is to calculate the Euclidean distance between the projected images and project input image, normalize it and then find the minimum Euclidean distance. The fourth and the final step is to compare the minimum Euclidean distance value with each digit of database for the recognition. Fig.8 shows the simulation results of the PCA technique. In order to analyze the above results properly draw a table which is illustrated below.

9 100 Principle Component Analysis 95 Accuracy (%) Digits (1-9 & 0) Fig 8: Accuracy of PCA Technique Table 4: Analyzation of PCA Table 4 shows that the accuracy of the PCA technique which is bit higher as compared with other techniques. When we compare table 4 with table 1, 2 and 3, it is clearly shown that the PCA gives better results. In PCA technique few of digits achieved 100% accuracy which is not achieved by the other techniques and most of digits achieved 95% accuracy. But the main difference is the accuracy of digits 4 and 5 which is drastically improved by using a PCA technique as compared to the other techniques. The main advantage of the PCA technique is that its efficiency can further be improved by using more eigenvectors. Here we use only few eigenvectors but by using more eigenvectors values, more accuracy can be achieved. Also there is a disadvantage that it will require more simulation time which may affect the system efficiency and capacity. To summarize the discussion, PCA technique is more accurate as compared to the Simple correlation technique, mean image correlation technique and mean filter base correlation technique. The next part explains the procedure of Mail sorting. 4. MAIL SORTING Principle Component Analysis Digits Accuracy After the recognition of postcode by using one of above defined techniques, the next part to sort the post according to the location. In the system each mailbox is allocated to the particular area. When system correctly recognizes the postcode then the system checks, that postcode belongs to which area. After making the decision the conveyer belt start moving again and place the envelope in a particular mailbox which belongs to that area. In this way the mail is sorted out area wise. 5. CONCLUSION AND FUTURE WORK In this paper, we implemented OCR algorithm by applying different techniques for the recognition of human written numeric postcodes for automatic mail sorting. On the basis of experimental results, it is revealed that the PCA technique is more accurate technique as compared to the other techniques. This work can be enhanced further by proposing a system which performs the autonomous orientation of the envelope according to the camera position during envelope detection, extraction of postcode from any line of the postal address, recognition of human written numeric digits of different font sizes and colours. The simulation results showed that the proposed techniques requires slightly higher processing time which can be reduced in future to improve the system performance.

10 REFERENCES [1] John Buck, The postal service revolution- A look at the past and where are we headed, Mailing Systems Technology Magazine, Dec [2] Alginaih, Yasser M., and Abdul Ahad Siddiqi, "Multistage hybrid Arabic/Indian numeral OCR system." International Journal of Computer Science and Information Security, IJCSIS, Vol. 8 No. 1, April 2010, USA. [3] John Buck, International mail-sorting automation in the low-volume environment, International Journal of Computer Science, vol. 8 no. 1, April [4] Xue Gao and Lianwen Jin, A Vision-Based Fast Chinese Postal Envelope Identification System, Journal of Information Science and Engineering, Vol 10, pp , [5] Yue Lu, Xiao Tu, Shujing Lu and Patrick S P Wang, Application of Pattern Recognition Technology to Postal Automation in China, Pattern Recognition and Machine Vision, Denmark, pp , [6] C. M. Velu1 and P. Vivekanandan, Automatic letter sorting for Indian Postal Address Recognition System based on PIN codes, Journal of Internet and Information System, Vol. 1, pp. 6-15, June [7] C.G. Leedham and P.E Jones, Automatic Sorting of Australian Handwritten Letter Mail using OCR and Address Feature Verification, IEEE Region 10 Conference, Melbourne. Australia, 11th - 13th November [8] P. Sri Rama Prasanna, S. Balaji, Thejavor Haralu Khezhie, C. Vasanthanayaki and S. Annadurai, Destination Address Interpretation for Automating the Sorting process of Indian Postal System, IEEE Conference on Convergent Technologies for the Asia-Pacific Region. Vol. 2, [9] The MNIST Handwritten Database Set. Available at [10] Nhat, Vo Dinh Minh, and Sungyoung Lee. "An improvement on PCA algorithm for face recognition", Advances in Neural Networks, Springer Berlin Heidelberg, pp , [11] Smith Lindsay I, "A tutorial on principal components analysis." Cornell University, USA, Available at. [12] Asadi, D. Srinivasulu, Ch DV Subba Rao, and V. Saikrishna. "A Comparative study of Face Recognition with Principal Component Analysis and Cross-Correlation Technique." International Journal of Computer Applications, vol 10, pp , 2010.