ICDAR2015 Writer Identification Competition using KHATT, AHTID/MW and IBHC Databases

Transcription

1 ICDAR2015 Writer Identification Competition using KHATT, AHTID/MW and IBHC Databases Handwriting is considered to be one of the commonly used modality to identify persons in commercial, governmental and forensic applications. In order to record recent advances in the field of writer identification, we are proposing to organize the ICDAR2015 writer identification competition using KHATT, AHTID/MW and IBHC Databases. A first edition of the Arabic Writer Identification Competition using KHATT and AHTID/MW Databases was organized in the 14th International Conference on Frontiers in Handwriting Recognition (ICFHR 14) [8]. The underlining objective is in the evolution of handwritten text recognition research. This competition will take place at the 13 th International Conference on Document Analysis and Recognition (ICDAR 15), during August 23-26, 2015, Tunis, Tunisia and will be organized using the freely available Arabic Handwritten Text Images Database written by Multiple Writers (AHTID/MW), the Arabic handwritten text database called KHATT and the Isolated Bangla Handwritten Characters (IBHC) database. Competition Tasks 1. Writer identification using handwritten Arabic paragraphs with the KHATT database. Main contact: Sameh Awaida (s.awaida@qu.edu.sa) 2. Writer identification using handwritten Arabic text lines with the AHTID/MW database. Main contact: Fouad Slimane (fouad.slimane@epfl.ch) 3. Writer identification using handwritten Arabic words with the AHTID/MW database. Main contact: Fouad Slimane (fouad.slimane@epfl.ch) 4. Writer Identification on all isolated Bangla characters (alphabets + numerals + vowel modifiers) with IBHC database. Main contact: Chayan Halder (chayan.halderz@gmail.com) 5. Writer identification on each Bangla character with IBHC database. Main contact: Chayan Halder (chayan.halderz@gmail.com) Participants can submit one or many executable systems depending to their choice to participate to all proposed tasks or some of them. Registration In order to participate in this competition, each team must send an to the main contact of the task(s) including: Subject: [ICDAR2015-WI_Task(s)_<Number>] Author names and affiliation The organizer will send you later the modalities to get the data. Scientific Objectives The scientific objectives of this competition are to measure the capacity of recognition systems to identify the writer using character, word, text line and paragraph images. The main difficulty is probably in the similarity between the writer styles, the quality of the images as they are scanned on grey level and in the possibility to recognize the writer using one character, word, one line or one paragraph image. Such evaluation can serve several purposes for researchers: It will encourage researchers to test their algorithms on real world challenging databases like KHATT, AHTID/MW and IBHC.

2 A competition on these databases can set some benchmarking results in writer identification research. A competition is an excellent opportunity for researchers to test their techniques on KHATT, AHTID/MW and IBHC databases and compare results. In addition, this may result in promoting the databases to the community of researchers. Competition on KHATT, AHTID/MW and IBHC may result in improving the state of the art in writer identification and text recognition using natural Arabic and Bangla text with large and open vocabulary. Used Databases Such evaluation can be useful in forensic applications. 1. The KHATT Database KHATT is a large and real-life Arabic handwritten text database, collected from 1000 writers of different origins. Each writer wrote six paragraphs, including a paragraph (written twice) that contains all the shapes of Arabic characters and two free form paragraphs on topics chosen by the writer himself. Figure 1 illustrates the paragraph forms from one writer in KHATT database. The database is very much suitable for research in Arabic writer identification and handwriting recognition. More details on the database can be found in khatt.ideas2serve.net KHATT database was first presented in ICFHR 12 [5] and received the Best Poster Award. It was later selected for publication (an extended and enhanced version) in Pattern Recognition as one of the selected best papers of ICFHR 12 [6]. The database is designed for tasks that are of central interests to document analysis and recognition community, like writer identification, line segmentation, and binarization and noise removal techniques beside handwritten text recognition. Already, there are a number of research groups around the world that have collected KHATT for their research purposes. The organizers will provide the competition participants with the following data. For each of the 1000 writers, four paragraph images (like the ones in Figure 1 (a) and (b)) will be provided as training and validation sets along with the verified ground truth values. Note that, the participant systems need not to segment the forms (as shown in Figure 1) to extract the paragraphs. Rather, the paragraphs are already extracted from the forms and will be provided to the participants (as in Figure 2), where each writer has written four paragraphs of Arabic text. The participants will use these paragraph images to train and test their writer identification systems. Two of these paragraph images are text-dependent and two are text-independent which helps the participant in developing both text-dependent and textindependent writer identification systems. (a) (b) (c) Figure 1: Samples of Training paragraphs from KHATT database.

3 The organizers will test these systems on a hidden evaluation set of around 2000 paragraph images (a maximum of 2 paragraph images per writer, like the ones in Figure 1 (c)). Note that, the second paragraph written by a single writer in the evaluation (hidden) set is written on ruled lines. A writer identification system may perform poorly [8] if proper pre-processing methods are not used to deal with ruled lines. Therefore, we separated the evaluation set into two parts (unlike the 2014 competition, where the evaluation set consisted of both paragraphs with and without lines). In this version of the competition, the participants may submit up to two systems for KHATT, one to be used for paragraphs without ruled lines (a sample is shown in Figure 2 (a)) and the other for paragraphs with ruled lines (a sample is shown in Figure 2 (b)). (a) Un-ruled Paragraph (b) Ruled Paragraph Figure 2: Samples of testing paragraphs from KHATT database. 2. The AHTID/MW Database The Arabic Handwritten Text Images Database written by Multiple Writers (AHTID/MW) has been built at the MIRACL Lab, ISIMS, University of Sfax - Tunisia in join collaboration with the Institute for Communications Technology (IfN), Braunschweig - Germany. It can be used for research in the recognition of Arabic handwritten text, word segmentation, word spotting and writer identification. The AHTID/MW contains 3710 text lines and 22,896 words written by 53 native writers of Arabic. These images are divided into five equilibrated sets. The four first sets are available for scientific community and the fifth set is kept internal for potential future evaluation of systems in blind mode. Each word/line image in AHTID/MW database is fully described using an XML file containing ground truth information. The database is freely available for worldwide researchers. Actually, AHTID/MW database is used by many groups all over the world working on Arabic handwritten text recognition. For more details, we refer to [7]. 3. The Isolated Bangla Handwritten Characters (IBHC) database The database is one of the important aspects for the development of a good writer identification system. So, it is important that we collect data for the database in a way such that the variation in writing style of different writers and the variation in writing style of the same writers are all remain intact in the datasets. For this reason we have collected 5 copies of isolated character and numerals datasets from the writers. There are instructions to each writer to write each copy of the datasets in different days to preserve inter and intra writer variations. Each participant (writer) was required to copy-out the printed characters in the particular box area of the sample document for each given set. Data collection has been done from different individuals of various ages, genders and professions. The collected Bangla handwritten isolated character database consists of 1241 datasets having characters (63291 Bangla alphabets, Bangla numerals and Bangla vowel modifiers) from 250 writers. There exists no boundary for writers regarding the type of pen and ink they have used. The documents are being scanned using a flatbed scanner for digitization. The images are in gray tone and digitized at 300 dpi and stored in Tagged Image File Format (TIFF). The isolated characters will be provided into groups of randomly selected characters of training set and groups of randomly selected characters of testing set. Remaining characters will be used for evaluation purpose. The training dataset will

4 be provided at the initial stage of the competition. The testing dataset will be provided afterwards and the evaluation dataset will be released after evaluation of all the submitted methods and announcement of the results. An example of isolated characters are shown in figure 3(a) and some isolated gray-scale characters from the datasets is shown in figure 3(b). Modalities of the evaluation Figure 3: Samples of isolated characters from IBHC database. The evaluation will be organized using a blind procedure. Participants are allowed to train their systems using the available sets of KHATT, AHTID/MW and IBHC databases. At a given date, participants have to send their executable that will be run on an unseen data set in our premises and a short description of the method submitted. We encourage participants to send us their pre-evaluation identification rates obtained using the available testing dataset before submitting their final executables. The results of the competition will be presented in a special session at ICDAR 15. The organizers will publish the results of the candidate methods on the evaluation dataset, as well as a comparative evaluation of the submitted systems. The organizers will test the submitted systems on the set 5 (tasks 2 and 3) and the 2000 paragraph images (task 1). For each image, these systems should produce the IDs of the 10 most probable writers for this image, sorted in descending order (from top-1 most probable writer ID to top-10 mostprobable writer ID). For tasks 4 and 5, the participant will be asked to provide a console application that will identify writers based on the isolated Bangla alphabets, isolated Bangla numerals and isolated Bangla vowel modifiers separately and also combined. The application will also identify writers depending on each character i.e. the application should able to identify writers depending on a single character. The performance evaluation will be grouped into two categories. In the first category the writer identification rate will be calculated depending on the total isolated Bangla characters (alphabets + numerals + vowel modifiers). Also in this category the calculation of writer identification on all Bangla alphabets, all Bangla numerals and all Bangla vowel modifiers separately will be considered to determine the strength of the methods. Evaluation Protocol The evaluation will be reported as character, paragraph, line, and word writer-identification rates. In this edition, systems will be tested using the paragraph, line, and word images. We accept recognizers developed with Windows or Linux OS. Recognizer Running Format We run a recognizer (say ProposedRec) by invoking it from the command line as follows: > ProposedRec input.txt output.txt input.txt Where the input and output formats are like: The input file is just a list of paths to all line, word, or paragraph images (*.png or *.tif) to be recognized. For example: /home/ahtid_mw/sentenceimage_0.png /home/ahtid_mw/sentenceimage_1.png

5 /home/ahtid_mw/sentenceimage_2.png output.txt The output file should be containing the path of recognized images and the writers. An example of output file is presented in the following: #!MLF!# "/home/ahtid_mw/sentenceimage_0.rec" writer_1 writer_2 writer_3 writer_6 writer_4 writer_10 writer_53 writer_50 writer_25 writer_20. "/home/ahtid_mw/sentenceimage_1.rec" writer_4... Performance measures The submitted systems are ranked according to different criteria. First, the top-1 accuracy results in writer identification are obtained by summing the number of hits for the most probable writer for each test sample. Correspondingly, the top-5 and top-10 accuracy results are the number of hits using the top-5 and top-10 most probable writer for each test sample. Important Dates Competitions open to participants: January 10, 2015 Deadline for submission of executables: March 15, 2015 Organizers Fouad Slimane fouad.slimane@epfl.ch Sameh Awaida - s.awaida@qu.edu.sa Anis Mezghani anis.mezghani@gmail.com Mohammad Tanvir Parvez - m.parvez@qu.edu.sa Slim Kanoun slim.kanoun@yahoo.fr Volker Märgner Maergner@ifn.ing.tu-bs.de Chayan Halder chayan.halderz@gmail.com Kaushik Roy kaushik.mrg@gmail.com Jaya Pal 5 1 Institute for Communications Technology (IfN), Germany

6 2 MIRACL Lab, ISIMS, University of Sfax, Tunisia 3 DHLAB, Swiss Federal Institute of Technology in Lausanne (EPFL), Switzerland 4 West Bengal State University, Kolkata, West Bengal, India) 5 Govt. College of Leather Technology, Kolkata, India) References [1] Awaida, S., and Mahmoud, S.A., State of the Art in Off-line Writer Identification of Handwritten Text and Survey of Writer Identification of Arabic text, Educational Research and Reviews, 7(20), doi: /err , [2] Parvez, M.T and Mahmoud, S.A., Off-line Arabic handwritten text recognition: a survey, ACM Computing Surveys, vol. 45, Issue 2, Article 23, [3] Märgner, V. and El Abed, H., ICDAR Arabic Handwriting Recognition Competition, 11 th International Conference on Document Analysis and Recognition (ICDAR), pp , [4] Märgner, V. and El Abed, H., ICDAR 2009 Arabic Handwriting Recognition Competition, 10 th International Conference on Document Analysis and Recognition (ICDAR), pp , [5] Mahmoud, S. A., Ahmad, I. Alshayeb, M., Al-Khatib, W. G., Parvez, M. T., Fink, G. A., Margner, V. and EL Abed, H., KHATT: Arabic Offline Handwritten Text Database, 13th International Conference on Frontiers in Handwriting Recognition (ICFHR-2012), pp , [6] Mahmoud, S. A., Ahmad, I. Alshayeb, M., Al-Khatib, W. G., Parvez, M. T., Margner, V., Fink, G. A., KHATT: An open Arabic offline handwritten text database, DOI: dx.doi.org/ /j.patcog , [7] Mezghani, A.; Kanoun, S.; Khemakhem, M.; Abed, H.E., "A Database for Arabic Handwritten Text Image Recognition and Writer Identification,", International Conference on Frontiers in Handwriting Recognition (ICFHR), pp.399,402, 2012 [8] Slimane, F.; Awaida, S.; Mezghani A.; Parvez, M. T.; Kanoun S.; Mahmoud, S. A.; Märgner, V., "ICFHR2014 Competition on Arabic Writer Identification Using AHTID/MW and KHATT Databases". Proceedings of 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp , 2014