MAAYA: Multimedia Analysis and Access for Documentation and Decipherment of Maya Epigraphy April Morton Gulcan Can Rui Hu February 27 th, 2013
1 The MAAYA Project Main goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers Dr. Daniel Gatica-Perez Gulcan Can Dr. Jean-Marc Odobez Dr Rui Hu Prof. Nikolai Grube Carlos Pallan Guido Krempel Prof. Stéphane Marchand-Maillet Dr. Edgar Roman-Rangel April Morton Oscar Dabrowski
2 Outline Mayan Overview Project Overview Data Preparation/Storage Computer Vision Algorithms Using Context to Improve Glyph Recognition Visualization Conclusion
3 Mayan Overview Ancient Mayan Culture Flourished from BC 2000-1500 AD
4 Mayan Overview Mayan Writing System Syllabographs Iconography Logographs
5 Mayan Overview Documentation Process Take Photographs Obtain Digital Line Drawings Identify Glyphs Annotate Glyphs Need better digital preservation and automatic analysis techniques
6 Project Overview The CODICES Project Statistical Shape Descriptors CBIR System Main goal: Develop statistical shape descriptors and a CBIR system
7 Project Overview The MAAYA Project Data Preparation/Storage Content-Based Data Processing/Computer Vision Algorithms High Level Analysis Visualization Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
8 Data Preparation/Storage Process Segment glyphs and blocks Upload segmented page into database Annotate and store glyphs
9 Computer Vision Algorithms Localization Segmentation Decomposition Where is the data? Glyph Representation How to recognize the data? Identification (Classification - Retrieval) Shape Descriptors Sparse Representation Attribute Learning Part-based Representation
10 Computer Vision Algorithms Localization
11 Computer Vision Algorithms Localization
12 Computer Vision Algorithms Segmentation
13 Computer Vision Algorithms Hierarchical Decomposition Codices Page T'ol Row of Blocks Block Glyph
14 Computer Vision Algorithms Recognition Glyph representation + Identification Can we utilize/capture diagnostic features?
15 Computer Vision Algorithms Diagnostic Features
16 Computer Vision Algorithms Histogram-of-Orientations Shape Context
17 Computer Vision Algorithms Learning Features with Autoencoders
18 Computer Vision Algorithms Identification Glyph Retrieval Compute pivots and HOOSC descriptors k-means clustering & bag-of-visual-words representation Ranking of bov s of each glyph
Computer Vision Algorithms Identification - Classification Compute pivots and HOOSC descriptors Classify each pivot Apply voting for glyph-level prediction 16 19
Using context to improve glyph recognition 20
Using context to improve glyph recognition 21
Using context to improve glyph recognition 22
23 Using context to improve glyph recognition Category:T668.T102 Cha-ki Meaning: Rain god Category:T227:T561.T23 xib?-ka an-na Meaning: man, sky,?
24 Using context to improve glyph recognition Data preparation
25 Using context to improve glyph recognition Glyph matching
26 Using context to improve glyph recognition Glyph matching
27 Using context to improve glyph recognition Statistic language model Icon Translation Category:T668.T102 Cha-ki Meaning: Rain god Category:T227:T561.T23 xib?-ka an-na Meaning: man, sky,?
28 Using context to improve glyph recognition Context Information
29 Using context to improve glyph recognition Improve recognition accuracy Help with noisy and partially missing examples High-level translation
30 Visualization A final goal is to advance state-of-the art in effective interactive visualization tools Input Data Update Parameters Apply Data Mining Technique Accept User Input Display Visualization
31 Visualization Interactive Clustering MDS projection based on weighted Euclidean distance function Data from UCI wine data set Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
32 Visualization Interactive Clustering Re-projection Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
33 Visualization Interactive Clustering Re-projection Could think of data points as glyphs Distribution of values among attributes Updated Weights Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
34 Visualization Topic Model Visualization Area encodes probabilities Can filter and sort words based on probability of co-occurrence Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
35 Visualization Topic Model Visualization Can think of terms as glyphs Area encodes probabilities Can filter and sort words based on probability of co-occurrence Can think of documents as codices (add) Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
36 Conclusion The MAAYA project aims to improve the entire process of digital glyph analysis Data Preparation/Storage Content-Based Data Processing/Computer Vision Algorithms High Level Analysis Visualization Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
37 Acknowledgements And a special thanks to the Swiss National Science Foundation: Project CR21I2L_144238
Questions 38
Backup Slides
The Online Wizard The webpage includes a user friendly upload wizard for segmented files
The Data Annotation Page As well as a standardized data annotation page
5 Mayan Overview Mayan Writing System
6 Mayan Overview Mayan Writing System