REAL TIME BRAILLE TRANSLATION. Andrew Petersen, Logan Schuelke, Marcus Turner

Size: px

Start display at page:

Download "REAL TIME BRAILLE TRANSLATION. Andrew Petersen, Logan Schuelke, Marcus Turner"

Alexis Byrd
6 years ago
Views:

1 REAL TIME BRAILLE TRANSLATION Andrew Petersen, Logan Schuelke, Marcus Turner

2 INTRODUCTION Motivations Increasing Accessibility (ADAAG) Harnessing Technological Advances Methods Dots Vs. Words Font Recognition

Francois, Optical recognition of Braille writing using standard equipment Kanagawa

3 PREVIOUS WORK Dot identification: Compute feature vectors and find peaks Laplacian of a gaussian, find circles Braille Point Translation J. Mennens, L. Van Tichelen, G. Francois, Optical recognition of Braille writing using standard equipment Kanagawa Univ., Yokohama A Braille Recognition System by the Mobile Phone with Embedded Camera

4 METHODOLOGY 1. Identifying braille dots from images 2. Group the dots into lines and words 3. Translating identified points into text

5 Steps to identify candidate points: POINT DETECTION / IDENTIFICATION 1. Convert image to binary (Otsu s algorithm) 2. Filter image (open/close) to eliminate small regions 3. Look at only regions that are a certain size with respect to the image Steps to filter out non-braille candidates 1. Find the minimum distance between region centroids 2. For every candidate, count how many other candidate regions there are within a given scaled version of that minimum distance. 3. If there are at least three others, good chance it is a braille dot

6 EXAMPLE: CLASSROOM NAMEPLATE finddotcoordinates

7 LIMITATIONS 1. Does NOT track points from frame to frame, has to start fresh every time. 2. Has to look for both black and white dots: adds a little extra computing time a. Hopefully, this gets filtered out at the beginning, because only small regions will be compared 3. Can pick up reflections that do not get filtered out:

8 GROUP SEPARATION/ ITERATIONS Hierarchical (non- Euclidian) Connected Components Cluster Estimation K-Means Clustering

9 groupdots GROUPING: EXAMPLE

10 ORGANIZING DOTS Order 1 3

11 POINT TRANSLATION Objective: Output from point identification step is an Nx2 array of points. Goal is to take set of points (of arbitrary order), and translate to text Minimum distance between braille dots (defining the grid) passed in from previous step Ideally orthorectification performed for robustness of algorithm (not implemented)

12 POINT TRANSLATION - STEP 1 Character Identification Braille characters are 3 rows by 2 columns, this is used to our advantage Braille characters, except for special characters and punctuation, always have a dot in the upper left corner, or two dots forming a 45 degree angle in the upper left

13 POINT TRANSLATION - STEP 2 Finding the sums (a given point s X + Y coordinates) of the input points gives easy and fast way to get upper left corner and special character patterns Example: if mindist = 1, upper (141,234) (sorted by ascending): Left Side Right Side

14 POINT TRANSLATION - STEP 2 Minimum sum corresponds to upper left point (the next greatest sums are >= sum + mindist) 45 degree gives two sums approximately the same (within small % of mindist) Left Side Right Side

POINT TRANSLATION - STEP 3 If 45 degree angle, virtual point created in upper left Corner Virtual Point (if 45 degree) Upper left corner used to create an occupancy grid,

15 POINT TRANSLATION - STEP 3 If 45 degree angle, virtual point created in upper left Corner Virtual Point (if 45 degree) Upper left corner used to create an occupancy grid, based off the X and Y coordinate of the upper left corner Iterate over point set to see where they fall within the grid Store, and remove from point set. Repeat until no more points

16 POINT TRANSLATION - STEP 4 Load in dictionary of braille characters Each stored occupancy grid corresponds to an entry in the dictionary Iterate over stored grids and find the respective entries; output translated string

17 POINT TRANSLATION ASSUMPTIONS AND LIMITATIONS Algorithm assumes points come from a single word or line Can handle letters, and numbers. Special symbols such as punctuation and capitalization currently unsupported Points orthorectified for best results

18 EXPERIMENT AND RESULTS

19 CONCLUSION Difficulties and limitations Slight offset could make letters untranslateable. Future work Orthorectification of the image Track points by frame instead of starting fresh every single time Find location of finger and read aloud as the person progresses

20 Questions?

6. Applications - Text recognition in videos - Semantic video analysis

6. Applications - Text recognition in videos - Semantic video analysis Stephan Kopf 1 Motivation Goal: Segmentation and classification of characters Only few significant features are visible in these simple