Detecting and Reading Text in Natural Scenes

Size: px

Start display at page:

Download "Detecting and Reading Text in Natural Scenes"

Octavia Foster
5 years ago
Views:

1 October 19, 2004 X. Chen, A. L. Yuille

2 Outline Outline Goals Example Main Ideas Results Goals

3 Outline Goals Example Main Ideas Results Given an image of an outdoor scene, goals are to: Identify regions where there is text Extract the text using OCR Convey this information to a blind person

4 Example Outline Goals Example Main Ideas Results

5 Main Ideas Outline Goals Example Main Ideas Results Build a strong classifier trained by AdaBoost Apply classifier to sub-regions of the image Binarize candidate text regions Use OCR on binarized candidate text regions

6 Results Outline Goals Example Main Ideas Results Reasonably fast 3 seconds on 3MP image 2.8% flase negatives 10% false positives 93% accuracy of OCR

7 General Idea Overview Selection of Features Find good features of regions containing text that sets them apart from other regions Construct classifiers that classify using these features Each classifier can be weak (slightly higher than 50% accuracy)

8 Overview Selection of Features How do we come up with text features in image? [1/3] Text has few common features with faces PCA Analysis leads to far more non-zero eigenvalues 1 1

9 Overview Selection of Features How do we come up with text features in image? [2/3] Examine x and y derivatives

10 Overview Selection of Features How do we come up with text features in image? [3/3] Histogram of pixel intensities Edge detection intensity gradient thresholding edge linking

11 What is boosting? Overview Gist of AdaBoost Algorithm Problems Start with an example: junk classification Basic idea: finding many rough rules of thumb is easier than finding single highly accurate rule History on boosting

12 Gist of AdaBoost Overview Gist of AdaBoost Algorithm Problems AdaBoost algorithm is used to combine weak classifiers to make a strong classifier Each weak classifier produces a yes/no answer for a particular feature Need to come up with lots of features and let AdaBoost decide how to combine them

13 Algorithm Overview Gist of AdaBoost Algorithm Problems Given a set of examples (x 1, y 1 ),..., (x N, y N ), where x i X, and y i Y = { 1, +1}: 1 Initialize D 1 (i) = 1/N 2 For t = 1,..., T : 1 Train weak classifier using distribution D t 2 Obtain weak hypothesis ( ) h t : X { 1, +1} with error ɛ t 3 Choose α t = 1 2 ln 1 ɛt ɛ t 4 Update distribution: D t+1 (i) = D t(i) Z t 3 Output final hypothesis: H(x) = sign { e αt e αt = D t(i)e αty i h t(x i ) Z t TX t=1 α th t(x) if h t (x i ) = y i if h t (x i ) y i!

14 Error Overview Gist of AdaBoost Algorithm Problems Z t = i D t(i)e y i h t(x i ) Upper bound on training error of strong classifier is t Z t. Minimizing t Z t is also equivalent to minimizing overall classification error

15 Problems with AdaBoost Overview Gist of AdaBoost Algorithm Problems Minimizes classification error - not number of false negatives. To fix, Viola and Jones propose modifying distribution D t give more weight to positive examples. They call this modification Asymmetric AdaBoost.

16 Cascade Classification Cascade Classification Integral Images Sub- Window 1 T 2 T 3 T Further Processing F F F Reject sub-window

17 Integral Images Cascade Classification Integral Images First 3 layers of Chen and Yuille cascade use only mean, STD, and derivative features These are easily calculated from integral images

18 Integral Images Cascade Classification Integral Images A B 1 2 C 3 D 4 D = I 4 + I 1 (I 2 + I 3 ) Figure 3: The sum of the pixels within rectangle D can be computed with four array references. The value of the integral image at location 1 is the sum of the pixels in rectangle A. The value at location 2 is A + B, at location 3 is A + C, and at location 4 is A + B + C + D. The sum within D can be computed as 4+1 (2 + 3).

19 Use Niblack s adaptive binarization method T r (x) = µ r (x) + kσ r (x) with a modification: r(x) = min r (σ r (x) > T σ )

20 [1/3] Is AdaBoost the best thing out there for detecting objects? Does it not depend on features used? P PP P PP False detections P PP Detector P PP PP Viola-Jones 78.3% 85.2% 88.8% 89.8% 90.1% 90.8% 91.1% 91.8% 93.7% Rowley-Baluja-Kanade 83.2% 86.0% % % 89.9% Schneiderman-Kanade % Roth-Yang-Ahuja (94.8%) Table 3: Detection rates for various numbers of false positives on the MIT+CMU test set containing 130 images and 507 faces. 2 The Sung and Poggio face detector [18] was tested on the MIT subset of the MIT+CMU test set since the CMU portion did not exist yet. The MIT test set contains 23 images with 149 faces. They achieved a detection rate of 79.9% with 5 false positives. Our detection rate with 5 false positives is 77.8% on the MIT 2 Viola & Jones. Robust Real-time Object Detection. test set.

21 [2/3] Sensitivity of the algorithm to the input data and parameters? Modulus of derivative feature?!? What is that?

22 [3/3] Does the order of weak classifiers used matter in AdaBoost? What about perspective distortion?

Object Category Detection: Sliding Windows

03/18/10 Object Category Detection: Sliding Windows Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Goal: Detect all instances of objects Influential Works in Detection Sung-Poggio