Introduction. How? Rapid Object Detection using a Boosted Cascade of Simple Features. Features. By Paul Viola & Michael Jones

Size: px

Start display at page:

Download "Introduction. How? Rapid Object Detection using a Boosted Cascade of Simple Features. Features. By Paul Viola & Michael Jones"

Maude Atkins
6 years ago
Views:

1 Rapid Object Detection using a Boosted Cascade of Simple Features By Paul Viola & Michael Jones Introduction The Problem we solve face/object detection What's new: Fast! 384X288 pixel images can be processed at 15 frames per second on a 700MHz Intel Pentium III. Ohad Hageby IDC Ohad Hageby IDC How? New representation (integral Image) using AdaBoost for improved learning. Cascaded detectors Features We classify images based on the value of simple features. Why not use pixels directly? - features can act to encode ad-hoc domain knowledge that is difficult to learn using a finite quantity of training data. - Working with features is much faster! Ohad Hageby IDC Ohad Hageby IDC

2 What are Features? Three kind of features are being used: Two-rectangle feature Three-rectangle rectangle feature Four-rectangle rectangle feature Two-rectangle feature A number consisted from: Sum of pixel values in black area - Sum of pixel values in white area Ohad Hageby IDC Ohad Hageby IDC Three-rectangle rectangle feature A number consisted from: Four-rectangle rectangle feature A number consisted from Diagonal difference: Sum of pixel values in black area - Sum of pixel values in both whites area Sum of pixel values in both black area - Sum of pixel values in both whites area Ohad Hageby IDC Ohad Hageby IDC

3 Intuition on how features used These are how features are detected in an image (sub-window). Note! The base resolution of the detector is 24x24 (the sub-window) In this resolution there are over 180,000 features! With so many features, how are we going to have an efficient implementation? Ohad Hageby IDC Ohad Hageby IDC New Image Representation the "Integral Image" To compute fast rectangle features we will use an intermediate representation Integral Image. Definition: The Integral image at location (x, y) is If ii(x,, y) is the integral image i(x,, y) is the original image (or rectangular part of it) Therefore s(x,y)=s(x,y-1) +i(x,y) : s is cumulative row sum: s(x,-1)=0. ii(x,y)=ii(x-1,y) + s(x,y) that s s means: ii(-1,y)=0 Ohad Hageby IDC Ohad Hageby IDC

s(x,y)=s(x,y-1) +i(x,y) Y ii(x,y)=ii(x-1,y) + s(x,y) ii(x-1,y) Y s(x,y-1) x The pixel i(x,y) x The row s(x,y) Ohad Hageby IDC 2005 13 Ohad Hageby IDC 2005 14 What it cost Example on use The integral

4 s(x,y)=s(x,y-1) +i(x,y) Y ii(x,y)=ii(x-1,y) + s(x,y) ii(x-1,y) Y s(x,y-1) x The pixel i(x,y) x The row s(x,y) Ohad Hageby IDC Ohad Hageby IDC What it cost Example on use The integral image can be computed in one pass over the original image Using the integral image,, any rectangular sum can be computed as four array references. Example: The sun within D can be computed as ii(4) + ii(1) - (ii(2) + ii(3)) Note that ii(at point1)=area of A etc. Ohad Hageby IDC Ohad Hageby IDC

5 Integral Image Cont. It takes six array references to compute the difference in two rectangular feature It takes eight array references to compute the difference in three rectangular feature It takes nine array references to compute the difference in four rectangular feature Feature Discussion Rectangular feature are coarse (vertical horizontal and diagonal). However, provide rich image representation which supports effective learning. Used together with Integral Image compensate their limited flexibility. Ohad Hageby IDC Ohad Hageby IDC Learning Classification Functions Problem: There are more than 180,000 rectangle features in an image sub-window (24x24).Although each feature can be computed efficiently this is too much Solution: Experimental conclusion was that only a small number of Rectangle feature can be combined to form an effective classifier. Learning Classification Functions Cont. Challenge: To find those special features and to assemble a classifier from them. A variant of AdaBoost was used both for selecting a small set of features and for training the classifier. We use a weak learning algorithm to select the single rectangle feature which best separates positive examples from negative. Ohad Hageby IDC Ohad Hageby IDC

6 The Weak Classifier hj (x) - the weak classifier (x is 24x24 pixel window) fj the feature Өj threshold, looking for optimal Өj j for fj. pj parity. Indicating direction of inequity sign. Learning goals What? We must choose the features that will be used for the classifier. How? We will use AdaBoost algorithm for classifier learning. Each round of boosting selects one feature from the ~180,000 potential features. Ohad Hageby IDC Ohad Hageby IDC Our AdaBoost algorithm Input: Training set (x1,y,y1), (x2,, y2), y ), (xn,, yn). y xi X X image, yiy Y={0,1} (positive/negative example) m the number of positive examples l the number of negative examples T the number of boosting rounds (t will be the counter) wt,i the i-th i weight in the t-th t th round. Our AdaBoost algorithm Initialize w 1, i 1 ; y = 2m 1 ; yi 2l i = 0 = 1 Ohad Hageby IDC Ohad Hageby IDC

7 Our AdaBoost algorithm For t=1, T 1. Normalize weights so that wtw is a probability distribution w t, i n w j = 1 t, i w t, i Our AdaBoost algorithm 2. For each feature fjf train a classifier hjh which is restricted to using a single feature. The training error is evaluated with respect to wt: w j i i ( xi ) yi ε = w h 3. Choose the classifier hth with lowest error Et. E j Ohad Hageby IDC Ohad Hageby IDC Our AdaBoost algorithm 4. Update the weights for next round: and the final classifier is The final strong classifier is: w t + 1, i = w t t, i β 1 ei t 1; example x ei = 0; ε t βt = 1 ε i classified otherwise correctly T T 1 1; αtht ( x) αt h( x) = t= 1 2 t= 1 0 otherwise 1 αt = log β t Ohad Hageby IDC Ohad Hageby IDC

The Attentional Cascade The Attentional Cascade Instead of using single complex classifier, we use many simple classifiers in a cascade structure.

8 The Attentional Cascade The Attentional Cascade Instead of using single complex classifier, we use many simple classifiers in a cascade structure. Increases detection performance while reducing computation time. The key idea: rejecting fast, big amount of sub-windows, to allow more computation time for the interesting sub-windows. Ohad Hageby IDC Ohad Hageby IDC The Attentional Cascade The Attentional Cascade Observation Simpler classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to achieve low false positive rates. The threshold of a boosted classifier can (and should! At least for the firsts in the cascade) be adjusted so that the false negative rate is close to zero!! (not to throw too much) Ohad Hageby IDC Ohad Hageby IDC

9 The Attentional Cascade Stages in the cascade constructed by training classifiers using AdaBoost. And then adjusting the threshold to minimize false negative. Note! The default AdaBoost designed to yield low error rate on the training set. This generally causes higher detection rate AND higher false positives. Down to Earth Example Example for a first-stage stage two feature classifier. Threshold was reduced to minimize false negatives. Yielded 100% detection with false positive rate of 40%!!!! Ohad Hageby IDC Ohad Hageby IDC Down to Earth Example cont. Complexity: Computation of the two feature classifier took 60 microprocessor instructions. Other methods like single layer perceptron costs more than 20 times this much!! Observation: Finding a good sub-window is a rare occasion! Training a Cascade of classifiers In theory we would like a mechanism which trade off: The number of classif. stages ( cascade( depth ). The number of feature in each stage (complexity of classifier). The threshold ( sensitivity( sensitivity ). in order to optimize computation time. VERY HARD PROBLEM! Ohad Hageby IDC Ohad Hageby IDC

10 Training a Cascade of classifiers cont. In practice we have simple framework : Each stage reduce FP and decrease detection rate. 1. Target selected for min reduction in FP and max decrease in detection. 2. A stage is trained (adding features) until targets are met (using validation set). 3. Adding stages until overall target for FP and detection rate is met. Results The complete face detector has: 38 stages Over 6000 features. Resulted in fast average detection time: On a difficult data set of 507 faces (75 million sub-windows) detection performed using average of 10 features per sub window. Ohad Hageby IDC Ohad Hageby IDC Detector training the classifier was trained to detect frontal upright faces hand labeled faces of 24x24 pixels non face images manually inspected not to contain faces. Total of 350 million sub windows within non-face images. Detector training cont Number of features in the first five layers of detector: 1, 10, 25, 25 and 50. Total number of features: Each classif. in the cascade trained with 9832 faces (4916 plus their vertical mirror) and 10,000 non-faces sub windows using AdaBoost. (non face = false positives of prev. layers) Ohad Hageby IDC Ohad Hageby IDC

11 Speed of final detector Evaluated on the MIT+CMU test set. Majority of sub-windows rejected by the first or second layer of the cascade (10 features out of 6061 are evaluated per sub window avg.) On PIII 700Mhz sec per 384x288 image. Starting scale: 1.25 Step size: 1.5 (TBE) Ohad Hageby IDC Variance normalization Both for training and detection SW were variance normalized. б = standard deviation, m = mean, x = pixel value within the SW, N = number of pixels in SW: σ = m N Sum of squared pix. Computed using ii and also the mean. The V.N. effect can be achieved by post multiplying feature values instead pre-mul mul.. the pixels. x 2 Ohad Hageby IDC Scanning the Detector Final detector is scanned at multiple scales and locations. Scaling is easier to do on the detector than on the image (feature scale eval at same cost!) Scale factor: 1.25 Scanning the Detector cont. The SW is shifted [scale]x[shift ] =1 in the results presented but a speedup is gained using =1.5 with small decrease in accuracy. Ohad Hageby IDC Ohad Hageby IDC

$ROC receiver operating characteristics Defined as the relation between the probability of detection (true positive fraction) and probability of false positive (false positive$ $fraction) Ohad Hageby IDC 2005 45 From: http://www.anaesthetist.$

12 Integration of multiple detection Our final detector is insensitive of small changes in place or scale. Disjoint sets overlapping detections an average was calculated for each set. ROC receiver operating characteristics Defined as the relation between the probability of detection (true positive fraction) and probability of false positive (false positive fraction) Ohad Hageby IDC From: Ohad Hageby IDC Cascade and ROC The more difficult examples faced by deeper classifiers push the entire ROC curve downwards. At a given detection rate deeper classifier have higher FP rates. How ROC looks like? Next slide Ohad Hageby IDC Ohad Hageby IDC

The Threshold Role To create this ROC curve threshold of last layer set to +. + When threshold of last layer is -, detection rate and false positive rate increases (to a limit).

13 The Threshold Role To create this ROC curve threshold of last layer set to +. + When threshold of last layer is -, detection rate and false positive rate increases (to a limit). Equivalent to removing last layer Setting threshold to + + yields detection of 0.0 and FP rate of 0.0. Kicking it up a notch: Voting Taking our 38 layers detector with similarly trained two other additional detectors outputting the majority vote of three slightly better. Would be better if all three were independent. Ohad Hageby IDC Ohad Hageby IDC Conclusions Summary Fast detection system, real time. Good results on hard data sets (light, scale, position etc.). Add to the validity of results. New and useful way to represent image Integral Image. Simple features which are robust and easy to scale. Ohad Hageby IDC Ohad Hageby IDC

14 Summary AdaBoost learning algorithm that ensures that achieve large margins rapidly. The training error zeros exponentially in the number of rounds. Ohad Hageby IDC Ohad Hageby IDC

Classifier Case Study: Viola-Jones Face Detector

Classifier Case Study: Viola-Jones Face Detector P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001. P. Viola and M. Jones. Robust real-time face detection.