Viola-Jones with CUDA

Size: px

Start display at page:

Download "Viola-Jones with CUDA"

Roger Blake
5 years ago
Views:

1 Seminar: Multi-Core Architectures and Programming Viola-Jones with CUDA Su Wu: Computational Engineering Pu Li: Information und Kommunikationstechnik

2 Viola-Jones Face Detection Overview Application Of Face Detection-HowOldRobot General Process Of Face Detection Rectangle Feature Integral Image Adaboost Training Cascade Of Classifiers Project and Simulation

3 Application HowOldRobot HOW OLD DO I LOOK?

General Process Of Face Detection Question: How do we usually find a face in a picture? hair eyes According to the features nose mouth How can a computer find a face in a picture?

4 General Process Of Face Detection Question: How do we usually find a face in a picture? hair eyes According to the features nose mouth How can a computer find a face in a picture? Training Process: 1. Collect large numbers of examples (both positives and negatives). 2. Extract the important features of these examples. 3. Construct a classifier to detect the face in a picture.

5 General Process Of Face Detection Training Process Face Detection Process Face Non-Face Input Image Integral Image Adaboost Training Face detection using the Cascade Classifier Cascade Classifier Mark the position of faces

6 Rectangle Feature Interpret the features of the face. e.g. The region of eyes is often darker than cheeks. e.g. The region of eyes is often darker than the bridge of the nose. The features that usually be used:

sum1 sum2 What means the value of pixel?

7 Rectangle Feature Idea: Give each rectangle feature a value. value= sum1(pixels in white area) sum2(pixels in dark area) sum1 sum2 sum1 sum2 sum1 sum2 What means the value of pixel? RGB Color model (8 bits) Value of Red=0-255 Value of Green=0-255 Value of Blue=0-255 Grey scale (8 bits) 255 White f (R)=f (G)=f (B) 0 Black

The value of the integral image at point(x,y) is the sum of all pixel above and to the left.

8 Integral Image Question: How can we calculate the value of feature fast? Method: Integral Image Scan the Image and get the value of each pixel. The value of the integral image at point(x,y) is the sum of all pixel above and to the left. x ii(x,y)=ii(x-1,y)+s(x,y) s(x,y)=s(x,y-1)+i(x,y) y Sum of the pixels within rectangle D can be computed with four array references: D=

9 Integral Image x A C E B D F Rectangle_Feature_value f = pixel sum in white area - pixel sum in dark area = (ii(f)-ii(d)-ii(e)+ii(c)) - (ii(d)-ii(b)-ii(c)+ii(a)) =17-3=14 y e.g. If the threshold is 10, 14>10, so this feature is about a face. If the threshold is 15, 14<15, so this feature is about non-face.

10 Adaboost Training Problem: The whole feature set is too large! e.g. a 24*24 window have more than 160,000 rectangle features! Solution: Select a small set of critical features to make it more efficient. What kinds of features are critical features? The features which best separate the positive and negative examples. Find the best threshold of these critical features. positive examples negative examples not critical features with bad threshold critical features with the best threshold

11 Adaboost Training Combine many weak classifiers to become a strong classifier! 1. Input positive and negative examples. 2. Initialize weights of examples. 3. Set up the loops T. 4. For each feature, train a weak classifier with different thresholds. 5. For each weak classifier choose the threshold, which makes the lowest error. 6. Choose the classifier, which makes the lowest error, as the 1 st weak classifier. 7. Look at where it makes errors, reweight the data so that the inputs where we made errors get higher weight in the learning process. 8. Now train the 2 nd weak classifier in the same way Combine the 1 st, 2 nd weak classifiers with the different weight α to become a strong classifier.

Classifiers The experiments show us: using Adaboost Training we just need 200

12 Adaboost Training Weak Classifier 1 Weights Increased Weak Classifier 2 Weights Increased Weak Classifier 3 Final Classifier is a combination of Weak Classifiers The experiments show us: using Adaboost Training we just need 200 critical features instead of the 160,000 features to construct a efficient face detector.

13 Cascade Of Classifiers Problem: For each position of the window, we have to calculate the value of 200 features, still too much calculations. Solution: Construct a cascade of classifiers which dramatically increases the speed of the detector by quickly discarding the negative sub-windows. Classifier 1 Classifier 2 Classifier 3

14 Cascade Of Classifiers Begin with simple classifiers to reject many negative sub-windows, so that many non-faces are rejected at the first few stages. The next stage has more features to distinguish unclear sub-windows. Window with no face Window with face

15 Project and Simulation Pseudo Code for number of scales in image pyramid do downsample image by one scale compute integral image for current scale for each shift step of the sliding detection window do for each stage in the cascade classifier do for each filter in the stage do filter the detection window end accumulate filter outputs within this stage if accumulation fails to pass per-stage threshold do break the for loop and reject this window as a face end end if this detection window passes all per-stage thresholds do accept this window as a face else reject this window as a face end end end

$<<16 and >>16 to avoid lost of precision int x_ratio = (int)((w1<<16)/w2) +1; int y_ratio = (int)((h1<<16)/h2) +1; int x2, y2 ; for (int i=0;i<h2;i++) { for (int j=0;j<w2;j++) { x2 =$

16 Nearest Neighbor Image Scaling public int[] resizepixels(int[] pixels,int w1,int h1,int w2,int h2) { int[] temp = new int[w2*h2] ; // EDIT: added +1 to account for an early rounding problem //using <<16 and >>16 to avoid lost of precision int x_ratio = (int)((w1<<16)/w2) +1; int y_ratio = (int)((h1<<16)/h2) +1; int x2, y2 ; for (int i=0;i<h2;i++) { for (int j=0;j<w2;j++) { x2 = ((j*x_ratio)>>16) ; y2 = ((i*y_ratio)>>16) ; temp[(i*w2)+j] = pixels[(y2*w1)+x2] ; } } return temp ; }

of the rectangular region in the corresponding integral image. The calculation becomes, 46 22 20 + 10 = 14.

17 Integral Image To calculate the summation of a sub-region of an image, you can use the corresponding region of its integral image. For example, in the input image below, the summation of the shaded region becomes a simple calculation using four reference values of the rectangular region in the corresponding integral image. The calculation becomes, = 14. The calculation subtracts the regions above and to the left of the shaded region. The area of overlap is added back to compensate for the double subtraction.

18 Sliding Window A detection window shifts around the whole image at each scale to detect the face, as shown in the figure below. In the provided implementation, the sliding window shifts pixel-by-pixel. Each time the window shifts, the image region within the window will go through the cascade classifier, which will be explained next.

19 Cascade Classifier A cascade classifier consists of multiple stages of filters, as shown in the figure below. Each time the sliding window shifts, the new region within the sliding window will go through the cascade classifier stage-by-stage. If the input region fails to pass the threshold of a stage, the cascade classifier will immediately reject the region as a face. If a region pass all stages successfully, it will be classified as a candidate of face, which may be refined by further processing.

20 Haar Filter * Read the filter parameters in class.txt * Each stage of the cascade filter has: * 18 parameter per filter * filters per stage * + 1 threshold per stage * * For example * the first stage has 9 filters, * the first stage is specified using * 18 * = 163 parameters * * The 18 parameters for each filter are: * 1 to 4: coordinates of rectangle 1 * 5: weight of rectangle 1 * 6 to 9: coordinates of rectangle 2 * 10: weight of rectangle 2 * 11 to 14: coordinates of rectangle 3 * 15: weight of rectangle 3 * 16: threshold of the filter * 17: alpha 1 of the filter * 18: alpha 2 of the filter

21 Face detection process 1. Slide the detect window from left to right along each row, calculate the value of 200 features, compare with threshold, then weighted voting. 2. Enlarge the window at each location. 3. Merge the overlapping windows. Demonstrate

22 Result

Learning to Detect Faces. A Large-Scale Application of Machine Learning

Learning to Detect Faces. A Large-Scale Application of Machine Learning Learning to Detect Faces A Large-Scale Application of Machine Learning (This material is not in the text: for further information see the paper by P. Viola and M. Jones, International Journal of Computer