Adaptive Learning of an Accurate Skin-Color Model

Transcription

1 Adaptive Learning of an Accurate Skin-Color Model Q. Zhu K.T. Cheng C. T. Wu Y. L. Wu Electrical & Computer Engineering University of California, Santa Barbara Presented by: H.T Wang

2 Outline Generic Skin Model Adaptive Skin Model Experiments & Applications Conclusions

3 Generic Skin Model Three different models Histogram Single Gaussian Model (SGM) Gaussian Mixture Model (GMM) Comparison Accuracy: GMM > SGM > Histogram Efficiency: Histogram > SGM > GMM Size of training set: Histogram > GMM > SGM Color space Normalized (rg( rg,, HSV, TSL) Un-normalized normalized (RGB)

4 Experimental Databases Skin (STD), Non-Skin (NSD) and Testing (TDSD) Datasets Dataset STD Number of Images 2720 Skin pixels 151 million Non-Skin 0 NSD million TDSD million 75 million All images randomly picked from Web Labeling the skin region manually, using Photoshop

5 Comparison: ROC Curves of Generic Skin Models on TDSD Detection Rate GMM SGM Histogram False Positive Rate

6 Outline Generic Skin Model Adaptive Skin Model Experiments & Applications Conclusions

7 Why Adaptive? Skin Distribution in HS space Non-Skin in HS space Probability Probability H Value S Value H Value S Value 151 million skin pixels 448 million non-skin pixels Non-trivial overlap between two distributions 98% of color bins containing skin pixels also contain non-skin pixels

8 Why Adaptive? A generic skin model An adaptive skin model

9 Two-Step Adaptive Framework All Image Pixels Skin-Similar Pixels EM Learning Training Dataset Generic Skin Model 1 st Skin Classification Gaussian Mixture Model (Skin and Non Skin Gaussian) SVM Classifier 2 nd Skin Classification Adaptive Skin Model Non-Skin Pixels Skin-Similar Pixels True-Skin Pixels False-Skin Pixels

10 Why two steps? Color distribution of all pixels in an image often spreads over the entire space Harder to analyze and model Color distribution of Skin-Similar Similar pixels quite compact and simple Effective analysis possible Divide one hard task into two easier subtasks

11 Skin-Similar Space Examples True skin a dominant Gaussian False skin a weak Gaussian Two Gaussians are separable Non-Skin Gaussian H Value Skin Gaussian S Value Non-Skin Gaussian Skin Gaussian Non-Skin Gaussian Skin Gaussian H Value S Value H Value S Value

12 Quantitative Analysis for 554 images (TDSD) Experiment1: goodness of fit of Skin Gaussian Assuming a single skin Gaussian in the Skin-Similar space. Calculating the mean and covariance matrix from the ground truth. Observing the ratio of true skin pixels covered by this Gaussian for a fixed Mahalanobis distance.

13 Goodness of fit Curve Ratio Image A high ratio maintained for a small distance A compact Skin Gaussian existing

14 Quantitative Analysis for 554 images (TDSD) Experiment2: Theoretical Upper Bound Analysis With ground truth, build two histograms: One for true-skin pixels The other for false-skin skin pixels Calculate the overlap between two histograms: This equation evaluates the upper bound of achievable performance improvement

15 Experimental Results Theoretical Upper Bound ROC Curve Detection Rate False Positive Rate An example In the first skin classification: DR 1 = 96% FP 1 = 50% Choose DR and FP from this curve in the second step: DR 2 = 95% FP 2 = 40% The final performance: DR = DR 1 DR 2 = 91.2% FP = FP 1 FP 2 = 20% Reduces the FP by 30% at the cost of 4.8% reduction in DR

16 Model the Skin-Similar Similar Space Gaussian Mixture Model One for true-skin pixels One for non-skin pixels Non-Skin Gaussian H Value Skin Gaussian S Value

17 EM Based Adaptive Modeling A structure assumption, i.e. the number of Gaussian kernels in a GMM (k = 2 in our application) An initial guess of mean and variance for each Gaussian

18 Step1: Obtain an Initial Guess Marginal Probability in HS color space Choosing two separated peaks as means A small covariance matrix is preferred Non-Skin Gaussian Skin Gaussian Probability H Value S Value S Value

19 Step 2: EM Learning Starting from the initial guess, update the GMM parameters as follows:

20 Step 2: EM Learning (result) Blue surface: Ground truth Red surface: Trained GMM Probability H Value S Value

21 Step 3: Identifying Skin Gaussian Extracting features for each Gaussian Group A features (color distribution related): weight, Gaussian mean, Gaussian variance. Group B features (spatial and shape related): spreadness,, elongation, X-direction X and Y-Y direction histograms. A SVM classifier is trained and used to identify the skin-gaussian from the trained GMM with two Gaussians

22 Step 3: Identifying Skin Gaussian Experimental Results Probability that the right Gaussian is identified: Feature Set Group A Group B Group A+B Training Accuracy 91.02% 88.96% 96.83% Testing Accuracy 87.73% 90.97% 96.57%

23 Outline Generic Skin Model Adaptive Skin Model Experiments & Applications Conclusions

24 Using Different Number of Gaussian Kernels (1,3,5) for Modeling False-Skin Pixels in Skin-Similar Space False Positive One Gaussian Three Gaussians Five Gaussians Detection Rate

25 Generic Skin-Model vs. Adaptive False Positive Adaptive Skin-Model using SVM classifier Adaptive Skin-Model using heuristic rules Generic Skin-Model Detection Rate

26 Skin Detection for Still Images

27 Skin Detection for Still Images

28 Skin Detection for Still Images

29 Skin Detection for Still Images

30 Face Tracking in Videos

31 Hand Tracking in Videos

32 Outline Generic Skin Model Adaptive Skin Model Experiments & Applications Conclusions

33 Conclusions Propose a two-step adaptive approach Skin-Similar Similar Space EM based Adaptive Modeling Experimental results demonstrate the effectiveness and advantage of our proposed method

34 Thanks!