Boosted Detection of Objects and Attributes

Transcription

1 L M M Boosted Detection of Objects and Attributes Abstract We present a new fraework for detection of object and attributes in iages based on boosted cobination of priitive classifiers. The fraework directly iniizes the detection error by learning a set of siple, coputationally efficient threshold-based detectors. We apply this fraework to segentation of huan skin and detection of faces in iages. We show that despite its siplicity the ethod perfors on par with ore coplex traditional odels in detection accuracy while outperforing the in scalability. This can be especially beneficial in applications of the fraework to real-tie detection tasks. 1. Introduction Statistical odels for detection of objects and their attributes in iages have been popular for a nuber of years. Different odels of varying degrees of coplexity have been proposed and utilized in tasks such as edge and color detection [1, 2, 3, 4, 5], and face detection and recognition[6, 7, 8, 9], to nae just a few. A coon thread of these approaches is that their iproved perforance usually cae at the cost of high coputational and algorithic coplexity. Siilarly, coputationally sipler techniques often required large aounts of data to copensate for the weaknesses of the odel. Exaples of these trends ay be seen in, for instance, face detectors of Rowley et al. [8] and skin odels of Jones and Rehg [5]. Recent developents in the theory of achine learning and applied statistics have shown that siple cobinations of weak odels (learners, classifiers, regressors) ay have a surprisingly effective cobined perforance. Techniques such as bagging and boosting[10] have proved both practically and theoretically that particular choices of cobining siple statistical learners ay significantly reduce not only the training error but also the ore elusive generalization error. In this paper we show how one such technique can be utilized for the task of object detection. We first describe a general theory of boosting, otivated by a nuber of its appealing theoretical properties. We then propose a boosted use of siple thresholding classifiers that leads to coputationally siple yet highly accurate detectors of objects and their attributes in iages. Finally, we deonstrate the utility of our approach by constructing siple boosted detectors for the probles of color-based iage segentation and face detection. 2. Boosted Cobination of Classifiers Boosting (c.f. [10]) is a learning technique which has caught attention in the statistical and achine learning counities. The Boosting algorith works by sequentially training a series of weak learners which are then cobined into a single strong classifier. Each weak learner (classifier) attepts to iniize classification error on a particular distribution of the training data. Freud and Schapire [10] proved that the boosted cobination of classifiers not only iniizes the epirical classification error ( error on the training set ) but also leads to a iniized bound on the true Bayesian error ( error on the test set. ) Various classifiers of different coplexity such as naive Bayes, decision trees and Bayesian networks, have been used in place of the weak learner and have all lead to significantly iproved (boosted) final classification perforance in a nuber of epirical studies. Consider a supervised binary classification proble with training data given by! " #%$&'$, where is the feature vector and is its label,. The (*)+-,/. goal of the learning algorith is to solve for a classifier where that iniizes the generalization error Pr 7%8 ( 9 :<;= is the distribution of the data?@a. However, direct iniization of is ipossible and one attepts to iniize the epirical error 03 CB Pr 7 D 8 ( 9 "; "GIHJ where now denotes?@a the epirical distribution on the training set and G HJ is the M #F$ classification loss defined as GCHNJ PO M ( RQ $S ( RT Nevertheless, iniization of the epirical error reains a difficult task. Schapire and Freud [10] instead suggest iniization of an upper bound on the classification M #:$ loss GIHJ?@ARU GRV4W "X4Y[Z\ ]#^ 1

2 l l J $ ( This task becoes feasible and can be accoplished using the Adaboost algorith: #%$&'$ 1. Given 2. For j $@k ADABOOST < _ `'@àb*c initialize d e]i $gf4hi and dis- (l (a) Train classifier using data tribution. (b) Choose l $^# 0 l noprq 0 lts where 0 l vu!w 7yx[8 (l za9 "{; (c) Update l e]i} 3. The final hypothesis is l _e6[x4y[z? ]# ~ _ C vnez ƒ E l. l (l =4 l ( l 6 " Adaboost algorith starts by assigning equal weight to all training exaples. In each iteration, the weight of the isclassified saples is increased and the weight of correctly classified saples is decreased. In other words, each successive classifier focuses on saples not learned by the previous odels. The weighting factor on data at step j depends on the k expected error 0 l ade by the previous classifier. After iterations the final classifier k is constructed as a linear cobination of individual odels. The weight given to each classifier in the cobination is proportional to the epirical error the classifier akes on the training set. As entioned before, the Adaboost algorith can be shown to iniize a bound on the epirical error. If the weights ( ) are chosen in the way described above, the training error is bounded by n Š 0 l ]$Œ# 0 l = l ˆ One can note that, if each of the individual weak classifiers (1) perfors at least slightly better than the rando, the training error decreases exponentially. Schapire et al. [10] have also shown that the generalization error of the final hypothesis 0 13]5 is bounded by U Pr 7 D ŽE l(l U <?& y hi for any positive and (the VC-diension of.) l ( l A can be viewed as the argin of point and Adaboost indeed axiizes the argin of the cobined classifier. Epirically it has been shown that k Adaboost is able to generalize well for any reasonable. We next show how boosting can be used to construct highly accurate yet siple detectors of objects and their attributes in iages. We eploy (l siple thresholding functions as the weak classifiers. Training of each weak classifier now reduces to optial selection of its threshold. We deonstrate the utility of the boosting approach on two coon coputer vision tasks: color-based iage segentation and face detection. 2.1 Boosted Skin Color Segentation Segentation of iages into skin color regions and nonskin color regions is a coon practical proble in coputer vision. Skin segentation is often the first step in solving ore coplex vision probles. For exaple, one can use skin color-based iage segentation to localize different body parts and then use ore coplex algoriths in just those portion of the iages to do iproved tracking, detection of hands and faces. However, because ost of this task relies on the perforance of the basic color based segentation technique, it becoes iperative that the ethod adopted is accurate. At the sae tie, it is also necessary to have a coputationally efficient technique as one cannot afford a high overhead fro the basic segentation. A nuber of skin detection techniques have been suggested in literature (c.f. [1, 2, 3, 4, 5]). Aong the ost successful are learning-based statistical techniques. In general the techniques can be divided into two categories: paraetric, such as Gaussians ixtures, and non-paraetric, such as histogras. Gaussian ixture ethods [1, 2] work well for saller aounts of training data (provided they are sufficient to learn the paraeters) but do not show good generalization. Moreover, coplex Gaussian odels require overhead in floating coputations which can ake the ipractical for ipleentation on, for instance, hand held devices. Histogra-based techniques [5] exhibit very good perforance, but they require large aounts of training data and fine bin size. Boosting provides an alternative to these ethods by reducing both the eory and the coputational requireents. By optially cobining a set of siple skin color 2

3 M p E classifiers, boosted skin detector optiizes skin detection rate on the training set of iages and, at the sae tie, generalizes well to unseen data. In our ipleentation, boosted skin detector classifies each iage pixel as either skin or non-skin. The detector consists of a nuber of weak, threshold-based detectors: (l š4 _ œ A@žNI Ÿez < I _ \ = # < Here, ž denotes the iage pixel, is the decision threshold, and ž 2ª@«selects the color coponent of the pixel, for instance. Therefore, each threshold detector takes one color coponent of the pixel as its input, copares it against threshold and labels it as the skin or non-skin. Optial choice of, or learning of weak classifiers in the boosting fraework, ž < corresponds to the threshold and coponent that yield the lowest isclassification rate on the training set of pixels. In this work we assue discrete-valued color coponents, hence a fixed $N set of possible thresholds,, suffices. For instance, can be 255 or soe other integer. For a coponent color space ( for RGB, for instance) %± the space of all possible threshold functions has eleents. However, since our ethod is treating coponents independently of each other, the effective space reduces to. The boosted color detector algorith is now siply the ADABOOST algorith of Section 2 with the following training step: SKINBOOST 2(a) Select a threshold function aong possible functions which iniizes 1 the classifica- ( l ²š4 _ 4 tion error on the training set, such that l I Ņ¹ l e] where is skin or non-skin. 8 š4 \œ A2žN9 º ;< k The upper liit on the nuber of weak classifiers is approached when the training error falls to zero or when each subsequent weak detector fails to yield further iproveent ( 0 l ¼» ). 1 It can be easily shown that the error of this classifier is less than ½¾4, provided we also allow negating the output of the threshold function. This eans that one effectively needs to double the nuber of weak detectors to À Á!ÂÄÃÆÅ, corresponding to Ç=ÈÉÊ and Ë\Ç=ÈÉÊ. Boosted skin detector ~ _ obtained in this fashion relies on the superposition of weighted thresholding functions. An exaple of the coplex threshold functions is shown in Figure 1. Note that the final boosted detector is a Red Green Blue Figure 1: Boosted color detector. Shown are linear cobinaitons of weak detectors (thresholding functions) assigned to R, G, and B channels, respectively. Final detector Ì*Í{Î[Ï is a weighted and then thresholded cobination of the three (R,G,B) functions. Horizontal axis corresponds to intensities in the three channels. thresholded linear cobination of the three functions. This superposition, in turn, results in adaptive partitioning of color space into regions assigned to one of the two classes, skin and non-skin. Even though the classifiers at each step of this algorith are trivial and the optiization/learning can be accoplished by a siple exhaustive search, the cobined perforance of the boosted odel is very good. Moreover, h detection process siply involves evaluation of threshold functions which can be efficiently ipleented using lookup tables. In the next section we present results of skin color segentation using our classification technique. We also include a coparison with both the Gaussian ixtures and the histogra odels in two different color space. 2.2 Boosted Face Detection Face detection is another iportant basic task often encountered in coputer vision. A nuber of applications have been developed that require recognition of huan faces. Even though face detection is a difficult task and various ethods that have been used are often coplex, we show how a siple boosted algorith can be used to accurately 3

4 Ð p E and efficiently solve this detection task. A nuber of odels, priarily statistical, have explored various ways of odeling faces [6, 7, 8, 9]. Unlike the task of color-based segentation which iposes little if any spatial constraints on the odel, detection of faces also relies on spatial constraints. To extend the proposed boosting fraework to detection of faces we consider a set of siple weak classifiers of the following for: (l š4 KÐ œ A@ÑzI vez < I +ÓÒÓ?# < now denotes a vectorized iage of gray-scale pixel values and +ÓÒÓ is its ÑÆ# š ( coponent (pixel). Analogous to the color detection task, each threshold detector takes one iage pixel as its input, copares it against threshold and labels it as the face or non-face. Learning of the weak classifiers now corresponds to the threshold and pixel Ñ that yield the lowest rate of error on the training set of iages. The rest of the face detection algorith assues the sae general for of ADABOOST where the weak classifier learning step becoes FACEBOOST 2(a) Select a threshold function which iniizes the ( classification error on the training set, l :š4 _ 4 such that l I v³ 2µI 4¹ Ò l _e6 where is face or non-face. 8 š4 KÐÔœ A2ÑKa9 "{;< Even though it sees trivial for the task of face detection, experiental results in the next section indicate that its perforance is better than that of soe coplex classifiers such as support vector achines. Boosted face detection Ñ=l <l algorith selects an optial subset of iage pixels and thresholds corresponding to those pixels that best discriinate faces fro non-faces, while iniizing the detection error. Figure 2(a), shows an exaple of an average face obtained fro training data. Figure 2(b) shows a typical face iage sapled fro the function learned using boosting. Each non-white location corresponds to a pixel selected by the boosting algoriths. Intensity of each selected pixel is in turn deterined by a linear cobination of the thresholding functions. Siilar to the case of the color classifier, the selected pixel ay have one or ore such thresholding functions that specify ranges of intensities associated with faces and non-faces. The exaple iages indicates that the pixels selected by the algorith are the ones which correspond to the iportant features of the face (eyes, outh, nose and hair). This follows our intuition and agrees with the average face shown in Figure 2(a). (a) face Average (b) Boosted face Figure 2: Points on the boosted face align with ost descriptive features on the average face. Boosted face was obtained by sapling fro the learned boosted threshold odel. It is once again iportant to stress that the boosted face detector selects only a subset of all iage pixels. This clearly reduces the coputational coplexity of detection because only a sall nuber of pixels needs to be exained. Moreover, coputation of the threshold function can be ipleented using a fast lookup table. In the next section we present details of the detection experients. We copare perforance of the boosting ethod with standard ethods like SVM and nearest neighbor classifier. 3. Experients We conducted two series of experients to evaluate the power of our boosted detector odels. The first set of experients involved segentation of iages into skin and non-skin color regions. The second set of experients dealt with the face detection Skin Color Segentation The algorith fro Section 2.1 was used for skin color segentation. We have copared its perforance to two standard skin color odels based on histogras and Gaussian ixtures. We selected a boosted skin detector with 16 threshold levels in each color channel. Coparable histogra and Gaussian ixture odels had 16 unifor bins in each channel and three ixture coponents, respectively. Finally, we considered three typical color spaces: RGB, noralized RGB, and HSV. All odels were trained and evaluated on the sae dataset of 1200 iages. Ground truth labeling of all iage pixels in the data set was done anually. 4

5 Method Err Det False Pos Boosting Mixture of Gaussians Histogra Table 1: Percentage error, detection rate, and false positives (per pixel) in RGB space. Figure 3.1 shows an exaple of the epirical error behavior on the training and the test sets during training. As expected, the test error continued to decrease even when the training error becae stationary, indicating good generalization perforance of the odel. Method Err Det False Pos Boosting Mixture of Gaussians Histogra Table 2: Percentage error, detection rate, and false positives (per pixel) in noralized RGB space. Method Err Det False Pos Boosting Mixture of Gaussians Histogra Table 3: Percentage error, detection rate, and false positives (per pixel) in HSV space. one of those iages. Figure 3: Error of classification on the training and test sets as a function of the nuber of weak detectors. Note that the test error continues to decrease even when the training error reaches a plateau, as the nuber of weak detectors increases. Skin segentation results are suarized in Tables 3.1 through 3.1. We observe that in each case the boosted detector copares favorably in error rate and false positive rate to both the histogra ethod and the ixture of Gaussian. This suggests that a siple boosted detector ay be the desired choice aong the three odels given its low coputational coplexity and eory requireents. Finally, we applied our boosted skin detector to a nuber of arbitrary color iages fro the web. Figure 4(b) illustrates very good skin segentation results obtained on 3.2. Face Detection Experient We conducted a set of preliinary experients to evaluate the boosted face detector. Our training set contained 1400 frontal face iages cropped out of iages downloaded fro the web. Each iage was rescaled to a 16x16 window and noralized for intensity. We randoly chose another 1400 iages which did not contain any faces. Part of the data was used for training and the rest was eployed for testing (five fold crossvalidation was done to obtain consistency in the perforance easure.) Table 3.2 outlines classification perforance for four different classifiers: boosted face detector, support vector achine (SVM) with a linear kernel function, SVM with an RBF kernel, and a nearest neighbor classifier. It is evident that the boosted detector outperfors two of the classifiers and perfors on par with the coplex RBF SVM in the nuber of false positives. Although nearest neighbor classifier gave the best detection perforance it also had a very high false positive rate. The error variance in all cases was less that $gõ, which indicates significance of our results. Figures 5(a) and 5(b) depict results of applying the boosted face detector on two arbitrary web iages selected to contain ultiple faces. Again, the coplexity of the boosted classifier is less than that of the ost other classifiers. Note also that the Method Detection False Positives Boosting SVM (linear) SVM (RBF) Nearest Neighbor Table 4: Percentage error in ters of percentage of faces correctly detected and the percentage of faces isclassified. 5

6 (a) Original (a) (b) Skin color ask Figure 4: Detection of skin color in an arbitrary web iage using the boosted skin detector. (b) Figure 5: Detection of faces in arbitrary web iages using the boosted face detector. 6

7 Õ boosted classifier facilitates efficient ipleentation of the ultiscale search because the evaluation is liited to a subset of iage pixels. 4. Discussion We have shown in the previous sections that optial cobination of siple classifiers, deterined by boosting, even in a basic state space (e.g., RGB or noralized intensities) can lead to detectors whose perforance is coparable to that of soe coplex classifiers. One reason for this ay lie in the coplexity of the state space partitions obtained in this fashion. Our fraework allows ultiple siple classifiers to for coplex partition of the state space, adapted to the statistics of the training set but, also, with soe guarantees with respect to the test set. These guarantees indeed see to hold well for the set of experients we conducted for instance, perforance of both skin and face detectors reains high over relatively adverse lighting conditions present in arbitrary web iages. The use of ore coplex state spaces ( features ) ay slightly iprove detection perforance in ters of (generalized) detection error, but it coes at the additional cost in coputational coplexity. This ay hinder applicability of these detectors in low-resource applications (e.g., handheld devices.) The inherent perforance score of the detector is the syetric total classification error, based on the G HJ syetric loss. As such, the score equally penalizes all types of errors (false and true positives and negatives). We have epirically shown that, in spite of that, our detectors exhibit a good false positive perforance, indicating their optiality and good generalization properties. Unfortunately, basic forulation of the boosting theory does not allow one to iediately utilize non-syetric loss functions and, thus, exhibit control over individual error types. This is one reason for the lack of ROC evaluation, coonly seen in other detection studies. 5. Previous Work Boosting theory has begun to find its application in coputer vision in recent years. For instance, Viola et al. [11] and Pavlovic et al. [12] have both used boosting to cobine classifiers of different coplexity. Viola s work focused on iage search in large databases. They used coplex filters and boosting of a single threshold classifier to generate discriinative features. On the other hand, in our fraework we allow ultiple thresholding functions to generate ore coplex partitions of the sipler state space. Pavlovic et al. used dynaic Bayesian networks for event detection in video and eployed boosting to iprove this detector s perforance. Unlike ours, this approaches attepted to utilize coplex base classifiers that were not always guaranteed to perfor better than» M, a constraint necessary for boosting. 6. Conclusions and Future Work We have presented a statistical fraework for detection of objects and attributes in iages based on boosting of weak classifiers [10]. By selecting a siple thresholded function as the weak classifier we have deonstrated how to obtain optial detectors of skin color and faces. These siple yet highly efficient classifiers guarantee iniization of the training error as well as good generalization perforance. Our preliinary results on two representative sets of web iages suggest that detection perforance of the boosted detectors is on par with that of the ore coplex traditional detection ethods such as histogras, Gaussian ixtures, and support vector achines. However, our ethod is of lower coplexity and trivial to ipleent and execute. As such, the boosted detection ethod ay prove appropriate for applications in real-tie and low-resource coputing devices, such as handhelds. We plan to conduct further coparative tests of the boosted detectors against other state-of-the-art odels. Furtherore, we will extend the study to include coparison with the weak detectors that operate on ore coplex features, such suary features on iage neighborhoods. References [1] T. S. Jebara and A. Pentland, Paraeterized structure fro otion for 3d adaptive feedback tracking of faces, in Proc. IEEE Conf. on Coputer Vision and Pattern Recognition, pp , [2] J. Yang, W. Lu, and A. Waibel, Skin-color odeling and adaptation, in Proc. ACCV, pp , [3] D. A. Forsyth, M. Fleck, and C. Bregler, Finding naked people, in Proc. European Conference on Coputer Vision, pp , [4] J. Z. Wang, J. Li, G. Wiederhold, and O. Firschein, Syste for screening objectionable iages using daubechies wavelets and color histogras, in Proc. IDMS, [5] M. J. Jones and J. Rehg, Statistical color odels with application to skin detection, in Proc. IEEE Conf. on Coputer Vision and Pattern Recognition, pp , [6] M. Turk and A. Pentland, Face recognition using eigenfaces, in Proc. IEEE Conf. on Coputer Vision and Pattern Recognition, pp ,

8 [7] K. Sung and T. Poggio, Exaple-based learning for view-based huan face detection, a.i. eo 1521, MIT, Deceber [8] H. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection, in Proc. IEEE Conf. on Coputer Vision and Pattern Recognition, pp , [9] T. Rikert, M. Jones, and P. Viola, A cluster-based statistical odel for object detection, in ICCV, [10] Y. Freund and R. E. Schapire, Experients with a new boosting algorith, in Machine Learning: Proceedings of the thriteenth international conference, pp , [11] K. Tieu and P. Viola, Boosting iage retrieval, in CVPR00, pp. I: , [12] V. Pavlovic, A. Garg, J. Rehg, and T. Huang, Multiodal speaker detection using error feedback dynaic bayesian networks, in CVPR00, pp. II:34 41,