Adaptive Feature Selection via Boosting-like Sparsity Regularization
|
|
- Percival Jacobs
- 5 years ago
- Views:
Transcription
1 Adaptive Feature Selection via Boosting-like Sparsity Regularization Libin Wang, Zhenan Sun, Tieniu Tan Center for Research on Intelligent Perception and Computing NLPR, Beijing, China {lbwang, znsun, Abstract In order to efficiently select a discriminative and complementary subset from a large feature pool, we propose a two-stage learning strategy considering both samples and their features simultaneously, namely sample selection and feature selection. The objective functions of both stages are consistent with a large margin loss. At the first stage, the support samples are selected by Support Vector Machine (SVM). At the second stage, a Boosting-like Sparsity Regularization (SRBoost) algorithm is presented to select a small number of complementary features. In detail, a weak learner is composed of a few features, which are selected by a sparsity enforcing model, and an intermediate variable is gracefully used to reweight the corresponding sample. Extensive experimental results on the CASIA-IrisV4. database demonstrate that our method outperforms the state-of-the-art methods. 1 Support Samples 2 Sparsity reweight S 1 3 Boosting-Like S 2 4 Keywords-feature selection; Boosting; sparse; Sample Selection Feature Selection I. INTRODUCTION Feature selection aims to select a small subset of compact and discriminative features. In biometrics, object classification and recognition, one image is usually represented by local feature descriptors, such as SIFT [1], and Ordinal Measures (OM) [13], which are extracted at every pixel by certain filters. So a large feature pool is generated, and it is over complete to describe the image itself. In this case, feature selection is brought to deal with high dimensional data. Many related algorithms have been presented during the past several decades. Among them, AdaBoost [6], [14] is a class of successful methods. They select a new feature (weak learner) on the reweighted samples heuristically. Recently, sparsity enforcing models [7][15][11] have attracted great attention, and achieved competitive performance especially in the case of small scale of training samples. These sparse models address the feature selection problem as an l or l 1 regularization optimization in common, which enforces the weights of features sparse. Besides the regularization term, the loss function is another significant element. Destrero et.al. [5] adopt Least Squares (LS) directly as a loss function with application to face detection. He et.al. [7] propose a correntropy based robust estimation loss to tackle the non-gaussian noise. And Wang et.al. [15] formulate feature selection as a Linear Programming (LP) model with a large margin loss, which is robust to noise and outliers as well. In the above models, complementary features are not explicitly taken into consideration, such Figure 1: Flowchart of the proposed method. (1) Sample selection ( 1-2 ). The points inside the dash ellipse line are selected samples. (2) Feature selection ( 3-4 ) by SRBoost. The bold dash lines (S 1, S 2 ) are the selected features by the Simplex algorithm (the polyhedron) respectively. that similar features will share large weights simultaneously. Moreover, the optimization of a sparse model usually involves the computation of matrix [11][7], which is timeconsuming in general. In summary, although the sparsity regularization methods have achieved promising performance, there are some limitations, e.g., they are lacking of considering the distribution of samples. And the complementation of features is not explicitly taken into account. To regard the above problems, in this paper, we propose a two-stage learning strategy, including sample selection and Boosting-like learning. The loss functions of two stages are consistent to promote the performance. At the first stage, the support samples are selected by SVM, which has the Hinge loss function with a large margin principle. At the second stage, a Boostinglike sparsity regularization (SRBoost) algorithm is designed to select the complementary features. In detail, SRBoost iteratively selects a small number of features by sparsity enforcing model, in which a large margin loss is added as well. And the complementary features are selected, because the features selected in each iteration classify training samples with different weights. In general, the proposed model
2 with the large margin loss can be formulated as a linear programming problem, which can be efficiently solved by an iterative Simplex algorithm. Figure 1 illustrates the flowchart of the proposed method. II. BOOSTING-LIKE SPARSITY REGULARIZATION MODEL A. Notations and primary settings Without loss of generality, we consider a binary classification problem, because multi-class problem can be transferred into intra-matching class and inter-matching class for feature selection problem. Assuming that class labels are linear mapping results of feature spaces, we learn the linear function by minimizing the mean squared error. Here {y y j, y j {+1, 1}} denotes the class label, and {X x, x {x +, x }} denotes a data set of D dimensional features, wherein {x +, x } represents the positive and negative samples respectively. The linear decision hyperplane is y Xw =, where w represents the weight vector. B. Sample selection In the first stage, a preprocessing strategy of sample selection is applied to reduce the scale of training samples, simultaneously, the distribution of samples will be maintained for classification. Sampling technique may be an offthe-shelf solution. The classic statistic bootstrap and n out of m bootstrap are important general resampling approaches. However, in order to hold the distribution of the original data, sufficient times of sampling should be taken. From another perspective, in this paper, we take good advantage of the Support Vector Machine (SVM). As we know, the output of SVM is a function of selected support vectors, therefore, sample selection here is implemented in this supervised way. Considering the efficiency, we use the linear SVM without kernels to generate samples. The objective function takes the form [2]: L(w, b, a) = 1 N 2 w 2 2 a n {y n (w T x n + b) 1} (1) n=1 And the linear SVM can be efficiently solved by the Sequential Minimal Optimization (SMO) algorithm [4], [3]. In addition, the support vectors have a good property that they are close to the decision boundary. And they reflect the distribution of samples relatively hard to be classified. Thus it is reasonable that the following feature selection can be just deployed on the selected samples. Figure 1( 1-2 ) shows the process of the sample selection. Generally, the time complexity of sample selection is worthy compared to the following selection step. It is worth mentioning that the Hinge loss of SVM is provided with large margin criterion, which has a close relationship with the following feature selection. Furthermore, the rest of training samples can be cast as a validation set for cross validation. C. SRBoost 1) learning weak classifiers: This step aims to construct weak learners as Adaboost, the difference lies in the learning approach rather than hand-craft one. Specifically, The first step of the second learning stage is to select features by sparsity regularization, the few learnt features constitute one weak classifier. As previously mentioned, the linear decision hyperplane is y Xw =, The original sparsity enforcing methods [5][9] can be summarized as: w = arg min y Xw λ w 1 (2) w To further improve the performance, a robust estimator φ( ) is introduced to deal with non-gaussian noise [7]: w = arg min w N φ((y i X i w)) + λ w 1 (3) i=1 The robust functions have the property that φ(x) is stable even if the independent variable x is very large, e.g., φ(x) = 1 exp( x 2 ) [7], which is different from original least squares loss. In order to be consistent with the objective function (Equation (1)) of sample selection, we still employ a large margin loss function [15] for learning weak classifiers. And considering the Boosting-like strategy, a weighted term k is gracefully introduced to update samples. Therefore, in this paper, we present a sparsity regularization model as: min w T 1 + λ (k T ξ) w T x + j C+ + ξ j, j = 1...N +. s.t. w T x j C ξ j j = 1...N. w i, ξ j i = 1...D, j = 1...N. (4) where 1 is a vector whose elements are all 1. C + and C are determined empirically constants, λ is a regularized parameter balancing the two parts of the objective function. k is a weighted term, it can control the variation of learnt ξ. The objective function is an l 1 minimization with nonnegative constraints, therefore, the weight w is sparse. The constraint terms are to classify the samples in a supervised way, additionally, loss functions with constraints are also yielded under large margin principle [15], which are elegantly consistent with that of the above sample selection step. Finally, the features are selected according to the weights. The important term in our model is the slack variable ξ. Naturally, it has almost the same effect with robust estimator φ, but it is an adaptively learning-based term. For example, if one sample is interfered with large noise, then the corresponding slack variable ξ is also large automatically. In this case, the response values of noisy samples will be suppressed to ensure the learning performance.
3 Algorithm 1 Boosting-like Sparsity Regularized Feature Selection (SRBoost) 1: Input: Data X = {X + R N + D, X R N D }. Output: Weight vector W R D. Initialization: k = 1/N. 2: sample selection: ˆX solving Eqn. (1); 3: for m = 1 : M do 4: w (m) solving Eqn. (4) by ˆX; 5: k (m+1) solving Eqn. (5) or (6); 6: end for 7: W = M w (m) m=1 From another perspective, this slack variable can be viewed as the classification rate of training samples. The smaller the slack variable is, the more confidence we have to classify the corresponding samples correctly. This is key to the idea of the following Boosting-like strategy. 2) Sample reweighting: The goal of this step is to reweight the training samples as Adaboost. Specific speaking, the second step of the second learning stage is boosting the selected features. Complementary features are not explicitly taken into consideration in sparsity enforcing selection methods, thus, there are some of the similar features sharing almost the same large weight. Therefore, complementary analysis is necessary to be added directly to reduce the redundancy. Generally, a pair of complementary features can classify different training samples. In order to implement the above idea, we adopt the sample reweighting strategy inspired by the success of Adaboost [6]. Here we deploy two different functions to update the weights of samples, i.e., linear and exponential penalty: k (m+1) = ξ (t) /(1 T ξ (m) ) (5) k (m+1) = exp(ρξ (m) )/Z (6) where ρ is a learning rate, and Z is the factor of normalization. From the objective function of (4), minimizing k j ξ j, we can see that the larger k j is, the smaller ξ j is learned, which means the samples with large weight should be correctly classified. And under the condition of Equation (6) or (5), the samples with large classification error ξ (m) j will have large k (m+1) j in the previous iteration, thus larger weights are added to these samples in the next round, so that these samples are enforced classified with smaller error ξ (m+1) j in the current iteration. In other words, the selected features in different iterations are complementary to deal with different samples. In the linear case, the update rate of samples is fixed without parameters. But the exponential penalty is more flexible with a tunable learning ratio. In the exponential case, different values of the update ratio ρ show different levels of penalty. If ρ is small, it updates the weights more gentle than the linear case. And if ρ is large, it updates the weights more severe than the linear case. Figure 1( 3-4 ) shows the flowchart of SRBoost. Finally, the selected features are the intersection of the results of M interations. The entire model is a LP problem, thus it can be efficiently solved by the Simplex algorithms. Algorithm 1 describes the proposed SRBoost in brief. In summary, the proposed two-stage learning strategy considers both efficiency and effectiveness, as shown in Algorithm 1. Sample selection is introduced to reduce the computation complexity of feature selection, and meanwhile ensures the local distribution of samples close to the decision boundary. Especially, the second stage SRBoost combines the advantages of sparsity regularization and AdaBoost like methods. In addition, the two stages have a close relationship sharing with a consistent large margin loss function. III. EXPERIMENTAL RESULTS In order to verify the performance of our method, we conduct experiments on feature selection of iris images [12], [15], because the local feature descriptors of biometrics are typically high dimensional. A. Datasets. We evaluate our method on two subsets of CASIA- Iris-V4. database [1]. CASIA-Iris-Thousand (Thousand) contains 2, iris images from 1, subjects. We use Distance subset to verify the generalization of these methods. They are both challenging databases. B. Settings. We use the same settings as in [15]. The iris images are all normalized to the size of 7 54 without preprocessing. 5 iris images from 25 subjects (1 images per eye) in the Thousand database are used for training. We generate 2,25 intra-class matching scores as the positive samples, and 4,9 inter-class matching scores as the negative samples, and the rest of the Thousand subset serves as the test set. We adopt the regional OM [13][8] as our local feature, and the matching scores are computed by Hamming distance. 47,42 regional OM features are extracted for selection. We select 15 features for comparison, and they are enough for a competitive performance. C + and C are set to be.4 and.8 respectively. The algorithms involved in comparison are GentleBoost [6], traditional l 1 regularized sparse methods [5], RRLP [15]. C. Evaluations. We will analyze the experimental results from the following two aspects: learning stage and analysis of the results including parameter selection, accuracy and efficiency.
4 The weight of features 5 Results of the 1st iteration Results of the 2nd iteration The index of features x 1 4 Figure 2: The results of feature selection at the first two iterations. Table I: Comparative results on the Thousand database Methods GentleBoost [6] l 1 [5] RRLP [15] Proposed SRBoost Table II: Comparative results on the Distance database Methods GentleBoost [6] l 1 [5] RRLP [15] Proposed SRBoost ) Learning stage: We train the models to select several numbers of features on Thousand database for iris recognition. Sample selection: The first step is sample selection. Totally, the number of training samples N is 715, including N + = 225, N = 49. The support samples are selected by linear SVM [3] with default parameters. Here we only focus on the samples rather than the performance of SVM. Then 147 support vectors are extracted which are only about 2% of the original training samples. The other samples distributed beyond the decision boundary are not so crucial for feature selection. Intuitively, they are used as an validation set in the following stage to select optimal parameters of SRBoost. SRBoost learning: In the inner loop, a sparsity regularized feature selection via LP is implemented with fixed C + and C. Initially, the weights of samples are set to be 1/N. Then the samples weights are updated according to the learned slack variable ξ via Equation (5) or (6) at each iteration. The former linear update function is convenient without extra parameters, however the latter exponential function is more flexible with tunable update ratio ρ. The large ρ is suitable for the cases with a small number of Boosting-like iterations. In order to compute conveniently, we implement two rounds of iterations, which is also enough for competitive results. For example, if ρ = 3, then the numbers of features selected at two iterations are 27 and 19, respectively. Figure 2 illustrates the feature selection results at the first two iterations. As shown in Figure 2, the features are different and complementary to some extent. Finally, to fairly compare the performance, we select 15 features for all algorithms. For Lasso and RRLP, we select top 15 features by the absolute value of weights. then in our proposed SRBoost algorithm, we select 8 and 7 features in the two iterations respectively. And SVM is applied as the classifier for iris recognition. 2) Performance analysis: In biometrics, ROC curves and Equal Error Rate (EER) are usually employed as measurements of performance. EER is the rate where False Accept Rate () and False Reject Rate () are equal in the ROC curve. The smaller EER is, the better the performance is. Parameter selection: Firstly, we study the impact of different update functions on the performance. Three models are trained with linear case and exponential case (ρ = 1, ρ = 3) respectively. For simplification, the first 5 classes (left eyes and right eyes of 25 subjects) in Thousand database are selected to test the performance except for the training data. As show in Figure 3(a), the exponential update function performs generally better than the linear case, and the large update ratio obtains the best results, that is because only two Boosting iterations are carried, the severe update function of samples ensures the better complementarity of features. The features selected at the second iteration are more prone to classify the hard samples misclassified at previous iteration. To further prove the explanation, the same three models are fed to the Distance database, which is different from the training data. From Figure 3(b), we can see the similar results. The performance of the three models is closer than results on the Thousand database because of the generalization, i.e., the capacity of the training models is not so strong, which suggests that the quality of two subset is of great differences. Comparative results: Secondly, we compare the proposed method to other three state-of-the-art algorithms. The rest images of Thousand database are all testing set, which has 8775 intra-class matching and interclass matching. The sufficient number of samples is enough for testing the algorithms. We adopt exponential update function (ρ = 3) due to the analysis above. As shown in Figure 3(c), RRLP has better results than l 1 sparse method, because it deploys a more robust large margin based loss function. Then the proposed SRBoost performs best, which explains that the Boosting strategy works better, it is necessary to explicitly consider the complementarity of features. The EER and at =% are illustrated in Table I. EER of our method is improved by nearly 5% compared with other methods. Considering the applicability, at =% is.77, which is much lower than
5 ρ=1 exp ρ=3 exp Linear EER curve ρ=1 exp ρ=3 exp Linear EER curve GentleBoost L1 RRLP SRBoost EER Curve GentleBoost L1 RRLP SRBoost EER Curve (a) (b) (c) (d) Figure 3: The ROC curves of feature selection under two kinds of update functions (a) on the Thousand database and (b) on the Distance database. The ROC curves of feature selection compared with other methods (c) on the Thousand database and (d) on the Distance database. The training data sets are from the Thousand database. classical methods. In order to verify the generalization, we also conduct the same experiments on the Distance database, and on this database, we generate 4766 intra-class matching and inter-class matching. ROC curves are shown in Figure 3(d). The recognition rates are consistent with the results on the Thousand database, SRBoost also obtains the best performance, although the results on the Distance database are not so good as those on the Thousand database, our algorithm still shows its potential of good generalization. IV. CONCLUSION In this paper, we have proposed a two-stage learning strategy, including sample selection and feature selection, to select features. Our method considers the samples and their high dimensional features simultaneously, additionally the loss functions of both stages are gracefully consistent based on large margin principle. At the first stage, the support samples are selected by SVM regarding the distribution of training samples. At the second stage, a Boosting-like sparsity regularization (SRBoost) algorithm is presented to select a small number of complementary features. The experimental results on the CASIA-IrisV4. database have demonstrated that our method outperforms the state-of-theart methods. ACKNOWLEDGMENT This work is funded by the National Basic Research Program of China (212CB3163), National Natural Science Foundation of China (Grant No , ), International S&T Cooperation Program of China (Grant No. 21DFB1411) and Instrument Developing Project of the Chinese Academy of Sciences (Grant No. YZ21266). REFERENCES [1] CASIA Iris-V4. Database, [2] M. Bishop, in Pattern Recognition and Machine Learning, 26. [3] C.-C. Chang and C.-J. Lin, Libsvm: A library for support vector machines, ACM TIST, vol. 2, no. 3, p. 27, 211. [4] C. Cortes and V. Vapnik, Support-vector networks, Machine Learning, vol. 2, no. 3, pp , [5] A. Destrero, C. De Mol, F. Odone, and A. Verri, A regularized framework for feature selection in face detection and authentication, IJCV, vol. 83, pp , 29. [6] J. Friedman, T. Hastie, and R. Tibshirani, Additive logistic regression: a statistical view of boosting, Annals of Statistics, vol. 28, p. 2, [7] R. He, T. Tan, L. Wang, and W.-S. Zheng, l2, 1 regularized correntropy for robust feature selection, in CVPR, 212, pp [8] Z. He, Z. Sun, T. Tan, X. Qiu, C. Zhong, and W. Dong, Boosting ordinal features for accurate and fast iris recognition, in CVPR, june 28, pp [9] Y. Liang, S. Liao, L. Wang, and B. Zou, Exploring regularized feature selection for person specific face verification, in ICCV, nov. 211, pp [1] D. G. Lowe, Distinctive image features from scale-invariant keypoints, IJCV, vol. 6, no. 2, pp , 24. [11] F. Nie, H. Huang, X. Cai, and C. H. Q. Ding, Efficient and robust feature selection via joint ;2, 1-norms minimization, in NIPS, 21, pp [12] J. Pillai, V. Patel, R. Chellappa, and N. Ratha, Secure and robust iris recognition using random projections and sparse representations, TPAMI, vol. 33, no. 9, pp , 211. [13] Z. Sun and T. Tan, Ordinal measures for iris recognition, TPAMI, vol. 31, no. 12, pp , dec. 29. [14] P. A. Viola, M. J. Jones, and D. Snow, Detecting pedestrians using patterns of motion and appearance, IJCV, vol. 63, no. 2, pp , 25. [15] L. Wang, Z. Sun, and T. Tan, Robust regularized feature selection for iris recognition via linear programming, in ICPR, Nov. 212, pp
Graph Matching Iris Image Blocks with Local Binary Pattern
Graph Matching Iris Image Blocs with Local Binary Pattern Zhenan Sun, Tieniu Tan, and Xianchao Qiu Center for Biometrics and Security Research, National Laboratory of Pattern Recognition, Institute of
More informationAn efficient face recognition algorithm based on multi-kernel regularization learning
Acta Technica 61, No. 4A/2016, 75 84 c 2017 Institute of Thermomechanics CAS, v.v.i. An efficient face recognition algorithm based on multi-kernel regularization learning Bi Rongrong 1 Abstract. A novel
More informationEfficient Iris Spoof Detection via Boosted Local Binary Patterns
Efficient Iris Spoof Detection via Boosted Local Binary Patterns Zhaofeng He, Zhenan Sun, Tieniu Tan, and Zhuoshi Wei Center for Biometrics and Security Research National Laboratory of Pattern Recognition,
More informationA Feature-level Solution to Off-angle Iris Recognition
A Feature-level Solution to Off-angle Iris Recognition Xingguang Li,2, Libin Wang 2, Zhenan Sun 2 and Tieniu Tan 2.Department of Automation,USTC 2.Center for Research on Intelligent Perception and Computing
More informationBoosting Ordinal Features for Accurate and Fast Iris Recognition
Boosting Ordinal Features for Accurate and Fast Iris Recognition Zhaofeng He, Zhenan Sun, Tieniu Tan, Xianchao Qiu, Cheng Zhong and Wenbo Dong Center for Biometrics and Security Research National Laboratory
More informationDiscriminative classifiers for image recognition
Discriminative classifiers for image recognition May 26 th, 2015 Yong Jae Lee UC Davis Outline Last time: window-based generic object detection basic pipeline face detection with boosting as case study
More informationThe Comparative Study of Machine Learning Algorithms in Text Data Classification*
The Comparative Study of Machine Learning Algorithms in Text Data Classification* Wang Xin School of Science, Beijing Information Science and Technology University Beijing, China Abstract Classification
More informationHW2 due on Thursday. Face Recognition: Dimensionality Reduction. Biometrics CSE 190 Lecture 11. Perceptron Revisited: Linear Separators
HW due on Thursday Face Recognition: Dimensionality Reduction Biometrics CSE 190 Lecture 11 CSE190, Winter 010 CSE190, Winter 010 Perceptron Revisited: Linear Separators Binary classification can be viewed
More informationSubject-Oriented Image Classification based on Face Detection and Recognition
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationExperts-Shift: Learning Active Spatial Classification Experts for Keyframe-based Video Segmentation
Experts-Shift: Learning Active Spatial Classification Experts for Keyframe-based Video Segmentation Yibiao Zhao 1,3, Yanbiao Duan 2,3, Xiaohan Nie 2,3, Yaping Huang 1, Siwei Luo 1 1 Beijing Jiaotong University,
More informationFace detection and recognition. Detection Recognition Sally
Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationK-Means Based Matching Algorithm for Multi-Resolution Feature Descriptors
K-Means Based Matching Algorithm for Multi-Resolution Feature Descriptors Shao-Tzu Huang, Chen-Chien Hsu, Wei-Yen Wang International Science Index, Electrical and Computer Engineering waset.org/publication/0007607
More informationA Cascade of Feed-Forward Classifiers for Fast Pedestrian Detection
A Cascade of eed-orward Classifiers for ast Pedestrian Detection Yu-ing Chen,2 and Chu-Song Chen,3 Institute of Information Science, Academia Sinica, aipei, aiwan 2 Dept. of Computer Science and Information
More informationVoxel selection algorithms for fmri
Voxel selection algorithms for fmri Henryk Blasinski December 14, 2012 1 Introduction Functional Magnetic Resonance Imaging (fmri) is a technique to measure and image the Blood- Oxygen Level Dependent
More informationA General Greedy Approximation Algorithm with Applications
A General Greedy Approximation Algorithm with Applications Tong Zhang IBM T.J. Watson Research Center Yorktown Heights, NY 10598 tzhang@watson.ibm.com Abstract Greedy approximation algorithms have been
More informationFace Detection Using Look-Up Table Based Gentle AdaBoost
Face Detection Using Look-Up Table Based Gentle AdaBoost Cem Demirkır and Bülent Sankur Boğaziçi University, Electrical-Electronic Engineering Department, 885 Bebek, İstanbul {cemd,sankur}@boun.edu.tr
More informationSupplementary material: Strengthening the Effectiveness of Pedestrian Detection with Spatially Pooled Features
Supplementary material: Strengthening the Effectiveness of Pedestrian Detection with Spatially Pooled Features Sakrapee Paisitkriangkrai, Chunhua Shen, Anton van den Hengel The University of Adelaide,
More informationAll lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
More informationCombining SVMs with Various Feature Selection Strategies
Combining SVMs with Various Feature Selection Strategies Yi-Wei Chen and Chih-Jen Lin Department of Computer Science, National Taiwan University, Taipei 106, Taiwan Summary. This article investigates the
More informationGeneric Object Detection Using Improved Gentleboost Classifier
Available online at www.sciencedirect.com Physics Procedia 25 (2012 ) 1528 1535 2012 International Conference on Solid State Devices and Materials Science Generic Object Detection Using Improved Gentleboost
More informationComputer Vision Group Prof. Daniel Cremers. 8. Boosting and Bagging
Prof. Daniel Cremers 8. Boosting and Bagging Repetition: Regression We start with a set of basis functions (x) =( 0 (x), 1(x),..., M 1(x)) x 2 í d The goal is to fit a model into the data y(x, w) =w T
More informationGeneric Face Alignment Using an Improved Active Shape Model
Generic Face Alignment Using an Improved Active Shape Model Liting Wang, Xiaoqing Ding, Chi Fang Electronic Engineering Department, Tsinghua University, Beijing, China {wanglt, dxq, fangchi} @ocrserv.ee.tsinghua.edu.cn
More informationPV211: Introduction to Information Retrieval
PV211: Introduction to Information Retrieval http://www.fi.muni.cz/~sojka/pv211 IIR 15-1: Support Vector Machines Handout version Petr Sojka, Hinrich Schütze et al. Faculty of Informatics, Masaryk University,
More informationA New Strategy of Pedestrian Detection Based on Pseudo- Wavelet Transform and SVM
A New Strategy of Pedestrian Detection Based on Pseudo- Wavelet Transform and SVM M.Ranjbarikoohi, M.Menhaj and M.Sarikhani Abstract: Pedestrian detection has great importance in automotive vision systems
More informationLecture 7: Support Vector Machine
Lecture 7: Support Vector Machine Hien Van Nguyen University of Houston 9/28/2017 Separating hyperplane Red and green dots can be separated by a separating hyperplane Two classes are separable, i.e., each
More informationKernel Methods & Support Vector Machines
& Support Vector Machines & Support Vector Machines Arvind Visvanathan CSCE 970 Pattern Recognition 1 & Support Vector Machines Question? Draw a single line to separate two classes? 2 & Support Vector
More informationSkin and Face Detection
Skin and Face Detection Linda Shapiro EE/CSE 576 1 What s Coming 1. Review of Bakic flesh detector 2. Fleck and Forsyth flesh detector 3. Details of Rowley face detector 4. Review of the basic AdaBoost
More informationBagging and Boosting Algorithms for Support Vector Machine Classifiers
Bagging and Boosting Algorithms for Support Vector Machine Classifiers Noritaka SHIGEI and Hiromi MIYAJIMA Dept. of Electrical and Electronics Engineering, Kagoshima University 1-21-40, Korimoto, Kagoshima
More informationThe flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in R
Journal of Machine Learning Research 6 (205) 553-557 Submitted /2; Revised 3/4; Published 3/5 The flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in R Xingguo Li Department
More informationDescribable Visual Attributes for Face Verification and Image Search
Advanced Topics in Multimedia Analysis and Indexing, Spring 2011, NTU. 1 Describable Visual Attributes for Face Verification and Image Search Kumar, Berg, Belhumeur, Nayar. PAMI, 2011. Ryan Lei 2011/05/05
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines CS 536: Machine Learning Littman (Wu, TA) Administration Slides borrowed from Martin Law (from the web). 1 Outline History of support vector machines (SVM) Two classes,
More informationIMAGE RETRIEVAL USING VLAD WITH MULTIPLE FEATURES
IMAGE RETRIEVAL USING VLAD WITH MULTIPLE FEATURES Pin-Syuan Huang, Jing-Yi Tsai, Yu-Fang Wang, and Chun-Yi Tsai Department of Computer Science and Information Engineering, National Taitung University,
More informationFace Recognition using SURF Features and SVM Classifier
International Journal of Electronics Engineering Research. ISSN 0975-6450 Volume 8, Number 1 (016) pp. 1-8 Research India Publications http://www.ripublication.com Face Recognition using SURF Features
More informationApplication of Support Vector Machine Algorithm in Spam Filtering
Application of Support Vector Machine Algorithm in E-Mail Spam Filtering Julia Bluszcz, Daria Fitisova, Alexander Hamann, Alexey Trifonov, Advisor: Patrick Jähnichen Abstract The problem of spam classification
More informationSupport Vector Machines + Classification for IR
Support Vector Machines + Classification for IR Pierre Lison University of Oslo, Dep. of Informatics INF3800: Søketeknologi April 30, 2014 Outline of the lecture Recap of last week Support Vector Machines
More informationTraffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers
Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers A. Salhi, B. Minaoui, M. Fakir, H. Chakib, H. Grimech Faculty of science and Technology Sultan Moulay Slimane
More informationVideo Inter-frame Forgery Identification Based on Optical Flow Consistency
Sensors & Transducers 24 by IFSA Publishing, S. L. http://www.sensorsportal.com Video Inter-frame Forgery Identification Based on Optical Flow Consistency Qi Wang, Zhaohong Li, Zhenzhen Zhang, Qinglong
More informationActive learning for visual object recognition
Active learning for visual object recognition Written by Yotam Abramson and Yoav Freund Presented by Ben Laxton Outline Motivation and procedure How this works: adaboost and feature details Why this works:
More informationObject Detection Design challenges
Object Detection Design challenges How to efficiently search for likely objects Even simple models require searching hundreds of thousands of positions and scales Feature design and scoring How should
More informationOptimal Extension of Error Correcting Output Codes
Book Title Book Editors IOS Press, 2003 1 Optimal Extension of Error Correcting Output Codes Sergio Escalera a, Oriol Pujol b, and Petia Radeva a a Centre de Visió per Computador, Campus UAB, 08193 Bellaterra
More informationBinary Online Learned Descriptors
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TPAMI.27.267993,
More informationMobile Human Detection Systems based on Sliding Windows Approach-A Review
Mobile Human Detection Systems based on Sliding Windows Approach-A Review Seminar: Mobile Human detection systems Njieutcheu Tassi cedrique Rovile Department of Computer Engineering University of Heidelberg
More informationFeature-level Fusion for Effective Palmprint Authentication
Feature-level Fusion for Effective Palmprint Authentication Adams Wai-Kin Kong 1, 2 and David Zhang 1 1 Biometric Research Center, Department of Computing The Hong Kong Polytechnic University, Kowloon,
More information[2008] IEEE. Reprinted, with permission, from [Yan Chen, Qiang Wu, Xiangjian He, Wenjing Jia,Tom Hintz, A Modified Mahalanobis Distance for Human
[8] IEEE. Reprinted, with permission, from [Yan Chen, Qiang Wu, Xiangian He, Wening Jia,Tom Hintz, A Modified Mahalanobis Distance for Human Detection in Out-door Environments, U-Media 8: 8 The First IEEE
More informationHand Posture Recognition Using Adaboost with SIFT for Human Robot Interaction
Hand Posture Recognition Using Adaboost with SIFT for Human Robot Interaction Chieh-Chih Wang and Ko-Chih Wang Department of Computer Science and Information Engineering Graduate Institute of Networking
More informationEmbedded Palmprint Recognition System on Mobile Devices
Embedded Palmprint Recognition System on Mobile Devices Yufei Han, Tieniu Tan, Zhenan Sun, and Ying Hao Center for Biometrics and Security Research National Labrotory of Pattern Recognition,Institue of
More informationIris Recognition for Eyelash Detection Using Gabor Filter
Iris Recognition for Eyelash Detection Using Gabor Filter Rupesh Mude 1, Meenakshi R Patel 2 Computer Science and Engineering Rungta College of Engineering and Technology, Bhilai Abstract :- Iris recognition
More informationA novel template matching method for human detection
University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2009 A novel template matching method for human detection Duc Thanh Nguyen
More informationTraining-Free, Generic Object Detection Using Locally Adaptive Regression Kernels
Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIENCE, VOL.32, NO.9, SEPTEMBER 2010 Hae Jong Seo, Student Member,
More informationChoosing the kernel parameters for SVMs by the inter-cluster distance in the feature space Authors: Kuo-Ping Wu, Sheng-De Wang Published 2008
Choosing the kernel parameters for SVMs by the inter-cluster distance in the feature space Authors: Kuo-Ping Wu, Sheng-De Wang Published 2008 Presented by: Nandini Deka UH Mathematics Spring 2014 Workshop
More informationFace Recognition Using Vector Quantization Histogram and Support Vector Machine Classifier Rong-sheng LI, Fei-fei LEE *, Yan YAN and Qiu CHEN
2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: 978-1-60595-389-2 Face Recognition Using Vector Quantization Histogram and Support Vector Machine
More informationFace Detection and Alignment. Prof. Xin Yang HUST
Face Detection and Alignment Prof. Xin Yang HUST Many slides adapted from P. Viola Face detection Face detection Basic idea: slide a window across image and evaluate a face model at every location Challenges
More informationLarge synthetic data sets to compare different data mining methods
Large synthetic data sets to compare different data mining methods Victoria Ivanova, Yaroslav Nalivajko Superviser: David Pfander, IPVS ivanova.informatics@gmail.com yaroslav.nalivayko@gmail.com June 3,
More informationCombine the PA Algorithm with a Proximal Classifier
Combine the Passive and Aggressive Algorithm with a Proximal Classifier Yuh-Jye Lee Joint work with Y.-C. Tseng Dept. of Computer Science & Information Engineering TaiwanTech. Dept. of Statistics@NCKU
More informationReal-time Object Classification in Video Surveillance Based on Appearance Learning
Real-time Object Classification in Video Surveillance Based on Appearance Learning Lun Zhang, Stan Z. Li, Xiaotong Yuan and Shiming Xiang Center for Biometrics and Security Research & National Laboratory
More informationPreliminary Local Feature Selection by Support Vector Machine for Bag of Features
Preliminary Local Feature Selection by Support Vector Machine for Bag of Features Tetsu Matsukawa Koji Suzuki Takio Kurita :University of Tsukuba :National Institute of Advanced Industrial Science and
More informationChap.12 Kernel methods [Book, Chap.7]
Chap.12 Kernel methods [Book, Chap.7] Neural network methods became popular in the mid to late 1980s, but by the mid to late 1990s, kernel methods have also become popular in machine learning. The first
More informationFacial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
More informationSoftware Documentation of the Potential Support Vector Machine
Software Documentation of the Potential Support Vector Machine Tilman Knebel and Sepp Hochreiter Department of Electrical Engineering and Computer Science Technische Universität Berlin 10587 Berlin, Germany
More informationSelf Lane Assignment Using Smart Mobile Camera For Intelligent GPS Navigation and Traffic Interpretation
For Intelligent GPS Navigation and Traffic Interpretation Tianshi Gao Stanford University tianshig@stanford.edu 1. Introduction Imagine that you are driving on the highway at 70 mph and trying to figure
More informationVideo annotation based on adaptive annular spatial partition scheme
Video annotation based on adaptive annular spatial partition scheme Guiguang Ding a), Lu Zhang, and Xiaoxu Li Key Laboratory for Information System Security, Ministry of Education, Tsinghua National Laboratory
More informationComputer Vision Group Prof. Daniel Cremers. 6. Boosting
Prof. Daniel Cremers 6. Boosting Repetition: Regression We start with a set of basis functions (x) =( 0 (x), 1(x),..., M 1(x)) x 2 í d The goal is to fit a model into the data y(x, w) =w T (x) To do this,
More informationSupport Vector Machines.
Support Vector Machines srihari@buffalo.edu SVM Discussion Overview 1. Overview of SVMs 2. Margin Geometry 3. SVM Optimization 4. Overlapping Distributions 5. Relationship to Logistic Regression 6. Dealing
More informationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun Presented by Tushar Bansal Objective 1. Get bounding box for all objects
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 12 Combining
More informationPost-Classification Change Detection of High Resolution Satellite Images Using AdaBoost Classifier
, pp.34-38 http://dx.doi.org/10.14257/astl.2015.117.08 Post-Classification Change Detection of High Resolution Satellite Images Using AdaBoost Classifier Dong-Min Woo 1 and Viet Dung Do 1 1 Department
More informationA Novel Extreme Point Selection Algorithm in SIFT
A Novel Extreme Point Selection Algorithm in SIFT Ding Zuchun School of Electronic and Communication, South China University of Technolog Guangzhou, China zucding@gmail.com Abstract. This paper proposes
More informationKeywords:- Object tracking, multiple instance learning, supervised learning, online boosting, ODFS tracker, classifier. IJSER
International Journal of Scientific & Engineering Research, Volume 5, Issue 2, February-2014 37 Object Tracking via a Robust Feature Selection approach Prof. Mali M.D. manishamali2008@gmail.com Guide NBNSCOE
More informationOut-of-Plane Rotated Object Detection using Patch Feature based Classifier
Available online at www.sciencedirect.com Procedia Engineering 41 (2012 ) 170 174 International Symposium on Robotics and Intelligent Sensors 2012 (IRIS 2012) Out-of-Plane Rotated Object Detection using
More informationClassification of Digital Photos Taken by Photographers or Home Users
Classification of Digital Photos Taken by Photographers or Home Users Hanghang Tong 1, Mingjing Li 2, Hong-Jiang Zhang 2, Jingrui He 1, and Changshui Zhang 3 1 Automation Department, Tsinghua University,
More informationhttps://en.wikipedia.org/wiki/the_dress Recap: Viola-Jones sliding window detector Fast detection through two mechanisms Quickly eliminate unlikely windows Use features that are fast to compute Viola
More informationGeneric Object Class Detection using Feature Maps
Generic Object Class Detection using Feature Maps Oscar Danielsson and Stefan Carlsson CVAP/CSC, KTH, Teknikringen 4, S- 44 Stockholm, Sweden {osda2,stefanc}@csc.kth.se Abstract. In this paper we describe
More informationLearning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li
Learning to Match Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li 1. Introduction The main tasks in many applications can be formalized as matching between heterogeneous objects, including search, recommendation,
More informationWriter Authentication Based on the Analysis of Strokes
Writer Authentication Based on the Analysis of Strokes Kun Yu, Yunhong Wang, Tieniu Tan * NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, 00080 P.R.China ABSTRACT This paper presents
More informationData mining with Support Vector Machine
Data mining with Support Vector Machine Ms. Arti Patle IES, IPS Academy Indore (M.P.) artipatle@gmail.com Mr. Deepak Singh Chouhan IES, IPS Academy Indore (M.P.) deepak.schouhan@yahoo.com Abstract: Machine
More informationAn R Package flare for High Dimensional Linear Regression and Precision Matrix Estimation
An R Package flare for High Dimensional Linear Regression and Precision Matrix Estimation Xingguo Li Tuo Zhao Xiaoming Yuan Han Liu Abstract This paper describes an R package named flare, which implements
More informationNaïve Bayes for text classification
Road Map Basic concepts Decision tree induction Evaluation of classifiers Rule induction Classification using association rules Naïve Bayesian classification Naïve Bayes for text classification Support
More informationIMPROVED FACE RECOGNITION USING ICP TECHNIQUES INCAMERA SURVEILLANCE SYSTEMS. Kirthiga, M.E-Communication system, PREC, Thanjavur
IMPROVED FACE RECOGNITION USING ICP TECHNIQUES INCAMERA SURVEILLANCE SYSTEMS Kirthiga, M.E-Communication system, PREC, Thanjavur R.Kannan,Assistant professor,prec Abstract: Face Recognition is important
More informationChakra Chennubhotla and David Koes
MSCBIO/CMPBIO 2065: Support Vector Machines Chakra Chennubhotla and David Koes Nov 15, 2017 Sources mmds.org chapter 12 Bishop s book Ch. 7 Notes from Toronto, Mark Schmidt (UBC) 2 SVM SVMs and Logistic
More informationGeneric Object-Face detection
Generic Object-Face detection Jana Kosecka Many slides adapted from P. Viola, K. Grauman, S. Lazebnik and many others Today Window-based generic object detection basic pipeline boosting classifiers face
More information6 Model selection and kernels
6. Bias-Variance Dilemma Esercizio 6. While you fit a Linear Model to your data set. You are thinking about changing the Linear Model to a Quadratic one (i.e., a Linear Model with quadratic features φ(x)
More informationA Feature Selection Method to Handle Imbalanced Data in Text Classification
A Feature Selection Method to Handle Imbalanced Data in Text Classification Fengxiang Chang 1*, Jun Guo 1, Weiran Xu 1, Kejun Yao 2 1 School of Information and Communication Engineering Beijing University
More informationComparison of Optimization Methods for L1-regularized Logistic Regression
Comparison of Optimization Methods for L1-regularized Logistic Regression Aleksandar Jovanovich Department of Computer Science and Information Systems Youngstown State University Youngstown, OH 44555 aleksjovanovich@gmail.com
More informationTable of Contents. Recognition of Facial Gestures... 1 Attila Fazekas
Table of Contents Recognition of Facial Gestures...................................... 1 Attila Fazekas II Recognition of Facial Gestures Attila Fazekas University of Debrecen, Institute of Informatics
More informationEye Detection by Haar wavelets and cascaded Support Vector Machine
Eye Detection by Haar wavelets and cascaded Support Vector Machine Vishal Agrawal B.Tech 4th Year Guide: Simant Dubey / Amitabha Mukherjee Dept of Computer Science and Engineering IIT Kanpur - 208 016
More informationA novel supervised learning algorithm and its use for Spam Detection in Social Bookmarking Systems
A novel supervised learning algorithm and its use for Spam Detection in Social Bookmarking Systems Anestis Gkanogiannis and Theodore Kalamboukis Department of Informatics Athens University of Economics
More informationClient Dependent GMM-SVM Models for Speaker Verification
Client Dependent GMM-SVM Models for Speaker Verification Quan Le, Samy Bengio IDIAP, P.O. Box 592, CH-1920 Martigny, Switzerland {quan,bengio}@idiap.ch Abstract. Generative Gaussian Mixture Models (GMMs)
More informationVariable Selection 6.783, Biomedical Decision Support
6.783, Biomedical Decision Support (lrosasco@mit.edu) Department of Brain and Cognitive Science- MIT November 2, 2009 About this class Why selecting variables Approaches to variable selection Sparsity-based
More informationSupport Vector Machines
Support Vector Machines About the Name... A Support Vector A training sample used to define classification boundaries in SVMs located near class boundaries Support Vector Machines Binary classifiers whose
More informationHuman detection using local shape and nonredundant
University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2010 Human detection using local shape and nonredundant binary patterns
More informationSeminary Iris Segmentation. BCC448 Pattern Recognition
Seminary Iris Segmentation BCC448 Pattern Recognition Students: Filipe Eduardo Mata dos Santos Pedro Henrique Lopes Silva Paper Robust Iris Segmentation Based on Learned Boundary Detectors Authors: Haiqing
More informationClassifier Case Study: Viola-Jones Face Detector
Classifier Case Study: Viola-Jones Face Detector P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001. P. Viola and M. Jones. Robust real-time face detection.
More informationLogistic Regression: Probabilistic Interpretation
Logistic Regression: Probabilistic Interpretation Approximate 0/1 Loss Logistic Regression Adaboost (z) SVM Solution: Approximate 0/1 loss with convex loss ( surrogate loss) 0-1 z = y w x SVM (hinge),
More information4820 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 24, NO. 12, DECEMBER 2015
4820 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 24, NO. 12, DECEMBER 2015 Local Multi-Grouped Binary Descriptor With Ring-Based Pooling Configuration and Optimization Yongqiang Gao, Weilin Huang, Member,
More informationSelection of Scale-Invariant Parts for Object Class Recognition
Selection of Scale-Invariant Parts for Object Class Recognition Gy. Dorkó and C. Schmid INRIA Rhône-Alpes, GRAVIR-CNRS 655, av. de l Europe, 3833 Montbonnot, France fdorko,schmidg@inrialpes.fr Abstract
More informationBagging for One-Class Learning
Bagging for One-Class Learning David Kamm December 13, 2008 1 Introduction Consider the following outlier detection problem: suppose you are given an unlabeled data set and make the assumptions that one
More informationGene Expression Based Classification using Iterative Transductive Support Vector Machine
Gene Expression Based Classification using Iterative Transductive Support Vector Machine Hossein Tajari and Hamid Beigy Abstract Support Vector Machine (SVM) is a powerful and flexible learning machine.
More informationSupport Vector Machines
Support Vector Machines SVM Discussion Overview. Importance of SVMs. Overview of Mathematical Techniques Employed 3. Margin Geometry 4. SVM Training Methodology 5. Overlapping Distributions 6. Dealing
More informationAggregating Descriptors with Local Gaussian Metrics
Aggregating Descriptors with Local Gaussian Metrics Hideki Nakayama Grad. School of Information Science and Technology The University of Tokyo Tokyo, JAPAN nakayama@ci.i.u-tokyo.ac.jp Abstract Recently,
More information