Neetha Das Prof. Andy Khong

Size: px

Start display at page:

Download "Neetha Das Prof. Andy Khong"

Aubrey Wilkins
5 years ago
Views:

1 Neetha Das Prof. Andy Khong

2 Contents Introduction and aim Current system at IMI Proposed new classification model Support Vector Machines Initial audio data collection and processing Features and their extraction Cross validation Future work Conclusion

3 Introduction and Aim Auditory awareness to the humanoid robot Areas source localization and sound classification Classification: Sounds Sounds Speech Impulsive sound Non-impulsive sound Speech Cough Keys jiggling Pen tapping Nails tapping Laughter Phone ringing

4 Current system in IMI Amplifier + A/D Converter Localization + Classification Algorithm Direction of sound Class of the sound segment

5 Current system in IMI 1. Source Localization using SRP-PHAT algorithm Time Difference of Arrival(TDOA) is compensated in each of the power spectrums of the microphones Tracking over a range of 0 to 180 degrees, the angle with the highest spectral power is chosen as the estimated source direction(dibiase et al, 2001, Dmochowski et al., 2008) PHAT based algorithm is seen to be more robust in highly reverberant environment(zhang et al, 2008)

6 Current system in IMI 2. Sound classification (Rashobh, 2012) Audio frame LPC error Sparsity Energy Audio Classification

7 New classification model - Machine learning

8 SVM Originally invented by Vladimir N. Vapnik, current version proposed by Vapnik and Corinna Cortis, published in 1995 Supervised learning models used for classification and regression analysis A basic SVM is a non-probabilistic binary linear classifier Given a set of training samples, each marked to belong to one of 2 categories, SVM builds a model that assigns new examples into one category or the other

9 SVM The training examples are mapped as points in the feature space SVM model divides the two categories with as wide a gap as possible Support vectors are the examples that lie closest to the decision surface

10 Finding the optimal hyper-plane Only support vectors influence optimality They are the most difficult to classify

11 The optimization problem Attribute vector for i th example in the training set: Training set of l samples x i = <x 1i,x 2i, x ni > Hyper plane H is given by, {(x 1,y 1 ), (x l,y l )}, where label y i Є {-1,1} H = w 0 x 0 + w 1 x 1 + w n x n + b = (w x) + b = 0 Optimal hyperplane drives the largest wedge between the two categories of data. Margin hyperplanes have equation H = ± 1

12 The optimization problem Distance of the i th example to the hyperplane H is given by Thus, minimizing magnitude of the weight vector results in maximizing distance of the example from the hyperplane The problem can be formulated as that of optimizing n variables with l side constraints: The problem can be solved using Lagrange multipliers

13 The optimization problem The Lagrange multiplier may also be subject to a constraint: 0< α i C for all i Penalty parameter C = is for an optimal hyper-plane that completely separates the data ( assuming one exists) For finite C, this changes to a problem of finding a soft-margin classifier Higher C signifies higher importance to classify all training data correctly Lower C is for a more flexible hyper plane that tries to minimize margin error

14 Kernels If data is not linearly separable, Kernels help pre-process it such that the problem is transformed into one of finding a simple hyperplane

15 Kernels A mapping z = Ф(x) transforms the dimension of the input vector and maps it in to a higher dimensional feature space By casting into a high enough dimensional space, we can separate any data set

Some common types of kernels The parameters relevant to the kernel being used, need to be specified during the training of the SVM model Multiclass SVM often uses Max Wins strategy One-against-one

16 Some common types of kernels The parameters relevant to the kernel being used, need to be specified during the training of the SVM model Multiclass SVM often uses Max Wins strategy One-against-one method K classes, K(K-1) classifiers data point classified to the class with the highest number of votes (Chang et al, 2010) SVM training involves application of kernel and then solving the optimization problem

17 Initial audio data collection and processing Read audio file Initial Data Collection Location : Anechoic chamber Audio type Sound No. of samples No. of samples Male scream(clean) Female scream(clean) Glass breaking(clean) 29 Glass breaking 29 Chop in to 20ms frames, 50% overlap Thresholding to eliminate silent frames Hamming window Fast Fourier Transform Feature Extraction SVM Training

18 Acoustic features and their extraction

19 Acoustic features and their extraction

20 Acoustic features and their extraction

21 Acoustic features and their extraction

22 Acoustic features and their extraction Mel-frequency cepstral coefficients (MFCCs) - popular in automatic speech recognition(asr) - modelled after human perception of sound - 12 coefficients; Delta and Delta-Delta(12 each) Audio frame FFT for power spectrum Apply mel filter bank and sum energy in each filter Log DCT (2-13 coefficients) MFCCs

23 Acoustic features and their extraction

24 Scaling Performed to reduce the influence of larger features compared to smaller ones Also simplifies computation Range [0,1]

25 Cross-validation K-fold cross validation divides the training set in to K equal groups. (K-1) groups are used to train and the model is tested for accuracy on the left out group Accuracy is calculated for all combinations of (K-1) groups Example of a 5-fold validation Blue represents training set and green represents testing set Mean of all the accuracies is found to analyze the model performance

26 Hyper-parameter selection Performance of SVMs depend on the selection of the kernel, kernel parameters, and soft margin parameter C (penalty parameter) RBF kernel is a common choice in audio classification Grid search is performed over exponentially growing sequences of C and γ The combination with the best cross-validation accuracy is chosen for the model that is used for classifying new data

27 Example of grid search result Sound No. of samples Male scream(clean) 15 Female scream(clean) 15 Glass breaking(clean) 15 Laughter(from internet) 8

28 Data collection at IMI Audio samples from 44 people, at 48kHz Down sampled to 16kHz Frame size of 32ms with 50% overlap 512 point audio frame and 512 point FFT applied 65 features used for training MFCCs, LPCs, frame energy Two ways label feature vector of frame or find average and standard deviation of each audio sample RBF kernel applied

29 Grid search result for data collected at IMI Sound (collected at IMI) No. of audio clips Cough 19 Keys jiggling 19 Laugh 19 Nails tapping on the table 19 Pen tapping on the table 19 Phone ringing 19 Speech 19 Silence clips 13

30 Future Work Use entire corpus to model the SVM Good accuracy Low accuracy Implement classification system in realtime Low accuracy Improve on feature set, kernel selection, thresholding, frame size, and pretraining processes Best classification model chosen to run real-time Artificial Neural Networks Comparison with SVM performance

31 Thank you!

Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning

Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning Robot Learning 1 General Pipeline 1. Data acquisition (e.g., from 3D sensors) 2. Feature extraction and representation construction 3. Robot learning: e.g., classification (recognition) or clustering (knowledge