Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA
|
|
- Cassandra Cross
- 5 years ago
- Views:
Transcription
1 Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 0012 Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA Zhu Li Dept of CSEE, UMKC Office: FH560E, Ph: x Z. Li, Image Analysis & Retrv p.1
2 Outline ReCap of Lecture 07 Image Retrieval System BoW VLAD Dense SIFT Fisher Vector Aggregation AKULA Summary Z. Li, Image Analysis & Retrv p.2
3 Precision, Recall, F-measure Precision, TPR = TP/(TP + FP), Recall = TP/(TP + FN), FPR=FP/(TP+FP) F-measure = 2*(precision*recall)/(precision + recall) Precision: is the probability that a retrieved document is relevant. Recall: is the probability that a relevant document is retrieved in a search. Z. Li, Image Analysis & Retrv p.3
4 Curse of Dimensionality Why Aggregation? + Decision Boundary / Indexing.. Z. Li, Image Analysis & Retrv p.4
5 Bag-of-Words: Histogram Coding Codebook: Feature space: R d, k-means to get k centroids, {μ 1, μ 2,, μ k } BoW Hard Encoding: For n feature points,{x 1, x 2,,x n } assignment matrix: kxn, with column only 1-non zero entry Aggregated dimension: k k n Z. Li, Image Analysis & Retrv p.5
6 Kernel Code Book Soft Encoding Kernel Code Book Soft Encoding Kernel Affinity: K x j, μ k = e k x j μ k 2 Assignment Matrix: A j,k = K(x j, μ k )/ k K(x j, μ k ) Encoding: k-dimensional: X(k)= 1 n j A j,k Z. Li, Image Analysis & Retrv p.6
7 VLAD- Vector of Locally Aggregated Descriptors Aggregate feature difference from the codebook Hard assignment by finding the NN of feature {x k } to {μ k } Compute aggregated differences 1 assign descriptors x 2 v k = j,s.t.nn x j =μ k x j μ k 2 compute x- i 5 L2 normalize v k = v k / v k 2 Final feature: k x d 3 v i =sum x- i for cell i v 1 v2 v 3 v 4 v 5 Z. Li, Image Analysis & Retrv p.7
8 VLAD on SIFT Example of aggregating SIFT with VLAD K=16 codebook entries Each cell is a SIFT visualized as centroids in blue, and VLAD difference in red Top row: left image, bottom row: right image, red: code book, blue: encoded VLAD Z. Li, Image Analysis & Retrv p.8
9 Outline ReCap of Lecture 07 Image Retrieval System BoW VLAD Dense SIFT Fisher Vector Aggregation AKULA Summary Z. Li, Image Analysis & Retrv p.9
10 One more trick Recall that SIFT is a powerful descriptor VL_FEAT: vl_dsift A dense description of image by computing SIFT descriptor (no spatial-scale space extrema detection) at predetermined grid Supplement HoG as an alternative texture descriptor Z. Li, Image Analysis & Retrv p.10
11 VL_FEAT: vl_dsift Compute dense SIFT as a texture descriptor for the image [f, dsift]=vl_dsift(single(rgb2gray(im)), step, 2); There s also a FAST option [f, dsift]=vl_dsift(single(rgb2gray(im)), fast, step, 2); Huge amount of SIFT data will be generated Z. Li, Image Analysis & Retrv p.11
12 Fisher Vector Fisher Vector and variations: Winning in image classification: Winning in the MPEG object re-identification: o SCFV(Scalable Coded Fisher Vec) in CDVS Z. Li, Image Analysis & Retrv p.12
13 Codebook: Gaussian Mixture Model (GMM) GMM is a generative model to express data Assuming data is generated from with parameters {w k, μ k, σ k } x k ~ K k=1 w k N(μ k,σ k ) N μ k,σ k = 1 2π d 2 Σ k 1/2 e 1 2 x μ k Σ k 1 (x μ k ) Z. Li, Image Analysis & Retrv p.13
14 A bit of Theory: Fisher Kernel Encode the derivation from the generative model Observed feature set, {x 1, x 2,,x n } in R d, e.g, d=128 for SIFT. How s these observations derivate from the given GMM model with a set of parameter, λ = w k, μ k, σ k? o i.e, how the parameter, e.g, mean will move to best fit the observation? μ 1 μ 4 μ 3 + X1 μ 2 Z. Li, Image Analysis & Retrv p.14
15 A bit of Theory: Fisher Kernel Score function w.r.t. the likelihood function μ λ (X) G λ X = λ log u λ (X): derivative on the log likelihood The dimension of score function is m, where m is the number of generative model parameters, m=3 for GMM Given the observed data X, score function indicate how likelihood function parameter (e.g, mean) should move to better fit the data. Distance/Derivation of two observation X, Y w.r.t the generative model Fisher Info Matrix (roughly the covariance in the Mahanolibis distance) F λ = E X G λ X G λ X Fisher Kernel Distance: normalized by the Fisher Info Matrix: K FK X, Y = G λ X F λ 1 G λ X Z. Li, Image Analysis & Retrv p.15
16 Fisher Vector K FK (X, Y) is a measure of similarity, w.r.t. the generative model Similar to the Mahanolibis distance case, we can decompose this kernel as, K FK X, Y = G λ X F λ 1 G λ X = G λ X L λ L λ G λ X That give us a kernel feature mapping of X to Fisher Vector For observed images features {x t }, can be computed as, Z. Li, Image Analysis & Retrv p.16
17 GMM Fisher Vector Encode the derivation from the generative model Observed feature set, {x 1, x 2,,x n } in R d, e.g, d=128 (!) for SIFT. How s these observations derivate from the given GMM model with a set of parameter, θ = a k, μ k, σ k? GMM Log Likelihood Gradient Let w k = ea k j ea j, Then we have weight mean variance Z. Li, Image Analysis & Retrv p.17
18 GMM Fisher Vector VL_FEAT implementation GMM codebook For a K-component GMM, we only allow 3K parameters, π k, μ k,σ k k = 1.. K}, i.e, iid Gaussian component Σ k = σ k σ k 0 0. σ k Posterior prob of feature point x i to GMM component k Z. Li, Image Analysis & Retrv p.18
19 GMM Fisher Vector VL_FEAT implementation FV encoding Gradient on the mean, for GMM component k, j=1..d In the end, we have 2K x D aggregation on the derivation w.r.t. the means and variances FV = [u 1, u 2,, u K, v 1, v 2,, v K ] Z. Li, Image Analysis & Retrv p.19
20 VL_FEAT GMM/FV API Compute GMM model with VL_FEAT Prepare data: numpoints = 1000 ; dimension = 2 ; data = rand(dimension,n) ; Call vl_gmm: numclusters = 30 ; [means, covariances, priors] = vl_gmm(data, numclusters) ; Visualize: figure ; hold on ; plot(data(1,:),data(2,:),'r.') ; for i=1:numclusters vl_plotframe([means(:,i)' sigmas(1,i) 0 sigmas(2,i)]); end Z. Li, Image Analysis & Retrv p.20
21 VL_FEAT API FV encoding encoding = vl_fisher(datatobeencoded, means, covariances, priors); Bonus points: Encode HoG features with Fisher Vector? randomly collect 2~3 images from each class Stack all HoG features together into an n x 36 data matrix Compute its GMM Use this GMM to encode all image HoG features (other than average) Z. Li, Image Analysis & Retrv p.21
22 Super Vector Aggregation Speaker ID Fisher Vector: Aggregates Features against a GMM Super Vector: Aggregates GMM against GMM Yes, We Can!? Ref: o William M. Campbell, Douglas E. Sturim, Douglas A. Reynolds: Support vector machines using GMM supervectors for speaker verification. IEEE Signal Process. Lett. 13(5): (2006) Z. Li, Image Analysis & Retrv p.22
23 Super Vector from MFCC Motivated from Speaker ID work Speech is a continuous evolution of the vocal tract Need to extract a sequence of spectra or sequence of spectral coefficients Use a sliding window - 25 ms window, 10 ms shift Log X(ω) DCT MFCC Z. Li, Image Analysis & Retrv p.23
24 GMM Model from MFCC GMM on MFCC feature Z. Li, Image Analysis & Retrv p.24 M j s j s j s j s p p 1 ) ( ) ( ) ( ) ( ), ( ) ( x x The acoustic vectors (MFCC) of speaker s is modeled by a prob. density function parameterized by M j s j s j s j s 1 ) ( ) ( ) ( ) ( },, { Gaussian mixture model (GMM) for speaker s: M j s j s j s j s 1 ) ( ) ( ) ( ) ( },, {
25 Universal Background Model UBM GMM Model: The acoustic vectors of a general population is modeled by another GMM called the universal background model (UBM): (ubm) (ubm) p( x ) p( x M j 1 j (ubm) j, (ubm) j ) Parameters of the UBM (ubm) (ubm) (ubm) (ubm) M { j, j, j } j 1 Z. Li, Image Analysis & Retrv p.25
26 MAP Adaption Given the UBM GMM, how is the new observation derivate? The adapted mean is given by: Z. Li, Image Analysis & Retrv p.26
27 Supervector Distance Assuming we have UBM GMM model λ UBM = {P k, μ k, Σ k }, with identical prior and covariance Then for two utterance samples a and b, with GMM models λ a = {P k, μ k a, Σ k }, λ b = {P k, μ k b, Σ k }, The SV distance is, K λ a,λ b = k P k Σ k ( 1 2 ) μ k a It means the means of two models need to be normalized by the UBM covariance induced Mahanolibis distance metric This is also a linear kernel function scaled by the UBM covariances T ( P k Σ k ( 1 2 ) μ k b ) Z. Li, Image Analysis & Retrv p.27
28 Supervector Performance in NIST Speaker ID System 5: Gaussian SV DCF (Detection Cost Function) Z. Li, Image Analysis & Retrv p.28
29 m31491 AKULA Adaptive KLUster Aggregation 2013/10/25 Abhishek Nagar, Zhu Li, Gaurav Srivastava and Kyungmo Park Z. Li, Image Analysis & Retrv p.29
30 Outline Motivation Adaptive Aggregation Results with TM7 Summary Z. Li, Image Analysis & Retrv p.30
31 Motivation Better Aggregation Fisher Vector and VLAD type aggregation depending on a global model AKULA removes this dependence, and directly coding the cluster centroids and sift count SCFV/RVD all having situations where clusters are turned off due to no assignment, this can be avoided in AKULA SIFT detection & selection K-means AKULA description Z. Li, Image Analysis & Retrv p.31
32 Motivation Better Subspace Choice Both SCFV and RVD do fixed normalization and PCA projection based on heuristic. What is the best possible subspace to do the aggregation? Using a boosting scheme to keep adding subspaces and aggregations in an iterative fashion, and tune TPR-FPR to the desired operating points on FPR. Z. Li, Image Analysis & Retrv p.32
33 CE2: AKULA Adaptive KLUster Aggregation AKULA Descriptor: cluster centroids + SIFT count A 1 ={yc 1 1, yc 1 2,, yc 1 k ; pc 1 1, pc 1 2,, pc 1 k }, A 2 ={yc 2 1, yc 2 2,, yc 2 k ; pc 2 1, pc 2 2,, pc 2 k } Distance metric: Min centroids distance, weighted by SIFT count d A 1, A 2 = 1 k 1 d k j=0 min j w min k 1 (j) + 1 k i=0 2 2 d min i w min (i) 1 d min d min j = min 2 i = min i j d j,i d j,i 1 w min j = w j,i, i = argmin i 2 i = w j,i, j = argmin w min j d j,i d j,i Z. Li, Image Analysis & Retrv p.33
34 AKULA implementation in TM7 Inner loop aggregation Dimension is fixed at 8 Numb of clusters, or nc=8, 16, 32, to hit 64, 128, and 256 bytes Quantization: scale by ½ and quantized to int8, sift count is 8 bits, total (nc+1)*dim bytes per aggregation Z. Li, Image Analysis & Retrv p.34
35 AKULA implementation in TM7 Outer loop subspace optimization by boosting Initial set of subspace models {A k } computed from MIR FLICKR data set SIFT extractions by k-means the space to 4096 clusters Iterative search on subspaces to generate AKULA aggregation that can improve performance in precisionrecall Notice that aggregation is de-coupled in subspace iteration, to allow more DoF in aggregation, to find subspaces that provides complimentary info. The algorithm is still being debugged, hence only having 1 st iteration results in TM7 Z. Li, Image Analysis & Retrv p.35
36 AKULA implementation in TM7 Outer loop subspace optimization by boosting Initial set of subspace models {A k } computed from MIR FLICKR data set SIFT extractions by k-means the space to 4096 clusters Iterative search on subspaces to generate AKULA aggregation that can improve performance in precision-recall Notice that aggregation is de-coupled in subspace iteration, to allow more DoF in aggregation, to find subspaces that provides complimentary info. The algorithm is still being debugged, hence only having 1 st iteration results in TM7 Indexing/Hashing is required for AKULA, it involves nc x dim multiplications and additions at this time. A binarization scheme will be considered once its performance is optimized in non-binary form. Z. Li, Image Analysis & Retrv p.36
37 GD Only TPR-FPR: AKULA vs SCFV Data set 1: AKULA (128bytes, dim=8, nc=16) distance is just 1-way dmin 1.*wt Forcing a weighted sum on SCFV (512 bytes) hamming distances without 2D decision fitting, i.e, count hamming distance between common active clusters, and sum up their distances Z. Li, Image Analysis & Retrv p.37
38 GD Only TPR-FPR: AKULA vs SCFV Data set 2, 3: AKULA distance is just 1-way dmin 1.*wt AKULA=128bytes, SCFV = 512 bytes. Z. Li, Image Analysis & Retrv p.38
39 3D object set: 4, 5 Data set4, 5: Z. Li, Image Analysis & Retrv p.39
40 FPR performance: AKULA in PM AKULA rates: pm rates m akula rates K K K_4K K_4K K K K Z. Li, Image Analysis & Retrv p.40
41 TPR (%) FPR TPR (%) 120 bitrate: bitrate: 1k TM7 AKULA TM7 AKULA a 1b 1c a 1b 1c Z. Li, Image Analysis & Retrv p.41
42 TPR (%) TPR (%) bitrate: 2k bitrate: 1k-4k TM7 60 TM7 40 AKULA 40 AKULA a 1b 1c a 1b 1c Z. Li, Image Analysis & Retrv p.42
43 TPR (%) TPR (%) bitrate: 2k-4k bitrate: 4k TM7 60 TM7 40 AKULA 40 AKULA a 1b 1c a 1b 1c Z. Li, Image Analysis & Retrv p.43
44 TPR (%) TPR (%) bitrate: 8k bitrate: 16k TM7 AKULA TM7 AKULA a 1b 1c a 1b 1c Z. Li, Image Analysis & Retrv p.44
45 AKULA Localization Quite some improvements: 2.7% Z. Li, Image Analysis & Retrv p.45
46 AKULA Summary Benefits: Allow more DoF in aggregation optimization, o by an outer loop boosting scheme for subspace projection optimization o And an inner loop adaptive clustering without the constraint of the global GMM model Simple weighted distance sum metric, with no need to tune a multi-dimensional decision boundary The overall pair wise matching matched up with TM7 SCFV with 2-dimensional decision boundary In GD only matching outperforms the TM7 GD Good improvements to the localization accuracy Light in extraction, but still heavy in pair wise matching, and need binarization scheme and/or indexing scheme to work for retrieval Future Improvements: Supervector AKULA? Z. Li, Image Analysis & Retrv p.46
47 Lec 08 Summary Fisher Vector Aggregate features {Xk} in R D against GMM Super Vector Aggregate GMM against a global GMM (UBM) AKULA Direct Aggregation Z. Li, Image Analysis & Retrv p.47
Image Analysis & Retrieval Lec 12 - Mid-Term Review
CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 12 - Mid-Term Review Zhu Li Dept of CSEE, UMKC http://l.web.umkc.edu/lizhu Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact:
More informationImage Analysis & Retrieval. CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W Lec 13
Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 13 Dimension Reduction: SVD and PCA Zhu Li Dept of CSEE, UMKC Office: FH560E, Email:
More informationImage Analysis & Retrieval. CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W Lec 18.
Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 18 Image Hashing Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph:
More informationMixtures of Gaussians and Advanced Feature Encoding
Mixtures of Gaussians and Advanced Feature Encoding Computer Vision Ali Borji UWM Many slides from James Hayes, Derek Hoiem, Florent Perronnin, and Hervé Why do good recognition systems go bad? E.g. Why
More informationAKULA Adaptive Cluster Aggregation for Visual Search
AKULA Adaptive Cluster Aggregation for Visual Search Abhishek Nagar *, Zhu Li *, Gaurav Srivastava *, and Kyungmo Park + * Samsung Research America 1301 E. Lookout Dr. Richardson, TX 75082, USA {a.nagar,
More informationFisher vector image representation
Fisher vector image representation Jakob Verbeek January 13, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.11.12.php Fisher vector representation Alternative to bag-of-words image representation
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationImage Analysis & Retrieval. CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W Lec 16
Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 16 Subspace/Transform Optimization Zhu Li Dept of CSEE, UMKC Office: FH560E, Email:
More informationComputer Vision. Exercise Session 10 Image Categorization
Computer Vision Exercise Session 10 Image Categorization Object Categorization Task Description Given a small number of training images of a category, recognize a-priori unknown instances of that category
More informationFrom Pixels to Information Recent Advances in Visual Search
From Pixels to Information Recent Advances in Visual Search Bernd Girod Stanford University bgirod@stanford.edu Augmented Reality 3 Augmented Reality 2014 2012 2015 4 Future: Smart Contact Lenses Sight:
More informationECE 5424: Introduction to Machine Learning
ECE 5424: Introduction to Machine Learning Topics: Unsupervised Learning: Kmeans, GMM, EM Readings: Barber 20.1-20.3 Stefan Lee Virginia Tech Tasks Supervised Learning x Classification y Discrete x Regression
More informationImage Analysis & Retrieval Lec 10 - Classification II
CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 10 - Classification II Zhu Li Dept of CSEE, UMKC http://l.web.umkc.edu/lizhu Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact:
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationMetric Learning for Large Scale Image Classification:
Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Thomas Mensink 1,2 Jakob Verbeek 2 Florent Perronnin 1 Gabriela Csurka 1 1 TVPA - Xerox Research Centre
More informationLarge scale object/scene recognition
Large scale object/scene recognition Image dataset: > 1 million images query Image search system ranked image list Each image described by approximately 2000 descriptors 2 10 9 descriptors to index! Database
More informationPattern Spotting in Historical Document Image
Pattern Spotting in historical document images Sovann EN, Caroline Petitjean, Stéphane Nicolas, Frédéric Jurie, Laurent Heutte LITIS, University of Rouen, France 1 Outline Introduction Commons Pipeline
More informationImageCLEF 2011
SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test
More informationMachine Learning Lecture 3
Machine Learning Lecture 3 Probability Density Estimation II 19.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Exam dates We re in the process
More informationExpectation Maximization: Inferring model parameters and class labels
Expectation Maximization: Inferring model parameters and class labels Emily Fox University of Washington February 27, 2017 Mixture of Gaussian recap 1 2/26/17 Jumble of unlabeled images HISTOGRAM blue
More informationWriter Identification and Verification Using GMM Supervectors
Writer Identification and Verification Using GMM Supervectors Vincent Christlein David Bernecker Florian Hönig Elli Angelopoulou Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität
More informationon learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim (100 100,000) Generic, unsupervised: BoW,
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationCS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
More informationMACHINE LEARNING: CLUSTERING, AND CLASSIFICATION. Steve Tjoa June 25, 2014
MACHINE LEARNING: CLUSTERING, AND CLASSIFICATION Steve Tjoa kiemyang@gmail.com June 25, 2014 Review from Day 2 Supervised vs. Unsupervised Unsupervised - clustering Supervised binary classifiers (2 classes)
More informationMethods for Intelligent Systems
Methods for Intelligent Systems Lecture Notes on Clustering (II) Davide Eynard eynard@elet.polimi.it Department of Electronics and Information Politecnico di Milano Davide Eynard - Lecture Notes on Clustering
More informationComparative Evaluation of Feature Normalization Techniques for Speaker Verification
Comparative Evaluation of Feature Normalization Techniques for Speaker Verification Md Jahangir Alam 1,2, Pierre Ouellet 1, Patrick Kenny 1, Douglas O Shaughnessy 2, 1 CRIM, Montreal, Canada {Janagir.Alam,
More informationBo#leneck Features from SNR- Adap9ve Denoising Deep Classifier for Speaker Iden9fica9on
Bo#leneck Features from SNR- Adap9ve Denoising Deep Classifier for Speaker Iden9fica9on TAN Zhili & MAK Man-Wai APSIPA 2015 Department of Electronic and Informa2on Engineering The Hong Kong Polytechnic
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationExpectation Maximization: Inferring model parameters and class labels
Expectation Maximization: Inferring model parameters and class labels Emily Fox University of Washington February 27, 2017 Mixture of Gaussian recap 1 2/27/2017 Jumble of unlabeled images HISTOGRAM blue
More informationRecognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213)
Recognition of Animal Skin Texture Attributes in the Wild Amey Dharwadker (aap2174) Kai Zhang (kz2213) Motivation Patterns and textures are have an important role in object description and understanding
More informationThree things everyone should know to improve object retrieval. Relja Arandjelović and Andrew Zisserman (CVPR 2012)
Three things everyone should know to improve object retrieval Relja Arandjelović and Andrew Zisserman (CVPR 2012) University of Oxford 2 nd April 2012 Large scale object retrieval Find all instances of
More informationILSVRC on a Smartphone
[DOI: 10.2197/ipsjtcva.6.83] Express Paper ILSVRC on a Smartphone Yoshiyuki Kawano 1,a) Keiji Yanai 1,b) Received: March 14, 2014, Accepted: April 24, 2014, Released: July 25, 2014 Abstract: In this work,
More informationProbabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information
Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information Mustafa Berkay Yilmaz, Hakan Erdogan, Mustafa Unel Sabanci University, Faculty of Engineering and Natural
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationSHOT-BASED OBJECT RETRIEVAL FROM VIDEO WITH COMPRESSED FISHER VECTORS. Luca Bertinetto, Attilio Fiandrotti, Enrico Magli
SHOT-BASED OBJECT RETRIEVAL FROM VIDEO WITH COMPRESSED FISHER VECTORS Luca Bertinetto, Attilio Fiandrotti, Enrico Magli Dipartimento di Elettronica e Telecomunicazioni, Politecnico di Torino (Italy) ABSTRACT
More informationClustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
More informationClustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
More informationImage Processing. Image Features
Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationTA Section: Problem Set 4
TA Section: Problem Set 4 Outline Discriminative vs. Generative Classifiers Image representation and recognition models Bag of Words Model Part-based Model Constellation Model Pictorial Structures Model
More informationMachine Learning Lecture 3
Many slides adapted from B. Schiele Machine Learning Lecture 3 Probability Density Estimation II 26.04.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course
More informationECE 484 Digital Image Processing Lec 17 - Part II Review & Final Projects Topics
ECE 484 Digital Image Processing Lec 17 - Part II Review & Final Projects opics Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu slides created with
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Image Segmentation Some material for these slides comes from https://www.csd.uwo.ca/courses/cs4487a/
More informationComputer vision: models, learning and inference. Chapter 13 Image preprocessing and feature extraction
Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction Preprocessing The goal of pre-processing is to try to reduce unwanted variation in image due to lighting,
More informationMachine Learning Lecture 3
Course Outline Machine Learning Lecture 3 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Probability Density Estimation II 26.04.206 Discriminative Approaches (5 weeks) Linear
More informationAggregating Descriptors with Local Gaussian Metrics
Aggregating Descriptors with Local Gaussian Metrics Hideki Nakayama Grad. School of Information Science and Technology The University of Tokyo Tokyo, JAPAN nakayama@ci.i.u-tokyo.ac.jp Abstract Recently,
More information10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
More informationFeature Descriptors. CS 510 Lecture #21 April 29 th, 2013
Feature Descriptors CS 510 Lecture #21 April 29 th, 2013 Programming Assignment #4 Due two weeks from today Any questions? How is it going? Where are we? We have two umbrella schemes for object recognition
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationExpectation-Maximization. Nuno Vasconcelos ECE Department, UCSD
Expectation-Maximization Nuno Vasconcelos ECE Department, UCSD Plan for today last time we started talking about mixture models we introduced the main ideas behind EM to motivate EM, we looked at classification-maximization
More informationAggregating local image descriptors into compact codes
Aggregating local image descriptors into compact codes Hervé Jégou, Florent Perronnin, Matthijs Douze, Jorge Sánchez, Patrick Pérez, Cordelia Schmid To cite this version: Hervé Jégou, Florent Perronnin,
More informationSparse coding for image classification
Sparse coding for image classification Columbia University Electrical Engineering: Kun Rong(kr2496@columbia.edu) Yongzhou Xiang(yx2211@columbia.edu) Yin Cui(yc2776@columbia.edu) Outline Background Introduction
More informationThe Approach of Mean Shift based Cosine Dissimilarity for Multi-Recording Speaker Clustering
The Approach of Mean Shift based Cosine Dissimilarity for Multi-Recording Speaker Clustering 1 D. Jareena Begum, 2 K Rajendra Prasad, 3 M Suleman Basha 1 M.Tech in SE, RGMCET, Nandyal 2 Assoc Prof, Dept
More informationLarge-scale visual recognition Efficient matching
Large-scale visual recognition Efficient matching Florent Perronnin, XRCE Hervé Jégou, INRIA CVPR tutorial June 16, 2012 Outline!! Preliminary!! Locality Sensitive Hashing: the two modes!! Hashing!! Embedding!!
More informationMetric Learning for Large-Scale Image Classification:
Metric Learning for Large-Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Florent Perronnin 1 work published at ECCV 2012 with: Thomas Mensink 1,2 Jakob Verbeek 2 Gabriela Csurka
More informationMixture Models and the EM Algorithm
Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Finite Mixture Models Say we have a data set D = {x 1,..., x N } where x i is
More informationLecture 24: Image Retrieval: Part II. Visual Computing Systems CMU , Fall 2013
Lecture 24: Image Retrieval: Part II Visual Computing Systems Review: K-D tree Spatial partitioning hierarchy K = dimensionality of space (below: K = 2) 3 2 1 3 3 4 2 Counts of points in leaf nodes Nearest
More informationGenerative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
More informationEfficient Object Localization with Gaussianized Vector Representation
Efficient Object Localization with Gaussianized Vector Representation ABSTRACT Xiaodan Zhuang xzhuang2@uiuc.edu Mark A. Hasegawa-Johnson jhasegaw@uiuc.edu Recently, the Gaussianized vector representation
More informationSegmentation: Clustering, Graph Cut and EM
Segmentation: Clustering, Graph Cut and EM Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL 60208 yingwu@northwestern.edu http://www.eecs.northwestern.edu/~yingwu
More informationRecap: Gaussian (or Normal) Distribution. Recap: Minimizing the Expected Loss. Topics of This Lecture. Recap: Maximum Likelihood Approach
Truth Course Outline Machine Learning Lecture 3 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Probability Density Estimation II 2.04.205 Discriminative Approaches (5 weeks)
More information( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components
Review Lecture 14 ! PRINCIPAL COMPONENT ANALYSIS Eigenvectors of the covariance matrix are the principal components 1. =cov X Top K principal components are the eigenvectors with K largest eigenvalues
More informationAn Introduction to PDF Estimation and Clustering
Sigmedia, Electronic Engineering Dept., Trinity College, Dublin. 1 An Introduction to PDF Estimation and Clustering David Corrigan corrigad@tcd.ie Electrical and Electronic Engineering Dept., University
More informationRegion-based Segmentation
Region-based Segmentation Image Segmentation Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation. Applications: Finding tumors, veins, etc.
More informationScene Recognition using Bag-of-Words
Scene Recognition using Bag-of-Words Sarthak Ahuja B.Tech Computer Science Indraprastha Institute of Information Technology Okhla, Delhi 110020 Email: sarthak12088@iiitd.ac.in Anchita Goel B.Tech Computer
More informationCS Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts
More informationData Analysis 3. Support Vector Machines. Jan Platoš October 30, 2017
Data Analysis 3 Support Vector Machines Jan Platoš October 30, 2017 Department of Computer Science Faculty of Electrical Engineering and Computer Science VŠB - Technical University of Ostrava Table of
More informationBossaNova at ImageCLEF 2012 Flickr Photo Annotation Task
BossaNova at ImageCLEF 2012 Flickr Photo Annotation Task S. Avila 1,2, N. Thome 1, M. Cord 1, E. Valle 3, and A. de A. Araújo 2 1 Pierre and Marie Curie University, UPMC-Sorbonne Universities, LIP6, France
More informationOverview of the MPEG-CDVS standard
JAN 2015 1 Overview of the MPEG-CDVS standard Ling-Yu Duan, Member, IEEE, Vijay Chandrasekhar, Member, IEEE, Jie Chen, Jie Lin, Member, IEEE, Zhe Wang, Tiejun Huang, Senior Member, IEEE, Bernd Girod, Fellow,
More informationImage Analysis & Retrieval
Outline CS/EE 5590 / ENG 401 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Image Analysis & Retrieval Background Objective of the class Prerequisite Lecture Plan Course Project
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationSpeech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute Slide Credit: Mehryar Mohri
Speech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute eugenew@cs.nyu.edu Slide Credit: Mehryar Mohri Speech Recognition Components Acoustic and pronunciation model:
More informationVideo event detection using subclass discriminant analysis and linear support vector machines
Video event detection using subclass discriminant analysis and linear support vector machines Nikolaos Gkalelis, Damianos Galanopoulos, Vasileios Mezaris / TRECVID 2014 Workshop, Orlando, FL, USA, November
More informationNormalized Texture Motifs and Their Application to Statistical Object Modeling
Normalized Texture Motifs and Their Application to Statistical Obect Modeling S. D. Newsam B. S. Manunath Center for Applied Scientific Computing Electrical and Computer Engineering Lawrence Livermore
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
More informationInference and Representation
Inference and Representation Rachel Hodos New York University Lecture 5, October 6, 2015 Rachel Hodos Lecture 5: Inference and Representation Today: Learning with hidden variables Outline: Unsupervised
More informationMultiple Kernel Learning for Emotion Recognition in the Wild
Multiple Kernel Learning for Emotion Recognition in the Wild Karan Sikka, Karmen Dykstra, Suchitra Sathyanarayana, Gwen Littlewort and Marian S. Bartlett Machine Perception Laboratory UCSD EmotiW Challenge,
More informationClassifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped
More information[Programming Assignment] (1)
http://crcv.ucf.edu/people/faculty/bagci/ [Programming Assignment] (1) Computer Vision Dr. Ulas Bagci (Fall) 2015 University of Central Florida (UCF) Coding Standard and General Requirements Code for all
More informationGaussian Mixture Models For Clustering Data. Soft Clustering and the EM Algorithm
Gaussian Mixture Models For Clustering Data Soft Clustering and the EM Algorithm K-Means Clustering Input: Observations: xx ii R dd ii {1,., NN} Number of Clusters: kk Output: Cluster Assignments. Cluster
More informationClustering web search results
Clustering K-means Machine Learning CSE546 Emily Fox University of Washington November 4, 2013 1 Clustering images Set of Images [Goldberger et al.] 2 1 Clustering web search results 3 Some Data 4 2 K-means
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
More informationClient Dependent GMM-SVM Models for Speaker Verification
Client Dependent GMM-SVM Models for Speaker Verification Quan Le, Samy Bengio IDIAP, P.O. Box 592, CH-1920 Martigny, Switzerland {quan,bengio}@idiap.ch Abstract. Generative Gaussian Mixture Models (GMMs)
More informationAdaptive Binary Quantization for Fast Nearest Neighbor Search
IBM Research Adaptive Binary Quantization for Fast Nearest Neighbor Search Zhujin Li 1, Xianglong Liu 1*, Junjie Wu 1, and Hao Su 2 1 Beihang University, Beijing, China 2 Stanford University, Stanford,
More informationMixture Models and EM
Table of Content Chapter 9 Mixture Models and EM -means Clustering Gaussian Mixture Models (GMM) Expectation Maximiation (EM) for Mixture Parameter Estimation Introduction Mixture models allows Complex
More informationLearning Compact Visual Attributes for Large-scale Image Classification
Learning Compact Visual Attributes for Large-scale Image Classification Yu Su and Frédéric Jurie GREYC CNRS UMR 6072, University of Caen Basse-Normandie, Caen, France {yu.su,frederic.jurie}@unicaen.fr
More informationSupport Vector Machines
Support Vector Machines About the Name... A Support Vector A training sample used to define classification boundaries in SVMs located near class boundaries Support Vector Machines Binary classifiers whose
More informationFeature Selection for Image Retrieval and Object Recognition
Feature Selection for Image Retrieval and Object Recognition Nuno Vasconcelos et al. Statistical Visual Computing Lab ECE, UCSD Presented by Dashan Gao Scalable Discriminant Feature Selection for Image
More informationImage Analysis & Retrieval Lec-01: Introduction
CS/EE 5590 / ENG 401 Special Topics Spring 2017, M/W 5:30-6:45pm@Haag 309 Image Analysis & Retrieval Lec-01: Introduction Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More informationCompressed local descriptors for fast image and video search in large databases
Compressed local descriptors for fast image and video search in large databases Matthijs Douze2 joint work with Hervé Jégou1, Cordelia Schmid2 and Patrick Pérez3 1: INRIA Rennes, TEXMEX team, France 2:
More informationContent-based image and video analysis. Machine learning
Content-based image and video analysis Machine learning for multimedia retrieval 04.05.2009 What is machine learning? Some problems are very hard to solve by writing a computer program by hand Almost all
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationSpeed-up Multi-modal Near Duplicate Image Detection
Open Journal of Applied Sciences, 2013, 3, 16-21 Published Online March 2013 (http://www.scirp.org/journal/ojapps) Speed-up Multi-modal Near Duplicate Image Detection Chunlei Yang 1,2, Jinye Peng 2, Jianping
More informationExpectation Maximization (EM) and Gaussian Mixture Models
Expectation Maximization (EM) and Gaussian Mixture Models Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 2 3 4 5 6 7 8 Unsupervised Learning Motivation
More informationObject Classification Problem
HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category
More informationComputer Vision II Lecture 14
Computer Vision II Lecture 14 Articulated Tracking I 08.07.2014 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Outline of This Lecture Single-Object Tracking Bayesian
More informationHomework #4 Programming Assignment Due: 11:59 pm, November 4, 2018
CSCI 567, Fall 18 Haipeng Luo Homework #4 Programming Assignment Due: 11:59 pm, ovember 4, 2018 General instructions Your repository will have now a directory P4/. Please do not change the name of this
More informationSUT Submission for NIST 2016 Speaker Recognition Evaluation: Description and Analysis
The 2017 Conference on Computational Linguistics and Speech Processing ROCLING 2017, pp. 276-286 The Association for Computational Linguistics and Chinese Language Processing SUT Submission for NIST 2016
More informationComputer Vision for HCI. Topics of This Lecture
Computer Vision for HCI Interest Points Topics of This Lecture Local Invariant Features Motivation Requirements, Invariances Keypoint Localization Features from Accelerated Segment Test (FAST) Harris Shi-Tomasi
More information