LECTURE : MANIFOLD LEARNING

Size: px
Start display at page:

Download "LECTURE : MANIFOLD LEARNING"

Transcription

1 LECTURE : MANIFOLD LEARNING Rta Osadchy Some sldes are due to L.Saul, V. C. Raykar, N. Verma

2 Topcs PCA MDS IsoMap LLE EgenMaps Done!

3 Dmensonalty Reducton Data representaton Inputs are real-valued vectors n a hgh dmensonal space. Lnear structure Does the data lve n a low dmensonal subspace? Nonlnear structure Does the data lve on a low dmensonal submanfold?

4 Notatons Inputs (hgh dmensonal) x 1,x 2,,x n ponts n R D Outputs (low dmensonal) y 1,y 2,,y n ponts n R d (d<<d) Goals Nearby ponts reman nearby. Dstant ponts reman dstant.

5 Non-metrc MDS for manfolds? Rank orderng of Eucldean dstances s NOT preserved n manfold learnng.

6 Nonlnear Manfolds A PCA and MDS measure the Eucldean dstance What s mportant s the geodesc dstance Unroll the manfold

7 To preserve structure preserve the geodesc dstance and not the eucldean dstance.

8 Graph-Based Methods Tenenbaum et.al s Isomap Algorthm Global approach. Preserves global parwse dstances. Rowes and Saul s Locally Lnear Embeddng Algorthm Local approach Nearby ponts should map nearby Belkn and Nyog Laplacan Egenmaps Algorthm Local approach mnmzes approxmately the same value as LLE

9 Isomap - Key Idea: Use geodesc nstead of Eucldean dstances n MDS. For neghborng ponts Eucldean dstance s a good approxmaton to the geodesc dstance. For dstant ponts estmate the dstance by a seres of short hops between neghborng ponts. Fnd shortest paths n a graph wth edges connectng neghborng data ponts.

10 Step 1. Buld adjacency graph. Adjacency graph Vertces represent nputs. Undrected edges connect neghbours. Neghbourhood selecton Many optons: k-nearest neghbours, nputs wthn radus r, pror knowledge. Graph s dscretzed approxmaton of submanfold.

11 Buldng the graph Computaton knn scales navely as Faster methods explot data structures. Assumptons 1. Graph s connected. O( n 2 D) 2. Neghbourhoods on graph reflect neghbourhoods on manfold.

12 Step 2. Estmate geodescs Dynamc programmng Weght edges by local dstances. Compute shortest paths through graph. Geodesc dstances Estmate by lengths of shortest paths: denser samplng = better estmates. Computaton Djkstra s algorthm for shortest paths O(n 2 log n + n 2 k).

13 Step 3. Metrc MDS Embeddng Top d egenvectors of Gram matrx yeld embeddng. Dmensonalty Number of sgnfcant egenvalues yeld estmate of dmensonalty. Computaton Top d egenvectors can be computed n O(n 2 d).

14 Summary Algorthm 1. k nearest neghbours 2. shortest paths through graph 3. MDS on geodesc dstances

15 Swss Roll n (ponts) =1024 k (neghbors) =12

16 Isomap: Two-dmensonal embeddng of hand mages (from Josh. Tenenbaum, Vn de Slva, John Langford 2000) n =2000, k =6, D=64x64

17 Isomap: two-dmensonal embeddng of hand-wrtten 2 (from Josh. Tenenbaum, Vn de Slva, John Langford 2000) n =1000, r=4.2, D=20x20

18 Isomap: three-dmensonal embeddng of faces (from Josh. Tenenbaum, Vn de Slva, John Langford 2000) n =698, k=6

19 Propertes of Isomap Strengths : Preserves the global data structure Performs global optmzaton Non-parametrc (Only heurstc s neghbourhood sze) Weaknesses : Senstve to shortcuts Very slow

20 Spectral Methods Common framework 1. Derve sparse graph from knn. 2. Derve matrx from graph weghts. 3. Derve embeddng from egenvectors. Vared solutons Algorthms dffer n step 2. Types of optmzaton: shortest paths, least squares fts, semdefnte programmng.

21 Locally Lnear Embeddng (LLE) Assume that data les on a manfold: each sample and ts neghbors le on approxmately lnear subspace Idea: 1. Approxmate data by a set of lnear patches 2. Glue these patches together on a low dmensonal subspace s.t. neghborhood relatonshps between patches are preserved. Algorthm:

22 LLE at glance Steps 1. Nearest neghbour search. 2. Least squares fts. 3. Sparse egenvalue problem. Propertes Obtans hghly nonlnear embeddngs. Not prone to local mnma. Sparse graphs yeld sparse problems.

23 Step 1. Nearest neghbours search Effect of Neghbourhood Sze

24 Step 2. Compute weghts Characterze local geometry of each neghbourhood by weghts Wj. Compute weghts by reconstructng each nput (lnearly) from neghbours.

25 Lnear reconstructons Local lnearty Assume neghbours le on locally lnear patches of a low dmensonal manfold. Mnmze reconstructon error Each pont can be wrtten as a lnear combnaton of ts neghbors. The weghts chosen to mnmze the reconstructon error: mn x Wj x W j j 2

26 Least squares fts (Computng W j ) Local reconstructons Choose weghts to mnmze: Constrants W = 0 x j Set f s not a neghbor of j Weghts must sum to one: Wj = 1 j Local nvarance W j nvarance to translaton Φ( W ) = x W Optmal weghts are nvarant to rotaton, translaton, and scalng. x j j x j 2

27 Step 3. Fndng the Embeddng Low dmensonal representaton Map nputs to outputs: Mnmze reconstructon errors Optmze outputs for fxed weghts: x R Ψ ( y) = y W j D j y j 2 y R d Constrants: Center outputs on orgn y = 0 1 N Impose unt covarance matrx y y = I d

28 Mnmzaton Quadratc form: ( y y ) Ψ( y) = M M δ j j j j =δ W W + W W, j j j 1 f = j = 0 otherwse j k k kj It can be shown that T M = ( I W ) ( I W )

29 Sparse egenvalue problem Optmal embeddng gven by bottom d+1 egenvectors, correspondng to the d+1 smallest egenvalues (Raylegh-Rtz theorem). Soluton Dscard bottom egenvector [1 1 1] (wth egenvalue zero). Other egenvectors satsfy constrants.

30 Surfaces N=1000 nputs k=8 nearest neghbors

31 Lps N=15960 mages K=24 neghbors D=65664 pxels d=2 (shown)

32 Pose and expresson N=1965 mages k=12 nearest neghbors D=560 pxels d=2 (shown)

33 Propertes of LLE Strengths: Fast No local mnma Non-teratve Non-parametrc (only heurstc s neghbourhood sze). Weaknesses: Senstve to shortcuts No estmate of dmensonalty

34 LLE versus Isomap Many smlartes Graph-based, spectral method No local mnma Essental dfferences Does not estmate dmensonalty No theoretcal guarantees Constructs sparse vs. dense matrx Preserves weghts vs. dstances Much faster

35 Laplacan Egenmaps Map nearby nputs to nearby outputs, where nearness s encoded by graph. Summary of the Algorthm 1. Identfy k-nearest neghbours (as n LLE) 2. Assgn weghts to neghbours 3. Sparse egenvalue problem

36 Step 2. Construct the graph Vertces represent nputs. Undrected edges connect neghbours. Assgn weghts to neghbours: Smple: W j = 1 or Heat kernel W j = exp β ( ) 2 x x j

37 Step 3. Graph Laplacan Compute outputs by mnmzng: = W y Ψ( y) y under approprate constrants Ψ = j j ( ) ( 2 2 y = W y + y 2y y ) y 2 j D + j j y 2 j D jj j j 2 2 j y j y j W j W j s symmetrc = 2y t Ly D = W j j Graph Laplacan L = D W

38 Step 3. Generalzed egenvalue problem Mnmze constraned by y t Ly Dy= 1 Optmal embeddng: y t ( Le = λde) gven by bottom d+1 egenvectors (correspondng to the d+1 smallest egenvalues). Soluton: Dscard bottom egenvector [1 1 1] (wth egenvalue zero). Other egenvectors satsfy constrants.

39 Analyss on Manfolds Consder Remannan manfold a real dfferentable manfold n whch tangent space s equpped wth dot product. Laplace Beltram operator Ω Ω R has a natural operator on dfferentable functons. s a second order dfferental operator defned as a dvergence of the gradent D = 2 x 2

40 Spectral desomposton of Assume L 2 (Ω) s space of all square ntegrable functons on Ω s a self-adjont postve sem- defnate operator and ts egenfunctons form the bass. Thus all f n L 2 (Ω) can be wrtten as ( x) = α e ( x) (provded Ω s compact) f

41 Smoothness functonal Defned as value close to zero mples f beng smooth. ( ) Ω Ω = = = 2, ) ( 2 L f f d f f d f f S ω ω value close to zero mples f beng smooth. Snce we have e e e S λ = =, ) ( = = = e e f f f S α λ α α,, ) ( choosng the lowest p egenfunctons provdes a maxmally smooth approxmaton to the manfold.

42 Spectral graph theory Weghted graph s dscretzed representaton of manfold. Laplacan measures smoothness of functons over manfold and graph. Manfold: Graph: Ω j f W 2 dω = f f dω ( ) 2 t f f f Lf j j =

43 Interpretng Laplacan Egenmaps Egenvectors functons from nodes to R n a way that "close by" ponts are assgned "close by" values. Egenvalues measure how close are the values of neghbourng ponts smoothness.

44 Example: S1 (the crcle) Contnuous Egenfunctons of Laplacan are bass for perodc functons on crcle, ordered by smoothness. Egenvalues measure smoothness.

45 Example: S1 (the crcle) Dscrete (n equally spaced ponts) Egenvectors of graph Laplacan are dscrete snes and cosnes. Egenvalues measure smoothness.

46 Laplacan vs LLE More smlar than dfferent Graph-based, spectral method Sparse egenvalue problem Smlar results n practce Essental dfferences Preserves localty vs local lnearty Uses graph Laplacan

12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification

12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification Introducton to Artfcal Intellgence V22.0472-001 Fall 2009 Lecture 24: Nearest-Neghbors & Support Vector Machnes Rob Fergus Dept of Computer Scence, Courant Insttute, NYU Sldes from Danel Yeung, John DeNero

More information

Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

Face Recognition University at Buffalo CSE666 Lecture Slides Resources: Face Recognton Unversty at Buffalo CSE666 Lecture Sldes Resources: http://www.face-rec.org/algorthms/ Overvew of face recognton algorthms Correlaton - Pxel based correspondence between two face mages Structural

More information

Feature Reduction and Selection

Feature Reduction and Selection Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components

More information

SELECTION OF THE NUMBER OF NEIGHBOURS OF EACH DATA POINT FOR THE LOCALLY LINEAR EMBEDDING ALGORITHM

SELECTION OF THE NUMBER OF NEIGHBOURS OF EACH DATA POINT FOR THE LOCALLY LINEAR EMBEDDING ALGORITHM ISSN 392 24X INFORMATION TECHNOLOGY AND CONTROL, 2007, Vol.36, No.4 SELECTION OF THE NUMBER OF NEIGHBOURS OF EACH DATA POINT FOR THE LOCALLY LINEAR EMBEDDING ALGORITHM Rasa Karbauskatė,2, Olga Kurasova,2,

More information

Laplacian Eigenmap for Image Retrieval

Laplacian Eigenmap for Image Retrieval Laplacan Egenmap for Image Retreval Xaofe He Partha Nyog Department of Computer Scence The Unversty of Chcago, 1100 E 58 th Street, Chcago, IL 60637 ABSTRACT Dmensonalty reducton has been receved much

More information

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points; Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features

More information

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces Range mages For many structured lght scanners, the range data forms a hghly regular pattern known as a range mage. he samplng pattern s determned by the specfc scanner. Range mage regstraton 1 Examples

More information

Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering

Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering Out-of-Sample Extensons for LLE, Isomap, MDS, Egenmaps, and Spectral Clusterng Yoshua Bengo, Jean-Franços Paement, Pascal Vncent Olver Delalleau, Ncolas Le Roux and Mare Oumet Département d Informatque

More information

S1 Note. Basis functions.

S1 Note. Basis functions. S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type

More information

Recognizing Faces. Outline

Recognizing Faces. Outline Recognzng Faces Drk Colbry Outlne Introducton and Motvaton Defnng a feature vector Prncpal Component Analyss Lnear Dscrmnate Analyss !"" #$""% http://www.nfotech.oulu.f/annual/2004 + &'()*) '+)* 2 ! &

More information

Support Vector Machines

Support Vector Machines /9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.

More information

Radial Basis Functions

Radial Basis Functions Radal Bass Functons Mesh Reconstructon Input: pont cloud Output: water-tght manfold mesh Explct Connectvty estmaton Implct Sgned dstance functon estmaton Image from: Reconstructon and Representaton of

More information

Graph-based Clustering

Graph-based Clustering Graphbased Clusterng Transform the data nto a graph representaton ertces are the data ponts to be clustered Edges are eghted based on smlarty beteen data ponts Graph parttonng Þ Each connected component

More information

Learning an Image Manifold for Retrieval

Learning an Image Manifold for Retrieval Learnng an Image Manfold for Retreval Xaofe He*, We-Yng Ma, and Hong-Jang Zhang Mcrosoft Research Asa Bejng, Chna, 100080 {wyma,hjzhang}@mcrosoft.com *Department of Computer Scence, The Unversty of Chcago

More information

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Decson surface s a hyperplane (lne n 2D) n feature space (smlar to the Perceptron) Arguably, the most mportant recent dscovery n machne learnng In a nutshell: map the data to a predetermned

More information

Fitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.

Fitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros. Fttng & Matchng Lecture 4 Prof. Bregler Sldes from: S. Lazebnk, S. Setz, M. Pollefeys, A. Effros. How do we buld panorama? We need to match (algn) mages Matchng wth Features Detect feature ponts n both

More information

Discriminative Dictionary Learning with Pairwise Constraints

Discriminative Dictionary Learning with Pairwise Constraints Dscrmnatve Dctonary Learnng wth Parwse Constrants Humn Guo Zhuoln Jang LARRY S. DAVIS UNIVERSITY OF MARYLAND Nov. 6 th, Outlne Introducton/motvaton Dctonary Learnng Dscrmnatve Dctonary Learnng wth Parwse

More information

Active Contours/Snakes

Active Contours/Snakes Actve Contours/Snakes Erkut Erdem Acknowledgement: The sldes are adapted from the sldes prepared by K. Grauman of Unversty of Texas at Austn Fttng: Edges vs. boundares Edges useful sgnal to ndcate occludng

More information

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like: Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 5 Luca Trevisan September 7, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 5 Luca Trevisan September 7, 2017 U.C. Bereley CS294: Beyond Worst-Case Analyss Handout 5 Luca Trevsan September 7, 207 Scrbed by Haars Khan Last modfed 0/3/207 Lecture 5 In whch we study the SDP relaxaton of Max Cut n random graphs. Quc

More information

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Learning the Kernel Parameters in Kernel Minimum Distance Classifier Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department

More information

The Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique

The Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique //00 :0 AM Outlne and Readng The Greedy Method The Greedy Method Technque (secton.) Fractonal Knapsack Problem (secton..) Task Schedulng (secton..) Mnmum Spannng Trees (secton.) Change Money Problem Greedy

More information

Unsupervised Learning and Clustering

Unsupervised Learning and Clustering Unsupervsed Learnng and Clusterng Why consder unlabeled samples?. Collectng and labelng large set of samples s costly Gettng recorded speech s free, labelng s tme consumng 2. Classfer could be desgned

More information

Unsupervised Learning

Unsupervised Learning Pattern Recognton Lecture 8 Outlne Introducton Unsupervsed Learnng Parametrc VS Non-Parametrc Approach Mxture of Denstes Maxmum-Lkelhood Estmates Clusterng Prof. Danel Yeung School of Computer Scence and

More information

Multi-stable Perception. Necker Cube

Multi-stable Perception. Necker Cube Mult-stable Percepton Necker Cube Spnnng dancer lluson, Nobuuk Kaahara Fttng and Algnment Computer Vson Szelsk 6.1 James Has Acknowledgment: Man sldes from Derek Hoem, Lana Lazebnk, and Grauman&Lebe 2008

More information

BAYESIAN MULTI-SOURCE DOMAIN ADAPTATION

BAYESIAN MULTI-SOURCE DOMAIN ADAPTATION BAYESIAN MULTI-SOURCE DOMAIN ADAPTATION SHI-LIANG SUN, HONG-LEI SHI Department of Computer Scence and Technology, East Chna Normal Unversty 500 Dongchuan Road, Shangha 200241, P. R. Chna E-MAIL: slsun@cs.ecnu.edu.cn,

More information

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1 4/14/011 Outlne Dscrmnatve classfers for mage recognton Wednesday, Aprl 13 Krsten Grauman UT-Austn Last tme: wndow-based generc obect detecton basc ppelne face detecton wth boostng as case study Today:

More information

CS 534: Computer Vision Model Fitting

CS 534: Computer Vision Model Fitting CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15 CS434a/541a: Pattern Recognton Prof. Olga Veksler Lecture 15 Today New Topc: Unsupervsed Learnng Supervsed vs. unsupervsed learnng Unsupervsed learnng Net Tme: parametrc unsupervsed learnng Today: nonparametrc

More information

Structure from Motion

Structure from Motion Structure from Moton Structure from Moton For now, statc scene and movng camera Equvalentl, rgdl movng scene and statc camera Lmtng case of stereo wth man cameras Lmtng case of multvew camera calbraton

More information

Hermite Splines in Lie Groups as Products of Geodesics

Hermite Splines in Lie Groups as Products of Geodesics Hermte Splnes n Le Groups as Products of Geodescs Ethan Eade Updated May 28, 2017 1 Introducton 1.1 Goal Ths document defnes a curve n the Le group G parametrzed by tme and by structural parameters n the

More information

Infrared face recognition using texture descriptors

Infrared face recognition using texture descriptors Infrared face recognton usng texture descrptors Moulay A. Akhlouf*, Abdelhakm Bendada Computer Vson and Systems Laboratory, Laval Unversty, Quebec, QC, Canada G1V0A6 ABSTRACT Face recognton s an area of

More information

MULTI-VIEW ANCHOR GRAPH HASHING

MULTI-VIEW ANCHOR GRAPH HASHING MULTI-VIEW ANCHOR GRAPH HASHING Saehoon Km 1 and Seungjn Cho 1,2 1 Department of Computer Scence and Engneerng, POSTECH, Korea 2 Dvson of IT Convergence Engneerng, POSTECH, Korea {kshkawa, seungjn}@postech.ac.kr

More information

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law)

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law) Machne Learnng Support Vector Machnes (contans materal adapted from talks by Constantn F. Alfers & Ioanns Tsamardnos, and Martn Law) Bryan Pardo, Machne Learnng: EECS 349 Fall 2014 Support Vector Machnes

More information

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline mage Vsualzaton mage Vsualzaton mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and Analyss outlne mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and

More information

Semi-Supervised Discriminant Analysis Based On Data Structure

Semi-Supervised Discriminant Analysis Based On Data Structure IOSR Journal of Computer Engneerng (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 3, Ver. VII (May Jun. 2015), PP 39-46 www.osrournals.org Sem-Supervsed Dscrmnant Analyss Based On Data

More information

GSLM Operations Research II Fall 13/14

GSLM Operations Research II Fall 13/14 GSLM 58 Operatons Research II Fall /4 6. Separable Programmng Consder a general NLP mn f(x) s.t. g j (x) b j j =. m. Defnton 6.. The NLP s a separable program f ts objectve functon and all constrants are

More information

Smoothing Spline ANOVA for variable screening

Smoothing Spline ANOVA for variable screening Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory

More information

Fitting: Deformable contours April 26 th, 2018

Fitting: Deformable contours April 26 th, 2018 4/6/08 Fttng: Deformable contours Aprl 6 th, 08 Yong Jae Lee UC Davs Recap so far: Groupng and Fttng Goal: move from array of pxel values (or flter outputs) to a collecton of regons, objects, and shapes.

More information

Lecture 4: Principal components

Lecture 4: Principal components /3/6 Lecture 4: Prncpal components 3..6 Multvarate lnear regresson MLR s optmal for the estmaton data...but poor for handlng collnear data Covarance matrx s not nvertble (large condton number) Robustness

More information

Manifold embedding for modeling spinal deformations

Manifold embedding for modeling spinal deformations Manfold embeddng for modelng spnal deformatons Samuel Kadoury Phlps Research North Amerca MICCAI 211 Tutoral on Manfold Learnng wth Medcal Images September 22 th, 211 Spnal deformtes Adolescent Idopathc

More information

Machine Learning 9. week

Machine Learning 9. week Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below

More information

Programming in Fortran 90 : 2017/2018

Programming in Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Exercse 1 : Evaluaton of functon dependng on nput Wrte a program who evaluate the functon f (x,y) for any two user specfed values

More information

Machine Learning: Algorithms and Applications

Machine Learning: Algorithms and Applications 14/05/1 Machne Learnng: Algorthms and Applcatons Florano Zn Free Unversty of Bozen-Bolzano Faculty of Computer Scence Academc Year 011-01 Lecture 10: 14 May 01 Unsupervsed Learnng cont Sldes courtesy of

More information

Learning a Locality Preserving Subspace for Visual Recognition

Learning a Locality Preserving Subspace for Visual Recognition Learnng a Localty Preservng Subspace for Vsual Recognton Xaofe He *, Shucheng Yan #, Yuxao Hu, and Hong-Jang Zhang Mcrosoft Research Asa, Bejng 100080, Chna * Department of Computer Scence, Unversty of

More information

Classification / Regression Support Vector Machines

Classification / Regression Support Vector Machines Classfcaton / Regresson Support Vector Machnes Jeff Howbert Introducton to Machne Learnng Wnter 04 Topcs SVM classfers for lnearly separable classes SVM classfers for non-lnearly separable classes SVM

More information

Image Alignment CSC 767

Image Alignment CSC 767 Image Algnment CSC 767 Image algnment Image from http://graphcs.cs.cmu.edu/courses/15-463/2010_fall/ Image algnment: Applcatons Panorama sttchng Image algnment: Applcatons Recognton of object nstances

More information

The Discriminate Analysis and Dimension Reduction Methods of High Dimension

The Discriminate Analysis and Dimension Reduction Methods of High Dimension Open Journal of Socal Scences, 015, 3, 7-13 Publshed Onlne March 015 n ScRes. http://www.scrp.org/journal/jss http://dx.do.org/10.436/jss.015.3300 The Dscrmnate Analyss and Dmenson Reducton Methods of

More information

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning Computer Anmaton and Vsualsaton Lecture 4. Rggng / Sknnng Taku Komura Overvew Sknnng / Rggng Background knowledge Lnear Blendng How to decde weghts? Example-based Method Anatomcal models Sknnng Assume

More information

All-Pairs Shortest Paths. Approximate All-Pairs shortest paths Approximate distance oracles Spanners and Emulators. Uri Zwick Tel Aviv University

All-Pairs Shortest Paths. Approximate All-Pairs shortest paths Approximate distance oracles Spanners and Emulators. Uri Zwick Tel Aviv University Approxmate All-Pars shortest paths Approxmate dstance oracles Spanners and Emulators Ur Zwck Tel Avv Unversty Summer School on Shortest Paths (PATH05 DIKU, Unversty of Copenhagen All-Pars Shortest Paths

More information

Unsupervised Co-segmentation of 3D Shapes via Functional Maps

Unsupervised Co-segmentation of 3D Shapes via Functional Maps Unsupervsed Co-segmentaton of 3D Shapes va Functonal aps Jun Yang School of Electronc and Informaton Engneerng, Lanzhou Jaotong Unversty, Lanzhou 730070, P. R. Chna yangj@mal.lzjtu.cn Zhenhua Tan School

More information

IMAGE matting is an important but still challenging problem

IMAGE matting is an important but still challenging problem IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Patch Algnment Manfold Mattng Xuelong L, Fellow, IEEE, Kang Lu, Member, IEEE, Yongsheng Dong, Member, IEEE, and Dacheng Tao, Fellow, IEEE arxv:1904.07588v1

More information

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal

More information

Announcements. Supervised Learning

Announcements. Supervised Learning Announcements See Chapter 5 of Duda, Hart, and Stork. Tutoral by Burge lnked to on web page. Supervsed Learnng Classfcaton wth labeled eamples. Images vectors n hgh-d space. Supervsed Learnng Labeled eamples

More information

Barycentric Coordinates. From: Mean Value Coordinates for Closed Triangular Meshes by Ju et al.

Barycentric Coordinates. From: Mean Value Coordinates for Closed Triangular Meshes by Ju et al. Barycentrc Coordnates From: Mean Value Coordnates for Closed Trangular Meshes by Ju et al. Motvaton Data nterpolaton from the vertces of a boundary polygon to ts nteror Boundary value problems Shadng Space

More information

Geometry-aware Metric Learning

Geometry-aware Metric Learning Zhengdong Lu LUZ@CS.UTEXAS.EDU Prateek Jan PJAIN@CS.UTEXAS.EDU Inderjt S. Dhllon INDERJIT@CS.UTEXAS.EDU Dept. of Computer Scence, Unversty of Texas at Austn, Unversty Staton C5, Austn, TX 78712 Abstract

More information

Unsupervised Learning and Clustering

Unsupervised Learning and Clustering Unsupervsed Learnng and Clusterng Supervsed vs. Unsupervsed Learnng Up to now we consdered supervsed learnng scenaro, where we are gven 1. samples 1,, n 2. class labels for all samples 1,, n Ths s also

More information

Calibrating a single camera. Odilon Redon, Cyclops, 1914

Calibrating a single camera. Odilon Redon, Cyclops, 1914 Calbratng a sngle camera Odlon Redon, Cclops, 94 Our goal: Recover o 3D structure Recover o structure rom one mage s nherentl ambguous??? Sngle-vew ambgut Sngle-vew ambgut Rashad Alakbarov shadow sculptures

More information

Harmonic Coordinates for Character Articulation PIXAR

Harmonic Coordinates for Character Articulation PIXAR Harmonc Coordnates for Character Artculaton PIXAR Pushkar Josh Mark Meyer Tony DeRose Bran Green Tom Sanock We have a complex source mesh nsde of a smpler cage mesh We want vertex deformatons appled to

More information

Ecient Computation of the Most Probable Motion from Fuzzy. Moshe Ben-Ezra Shmuel Peleg Michael Werman. The Hebrew University of Jerusalem

Ecient Computation of the Most Probable Motion from Fuzzy. Moshe Ben-Ezra Shmuel Peleg Michael Werman. The Hebrew University of Jerusalem Ecent Computaton of the Most Probable Moton from Fuzzy Correspondences Moshe Ben-Ezra Shmuel Peleg Mchael Werman Insttute of Computer Scence The Hebrew Unversty of Jerusalem 91904 Jerusalem, Israel Emal:

More information

Interactive Rendering of Translucent Objects

Interactive Rendering of Translucent Objects Interactve Renderng of Translucent Objects Hendrk Lensch Mchael Goesele Phlppe Bekaert Jan Kautz Marcus Magnor Jochen Lang Hans-Peter Sedel 2003 Presented By: Mark Rubelmann Outlne Motvaton Background

More information

A Bilinear Model for Sparse Coding

A Bilinear Model for Sparse Coding A Blnear Model for Sparse Codng Davd B. Grmes and Rajesh P. N. Rao Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA 98195-2350, U.S.A. grmes,rao @cs.washngton.edu Abstract

More information

Multicriteria Decision Making

Multicriteria Decision Making Multcrtera Decson Makng Andrés Ramos (Andres.Ramos@comllas.edu) Pedro Sánchez (Pedro.Sanchez@comllas.edu) Sonja Wogrn (Sonja.Wogrn@comllas.edu) Contents 1. Basc concepts 2. Contnuous methods 3. Dscrete

More information

Data-dependent Hashing Based on p-stable Distribution

Data-dependent Hashing Based on p-stable Distribution Data-depent Hashng Based on p-stable Dstrbuton Author Ba, Xao, Yang, Hachuan, Zhou, Jun, Ren, Peng, Cheng, Jan Publshed 24 Journal Ttle IEEE Transactons on Image Processng DOI https://do.org/.9/tip.24.2352458

More information

Human Action Recognition Using Dynamic Time Warping Algorithm and Reproducing Kernel Hilbert Space for Matrix Manifold

Human Action Recognition Using Dynamic Time Warping Algorithm and Reproducing Kernel Hilbert Space for Matrix Manifold IJCTA, 10(07), 2017, pp 79-85 Internatonal Scence Press Closed Loop Control of Soft Swtched Forward Converter Usng Intellgent Controller 79 Human Acton Recognton Usng Dynamc Tme Warpng Algorthm and Reproducng

More information

Semi-Supervised Kernel Mean Shift Clustering

Semi-Supervised Kernel Mean Shift Clustering IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. XX, JANUARY XXXX 1 Sem-Supervsed Kernel Mean Shft Clusterng Saket Anand, Student Member, IEEE, Sushl Mttal, Member, IEEE, Oncel

More information

New Extensions of the 3-Simplex for Exterior Orientation

New Extensions of the 3-Simplex for Exterior Orientation New Extensons of the 3-Smplex for Exteror Orentaton John M. Stenbs Tyrone L. Vncent Wllam A. Hoff Colorado School of Mnes jstenbs@gmal.com tvncent@mnes.edu whoff@mnes.edu Abstract Object pose may be determned

More information

Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations

Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations Fxng Max-Product: Convergent Message Passng Algorthms for MAP LP-Relaxatons Amr Globerson Tomm Jaakkola Computer Scence and Artfcal Intellgence Laboratory Massachusetts Insttute of Technology Cambrdge,

More information

SIGGRAPH Interactive Image Cutout. Interactive Graph Cut. Interactive Graph Cut. Interactive Graph Cut. Hard Constraints. Lazy Snapping.

SIGGRAPH Interactive Image Cutout. Interactive Graph Cut. Interactive Graph Cut. Interactive Graph Cut. Hard Constraints. Lazy Snapping. SIGGRAPH 004 Interactve Image Cutout Lazy Snappng Yn L Jan Sun Ch-Keung Tang Heung-Yeung Shum Mcrosoft Research Asa Hong Kong Unversty Separate an object from ts background Compose the object on another

More information

11. APPROXIMATION ALGORITHMS

11. APPROXIMATION ALGORITHMS Copng wth NP-completeness 11. APPROXIMATION ALGORITHMS load balancng center selecton prcng method: vertex cover LP roundng: vertex cover generalzed load balancng knapsack problem Q. Suppose I need to solve

More information

CS246: Mining Massive Datasets Jure Leskovec, Stanford University

CS246: Mining Massive Datasets Jure Leskovec, Stanford University CS46: Mnng Massve Datasets Jure Leskovec, Stanford Unversty http://cs46.stanford.edu /19/013 Jure Leskovec, Stanford CS46: Mnng Massve Datasets, http://cs46.stanford.edu Perceptron: y = sgn( x Ho to fnd

More information

Intra-Parametric Analysis of a Fuzzy MOLP

Intra-Parametric Analysis of a Fuzzy MOLP Intra-Parametrc Analyss of a Fuzzy MOLP a MIAO-LING WANG a Department of Industral Engneerng and Management a Mnghsn Insttute of Technology and Hsnchu Tawan, ROC b HSIAO-FAN WANG b Insttute of Industral

More information

On the Optimality of Spectral Compression of Meshes

On the Optimality of Spectral Compression of Meshes On the Optmalty of Spectral Compresson of Meshes MIRELA BEN-CHEN AND CRAIG GOTSMAN Center for Graphcs and Geometrc Computng Technon Israel Insttute of Technology Spectral compresson of trangle meshes has

More information

Lecture 5: Multilayer Perceptrons

Lecture 5: Multilayer Perceptrons Lecture 5: Multlayer Perceptrons Roger Grosse 1 Introducton So far, we ve only talked about lnear models: lnear regresson and lnear bnary classfers. We noted that there are functons that can t be represented

More information

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 1. SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 1. SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY SSDH: Sem-supervsed Deep Hashng for Large Scale Image Retreval Jan Zhang, and Yuxn Peng arxv:607.08477v2 [cs.cv] 8 Jun 207 Abstract Hashng

More information

Cost-efficient deployment of distributed software services

Cost-efficient deployment of distributed software services 1/30 Cost-effcent deployment of dstrbuted software servces csorba@tem.ntnu.no 2/30 Short ntroducton & contents Cost-effcent deployment of dstrbuted software servces Cost functons Bo-nspred decentralzed

More information

Reading. 14. Subdivision curves. Recommended:

Reading. 14. Subdivision curves. Recommended: eadng ecommended: Stollntz, Deose, and Salesn. Wavelets for Computer Graphcs: heory and Applcatons, 996, secton 6.-6., A.5. 4. Subdvson curves Note: there s an error n Stollntz, et al., secton A.5. Equaton

More information

Hierarchical clustering for gene expression data analysis

Hierarchical clustering for gene expression data analysis Herarchcal clusterng for gene expresson data analyss Gorgo Valentn e-mal: valentn@ds.unm.t Clusterng of Mcroarray Data. Clusterng of gene expresson profles (rows) => dscovery of co-regulated and functonally

More information

Sum of Linear and Fractional Multiobjective Programming Problem under Fuzzy Rules Constraints

Sum of Linear and Fractional Multiobjective Programming Problem under Fuzzy Rules Constraints Australan Journal of Basc and Appled Scences, 2(4): 1204-1208, 2008 ISSN 1991-8178 Sum of Lnear and Fractonal Multobjectve Programmng Problem under Fuzzy Rules Constrants 1 2 Sanjay Jan and Kalash Lachhwan

More information

Two-Dimensional Supervised Discriminant Projection Method For Feature Extraction

Two-Dimensional Supervised Discriminant Projection Method For Feature Extraction Appl. Math. Inf. c. 6 No. pp. 8-85 (0) Appled Mathematcs & Informaton cences An Internatonal Journal @ 0 NP Natural cences Publshng Cor. wo-dmensonal upervsed Dscrmnant Proecton Method For Feature Extracton

More information

IMAGE MATCHING WITH SIFT FEATURES A PROBABILISTIC APPROACH

IMAGE MATCHING WITH SIFT FEATURES A PROBABILISTIC APPROACH IMAGE MATCHING WITH SIFT FEATURES A PROBABILISTIC APPROACH Jyot Joglekar a, *, Shrsh S. Gedam b a CSRE, IIT Bombay, Doctoral Student, Mumba, Inda jyotj@tb.ac.n b Centre of Studes n Resources Engneerng,

More information

A Multivariate Analysis of Static Code Attributes for Defect Prediction

A Multivariate Analysis of Static Code Attributes for Defect Prediction Research Paper) A Multvarate Analyss of Statc Code Attrbutes for Defect Predcton Burak Turhan, Ayşe Bener Department of Computer Engneerng, Bogazc Unversty 3434, Bebek, Istanbul, Turkey {turhanb, bener}@boun.edu.tr

More information

Local Quaternary Patterns and Feature Local Quaternary Patterns

Local Quaternary Patterns and Feature Local Quaternary Patterns Local Quaternary Patterns and Feature Local Quaternary Patterns Jayu Gu and Chengjun Lu The Department of Computer Scence, New Jersey Insttute of Technology, Newark, NJ 0102, USA Abstract - Ths paper presents

More information

Monte Carlo Rendering

Monte Carlo Rendering Monte Carlo Renderng Last Tme? Modern Graphcs Hardware Cg Programmng Language Gouraud Shadng vs. Phong Normal Interpolaton Bump, Dsplacement, & Envronment Mappng Cg Examples G P R T F P D Today Does Ray

More information

An Improved Spectral Clustering Algorithm Based on Local Neighbors in Kernel Space 1

An Improved Spectral Clustering Algorithm Based on Local Neighbors in Kernel Space 1 DOI: 10.98/CSIS110415064L An Improved Spectral Clusterng Algorthm Based on Local Neghbors n Kernel Space 1 Xnyue Lu 1,, Xng Yong and Hongfe Ln 1 1 School of Computer Scence and Technology, Dalan Unversty

More information

5 The Primal-Dual Method

5 The Primal-Dual Method 5 The Prmal-Dual Method Orgnally desgned as a method for solvng lnear programs, where t reduces weghted optmzaton problems to smpler combnatoral ones, the prmal-dual method (PDM) has receved much attenton

More information

LEAST SQUARES. RANSAC. HOUGH TRANSFORM.

LEAST SQUARES. RANSAC. HOUGH TRANSFORM. LEAS SQUARES. RANSAC. HOUGH RANSFORM. he sldes are from several sources through James Has (Brown); Srnvasa Narasmhan (CMU); Slvo Savarese (U. of Mchgan); Bll Freeman and Antono orralba (MI), ncludng ther

More information

Snakes-based approach for extraction of building roof contours from digital aerial images

Snakes-based approach for extraction of building roof contours from digital aerial images Snakes-based approach for extracton of buldng roof contours from dgtal aeral mages Alur P. Dal Poz and Antono J. Fazan São Paulo State Unversty Dept. of Cartography, R. Roberto Smonsen 305 19060-900 Presdente

More information

Simplification of 3D Meshes

Simplification of 3D Meshes Smplfcaton of 3D Meshes Addy Ngan /4/00 Outlne Motvaton Taxonomy of smplfcaton methods Hoppe et al, Mesh optmzaton Hoppe, Progressve meshes Smplfcaton of 3D Meshes 1 Motvaton Hgh detaled meshes becomng

More information

DISTRIBUTED POSE AVERAGING IN CAMERA SENSOR NETWORKS USING CONSENSUS ON MANIFOLDS

DISTRIBUTED POSE AVERAGING IN CAMERA SENSOR NETWORKS USING CONSENSUS ON MANIFOLDS DISTRIBUTED POSE AVERAGING IN CAMERA SENSOR NETWORKS USING CONSENSUS ON MANIFOLDS Roberto Tron, René Vdal Johns Hopns Unversty Center for Imagng Scence 32B Clar Hall, 34 N. Charles St., Baltmore MD 21218,

More information

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 48 CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 3.1 INTRODUCTION The raw mcroarray data s bascally an mage wth dfferent colors ndcatng hybrdzaton (Xue

More information

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION Paulo Quntlano 1 & Antono Santa-Rosa 1 Federal Polce Department, Brasla, Brazl. E-mals: quntlano.pqs@dpf.gov.br and

More information

Semi-supervised Classification Using Local and Global Regularization

Semi-supervised Classification Using Local and Global Regularization Proceedngs of the Twenty-Thrd AAAI Conference on Artfcal Intellgence (2008) Sem-supervsed Classfcaton Usng Local and Global Regularzaton Fe Wang 1, Tao L 2, Gang Wang 3, Changshu Zhang 1 1 Department of

More information

Lobachevsky State University of Nizhni Novgorod. Polyhedron. Quick Start Guide

Lobachevsky State University of Nizhni Novgorod. Polyhedron. Quick Start Guide Lobachevsky State Unversty of Nzhn Novgorod Polyhedron Quck Start Gude Nzhn Novgorod 2016 Contents Specfcaton of Polyhedron software... 3 Theoretcal background... 4 1. Interface of Polyhedron... 6 1.1.

More information

Denoising Manifold and Non-Manifold Point Clouds

Denoising Manifold and Non-Manifold Point Clouds Denosng Manfold and Non-Manfold Pont Clouds Ranjth Unnkrshnan Martal Hebert Carnege Mellon Unversty, Pttsburgh, PA 23 ranjth,hebert@cs.cmu.edu Abstract The fathful reconstructon of 3-D models from rregular

More information

Discriminative classifiers for object classification. Last time

Discriminative classifiers for object classification. Last time Dscrmnatve classfers for object classfcaton Thursday, Nov 12 Krsten Grauman UT Austn Last tme Supervsed classfcaton Loss and rsk, kbayes rule Skn color detecton example Sldng ndo detecton Classfers, boostng

More information

Support Vector Machines. CS534 - Machine Learning

Support Vector Machines. CS534 - Machine Learning Support Vector Machnes CS534 - Machne Learnng Perceptron Revsted: Lnear Separators Bnar classfcaton can be veed as the task of separatng classes n feature space: b > 0 b 0 b < 0 f() sgn( b) Lnear Separators

More information

Epitomic image factorization via neighbor-embedding

Epitomic image factorization via neighbor-embedding Eptomc mage factorzaton va neghbor-embeddng Mehmet Turkan, Martn Alan, Domnque Thoreau, Phlppe Gullotel, Chrstne Gullemot To cte ths verson: Mehmet Turkan, Martn Alan, Domnque Thoreau, Phlppe Gullotel,

More information

Multi-View Surveillance Video Summarization via Joint Embedding and Sparse Optimization

Multi-View Surveillance Video Summarization via Joint Embedding and Sparse Optimization IEEE TRANSACTIONS ON MULTIMEDIA, VOL. XX, NO. XX, DECEMBER 20XX 1 Mult-Vew Survellance Vdeo Summarzaton va Jont Embeddng and Sparse Optmzaton Rameswar Panda and Amt K. Roy-Chowdhury, Senor Member, IEEE

More information