K-means and Hierarchical Clustering
|
|
- Leslie Stafford
- 5 years ago
- Views:
Transcription
1 Note to other teachers and users of these sldes. Andrew would be delghted f you found ths source materal useful n gvng your own lectures. Feel free to use these sldes verbatm, or to modfy them to ft your own needs. PowerPont orgnals are avalable. If you make use of a sgnfcant porton of these sldes n your own lecture, please nclude ths message, or the followng lnk to the source repostory of Andrew s tutorals: Comments and correctons gratefully receved. K-means and Herarchcal Clusterng Andrew W. Moore Assocate Professor School of Computer Scence Carnege Mellon Unversty awm@cs.cmu.edu Copyrght 00, Andrew W. Moore Nov 6th, 00 Some Data Ths could easly be modeled by a Gaussan Mxture (wth 5 components) But let s look at an satsfyng, frendly and nfntely popular alternatve Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde
2 Suppose you transmt the coordnates of ponts drawn randomly from ths dataset. You can nstall decodng software at the recever. You re only allowed to send two bts per pont. It ll have to be a lossy transmsson. Loss = Sum Squared Error between decoded coords and orgnal coords. What encoder/decoder wll lose the least nformaton? Lossy Compresson Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 3 Suppose you transmt the coordnates of ponts drawn randomly from ths dataset. You can nstall decodng software at the recever. You re only allowed to send two bts per pont. It ll have to be a lossy transmsson. Loss = Sum Squared Error between decoded coords and orgnal coords. What encoder/decoder wll lose the least nformaton? Break nto a grd, decode each bt-par as the mddle of each grd-cell Idea One Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde Any Better Ideas?
3 Suppose you transmt the coordnates of ponts drawn randomly from ths dataset. You can nstall decodng software at the recever. You re only allowed to send two bts per pont. It ll have to be a lossy transmsson. Loss = Sum Squared Error between decoded coords and orgnal coords. What encoder/decoder wll lose the least nformaton? Break nto a grd, decode each bt-par as the centrod of all data n that grd-cell Idea Two Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde Any Further Ideas? K-means. Ask user how many clusters they d lke. (e.g. k=5) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 6 3
4 K-means. Ask user how many clusters they d lke. (e.g. k=5). Randomly guess k cluster Center locatons Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 7 K-means. Ask user how many clusters they d lke. (e.g. k=5). Randomly guess k cluster Center locatons 3. Each datapont fnds out whch Center t s closest to. (Thus each Center owns a set of dataponts) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 8 4
5 K-means. Ask user how many clusters they d lke. (e.g. k=5). Randomly guess k cluster Center locatons 3. Each datapont fnds out whch Center t s closest to. 4. Each Center fnds the centrod of the ponts t owns Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 9 K-means. Ask user how many clusters they d lke. (e.g. k=5). Randomly guess k cluster Center locatons 3. Each datapont fnds out whch Center t s closest to. 4. Each Center fnds the centrod of the ponts t owns 5. and umps there 6. Repeat untl termnated! Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 0 5
6 K-means Start Advance apologes: n Black and Whte ths example wll deterorate Example generated by Dan Pelleg s super-duper fast K-means system: Dan Pelleg and Andrew Moore. Acceleratng Exact k-means Algorthms wth Geometrc Reasonng. Proc. Conference on Knowledge Dscovery n Databases 999, (KDD99) (avalable on Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 6
7 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 3 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 4 7
8 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 5 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 6 8
9 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 7 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 8 9
10 K-means contnues Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 9 K-means termnates Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 0 0
11 K-means Questons What s t tryng to optmze? Are we sure t wll termnate? Are we sure t wll fnd an optmal clusterng? How should we start t? How could we automatcally choose the number of centers?.we ll deal wth these questons over the next few sldes Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde Dstorton Gven.. an encoder functon: ENCODE : R m [..k] a decoder functon: DECODE : [..k] R m Defne Dstorton ( x DECODE[ ENCODE( x)] ) = R = Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde
12 Dstorton Gven.. an encoder functon: ENCODE : R m [..k] a decoder functon: DECODE : [..k] R m Defne Dstorton ( x DECODE[ ENCODE( x)] ) = R = We may as well wrte DECODE[ ] = c so Dstorton = R = ( x c ENCODE ( x ) ) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 3 The Mnmal Dstorton Dstorton = R = ( x c What propertes must centers c, c,, c k have when dstorton s mnmzed? ENCODE ( x )) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 4
13 The Mnmal Dstorton () Dstorton What propertes must centers c, c,, c k have when dstorton s mnmzed? () x must be encoded by ts nearest center.why? = R = ( x c ENCODE ( x )) c ENCODE ( x ) = argmn ( x c ) c { c, c,... c }..at the mnmal dstorton k Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 5 The Mnmal Dstorton () Dstorton = R = ( x c What propertes must centers c, c,, c k have when dstorton s mnmzed? () x must be encoded by ts nearest center.why? Otherwse dstorton could be reduced by replacng ENCODE[x ] by the nearest center c ENCODE ( x )) ENCODE ( x ) = argmn ( x c ) c { c, c,... c }..at the mnmal dstorton k Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 6 3
14 The Mnmal Dstorton () Dstorton = R = ( x c What propertes must centers c, c,, c k have when dstorton s mnmzed? () The partal dervatve of Dstorton wth respect to each center locaton must be zero. ENCODE ( x )) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 7 () The partal dervatve of Dstorton wth respect to each center locaton must be zero. Dstorton Dstorton c = = = = = R = k = OwnedBy( c ) c ( x c ENCODE ( x ) ( x c ( x c OwnedBy( c ) ( x OwnedBy( c ) c 0 (for a mnmum) ) ) ) ) OwnedBy(c ) = the set of records owned by Center c. Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 8 4
15 () The partal dervatve of Dstorton wth respect to each center locaton must be zero. Dstorton = = R = k ( x c ENCODE ( x ) ( x c = OwnedBy( c ) ) ) Dstorton c = c = ( x c OwnedBy( c ) ( x OwnedBy( c ) c = 0 (for a mnmum) Thus, at a mnmum: c = OwnedBy( c Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 9 ) ) ) x OwnedBy( c ) At the mnmum dstorton Dstorton = R = ( x c What propertes must centers c, c,, c k have when dstorton s mnmzed? () x must be encoded by ts nearest center () Each Center must be at the centrod of ponts t owns. ENCODE ( x )) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 30 5
16 Improvng a suboptmal confguraton Dstorton What propertes can be changed for centers c, c,, c k have when dstorton s not mnmzed? () Change encodng so that x s encoded by ts nearest center () Set each Center to the centrod of ponts t owns. There s no pont applyng ether operaton twce n successon. But t can be proftable to alternate. And that s K-means! = R = ( x c Easy to prove ths procedure wll termnate n a state at whch nether () or () change the confguraton. Why? Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 3 ENCODE ( x )) Improvng a suboptmal confguraton What propertes can be changed for centers c, c,, c k have when dstorton s not mnmzed? () Change encodng so that x s encoded by ts nearest center () Set each Center to the centrod of ponts t owns. There s no pont applyng ether operaton twce n successon. But t can be proftable to alternate. And that s K-means! There are only a fnte number of ways of parttonng R records nto k groups. Dstorton = R = ( x c Easy to prove ths procedure wll termnate n a state at whch nether () or () change the confguraton. Why? Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 3 ENCODE ( x )) So there are only a fnte number of possble confguratons n whch all Centers are the centrods of the ponts they own. If the confguraton changes on an teraton, t must have mproved the dstorton. So each tme the confguraton changes t must go to a confguraton t s never been to before. So f t tred to go on forever, t would eventually run out of confguratons. 6
17 Wll we fnd the optmal confguraton? Not necessarly. Can you nvent a confguraton that has converged, but does not have the mnmum dstorton? Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 33 Wll we fnd the optmal confguraton? Not necessarly. Can you nvent a confguraton that has converged, but does not have the mnmum dstorton? (Hnt: try a fendsh k=3 confguraton here ) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 34 7
18 Wll we fnd the optmal confguraton? Not necessarly. Can you nvent a confguraton that has converged, but does not have the mnmum dstorton? (Hnt: try a fendsh k=3 confguraton here ) Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 35 Tryng to fnd good optma Idea : Be careful about where you start Idea : Do many runs of k-means, each from a dfferent random start confguraton Many other deas floatng around. Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 36 8
19 Tryng to fnd good optma Idea : Be careful about where you start Idea : Do many runs of k-means, each from a dfferent random start confguraton Neat trck: Place frst center on top of randomly chosen datapont. Many other deas floatng around. Place second center on datapont that s as far away as possble from frst center : Place th center on datapont that s as far away as possble from the closest of Centers through - : Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 37 Choosng the number of Centers A dffcult problem Most common approach s to try to fnd the soluton that mnmzes the Schwarz Crteron (also related to the BIC) Dstorton +? (# parameters ) log R = Dstorton +?mk log R m=#dmensons k=#centers R=#Records Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 38 9
20 Common uses of K-means Often used as an exploratory data analyss tool In one-dmenson, a good way to quantze realvalued varables nto k non-unform buckets Used on acoustc data n speech understandng to convert waveforms nto one of k categores (known as Vector Quantzaton) Also used for choosng color palettes on old fashoned graphcal dsplay devces! Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 39 Sngle Lnkage Herarchcal Clusterng. Say Every pont s t s own cluster Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 40 0
21 Sngle Lnkage Herarchcal Clusterng. Say Every pont s t s own cluster. Fnd most smlar par of clusters Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 4 Sngle Lnkage Herarchcal Clusterng. Say Every pont s t s own cluster. Fnd most smlar par of clusters 3. Merge t nto a parent cluster Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 4
22 Sngle Lnkage Herarchcal Clusterng. Say Every pont s t s own cluster. Fnd most smlar par of clusters 3. Merge t nto a parent cluster 4. Repeat Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 43 Sngle Lnkage Herarchcal Clusterng. Say Every pont s t s own cluster. Fnd most smlar par of clusters 3. Merge t nto a parent cluster 4. Repeat Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 44
23 Sngle Lnkage Herarchcal Clusterng How do we defne smlarty between clusters? Mnmum dstance between ponts n clusters (n whch case we re smply dong Eucldan Mnmum Spannng Trees) Maxmum dstance between ponts n clusters Average dstance between ponts n clusters You re left wth a nce dendrogram, or taxonomy, or herarchy of dataponts (not shown here). Say Every pont s t s own cluster. Fnd most smlar par of clusters 3. Merge t nto a parent cluster 4. Repeat untl you ve merged the whole dataset nto one cluster Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 45 Also known n the trade as Herarchcal Agglomeratve Clusterng (note the acronym) Sngle Lnkage Comments It s nce that you get a herarchy nstead of an amorphous collecton of groups If you want k groups, ust cut the (k-) longest lnks There s no real statstcal or nformatontheoretc foundaton to ths. Makes your lecturer feel a bt queasy. Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 46 3
24 What you should know All the detals of K-means The theory behnd K-means as an optmzaton algorthm How K-means can get stuck The outlne of Herarchcal clusterng Be able to contrast between whch problems would be relatvely well/poorly suted to K- means vs Gaussan Mxtures vs Herarchcal clusterng Copyrght 00, Andrew W. Moore K-means and Herarchcal Clusterng: Slde 47 4
K-means and Hierarchical Clustering
K-means and Hierarchical Clustering Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to use these
More informationHierarchical clustering for gene expression data analysis
Herarchcal clusterng for gene expresson data analyss Gorgo Valentn e-mal: valentn@ds.unm.t Clusterng of Mcroarray Data. Clusterng of gene expresson profles (rows) => dscovery of co-regulated and functonally
More informationMachine Learning: Algorithms and Applications
14/05/1 Machne Learnng: Algorthms and Applcatons Florano Zn Free Unversty of Bozen-Bolzano Faculty of Computer Scence Academc Year 011-01 Lecture 10: 14 May 01 Unsupervsed Learnng cont Sldes courtesy of
More informationMachine Learning 9. week
Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below
More informationMachine Learning. Topic 6: Clustering
Machne Learnng Topc 6: lusterng lusterng Groupng data nto (hopefully useful) sets. Thngs on the left Thngs on the rght Applcatons of lusterng Hypothess Generaton lusters mght suggest natural groups. Hypothess
More informationSmoothing Spline ANOVA for variable screening
Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory
More informationOutline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:
Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A
More informationOutline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1
4/14/011 Outlne Dscrmnatve classfers for mage recognton Wednesday, Aprl 13 Krsten Grauman UT-Austn Last tme: wndow-based generc obect detecton basc ppelne face detecton wth boostng as case study Today:
More informationRange images. Range image registration. Examples of sampling patterns. Range images and range surfaces
Range mages For many structured lght scanners, the range data forms a hghly regular pattern known as a range mage. he samplng pattern s determned by the specfc scanner. Range mage regstraton 1 Examples
More informationUnsupervised Learning and Clustering
Unsupervsed Learnng and Clusterng Supervsed vs. Unsupervsed Learnng Up to now we consdered supervsed learnng scenaro, where we are gven 1. samples 1,, n 2. class labels for all samples 1,, n Ths s also
More informationThe Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique
//00 :0 AM Outlne and Readng The Greedy Method The Greedy Method Technque (secton.) Fractonal Knapsack Problem (secton..) Task Schedulng (secton..) Mnmum Spannng Trees (secton.) Change Money Problem Greedy
More informationGraph-based Clustering
Graphbased Clusterng Transform the data nto a graph representaton ertces are the data ponts to be clustered Edges are eghted based on smlarty beteen data ponts Graph parttonng Þ Each connected component
More informationUnsupervised Learning and Clustering
Unsupervsed Learnng and Clusterng Why consder unlabeled samples?. Collectng and labelng large set of samples s costly Gettng recorded speech s free, labelng s tme consumng 2. Classfer could be desgned
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15
CS434a/541a: Pattern Recognton Prof. Olga Veksler Lecture 15 Today New Topc: Unsupervsed Learnng Supervsed vs. unsupervsed learnng Unsupervsed learnng Net Tme: parametrc unsupervsed learnng Today: nonparametrc
More informationHierarchical agglomerative. Cluster Analysis. Christine Siedle Clustering 1
Herarchcal agglomeratve Cluster Analyss Chrstne Sedle 19-3-2004 Clusterng 1 Classfcaton Basc (unconscous & conscous) human strategy to reduce complexty Always based Cluster analyss to fnd or confrm types
More informationOutline. Type of Machine Learning. Examples of Application. Unsupervised Learning
Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton
More informationUnsupervised Learning
Pattern Recognton Lecture 8 Outlne Introducton Unsupervsed Learnng Parametrc VS Non-Parametrc Approach Mxture of Denstes Maxmum-Lkelhood Estmates Clusterng Prof. Danel Yeung School of Computer Scence and
More informationCS 534: Computer Vision Model Fitting
CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust
More informationLearning the Kernel Parameters in Kernel Minimum Distance Classifier
Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department
More informationBayesian Networks: Independencies and Inference. What Independencies does a Bayes Net Model?
Bayesan Networks: Indeendences and Inference Scott Daves and Andrew Moore Note to other teachers and users of these sldes. Andrew and Scott would be delghted f you found ths source materal useful n gvng
More informationClustering and Dimensionality Reduction
Clustering and Dimensionality Reduction Some material on these is slides borrowed from Andrew Moore's excellent machine learning tutorials located at: Data Mining Automatically extracting meaning from
More informationAn Optimal Algorithm for Prufer Codes *
J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,
More informationVirtual Memory. Background. No. 10. Virtual Memory: concept. Logical Memory Space (review) Demand Paging(1) Virtual Memory
Background EECS. Operatng System Fundamentals No. Vrtual Memory Prof. Hu Jang Department of Electrcal Engneerng and Computer Scence, York Unversty Memory-management methods normally requres the entre process
More informationModule Management Tool in Software Development Organizations
Journal of Computer Scence (5): 8-, 7 ISSN 59-66 7 Scence Publcatons Management Tool n Software Development Organzatons Ahmad A. Al-Rababah and Mohammad A. Al-Rababah Faculty of IT, Al-Ahlyyah Amman Unversty,
More informationCompiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz
Compler Desgn Sprng 2014 Regster Allocaton Sample Exercses and Solutons Prof. Pedro C. Dnz USC / Informaton Scences Insttute 4676 Admralty Way, Sute 1001 Marna del Rey, Calforna 90292 pedro@s.edu Regster
More informationSubspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;
Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features
More informationSIGGRAPH Interactive Image Cutout. Interactive Graph Cut. Interactive Graph Cut. Interactive Graph Cut. Hard Constraints. Lazy Snapping.
SIGGRAPH 004 Interactve Image Cutout Lazy Snappng Yn L Jan Sun Ch-Keung Tang Heung-Yeung Shum Mcrosoft Research Asa Hong Kong Unversty Separate an object from ts background Compose the object on another
More informationKent State University CS 4/ Design and Analysis of Algorithms. Dept. of Math & Computer Science LECT-16. Dynamic Programming
CS 4/560 Desgn and Analyss of Algorthms Kent State Unversty Dept. of Math & Computer Scence LECT-6 Dynamc Programmng 2 Dynamc Programmng Dynamc Programmng, lke the dvde-and-conquer method, solves problems
More informationSupport Vector Machines. CS534 - Machine Learning
Support Vector Machnes CS534 - Machne Learnng Perceptron Revsted: Lnear Separators Bnar classfcaton can be veed as the task of separatng classes n feature space: b > 0 b 0 b < 0 f() sgn( b) Lnear Separators
More informationNUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS
ARPN Journal of Engneerng and Appled Scences 006-017 Asan Research Publshng Network (ARPN). All rghts reserved. NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS Igor Grgoryev, Svetlana
More informationAMath 483/583 Lecture 21 May 13, Notes: Notes: Jacobi iteration. Notes: Jacobi with OpenMP coarse grain
AMath 483/583 Lecture 21 May 13, 2011 Today: OpenMP and MPI versons of Jacob teraton Gauss-Sedel and SOR teratve methods Next week: More MPI Debuggng and totalvew GPU computng Read: Class notes and references
More informationSupport Vector Machines
/9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.
More informationAssignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009.
Farrukh Jabeen Algorthms 51 Assgnment #2 Due Date: June 15, 29. Assgnment # 2 Chapter 3 Dscrete Fourer Transforms Implement the FFT for the DFT. Descrbed n sectons 3.1 and 3.2. Delverables: 1. Concse descrpton
More informationActive Contours/Snakes
Actve Contours/Snakes Erkut Erdem Acknowledgement: The sldes are adapted from the sldes prepared by K. Grauman of Unversty of Texas at Austn Fttng: Edges vs. boundares Edges useful sgnal to ndcate occludng
More informationFitting: Deformable contours April 26 th, 2018
4/6/08 Fttng: Deformable contours Aprl 6 th, 08 Yong Jae Lee UC Davs Recap so far: Groupng and Fttng Goal: move from array of pxel values (or flter outputs) to a collecton of regons, objects, and shapes.
More informationParallelism for Nested Loops with Non-uniform and Flow Dependences
Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr
More informationEXTENDED BIC CRITERION FOR MODEL SELECTION
IDIAP RESEARCH REPORT EXTEDED BIC CRITERIO FOR ODEL SELECTIO Itshak Lapdot Andrew orrs IDIAP-RR-0-4 Dalle olle Insttute for Perceptual Artfcal Intellgence P.O.Box 59 artgny Valas Swtzerland phone +4 7
More informationLOOP ANALYSIS. The second systematic technique to determine all currents and voltages in a circuit
LOOP ANALYSS The second systematic technique to determine all currents and voltages in a circuit T S DUAL TO NODE ANALYSS - T FRST DETERMNES ALL CURRENTS N A CRCUT AND THEN T USES OHM S LAW TO COMPUTE
More informationAlgorithm To Convert A Decimal To A Fraction
Algorthm To Convert A ecmal To A Fracton by John Kennedy Mathematcs epartment Santa Monca College 1900 Pco Blvd. Santa Monca, CA 90405 jrkennedy6@gmal.com Except for ths comment explanng that t s blank
More information6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour
6.854 Advanced Algorthms Petar Maymounkov Problem Set 11 (November 23, 2005) Wth: Benjamn Rossman, Oren Wemann, and Pouya Kheradpour Problem 1. We reduce vertex cover to MAX-SAT wth weghts, such that the
More informationLecture #15 Lecture Notes
Lecture #15 Lecture Notes The ocean water column s very much a 3-D spatal entt and we need to represent that structure n an economcal way to deal wth t n calculatons. We wll dscuss one way to do so, emprcal
More informationCMPS 10 Introduction to Computer Science Lecture Notes
CPS 0 Introducton to Computer Scence Lecture Notes Chapter : Algorthm Desgn How should we present algorthms? Natural languages lke Englsh, Spansh, or French whch are rch n nterpretaton and meanng are not
More informationSCALABLE AND VISUALIZATION-ORIENTED CLUSTERING FOR EXPLORATORY SPATIAL ANALYSIS
SCALABLE AND VISUALIZATION-ORIENTED CLUSTERING FOR EXPLORATORY SPATIAL ANALYSIS J.H.Guan, F.B.Zhu, F.L.Ban a School of Computer, Spatal Informaton & Dgtal Engneerng Center, Wuhan Unversty, Wuhan, 430079,
More informationMulti-stable Perception. Necker Cube
Mult-stable Percepton Necker Cube Spnnng dancer lluson, Nobuuk Kaahara Fttng and Algnment Computer Vson Szelsk 6.1 James Has Acknowledgment: Man sldes from Derek Hoem, Lana Lazebnk, and Grauman&Lebe 2008
More informationLecture 15: Memory Hierarchy Optimizations. I. Caches: A Quick Review II. Iteration Space & Loop Transformations III.
Lecture 15: Memory Herarchy Optmzatons I. Caches: A Quck Revew II. Iteraton Space & Loop Transformatons III. Types of Reuse ALSU 7.4.2-7.4.3, 11.2-11.5.1 15-745: Memory Herarchy Optmzatons Phllp B. Gbbons
More informationCSCI 104 Sorting Algorithms. Mark Redekopp David Kempe
CSCI 104 Sortng Algorthms Mark Redekopp Davd Kempe Algorthm Effcency SORTING 2 Sortng If we have an unordered lst, sequental search becomes our only choce If we wll perform a lot of searches t may be benefcal
More informationSolving two-person zero-sum game by Matlab
Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by
More informationBrave New World Pseudocode Reference
Brave New World Pseudocode Reference Pseudocode s a way to descrbe how to accomplsh tasks usng basc steps lke those a computer mght perform. In ths week s lab, you'll see how a form of pseudocode can be
More informationSLAM Summer School 2006 Practical 2: SLAM using Monocular Vision
SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,
More informationProblem Set 3 Solutions
Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n,
More information12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification
Introducton to Artfcal Intellgence V22.0472-001 Fall 2009 Lecture 24: Nearest-Neghbors & Support Vector Machnes Rob Fergus Dept of Computer Scence, Courant Insttute, NYU Sldes from Danel Yeung, John DeNero
More informationCSE 326: Data Structures Quicksort Comparison Sorting Bound
CSE 326: Data Structures Qucksort Comparson Sortng Bound Steve Setz Wnter 2009 Qucksort Qucksort uses a dvde and conquer strategy, but does not requre the O(N) extra space that MergeSort does. Here s the
More informationProgramming in Fortran 90 : 2017/2018
Programmng n Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Exercse 1 : Evaluaton of functon dependng on nput Wrte a program who evaluate the functon f (x,y) for any two user specfed values
More informationPerformance Evaluation of Information Retrieval Systems
Why System Evaluaton? Performance Evaluaton of Informaton Retreval Systems Many sldes n ths secton are adapted from Prof. Joydeep Ghosh (UT ECE) who n turn adapted them from Prof. Dk Lee (Unv. of Scence
More informationOPTIMAL CONFIGURATION FOR NODES IN MIXED CELLULAR AND MOBILE AD HOC NETWORK FOR INET
OPTIMAL CONFIGURATION FOR NODE IN MIED CELLULAR AND MOBILE AD HOC NETWORK FOR INET Olusola Babalola D.E. Department of Electrcal and Computer Engneerng Morgan tate Unversty Dr. Rchard Dean Faculty Advsor
More information2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements
Module 3: Element Propertes Lecture : Lagrange and Serendpty Elements 5 In last lecture note, the nterpolaton functons are derved on the bass of assumed polynomal from Pascal s trangle for the fled varable.
More informationImproving Low Density Parity Check Codes Over the Erasure Channel. The Nelder Mead Downhill Simplex Method. Scott Stransky
Improvng Low Densty Party Check Codes Over the Erasure Channel The Nelder Mead Downhll Smplex Method Scott Stransky Programmng n conjuncton wth: Bors Cukalovc 18.413 Fnal Project Sprng 2004 Page 1 Abstract
More informationParallel Numerics. 1 Preconditioning & Iterative Solvers (From 2016)
Technsche Unverstät München WSe 6/7 Insttut für Informatk Prof. Dr. Thomas Huckle Dpl.-Math. Benjamn Uekermann Parallel Numercs Exercse : Prevous Exam Questons Precondtonng & Iteratve Solvers (From 6)
More informationA Deflected Grid-based Algorithm for Clustering Analysis
A Deflected Grd-based Algorthm for Clusterng Analyss NANCY P. LIN, CHUNG-I CHANG, HAO-EN CHUEH, HUNG-JEN CHEN, WEI-HUA HAO Department of Computer Scence and Informaton Engneerng Tamkang Unversty 5 Yng-chuan
More informationGSLM Operations Research II Fall 13/14
GSLM 58 Operatons Research II Fall /4 6. Separable Programmng Consder a general NLP mn f(x) s.t. g j (x) b j j =. m. Defnton 6.. The NLP s a separable program f ts objectve functon and all constrants are
More informationPositive Semi-definite Programming Localization in Wireless Sensor Networks
Postve Sem-defnte Programmng Localzaton n Wreless Sensor etworks Shengdong Xe 1,, Jn Wang, Aqun Hu 1, Yunl Gu, Jang Xu, 1 School of Informaton Scence and Engneerng, Southeast Unversty, 10096, anjng Computer
More informationClassifier Selection Based on Data Complexity Measures *
Classfer Selecton Based on Data Complexty Measures * Edth Hernández-Reyes, J.A. Carrasco-Ochoa, and J.Fco. Martínez-Trndad Natonal Insttute for Astrophyscs, Optcs and Electroncs, Lus Enrque Erro No.1 Sta.
More informationEfficient Distributed File System (EDFS)
Effcent Dstrbuted Fle System (EDFS) (Sem-Centralzed) Debessay(Debsh) Fesehaye, Rahul Malk & Klara Naherstedt Unversty of Illnos-Urbana Champagn Contents Problem Statement, Related Work, EDFS Desgn Rate
More informationCourse Introduction. Algorithm 8/31/2017. COSC 320 Advanced Data Structures and Algorithms. COSC 320 Advanced Data Structures and Algorithms
Course Introducton Course Topcs Exams, abs, Proects A quc loo at a few algorthms 1 Advanced Data Structures and Algorthms Descrpton: We are gong to dscuss algorthm complexty analyss, algorthm desgn technques
More informationEnhanced AMBTC for Image Compression using Block Classification and Interpolation
Internatonal Journal of Computer Applcatons (0975 8887) Volume 5 No.0, August 0 Enhanced AMBTC for Image Compresson usng Block Classfcaton and Interpolaton S. Vmala Dept. of Comp. Scence Mother Teresa
More informationA mathematical programming approach to the analysis, design and scheduling of offshore oilfields
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 A mathematcal programmng approach to the analyss, desgn and
More informationFitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.
Fttng & Matchng Lecture 4 Prof. Bregler Sldes from: S. Lazebnk, S. Setz, M. Pollefeys, A. Effros. How do we buld panorama? We need to match (algn) mages Matchng wth Features Detect feature ponts n both
More informationComputer Animation and Visualisation. Lecture 4. Rigging / Skinning
Computer Anmaton and Vsualsaton Lecture 4. Rggng / Sknnng Taku Komura Overvew Sknnng / Rggng Background knowledge Lnear Blendng How to decde weghts? Example-based Method Anatomcal models Sknnng Assume
More informationMotivation. EE 457 Unit 4. Throughput vs. Latency. Performance Depends on View Point?! Computer System Performance. An individual user wants to:
4.1 4.2 Motvaton EE 457 Unt 4 Computer System Performance An ndvdual user wants to: Mnmze sngle program executon tme A datacenter owner wants to: Maxmze number of Mnmze ( ) http://e-tellgentnternetmarketng.com/webste/frustrated-computer-user-2/
More informationAll-Pairs Shortest Paths. Approximate All-Pairs shortest paths Approximate distance oracles Spanners and Emulators. Uri Zwick Tel Aviv University
Approxmate All-Pars shortest paths Approxmate dstance oracles Spanners and Emulators Ur Zwck Tel Avv Unversty Summer School on Shortest Paths (PATH05 DIKU, Unversty of Copenhagen All-Pars Shortest Paths
More informationLobachevsky State University of Nizhni Novgorod. Polyhedron. Quick Start Guide
Lobachevsky State Unversty of Nzhn Novgorod Polyhedron Quck Start Gude Nzhn Novgorod 2016 Contents Specfcaton of Polyhedron software... 3 Theoretcal background... 4 1. Interface of Polyhedron... 6 1.1.
More informationCluster Analysis of Electrical Behavior
Journal of Computer and Communcatons, 205, 3, 88-93 Publshed Onlne May 205 n ScRes. http://www.scrp.org/ournal/cc http://dx.do.org/0.4236/cc.205.350 Cluster Analyss of Electrcal Behavor Ln Lu Ln Lu, School
More informationA PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION
1 THE PUBLISHING HOUSE PROCEEDINGS OF THE ROMANIAN ACADEMY, Seres A, OF THE ROMANIAN ACADEMY Volume 4, Number 2/2003, pp.000-000 A PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION Tudor BARBU Insttute
More informationClustering. A. Bellaachia Page: 1
Clusterng. Obectves.. Clusterng.... Defntons... General Applcatons.3. What s a good clusterng?. 3.4. Requrements 3 3. Data Structures 4 4. Smlarty Measures. 4 4.. Standardze data.. 5 4.. Bnary varables..
More informationAnnouncements. Supervised Learning
Announcements See Chapter 5 of Duda, Hart, and Stork. Tutoral by Burge lnked to on web page. Supervsed Learnng Classfcaton wth labeled eamples. Images vectors n hgh-d space. Supervsed Learnng Labeled eamples
More informationNews. Recap: While Loop Example. Reading. Recap: Do Loop Example. Recap: For Loop Example
Unversty of Brtsh Columba CPSC, Intro to Computaton Jan-Apr Tamara Munzner News Assgnment correctons to ASCIIArtste.java posted defntely read WebCT bboards Arrays Lecture, Tue Feb based on sldes by Kurt
More informationAPPLIED MACHINE LEARNING
Methods for Clusterng K-means, Soft K-means DBSCAN 1 Objectves Learn basc technques for data clusterng K-means and soft K-means, GMM (next lecture) DBSCAN Understand the ssues and major challenges n clusterng
More informationELEC 377 Operating Systems. Week 6 Class 3
ELEC 377 Operatng Systems Week 6 Class 3 Last Class Memory Management Memory Pagng Pagng Structure ELEC 377 Operatng Systems Today Pagng Szes Vrtual Memory Concept Demand Pagng ELEC 377 Operatng Systems
More informationSteps for Computing the Dissimilarity, Entropy, Herfindahl-Hirschman and. Accessibility (Gravity with Competition) Indices
Steps for Computng the Dssmlarty, Entropy, Herfndahl-Hrschman and Accessblty (Gravty wth Competton) Indces I. Dssmlarty Index Measurement: The followng formula can be used to measure the evenness between
More informationComplex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following.
Complex Numbers The last topc n ths secton s not really related to most of what we ve done n ths chapter, although t s somewhat related to the radcals secton as we wll see. We also won t need the materal
More informationIntro. Iterators. 1. Access
Intro Ths mornng I d lke to talk a lttle bt about s and s. We wll start out wth smlartes and dfferences, then we wll see how to draw them n envronment dagrams, and we wll fnsh wth some examples. Happy
More informationDESIGNING TRANSMISSION SCHEDULES FOR WIRELESS AD HOC NETWORKS TO MAXIMIZE NETWORK THROUGHPUT
DESIGNING TRANSMISSION SCHEDULES FOR WIRELESS AD HOC NETWORKS TO MAXIMIZE NETWORK THROUGHPUT Bran J. Wolf, Joseph L. Hammond, and Harlan B. Russell Dept. of Electrcal and Computer Engneerng, Clemson Unversty,
More information5 The Primal-Dual Method
5 The Prmal-Dual Method Orgnally desgned as a method for solvng lnear programs, where t reduces weghted optmzaton problems to smpler combnatoral ones, the prmal-dual method (PDM) has receved much attenton
More informationNAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics
Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson
More informationExercises (Part 4) Introduction to R UCLA/CCPR. John Fox, February 2005
Exercses (Part 4) Introducton to R UCLA/CCPR John Fox, February 2005 1. A challengng problem: Iterated weghted least squares (IWLS) s a standard method of fttng generalzed lnear models to data. As descrbed
More informationA Clustering Algorithm for Chinese Adjectives and Nouns 1
Clusterng lgorthm for Chnese dectves and ouns Yang Wen, Chunfa Yuan, Changnng Huang 2 State Key aboratory of Intellgent Technology and System Deptartment of Computer Scence & Technology, Tsnghua Unversty,
More informationOn Some Entertaining Applications of the Concept of Set in Computer Science Course
On Some Entertanng Applcatons of the Concept of Set n Computer Scence Course Krasmr Yordzhev *, Hrstna Kostadnova ** * Assocate Professor Krasmr Yordzhev, Ph.D., Faculty of Mathematcs and Natural Scences,
More informationLEAST SQUARES. RANSAC. HOUGH TRANSFORM.
LEAS SQUARES. RANSAC. HOUGH RANSFORM. he sldes are from several sources through James Has (Brown); Srnvasa Narasmhan (CMU); Slvo Savarese (U. of Mchgan); Bll Freeman and Antono orralba (MI), ncludng ther
More informationBiostatistics 615/815
The E-M Algorthm Bostatstcs 615/815 Lecture 17 Last Lecture: The Smplex Method General method for optmzaton Makes few assumptons about functon Crawls towards mnmum Some recommendatons Multple startng ponts
More informationcos(a, b) = at b a b. To get a distance measure, subtract the cosine similarity from one. dist(a, b) =1 cos(a, b)
8 Clusterng 8.1 Some Clusterng Examples Clusterng comes up n many contexts. For example, one mght want to cluster journal artcles nto clusters of artcles on related topcs. In dong ths, one frst represents
More informationOptimizing Document Scoring for Query Retrieval
Optmzng Document Scorng for Query Retreval Brent Ellwen baellwe@cs.stanford.edu Abstract The goal of ths project was to automate the process of tunng a document query engne. Specfcally, I used machne learnng
More informationR s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes
SPH3UW Unt 7.3 Sphercal Concave Mrrors Page 1 of 1 Notes Physcs Tool box Concave Mrror If the reflectng surface takes place on the nner surface of the sphercal shape so that the centre of the mrror bulges
More informationInsertion Sort. Divide and Conquer Sorting. Divide and Conquer. Mergesort. Mergesort Example. Auxiliary Array
Inserton Sort Dvde and Conquer Sortng CSE 6 Data Structures Lecture 18 What f frst k elements of array are already sorted? 4, 7, 1, 5, 1, 16 We can shft the tal of the sorted elements lst down and then
More informationLoad Balancing for Hex-Cell Interconnection Network
Int. J. Communcatons, Network and System Scences,,, - Publshed Onlne Aprl n ScRes. http://www.scrp.org/journal/jcns http://dx.do.org/./jcns.. Load Balancng for Hex-Cell Interconnecton Network Saher Manaseer,
More informationAn Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices
Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal
More informationA Simple Methodology for Database Clustering. Hao Tang 12 Guangdong University of Technology, Guangdong, , China
for Database Clusterng Guangdong Unversty of Technology, Guangdong, 0503, Chna E-mal: 6085@qq.com Me Zhang Guangdong Unversty of Technology, Guangdong, 0503, Chna E-mal:64605455@qq.com Database clusterng
More informationThis excerpt from. Foundations of Statistical Natural Language Processing. Christopher D. Manning and Hinrich Schütze The MIT Press.
Ths excerpt from Foundatons of Statstcal Natural Language Processng. Chrstopher D. Mannng and Hnrch Schütze. 1999 The MIT Press. s provded n screen-vewable form for personal use only by members of MIT
More informationCSE 326: Data Structures Quicksort Comparison Sorting Bound
CSE 326: Data Structures Qucksort Comparson Sortng Bound Bran Curless Sprng 2008 Announcements (5/14/08) Homework due at begnnng of class on Frday. Secton tomorrow: Graded homeworks returned More dscusson
More informationAn Iterative Solution Approach to Process Plant Layout using Mixed Integer Optimisation
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 An Iteratve Soluton Approach to Process Plant Layout usng Mxed
More informationSurvey of Cluster Analysis and its Various Aspects
Harmnder Kaur et al, Internatonal Journal of Computer Scence and Moble Computng, Vol.4 Issue.0, October- 05, pg. 353-363 Avalable Onlne at www.csmc.com Internatonal Journal of Computer Scence and Moble
More information