Multi-Modal Metropolis Nested Sampling For Inspiralling Binaries
|
|
- Clifton Small
- 6 years ago
- Views:
Transcription
1 Multi-Modal Metropolis Nested Sampling For Inspiralling Binaries Ed Porter (AEI) & Jon Gair (IOA) Workshop AEI September 2008 (We acknowledge useful converstions with F. Feroz and M. Hobson (Cavendish Laboratory, Cambridge) regarding the MultiNest algorithm (arxiv:0704:3704, arxiv:0810:0781)) 1
2 Outline 1 Motivation : SMBHBs, multiple sources, multiple mode solutions... 2 Evolutionary Algorithms : Concepts, variants... 3 The Algorithm : the nested sampling, the clustering, x-means, k-means... 4 Current Results : Yes, we have results 5 Future Plans : EMRIs, spinning SMBHBs, smarter algorithms... 2
3 1 Motivation 3
4 Motivation & Terminology If we have two events A and B, we define P(A) = prior or marginal probability of A P(A B) = conditional or posterior probability of A given B Bayes Theorem : P(A B) = P(B A)P(A) / P(B) where P(B) acts as a normalising constant. Can also write Bayes theorem as Posterior = Likelihood x Prior Probability / Evidence Problem is P(B) is always very difficult to calculate. 4
5 SMBHBs 9 Parameter Set Can use F-Statistic to maximize over Non-spinning No extra harmonics Already have algorithm which uses MHMC algorithm which works very efficiently 5
6 LISA Response To A Source 6
7 Finding the Modes of the Solution Sky degeneracy From previous Metropolis-Hastings algorithms we can find either the primary or the antipodal solution, but not both at the same time : Cornish & Porter, CQG 24, 5729 (2007) 7
8 System Knowledge Main acceleration can come from knowledge of the system in question : e.g. degeneracies, symmetries etc Cornish & Porter, CQG 24, 5729 (2007) 8
9 2 Evolutionary Algorithms 9
10 What are they? Origins in artificial intelligence Uses a population based optimization algorithm Inspired by biological evolution : reproduction, death, mutation etc Candidate solutions play the role of organisms Fitness criteria determine the environment 10
11 Metropolis-Hastings Ratio Priors 11
12 Metropolis-Hastings Ratio Likelihoods 12
13 Metropolis-Hastings Ratio Transition Probability 13
14 Should I stay or should I go? First calculate then generate a = U(0,1) and move with probability a = min{1, H}. 14
15 Metropolis Sampling Developed in 1953 for Boltzmann distributions. Requires symmetric proposals i.e. q(x y) = q(y x) Now only likelihood ratio is important. Can also preserve detailed balance i.e. p(s x)q(y x)=p(s y)q(x y) 15
16 Metropolis-Hastings Sampling Updated in 1970 to include non-symmetric proposal distributions. Pros : 1) Faster mixing of chains. 2) Can use multiple proposal distributions which only depend on current state. 3) Just requires that a function proportional to a density can be calculated at a point in the parameter space. This allows us to generate a sample without knowing the normalising constant or the evidence. Cons : 1) Multiple proposal distributions require some knowledge of the system. 2) Can still get stuck on secondary solutions. 16
17 Simulated Annealing Suppose we want to sample from a distribution of the form P(x) = exp(- L(x) / 2). It is usually easier to sample instead from the distribution P*(x) = exp(- L(x) / kt) where k = 2 and we gradually cool from T 1 17
18 Simulated Annealing Pros : 1) Can prevent chains from getting stuck on secondaries. 2) Shortens and fattens high probability regions. 3) Can greatly speed up burn-in. Cons : 1) No information on initial temperature a priori (can get around this using thermostated annealing, see Cornish & Porter, Class.Quant.Grav.24: 5729,2007) 2) Cool down needs to be slow. 3) If cool down is too fast, the chain WILL get stuck. 4) Makes chain non-markovian (not really a Con for practical purposes). 18
19 Nested Sampling (Skilling 2004) Method to evaluate evidence Climb likelihood surface by passing through nested equi-likelihood contours Shrink prior volume by rejecting lowest likelihood point Always search for higher likelihood point within lowest equi-likelihood contour Can also return PDFs 19
20 BIC Scoring Statistical criterion for model selection Also called the Schwarz criterion or SIC If n = sample size, k = # of free parameters and L = maximized likelihood for the estimated model, then Properties BIC = -2 ln L + k ln(n) (a) Independent of priors (b) can measure efficiency of the parameterized model (c) penalizes complexity (d) Good for clustering 20
21 k-means Clustering (Duda & Hart 1973, Bishop 1995) k stands for number of clusters Goal : given n points, separate into k clusters Starting centroids chosen at random Can converge to the local minimum of a distortion measure Can be quite slow Issues with choosing value of k 21
22 x-means Clustering (Pelleg & Moore 1999) Define a k(min) and a k(max) Step 1 : Improve parameters (i.e. do k-means for a cluster) Step 2 : Improve structure (if and where new centroids are needed) 22
23 4 The Algorithm 23
24 Initial Population Selection Choose fitness threshold, e.g. SNR = 15 Generate organisms until N = 100 Allow organisms to improve their fitness using an uphill climber i.e. Deterministic search where higher likelihood points are accepted without question Generate k centroids according to x-means clustering and associate each of the the organisms with a centroid Centroids with highest BIC scores survive, while unfit centroids are killed off. 24
25 Population Evolution Population evolves according to a Swarm Intelligence Model i.e. allow organisms in a swarm to evolve on their own, but track center of mass of the swarm Every 20 iterations, recalculate the position and number of centroids, as swarms can break up Re-allocate the organisms with centroids, such that centroids and centroid number evolves along with the children The goal is that the organisms will evolve towards the different modes of a solution And while the organisms do all the work, it is the centroids that we are most interested in as they define the global fitness of a cluster 25
26 Moving The Clusters Nested Sampling : Try and replace ln L by jumping within a 1-sigma range Uphill Climber : Try proposed point and accept if ln L is better Metropolis : Uniform proposal distribution Metropolis-Hastings : Include non-uniform proposal distributions 26
27 4 Current Results 27
28 Binary Parameters 2 Sources 1) m1 = 1e7, m2 = 1e6, z=1, tc = 0.9 yrs, SNR ~ 200 2) m1 = 4e6, m2 = 1e6, z=1.5, tc = 1.02 yrs, SNR ~ 50 Assume Observation time of 1 year Low frequency approximation for LISA response 28
29 Single Source SMBHB Initial SA heat = 100, TA threshold set to SNR = 10 Used prior information to cluster in sky position Worked very well, clusters found both real and antipodal solutions Used F-Statistic Fitness threshold for initial population set at SNR of 5 No time of coalescence maximization Small number of organisms needed, ~20-30 Code took 3 hours to run on a dual Xeon processor desktop 29
30 30
31 31
32 32
33 33
34 Double Source SMBHB Same initial SA and TA conditions Initial fitness threshold set to SNR of 15 While sky clustering worked quite well, needed more time of coalescence and sky clustering worked better. More organisms needed (~100) to account for multiple modes Code takes ~24 hours to run Set k(min) = 2 and k(max) = 10 34
35 35
36 36
37 37
38 38
39 39
40 40
41 41
42 42
43 43
44 5 For the future... 44
45 Things To Try... Birth and death of weak organisms/clusters Growing/pruning of clusters to constant population Ant colony optimization More efficient clustering Using cluster to approximate covariance matrix, obtain size and direction of moves Fisher matrix less algorithm? Cross cluster learning 45
46 Sources To Try... 1) Spinning black hole binaries / EMRIs 2) Will be slower, will need more organisms 3) Can handle degenerate parameters very well 4) May be successful as we are not relying on a single chain which can get stuck 5) Method is designed to not only find the primary modes, but also the secondaries 46
47 Conclusion Entirely new algorithm for non-spinning SMBHBs Based on an Evolutionary Algorithm Not only finds multiple mode solutions but also maps PDFs Can also return the evidence Works very well for multiple sources Scaling time compared to MHMC algorithm is comparable Application to EMRIs and spinning black holes 47
Overview. Monte Carlo Methods. Statistics & Bayesian Inference Lecture 3. Situation At End Of Last Week
Statistics & Bayesian Inference Lecture 3 Joe Zuntz Overview Overview & Motivation Metropolis Hastings Monte Carlo Methods Importance sampling Direct sampling Gibbs sampling Monte-Carlo Markov Chains Emcee
More informationArtificial Intelligence
Artificial Intelligence Informed Search and Exploration Chapter 4 (4.3 4.6) Searching: So Far We ve discussed how to build goal-based and utility-based agents that search to solve problems We ve also presented
More informationNon-deterministic Search techniques. Emma Hart
Non-deterministic Search techniques Emma Hart Why do local search? Many real problems are too hard to solve with exact (deterministic) techniques Modern, non-deterministic techniques offer ways of getting
More informationLecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman
Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing non-convex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization
More informationIntroduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.
Chapter 7: Derivative-Free Optimization Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.5) Jyh-Shing Roger Jang et al.,
More informationMCMC Methods for data modeling
MCMC Methods for data modeling Kenneth Scerri Department of Automatic Control and Systems Engineering Introduction 1. Symposium on Data Modelling 2. Outline: a. Definition and uses of MCMC b. MCMC algorithms
More informationEscaping Local Optima: Genetic Algorithm
Artificial Intelligence Escaping Local Optima: Genetic Algorithm Dae-Won Kim School of Computer Science & Engineering Chung-Ang University We re trying to escape local optima To achieve this, we have learned
More informationMSA101/MVE Lecture 5
MSA101/MVE187 2017 Lecture 5 Petter Mostad Chalmers University September 12, 2017 1 / 15 Importance sampling MC integration computes h(x)f (x) dx where f (x) is a probability density function, by simulating
More informationNested Sampling: Introduction and Implementation
UNIVERSITY OF TEXAS AT SAN ANTONIO Nested Sampling: Introduction and Implementation Liang Jing May 2009 1 1 ABSTRACT Nested Sampling is a new technique to calculate the evidence, Z = P(D M) = p(d θ, M)p(θ
More informationBayesian Methods in Vision: MAP Estimation, MRFs, Optimization
Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization CS 650: Computer Vision Bryan S. Morse Optimization Approaches to Vision / Image Processing Recurring theme: Cast vision problem as an optimization
More informationIntroduction to Optimization Using Metaheuristics. The Lecturer: Thomas Stidsen. Outline. Name: Thomas Stidsen: Nationality: Danish.
The Lecturer: Thomas Stidsen Name: Thomas Stidsen: tks@imm.dtu.dk Outline Nationality: Danish. General course information Languages: Danish and English. Motivation, modelling and solving Education: Ph.D.
More informationChapter 14 Global Search Algorithms
Chapter 14 Global Search Algorithms An Introduction to Optimization Spring, 2015 Wei-Ta Chu 1 Introduction We discuss various search methods that attempts to search throughout the entire feasible set.
More informationMAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS
In: Journal of Applied Statistical Science Volume 18, Number 3, pp. 1 7 ISSN: 1067-5817 c 2011 Nova Science Publishers, Inc. MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS Füsun Akman
More informationIntroduction to Optimization Using Metaheuristics. Thomas J. K. Stidsen
Introduction to Optimization Using Metaheuristics Thomas J. K. Stidsen Outline General course information Motivation, modelling and solving Hill climbers Simulated Annealing 1 Large-Scale Optimization
More informationLevel-set MCMC Curve Sampling and Geometric Conditional Simulation
Level-set MCMC Curve Sampling and Geometric Conditional Simulation Ayres Fan John W. Fisher III Alan S. Willsky February 16, 2007 Outline 1. Overview 2. Curve evolution 3. Markov chain Monte Carlo 4. Curve
More informationCS 331: Artificial Intelligence Local Search 1. Tough real-world problems
CS 331: Artificial Intelligence Local Search 1 1 Tough real-world problems Suppose you had to solve VLSI layout problems (minimize distance between components, unused space, etc.) Or schedule airlines
More informationAn Introduction to Markov Chain Monte Carlo
An Introduction to Markov Chain Monte Carlo Markov Chain Monte Carlo (MCMC) refers to a suite of processes for simulating a posterior distribution based on a random (ie. monte carlo) process. In other
More informationSimulated Annealing. G5BAIM: Artificial Intelligence Methods. Graham Kendall. 15 Feb 09 1
G5BAIM: Artificial Intelligence Methods Graham Kendall 15 Feb 09 1 G5BAIM Artificial Intelligence Methods Graham Kendall Simulated Annealing Simulated Annealing Motivated by the physical annealing process
More informationGlobal Optimization. for practical engineering applications. Harry Lee 4/9/2018 CEE 696
Global Optimization for practical engineering applications Harry Lee 4/9/2018 CEE 696 Table of contents 1. Global Optimization 1 Global Optimization Global optimization Figure 1: Fig 2.2 from Nocedal &
More informationEvolutionary Computation Algorithms for Cryptanalysis: A Study
Evolutionary Computation Algorithms for Cryptanalysis: A Study Poonam Garg Information Technology and Management Dept. Institute of Management Technology Ghaziabad, India pgarg@imt.edu Abstract The cryptanalysis
More informationMarkov chain Monte Carlo methods
Markov chain Monte Carlo methods (supplementary material) see also the applet http://www.lbreyer.com/classic.html February 9 6 Independent Hastings Metropolis Sampler Outline Independent Hastings Metropolis
More informationAlgorithms & Complexity
Algorithms & Complexity Nicolas Stroppa - nstroppa@computing.dcu.ie CA313@Dublin City University. 2006-2007. November 21, 2006 Classification of Algorithms O(1): Run time is independent of the size of
More informationSIMULATED ANNEALING TECHNIQUES AND OVERVIEW. Daniel Kitchener Young Scholars Program Florida State University Tallahassee, Florida, USA
SIMULATED ANNEALING TECHNIQUES AND OVERVIEW Daniel Kitchener Young Scholars Program Florida State University Tallahassee, Florida, USA 1. INTRODUCTION Simulated annealing is a global optimization algorithm
More informationStatistical techniques for data analysis in Cosmology
Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction
More informationDiscussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg
Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg Phil Gregory Physics and Astronomy Univ. of British Columbia Introduction Martin Weinberg reported
More informationTheoretical Concepts of Machine Learning
Theoretical Concepts of Machine Learning Part 2 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Outline 1 Introduction 2 Generalization Error 3 Maximum Likelihood 4 Noise Models 5
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationAn evolutionary annealing-simplex algorithm for global optimisation of water resource systems
FIFTH INTERNATIONAL CONFERENCE ON HYDROINFORMATICS 1-5 July 2002, Cardiff, UK C05 - Evolutionary algorithms in hydroinformatics An evolutionary annealing-simplex algorithm for global optimisation of water
More informationMesh segmentation. Florent Lafarge Inria Sophia Antipolis - Mediterranee
Mesh segmentation Florent Lafarge Inria Sophia Antipolis - Mediterranee Outline What is mesh segmentation? M = {V,E,F} is a mesh S is either V, E or F (usually F) A Segmentation is a set of sub-meshes
More informationAlgorithm Design (4) Metaheuristics
Algorithm Design (4) Metaheuristics Takashi Chikayama School of Engineering The University of Tokyo Formalization of Constraint Optimization Minimize (or maximize) the objective function f(x 0,, x n )
More informationExpectation Maximization (EM) and Gaussian Mixture Models
Expectation Maximization (EM) and Gaussian Mixture Models Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 2 3 4 5 6 7 8 Unsupervised Learning Motivation
More informationSegmentation Computer Vision Spring 2018, Lecture 27
Segmentation http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 218, Lecture 27 Course announcements Homework 7 is due on Sunday 6 th. - Any questions about homework 7? - How many of you have
More informationClustering Relational Data using the Infinite Relational Model
Clustering Relational Data using the Infinite Relational Model Ana Daglis Supervised by: Matthew Ludkin September 4, 2015 Ana Daglis Clustering Data using the Infinite Relational Model September 4, 2015
More informationx n+1 = x n f(x n) f (x n ), (1)
1 Optimization The field of optimization is large and vastly important, with a deep history in computer science (among other places). Generally, an optimization problem is defined by having a score function
More informationMontePython. Thejs Brinckmann, Deanna C. Hooper, Julien Lesgourgues. MontePython + CLASS Kavli workshop
MontePython Thejs Brinckmann, Deanna C. Hooper, Julien Lesgourgues MontePython + CLASS Kavli workshop Code developed by Audren, Brinckmann, Lesgourgues & many others Lecture, Cambridge, 12/09/18 Overview
More informationPredicting Diabetes using Neural Networks and Randomized Optimization
Predicting Diabetes using Neural Networks and Randomized Optimization Kunal Sharma GTID: ksharma74 CS 4641 Machine Learning Abstract This paper analysis the following randomized optimization techniques
More informationMETAHEURISTICS. Introduction. Introduction. Nature of metaheuristics. Local improvement procedure. Example: objective function
Introduction METAHEURISTICS Some problems are so complicated that are not possible to solve for an optimal solution. In these problems, it is still important to find a good feasible solution close to the
More informationSimulated Annealing. Slides based on lecture by Van Larhoven
Simulated Annealing Slides based on lecture by Van Larhoven Iterative Improvement 1 General method to solve combinatorial optimization problems Principle: Start with initial configuration Repeatedly search
More informationRJaCGH, a package for analysis of
RJaCGH, a package for analysis of CGH arrays with Reversible Jump MCMC 1. CGH Arrays: Biological problem: Changes in number of DNA copies are associated to cancer activity. Microarray technology: Oscar
More informationMarkov Chain Monte Carlo (part 1)
Markov Chain Monte Carlo (part 1) Edps 590BAY Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2018 Depending on the book that you select for
More informationArtificial Intelligence
Artificial Intelligence Local Search Vibhav Gogate The University of Texas at Dallas Some material courtesy of Luke Zettlemoyer, Dan Klein, Dan Weld, Alex Ihler, Stuart Russell, Mausam Systematic Search:
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationSimulated annealing/metropolis and genetic optimization
Simulated annealing/metropolis and genetic optimization Eugeniy E. Mikhailov The College of William & Mary Lecture 18 Eugeniy Mikhailov (W&M) Practical Computing Lecture 18 1 / 8 Nature s way to find a
More informationData Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Search & Optimization Search and Optimization method deals with
More informationLecture 7: Segmentation. Thursday, Sept 20
Lecture 7: Segmentation Thursday, Sept 20 Outline Why segmentation? Gestalt properties, fun illusions and/or revealing examples Clustering Hierarchical K-means Mean Shift Graph-theoretic Normalized cuts
More informationGenetic.io. Genetic Algorithms in all their shapes and forms! Genetic.io Make something of your big data
Genetic Algorithms in all their shapes and forms! Julien Sebrien Self-taught, passion for development. Java, Cassandra, Spark, JPPF. @jsebrien, julien.sebrien@genetic.io Distribution of IT solutions (SaaS,
More informationEvolutionary Algorithms. CS Evolutionary Algorithms 1
Evolutionary Algorithms CS 478 - Evolutionary Algorithms 1 Evolutionary Computation/Algorithms Genetic Algorithms l Simulate natural evolution of structures via selection and reproduction, based on performance
More informationCHAPTER 4 GENETIC ALGORITHM
69 CHAPTER 4 GENETIC ALGORITHM 4.1 INTRODUCTION Genetic Algorithms (GAs) were first proposed by John Holland (Holland 1975) whose ideas were applied and expanded on by Goldberg (Goldberg 1989). GAs is
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture 17 EM CS/CNS/EE 155 Andreas Krause Announcements Project poster session on Thursday Dec 3, 4-6pm in Annenberg 2 nd floor atrium! Easels, poster boards and cookies
More informationMidterm Examination CS540-2: Introduction to Artificial Intelligence
Midterm Examination CS540-2: Introduction to Artificial Intelligence March 15, 2018 LAST NAME: FIRST NAME: Problem Score Max Score 1 12 2 13 3 9 4 11 5 8 6 13 7 9 8 16 9 9 Total 100 Question 1. [12] Search
More informationOptimization Techniques for Design Space Exploration
0-0-7 Optimization Techniques for Design Space Exploration Zebo Peng Embedded Systems Laboratory (ESLAB) Linköping University Outline Optimization problems in ERT system design Heuristic techniques Simulated
More informationShort-Cut MCMC: An Alternative to Adaptation
Short-Cut MCMC: An Alternative to Adaptation Radford M. Neal Dept. of Statistics and Dept. of Computer Science University of Toronto http://www.cs.utoronto.ca/ radford/ Third Workshop on Monte Carlo Methods,
More informationMarch 19, Heuristics for Optimization. Outline. Problem formulation. Genetic algorithms
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis and Dimensioning II Department of Electronics and Communications Engineering Tampere University of Technology, Tampere, Finland March 19, 2014
More informationDERIVATIVE-FREE OPTIMIZATION
DERIVATIVE-FREE OPTIMIZATION Main bibliography J.-S. Jang, C.-T. Sun and E. Mizutani. Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. Prentice Hall, New Jersey,
More informationSuppose you have a problem You don t know how to solve it What can you do? Can you use a computer to somehow find a solution for you?
Gurjit Randhawa Suppose you have a problem You don t know how to solve it What can you do? Can you use a computer to somehow find a solution for you? This would be nice! Can it be done? A blind generate
More informationIntroduction to Design Optimization: Search Methods
Introduction to Design Optimization: Search Methods 1-D Optimization The Search We don t know the curve. Given α, we can calculate f(α). By inspecting some points, we try to find the approximated shape
More informationChapter 1. Introduction
Chapter 1 Introduction A Monte Carlo method is a compuational method that uses random numbers to compute (estimate) some quantity of interest. Very often the quantity we want to compute is the mean of
More informationHierarchical Bayesian Modeling with Ensemble MCMC. Eric B. Ford (Penn State) Bayesian Computing for Astronomical Data Analysis June 12, 2014
Hierarchical Bayesian Modeling with Ensemble MCMC Eric B. Ford (Penn State) Bayesian Computing for Astronomical Data Analysis June 12, 2014 Simple Markov Chain Monte Carlo Initialise chain with θ 0 (initial
More informationClustering web search results
Clustering K-means Machine Learning CSE546 Emily Fox University of Washington November 4, 2013 1 Clustering images Set of Images [Goldberger et al.] 2 1 Clustering web search results 3 Some Data 4 2 K-means
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 2 Outline Local search techniques and optimization Hill-climbing
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 2 Outline Local search techniques and optimization Hill-climbing
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationAutomata Construct with Genetic Algorithm
Automata Construct with Genetic Algorithm Vít Fábera Department of Informatics and Telecommunication, Faculty of Transportation Sciences, Czech Technical University, Konviktská 2, Praha, Czech Republic,
More informationAutomated Video Analysis of Crowd Behavior
Automated Video Analysis of Crowd Behavior Robert Collins CSE Department Mar 30, 2009 Computational Science Seminar Series, Spring 2009. We Are... Lab for Perception, Action and Cognition Research Interest:
More informationEstimation of Item Response Models
Estimation of Item Response Models Lecture #5 ICPSR Item Response Theory Workshop Lecture #5: 1of 39 The Big Picture of Estimation ESTIMATOR = Maximum Likelihood; Mplus Any questions? answers Lecture #5:
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationQuantitative Biology II!
Quantitative Biology II! Lecture 3: Markov Chain Monte Carlo! March 9, 2015! 2! Plan for Today!! Introduction to Sampling!! Introduction to MCMC!! Metropolis Algorithm!! Metropolis-Hastings Algorithm!!
More information10. MLSP intro. (Clustering: K-means, EM, GMM, etc.)
10. MLSP intro. (Clustering: K-means, EM, GMM, etc.) Rahil Mahdian 01.04.2016 LSV Lab, Saarland University, Germany What is clustering? Clustering is the classification of objects into different groups,
More informationARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS
ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Optimisation problems Optimisation & search Two Examples The knapsack problem
More informationFall 09, Homework 5
5-38 Fall 09, Homework 5 Due: Wednesday, November 8th, beginning of the class You can work in a group of up to two people. This group does not need to be the same group as for the other homeworks. You
More informationWALLABY kinematic parametre extraction:
WALLABY kinematic parametre extraction: A sub-pipeline for 2D tilted-ring fits (ICRAR/UWA) with WALLABY kinematics working group Contents A brief recap of WALLABY rotation curve pipeline 2D tilted-ring
More informationHardware-Software Codesign
Hardware-Software Codesign 4. System Partitioning Lothar Thiele 4-1 System Design specification system synthesis estimation SW-compilation intellectual prop. code instruction set HW-synthesis intellectual
More informationGENETIC ALGORITHM with Hands-On exercise
GENETIC ALGORITHM with Hands-On exercise Adopted From Lecture by Michael Negnevitsky, Electrical Engineering & Computer Science University of Tasmania 1 Objective To understand the processes ie. GAs Basic
More informationL10. PARTICLE FILTERING CONTINUED. NA568 Mobile Robotics: Methods & Algorithms
L10. PARTICLE FILTERING CONTINUED NA568 Mobile Robotics: Methods & Algorithms Gaussian Filters The Kalman filter and its variants can only model (unimodal) Gaussian distributions Courtesy: K. Arras Motivation
More informationApproximate Bayesian Computation. Alireza Shafaei - April 2016
Approximate Bayesian Computation Alireza Shafaei - April 2016 The Problem Given a dataset, we are interested in. The Problem Given a dataset, we are interested in. The Problem Given a dataset, we are interested
More informationMachine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
More informationLuo, W., and Li, Y. (2016) Benchmarking Heuristic Search and Optimisation Algorithms in Matlab. In: 22nd International Conference on Automation and Computing (ICAC), 2016, University of Essex, Colchester,
More informationRandom Search Report An objective look at random search performance for 4 problem sets
Random Search Report An objective look at random search performance for 4 problem sets Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA dwai3@gatech.edu Abstract: This report
More information1 Methods for Posterior Simulation
1 Methods for Posterior Simulation Let p(θ y) be the posterior. simulation. Koop presents four methods for (posterior) 1. Monte Carlo integration: draw from p(θ y). 2. Gibbs sampler: sequentially drawing
More informationOptimization Methods III. The MCMC. Exercises.
Aula 8. Optimization Methods III. Exercises. 0 Optimization Methods III. The MCMC. Exercises. Anatoli Iambartsev IME-USP Aula 8. Optimization Methods III. Exercises. 1 [RC] A generic Markov chain Monte
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 Outline Local search techniques and optimization Hill-climbing
More informationEvolutionary Algorithms: Perfecting the Art of Good Enough. Liz Sander
Evolutionary Algorithms: Perfecting the Art of Good Enough Liz Sander Source: wikipedia.org Source: fishbase.org Source: youtube.com Sometimes, we can t find the best solution. Sometimes, we can t find
More informationActive contour: a parallel genetic algorithm approach
id-1 Active contour: a parallel genetic algorithm approach Florence Kussener 1 1 MathWorks, 2 rue de Paris 92196 Meudon Cedex, France Florence.Kussener@mathworks.fr Abstract This paper presents an algorithm
More informationClustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
More informationLatent Variable Models and Expectation Maximization
Latent Variable Models and Expectation Maximization Oliver Schulte - CMPT 726 Bishop PRML Ch. 9 2 4 6 8 1 12 14 16 18 2 4 6 8 1 12 14 16 18 5 1 15 2 25 5 1 15 2 25 2 4 6 8 1 12 14 2 4 6 8 1 12 14 5 1 15
More informationAn Approach to Polygonal Approximation of Digital CurvesBasedonDiscreteParticleSwarmAlgorithm
Journal of Universal Computer Science, vol. 13, no. 10 (2007), 1449-1461 submitted: 12/6/06, accepted: 24/10/06, appeared: 28/10/07 J.UCS An Approach to Polygonal Approximation of Digital CurvesBasedonDiscreteParticleSwarmAlgorithm
More informationMissing variable problems
Missing variable problems In many vision problems, if some variables were known the maximum likelihood inference problem would be easy fitting; if we knew which line each token came from, it would be easy
More informationAlso, for all analyses, two other files are produced upon program completion.
MIXOR for Windows Overview MIXOR is a program that provides estimates for mixed-effects ordinal (and binary) regression models. This model can be used for analysis of clustered or longitudinal (i.e., 2-level)
More informationPATTERN CLASSIFICATION AND SCENE ANALYSIS
PATTERN CLASSIFICATION AND SCENE ANALYSIS RICHARD O. DUDA PETER E. HART Stanford Research Institute, Menlo Park, California A WILEY-INTERSCIENCE PUBLICATION JOHN WILEY & SONS New York Chichester Brisbane
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationJednociljna i višeciljna optimizacija korištenjem HUMANT algoritma
Seminar doktoranada i poslijedoktoranada 2015. Dani FESB-a 2015., Split, 25. - 31. svibnja 2015. Jednociljna i višeciljna optimizacija korištenjem HUMANT algoritma (Single-Objective and Multi-Objective
More informationUsing a genetic algorithm for editing k-nearest neighbor classifiers
Using a genetic algorithm for editing k-nearest neighbor classifiers R. Gil-Pita 1 and X. Yao 23 1 Teoría de la Señal y Comunicaciones, Universidad de Alcalá, Madrid (SPAIN) 2 Computer Sciences Department,
More informationUsing Genetic Algorithms to optimize ACS-TSP
Using Genetic Algorithms to optimize ACS-TSP Marcin L. Pilat and Tony White School of Computer Science, Carleton University, 1125 Colonel By Drive, Ottawa, ON, K1S 5B6, Canada {mpilat,arpwhite}@scs.carleton.ca
More informationEvolutionary Methods for State-based Testing
Evolutionary Methods for State-based Testing PhD Student Raluca Lefticaru Supervised by Florentin Ipate University of Piteşti, Romania Department of Computer Science Outline Motivation Search-based software
More informationConvexization in Markov Chain Monte Carlo
in Markov Chain Monte Carlo 1 IBM T. J. Watson Yorktown Heights, NY 2 Department of Aerospace Engineering Technion, Israel August 23, 2011 Problem Statement MCMC processes in general are governed by non
More informationA Distributed Hybrid Algorithm for Composite Stock Cutting
Appl. Math. Inf. Sci. 6 No. 2S pp. 661S-667S (2012) Applied Mathematics & Information Sciences An International Journal @ 2012 NSP Natural Sciences Publishing Cor. A Distributed Hybrid Algorithm for Composite
More informationSYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationModel Selection - Which Curve?
Model Selection - Which Curve? Data Modelling There are two distinct requirements for a complete analysis. Parameter Estimation Find the parameter values that achieve closest fit to the data. Model Selection
More informationSolving Traveling Salesman Problem Using Parallel Genetic. Algorithm and Simulated Annealing
Solving Traveling Salesman Problem Using Parallel Genetic Algorithm and Simulated Annealing Fan Yang May 18, 2010 Abstract The traveling salesman problem (TSP) is to find a tour of a given number of cities
More information