Combinatorial Methods in Density Estimation

Size: px
Start display at page:

Download "Combinatorial Methods in Density Estimation"

Transcription

1 Luc Devroye Gabor Lugosi Combinatorial Methods in Density Estimation Springer

2 Contents Preface vii 1. Introduction 1 a 1.1. References 3 2. Concentration Inequalities Hoeffding's Inequality An Inequality for the Expected Maximal Deviation The Bounded Difference Inequality 7 y 2.4. Examples Bibliographic Remarks Exercises References Uniform Deviation Inequalities The Vapnik-Chervonenkis Inequality Covering Numbers and Chaining Example: The Dvoretzky-Kiefer-Wolfowitz Theorem Bibliographic Remarks Exercises References Combinatorial Tools Shatter Coefficients Vapnik-Chervonenkis Dimension and Shatter Coefficients Vapnik-Chervonenkis Dimension and Covering Numbers Examples Bibliographic Remarks Exercises References Total Variation Density Estimation The Total Variation^ Invariance Mappings Convolutions Normalization The Lebesgue Density Theorem 42

3 x/ CONTENTS 5.8. LeCam's Inequality Bibliographic Remarks Exercises References Choosing a Density Estimate Choosing Between Two Densities Examples Is the Factor of Three Necessary? Maximum Likelihood Does not Work *2 Distances Are To Be Avoided Selection from A; Densities Examples Continued Selection from an Infinite Class Bibliographic Remarks Exercises References Skeleton Estimates Kolmogorov Entropy Skeleton Estimates Robustness Finite Mixtures Monotone Densities on the Hypercube How To Make Gigantic Totally Bounded Classes Bibliographic Remarks 66 ' 7.8. Exercises References The Minimum Distance Estimate: Examples Problem Formulation Series Estimates Parametric Estimates: Exponential Families Neural Network Estimates Mixture Classes, Radial Basis Function Networks Bibliographic Remarks Exercises References The Kernel Density Estimate Approximating Functions by Convolutions Definition of the Kernel Estimate Consistency of the Kernel Estimate Concentration Choosing the Bandwidth Choosing the Kernel Rates of Convergence 85

4 CONTENTS / Xi 9.8. Uniform Rate of Convergence Shrinkage, and the Combination of Density Estimates Bibliographic Remarks Exercises References Additive Estimates and Data Splitting Data Splitting Additive Estimates Histogram Estimates Bibliographic Remarks Exercises References Bandwidth Selection for Kernel Estimates The Kernel Estimate with Riemann Kernel General Kernels, Kernel Complexity Kernel Complexity: Univariate Examples Kernel Complexity: Multivariate Kernels Asymptotic Optimality Bibliographic Remarks Exercises References Multiparameter Kernel Estimates Multivariate Kernel Estimates Product Kernels 118, Multivariate Kernel Estimates Ellipsoidal Kernels Variable Kernel Estimates Tree-Structured Partitions Changepoints and Bump Hunting Bibliographic Remarks Exercises References Wavelet Estimates Definitions Smoothing Thresholding Soft Thresholding Bibliographic Remarks Exercises References The Transformed Kernel Estimate The Transformed Kernel Estimate Box-Cox Transformations Piecewise Linear Transformations Bibliographic Remarks 148

5 xii/ CONTENTS Exercises References Minimax Theory Estimating a Density from One Data Point The General Minimax Problem Rich Classes Assouad's Lemma ^Example: The Class of Convex Densities Additional Examples Tuning the Parameters of Variable Kernel Estimates Sufficient Statistics Bibliographic Remarks Exercises References Choosing the Kernel Order Introduction Standard Kernel Estimate: Riemann Kernels Standard Kernel Estimates: General Kernels An Infinite Family of Kernels Bibliographic Remarks Exercises References Bandwidth Choice with SuperMerlieis Superkernels The Trapezoidal Kernel Bandwidth Selection Bibliographic Remarks Exercises References 196 Author Index 199 Subject Index 203

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1 Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches

More information

Stochastic Simulation: Algorithms and Analysis

Stochastic Simulation: Algorithms and Analysis Soren Asmussen Peter W. Glynn Stochastic Simulation: Algorithms and Analysis et Springer Contents Preface Notation v xii I What This Book Is About 1 1 An Illustrative Example: The Single-Server Queue 1

More information

Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221. Yixiao Sun Department of Economics, University of California, San Diego

Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221. Yixiao Sun Department of Economics, University of California, San Diego Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221 Yixiao Sun Department of Economics, University of California, San Diego Winter 2007 Contents Preface ix 1 Kernel Smoothing: Density

More information

PATTERN CLASSIFICATION AND SCENE ANALYSIS

PATTERN CLASSIFICATION AND SCENE ANALYSIS PATTERN CLASSIFICATION AND SCENE ANALYSIS RICHARD O. DUDA PETER E. HART Stanford Research Institute, Menlo Park, California A WILEY-INTERSCIENCE PUBLICATION JOHN WILEY & SONS New York Chichester Brisbane

More information

Image Segmentation. Shengnan Wang

Image Segmentation. Shengnan Wang Image Segmentation Shengnan Wang shengnan@cs.wisc.edu Contents I. Introduction to Segmentation II. Mean Shift Theory 1. What is Mean Shift? 2. Density Estimation Methods 3. Deriving the Mean Shift 4. Mean

More information

Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate

Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate Density estimation In density estimation problems, we are given a random sample from an unknown density Our objective is to estimate? Applications Classification If we estimate the density for each class,

More information

IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING

IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING SECOND EDITION IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING ith Algorithms for ENVI/IDL Morton J. Canty с*' Q\ CRC Press Taylor &. Francis Group Boca Raton London New York CRC

More information

Spline Functions on Triangulations

Spline Functions on Triangulations Spline Functions on Triangulations MING-JUN LAI AND LARRY L. SCHUMAKER CAMBRIDGE UNIVERSITY PRESS Contents Preface xi Chapter 1. Bivariate Polynomials 1.1. Introduction 1 1.2. Norms of Polynomials on Triangles

More information

w KLUWER ACADEMIC PUBLISHERS Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms Roman G. Strongin Yaroslav D.

w KLUWER ACADEMIC PUBLISHERS Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms Roman G. Strongin Yaroslav D. Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms by Roman G. Strongin Nizhni Novgorod State University, Nizhni Novgorod, Russia and Yaroslav D. Sergeyev Institute of Systems

More information

Contents. Preface... VII. Part I Classical Topics Revisited

Contents. Preface... VII. Part I Classical Topics Revisited Contents Preface........................................................ VII Part I Classical Topics Revisited 1 Sphere Packings........................................... 3 1.1 Kissing Numbers of Spheres..............................

More information

Random Number Generation and Monte Carlo Methods

Random Number Generation and Monte Carlo Methods James E. Gentle Random Number Generation and Monte Carlo Methods With 30 Illustrations Springer Contents Preface vii 1 Simulating Random Numbers from a Uniform Distribution 1 1.1 Linear Congruential Generators

More information

Risk bounds for some classification and regression models that interpolate

Risk bounds for some classification and regression models that interpolate Risk bounds for some classification and regression models that interpolate Daniel Hsu Columbia University Joint work with: Misha Belkin (The Ohio State University) Partha Mitra (Cold Spring Harbor Laboratory)

More information

DETERMINISTIC OPERATIONS RESEARCH

DETERMINISTIC OPERATIONS RESEARCH DETERMINISTIC OPERATIONS RESEARCH Models and Methods in Optimization Linear DAVID J. RADER, JR. Rose-Hulman Institute of Technology Department of Mathematics Terre Haute, IN WILEY A JOHN WILEY & SONS,

More information

Digital Image Processing

Digital Image Processing Digital Image Processing Third Edition Rafael C. Gonzalez University of Tennessee Richard E. Woods MedData Interactive PEARSON Prentice Hall Pearson Education International Contents Preface xv Acknowledgments

More information

Kernel Density Estimation

Kernel Density Estimation Kernel Density Estimation An Introduction Justus H. Piater, Université de Liège Overview 1. Densities and their Estimation 2. Basic Estimators for Univariate KDE 3. Remarks 4. Methods for Particular Domains

More information

Finite Math Linear Programming 1 May / 7

Finite Math Linear Programming 1 May / 7 Linear Programming Finite Math 1 May 2017 Finite Math Linear Programming 1 May 2017 1 / 7 General Description of Linear Programming Finite Math Linear Programming 1 May 2017 2 / 7 General Description of

More information

Unified Methods for Censored Longitudinal Data and Causality

Unified Methods for Censored Longitudinal Data and Causality Mark J. van der Laan James M. Robins Unified Methods for Censored Longitudinal Data and Causality Springer Preface v Notation 1 1 Introduction 8 1.1 Motivation, Bibliographic History, and an Overview of

More information

Convex Analysis and Minimization Algorithms I

Convex Analysis and Minimization Algorithms I Jean-Baptiste Hiriart-Urruty Claude Lemarechal Convex Analysis and Minimization Algorithms I Fundamentals With 113 Figures Springer-Verlag Berlin Heidelberg New York London Paris Tokyo Hong Kong Barcelona

More information

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer Ludwig Fahrmeir Gerhard Tute Statistical odelling Based on Generalized Linear Model íecond Edition. Springer Preface to the Second Edition Preface to the First Edition List of Examples List of Figures

More information

CLASSIFICATION AND CHANGE DETECTION

CLASSIFICATION AND CHANGE DETECTION IMAGE ANALYSIS, CLASSIFICATION AND CHANGE DETECTION IN REMOTE SENSING With Algorithms for ENVI/IDL and Python THIRD EDITION Morton J. Canty CRC Press Taylor & Francis Group Boca Raton London NewYork CRC

More information

Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited.

Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited. page v Preface xiii I Basics 1 1 Optimization Models 3 1.1 Introduction... 3 1.2 Optimization: An Informal Introduction... 4 1.3 Linear Equations... 7 1.4 Linear Optimization... 10 Exercises... 12 1.5

More information

Fundamentals of Digital Image Processing

Fundamentals of Digital Image Processing \L\.6 Gw.i Fundamentals of Digital Image Processing A Practical Approach with Examples in Matlab Chris Solomon School of Physical Sciences, University of Kent, Canterbury, UK Toby Breckon School of Engineering,

More information

CS 229 Midterm Review

CS 229 Midterm Review CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask

More information

Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate

Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate Density estimation In density estimation problems, we are given a random sample from an unknown density Our objective is to estimate? Applications Classification If we estimate the density for each class,

More information

Curve and Surface Fitting with Splines. PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium

Curve and Surface Fitting with Splines. PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium Curve and Surface Fitting with Splines PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium CLARENDON PRESS OXFORD 1995 - Preface List of Figures List of Tables

More information

Curves and Fractal Dimension

Curves and Fractal Dimension Claude Tricot Curves and Fractal Dimension With a Foreword by Michel Mendes France With 163 Illustrations Springer-Verlag New York Berlin Heidelberg London Paris Tokyo Hong Kong Barcelona Budapest Contents

More information

Communication Complexity and Parallel Computing

Communication Complexity and Parallel Computing Juraj Hromkovic Communication Complexity and Parallel Computing With 40 Figures Springer Table of Contents 1 Introduction 1 1.1 Motivation and Aims 1 1.2 Concept and Organization 4 1.3 How to Read the

More information

Clustering. SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic

Clustering. SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic Clustering SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic Clustering is one of the fundamental and ubiquitous tasks in exploratory data analysis a first intuition about the

More information

David G. Luenberger Yinyu Ye. Linear and Nonlinear. Programming. Fourth Edition. ö Springer

David G. Luenberger Yinyu Ye. Linear and Nonlinear. Programming. Fourth Edition. ö Springer David G. Luenberger Yinyu Ye Linear and Nonlinear Programming Fourth Edition ö Springer Contents 1 Introduction 1 1.1 Optimization 1 1.2 Types of Problems 2 1.3 Size of Problems 5 1.4 Iterative Algorithms

More information

Clustering. Discover groups such that samples within a group are more similar to each other than samples across groups.

Clustering. Discover groups such that samples within a group are more similar to each other than samples across groups. Clustering 1 Clustering Discover groups such that samples within a group are more similar to each other than samples across groups. 2 Clustering Discover groups such that samples within a group are more

More information

J. Weston, A. Gammerman, M. Stitson, V. Vapnik, V. Vovk, C. Watkins. Technical Report. February 5, 1998

J. Weston, A. Gammerman, M. Stitson, V. Vapnik, V. Vovk, C. Watkins. Technical Report. February 5, 1998 Density Estimation using Support Vector Machines J. Weston, A. Gammerman, M. Stitson, V. Vapnik, V. Vovk, C. Watkins. Technical Report CSD-TR-97-3 February 5, 998!()+, -./ 3456 Department of Computer Science

More information

Homework. Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression Pod-cast lecture on-line. Next lectures:

Homework. Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression Pod-cast lecture on-line. Next lectures: Homework Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression 3.0-3.2 Pod-cast lecture on-line Next lectures: I posted a rough plan. It is flexible though so please come with suggestions Bayes

More information

MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE

MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE REGULARIZATION METHODS FOR HIGH DIMENSIONAL LEARNING Francesca Odone and Lorenzo Rosasco odone@disi.unige.it - lrosasco@mit.edu June 6, 2011 ABOUT THIS

More information

Leave-One-Out Support Vector Machines

Leave-One-Out Support Vector Machines Leave-One-Out Support Vector Machines Jason Weston Department of Computer Science Royal Holloway, University of London, Egham Hill, Egham, Surrey, TW20 OEX, UK. Abstract We present a new learning algorithm

More information

Optimization Methods for Machine Learning (OMML)

Optimization Methods for Machine Learning (OMML) Optimization Methods for Machine Learning (OMML) 2nd lecture Prof. L. Palagi References: 1. Bishop Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1) 2. V. Cherlassky, F. Mulier - Learning

More information

Area, Lattice Points, and Exponential Sums

Area, Lattice Points, and Exponential Sums Area, Lattice Points, and Exponential Sums M. N. Huxley College of Cardiff University of Wales CLARENDON PRESS OXFORD 1996 Notation Introduction Part I Elementary methods 1. The rational line 1.1 Height

More information

Machine Learning Lecture 3

Machine Learning Lecture 3 Machine Learning Lecture 3 Probability Density Estimation II 19.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Exam dates We re in the process

More information

Image Analysis, Classification and Change Detection in Remote Sensing

Image Analysis, Classification and Change Detection in Remote Sensing Image Analysis, Classification and Change Detection in Remote Sensing WITH ALGORITHMS FOR ENVI/IDL Morton J. Canty Taylor &. Francis Taylor & Francis Group Boca Raton London New York CRC is an imprint

More information

MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE

MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE MODEL SELECTION AND REGULARIZATION PARAMETER CHOICE REGULARIZATION METHODS FOR HIGH DIMENSIONAL LEARNING Francesca Odone and Lorenzo Rosasco odone@disi.unige.it - lrosasco@mit.edu June 3, 2013 ABOUT THIS

More information

Let be a function. We say, is a plane curve given by the. Let a curve be given by function where is differentiable with continuous.

Let be a function. We say, is a plane curve given by the. Let a curve be given by function where is differentiable with continuous. Module 8 : Applications of Integration - II Lecture 22 : Arc Length of a Plane Curve [Section 221] Objectives In this section you will learn the following : How to find the length of a plane curve 221

More information

Machine Learning (BSMC-GA 4439) Wenke Liu

Machine Learning (BSMC-GA 4439) Wenke Liu Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning

More information

Machine Learning (BSMC-GA 4439) Wenke Liu

Machine Learning (BSMC-GA 4439) Wenke Liu Machine Learning (BSMC-GA 4439) Wenke Liu 01-25-2018 Outline Background Defining proximity Clustering methods Determining number of clusters Other approaches Cluster analysis as unsupervised Learning Unsupervised

More information

Contents. I The Basic Framework for Stationary Problems 1

Contents. I The Basic Framework for Stationary Problems 1 page v Preface xiii I The Basic Framework for Stationary Problems 1 1 Some model PDEs 3 1.1 Laplace s equation; elliptic BVPs... 3 1.1.1 Physical experiments modeled by Laplace s equation... 5 1.2 Other

More information

Chapter 7: Competitive learning, clustering, and self-organizing maps

Chapter 7: Competitive learning, clustering, and self-organizing maps Chapter 7: Competitive learning, clustering, and self-organizing maps António R. C. Paiva EEL 6814 Spring 2008 Outline Competitive learning Clustering Self-Organizing Maps What is competition in neural

More information

Research on the New Image De-Noising Methodology Based on Neural Network and HMM-Hidden Markov Models

Research on the New Image De-Noising Methodology Based on Neural Network and HMM-Hidden Markov Models Research on the New Image De-Noising Methodology Based on Neural Network and HMM-Hidden Markov Models Wenzhun Huang 1, a and Xinxin Xie 1, b 1 School of Information Engineering, Xijing University, Xi an

More information

Empirical risk minimization (ERM) A first model of learning. The excess risk. Getting a uniform guarantee

Empirical risk minimization (ERM) A first model of learning. The excess risk. Getting a uniform guarantee A first model of learning Let s restrict our attention to binary classification our labels belong to (or ) Empirical risk minimization (ERM) Recall the definitions of risk/empirical risk We observe the

More information

Combinatorial Selection and Least Absolute Shrinkage via The CLASH Operator

Combinatorial Selection and Least Absolute Shrinkage via The CLASH Operator Combinatorial Selection and Least Absolute Shrinkage via The CLASH Operator Volkan Cevher Laboratory for Information and Inference Systems LIONS / EPFL http://lions.epfl.ch & Idiap Research Institute joint

More information

Non-Parametric Modeling

Non-Parametric Modeling Non-Parametric Modeling CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Non-Parametric Density Estimation Parzen Windows Kn-Nearest Neighbor

More information

Probabilistic Graphical Models

Probabilistic Graphical Models School of Computer Science Probabilistic Graphical Models Theory of Variational Inference: Inner and Outer Approximation Eric Xing Lecture 14, February 29, 2016 Reading: W & J Book Chapters Eric Xing @

More information

Expectation Maximization (EM) and Gaussian Mixture Models

Expectation Maximization (EM) and Gaussian Mixture Models Expectation Maximization (EM) and Gaussian Mixture Models Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 2 3 4 5 6 7 8 Unsupervised Learning Motivation

More information

Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning

Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning Robot Learning 1 General Pipeline 1. Data acquisition (e.g., from 3D sensors) 2. Feature extraction and representation construction 3. Robot learning: e.g., classification (recognition) or clustering (knowledge

More information

TOPOLOGICAL ALGEBRAS SELECTED TOPICS

TOPOLOGICAL ALGEBRAS SELECTED TOPICS TOPOLOGICAL ALGEBRAS SELECTED TOPICS Anastasios MALLIOS Mathematical Institute University ofathens Greece 1986 NORTH-HOLLAND-AMSTERDAM NEW YORK»OXFORD»TOKYO xiii Contents Preface ix PART I. GENERAL THEORY

More information

Lecture 7: Support Vector Machine

Lecture 7: Support Vector Machine Lecture 7: Support Vector Machine Hien Van Nguyen University of Houston 9/28/2017 Separating hyperplane Red and green dots can be separated by a separating hyperplane Two classes are separable, i.e., each

More information

Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06

Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06 Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,

More information

I How does the formulation (5) serve the purpose of the composite parameterization

I How does the formulation (5) serve the purpose of the composite parameterization Supplemental Material to Identifying Alzheimer s Disease-Related Brain Regions from Multi-Modality Neuroimaging Data using Sparse Composite Linear Discrimination Analysis I How does the formulation (5)

More information

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 5, MAY

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 5, MAY IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 55, NO 5, MAY 2007 1911 Game Theoretic Cross-Layer Transmission Policies in Multipacket Reception Wireless Networks Minh Hanh Ngo, Student Member, IEEE, and

More information

SECTION 5 IMAGE PROCESSING 2

SECTION 5 IMAGE PROCESSING 2 SECTION 5 IMAGE PROCESSING 2 5.1 Resampling 3 5.1.1 Image Interpolation Comparison 3 5.2 Convolution 3 5.3 Smoothing Filters 3 5.3.1 Mean Filter 3 5.3.2 Median Filter 4 5.3.3 Pseudomedian Filter 6 5.3.4

More information

Lecture 3 January 22

Lecture 3 January 22 EE 38V: Large cale Learning pring 203 Lecture 3 January 22 Lecturer: Caramanis & anghavi cribe: ubhashini Krishsamy 3. Clustering In the last lecture, we saw Locality ensitive Hashing (LH) which uses hash

More information

F. THOMSON LEIGHTON INTRODUCTION TO PARALLEL ALGORITHMS AND ARCHITECTURES: ARRAYS TREES HYPERCUBES

F. THOMSON LEIGHTON INTRODUCTION TO PARALLEL ALGORITHMS AND ARCHITECTURES: ARRAYS TREES HYPERCUBES F. THOMSON LEIGHTON INTRODUCTION TO PARALLEL ALGORITHMS AND ARCHITECTURES: ARRAYS TREES HYPERCUBES MORGAN KAUFMANN PUBLISHERS SAN MATEO, CALIFORNIA Contents Preface Organization of the Material Teaching

More information

Dynamic Thresholding for Image Analysis

Dynamic Thresholding for Image Analysis Dynamic Thresholding for Image Analysis Statistical Consulting Report for Edward Chan Clean Energy Research Center University of British Columbia by Libo Lu Department of Statistics University of British

More information

Nonsmooth Optimization and Related Topics

Nonsmooth Optimization and Related Topics Nonsmooth Optimization and Related Topics Edited by F. H. Clarke University of Montreal Montreal, Quebec, Canada V. F. Dem'yanov Leningrad State University Leningrad, USSR I and F. Giannessi University

More information

Manifold Learning Theory and Applications

Manifold Learning Theory and Applications Manifold Learning Theory and Applications Yunqian Ma and Yun Fu CRC Press Taylor Si Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business Contents

More information

Machine Learning / Jan 27, 2010

Machine Learning / Jan 27, 2010 Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,

More information

Generalized Additive Models

Generalized Additive Models :p Texts in Statistical Science Generalized Additive Models An Introduction with R Simon N. Wood Contents Preface XV 1 Linear Models 1 1.1 A simple linear model 2 Simple least squares estimation 3 1.1.1

More information

INDEPENDENT COMPONENT ANALYSIS WITH QUANTIZING DENSITY ESTIMATORS. Peter Meinicke, Helge Ritter. Neuroinformatics Group University Bielefeld Germany

INDEPENDENT COMPONENT ANALYSIS WITH QUANTIZING DENSITY ESTIMATORS. Peter Meinicke, Helge Ritter. Neuroinformatics Group University Bielefeld Germany INDEPENDENT COMPONENT ANALYSIS WITH QUANTIZING DENSITY ESTIMATORS Peter Meinicke, Helge Ritter Neuroinformatics Group University Bielefeld Germany ABSTRACT We propose an approach to source adaptivity in

More information

Nonparametric Estimation of Distribution Function using Bezier Curve

Nonparametric Estimation of Distribution Function using Bezier Curve Communications for Statistical Applications and Methods 2014, Vol. 21, No. 1, 105 114 DOI: http://dx.doi.org/10.5351/csam.2014.21.1.105 ISSN 2287-7843 Nonparametric Estimation of Distribution Function

More information

INTRODUCTION TO The Uniform Geometrical Theory of Diffraction

INTRODUCTION TO The Uniform Geometrical Theory of Diffraction INTRODUCTION TO The Uniform Geometrical Theory of Diffraction D.A. McNamara, C.W.I. Pistorius J.A.G. Malherbe University of Pretoria Artech House Boston London CONTENTS Preface xiii Chapter 1 The Nature

More information

Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition

Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant

More information

An introduction to multi-armed bandits

An introduction to multi-armed bandits An introduction to multi-armed bandits Henry WJ Reeve (Manchester) (henry.reeve@manchester.ac.uk) A joint work with Joe Mellor (Edinburgh) & Professor Gavin Brown (Manchester) Plan 1. An introduction to

More information

Lecture 27, April 24, Reading: See class website. Nonparametric regression and kernel smoothing. Structured sparse additive models (GroupSpAM)

Lecture 27, April 24, Reading: See class website. Nonparametric regression and kernel smoothing. Structured sparse additive models (GroupSpAM) School of Computer Science Probabilistic Graphical Models Structured Sparse Additive Models Junming Yin and Eric Xing Lecture 7, April 4, 013 Reading: See class website 1 Outline Nonparametric regression

More information

On Multiple-Instance Learning of Halfspaces

On Multiple-Instance Learning of Halfspaces On Multiple-Instance Learning of Halfspaces D. I. Diochnos 1, R. H. Sloan 1, and Gy. Turán 1,2 1 University of Illinois at Chicago ddioch2 sloan gyt @uic.edu 2 Research Group on AI, Hungarian Acad. Sciences

More information

Contents III. 1 Introduction 1

Contents III. 1 Introduction 1 III Contents 1 Introduction 1 2 The Parametric Distributional Clustering Model 5 2.1 The Data Acquisition Process.................... 5 2.2 The Generative Model........................ 8 2.3 The Likelihood

More information

Principles of Network Economics

Principles of Network Economics Hagen Bobzin Principles of Network Economics SPIN Springer s internal project number, if known unknown Monograph August 12, 2005 Springer Berlin Heidelberg New York Hong Kong London Milan Paris Tokyo Contents

More information

Kernels and representation

Kernels and representation Kernels and representation Corso di AA, anno 2017/18, Padova Fabio Aiolli 20 Dicembre 2017 Fabio Aiolli Kernels and representation 20 Dicembre 2017 1 / 19 (Hierarchical) Representation Learning Hierarchical

More information

MapReduce Algorithms. Barna Saha. March 28, 2016

MapReduce Algorithms. Barna Saha. March 28, 2016 MapReduce Algorithms Barna Saha March 28, 2016 Complexity Model for MapReduce Minimum Spanning Tree in MapReduce Computing Dense Subgraph in MapReduce Complexity Model for MapReduce:MRC i Input: finite

More information

epub WU Institutional Repository

epub WU Institutional Repository epub WU Institutional Repository Josef Leydold and Wolfgang Hörmann The Automatic Generation of One- and Multi-dimensional Distributions with Transformed Density Rejection Paper Original Citation: Leydold,

More information

Convex or non-convex: which is better?

Convex or non-convex: which is better? Sparsity Amplified Ivan Selesnick Electrical and Computer Engineering Tandon School of Engineering New York University Brooklyn, New York March 7 / 4 Convex or non-convex: which is better? (for sparse-regularized

More information

Generative and discriminative classification techniques

Generative and discriminative classification techniques Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14

More information

Bayesian Estimation for Skew Normal Distributions Using Data Augmentation

Bayesian Estimation for Skew Normal Distributions Using Data Augmentation The Korean Communications in Statistics Vol. 12 No. 2, 2005 pp. 323-333 Bayesian Estimation for Skew Normal Distributions Using Data Augmentation Hea-Jung Kim 1) Abstract In this paper, we develop a MCMC

More information

SVMs for Structured Output. Andrea Vedaldi

SVMs for Structured Output. Andrea Vedaldi SVMs for Structured Output Andrea Vedaldi SVM struct Tsochantaridis Hofmann Joachims Altun 04 Extending SVMs 3 Extending SVMs SVM = parametric function arbitrary input binary output 3 Extending SVMs SVM

More information

of Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures

of Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures 2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Jean-Baptiste Hiriart-Urruty Claude Lemarechal Fundamentals of Convex

More information

Cellular Tree Classifiers. Gérard Biau & Luc Devroye

Cellular Tree Classifiers. Gérard Biau & Luc Devroye Cellular Tree Classifiers Gérard Biau & Luc Devroye Paris, December 2013 Outline 1 Context 2 Cellular tree classifiers 3 A mathematical model 4 Are there consistent cellular tree classifiers? 5 A non-randomized

More information

Classification and Trees

Classification and Trees Classification and Trees Luc Devroye School of Computer Science, McGill University Montreal, Canada H3A 2K6 This paper is dedicated to the memory of Pierre Devijver. Abstract. Breiman, Friedman, Gordon

More information

Package benchden. February 19, 2015

Package benchden. February 19, 2015 Package benchden February 19, 2015 Type Package Title 28 benchmark densities from Berlinet/Devroye (1994) Version 1.0.5 Date 2012-02-29 Author Thoralf Mildenberger, Henrike Weinert, Sebastian Tiemeyer

More information

Medical Image Segmentation Based on Mutual Information Maximization

Medical Image Segmentation Based on Mutual Information Maximization Medical Image Segmentation Based on Mutual Information Maximization J.Rigau, M.Feixas, M.Sbert, A.Bardera, and I.Boada Institut d Informatica i Aplicacions, Universitat de Girona, Spain {jaume.rigau,miquel.feixas,mateu.sbert,anton.bardera,imma.boada}@udg.es

More information

CHAPTER 1 Introduction 1. CHAPTER 2 Images, Sampling and Frequency Domain Processing 37

CHAPTER 1 Introduction 1. CHAPTER 2 Images, Sampling and Frequency Domain Processing 37 Extended Contents List Preface... xi About the authors... xvii CHAPTER 1 Introduction 1 1.1 Overview... 1 1.2 Human and Computer Vision... 2 1.3 The Human Vision System... 4 1.3.1 The Eye... 5 1.3.2 The

More information

Support Vector. Machines. Algorithms, and Extensions. Optimization Based Theory, Naiyang Deng YingjieTian. Chunhua Zhang.

Support Vector. Machines. Algorithms, and Extensions. Optimization Based Theory, Naiyang Deng YingjieTian. Chunhua Zhang. Support Vector Machines Optimization Based Theory, Algorithms, and Extensions Naiyang Deng YingjieTian Chunhua Zhang CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an imprint

More information

Time Series Analysis by State Space Methods

Time Series Analysis by State Space Methods Time Series Analysis by State Space Methods Second Edition J. Durbin London School of Economics and Political Science and University College London S. J. Koopman Vrije Universiteit Amsterdam OXFORD UNIVERSITY

More information

^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held

^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held Rudolf Kruse Christian Borgelt Frank Klawonn Christian Moewes Matthias Steinbrecher Pascal Held Computational Intelligence A Methodological Introduction ^ Springer Contents 1 Introduction 1 1.1 Intelligent

More information

Proximal operator and methods

Proximal operator and methods Proximal operator and methods Master 2 Data Science, Univ. Paris Saclay Robert M. Gower Optimization Sum of Terms A Datum Function Finite Sum Training Problem The Training Problem Convergence GD I Theorem

More information

Deep Learning. Architecture Design for. Sargur N. Srihari

Deep Learning. Architecture Design for. Sargur N. Srihari Architecture Design for Deep Learning Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation

More information

MEDICAL IMAGE ANALYSIS

MEDICAL IMAGE ANALYSIS SECOND EDITION MEDICAL IMAGE ANALYSIS ATAM P. DHAWAN g, A B IEEE Engineering in Medicine and Biology Society, Sponsor IEEE Press Series in Biomedical Engineering Metin Akay, Series Editor +IEEE IEEE PRESS

More information

Model Assessment and Selection. Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer

Model Assessment and Selection. Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer Model Assessment and Selection Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 Model Training data Testing data Model Testing error rate Training error

More information

MR IMAGE SEGMENTATION

MR IMAGE SEGMENTATION MR IMAGE SEGMENTATION Prepared by : Monil Shah What is Segmentation? Partitioning a region or regions of interest in images such that each region corresponds to one or more anatomic structures Classification

More information

Contents. Preface to the Second Edition

Contents. Preface to the Second Edition Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................

More information

Algorithms for Approximation II. J. C. Mason Professor of Computational Mathematics, Royal Military College of Science, Shrivenham, UK and

Algorithms for Approximation II. J. C. Mason Professor of Computational Mathematics, Royal Military College of Science, Shrivenham, UK and Algorithms for Approximation II Based on the proceedings of the Second International Conference on Algorithms for Approximation, held at Royal Military College of Science, Shrivenham, July 1988 Edited

More information

Greed Considered Harmful

Greed Considered Harmful Greed Considered Harmful Nonlinear (in)stabilities in network resource allocation Priya Ranjan Indo-US workshop 2009 Outline Background Model & Motivation Main results Fixed delays Single-user, single-link

More information

Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask

Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of

More information

INTERNATIONAL COMPUTER SCIENCE INSTITUTE. Semi-Supervised Model Selection Based on Cross-Validation

INTERNATIONAL COMPUTER SCIENCE INSTITUTE. Semi-Supervised Model Selection Based on Cross-Validation INTERNATIONAL OMPUTER SIENE INSTITUTE 947 enter St. Suite 6 Berkeley, alifornia 9474-98 (5) 666-29 FAX (5) 666-2956 Semi-Supervised Model Selection Based on ross-validation Matti Kääriäinen International

More information

Minimal Universal Bipartite Graphs

Minimal Universal Bipartite Graphs Minimal Universal Bipartite Graphs Vadim V. Lozin, Gábor Rudolf Abstract A graph U is (induced)-universal for a class of graphs X if every member of X is contained in U as an induced subgraph. We study

More information