OPTIMIZATION. Optimization. Derivative-based optimization. Derivative-free optimization. Steepest descent (gradient) methods Newton s method
|
|
- Archibald Russell
- 6 years ago
- Views:
Transcription
1 OPTIMIZATION Optimization Derivative-based optimization Steepest descent (gradient) methods Newton s method Derivative-free optimization Simplex Simulated Annealing Genetic Algorithms Ant Colony Optimization... 2
2 Derivative-based optimization Goal: Solving optimization (nonlinear) problems using derivative information Methods: Gradient based optimization Steepest descent Newton methods Conjugate gradient Nonlinear least-squares problems 22 Derivative-based optimization Methods used in: Deriving membership functions Deriving consequents of Takagi-Sugeno models Neural network learning Regression analysis in nonlinear models Optimization of nonlinear neuro-fuzzy models... Methods will be introduced as needed in the next lectures. 23 2
3 DATA CLUSTERING Data clustering Extensively used for data categorization (e.g. classification) data compression model construction Two types Hierarchical clustering Objective function-based clustering Partitions data into several groups such that withingroup similarity is larger than between-group similarity 25 3
4 Example Hierarchical clustering Proceed by a series of successive mergers of data points (bottom-up) or successive division of data set (top-down) Induce a tree-like structure in the data set No pre-determined number of clusters Clusters are induced after partitioning by thresholding the induced tree based on similarity 27 4
5 Similarity Based on some notion of distance Inputs are usually normalized to the range [,] Inversely related to distance, e.g. S( xi, x j) d ( x, x ) Similarity is a reflexive and symmetric relation Suitable distance metrics can be defined, even for nominal data i j 28 Linkage algorithms Given the data T x [ x, x,, x ], k,, N k k 2k nk. Start with N clusters and compute the NN matrix of similarities 2. Determine most similar clusters i *, j * 3. Merge clusters i * and j * for form new cluster i. 4. Delete the rows and columns corresponding to i * and j * in the similarity matrix. 5. Determine the similarity between i and all other remaining clusters. 29 5
6 Cost function-based clustering Partition data by optimizing an objective function Clusters represented by cluster prototypes Objective function usually minimizes the distance to cluster prototypes Pre-determined number of clusters Simultaneous estimation of partition and cluster prototypes 2 Crisp vs. fuzzy clustering Crisp clustering algorithms partition the data set into disjoint groups, i.e. each data point belongs to one cluster only similarity is quantified using some metric Fuzzy clustering algorithms partition the data set into overlapping groups, i.e. each data point belongs to multiple clusters with varying degree of membership similarity is quantified using some metric which is modified by membership values 2 6
7 Clustering algorithms Hard c-means (K-means) Fuzzy c-means Gustafson-Kessel Possibilistic clustering Mountain clustering Subtractive clustering And many, many others 22 K(C)-means clustering Partition data into disjoint sets based on similarity amongst patterns Given the data T n x [ x, x,, x ], k,, N k k 2k nk Find the crisp partition matrix: N U, ij {,} C CN and the cluster centres: n V { v, v }, v C i 23 7
8 K-means clustering The following cost function of dissimilarity measures is minimized C C J J d( x v ), often d( x v ) x v i k i k i k i i i k, xkgi Partition matrix can be calculated if centers are fixed 2 2 if x j vi x j vk, k ij otherwise Centers can be calculated if partition matrix is fixed vi xk G i k xkgi i 2 24 C-means algorithm. Initialize cluster centers V 2. Determine the partition matrix U 3. Compute cost function J Stop if J is below a threshold or if it has not improved 4. Update the cluster centers V and iterate from step 2. Note that C C N, j,, N and N ij i i j ij 25 8
9 FUZZY CLUSTERING Fuzzy c means Partition data into overlapping sets based on similarity amongst patterns Given the data T n x [ x, x,, x ], k,, N k k 2k nk Find the fuzzy partition matrix: N U, ij [,] C CN This is a generalization and the cluster centres: of hard c-means! n V { v, v }, v C i 27 9
10 Fuzzy clustering Minimize objective function J C N m 2 ( X, U, V) ikd ( xk, vi) i k subject to ik, i,, C, k,, N C i N ik k, k,, N N, i,, C ik m(, ) is the fuzziness parameter membership degree total membership no cluster empty 28 Fuzzy c-means algorithm Initialize V or U Repeat. Compute cluster centers N m k ik x k vi N Assumes partition matrix is fixed m k 2. Calculate distances 2 T dik ( xk vi) ( xk vi) 3. Update partition matrix Until U ik C 2 2 /( ) ( / ) m d j ik d jk ik Assumes cluster centers are fixed Other stopping criteria are possible 29
11 Fuzzy c-means example MF.5 MF.5.5 Y X.5.5 Y X.5 MF Y X 22 Fuzzy c-means example 22
12 Distance measures Euclidian norm: 2 T d ( x, v ) ( x v ) ( x v ) k i k i k i Inner-product norm: 2 T d ( x, v ) ( x v ) A ( x v ) k i k i k i Ais diagonal v i Mahalanobis norm: d ( x, v ) ( x v ) F ( x v ) 2 T k i k i i k i Rotated clusters 222 Gustafson-Kessel clustering Uses an adaptive distance metric d 2 ( x v ) ( x v ) T A ( x v ) k i k i i k i / n i i i A F F Fuzzy covariance matrix ( ) ( )( ) ik xk v k i xk v i Fi N m ( ) N m T k Clusters are constrained by volume ik Clusters adapt themselves to the shape and location of data 223 2
13 GK algorithm Repeat. Compute cluster centers N m k ik x k v Assumes partition matrix is fixed i N m k ik 2. Calculate covariance matrices and distances / n N m T ( ) ( x v )( x v ) 2 T k ik k i k i d ik F i ( x k v i ) F i ( x k v i ) Fi N m ( ) k ik 3. Update partition matrix Assumes cluster centers are fixed ik C 2 2 /( ) ( d / ) m j ik djk Until U Other stopping criteria are possible 224 GK algorithm example 225 3
14 GK algorithm example smallest eigenvector d(z,v) k data cluster centers z k d(z,v) k 2 smallest eigenvector A A Mountain clustering Algorithm: Lay a grid (any type) on the data space Compute mountain functions (measure of data density) f( v ) i N k e 2 d ( xkvi) 2 2 Sequentially destroy mountain function after selecting grid point with highest value C ( C) ( j) vi vi v j j f ( ) f( ) f ( ) e 2 d ( viv j)
15 Mountain construction.8.6 (a) (b) = (c) (d) =. = Mountain destruction (a) =., =. (b) (c) (d)
16 Subtractive clustering Mountain clustering with four variables and grid lines results in 4 grid points to be evaluated. In subtractive clustering, the grid consists of the data points themselves. Computational complexity independent of the dimension of the data vectors Rule of thumb: =.5 23 Effect of probabilistic constraint Probabilistic constraint: C i ik, k,, N Problematic if a data point lies far away from all clusters (e.g. outliers) Leads to nonconvex clusters Possibilistic constraint: C i ik, k,, N 23 6
17 Possibilistic clustering Minimize objective function: C N m C N 2 m ( X, U, V, ) ( xk, vi ) i( ik ) ik i k i k J d m (,) is the fuzziness parameter determine the size of the clusters suitable values from average inter-cluster distance N m 2 k ik d ik i N m k ik The optimization problem can now be decomposed into C independent optimization problems. 232 Possibilistic clustering algorithm Repeat:. Compute cluster centers N m k ik x k vi N m Until k 2. Calculate distances ik d ( x v ) A( x v ) 2 T ik k i k i 3. Update partition matrix ik Membership value does not depend 2 d m ik 2 on the membership of other clusters U i 233 7
18 Effect of fuzziness index As m increases, clusters overlap more; their centres become more isolated As m decreases, clusters overlap less; they become crisp Often m = 2 is selected m =.2 m = Issues in fuzzy clustering Normalization Initialization Cluster volumes Number of clusters Cluster convexity Selection of parameters (e.g. fuzziness m) Categorical variables Outliers Missing values 235 8
19 Normalization How to compare measurements on different scales? Data box normalization xjl min xjl ' j xjl, j,, N and l,, n max x min x j jl j Standard deviation normalization x ' jl xl xjl, j,, N and l,, n l jl Adaptive distance metrics as in Gustafson-Kessel clustering are less sensitive to normalization 236 Initialization How to avoid local minima during the optimization? Randomly select a set of cluster prototypes V Randomly select a set of data points as cluster centers V Randomly initialize the partition matrix U Use information (e.g. cluster center locations) from a separate clustering step Initialize centers far away from data 237 9
20 Cluster volumes How large should clusters be? Extent of clusters Data density and distribution Size of cluster prototypes Cluster volume can be a parameter in Gustafson- Kessel clustering FCM cluster centres Cluster validity How good are the clustering results? Correct number of clusters? Well-separated clusters? Compact clusters? Cluster validity measures try to quantify the answers to these questions in a formula Optimal number of clusters at a local minimum of the validity measure 239 2
21 Validity measures Gath and Geva index S N C m T ( ) ( )( ) ik xk vi x k k v i G N m i ( ) k ik C Xie-Beni index S X C N m 2 d x ik k vi i k N d (, ) 2 min ( vi v j) i, j, ij 24 Validity measures - example Validity (Gath and Geva) minimum Number of clusters Local Models (Gustafson-Kessel) Clusters
22 CLUSTERING FOR IDENTIFICATION Modeling based on clustering. Determine relevant input and output variables/features and collect data 2. Select model structure (Mamdani, Takagi-Sugeno, ) 3. Select number of clusters and clustering algorithm 4. Cluster the data 5. Obtain antecedent membership functions (MF) 6. Obtain consequents (MF or parameters) 7. Simplify the model, if necessary 8. Validate the model
23 Linguistic models fr. clustering R: Ifxis A thenyis B, i,2,, K i i i Use fuzzy c-means algorithm. Cluster data in input output product space. Membership functions obtained by: projection onto variables, membership function parameterization. One rule per cluster 244 Membership functions After clustering, MFs are obtained by projections of partition matrix values
24 Example of linguistic model If income is Low then tax is Low If income is High then tax is High 246 Data for clustering T Inputs and output: x y X, y ZX y T x y N N Example of data: what is the structure of the model? y(2) y2(2) y2() u(2) y(3) y(3) y2(3) y2(2) u(3) y(4) Z y( N ) y2( N ) y2( N 2) u( N ) y( N) y ( k ) f( y ( k), y ( k), y ( k ), u ( k))
25 TS models from clustering R: If xis A then y f ( x), i,2,, K i i i i Takagi-Sugeno order zero: Takagi-Sugeno order one: Degree of fulfillment ( x) i A i Model output given by the weighted fuzzy-mean: y K k y i b y a xb i T i i i k kyk, with k (normalized k) K j j 248 TS models from clustering Use fuzzy clustering algorithm (one rule per cluster) Cluster data in input output product space Project clusters onto input variables and fit parametric membership functions to projected clusters Estimate consequent parameters
26 Matrix notation T x y X, y T x y N N W k k kn Consequent parameters: TS order zero: b, k,, K T TS order one: k ak b k Extended regression matrix k e k X X 25 Estimation of consequents Global least squares: W WX WX WX e 2 e K e T T T 2 K Resulting least squares problem: Solution: T T WW Wy y W Solution of local least squares: T T k e k e e k X W X X W y 25 26
27 Example: TS order one Consequents can approximate local linear models of the system y y a xb y a xb 2 2 y a xb 3 3 A A 2 A 3 x 252 Example: TS model (a) Local Models 6 4 Output (y) Input (x) (b) Clusters Membership degree Input (x)
28 Example: TS order zero y b 2 y = f(x) b 4 b b 3 x A 2 A 3 A Example: pressure control dp RT 2 2P P g ( RH) ln dt 22.4Vh Kf P
29 Fermenter: parameters R gas constant (8.34 J mol K ) T temperature (35 K) V h gas volume (.5 m ) g gas flow-rate (3.75 m s ) R H radius of the outlet pipe (.78 m) P reference pressure (.3 N m ) outside air density (.2 Kg m ) P pressure in the tank (N m ) K f valve friction factor (J mol ) 256 Takagi-Sugeno fuzzy model. If yk ( ) is LOW and uk ( ) is OPEN then yk ( ).67 yk ( ).7 uk ( ) If yk ( ) is MEDIUM and uk ( ) is HALF CLOSED then yk ( ).8 yk ( ).28 uk ( ).7 3. If yk ( ) is HIGH and uk ( ) is CLOSED then yk ( ).9 yk ( ).7 uk ( )
30 Validation 258 Example: 3-D representation Fuzzy partition of the input-state product space Corresponding local linear models membership grade current pressure fuzzyregion If current pressure yk () is LOW then and valve uk () is OPEN 5 valve position new pressure Fuzzy-linear rules current pressure valve position new pressure y(+)=a k y()+ k bu()+ k c 259 3
31 Interpretability in fuzzy models Interpretability is not obtained automatically Knowledge acquisition ensures transparency. Based on data: some redundancy is unavoidable. Redundancy manifests itself in two ways: High number of rules. Trade-off between: model accuracy / model complexity generalization capability / approximating data Very similar fuzzy sets similarity between fuzzy sets similarity to the universal fuzzy set 26 Redundancy in fuzzy models High number of rules Overlapping similar fuzzy sets Unnecessary complexity Difficult to assign linguistic labels Less transparency and generality 26 3
32 Redundancy in fuzzy clustering Number of clusters Projection of clusters onto antecedent variables 262 Methods to solve redundancy Cluster merging Method to determine the best number of clusters Merge clusters that are compatible Cluster again and continue merging until there are no compatible clusters Similarity based simplification Merge similar antecedent fuzzy sets Remove sets similar to universal set Combine / merge similar consequents Combine rules with equal antecedents
33 Cluster merging Select number of clusters larger than needed and do clustering Merge clusters that are compatible Cluster again and continue merging until there are no compatible clusters Cluster compatibility measured by Compatibility criteria How close are clusters? How similar are their characteristics? Etc. Similarity measures 264 Compatible cluster merging Select K number of clusters larger than needed. Repeat. Perform clustering 2. Evaluate compatibility criteria: jp' smallest eigenvector k, k close to ip' jp' i v v j k, k close to 2 2 smallest eigenvector ip' v i d 2 (v i,v j ) v j
34 Compatible cluster merging 3. Compute parallelism and closeness and 2 ij ij ij.8 parallel 2 ij.8 close membership.6.4 membership inner product distance 4. Compute compatibility s ij s ij = 2 ij ij Compatible cluster merging 5. Compute transitive closure of compatibility matrix S and threshold with s *. 6. Merge clusters if min max d v M v Until no clusters can be merged ik max v, v M i i j km d ij
35 Example (a) Local Models (a) Local Models 5 5 Output (y) Output (y) Input (x) Input (x) (b) Clusters (b) Clusters Membership degree Input (x) Initial ( clusters) Membership degree Input (x) After CCM (6 clusters) 268 Example (a) Local Models 6 4 Output (y) Input (x) (b) Clusters Membership degree Input (x)
36 Similarity based simplification Reduce redundancy in rule and term set Merge similar antecedent fuzzy sets create generalized concepts reduce the number of terms Remove sets similar to universal set (always fires) reduce number of terms Combine/merge similar consequents reduce the number of consequent values Combine rules with equal antecedents reduce number of rules 27 Similarity measures Similarity measure: AB S( AB, ) AB Merging of similar sets Gives generalizing term Merging by aggregating the parameters of the individual sets 27 36
37 Simplification and reduction 272 Example: algae growth Prediction of chlorophyll-a concentration in lake ecosystems. 998 observations from nine different lakes. Inputs: temperature, N,P,Si, day length, light intensity Output: chlorophyll-a concentration Takagi-Sugeno model: If T is A T, N is A N, P is A P, Si is A Si, D is A D and I is A I then Chl = p + p T + p 2 N + p 3 P + p 4 Si + p 5 D + p 6 I Method: fuzzy clustering and similarity analysis
38 Initial rule base Seven rules with a total of 42 fuzzy sets; Difficult to assign linguistic labels, inspection is virtually impossible 274 Simplified rule base
Clustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
More informationMODELING FOR RESIDUAL STRESS, SURFACE ROUGHNESS AND TOOL WEAR USING AN ADAPTIVE NEURO FUZZY INFERENCE SYSTEM
CHAPTER-7 MODELING FOR RESIDUAL STRESS, SURFACE ROUGHNESS AND TOOL WEAR USING AN ADAPTIVE NEURO FUZZY INFERENCE SYSTEM 7.1 Introduction To improve the overall efficiency of turning, it is necessary to
More informationIn the Name of God. Lecture 17: ANFIS Adaptive Network-Based Fuzzy Inference System
In the Name of God Lecture 17: ANFIS Adaptive Network-Based Fuzzy Inference System Outline ANFIS Architecture Hybrid Learning Algorithm Learning Methods that Cross-Fertilize ANFIS and RBFN ANFIS as a universal
More informationClustering & Classification (chapter 15)
Clustering & Classification (chapter 5) Kai Goebel Bill Cheetham RPI/GE Global Research goebel@cs.rpi.edu cheetham@cs.rpi.edu Outline k-means Fuzzy c-means Mountain Clustering knn Fuzzy knn Hierarchical
More informationImproving interpretability in approximative fuzzy models via multi-objective evolutionary algorithms.
Improving interpretability in approximative fuzzy models via multi-objective evolutionary algorithms. Gómez-Skarmeta, A.F. University of Murcia skarmeta@dif.um.es Jiménez, F. University of Murcia fernan@dif.um.es
More informationMachine Learning & Statistical Models
Astroinformatics Machine Learning & Statistical Models Neural Networks Feed Forward Hybrid Decision Analysis Decision Trees Random Decision Forests Evolving Trees Minimum Spanning Trees Perceptron Multi
More informationCS 2750 Machine Learning. Lecture 19. Clustering. CS 2750 Machine Learning. Clustering. Groups together similar instances in the data sample
Lecture 9 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem: distribute data into k different groups
More informationNeuro-fuzzy systems 1
1 : Trends and Applications International Conference on Control, Engineering & Information Technology (CEIT 14), March 22-25, Tunisia Dr/ Ahmad Taher Azar Assistant Professor, Faculty of Computers and
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationApplication of fuzzy set theory in image analysis. Nataša Sladoje Centre for Image Analysis
Application of fuzzy set theory in image analysis Nataša Sladoje Centre for Image Analysis Our topics for today Crisp vs fuzzy Fuzzy sets and fuzzy membership functions Fuzzy set operators Approximate
More informationCHAPTER 3 FUZZY RULE BASED MODEL FOR FAULT DIAGNOSIS
39 CHAPTER 3 FUZZY RULE BASED MODEL FOR FAULT DIAGNOSIS 3.1 INTRODUCTION Development of mathematical models is essential for many disciplines of engineering and science. Mathematical models are used for
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationFuzzy Modeling for Control.,,i.
Fuzzy Modeling for Control,,i. INTERNATIONAL SERIES IN INTELLIGENT TECHNOLOGIES Prof. Dr. Dr. h.c. Hans-Jiirgen Zimmermann, Editor European Laboratory for Intelligent Techniques Engineering Aachen, Germany
More informationFuzzy if-then rules fuzzy database modeling
Fuzzy if-then rules Associates a condition described using linguistic variables and fuzzy sets to a conclusion A scheme for capturing knowledge that involves imprecision 23.11.2010 1 fuzzy database modeling
More informationGRANULAR COMPUTING AND EVOLUTIONARY FUZZY MODELLING FOR MECHANICAL PROPERTIES OF ALLOY STEELS. G. Panoutsos and M. Mahfouf
GRANULAR COMPUTING AND EVOLUTIONARY FUZZY MODELLING FOR MECHANICAL PROPERTIES OF ALLOY STEELS G. Panoutsos and M. Mahfouf Institute for Microstructural and Mechanical Process Engineering: The University
More informationUnsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
More informationCHAPTER 4: CLUSTER ANALYSIS
CHAPTER 4: CLUSTER ANALYSIS WHAT IS CLUSTER ANALYSIS? A cluster is a collection of data-objects similar to one another within the same group & dissimilar to the objects in other groups. Cluster analysis
More informationWeb Based Fuzzy Clustering Analysis
Research Inventy: International Journal Of Engineering And Science Vol.4, Issue 11 (November2014), PP 51-57 Issn (e): 2278-4721, Issn (p):2319-6483, www.researchinventy.com Web Based Fuzzy Clustering Analysis
More informationPattern Recognition Lecture Sequential Clustering
Pattern Recognition Lecture Prof. Dr. Marcin Grzegorzek Research Group for Pattern Recognition Institute for Vision and Graphics University of Siegen, Germany Pattern Recognition Chain patterns sensor
More informationCS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
More informationCluster Analysis. Prof. Thomas B. Fomby Department of Economics Southern Methodist University Dallas, TX April 2008 April 2010
Cluster Analysis Prof. Thomas B. Fomby Department of Economics Southern Methodist University Dallas, TX 7575 April 008 April 010 Cluster Analysis, sometimes called data segmentation or customer segmentation,
More informationANFIS: ADAPTIVE-NETWORK-BASED FUZZY INFERENCE SYSTEMS (J.S.R. Jang 1993,1995) bell x; a, b, c = 1 a
ANFIS: ADAPTIVE-NETWORK-ASED FUZZ INFERENCE SSTEMS (J.S.R. Jang 993,995) Membership Functions triangular triangle( ; a, a b, c c) ma min = b a, c b, 0, trapezoidal trapezoid( ; a, b, a c, d d) ma min =
More informationUnsupervised Learning. Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi
Unsupervised Learning Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi Content Motivation Introduction Applications Types of clustering Clustering criterion functions Distance functions Normalization Which
More informationFuzzy Segmentation. Chapter Introduction. 4.2 Unsupervised Clustering.
Chapter 4 Fuzzy Segmentation 4. Introduction. The segmentation of objects whose color-composition is not common represents a difficult task, due to the illumination and the appropriate threshold selection
More informationCluster Analysis. Angela Montanari and Laura Anderlucci
Cluster Analysis Angela Montanari and Laura Anderlucci 1 Introduction Clustering a set of n objects into k groups is usually moved by the aim of identifying internally homogenous groups according to a
More informationImage Analysis - Lecture 5
Texture Segmentation Clustering Review Image Analysis - Lecture 5 Texture and Segmentation Magnus Oskarsson Lecture 5 Texture Segmentation Clustering Review Contents Texture Textons Filter Banks Gabor
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2008 CS 551, Spring 2008 c 2008, Selim Aksoy (Bilkent University)
More informationCHAPTER 4 AN IMPROVED INITIALIZATION METHOD FOR FUZZY C-MEANS CLUSTERING USING DENSITY BASED APPROACH
37 CHAPTER 4 AN IMPROVED INITIALIZATION METHOD FOR FUZZY C-MEANS CLUSTERING USING DENSITY BASED APPROACH 4.1 INTRODUCTION Genes can belong to any genetic network and are also coordinated by many regulatory
More informationMachine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
More informationUnsupervised Learning : Clustering
Unsupervised Learning : Clustering Things to be Addressed Traditional Learning Models. Cluster Analysis K-means Clustering Algorithm Drawbacks of traditional clustering algorithms. Clustering as a complex
More informationFuzzy clustering with volume prototypes and adaptive cluster merging
Fuzzy clustering with volume prototypes and adaptive cluster merging Kaymak, U and Setnes, M http://dx.doi.org/10.1109/tfuzz.2002.805901 Title Authors Type URL Fuzzy clustering with volume prototypes and
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany
More information10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
More informationFuzzy Clustering, Feature Selection, and Membership Function Optimization
Fuzzy Clustering, Feature Selection, and Membership Function Optimization Muriel Bowie Seminar Paper 2004 DIUF Department of Informatics University of Fribourg, Switzerland muriel.bowie@unifr.ch Abstract
More informationEuropean Journal of Science and Engineering Vol. 1, Issue 1, 2013 ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM IDENTIFICATION OF AN INDUCTION MOTOR
ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM IDENTIFICATION OF AN INDUCTION MOTOR Ahmed A. M. Emam College of Engineering Karrary University SUDAN ahmedimam1965@yahoo.co.in Eisa Bashier M. Tayeb College of Engineering
More informationOlmo S. Zavala Romero. Clustering Hierarchical Distance Group Dist. K-means. Center of Atmospheric Sciences, UNAM.
Center of Atmospheric Sciences, UNAM November 16, 2016 Cluster Analisis Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster)
More informationCluster Analysis: Agglomerate Hierarchical Clustering
Cluster Analysis: Agglomerate Hierarchical Clustering Yonghee Lee Department of Statistics, The University of Seoul Oct 29, 2015 Contents 1 Cluster Analysis Introduction Distance matrix Agglomerative Hierarchical
More informationCHAPTER 4 FUZZY LOGIC, K-MEANS, FUZZY C-MEANS AND BAYESIAN METHODS
CHAPTER 4 FUZZY LOGIC, K-MEANS, FUZZY C-MEANS AND BAYESIAN METHODS 4.1. INTRODUCTION This chapter includes implementation and testing of the student s academic performance evaluation to achieve the objective(s)
More informationClustering and Dissimilarity Measures. Clustering. Dissimilarity Measures. Cluster Analysis. Perceptually-Inspired Measures
Clustering and Dissimilarity Measures Clustering APR Course, Delft, The Netherlands Marco Loog May 19, 2008 1 What salient structures exist in the data? How many clusters? May 19, 2008 2 Cluster Analysis
More information7/19/09 A Primer on Clustering: I. Models and Algorithms 1
7/19/9 A Primer on Clustering: I. Models and Algorithms 1 7/19/9 A Primer on Clustering: I. Models and Algorithms 2 7/19/9 A Primer on Clustering: I. Models and Algorithms 3 A Primer on Clustering : I.Models
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationCSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University 09/25/2017 Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10.
More informationSupervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationNon-convex Multi-objective Optimization
Non-convex Multi-objective Optimization Multi-objective Optimization Real-world optimization problems usually involve more than one criteria multi-objective optimization. Such a kind of optimization problems
More informationUniversity of Florida CISE department Gator Engineering. Clustering Part 5
Clustering Part 5 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville SNN Approach to Clustering Ordinary distance measures have problems Euclidean
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationSimplicial Global Optimization
Simplicial Global Optimization Julius Žilinskas Vilnius University, Lithuania September, 7 http://web.vu.lt/mii/j.zilinskas Global optimization Find f = min x A f (x) and x A, f (x ) = f, where A R n.
More informationMachine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
More informationFuzzy Expert Systems Lecture 8 (Fuzzy Systems)
Fuzzy Expert Systems Lecture 8 (Fuzzy Systems) Soft Computing is an emerging approach to computing which parallels the remarkable ability of the human mind to reason and learn in an environment of uncertainty
More informationMethods for Intelligent Systems
Methods for Intelligent Systems Lecture Notes on Clustering (II) Davide Eynard eynard@elet.polimi.it Department of Electronics and Information Politecnico di Milano Davide Eynard - Lecture Notes on Clustering
More informationA new fuzzy set merging technique using inclusion based fuzzy clustering
A new fuzzy set merging technique using inclusion based fuzzy clustering Nefti Meziani, S, Kaymak, U and Oussalah, M http://dx.doi.org/10.1109/tfuzz.2007.902011 Title Authors Type URL A new fuzzy set merging
More informationMarket basket analysis
Market basket analysis Find joint values of the variables X = (X 1,..., X p ) that appear most frequently in the data base. It is most often applied to binary-valued data X j. In this context the observations
More informationCluster Tendency Assessment for Fuzzy Clustering of Incomplete Data
EUSFLAT-LFA 2011 July 2011 Aix-les-Bains, France Cluster Tendency Assessment for Fuzzy Clustering of Incomplete Data Ludmila Himmelspach 1 Daniel Hommers 1 Stefan Conrad 1 1 Institute of Computer Science,
More informationCHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES
70 CHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES 3.1 INTRODUCTION In medical science, effective tools are essential to categorize and systematically
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationA New Fuzzy Neural System with Applications
A New Fuzzy Neural System with Applications Yuanyuan Chai 1, Jun Chen 1 and Wei Luo 1 1-China Defense Science and Technology Information Center -Network Center Fucheng Road 26#, Haidian district, Beijing
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim,
More informationWorking with Unlabeled Data Clustering Analysis. Hsiao-Lung Chan Dept Electrical Engineering Chang Gung University, Taiwan
Working with Unlabeled Data Clustering Analysis Hsiao-Lung Chan Dept Electrical Engineering Chang Gung University, Taiwan chanhl@mail.cgu.edu.tw Unsupervised learning Finding centers of similarity using
More informationANALYSIS AND REASONING OF DATA IN THE DATABASE USING FUZZY SYSTEM MODELLING
ANALYSIS AND REASONING OF DATA IN THE DATABASE USING FUZZY SYSTEM MODELLING Dr.E.N.Ganesh Dean, School of Engineering, VISTAS Chennai - 600117 Abstract In this paper a new fuzzy system modeling algorithm
More informationIndex. ,a-possibility efficient solution, programming, 275
Index,a-possibility efficient solution, 196 0-1 programming, 275 aggregation, 31 of fuzzy number, 55 operator, 37, 45 agreement-discordance principle, 61 agricultural economics, 205 Arrow's impossibility
More informationEnsemble methods in machine learning. Example. Neural networks. Neural networks
Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you
More informationUniversity of Florida CISE department Gator Engineering. Clustering Part 2
Clustering Part 2 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Partitional Clustering Original Points A Partitional Clustering Hierarchical
More informationFEATURE EXTRACTION USING FUZZY RULE BASED SYSTEM
International Journal of Computer Science and Applications, Vol. 5, No. 3, pp 1-8 Technomathematics Research Foundation FEATURE EXTRACTION USING FUZZY RULE BASED SYSTEM NARENDRA S. CHAUDHARI and AVISHEK
More informationHARD, SOFT AND FUZZY C-MEANS CLUSTERING TECHNIQUES FOR TEXT CLASSIFICATION
HARD, SOFT AND FUZZY C-MEANS CLUSTERING TECHNIQUES FOR TEXT CLASSIFICATION 1 M.S.Rekha, 2 S.G.Nawaz 1 PG SCALOR, CSE, SRI KRISHNADEVARAYA ENGINEERING COLLEGE, GOOTY 2 ASSOCIATE PROFESSOR, SRI KRISHNADEVARAYA
More informationWorkload Characterization Techniques
Workload Characterization Techniques Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu These slides are available on-line at: http://www.cse.wustl.edu/~jain/cse567-08/
More informationContents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited.
page v Preface xiii I Basics 1 1 Optimization Models 3 1.1 Introduction... 3 1.2 Optimization: An Informal Introduction... 4 1.3 Linear Equations... 7 1.4 Linear Optimization... 10 Exercises... 12 1.5
More informationCSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10. Cluster
More informationCompact and Transparent Fuzzy Models and Classifiers Through Iterative Complexity Reduction
516 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 9, NO. 4, AUGUST 2001 Compact and Transparent Fuzzy Models and Classifiers Through Iterative Complexity Reduction Hans Roubos, Student Member, IEEE, and Magne
More informationConnected components - 1
Connected Components Basic definitions Connectivity, Adjacency, Connected Components Background/Foreground, Boundaries Run-length encoding Component Labeling Recursive algorithm Two-scan algorithm Chain
More informationECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
More informationINF4820 Algorithms for AI and NLP. Evaluating Classifiers Clustering
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Murhaf Fares & Stephan Oepen Language Technology Group (LTG) September 27, 2017 Today 2 Recap Evaluation of classifiers Unsupervised
More informationPredicting Porosity through Fuzzy Logic from Well Log Data
International Journal of Petroleum and Geoscience Engineering (IJPGE) 2 (2): 120- ISSN 2289-4713 Academic Research Online Publisher Research paper Predicting Porosity through Fuzzy Logic from Well Log
More informationSYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
More information12 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 1, FEBRUARY An On-Line Self-Constructing Neural Fuzzy Inference Network and Its Applications
12 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 1, FEBRUARY 1998 An On-Line Self-Constructing Neural Fuzzy Inference Network Its Applications Chia-Feng Juang Chin-Teng Lin Abstract A self-constructing
More informationECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
More informationClustering. Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationEFFICIENT CLUSTERING WITH FUZZY ANTS
EFFICIENT CLUSTERING WITH FUZZY ANTS S. SCHOCKAERT, M. DE COCK, C. CORNELIS AND E. E. KERRE Fuzziness and Uncertainty Modelling Research Unit, Department of Applied Mathematics and Computer Science, Ghent
More informationCluster analysis. Agnieszka Nowak - Brzezinska
Cluster analysis Agnieszka Nowak - Brzezinska Outline of lecture What is cluster analysis? Clustering algorithms Measures of Cluster Validity What is Cluster Analysis? Finding groups of objects such that
More informationLesson 3. Prof. Enza Messina
Lesson 3 Prof. Enza Messina Clustering techniques are generally classified into these classes: PARTITIONING ALGORITHMS Directly divides data points into some prespecified number of clusters without a hierarchical
More informationSimilarity-Driven Cluster Merging Method for Unsupervised Fuzzy Clustering
Similarity-Driven Cluster Merging Method for Unsupervised Fuzzy Clustering Xuejian Xiong, Kian Lee Tan Singapore-MIT Alliance E4-04-10, 4 Engineering Drive 3 Singapore 117576 Abstract In this paper, a
More informationINF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
More informationCse634 DATA MINING TEST REVIEW. Professor Anita Wasilewska Computer Science Department Stony Brook University
Cse634 DATA MINING TEST REVIEW Professor Anita Wasilewska Computer Science Department Stony Brook University Preprocessing stage Preprocessing: includes all the operations that have to be performed before
More informationGene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
More informationFUZZY INFERENCE SYSTEMS
CHAPTER-IV FUZZY INFERENCE SYSTEMS Fuzzy inference is the process of formulating the mapping from a given input to an output using fuzzy logic. The mapping then provides a basis from which decisions can
More informationFuzzy Logic Controller
Fuzzy Logic Controller Debasis Samanta IIT Kharagpur dsamanta@iitkgp.ac.in 23.01.2016 Debasis Samanta (IIT Kharagpur) Soft Computing Applications 23.01.2016 1 / 34 Applications of Fuzzy Logic Debasis Samanta
More informationHierarchical clustering
Hierarchical clustering Based in part on slides from textbook, slides of Susan Holmes December 2, 2012 1 / 1 Description Produces a set of nested clusters organized as a hierarchical tree. Can be visualized
More informationClustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
More informationECM A Novel On-line, Evolving Clustering Method and Its Applications
ECM A Novel On-line, Evolving Clustering Method and Its Applications Qun Song 1 and Nikola Kasabov 2 1, 2 Department of Information Science, University of Otago P.O Box 56, Dunedin, New Zealand (E-mail:
More informationSGN (4 cr) Chapter 11
SGN-41006 (4 cr) Chapter 11 Clustering Jussi Tohka & Jari Niemi Department of Signal Processing Tampere University of Technology February 25, 2014 J. Tohka & J. Niemi (TUT-SGN) SGN-41006 (4 cr) Chapter
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationCT79 SOFT COMPUTING ALCCS-FEB 2014
Q.1 a. Define Union, Intersection and complement operations of Fuzzy sets. For fuzzy sets A and B Figure Fuzzy sets A & B The union of two fuzzy sets A and B is a fuzzy set C, written as C=AUB or C=A OR
More informationPattern Classification Algorithms for Face Recognition
Chapter 7 Pattern Classification Algorithms for Face Recognition 7.1 Introduction The best pattern recognizers in most instances are human beings. Yet we do not completely understand how the brain recognize
More informationChapter DM:II. II. Cluster Analysis
Chapter DM:II II. Cluster Analysis Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster Analysis DM:II-1
More informationUnsupervised Learning. Andrea G. B. Tettamanzi I3S Laboratory SPARKS Team
Unsupervised Learning Andrea G. B. Tettamanzi I3S Laboratory SPARKS Team Table of Contents 1)Clustering: Introduction and Basic Concepts 2)An Overview of Popular Clustering Methods 3)Other Unsupervised
More informationMachine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning
More informationCHAPTER 3 FUZZY RELATION and COMPOSITION
CHAPTER 3 FUZZY RELATION and COMPOSITION Crisp relation! Definition (Product set) Let A and B be two non-empty sets, the prod uct set or Cartesian product A B is defined as follows, A B = {(a, b) a A,
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Slides From Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Slides From Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining
More informationCHAPTER 3 FUZZY INFERENCE SYSTEM
CHAPTER 3 FUZZY INFERENCE SYSTEM Fuzzy inference is the process of formulating the mapping from a given input to an output using fuzzy logic. There are three types of fuzzy inference system that can be
More informationImage Segmentation. Selim Aksoy. Bilkent University
Image Segmentation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Examples of grouping in vision [http://poseidon.csd.auth.gr/lab_research/latest/imgs/s peakdepvidindex_img2.jpg]
More information