Kernel PCA in application to leakage detection in drinking water distribution system
|
|
- Cecilia Clarke
- 5 years ago
- Views:
Transcription
1 Kernel PCA in application to leakage detection in drinking water distribution system Adam Nowicki, Michał Grochowski Faculty of Electrical and Control Engineering, Gdansk University of Technology, Narutowicza str. 11/12, Gdansk, Poland Abstract. Monitoring plays an important role in advanced control of complex dynamic systems. Precise information about system s behaviour, including faults detection, enables efficient control. Proposed method- Kernel Principal Component Analysis (KPCA), a representative of machine learning, skilfully takes full advantage of the well known PCA method and extends its application to nonlinear case. The paper explains the general idea of KPCA and provides an example of how to utilize it for fault detection problem. The efficiency of described method is presented for application of leakage detection in drinking water systems, representing a complex and distributed dynamic system of a large scale. Simulations for Chojnice town show promising results of detecting and even localising the leakages, using limited number of measuring points. Keywords: machine learning, kernel PCA, fault detection, monitoring, water leakage detection. 1 Introduction Several studies aims at estimating the losses in drinking water distribution systems (DWDS). Even though they differ with respect to the measurement methods and hence, are difficult to compare, the results are always alarming; European Commission studies show that they can be as high as 50% in certain areas of Europe. The losses can be considered as a difference between the volume of the water delivered to the system and the volume of authorized consumption (nonrevenue water-nrw). The World Bank estimates the worldwide NRW volume to be 48.6 billion m 3 /year [1] - most of it is caused by leakages. The most popular approach for detecting the leakages is acoustic based method [2-3]; overview of other methods can be found in [4-5]. The approach presented in this paper makes use of the fact that the appearance of the leakage can be discovered through analysis of flow and pressure measurements.
2 2 Adam Nowicki, Michał Grochowski Water supply networks usually occupy large territories and are often subject to local disturbances which have limited effect on the remaining part of the network. This motivates building a number of local models rather than a single model of the entire network. The typical quantities measured are flows and pressures. The place the measurements are taken has an important impact on the efficiency of the monitoring system. In order to reduce the cost of the whole system it is desirable to deploy the instruments in a concentrated manner around pipe junctions (called nodes), preferably with as many pipes crossing as possible. Then, for a node with pipes, 1 measurements are available: flows and a pressure. A model of such a node serves as a local model. A variety of methods for fault detection can be applied for leakage detection problem. An extensive review of most common approaches can be found in [6]. Water distribution system is dynamic, complex, nonlinear system with varying parameters. Clearly, in this case the quantitative modelling is a very demanding task, while there is no straightforward solution for qualitative approach. Moreover, the values of flows and pressure measured in a real-life networks at a given node are proven to be highly repeatable on the daily basis with a predictable variations depending on the season. During the leakage the relationship between measurements is disturbed thus providing a fault symptom. These aspects motivates the choice of data-driven approach for a problem of leakage detection. This paper presents the results of employing the Kernel Principal Component Analysis to this problem. Results of application of other data-driven approaches for the leakage detection can be found in [5],[7]. 2 Data-driven approach for novelty detection using Kernel PCA Consider a single row vector of a size 1 containing a set of measurements taken at the same time, denoted as : (1) This vector belongs to -dimensional space called input space. The measured signals are assumed to be chosen so that the vector determines operational state of the process, but not necessarily uniquely. It is sufficient that for any two measurements:, (2) where and correspond to data collected during normal and faulty state of the process, respectively. This means that a single vector is capable of carrying a symptom of the fault. When dealing with data-driven methods for a fault detection, there are two approaches to determine whether the fault has occurred: novelty detection and classification. This paper presents solution based on novelty detection, where
3 Kernel PCA in application to leakage detection in drinking water distribution system 3 the model is built using the data from set only a training set with a number of data points significantly larger than the dimension of an input space and covering all operational states of the process. Then, the task of fault detection can be reduced to the problem of finding a decision boundary in -dimensional input space that tightly encloses the training set. Hence, when previously unknown data is presented, the fault detection system is able to separate ordinary from novel patterns. If the data follows a continues linear pattern, PCA is a method of choice. This purely statistical method uses hypershpere as a decision boundary [8]. Unfortunately, most of real world applications involves dealing with non-linear patterns. A remedy to this might be VQPCA: it uses a number of local PCA models which are built using Voronoi scheme. However, its application is restricted to the cases where pattern can be approximated with piecewise linear patterns and no determinism is required. A relatively new method that does not suffer from these drawbacks is the Kernel PCA, introduced in [9]. It can be considered as a non-linear extension of PCA that combines multivariate analysis and machine learning. Instead of looking for a linear relationship between the variables in the input space, all measurements are mapped into a higher dimensional feature space through a non-linear mapping :, 1,2, (3) A subspace is identified within the feature space where PCA is used. Linear patterns detected in this space corresponds to non-linear patterns in the input space. The desirable size of is such that it allows to capture only the general pattern of the data; normally is of a higher dimension than. In a classical approach operations in large-dimensional spaces yields considerable workload since each vector is represented by a number of coordinates. Kernel PCA, which represents a group of so-called kernel methods, solves this problem using the kernel trick described in [10]. For any algorithm which operates exclusively on inner product between data vectors, it is possible to substitute each occurrence of inner product with its kernel representation,, (4) Inner product can be interpreted as a similarity measure between data points: if the angle between two different vectors is small it means that both data points follows the same linear pattern. A value of, is calculated using chosen kernel function, which operates on data in input space, but corresponds to the inner product between data in, thus allowing to detect linear patterns there. This means that there is no need to carry out the mapping from the input space into the feature space, explicitly. Moreover, neither coordinates of the, nor even mapping is needed to be known - from this point of view it is the kernel function that defines this mapping. The choice of a proper function can be motivated by specific domain knowledge this enables to incorporate heuristics into the method. In order to check if new data follows the pattern discovered in a training set, a mechanism
4 4 Adam Nowicki, Michał Grochowski based on the reconstruction error may be used [8],[11]. This solution assumes that a subspace of feature space found during the training is suitable only for the data similar to the training set. This means that for such a data, during the mapping : minimum information is lost and gives almost the same result as mapping :, thus not producing large reconstruction error. 3 Kernel PCA model Application of the Kernel PCA method for monitoring purposes is a two-step process: in the first step the model is built and then, in the second step this model is used to determine the status of the system based on the latest measurements. In order to simplify the problem it is assumed that the model is non-adaptive which means that training set remains constant and therefore model is built only once. Let be the matrix containing normalized training set with data points,1,2 given as row vectors in the input space : (5), Since this is novelty detection it is assumed that for 1,2 (data set represents only normal operating states). In kernel methods all the information about the data is stored in Kernel matrix which contains the value of the kernel function, calculated for each pair of vectors from. For gauss function:,, exp 2, 0 (6) The value of the free parameter is chosen empirically. Since 1 and, only elements above the diagonal need to be computed, which means that computation of Kernel matrix of [ size requires evaluations of the kernel function. Classic PCA requires that data is normalized, i.e. 0 for 1,2,. This is also the case when using Kernel PCA, but since data is expressed in terms of inner product, the normalization is applied indirectly, through Kernel matrix. Each element of normalized Kernel matrix can be expressed in terms of : 1 1 1, A mapping which takes into account that the centre of the mass of the training set is moved to the origin of the coordinate system in the feature space, is associated with centring procedure given in (7). (7)
5 Kernel PCA in application to leakage detection in drinking water distribution system 5 Normally, when applying classical PCA to the linear problems, eigenvectors of the covariance matrix are searched for, since they define principal components. In Kernel PCA the algorithm is applied to data in the feature space, so the primal PCA problem could be solved by finding eigenvectors of, only the mapped data points are not available explicitly. This problem is solved with different version of PCA, called dual PCA, which allows to compute eigenvectors of using by: (8) where is -th eigenvalue associated with -th eigenvector of given as a column vector. Although in (8) is not available explicitly, it will be possible to use in this form later on. It is worth noting that with primal PCA at most eigenvectors for the covariance matrix can be evaluated, while in Kernel PCA there can be evaluated as many as eigenvectors that spans -dimensional feature space. Since it is always possible to construct a single hyperplane consisting of any points in -dimensional space, thus it is possible to find a linear relationship between all mapped points; however, this might lead to overfitting. Therefore only eigenvectors corresponding to largest eigenvalues are used, resulting in the subspace that captures the general pattern in the data. The value of is chosen empirically. These eigenvalues are stored as column vectors in matrix. Having a model defined, it is possible to evaluate a reconstruction error. For a new data point, this error, denoted as, can be regarded as the squared distance between the exact mapping of into the feature space and its projection onto the chosen subspace (Fig. 1). Let and denote PC scores of associated with the feature space and reduced feature space, respectively. Since principal components originates from the origin of coordinate system, using Pythagoras theorem: (9) The term is a distance of the from the origin of the coordinates in the feature space and can be calculated using inner product calculation:,, 2,, 1 2, The PC score of the test point in the reduced feature space is equal to projection of its image onto the eigenvectors : (10)
6 6 Adam Nowicki, Michał Grochowski representing Fig. 1. Geometrical interpretation of the reconstruction error. Λ (11) where and Λ contain first eigenvectors and eigenvalues of, respectively. The term can be calculated using kernel function with correction that takes into account centring in the feature space resulting in vector :,, 1 1, 1 Using (10) and combination of (11) and (12) the expression for reconstruction error in (9) can be calculated. The value of the error is always between zero and a value close to one. An error exceeding some selected maximal value of the reconstruction error indicates that the test point is not following the pattern defined by the training set. This means that serves as a decision boundary (Fig. 2) that enables to classify the current state of the system: (13) a) b) (12) Fig. 2. a) Reconstruction error for a test set containing measurements from normal state (green) and leakage (red) b) The same test set in the input space : data points from the leakage (red) are separated from data points from normal operation (green) by a decision boundary (bold). Model built from the training set (blue)
7 Kernel PCA in application to leakage detection in drinking water distribution system 7 4 Chojnice case study In order to prove the efficiency of the presented method, a number of experiments has been carried out. Each single experiment aimed at answering the following question: Is it possible to detect a leakage that occurred in node A basing entirely on measurements taken at node B?. All the experiments were based on the measurements provided by Epanet with simulations carried out using a calibrated model of a real network. This network has been divided into a number of regions that are partly independent in the sense of leakage discovery as described later. Training set corresponds to measurements collected during 6 days every 5 minutes, while the test set was collected during the 7th day, with the leakage being simulated in the middle of the day. For each of the monitoring nodes a KPCA model was built, with kernel width 1 and a size of the feature space set to 70. Values of these parameters were chosen as a result of an analysis. Since the data is normalized and the training set has the same size yielding the same dimension of the full feature space, the result of applying common parameters for all nodes provides satisfactory outcome, however this leaves the place for further improvements. The third parameter - maximal allowed reconstruction error was chosen so that 99,9% of training set is considered to be in the normal state. In order to check what is the monitoring potential of each node, results from a set of following experiments were brought together: for a fixed monitored node leakages of a chosen size were consecutively simulated in adjoining nodes. This has provided a monitoring range map of a chosen node. Some nodes presents better performance than others. This is caused by the diverse effect of a certain leakage on the measurements. Simulations carried out proved that there are several factors that have strong influence on the performance of the node: The position of monitored node in respect to the leakage node. As explained earlier a leakage causes disturbance within a limited range. This range is different for each node since compensation of adjoining nodes for water loss is never the same (Fig. 3). Furthermore, it turns out that the large supply pipes have the significant ability to mask the leakage symptom as they can easily deliver increased amount of water without affecting its own state. This motivates dividing the network into separate regions with the arrangement of supply pipes taken into account [7]. Size of the leakage. The influence of the leakage on the measured variables is in general proportional to the relative amount of water lost in a leakage. Larger leakages cause stronger disturbances and as a result larger area is affected. This explains why a monitoring node performs much better in case of larger leakages (Fig 3c). The time of a day. The amount of the water flowing through the pipes changes throughout the day with the least water being supplied at night. Although the value of the pressure is generally lower at this time of the day resulting in less water being
8 8 Adam Nowicki, Michał Grochowski a) b) c) monitoring node leakages detectable from monitoring node leakages undetectable from monitoring node not covered by the experiment Fig. 3. a) (top) leakages simulated in different nodes have different impact on the reconstruction error E(t) evaluated for monitored node, (bottom) results of the experiment presented on the map; b) poor (top) and good (bottom) candidate for monitoring node simulated leakage 2 m /h, c) the range of detection depends on the size of the leakage: 1.5m /h (top), 12m /h (bottom) lost, it is much easier to observe the leakage as it has relatively stronger impact on the network. This means that the sensitivity of the monitoring node changes throughout the day. Even though the area covered by the detection system differs for each monitoring node, they share some common properties: the area is always continuous and concentrated around monitoring node in a non-symmetric manner. The monitored nodes should be chosen carefully as there might be a significant difference in performance between a good candidate and a poor one (Fig. 3b). Setting a number of monitoring nodes provides a possibility to monitor an entire network. Since the area monitored by each node is subject to change depending on the leakage size, the number of required nodes heavily depends on the expected sensitivity of the system: if one needs to detect and to precisely localise even small leakages this requires setting a large number of monitoring nodes close to each other. The possibility to detect the leakages only in close neighbourhood of monitored node extends application of the method to localization of the potential fault (Fig. 4). If for
9 Kernel PCA in application to leakage detection in drinking water distribution system 9 a) b) c) Fig. 4. The idea of leakage localisation using local models: a) an example of the detection range for three monitored nodes with simulated leakages Q=2 m /h, b) monitored nodes marked in blue, place of the simulated leakage marked in red c) values of reconstruction error in monitored nodes for situation given in b). a single node current value of reconstruction error exceeds, it indicates that a leakage occurred in some close neighbourhood. If several nodes report an error at the same time this suggest that a larger leakage occurred somewhere in between. 5 Conclusions and future work The paper describes an approach to detect the leakages in water distribution system using Kernel PCA method with a limited number of measurements. The arrangement of the measuring points is determined through simulations and heuristics in order to ensure efficient fault detecting abilities of local KPCA models. By adjusting the number of controlled nodes, one can set a sensitivity of the system to maintain economic level of real losses. The usage of KPCA, instead of conventional PCA, reduces number of false alarms and prevents model conservatism. The methodology was verified on calibrated model and data of Chojnice town (Northern Poland). At this stage of the research localisation of the leakages is supervised by a man, however promising results completing the process of automatic fault detection and localisation are obtained by the paper Authors with usage of Self Organising Maps. Other faults (such as pump or valve breakdown, water contamination) can be identified and isolated using this approach. Moreover, the method is rather of a generic nature, hence might be transferred into similar systems e.g. pipeline systems,
10 10 Adam Nowicki, Michał Grochowski telecommunication systems, power systems etc, known as a network systems. Optimal and adaptive parameters of KPCA models selecting predispose the method to real time diagnostic and control systems e.g. Fault Tolerant Model Predictive Control. References 1. Thornton, J., Sturm, R., Kunkel, G.: Water Loss Control. The McGraw-Hill Companies, New York (2008) 2. Water Audits and Loss Control Programs - Manual of Water Supply Practices, M36. American Water Works Association, (2009). 3. Jin, Y., Yumei, W., Ping, L.: Leak Acoustic Detection in Water Distribution Pipeline, Proceedings of the 7th World Congress on Intelligent Control and Automation, pp (2008) 4. Xiao-Li, C., Chao-Yuan, J., Si-Yuan, G., Leakage monitoring and locating method of water supply pipe network. Proceedings of the Seventh International Conference on Machine Learning and Cybernetics, pp , (2008). 5. Mashford, J., Silva, D.D., Marney, D., Burn, S.: An approach to leak detection in pipe networks using analysis of monitored pressure values by support vector machine. In: 2009 Third International Conference on Network and System Security, pp (2009). 6. Venkatasubramanian, V., Rengaswamy, R., Yin, K., Kavuri, S.: A review of process fault detection and diagnosis. Part I, II, III. Computers and Chemical Engineering, 27, pp , (2003) 7. Duzinkiewicz, K., Borowa, A., Mazur, K., Grochowski, M., Brdys, M.A., Jezior, K.: Leakage Detection and Localization in Drinking Water Distribuition Networks by MultiRegional PCA. Studies in Informatics and Control, 17 (2), pp (2008) 8. Jackson, J.E., Mudholkar, G.,: Control procedures for residuals associated with principal component analysis. Technometrics, 21, pp (1979). 9. Schölkopf, B., Smola, A.J., Müller, K.R,: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10, pp (1998). 10. Aizerman, M., Braverman, E., Rozonoer, L.: Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25, pp (1964). 11. Hoffman, H.: Kernel PCA for novelty detection. Pattern Recognition, 40, pp (2007).
DATA DRIVEN MODELS FOR FAULT DETECTION USING KERNEL PCA: A WATER DISTRIBUTION SYSTEM CASE STUDY
Int. J. Appl. Math. Comput. Sci., 202, Vol. 22, No. 4, 939 949 DOI: 0.2478/v0006-02-0070- DATA DRIVEN MODELS FOR FAULT DETECTION USING KERNEL PCA: A WATER DISTRIBUTION SYSTEM CASE STUDY ADAM NOWICKI, MICHAŁ
More informationKernel PCA in nonlinear visualization of a healthy and a faulty planetary gearbox data
Kernel PCA in nonlinear visualization of a healthy and a faulty planetary gearbox data Anna M. Bartkowiak 1, Radoslaw Zimroz 2 1 Wroclaw University, Institute of Computer Science, 50-383, Wroclaw, Poland,
More informationBagging for One-Class Learning
Bagging for One-Class Learning David Kamm December 13, 2008 1 Introduction Consider the following outlier detection problem: suppose you are given an unlabeled data set and make the assumptions that one
More informationThe Curse of Dimensionality
The Curse of Dimensionality ACAS 2002 p1/66 Curse of Dimensionality The basic idea of the curse of dimensionality is that high dimensional data is difficult to work with for several reasons: Adding more
More informationLeave-One-Out Support Vector Machines
Leave-One-Out Support Vector Machines Jason Weston Department of Computer Science Royal Holloway, University of London, Egham Hill, Egham, Surrey, TW20 OEX, UK. Abstract We present a new learning algorithm
More informationApplication of Support Vector Machine Algorithm in Spam Filtering
Application of Support Vector Machine Algorithm in E-Mail Spam Filtering Julia Bluszcz, Daria Fitisova, Alexander Hamann, Alexey Trifonov, Advisor: Patrick Jähnichen Abstract The problem of spam classification
More informationLocal Linear Approximation for Kernel Methods: The Railway Kernel
Local Linear Approximation for Kernel Methods: The Railway Kernel Alberto Muñoz 1,JavierGonzález 1, and Isaac Martín de Diego 1 University Carlos III de Madrid, c/ Madrid 16, 890 Getafe, Spain {alberto.munoz,
More informationFAULT DIAGNOSIS BASED ON MULTI-SCALE CLASSIFICATION USING KERNEL FISHER DISCRIMINANT ANALYSIS AND GAUSSIAN MIXTURE MODEL AND K-NEAREST NEIGHBOR METHOD
Universiti Kebangsaan Malaysia FAULT DIAGNOSIS BASED ON MULTI-SCALE CLASSIFICATION USING KERNEL FISHER DISCRIMINANT ANALYSIS AND GAUSSIAN MIXTURE MODEL AND K-NEAREST NEIGHBOR METHOD NORAZWAN M. NOR*, MOHD
More informationKernel Methods & Support Vector Machines
& Support Vector Machines & Support Vector Machines Arvind Visvanathan CSCE 970 Pattern Recognition 1 & Support Vector Machines Question? Draw a single line to separate two classes? 2 & Support Vector
More informationThe Pre-Image Problem and Kernel PCA for Speech Enhancement
The Pre-Image Problem and Kernel PCA for Speech Enhancement Christina Leitner and Franz Pernkopf Signal Processing and Speech Communication Laboratory, Graz University of Technology, Inffeldgasse 6c, 8
More informationTensor Based Approaches for LVA Field Inference
Tensor Based Approaches for LVA Field Inference Maksuda Lillah and Jeff Boisvert The importance of locally varying anisotropy (LVA) in model construction can be significant; however, it is often ignored
More informationAdvanced Machine Learning Practical 1: Manifold Learning (PCA and Kernel PCA)
Advanced Machine Learning Practical : Manifold Learning (PCA and Kernel PCA) Professor: Aude Billard Assistants: Nadia Figueroa, Ilaria Lauzana and Brice Platerrier E-mails: aude.billard@epfl.ch, nadia.figueroafernandez@epfl.ch
More informationKernel SVM. Course: Machine Learning MAHDI YAZDIAN-DEHKORDI FALL 2017
Kernel SVM Course: MAHDI YAZDIAN-DEHKORDI FALL 2017 1 Outlines SVM Lagrangian Primal & Dual Problem Non-linear SVM & Kernel SVM SVM Advantages Toolboxes 2 SVM Lagrangian Primal/DualProblem 3 SVM LagrangianPrimalProblem
More informationCOMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
More informationDimension Reduction CS534
Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationTransductive Learning: Motivation, Model, Algorithms
Transductive Learning: Motivation, Model, Algorithms Olivier Bousquet Centre de Mathématiques Appliquées Ecole Polytechnique, FRANCE olivier.bousquet@m4x.org University of New Mexico, January 2002 Goal
More informationLinear Methods for Regression and Shrinkage Methods
Linear Methods for Regression and Shrinkage Methods Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 Linear Regression Models Least Squares Input vectors
More informationFeature Selection Using Principal Feature Analysis
Feature Selection Using Principal Feature Analysis Ira Cohen Qi Tian Xiang Sean Zhou Thomas S. Huang Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign Urbana,
More informationLinear methods for supervised learning
Linear methods for supervised learning LDA Logistic regression Naïve Bayes PLA Maximum margin hyperplanes Soft-margin hyperplanes Least squares resgression Ridge regression Nonlinear feature maps Sometimes
More informationAnomaly Detection on Data Streams with High Dimensional Data Environment
Anomaly Detection on Data Streams with High Dimensional Data Environment Mr. D. Gokul Prasath 1, Dr. R. Sivaraj, M.E, Ph.D., 2 Department of CSE, Velalar College of Engineering & Technology, Erode 1 Assistant
More informationData Mining: Concepts and Techniques. Chapter 9 Classification: Support Vector Machines. Support Vector Machines (SVMs)
Data Mining: Concepts and Techniques Chapter 9 Classification: Support Vector Machines 1 Support Vector Machines (SVMs) SVMs are a set of related supervised learning methods used for classification Based
More informationBasics of Multivariate Modelling and Data Analysis
Basics of Multivariate Modelling and Data Analysis Kurt-Erik Häggblom 9. Linear regression with latent variables 9.1 Principal component regression (PCR) 9.2 Partial least-squares regression (PLS) [ mostly
More informationData-driven fault detection with process topology for fault identification
Preprints of the 19th World Congress The International Federation of Automatic Control Data-driven fault detection with process topology for fault identification Brian S. Lindner*. Lidia Auret** * Department
More informationPerformance Degradation Assessment and Fault Diagnosis of Bearing Based on EMD and PCA-SOM
Performance Degradation Assessment and Fault Diagnosis of Bearing Based on EMD and PCA-SOM Lu Chen and Yuan Hang PERFORMANCE DEGRADATION ASSESSMENT AND FAULT DIAGNOSIS OF BEARING BASED ON EMD AND PCA-SOM.
More informationLinear Discriminant Analysis in Ottoman Alphabet Character Recognition
Linear Discriminant Analysis in Ottoman Alphabet Character Recognition ZEYNEB KURT, H. IREM TURKMEN, M. ELIF KARSLIGIL Department of Computer Engineering, Yildiz Technical University, 34349 Besiktas /
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationPCA and KPCA algorithms for Face Recognition A Survey
PCA and KPCA algorithms for Face Recognition A Survey Surabhi M. Dhokai 1, Vaishali B.Vala 2,Vatsal H. Shah 3 1 Department of Information Technology, BVM Engineering College, surabhidhokai@gmail.com 2
More informationKernel Methods and Visualization for Interval Data Mining
Kernel Methods and Visualization for Interval Data Mining Thanh-Nghi Do 1 and François Poulet 2 1 College of Information Technology, Can Tho University, 1 Ly Tu Trong Street, Can Tho, VietNam (e-mail:
More informationDesign of Fault Diagnosis System of FPSO Production Process Based on MSPCA
2009 Fifth International Conference on Information Assurance and Security Design of Fault Diagnosis System of FPSO Production Process Based on MSPCA GAO Qiang, HAN Miao, HU Shu-liang, DONG Hai-jie ianjin
More informationCSC 411: Lecture 14: Principal Components Analysis & Autoencoders
CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Raquel Urtasun & Rich Zemel University of Toronto Nov 4, 2015 Urtasun & Zemel (UofT) CSC 411: 14-PCA & Autoencoders Nov 4, 2015 1 / 18
More informationChemometrics. Description of Pirouette Algorithms. Technical Note. Abstract
19-1214 Chemometrics Technical Note Description of Pirouette Algorithms Abstract This discussion introduces the three analysis realms available in Pirouette and briefly describes each of the algorithms
More informationBest First and Greedy Search Based CFS and Naïve Bayes Algorithms for Hepatitis Diagnosis
Best First and Greedy Search Based CFS and Naïve Bayes Algorithms for Hepatitis Diagnosis CHAPTER 3 BEST FIRST AND GREEDY SEARCH BASED CFS AND NAÏVE BAYES ALGORITHMS FOR HEPATITIS DIAGNOSIS 3.1 Introduction
More informationCSC 411: Lecture 14: Principal Components Analysis & Autoencoders
CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 14-PCA & Autoencoders 1 / 18
More informationFeature Selection Using Modified-MCA Based Scoring Metric for Classification
2011 International Conference on Information Communication and Management IPCSIT vol.16 (2011) (2011) IACSIT Press, Singapore Feature Selection Using Modified-MCA Based Scoring Metric for Classification
More informationCS6375: Machine Learning Gautam Kunapuli. Mid-Term Review
Gautam Kunapuli Machine Learning Data is identically and independently distributed Goal is to learn a function that maps to Data is generated using an unknown function Learn a hypothesis that minimizes
More informationApplying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
More informationKernel Principal Component Analysis: Applications and Implementation
Kernel Principal Component Analysis: Applications and Daniel Olsson Royal Institute of Technology Stockholm, Sweden Examiner: Prof. Ulf Jönsson Supervisor: Prof. Pando Georgiev Master s Thesis Presentation
More informationRobust Kernel Methods in Clustering and Dimensionality Reduction Problems
Robust Kernel Methods in Clustering and Dimensionality Reduction Problems Jian Guo, Debadyuti Roy, Jing Wang University of Michigan, Department of Statistics Introduction In this report we propose robust
More informationUltrasonic Multi-Skip Tomography for Pipe Inspection
18 th World Conference on Non destructive Testing, 16-2 April 212, Durban, South Africa Ultrasonic Multi-Skip Tomography for Pipe Inspection Arno VOLKER 1, Rik VOS 1 Alan HUNTER 1 1 TNO, Stieltjesweg 1,
More informationEvaluating Classifiers
Evaluating Classifiers Charles Elkan elkan@cs.ucsd.edu January 18, 2011 In a real-world application of supervised learning, we have a training set of examples with labels, and a test set of examples with
More information12 Classification using Support Vector Machines
160 Bioinformatics I, WS 14/15, D. Huson, January 28, 2015 12 Classification using Support Vector Machines This lecture is based on the following sources, which are all recommended reading: F. Markowetz.
More informationData Analysis 3. Support Vector Machines. Jan Platoš October 30, 2017
Data Analysis 3 Support Vector Machines Jan Platoš October 30, 2017 Department of Computer Science Faculty of Electrical Engineering and Computer Science VŠB - Technical University of Ostrava Table of
More informationAll lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
More informationInvariant Recognition of Hand-Drawn Pictograms Using HMMs with a Rotating Feature Extraction
Invariant Recognition of Hand-Drawn Pictograms Using HMMs with a Rotating Feature Extraction Stefan Müller, Gerhard Rigoll, Andreas Kosmala and Denis Mazurenok Department of Computer Science, Faculty of
More informationDimensionality Reduction, including by Feature Selection.
Dimensionality Reduction, including by Feature Selection www.cs.wisc.edu/~dpage/cs760 Goals for the lecture you should understand the following concepts filtering-based feature selection information gain
More information3 Feature Selection & Feature Extraction
3 Feature Selection & Feature Extraction Overview: 3.1 Introduction 3.2 Feature Extraction 3.3 Feature Selection 3.3.1 Max-Dependency, Max-Relevance, Min-Redundancy 3.3.2 Relevance Filter 3.3.3 Redundancy
More informationColor Space Projection, Feature Fusion and Concurrent Neural Modules for Biometric Image Recognition
Proceedings of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS AND CYBERNETICS, Venice, Italy, November 20-22, 2006 286 Color Space Projection, Fusion and Concurrent Neural
More informationClassification by Nearest Shrunken Centroids and Support Vector Machines
Classification by Nearest Shrunken Centroids and Support Vector Machines Florian Markowetz florian.markowetz@molgen.mpg.de Max Planck Institute for Molecular Genetics, Computational Diagnostics Group,
More informationNikolaos Tsapanos, Anastasios Tefas, Nikolaos Nikolaidis and Ioannis Pitas. Aristotle University of Thessaloniki
KERNEL MATRIX TRIMMING FOR IMPROVED KERNEL K-MEANS CLUSTERING Nikolaos Tsapanos, Anastasios Tefas, Nikolaos Nikolaidis and Ioannis Pitas Aristotle University of Thessaloniki ABSTRACT The Kernel k-means
More informationFlexibility and Robustness of Hierarchical Fuzzy Signature Structures with Perturbed Input Data
Flexibility and Robustness of Hierarchical Fuzzy Signature Structures with Perturbed Input Data B. Sumudu U. Mendis Department of Computer Science The Australian National University Canberra, ACT 0200,
More informationDiagonal Principal Component Analysis for Face Recognition
Diagonal Principal Component nalysis for Face Recognition Daoqiang Zhang,2, Zhi-Hua Zhou * and Songcan Chen 2 National Laboratory for Novel Software echnology Nanjing University, Nanjing 20093, China 2
More informationFAULT DETECTION AND ISOLATION USING SPECTRAL ANALYSIS. Eugen Iancu
FAULT DETECTION AND ISOLATION USING SPECTRAL ANALYSIS Eugen Iancu Automation and Mechatronics Department University of Craiova Eugen.Iancu@automation.ucv.ro Abstract: In this work, spectral signal analyses
More informationQualitative Multi-faults Diagnosis Based on Automated Planning II: Algorithm and Case Study
Qualitative Multi-faults Diagnosis Based on Automated Planning II: Algorithm and Case Study He-xuan Hu, Anne-lise Gehin, and Mireille Bayart Laboratoire d Automatique, Génie Informatique & Signal, UPRESA
More information6. Dicretization methods 6.1 The purpose of discretization
6. Dicretization methods 6.1 The purpose of discretization Often data are given in the form of continuous values. If their number is huge, model building for such data can be difficult. Moreover, many
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines CS 536: Machine Learning Littman (Wu, TA) Administration Slides borrowed from Martin Law (from the web). 1 Outline History of support vector machines (SVM) Two classes,
More informationFault detection with principal component pursuit method
Journal of Physics: Conference Series PAPER OPEN ACCESS Fault detection with principal component pursuit method Recent citations - Robust Principal Component Pursuit for Fault Detection in a Blast Furnace
More informationSELECTION OF A MULTIVARIATE CALIBRATION METHOD
SELECTION OF A MULTIVARIATE CALIBRATION METHOD 0. Aim of this document Different types of multivariate calibration methods are available. The aim of this document is to help the user select the proper
More informationStructural and Syntactic Pattern Recognition
Structural and Syntactic Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent
More informationECE 285 Class Project Report
ECE 285 Class Project Report Based on Source localization in an ocean waveguide using supervised machine learning Yiwen Gong ( yig122@eng.ucsd.edu), Yu Chai( yuc385@eng.ucsd.edu ), Yifeng Bu( ybu@eng.ucsd.edu
More informationFace recognition based on improved BP neural network
Face recognition based on improved BP neural network Gaili Yue, Lei Lu a, College of Electrical and Control Engineering, Xi an University of Science and Technology, Xi an 710043, China Abstract. In order
More informationPARAMETERIZATION AND SAMPLING DESIGN FOR WATER NETWORKS DEMAND CALIBRATION USING THE SINGULAR VALUE DECOMPOSITION: APPLICATION TO A REAL NETWORK
11 th International Conference on Hydroinformatics HIC 2014, New York City, USA PARAMETERIZATION AND SAMPLING DESIGN FOR WATER NETWORKS DEMAND CALIBRATION USING THE SINGULAR VALUE DECOMPOSITION: APPLICATION
More information4.12 Generalization. In back-propagation learning, as many training examples as possible are typically used.
1 4.12 Generalization In back-propagation learning, as many training examples as possible are typically used. It is hoped that the network so designed generalizes well. A network generalizes well when
More informationThe Anatomical Equivalence Class Formulation and its Application to Shape-based Computational Neuroanatomy
The Anatomical Equivalence Class Formulation and its Application to Shape-based Computational Neuroanatomy Sokratis K. Makrogiannis, PhD From post-doctoral research at SBIA lab, Department of Radiology,
More informationLocalization from Pairwise Distance Relationships using Kernel PCA
Center for Robotics and Embedded Systems Technical Report Localization from Pairwise Distance Relationships using Kernel PCA Odest Chadwicke Jenkins cjenkins@usc.edu 1 Introduction In this paper, we present
More informationSupport Vector Machines
Support Vector Machines Michael Tagare De Guzman May 19, 2012 Support Vector Machines Linear Learning Machines and The Maximal Margin Classifier In Supervised Learning, a learning machine is given a training
More informationAnalyzing Vocal Patterns to Determine Emotion Maisy Wieman, Andy Sun
Analyzing Vocal Patterns to Determine Emotion Maisy Wieman, Andy Sun 1. Introduction The human voice is very versatile and carries a multitude of emotions. Emotion in speech carries extra insight about
More informationSupervised Learning with Neural Networks. We now look at how an agent might learn to solve a general problem by seeing examples.
Supervised Learning with Neural Networks We now look at how an agent might learn to solve a general problem by seeing examples. Aims: to present an outline of supervised learning as part of AI; to introduce
More informationLearning Inverse Dynamics: a Comparison
Learning Inverse Dynamics: a Comparison Duy Nguyen-Tuong, Jan Peters, Matthias Seeger, Bernhard Schölkopf Max Planck Institute for Biological Cybernetics Spemannstraße 38, 72076 Tübingen - Germany Abstract.
More informationPrincipal Component Analysis (PCA) is a most practicable. statistical technique. Its application plays a major role in many
CHAPTER 3 PRINCIPAL COMPONENT ANALYSIS ON EIGENFACES 2D AND 3D MODEL 3.1 INTRODUCTION Principal Component Analysis (PCA) is a most practicable statistical technique. Its application plays a major role
More informationComments on the randomized Kaczmarz method
Comments on the randomized Kaczmarz method Thomas Strohmer and Roman Vershynin Department of Mathematics, University of California Davis, CA 95616-8633, USA. strohmer@math.ucdavis.edu, vershynin@math.ucdavis.edu
More informationPrediction-based diagnosis and loss prevention using qualitative multi-scale models
European Symposium on Computer Arded Aided Process Engineering 15 L. Puigjaner and A. Espuña (Editors) 2005 Elsevier Science B.V. All rights reserved. Prediction-based diagnosis and loss prevention using
More informationThe Use of Biplot Analysis and Euclidean Distance with Procrustes Measure for Outliers Detection
Volume-8, Issue-1 February 2018 International Journal of Engineering and Management Research Page Number: 194-200 The Use of Biplot Analysis and Euclidean Distance with Procrustes Measure for Outliers
More informationFace detection and recognition. Detection Recognition Sally
Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification
More informationMultiresponse Sparse Regression with Application to Multidimensional Scaling
Multiresponse Sparse Regression with Application to Multidimensional Scaling Timo Similä and Jarkko Tikka Helsinki University of Technology, Laboratory of Computer and Information Science P.O. Box 54,
More informationIMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS
IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS BOGDAN M.WILAMOWSKI University of Wyoming RICHARD C. JAEGER Auburn University ABSTRACT: It is shown that by introducing special
More informationCOSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
More informationClassification by Support Vector Machines
Classification by Support Vector Machines Florian Markowetz Max-Planck-Institute for Molecular Genetics Computational Molecular Biology Berlin Practical DNA Microarray Analysis 2003 1 Overview I II III
More informationGENDER CLASSIFICATION USING SUPPORT VECTOR MACHINES
GENDER CLASSIFICATION USING SUPPORT VECTOR MACHINES Ashwin Swaminathan ashwins@umd.edu ENEE633: Statistical and Neural Pattern Recognition Instructor : Prof. Rama Chellappa Project 2, Part (a) 1. INTRODUCTION
More informationRecognition: Face Recognition. Linda Shapiro EE/CSE 576
Recognition: Face Recognition Linda Shapiro EE/CSE 576 1 Face recognition: once you ve detected and cropped a face, try to recognize it Detection Recognition Sally 2 Face recognition: overview Typical
More informationProf. Fanny Ficuciello Robotics for Bioengineering Visual Servoing
Visual servoing vision allows a robotic system to obtain geometrical and qualitative information on the surrounding environment high level control motion planning (look-and-move visual grasping) low level
More informationData mining with Support Vector Machine
Data mining with Support Vector Machine Ms. Arti Patle IES, IPS Academy Indore (M.P.) artipatle@gmail.com Mr. Deepak Singh Chouhan IES, IPS Academy Indore (M.P.) deepak.schouhan@yahoo.com Abstract: Machine
More informationLECTURE 5: DUAL PROBLEMS AND KERNELS. * Most of the slides in this lecture are from
LECTURE 5: DUAL PROBLEMS AND KERNELS * Most of the slides in this lecture are from http://www.robots.ox.ac.uk/~az/lectures/ml Optimization Loss function Loss functions SVM review PRIMAL-DUAL PROBLEM Max-min
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationDesign and Performance Improvements for Fault Detection in Tightly-Coupled Multi-Robot Team Tasks
Design and Performance Improvements for Fault Detection in Tightly-Coupled Multi-Robot Team Tasks Xingyan Li and Lynne E. Parker Distributed Intelligence Laboratory, Department of Electrical Engineering
More informationProbabilistic Approaches
Probabilistic Approaches Chirayu Wongchokprasitti, PhD University of Pittsburgh Center for Causal Discovery Department of Biomedical Informatics chw20@pitt.edu http://www.pitt.edu/~chw20 Overview Independence
More informationTime Series Clustering Ensemble Algorithm Based on Locality Preserving Projection
Based on Locality Preserving Projection 2 Information & Technology College, Hebei University of Economics & Business, 05006 Shijiazhuang, China E-mail: 92475577@qq.com Xiaoqing Weng Information & Technology
More informationCHAPTER 4 FUZZY LOGIC, K-MEANS, FUZZY C-MEANS AND BAYESIAN METHODS
CHAPTER 4 FUZZY LOGIC, K-MEANS, FUZZY C-MEANS AND BAYESIAN METHODS 4.1. INTRODUCTION This chapter includes implementation and testing of the student s academic performance evaluation to achieve the objective(s)
More informationProgramming Exercise 7: K-means Clustering and Principal Component Analysis
Programming Exercise 7: K-means Clustering and Principal Component Analysis Machine Learning May 13, 2012 Introduction In this exercise, you will implement the K-means clustering algorithm and apply it
More informationImproving Image Segmentation Quality Via Graph Theory
International Symposium on Computers & Informatics (ISCI 05) Improving Image Segmentation Quality Via Graph Theory Xiangxiang Li, Songhao Zhu School of Automatic, Nanjing University of Post and Telecommunications,
More informationOverview Citation. ML Introduction. Overview Schedule. ML Intro Dataset. Introduction to Semi-Supervised Learning Review 10/4/2010
INFORMATICS SEMINAR SEPT. 27 & OCT. 4, 2010 Introduction to Semi-Supervised Learning Review 2 Overview Citation X. Zhu and A.B. Goldberg, Introduction to Semi- Supervised Learning, Morgan & Claypool Publishers,
More informationA Direct Evolutionary Feature Extraction Algorithm for Classifying High Dimensional Data
A Direct Evolutionary Feature Extraction Algorithm for Classifying High Dimensional Data Qijun Zhao and David Zhang Department of Computing The Hong Kong Polytechnic University Hung Hom, Kowloon, Hong
More informationSupport Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
More information2 Second Derivatives. As we have seen, a function f (x, y) of two variables has four different partial derivatives: f xx. f yx. f x y.
2 Second Derivatives As we have seen, a function f (x, y) of two variables has four different partial derivatives: (x, y), (x, y), f yx (x, y), (x, y) It is convenient to gather all four of these into
More informationDetermination of 3-D Image Viewpoint Using Modified Nearest Feature Line Method in Its Eigenspace Domain
Determination of 3-D Image Viewpoint Using Modified Nearest Feature Line Method in Its Eigenspace Domain LINA +, BENYAMIN KUSUMOPUTRO ++ + Faculty of Information Technology Tarumanagara University Jl.
More informationHsiaochun Hsu Date: 12/12/15. Support Vector Machine With Data Reduction
Support Vector Machine With Data Reduction 1 Table of Contents Summary... 3 1. Introduction of Support Vector Machines... 3 1.1 Brief Introduction of Support Vector Machines... 3 1.2 SVM Simple Experiment...
More informationA NEW VARIABLES SELECTION AND DIMENSIONALITY REDUCTION TECHNIQUE COUPLED WITH SIMCA METHOD FOR THE CLASSIFICATION OF TEXT DOCUMENTS
A NEW VARIABLES SELECTION AND DIMENSIONALITY REDUCTION TECHNIQUE COUPLED WITH SIMCA METHOD FOR THE CLASSIFICATION OF TEXT DOCUMENTS Ahmed Abdelfattah Saleh University of Brasilia, Brasil ahmdsalh@yahoo.com
More informationSupport Vector Machines
Support Vector Machines . Importance of SVM SVM is a discriminative method that brings together:. computational learning theory. previously known methods in linear discriminant functions 3. optimization
More informationRecognizing Handwritten Digits Using the LLE Algorithm with Back Propagation
Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation Lori Cillo, Attebury Honors Program Dr. Rajan Alex, Mentor West Texas A&M University Canyon, Texas 1 ABSTRACT. This work is
More informationData mining with sparse grids
Data mining with sparse grids Jochen Garcke and Michael Griebel Institut für Angewandte Mathematik Universität Bonn Data mining with sparse grids p.1/40 Overview What is Data mining? Regularization networks
More information