Local Context Selection for Outlier Ranking in Graphs with Multiple Numeric Node Attributes
|
|
|
- Claire Oliver
- 5 years ago
- Views:
Transcription
1 Local Context Selection for Outlier Ranking in Graphs with Multiple Numeric Node Attributes Patricia Iglesias, Emmanuel Müller, Oretta Irmler, Klemens Böhm International Conference on Scientific and Statistical Database Management (SSDBM 2014)
2 Attributed Graphs Complex databases: Attributed graphs age income shoe size #children... attributes graph structure Several application domains: Communication networks, co-purchased networks, social networks Bibliographic networks, biological networks Outlier mining: Fraud detection, network intrusion, data cleaning... 2
3 Problem Overview: Example attributes: product description Node sales #reviews price graph structure: nodes: products edges: co-purchased products Tolkien Books 3
4 Challenges How to define a local context for each node? How to efficiently select only the relevant attributes? How to rank each node w.r.t. graph and attribute information? Node #reviews price
5 Comparison: Outlier Mining on Attributed Graphs Algorithm Local Selection of attributes CODA [Gao 2010] CONSUB [Iglesias 2013] GoutRank [Müller 2013] ConOut Ranking Time Complexity (#attributes) O(d 2 ) O(2 d ) O(2 d ) O(d) [Gao 2010] Gao et al. On community outliers and their efficient detection in information networks" In ACM SIGKDD 2010 [Iglesias 2003] Iglesias et al. Statistical Selection of Congruent Subspaces for Mining Attributed Graphs. In IEEE ICDM [Müller 2013] Müller et al. Ranking outlier nodes in subspaces of attributed graphs In GDM at IEEE ICDE
6 Our Approach: ConOut Local vs. global Attribute projection vs. subspace selection Avoid exponential runtimes w.r.t. the number of the attributes Time complexity: Linear Ranking vs. Binary Assessment of the outlierness w.r.t. both: graph and attributes 6
7 ConOut I: Context Definition Local Context of object o: C(o), R(o) Graph Context C(o) Nodes similar w.r.t. graph structure Graph similarity based on shared nearest neighborhood (SNN): sim v, p = Adj v Adj p Adj v Adj p Adj v = p E v, p E} {v} Other local graph context definitions possible Relevant Attributes R(o)? p Node #reviews price v C(o) R(o) 7
8 ConOut II: Statistical Selection Attribute A i has significantly lower variance in C(o) than the overall database reviews sales Statistical test: Example of instantiation 2 2 H 0 : σ local = σ global 2 2 H 1 : σ local < σ global P H o is rejected H o = true α 8
9 ConOut III: Context Based Ranking Local context selection enables a high contrast between inliers and outliers Goal: Compare deviation of the attribute values and the graph density of each node to its local context C(o) Local attribute deviation (LAD(o)) Local graph density (LGD(o)) Context based score combines both: score o = LGD(o) LAD(o) R(o) 9
![Experimental Setup Synthetic data Real world data Traditional Algorithms: LOF [Breunig 2001]: attributes SOF [Aggarwal 2001]: attributes SCAN [Xiu 2007]: graph Algorithms for attributed graphs: CODA](/docs-images/88/115732282/images/10-1.jpg "[Gao 2010] GOutRank [Müller 2013] ConSub [Iglesias 2013] quality (AUC) runtime [Breunig 2001] Breunig et al. \"LOF: identifying density-based local outliers.")
10 Experimental Setup Synthetic data Real world data Traditional Algorithms: LOF [Breunig 2001]: attributes SOF [Aggarwal 2001]: attributes SCAN [Xiu 2007]: graph Algorithms for attributed graphs: CODA [Gao 2010] GOutRank [Müller 2013] ConSub [Iglesias 2013] quality (AUC) runtime [Breunig 2001] Breunig et al. "LOF: identifying density-based local outliers." In ACM SIGMOD 2000 [Aggarwal 2001] Aggarwal et al. "Outlier detection for high dimensional data." In ACM SIGMOD 2001 [Xiu 2007] Xiu et al. "Scan: a structural clustering algorithm for networks." In ACM SIGKDD
11 Synthetic Data Scalability w.r.t. increasing number of attributes and graph size High quality for the detection of contextual outliers 11
12 Real World Data Benchmark [Müller 2013] 124 nodes, 333 edges and 28 attributes Algorithm AUC [%] run.[ms] Attributes full space LOF subspace selection SOF Graph graph clustering SCAN Both full space CODA subspace cluster analysis GOutRank global subspace selection ConSub Local context selection ConOut
13 Real World Data Benchmark [Müller 2013] 124 nodes, 333 edges and 28 attributes Algorithm AUC [%] run.[ms] Attributes full space LOF subspace selection SOF Graph graph clustering SCAN Both full space CODA subspace cluster analysis GOutRank global subspace selection ConSub Local context selection ConOut
![Real World Data Benchmark on a co-purchased network [Müller 2013] 124 nodes, 333 edges and 28 attributes Algorithm AUC [%] run.[ms] Attributes full space LOF 56.85 41 subspace selection SOF 65.](/docs-images/88/115732282/images/14-1.jpg "88 825 Graph graph clustering SCAN 52.68 4 Both full space CODA 50.56 2596 subspace cluster analysis GOutRank 86.86 26648 global subspace selection ConSub 81.77 8930 Local context selection ConOut 81.")
14 Real World Data Benchmark on a co-purchased network [Müller 2013] 124 nodes, 333 edges and 28 attributes Algorithm AUC [%] run.[ms] Attributes full space LOF subspace selection SOF Graph graph clustering SCAN Both full space CODA subspace cluster analysis GOutRank global subspace selection ConSub Local context selection ConOut
15 Conclusions & Future Work Challenge: attributed graphs Local context definition Irrelevant Attributes Statistical selection Outlierness Scoring Combined ranking functions Algorithm Efficiency Future Work Mixed attribute types Local correlations between attributes Other graph definitions (directed, weighted, ) 15
16 Thank you for your attention Our datasets and parameter settings are available online: 16
Graph Exploration: Taking the User into the Loop
Graph Exploration: Taking the User into the Loop Davide Mottin, Anja Jentzsch, Emmanuel Müller Hasso Plattner Institute, Potsdam, Germany 2016/10/24 CIKM2016, Indianapolis, US Where we are Background (5
Ranking Outlier Nodes in Subspaces of Attributed Graphs
Ranking Outlier Nodes in Subspaces of Attributed Graphs Emmanuel Müller Patricia Iglesias Sánchez Yvonne Mülle Klemens Böhm Karlsruhe Institute of Technology (KIT), Germany {emmanuel.mueller, patricia.iglesias,
Adaptive Sampling and Learning for Unsupervised Outlier Detection
Proceedings of the Twenty-Ninth International Florida Artificial Intelligence Research Society Conference Adaptive Sampling and Learning for Unsupervised Outlier Detection Zhiruo Zhao and Chilukuri K.
International Journal of Research in Advent Technology, Vol.7, No.3, March 2019 E-ISSN: Available online at
Performance Evaluation of Ensemble Method Based Outlier Detection Algorithm Priya. M 1, M. Karthikeyan 2 Department of Computer and Information Science, Annamalai University, Annamalai Nagar, Tamil Nadu,
LODES: Local Density Meets Spectral Outlier Detection
LODES: Local Density Meets Spectral Outlier Detection Saket Sathe * Charu Aggarwal Abstract The problem of outlier detection has been widely studied in existing literature because of its numerous applications
Outlier Ensembles. Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY Keynote, Outlier Detection and Description Workshop, 2013
Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier
Chapter 5: Outlier Detection
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases SS 2016 Chapter 5: Outlier Detection Lecture: Prof. Dr.
Outlier Detection Using Unsupervised and Semi-Supervised Technique on High Dimensional Data
Outlier Detection Using Unsupervised and Semi-Supervised Technique on High Dimensional Data Ms. Gayatri Attarde 1, Prof. Aarti Deshpande 2 M. E Student, Department of Computer Engineering, GHRCCEM, University
An Unsupervised Approach for Combining Scores of Outlier Detection Techniques, Based on Similarity Measures
An Unsupervised Approach for Combining Scores of Outlier Detection Techniques, Based on Similarity Measures José Ramón Pasillas-Díaz, Sylvie Ratté Presenter: Christoforos Leventis 1 Basic concepts Outlier
arxiv: v1 [cs.ai] 1 Nov 2016
![arxiv: v1 [cs.ai] 1 Nov 2016](/thumbs/85/92794056.jpg "arxiv: v1 [cs.ai] 1 Nov 2016")
Local Subspace-Based Outlier Detection using Global Neighbourhoods Bas van Stein, Matthijs van Leeuwen and Thomas Bäck LIACS, Leiden University, Leiden, The Netherlands Email: {b.van.stein,m.van.leeuwen,t.h.w.baeck}@liacs.leidenuniv.nl
Subspace Outlier Detection in Linear Time with Randomized Hashing
Subspace Outlier Detection in Linear Time with Randomized Hashing Saket Sathe IBM T. J. Watson Research Center Yorktown Heights, New York 10598 Email: ssathe@us.ibm.com Charu C. Aggarwal IBM T. J. Watson
DATA MINING II - 1DL460
DATA MINING II - 1DL460 Spring 2016 A second course in data mining!! http://www.it.uu.se/edu/course/homepage/infoutv2/vt16 Kjell Orsborn! Uppsala Database Laboratory! Department of Information Technology,
Computer Technology Department, Sanjivani K. B. P. Polytechnic, Kopargaon
Outlier Detection Using Oversampling PCA for Credit Card Fraud Detection Amruta D. Pawar 1, Seema A. Dongare 2, Amol L. Deokate 3, Harshal S. Sangle 4, Panchsheela V. Mokal 5 1,2,3,4,5 Computer Technology
9. Conclusions. 9.1 Definition KDD
9. Conclusions Contents of this Chapter 9.1 Course review 9.2 State-of-the-art in KDD 9.3 KDD challenges SFU, CMPT 740, 03-3, Martin Ester 419 9.1 Definition KDD [Fayyad, Piatetsky-Shapiro & Smyth 96]
On Exploring Complex Relationships of Correlation Clusters
In Proc. 19th International Conference on Scientific and Statistical Database Management (SSDBM 2007), Banff, Canada, 2007 On Exploring Complex Relationships of Correlation Clusters Elke Achtert, Christian
Keywords: Clustering, Anomaly Detection, Multivariate Outlier Detection, Mixture Model, EM, Visualization, Explanation, Mineset.
ISSN 2319-8885 Vol.03,Issue.35 November-2014, Pages:7140-7144 www.ijsetr.com Accurate and Efficient Anomaly Detection via Online Oversampling Principal Component Analysis K. RAJESH KUMAR 1, S.S.N ANJANEYULU
Subspace Outlier Detection in Linear Time with Randomized Hashing
2016 IEEE 16th International Conference on Data Mining Subspace Outlier Detection in Linear Time with Randomized Hashing Saket Sathe IBM T. J. Watson Research Center Yorktown Heights, New York 10598 Email:
Computer Department, Savitribai Phule Pune University, Nashik, Maharashtra, India
International Journal of Scientific Research in Computer Science, Engineering and Information Technology 2018 IJSRCSEIT Volume 3 Issue 5 ISSN : 2456-3307 A Review on Various Outlier Detection Techniques
Robust Contextual Outlier Detection: Where Context Meets Sparsity
Robust Contextual Outlier Detection: Where Context Meets Sparsity Jiongqian Liang and Srinivasan Parthasarathy Computer Science and Engineering, The Ohio State University, Columbus, OH, USA {liangji,srini}@cse.ohio-state.edu
DETECTION OF ANOMALIES FROM DATASET USING DISTRIBUTED METHODS
DETECTION OF ANOMALIES FROM DATASET USING DISTRIBUTED METHODS S. E. Pawar and Agwan Priyanka R. Dept. of I.T., University of Pune, Sangamner, Maharashtra, India M.E. I.T., Dept. of I.T., University of
Machine Learning in the Wild. Dealing with Messy Data. Rajmonda S. Caceres. SDS 293 Smith College October 30, 2017
Machine Learning in the Wild Dealing with Messy Data Rajmonda S. Caceres SDS 293 Smith College October 30, 2017 Analytical Chain: From Data to Actions Data Collection Data Cleaning/ Preparation Analysis
Distance-based Outlier Detection: Consolidation and Renewed Bearing
Distance-based Outlier Detection: Consolidation and Renewed Bearing Gustavo. H. Orair, Carlos H. C. Teixeira, Wagner Meira Jr., Ye Wang, Srinivasan Parthasarathy September 15, 2010 Table of contents Introduction
PCA Based Anomaly Detection
PCA Based Anomaly Detection P. Rameswara Anand 1,, Tulasi Krishna Kumar.K 2 Department of Computer Science and Engineering, Jigjiga University, Jigjiga, Ethiopi 1, Department of Computer Science and Engineering,Yogananda
NDoT: Nearest Neighbor Distance Based Outlier Detection Technique
NDoT: Nearest Neighbor Distance Based Outlier Detection Technique Neminath Hubballi 1, Bidyut Kr. Patra 2, and Sukumar Nandi 1 1 Department of Computer Science & Engineering, Indian Institute of Technology
Anomaly Detection on Data Streams with High Dimensional Data Environment
Anomaly Detection on Data Streams with High Dimensional Data Environment Mr. D. Gokul Prasath 1, Dr. R. Sivaraj, M.E, Ph.D., 2 Department of CSE, Velalar College of Engineering & Technology, Erode 1 Assistant
Table Of Contents: xix Foreword to Second Edition
Data Mining : Concepts and Techniques Table Of Contents: Foreword xix Foreword to Second Edition xxi Preface xxiii Acknowledgments xxxi About the Authors xxxv Chapter 1 Introduction 1 (38) 1.1 Why Data
Real-Time Model-Free Detection of Low-Quality Synchrophasor Data
Real-Time Model-Free Detection of Low-Quality Synchrophasor Data Meng Wu and Le Xie Department of Electrical and Computer Engineering Texas A&M University College Station, TX NASPI Work Group meeting March
Efficient Algorithms for a Robust Modularity-Driven Clustering of Attributed Graphs
Efficient Algorithms for a Robust Modularity-Driven Clustering of Attributed Graphs Patricia Iglesias Sánchez Emmanuel Müller Uwe Leo Korn Klemens Böhm Andrea Kappes Tanja Hartmann Dorothea Wagner Karlsruhe
OUTLIER MINING IN HIGH DIMENSIONAL DATASETS
OUTLIER MINING IN HIGH DIMENSIONAL DATASETS DATA MINING DISCUSSION GROUP OUTLINE MOTIVATION OUTLIERS IN MULTIVARIATE DATA OUTLIERS IN HIGH DIMENSIONAL DATA Distribution-based Distance-based NN-based Density-based
Data Preprocessing UE 141 Spring 2013
Data Preprocessing UE 141 Spring 2013 Jing Gao SUNY Buffalo 1 Outline Data Data Preprocessing Improve data quality Prepare data for analysis Exploring Data Statistics Visualization 2 Document Data Each
Data Mining. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395
Data Mining Introduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 21 Table of contents 1 Introduction 2 Data mining
A Fast Randomized Method for Local Density-based Outlier Detection in High Dimensional Data
A Fast Randomized Method for Local Density-based Outlier Detection in High Dimensional Data Minh Quoc Nguyen, Edward Omiecinski, and Leo Mark College of Computing, Georgia Institute of Technology, Atlanta,
CPSC 340: Machine Learning and Data Mining. Outlier Detection Fall 2016
CPSC 340: Machine Learning and Data Mining Outlier Detection Fall 2016 Admin Assignment 1 solutions will be posted after class. Assignment 2 is out: Due next Friday, but start early! Calculus and linear
Clustering methods: Part 7 Outlier removal Pasi Fränti
Clustering methods: Part 7 Outlier removal Pasi Fränti 6.5.207 Machine Learning University of Eastern Finland Outlier detection methods Distance-based methods Knorr & Ng Density-based methods KDIST: K
Epilog: Further Topics
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases SS 2016 Epilog: Further Topics Lecture: Prof. Dr. Thomas
Contents. Preface to the Second Edition
Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................
Outlier Detection Using Random Walks
Outlier Detection Using Random Walks H. D. K. Moonesinghe, Pang-Ning Tan Department of Computer Science & Engineering Michigan State University East Lansing, MI 88 (moonesin, ptan)@cse.msu.edu Abstract
Data Mining. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1394
Data Mining Introduction Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1394 1 / 20 Table of contents 1 Introduction 2 Data mining
Outlier Detection in Arbitrarily Oriented Subspaces
to appear in: Proceedings of the 12th IEEE International Conference on Data Mining (ICDM), Brussels, Belgium, 2012. Outlier Detection in Arbitrarily Oriented Subspaces Hans-Peter Kriegel, Peer Kröger,
Contents. Foreword to Second Edition. Acknowledgments About the Authors
Contents Foreword xix Foreword to Second Edition xxi Preface xxiii Acknowledgments About the Authors xxxi xxxv Chapter 1 Introduction 1 1.1 Why Data Mining? 1 1.1.1 Moving toward the Information Age 1
Outlier Detection with Two-Stage Area-Descent Method for Linear Regression
Proceedings of the 6th WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 2006 463 Outlier Detection with Two-Stage Area-Descent Method for Linear
Explaining outliers by subspace separability
Explaining outliers by subspace separability Barbora Micenková and Xuan-Hong Dang and Ira Assent Aarhus University Aabogade 34, 2 Aarhus, Denmark Email: {barbora,dang,ira}@cs.au.dk Raymond T. Ng University
I. INTRODUCTION II. RELATED WORK.
ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: A New Hybridized K-Means Clustering Based Outlier Detection Technique
Course Content. What is an Outlier? Chapter 7 Objectives
Principles of Knowledge Discovery in Data Fall 2007 Chapter 7: Outlier Detection Dr. Osmar R. Zaïane University of Alberta Course Content Introduction to Data Mining Association Analysis Sequential Pattern
Unsupervised Interaction-Preserving Discretization of Multivariate Data
Unsupervised Interaction-Preserving Discretization of Multivariate Data Hoang Vu Nguyen Emmanuel Müller Jilles Vreeken Klemens Böhm Karlsruhe Institute of Technology, Germany University of Antwerp, Belgium
Authors: Coman Gentiana. Asparuh Hristov. Daniel Corteso. Fernando Nunez
OUTLIER DETECTOR DOCUMENTATION VERSION 1.0 Authors: Coman Gentiana Asparuh Hristov Daniel Corteso Fernando Nunez Copyright Team 6, 2011 Contents 1. Introduction... 1 2. Global variables used... 1 3. Scientific
COW: Malware Classification in an Open World
: Malware Classification in an Open World Abstract A large number of new malware families are released on a daily basis. However, most of the existing works in the malware classification domain are still
Statistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte
Statistical Analysis of Metabolomics Data Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Outline Introduction Data pre-treatment 1. Normalization 2. Centering,
arxiv: v1 [cs.lg] 18 Sep 2016
![arxiv: v1 [cs.lg] 18 Sep 2016](/thumbs/75/71505813.jpg "arxiv: v1 [cs.lg] 18 Sep 2016")
Sequential Ensemble Learning for Outlier Detection: A Bias-Variance Perspective Shebuti Rayana Wen Zhong Leman Akoglu Department of Computer Science Stony Brook University, Stony Brook, NY 11794 Email:
Outlier Analysis Second Edition
Outlier Analysis Second Edition Charu C. Aggarwal IBM T. J. Watson Research Center Yorktown Heights, New York November 25, 2016 ii To my wife, my daughter Sayani, and my late parents Dr. Prem Sarup and
Jing Gao 1, Feng Liang 1, Wei Fan 2, Chi Wang 1, Yizhou Sun 1, Jiawei i Han 1 University of Illinois, IBM TJ Watson.
Jing Gao 1, Feng Liang 1, Wei Fan 2, Chi Wang 1, Yizhou Sun 1, Jiawei i Han 1 University of Illinois, IBM TJ Watson Debapriya Basu Determine outliers in information networks Compare various algorithms
Mean-shift outlier detection
Mean-shift outlier detection Jiawei YANG a, Susanto RAHARDJA b a,1 and Pasi FRÄNTI a School of Computing, University of Eastern Finland b Northwestern Polytechnical University, Xi an, China Abstract. We
Compare the density around a point with the density around its local neighbors. The density around a normal data object is similar to the density
6.6 Density-based Approaches General idea Compare the density around a point with the density around its local neighbors The relative density of a point compared to its neighbors is computed as an outlier
3. Data Preprocessing. 3.1 Introduction
3. Data Preprocessing Contents of this Chapter 3.1 Introduction 3.2 Data cleaning 3.3 Data integration 3.4 Data transformation 3.5 Data reduction SFU, CMPT 740, 03-3, Martin Ester 84 3.1 Introduction Motivation
2. Data Preprocessing
2. Data Preprocessing Contents of this Chapter 2.1 Introduction 2.2 Data cleaning 2.3 Data integration 2.4 Data transformation 2.5 Data reduction Reference: [Han and Kamber 2006, Chapter 2] SFU, CMPT 459
Chapter 1, Introduction
CSI 4352, Introduction to Data Mining Chapter 1, Introduction Young-Rae Cho Associate Professor Department of Computer Science Baylor University What is Data Mining? Definition Knowledge Discovery from
Statistics 202: Data Mining. c Jonathan Taylor. Outliers Based in part on slides from textbook, slides of Susan Holmes.
Outliers Based in part on slides from textbook, slides of Susan Holmes December 2, 2012 1 / 1 Concepts What is an outlier? The set of data points that are considerably different than the remainder of the
Data Preprocessing Yudho Giri Sucahyo y, Ph.D , CISA
Obj ti Objectives Motivation: Why preprocess the Data? Data Preprocessing Techniques Data Cleaning Data Integration and Transformation Data Reduction Data Preprocessing Lecture 3/DMBI/IKI83403T/MTI/UI
Knowledge Discovery in Databases II Winter Term 2015/2016. Optional Lecture: Pattern Mining & High-D Data Mining
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases II Winter Term 2015/2016 Optional Lecture: Pattern Mining
Automatic Group-Outlier Detection
Automatic Group-Outlier Detection Amine Chaibi and Mustapha Lebbah and Hanane Azzag LIPN-UMR 7030 Université Paris 13 - CNRS 99, av. J-B Clément - F-93430 Villetaneuse {firstname.secondname}@lipn.univ-paris13.fr
Machine Learning Feature Creation and Selection
Machine Learning Feature Creation and Selection Jeff Howbert Introduction to Machine Learning Winter 2012 1 Feature creation Well-conceived new features can sometimes capture the important information
Mining Data Streams. Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction. Summarization Methods. Clustering Data Streams
![Mining Data Streams. Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction. Summarization Methods. Clustering Data Streams](/thumbs/85/93020188.jpg "Mining Data Streams. Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction. Summarization Methods. Clustering Data Streams")
Mining Data Streams Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction Summarization Methods Clustering Data Streams Data Stream Classification Temporal Models CMPT 843, SFU, Martin Ester, 1-06
CS570: Introduction to Data Mining
CS570: Introduction to Data Mining Scalable Clustering Methods: BIRCH and Others Reading: Chapter 10.3 Han, Chapter 9.5 Tan Cengiz Gunay, Ph.D. Slides courtesy of Li Xiong, Ph.D., 2011 Han, Kamber & Pei.
Co-Selection of Features and Instances for Unsupervised Rare Category Analysis
Co-Selection of Features and Instances for Unsupervised Rare Category Analysis Jingrui He Jaime Carbonell Abstract Rare category analysis is of key importance both in theory and in practice. Previous research
Data Mining. Part 2. Data Understanding and Preparation. 2.4 Data Transformation. Spring Instructor: Dr. Masoud Yaghini. Data Transformation
Data Mining Part 2. Data Understanding and Preparation 2.4 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Normalization Attribute Construction Aggregation Attribute Subset Selection Discretization
node2vec: Scalable Feature Learning for Networks
node2vec: Scalable Feature Learning for Networks A paper by Aditya Grover and Jure Leskovec, presented at Knowledge Discovery and Data Mining 16. 11/27/2018 Presented by: Dharvi Verma CS 848: Graph Database
Graph mining assisted semi-supervised learning for fraudulent cash-out detection
Graph mining assisted semi-supervised learning for fraudulent cash-out detection Yuan Li Yiheng Sun Noshir Contractor Aug 2, 2017 Outline Introduction Method Experiments and Results Conculsion and Future
CPSC 340: Machine Learning and Data Mining. Outlier Detection Fall 2018
CPSC 340: Machine Learning and Data Mining Outlier Detection Fall 2018 Admin Assignment 2 is due Friday. Assignment 1 grades available? Midterm rooms are now booked. October 18 th at 6:30pm (BUCH A102
Data Distortion for Privacy Protection in a Terrorist Analysis System
Data Distortion for Privacy Protection in a Terrorist Analysis System Shuting Xu, Jun Zhang, Dianwei Han, and Jie Wang Department of Computer Science, University of Kentucky, Lexington KY 40506-0046, USA
A Survey on Intrusion Detection Using Outlier Detection Techniques
A Survey on Intrusion Detection Using Detection Techniques V. Gunamani, M. Abarna Abstract- In a network unauthorised access to a computer is more prevalent that involves a choice of malicious activities.
Database and Knowledge-Base Systems: Data Mining. Martin Ester
Database and Knowledge-Base Systems: Data Mining Martin Ester Simon Fraser University School of Computing Science Graduate Course Spring 2006 CMPT 843, SFU, Martin Ester, 1-06 1 Introduction [Fayyad, Piatetsky-Shapiro
UAPRIORI: AN ALGORITHM FOR FINDING SEQUENTIAL PATTERNS IN PROBABILISTIC DATA
UAPRIORI: AN ALGORITHM FOR FINDING SEQUENTIAL PATTERNS IN PROBABILISTIC DATA METANAT HOOSHSADAT, SAMANEH BAYAT, PARISA NAEIMI, MAHDIEH S. MIRIAN, OSMAR R. ZAÏANE Computing Science Department, University
PSS718 - Data Mining
Lecture 5 - Hacettepe University October 23, 2016 Data Issues Improving the performance of a model To improve the performance of a model, we mostly improve the data Source additional data Clean up the
Data Preprocessing. Why Data Preprocessing? MIT-652 Data Mining Applications. Chapter 3: Data Preprocessing. Multi-Dimensional Measure of Data Quality
Why Data Preprocessing? Data in the real world is dirty incomplete: lacking attribute values, lacking certain attributes of interest, or containing only aggregate data e.g., occupation = noisy: containing
COMPARISON OF DENSITY-BASED CLUSTERING ALGORITHMS
COMPARISON OF DENSITY-BASED CLUSTERING ALGORITHMS Mariam Rehman Lahore College for Women University Lahore, Pakistan mariam.rehman321@gmail.com Syed Atif Mehdi University of Management and Technology Lahore,
Chapter 9: Outlier Analysis
Chapter 9: Outlier Analysis Jilles Vreeken 8 Dec 2015 IRDM Chapter 9, overview 1. Basics & Motivation 2. Extreme Value Analysis 3. Probabilistic Methods 4. Cluster-based Methods 5. Distance-based Methods
Anomaly Detection. You Chen
Anomaly Detection You Chen 1 Two questions: (1) What is Anomaly Detection? (2) What are Anomalies? Anomaly detection refers to the problem of finding patterns in data that do not conform to expected behavior
Lecture 7: Linear Regression (continued)
Lecture 7: Linear Regression (continued) Reading: Chapter 3 STATS 2: Data mining and analysis Jonathan Taylor, 10/8 Slide credits: Sergio Bacallado 1 / 14 Potential issues in linear regression 1. Interactions
Feature Selection. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Feature Selection CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Dimensionality reduction Feature selection vs. feature extraction Filter univariate
AN IMPROVED DENSITY BASED k-means ALGORITHM
AN IMPROVED DENSITY BASED k-means ALGORITHM Kabiru Dalhatu 1 and Alex Tze Hiang Sim 2 1 Department of Computer Science, Faculty of Computing and Mathematical Science, Kano University of Science and Technology
Knowledge Discovery and Data Mining
Knowledge Discovery and Data Mining Unit # 1 1 Acknowledgement Several Slides in this presentation are taken from course slides provided by Han and Kimber (Data Mining Concepts and Techniques) and Tan,
Data Preprocessing. S1 Teknik Informatika Fakultas Teknologi Informasi Universitas Kristen Maranatha
Data Preprocessing S1 Teknik Informatika Fakultas Teknologi Informasi Universitas Kristen Maranatha 1 Why Data Preprocessing? Data in the real world is dirty incomplete: lacking attribute values, lacking
E-Stream: Evolution-Based Technique for Stream Clustering
E-Stream: Evolution-Based Technique for Stream Clustering Komkrit Udommanetanakit, Thanawin Rakthanmanon, and Kitsana Waiyamai Department of Computer Engineering, Faculty of Engineering Kasetsart University,
Detection of Anomalies using Online Oversampling PCA
Detection of Anomalies using Online Oversampling PCA Miss Supriya A. Bagane, Prof. Sonali Patil Abstract Anomaly detection is the process of identifying unexpected behavior and it is an important research
Performance Evaluation of Density-Based Outlier Detection on High Dimensional Data
Performance Evaluation of Density-Based Outlier Detection on High Dimensional Data P. Murugavel Research Scholar, Manonmaniam Sundaranar University, Tirunelveli, Tamil Nadu, India Dr. M. Punithavalli Research
ECLT 5810 Data Preprocessing. Prof. Wai Lam
ECLT 5810 Data Preprocessing Prof. Wai Lam Why Data Preprocessing? Data in the real world is imperfect incomplete: lacking attribute values, lacking certain attributes of interest, or containing only aggregate
UNSUPERVISED LEARNING FOR ANOMALY INTRUSION DETECTION Presented by: Mohamed EL Fadly
UNSUPERVISED LEARNING FOR ANOMALY INTRUSION DETECTION Presented by: Mohamed EL Fadly Outline Introduction Motivation Problem Definition Objective Challenges Approach Related Work Introduction Anomaly detection
Data Mining Based Online Intrusion Detection
International Journal of Engineering Research and Development e-issn: 2278-067X, p-issn: 2278-800X, www.ijerd.com Volume 3, Issue 12 (September 2012), PP. 59-63 Data Mining Based Online Intrusion Detection
PSCEG: An unbiased Parallel Subspace Clustering algorithm using Exact Grids
PSCEG: An unbiased Parallel Subspace Clustering algorithm using Exact Grids Bo Zhu, Bruno Ordozgoiti, and Alberto Mozo Universidad Polite cnica de Madrid - Departamento de Sistemas Informa ticos bozhumatias@ict-ontic.eu,
CLUS: Parallel subspace clustering algorithm on Spark
CLUS: Parallel subspace clustering algorithm on Spark Bo Zhu, Alexandru Mara, and Alberto Mozo Universidad Politécnica de Madrid, Madrid, Spain, bozhumatias@ict-ontic.eu alexandru.mara@alumnos.upm.es a.mozo@upm.es
Anomaly detection. Luboš Popelínský. Luboš Popelínský KDLab, Faculty of Informatics, MU
Anomaly detection Luboš Popelínský Luboš Popelínský KDLab, Faculty of Informatics, MU Thanks to Luis Torgo, Lea Nezvalová, Karel Vaculík and other members of the KDLab KEG VŠE Mar 1, 2018 What is an Outlier?!
Data Clustering With Leaders and Subleaders Algorithm
IOSR Journal of Engineering (IOSRJEN) e-issn: 2250-3021, p-issn: 2278-8719, Volume 2, Issue 11 (November2012), PP 01-07 Data Clustering With Leaders and Subleaders Algorithm Srinivasulu M 1,Kotilingswara
Data Preprocessing. Javier Béjar AMLT /2017 CS - MAI. (CS - MAI) Data Preprocessing AMLT / / 71 BY: $\
![Data Preprocessing. Javier Béjar AMLT /2017 CS - MAI. (CS - MAI) Data Preprocessing AMLT / / 71 BY: $\](/thumbs/73/69052607.jpg "Data Preprocessing. Javier Béjar AMLT /2017 CS - MAI. (CS - MAI) Data Preprocessing AMLT / / 71 BY: $\")
Data Preprocessing S - MAI AMLT - 2016/2017 (S - MAI) Data Preprocessing AMLT - 2016/2017 1 / 71 Outline 1 Introduction Data Representation 2 Data Preprocessing Outliers Missing Values Normalization Discretization
Adaptive Parallel Compressed Event Matching
Adaptive Parallel Compressed Event Matching Mohammad Sadoghi 1,2 Hans-Arno Jacobsen 2 1 IBM T.J. Watson Research Center 2 Middleware Systems Research Group, University of Toronto April 2014 Mohammad Sadoghi
OPTICS-OF: Identifying Local Outliers
Proceedings of the 3rd European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 99), Prague, September 1999. OPTICS-OF: Identifying Local Outliers Markus M. Breunig, Hans-Peter
Large Scale Data Analysis for Policy
Large Scale Data Analysis for Policy 90-866, Fall 2012 Lecture 9: Anomaly and Outlier Detection Parts of this lecture were adapted from Banerjee et al., Anomaly Detection: A Tutorial, presented at SDM
An Enhanced Density Clustering Algorithm for Datasets with Complex Structures
An Enhanced Density Clustering Algorithm for Datasets with Complex Structures Jieming Yang, Qilong Wu, Zhaoyang Qu, and Zhiying Liu Abstract There are several limitations of DBSCAN: 1) parameters have
Capturing Topology in Graph Pattern Matching
Capturing Topology in Graph Pattern Matching Shuai Ma, Yang Cao, Wenfei Fan, Jinpeng Huai, Tianyu Wo University of Edinburgh Graphs are everywhere, and quite a few are huge graphs! File systems Databases
Fast or furious? - User analysis of SF Express Inc
CS 229 PROJECT, DEC. 2017 1 Fast or furious? - User analysis of SF Express Inc Gege Wen@gegewen, Yiyuan Zhang@yiyuan12, Kezhen Zhao@zkz I. MOTIVATION The motivation of this project is to predict the likelihood
Filtered Clustering Based on Local Outlier Factor in Data Mining
, pp.275-282 http://dx.doi.org/10.14257/ijdta.2016.9.5.28 Filtered Clustering Based on Local Outlier Factor in Data Mining 1 Vishal Bhatt, 2 Mradul Dhakar and 3 Brijesh Kumar Chaurasia 1,2,3 Deptt. of