Introduction. IST557 Data Mining: Techniques and Applications. Jessie Li, Penn State University

Transcription

1 Introduction IST557 Data Mining: Techniques and Applications Jessie Li, Penn State University 1

2 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 2

3 Why Data Mining? The Explosive Growth of Data: from terabytes to petabytes Data collec3on and data availability Automated data collec3on tools, database systems, Web, computerized society Major sources of abundant data Business: Web, e-commerce, transac3ons, stocks, Science: Remote sensing, bioinforma3cs, scien3fic simula3on, Society and everyone: news, digital cameras, YouTube We are drowning in data, but starving for knowledge! Necessity is the mother of inven3on Data mining Automated analysis of massive data sets 3

4 4

5 5

6 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 6

7 What Is Data Mining? Data mining (knowledge discovery from data) Extrac3on of interes3ng (non-trivial, implicit, previously unknown and poten3ally useful) or knowledge from huge amount of data Data mining: a misnomer? Alterna3ve names Knowledge discovery (mining) in databases (KDD), knowledge extrac3on, data/pa@ern analysis, data archeology, data dredging, informa3on harves3ng, business intelligence, etc. Watch out: Is everything data mining? Simple search and query processing Data analysis vs. data mining 7

8 Knowledge Discovery (KDD) Process This is a view from typical Pattern Evaluation database systems and data warehousing communi3es Data mining plays an essen3al Data Mining role in the knowledge discovery process Task-relevant Data Data Warehouse Selection Data Cleaning Data Integration Databases 8

9 Data Mining in Business Intelligence Increasing potential to support business decisions Decision Making Data Presentation Visualization Techniques Data Mining Information Discovery End User Business Analyst Data Analyst Data Exploration Statistical Summary, Querying, and Reporting Data Preprocessing/Integration, Data Warehouses Data Sources Paper, Files, Web documents, Scientific experiments, Database Systems DBA 9

10 KDD Process: A Typical View from ML and Statistics Input Data Data Pre- Processing Data Mining Post- Processing Data integration Normalization Feature selection Dimension reduction Pattern discovery Association & correlation Classification Clustering Outlier analysis Pattern evaluation Pattern selection Pattern interpretation Pattern visualization This is a view from typical machine learning and sta3s3cs communi3es 10

11 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 11

12 Data Mining: On What Kinds of Data? Database-oriented data sets and applica3ons Rela3onal database, data warehouse, transac3onal database Object-rela3onal databases, Heterogeneous databases and legacy databases Advanced data sets and advanced applica3ons Data streams and sensor data Time-series data, temporal data, sequence data (incl. bio-sequences) Structure data, graphs, social networks and informa3on networks Spa3al data and spa3otemporal data Mul3media database Text databases The World-Wide Web 12

13 Matrix Data 13

14 Set Data 14

15 Sequence Data 15

16 Time Series Data 16

17 Graph/Network Data 17

18 Spatiotemporal Data 18

19 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 19

20 Data Mining Function: (1) Frequent Pattern and Association Rules Walmart Transac3ons Tid Items bought 10 Beer, Nuts, Diaper 20 Beer, Coffee, Diaper 30 Beer, Diaper, Eggs 40 Nuts, Eggs, Milk 50 Nuts, Coffee, Diaper, Eggs, Milk How to mine such and rules efficiently in large datasets? Frequent {Beer, Diaper}: 3 {Nuts, Eggs, Milk}: 3 In 1992, Thomas Blischok, manager of a retail consul3ng group at Teradata, and his staff prepared an analysis of 1.2 million market baskets from about 25 Osco Drug stores. Database queries were developed to iden3fy affini3es. The analysis "did discover that between 5:00 and 7:00 p.m. that consumers bought beer and diapers". 20

21 Frequent Pattern Mining: Example IST210 21

22 Frequent Pattern Mining: Example (Cont.) IST210 22

23 Data Mining Function: (2) Classification (Supervised Learning) Training This is a dog This is a dog This is a cat What is this? Classifica3on: Categorical output Regression: Con3nuous output This is a dog! Tes3ng 23

24 Data Mining Function: (2) Classification Classifica3on and label predic3on Construct models (func3ons) based on some training examples Describe and dis3nguish classes or concepts for future predic3on E.g., classify disease based on symptoms of pa3ents; classify image categories based on image features Predict some unknown class labels Typical methods Decision trees, naïve Bayesian classifica3on, support vector machines, neural networks, rule-based classifica3on, classifica3on, logis3c regression, Typical applica3ons: Credit card fraud detec3on, direct marke3ng, classifying diseases, web-pages, 24

25 Data Mining Function: (3) Cluster Analysis (Unsupervised Learning) Cluster 2 Cluster 1 Dog, cat, cow? Unsupervised: seman3c meanings of clusters are not clear Supervised learning (with training): classes are predefined by human 25

26 Data Mining Function: (3) Cluster Analysis Unsupervised learning (i.e., Class label is unknown) Group data to form new categories (i.e., clusters), e.g., cluster houses to find distribu3on Principle: Maximizing intra-class similarity & minimizing interclass similarity Many methods and applica3ons 26

27 Data Mining Function: Many Others Data-dependent (or domain specific) Text data Topic modeling, sen3ment analysis Time series Predic3on, outbreak detec3on Graph data Finding frequent subgraphs (e.g., chemical compounds) Network analysis Social networks analysis, community detec3on, influence propaga3on Recommenda3on system Movie recommenda3on, purchase recommenda3on Web mining Ranking, personalized search, opinion mining 27

28 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 28

29 Data Mining: Confluence of Multiple Disciplines Machine Learning Database Sta3s3cs Algorithm Data Mining Visualiza3on Applica3ons Recogni3on High-Performance Compu3ng 29

30 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 30

31 Major Issues in Data Mining (1) Mining Methodology Mining various and new kinds of knowledge Mining knowledge in mul3-dimensional space Data mining: An interdisciplinary effort Boos3ng the power of discovery in a networked environment Handling noise, uncertainty, and incompleteness of data Pa@ern evalua3on and pa@ern- or constraint-guided mining User Interac3on Interac3ve mining Incorpora3on of background knowledge Presenta3on and visualiza3on of data mining results 31

32 Major Issues in Data Mining (2) Efficiency and Scalability Efficiency and scalability of data mining algorithms Parallel, distributed, stream, and incremental mining methods Diversity of data types Handling complex types of data Mining dynamic, networked, and global data repositories Data mining and society Social impacts of data mining Privacy-preserving data mining Invisible data mining 32

33 Introduction Why Data Mining? What Is Data Mining? A Mul3-Dimensional View of Data Mining What Kinds of Data Can Be Mined? What Kinds of Pa@erns Can Be Mined? What Kinds of Technologies Are Used? Major Issues in Data Mining A Brief History of Data Mining and Data Mining Society 33

34 A Brief History of Data Mining Society 1989 IJCAI Workshop on Knowledge Discovery in Databases Knowledge Discovery in Databases (G. Piatetsky-Shapiro and W. Frawley, 1991) Workshops on Knowledge Discovery in Databases Advances in Knowledge Discovery and Data Mining (U. Fayyad, G. Piatetsky- Shapiro, P. Smyth, and R. Uthurusamy, 1996) Interna3onal Conferences on Knowledge Discovery in Databases and Data Mining (KDD 95-98) Journal of Data Mining and Knowledge Discovery (1997) ACM SIGKDD conferences since 1998 and SIGKDD Explora3ons More conferences on data mining PAKDD (1997), PKDD (1997), SIAM-Data Mining (2001), (IEEE) ICDM (2001), WSDM (2008), etc. ACM Transac3ons on KDD (2007) 34

35 Conferences and Journals on Data Mining KDD Conferences ACM SIGKDD Int. Conf. on Knowledge Discovery in Databases and Data Mining (KDD) SIAM Data Mining Conf. (SDM) (IEEE) Int. Conf. on Data Mining (ICDM) European Conf. on Machine Learning and Principles and prac3ces of Knowledge Discovery and Data Mining (ECML-PKDD) Pacific-Asia Conf. on Knowledge Discovery and Data Mining (PAKDD) Int. Conf. on Web Search and Data Mining (WSDM) n n Other related conferences n n n n DB conferences: ACM SIGMOD, VLDB, ICDE, EDBT, ICDT, Web and IR conferences: WWW, SIGIR, WSDM Journals n n n n ML conferences: ICML, NIPS PR conferences: CVPR, ICCV Data Mining and Knowledge Discovery (DAMI or DMKD) IEEE Trans. On Knowledge and Data Eng. (TKDE) KDD Explora3ons ACM Trans. on KDD 35

36 Where to Find References? DBLP, CiteSeer, Google Data mining and KDD (SIGKDD: CDROM) Conferences: ACM-SIGKDD, IEEE-ICDM, SIAM-DM, PKDD, PAKDD, etc. Journal: Data Mining and Knowledge Discovery, KDD Explora3ons, ACM TKDD Database systems (SIGMOD: ACM SIGMOD Anthology CD ROM) Conferences: ACM-SIGMOD, ACM-PODS, VLDB, IEEE-ICDE, EDBT, ICDT, DASFAA Journals: IEEE-TKDE, ACM-TODS/TOIS, JIIS, J. ACM, VLDB J., Info. Sys., etc. AI & Machine Learning Conferences: Machine learning (ML), AAAI, IJCAI, COLT (Learning Theory), CVPR, NIPS, etc. Journals: Machine Learning, Ar3ficial Intelligence, Knowledge and Informa3on Systems, IEEE-PAMI, etc. Web and IR Conferences: SIGIR, WWW, CIKM, etc. Journals: WWW: Internet and Web Informa3on Systems, Sta3s3cs Conferences: Joint Stat. Mee3ng, etc. Journals: Annals of sta3s3cs, etc. Visualiza3on Conference proceedings: CHI, ACM-SIGGraph, etc. Journals: IEEE Trans. visualiza3on and computer graphics, etc. 36

37 Summary Data mining: Discovering interes3ng and knowledge from massive amount of data A natural evolu3on of science and informa3on technology, in great demand, with wide applica3ons A KDD process includes data cleaning, data integra3on, data selec3on, transforma3on, data mining, pa@ern evalua3on, and knowledge presenta3on Mining can be performed in a variety of data Data mining func3onali3es: characteriza3on, discrimina3on, associa3on, classifica3on, clustering, trend and outlier analysis, etc. Data mining technologies and applica3ons Major issues in data mining 37

38 What you will learn in this course Fundamental data mining techniques Classification, clustering, frequent pattern techniques 16 lectures, 3-4 assignments, 3 assignment labs Data mining techniques in specific applications Learnt from your own project and other students projects Team project (2 students) Literature survey Project 38

39 What you should not expect at the end of the semester Become an expert in data mining! Know how to mine any kind of data! 39

40 What you should expect to learn (very) basic data mining techniques The way to learn data mining techniques by yourself in the future How to formulate problem How to apply simple methods and evaluate the methods How to look for solutions How to learn from failures How to improve your methods 40