Data Mining An Overview ITEV, F /18

Transcription

1 Data Mining An Overview ITEV, F /18

2 ITEV, F /18 What is Data Mining??

3 ITEV, F /18 What is Data Mining??

4 ITEV, F /18 What is Data Mining?!

5 ITEV, F /18 What is Data Mining? Definitions Knowledge Discovery is the nontrivial extraction of implicit, previously unknown, and potentially useful information from data [Frawley, Piatetsky-Shapiro, Matheus 1991]. Data Mining is the analysis of (often large) observational data sets to find unsuspected relationships and to summarize the data in novel ways that are both understandable and useful to the data owner [Hand, Mannila, Smyth 2001].

6 ITEV, F /18 What is Data Mining? Data Mining in practice

7 ITEV, F /18 What is Data Mining? Data Mining in practice Real life data Off the shelf algorithm preprocess adapt

8 ITEV, F /18 What is Data Mining? Data Mining in practice Real life data Off the shelf algorithm preprocess adapt evaluate + iterate

9 ITEV, F /18 What is Data Mining? Data Mining in practice Real life data Off the shelf algorithm preprocess adapt evaluate + iterate general algorithmic methods data/domain specific operations

10 ITEV, F /18 The CRISP model Background Developed by a four member consortium in a EU project. Members of the consortium: Teradata (NCR) SPSS (statistical software) DaimlerChrysler OHRA (Insurance and Banking) Consortium supported by a special interest group composed of over 300 organizations involved in data mining projects. Aim From The CRISP-DM project has developed an industry- and tool-neutral Data Mining process model. [... ] this project defined and validated a data mining process that is applicable in diverse industry sectors. This will make large data mining projects faster, cheaper, more reliable and more manageable. Even small scale data mining investigations will benefit from using CRISP-DM.

11 ITEV, F /18 The CRISP model Phases of the CRISP DM Process Model (Illustration from

12 ITEV, F /18 The CRISP model Business/Data understanding Vision: Data Mining extracts whatever interesting hidden information there is in the data Reality: Data Mining techniques solve several types of well-defined tasks Reality: The data used must support the task at hand Reality: The data miner must understand the background of the data, in order to select an appropriate data mining technique

13 ITEV, F /18 The CRISP model Our Focus

14 ITEV, F /18 The CRISP model Selecting the Modeling Technique Universe of Techniques (Defined by Tool) Techniques Appropriate for Problem Political Requirements (Management,Understandability) Constraints (Time, Data Characteristics, Staff Training/Knowledge) Tool(s) Selected

15 ITEV, F /18 Types of Tasks and Models Prediction (Supervised Learning) Task: predict some (unobserved) target variable based on observed values of attribute variables - Regression (Larose: Estimation), if target is continuous - Classification, if target is discrete Models e.g.: Decision trees, k nearest neighbors, Neural networks, Naive Bayes,... Clustering Task: identify coherent subgroups in data Models e.g.: k-means, hierarchical clustering, Self-organizing maps, probabilistic clustering,... Association analysis Task: identify patterns of co-occurrence of attribute values Models: Apriori and extensions

16 ITEV, F /18 Examples: Classification Text Categorization The Association for Computing Machinery (ACM) maintains a subject classification scheme for computer science research papers. Part of the subject hierarchy (1998 version): I. Computing Methodologies I.2 Artificial Intelligence I.2.6 Learning - Analogies - Concept learning - Connectionism and neural nets - Induction - Knowledge acquisition - Language acquisition - Parameter learning Papers are manually classified by authors or editors. Data: collection of classified papers (full text or abstracts) Task: build a classifier that automatically assigns a subject index to new, unclassified papers.

17 ITEV, F /18 Examples: Classification Spam Filtering Spam filtering in Mozilla: user trains the mail reader to recognize spam by manually labeling incoming mails as spam/no spam. Data: collection of user-classified s (full text). Task: build a classifier that automatically categorizes an incoming as spam/no spam

18 ITEV, F /18 Examples: Classification Character Recognition Example for a Pattern Recognition problem (pattern recognition is an older discipline than data mining, but now can also be seen as a sub-area of data mining): Data: collection of handwritten characters, correctly labeled. Task: build a classifier that identifies new handwritten characters.

19 ITEV, F /18 Examples: Classification Credit Rating From existing customer data predict whether a person applying for a new loan will repay or default on the loan. Data: existing customer records with attributes like age, employment type, income,... and information on payback history. Task: build a classifier that predicts whether a new customer will repay the loan.

20 ITEV, F /18 Examples: Clustering Text Categorization Web mining: automatically detect similarity between web pages (e.g. to support search engines or automatic construction of internet directories). Data: the WWW. Task: Construct a (similarity) model for pages on the WWW.

21 ITEV, F /18 Examples: Clustering Bioinformatics: Phylogenetic Trees From biological data construct a model of evolution. Lactococcus Lactis Caulobacter Crescentus Bacillus Halodurans Bacillus Subtilis Rattus Norvegicus Pan Troglodytes Homo Sapiens Data: e.g. genome sequences of different animal species. Task: construct a hierarchical model of similarity between the species.

22 ITEV, F /18 Examples: Association Analysis Association Rules Data: transaction data Task: infer association rules Transaction Items bought 1 Beer,Soap,Milk,Butter 2 Beer,Chips,Butter 3 Milk,Spaghetti,Butter,Tomatos {Beer} {Chips} {Spaghetti,Tomatos} {Wine}...

23 ITEV, F /18 Tools Commercial system, Windows only Clementine Many methods, good interface, integrated use of MS SQL server WEKA Free open source Java toolbox ( Many methods, good interface For all toolboxes: easy use of methods can be dangerous correct interpretation of results requires understanding of methods. Documentation essential (and often a weak point...)!