Data Mining. Allan Tucker School of Information Systems Computing and Mathematics Brunel University, London. UB8 3PH. UK

Transcription

1 Data Mining Allan Tucker School of Information Systems Computing and Mathematics Brunel University, London. UB8 3PH. UK

2 The talk The Data Explosion Data Mining techniques & Application Data Mining in the Media Some of our work on Biomedical Data Mining Some Caveats

3 Data historically... Preserve of scientists: Darwin, 1800s Newton, 1600s Galton, 1800s Pearson, 1900s

4 Database Technology Timeline 1960s: Data collection, database creation 1970s: Relational data model Relational DBMS implementation 1980s: Advanced data models (extended-relational, OO, deductive, etc.) Application-oriented DBMS (spatial, scientific, engineering, etc.) 1990s 2000s: Data Warehousing Multimedia and Web databases Distributed DW: The Cloud

5 Data Generation examples Data collected from online forms Amazon purchases Google Loan requests searches Massively parallel sequencing of biological data (gene expression) Telescopes scanning the skies

6 The Data Explosion We are drowning in information, but starving for knowledge John Naisbett (Futurologist) Due to the advance of IT and the Internet Massive increase in ability to: Record: Electronic records and forms, the Internet Store: Data Warehouses, the Cloud Risk of Information Overload

7 The Data Explosion Need to Analyse: Data Mining, Machine Learning, Intelligent Data Analysis, Knowledge Discovery in Databases, Bioinformatics Knowledge

8 Overlap with Statistics Statistics is the science of the collection, organization, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments, OED... statistics, that is, the mathematical treatment of reality... Hannah Arendt He uses statistics as a drunken man uses lampposts - for support rather than for illumination. Andrew Lang There are lies, damned lies, and statistics. Benjamin Disraeli

9 Overlap with Statistics DM is the process of extracting patterns from large data sets by combining methods from statistics and artificial intelligence with database management., Jason Frand, UCLA More explorative Not always an hypothesis Works with Historical Data Rarely any experimental design! Makes less assumptions about the data

10 Data Mining Process (or Knowledge Discovery) Knowledge Discovery in Databases (KDD) The Process (from Advances in KDD and Data mining): Data Knowledge Target Data Pre-processed Data Transformed Data Patterns

11 Typical Tasks Descriptive: Clustering (customer profiling) Association Rule Mining (basket analysis) Predictive: Classification (medical diagnosis) Forecasting (stock forecasts) Regression (interpolation / extrapolation)

12 Clustering (unsupervised learning) Looking for data points that are similar Depends on how you measure difference or similarity!

13 Clustering (unsupervised learning) Customer Relationship Management

14 Clustering (unsupervised learning) Patients with similar symptoms

15 Classification (supervised learning) Separate Classes with: 3 or... A simple model: Generalisable but biased a complex model: Good fit but risks overfitting

16 Classification (supervised learning) For example, Purchases bought online Patient data healthy vs disease Agreeing loans

17 Decision Trees Credit Rating / Pregnancy Screening examples

18 Feature Selection How data clusters / classifies depends very much upon selected variables (features) What if we select customers based upon their purchases only? Or also include demographics? We get very different results. Feature selection involves automatically identifying the important variables

19 Feature Selection Clusters plotted with different features Series1 Series2 Series Series1 Series2 Series Series1 Series2 Series Filter methods score each variable independently (e.g. chi squared) Wrapper approaches model the interactions between variables

20 Association Rules Based upon Basket Analysis Supermarkets use this all the time Given a large amount of basket data, generate rules: <Set of items> <Set of items> <Set of items> If <items> then <items> (confidence / support)

21 Association Rules Based upon Basket Analysis Supermarkets use this all the time Given a large amount of basket data, generate rules: <Set of items> <Set of items> <Set of items> If <items> then <items> (confidence / support)

22 Association Rules Why do they find this knowledge useful? Loyalty Cards Shop layout Special offers Think of amazon...

23 Association Rules

24 Association Rules

25 Time-Series Models Statistical & AI Models (Neural Networks) Temporal Abstractions For example, EEG & ECG in ICUs, Stock markets

26 Time-Series Models Statistical & AI Models (Neural Networks) Temporal Abstractions For example, EEG & ECG in ICUs, Stock markets

27 Bayesian Networks A probabilistic method to model data Easily interpreted by non-statisticians Can be used to combine existing knowledge with data Essentially use independence assumptions to model the joint distribution of a domain

28 Bayesian Networks Simple 2 variable Joint Distribution P(Gene, Disease) Gene Gene Disease Disease Can use it to ask many useful questions But requires k N probabilities

29 Bayesian Network for Toy Domain Gene A Gene B P(A) P(B) A B P(C) T T.95 T F.94 F T.29 F F.001 Gene C C P(D) T.70 F.01 Gene D Gene E C P(E) T.90 F.05

30 Bayesian Networks for Classification & Feature Selection & forecasting Nodes that can represents class labels or variables at points in time t-1 t Also latent variables via EM X 1 X 1 X 1 X 2 P(X 1 ) P(X 2 ) X 2 X 3 X 2 X 3 t-1 t H H X 3 P(X 3 X 1, X 2 ) X 4 X 4 X 2 X 2 X 4 X 5 P(X 4 X 3 ) P(X 5 X 3 ) C X N X N X N X N X 1 X 2 X 3 X N

31 Bayesian Networks for Classification & Feature Selection & forecasting Diagnosing Aircraft Failure

32 Data Mining - Successes Some successful examples of its use: Search Engines Bayesian networks Pharmaceutical companies Drug Discovery Credit card companies Fraud Detection Transportation companies - Routing Large consumer package goods companies (to improve the sales process to retailers) Hospital Organisation Decision Analysis Online businesses Market Research

33 On Business Intelligence & OLAP Application of DM & DW to Business Data On-Line Analytic Processing: Overlap with Data Mining More focussed on interactive ad-hoc analysis Exploits multidimensional modelling Concepts of: Drill-down Consolidation Slicing & Dicing Visualisation & Dashboards

34 On Business Intelligence & OLAP

35 On Social Media / Market Research LinkedIn (professional contacts) Skype (voice / video) Ipods (location) Flickr (images) Facebook (personal contacts)

36 On Social Media / Market Research LinkedIn (professional contacts) Skype (voice / video) Ipods (location) Flickr (images) Facebook (personal contacts)

37 Data Mining In the media part 1 Most positive news stories relate to other names for DM

38 Some of our Data Mining work in Biomedical / Eco Informatics Building Gene Regulatory Networks Building Trajectories of Disease from Medical Data Building Dynamic Models of Ecological Data

39 Microarray Data & Bioinformatics Major source of data for gene expression activity Technology takes measurements over 1000s of genes simultaneously Gene Regulatory Networks (GRNs) model how genes interact Eliciting reliable GRNs from data key to understanding biological mechanisms

40 Yeast

41 The Importance of Independent Test Data Prediction Train a network on one dataset Test it on the others sets (Independent Data) As opposed to Cross Validation (testing on the same dataset)

42 Models of Increasing Complexity (MIC) Extending the Consensus across platforms Select one dataset for training, others become test sets Score mean and var of SSE using CV and independent test sets Use these to rank genes (this is feature selection) (2010) Anvar, S.Y., t' Hoen, P.A.C. and Tucker, A., The Identification of Informative Genes from Multiple Datasets with Increasing Complexity, BMC Bioinformatics 11 : 32

43 Mechanisms Between Species? Dandelion Algorithm extension of MIC (submitted) Anvar, Y. Tucker, A. Venema, A. van Ommen, G.J.B. van der Maarel, S.M. Raz, V. t Hoen, P.A.C. Interspecies translation of gene disease networks increase robustness and predictive accuracy, PLOS Computational Biology

44 Inter-species Mechanisms

45 Modelling Clinical Data Biomedical studies often involve data sampled from a cross-section of a population Collecting medical information on patients suffering from a particular disease and controls These studies show a snapshot of the disease process but disease is inherently temporal: Previously healthy people can develop a disease over time going through different stages of severity If we want to model the development of such processes, usually require longitudinal data (expensive)

46 Models of Disease: Visual Field and Retinal Image Data Progressive loss of the field of vision is characteristic of many eye diseases Glaucoma is a leading cause of irreversible blindness in the world. VF Data: sensitivity of field of vision HRT Data: anatomical info of retina

47 b) Pseudo Time-Series for CS Data Tucker, A. and Garway-Heath, D., The Pseudo Temporal Bootstrap for Predicting Glaucoma from Cross-Sectional Visual Field Data, IEEE Transactions on IT in Biomedicine 14 (1) : 79-85, 2010

48 Fisheries Population Modelling

49 Cod Collapse in G Bank, N Sea & ESS George s Bank Functional Collapse in late 80s North Sea No Functional Collapse Biomass Catch East Scotian Shelf Functional Collapse in early 90s

50 Dynamic Functional Models Predicting ESS event & Cod biomass from G Bank G Bank Th Skate Cod Cusk Cod Catch ESS

51 Summary What is Data Mining Potential (& Successful) Applications Business Intelligence Medical Informatics Bio Informatics Ecological Data Engineering What about some of the downsides

52 Caveats to Data Mining Data Quality Spurious Correlations Over-fitting Black Box Modelling Over-reliance slave to the data? Can t see the wood for the trees?

53 Data Mining in the media part 2 Data mining government /commercial data sets for national security or law enforcement purposes has raised privacy concerns EU The right to be forgotten e.g. Facebook Patenting Genetic information

54 Data Mining in the media part 2

55 Data Mining Video , _ ,00.html

56 Data Mining in the Future Maybe a rebranding is needed? Medical Informatics & Business Intelligence Data to Knowledge Knowledge Discovery in Databases & KDnuggets In the cloud: Cloud Analytics

57 Data Mining in the Future Maybe different names? Medical Informatics & Business Intelligence Data to Knowledge Knowledge Discovery in Databases In the cloud: Cloud Analytics

58 Thanks for listening Emma Steele, Yahya Anvar & Peter- Bram t Hoen for their work on the microarray research Daniel Duplisea for the work on the fish biomass research Stefano Ceccon, Yuanxi Li & David Garway-Heath for work on Glaucoma research