Data Mining. ❸Chapter 3 Data warehouse, ETL and OLAP. Asso.Prof.Dr. Xiao-dong Zhu. Business School, University of Shanghai for Science & Technology

Transcription

1 ❸Chapter 3 Data warehouse, and Business School, University of Shanghai for Science & Technology nd Semester, Spring2017

2 Contents of chapter 2 1 KDD Process

3 What is KDD? KDD Process the process of KDD the architecture of data mining KDD, Knowledge in database

4 the process of KDD the architecture of data mining Figure: the process of KDD

5 the process of KDD the architecture of data mining Figure: the process of KDD

6 the process of KDD the architecture of data mining Figure: the architecture of data mining

7 the process of KDD the architecture of data mining Figure: Data mining and

8 Extraction KDD Process E T L *.txt, *.exl, *.doc, *.xml

9 T ransfomation KDD Process E T L dirty data, chaos data, missing data. Types? (string, integer, real,,) Unit? (meter, centimeter, inch, feet, 1inch = 2.54 centimeter)

10 Loading KDD Process E T L loading clean data into for mining

11 define W.H.Inmon. father of.

12 define W.H.Inmon. father of. 1 a database used for reporting and data analysis. 2 A is a relational database that is designed for query and analysis rather than for transaction processing. It usually contains historical data derived from transaction data, but can include data from other sources.

13 4 characters KDD Process define 1 Subject Oriented 2 Integrated 3 Nonvolatile 4 Time Variant

14 define Figure: Data warehouse architecture of IBM

15 define Figure: star schema of

16 define Figure: snow flake schema of data mining

17 define Figure: constellation schema of data miningp

18 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Table: Add caption Characteristics OLTP Original? Original data Exported data Integrated? Detailed data Integrated, refined data Historical? Current data Historical data Update cycle? Updatable, frequently Non-updatable, Periodic refresh Access? Read, write Read, seldom write Focus on what? Data input Data output Size? Data volume one time is small, MB-GB level Data volume one time is big, GB-TB level Metric? Transaction throughput Query throughput what operation? Application oriented transaction driven Analysis oriented analysis driven Who? Operator oriented, supporting daily operation Decision people oriented, supporting management requirement

19 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Dimension Dimension is a viewpoint, such as time dimension, product dimension, geography dimension. ( on mathematics or physics, it is in a space, is defined as the minimum number of coordinates needed to specify any point within it.) Dimension Granular Dimension granular. For a dimension, when people observe data, the dimension can be divided into a subdivision.

20 Example KDD Process Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence (1) Year month day (2) Electronic equipment refrigerator (3) drink coffee latte

21 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Granular of dimension

22 Data cube KDD Process Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Using three dimensions to describe data, the value occur at a concrete cross point in the three dimensional data space. Super data cube.

23 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Example: Some household appliance sales company. Figure: Data cube with two dimensions

24 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Data cube with three dimensions

25 has following 4 operations Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence

26 has following 4 operations Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence 1 slice 2 drill up 3 drill down 4 rotate

27 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Slice

28 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Drill up and Drill down

29 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Rotate

30 Business Intelligence KDD Process Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence BI: A Set of theories, methodologies, architectures, and technologies that transform raw data into meaningful and useful information for business purposes. BI can handle large amounts of information to help identify and develop new opportunities. For: enterprise.

31 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Figure: Location of Data mining in Business Intelligence

32 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence Data mining and decision support system decision support system DSS, comprehensive computer systems and related tools that assist managers in making decisions and solving problems. For manager, ceo, using data mining technologies.

33 Dimension, and Dimension Granular Data cube Four operation Data mining, in Business Intelligence 1 KDD Process

34 Describe some concepts in our data mining course 1 data mining 2 data 3 knowledge 4 5 6