Big Data 13. Data Warehousing

Size: px
Start display at page:

Download "Big Data 13. Data Warehousing"

Transcription

1 Ghislain Fourny Big Data 13. Data Warehousing fotoreactor / 123RF Stock Photo

2 The road to analytics Aurelio Scetta / 123RF Stock Photo

3 Another history of data management (T. Hofmann) 1970s 2000s Age of Transactions

4 Another history of data management (T. Hofmann) 1970s 2000s Age of Transactions Age of Business Intelligence

5 Another history of data management (T. Hofmann) 1970s 2000s Age of Transactions Age of Business Intelligence 2000s - Age of Big Data

6 Paradigms OLTP

7 Paradigms vs. OLTP OLAP

8 OnLine Transaction Processing Consistent and Reliable Record-Keeping

9 OnLine Transaction Processing Transactions and results on small portions of data

10 OnLine Transaction Processing Lots of transactions on small portions of data

11 OnLine Transaction Processing Normalized Data

12 OnLine Analytical Processing Data-based Decision Support arturaliev / 123RF Stock Photo

13 OLAP is Big Large portions of the data

14 OLAP is Big Large portions of the data Possibly many joins

15 OLAP is Big Large portions of the data Possibly many joins Few long heavy queries

16 OLAP Examples Web analytics Sales analytics Management support Statistical analysis (census) Scientific databases (e.g., bio-informatics)

17 OLTP vs. OLAP OLTP Detailed Individual Records

18 OLTP vs. OLAP Detailed Individual Records vs. Historical Summarized Consolidated Data Aurelio Scetta / 123RF Stock Photo OLTP OLAP

19 OLTP vs. OLAP Lots of writes OLTP

20 OLTP vs. OLAP vs. Lots of writes OLTP Lots of reads OLAP

21 OLTP vs. OLAP Small sets of records OLTP

22 OLTP vs. OLAP Small sets of records vs. Analysis over big chunks OLTP OLAP

23 OLTP vs. OLAP fully interactive (< 1s) OLTP

24 OLTP vs. OLAP fully interactive (< 1s) vs. Slow interactive OLTP OLAP

25 OLTP vs. OLAP Redundancy Redundancy Redundancy Consistency OLTP OLAP

26 OLAP Aurelio Scetta / 123RF Stock Photo

27 A data warehouse... is a

28 A data warehouse... is a subject-oriented

29 A data warehouse... is a subject-oriented integrated

30 A data warehouse... is a subject-oriented integrated time-variant

31 A data warehouse... is a subject-oriented integrated time-variant nonvolatile

32 A data warehouse... is a subject-oriented integrated time-variant nonvolatile collection of data in support of management's decision-making process

33 Subject-oriented customers sales products events

34 Integrated

35 Integrated

36 Time-variant Time in data warehouses is paramount (not so in OLTP systems)

37 Time-variant Y-9 Y-8 Y-7 Y-6 Y-5 Y-4 Y-3 Y-2 Y Often past 5-10 years

38 Non-volatile Load. Milosh Kojadinovich / 123RF Stock Photo

39 Non-volatile Load. Access. Milosh Kojadinovich / 123RF Stock Photo

40 Non-volatile Load. Access. Period. Milosh Kojadinovich / 123RF Stock Photo

41 Non-volatile Load. Access. Period. no updates Milosh Kojadinovich / 123RF Stock Photo

42 Architecture ERP CRM OLTP Files

43 Architecture ERP CRM ETL OLTP Files

44 Architecture ERP Analyze CRM ETL Report OLTP Files Mine

45 OLAP: Redundancy

46 OLAP: Redundancy Materialized views (denormalized)

47 1st Normal Form (tabular) The Key

48 2nd Normal Form (not joined) The Whole Key

49 3rd Normal Form Nothing But The Key

50 Why materialize? Operational data sources are too heterogeneous

51 OLAP: Special-purpose indices

52 OLAP: Derived data

53 Querying OLAP 6 Slow interactive Category 1 Category 2 Category 3 Category 4 Series 1 Series 2 Series s

54 Querying OLAP 6 Slow interactive Category 1 Category 2 Category 3 Category 4 Series 1 Series 2 Series 3 vs. Continuous monitoring/tracking 1-10s hours

55 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated)

56 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support

57 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views

58 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs

59 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating

60 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes

61 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals

62 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals Freshness Serializability Reproducibility

63 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals Freshness Serializability Reproducibility Speed Very fast Often slow

64 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals Freshness Serializability Reproducibility Speed Very fast Often slow Optimization Inter-query Intra-query

65 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals Freshness Serializability Reproducibility Speed Very fast Often slow Optimization Inter-query Intra-query Space Small, archiving old data Large, less space efficient

66 Summary of differences OLTP OLAP Source Original (operational) Derived (consolidated) Purpose Business tasks Decision support Interface Snapshot Multidimensional views Writing short and fast, by end user period refreshes, by batch jobs Queries Simple, small results Complex and aggregating Design Many normalized tables Few denormalized cubes Precision ACID Sampling, confidence intervals Freshness Serializability Reproducibility Speed Very fast Often slow Optimization Inter-query Intra-query Space Small, archiving old data Large, less space efficient Backup Very important Re-ETL

67 Data Model

68 Data Cubes Data is stored in multidimensional hypercubes

69 Data Cubes Year

70 Data Cubes Country

71 Data Cubes Product

72 Fact 2016 CH Server

73 Dimensions Which currency? Where What? Who? When? etc.

74 Fact table Where? Germany 2016 Peter 1,000$ Germany 2015 Mary 15,000$ Switzerland 2016 Mary 1,500$ Switzerland 2015 Peter 3,000$ Australia 2015 Peter 6,000$ China 2015 Mary 1,000$

75 Aggregation Where? Germany 2016 Peter 1,000$ Germany 2015 Mary 15,000$ Switzerland 2016 Mary 1,500$ Switzerland 2015 Peter 3,000$ Australia 2015 Peter 6,000$ China 2015 Mary 1,000$

76 Aggregation

77 Aggregation Where? Germany 2016 Peter 1,000$ Germany 2015 Mary 15,000$ Switzerland 2016 Mary 1,500$ Switzerland 2015 Peter 3,000$ Australia 2015 Peter 6,000$ China 2015 Mary 1,000$

78 Aggregation 2016 Peter 1,000$ 2015 Mary 16,000$ 2016 Mary 1,500$ 2015 Peter 9,000$

79 Slicing

80 Slicers and Dicers Slicers Dicers

81 Slicers and Dicers Usually between 1 and 3 dicers, often 2 Slicers Dicers

82 Slicers and Dicers Slicers Servers World USD

83 Slicers and Dicers Dicers Slicers Servers World USD Peter 1,000,000$ 1,500,000$ 1,400,000$ Mary 2,000,000$ 2,300,000$ 2,200,000$

84 Products: the big three Essbase Cognos Analysis Services

85 ETLing

86 OLAP: Derived data

87 OLAP: Derived data ETL

88 ETL Extract Transform Load

89 Extract Triggers Incremental updates

90 Extract Triggers Gateways Incremental updates

91 Extract Triggers Gateways Incremental updates Log extraction

92 Transform Derivation

93 Transform Herr Derivation Mister Value transformation

94 Transform Herr Derivation Mister Value transformation Cleaning

95 Transform Herr Derivation Mister Value transformation Cleaning Filter, split, merge, join

96 Load Integrity constraints

97 Load Integrity constraints Sorting

98 Load Integrity constraints Sorting Build indices

99 Load Integrity constraints Sorting Build indices Partition

100 Considerations When?

101 Considerations When? Granularity

102 Considerations When? Granularity Infrastructure

103 Implementation

104 Two flavors of OLAP ROLAP

105 Two flavors of OLAP ROLAP MOLAP

106 Fact table (ROLAP) Dim1 Dim2 Dim3 Dim4 Dim5 Value

107 Star Schema Dim1 Dim2 Dim3 Dim4 Dim5 Value

108 Snow-flake schema Dim1 Dim2 Dim3 Dim4 Dim5 Value Normalize More

109 Querying

110 Querying cubes Tables: SQL

111 Querying cubes Tables: SQL Cubes: MDX

112 Sample Cube (ROLAP) Sales Date Product Customer Quantity iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary 3

113 Satellite tables Time Date Year Quarter Customer Name Country Phone Peter UK Mary Switzerland Product Name Brand Price iphone X Apple 1000 Samsung Galaxy S9 Samsung 800

114 Querying with SQL SELECT * FROM Sales s

115 Querying with SQL SELECT * FROM Sales s Sales Date Product Customer Quantity iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary 3

116 Joining with satellite tables SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t WHERE s.date = t.date

117 Joining with satellite tables SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t WHERE s.date = t.date Year Product Customer Quantity 2017 iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary 3

118 Slicing SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t WHERE s.date = t.date AND t.year = 2017

119 Slicing SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t WHERE s.date = t.date AND t.year = 2017 Year Product Customer Quantity 2017 iphone X Peter Samsung Galaxy S9 Peter iphone X Mary Samsung Galaxy S9 Mary 3

120 Slicing SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name AND t.year = 2017 AND p.brand = 'Samsung'

121 Slicing SELECT t.year, s.product, s.customer, s.quantity FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name AND t.year = 2017 AND p.brand = 'Samsung' Year Product Customer Quantity 2017 Samsung Galaxy S9 Peter Samsung Galaxy S9 Mary 3

122 Aggregating SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name AND p.brand = 'Samsung' GROUP BY t.year

123 Aggregating SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name AND p.brand = 'Samsung' GROUP BY t.year Date Quantity

124 Aggregating (Rolling up) SELECT SUM(s.Quantity) FROM Sales s

125 Aggregating (Rolling up) SELECT SUM(s.Quantity) FROM Sales s Quantity 34

126 Aggregating (Drilling down) SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand

127 Aggregating (Drilling down) SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand Sales Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4

128 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT SUM(s.Quantity) FROM Sales s

129 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year Roll up SELECT SUM(s.Quantity) FROM Sales s

130 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4 SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT SUM(s.Quantity) FROM Sales s

131 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4 Year Quantity SELECT SUM(s.Quantity) FROM Sales s

132 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT SUM(s.Quantity) FROM Sales s Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4 Year Quantity Quantity 34

133 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4 SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT null, null, SUM(s.Quantity) FROM Sales s

134 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT null, null, SUM(s.Quantity) FROM Sales s Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung 4 Year Quantity 2017 NULL NULL NULL 9 Quantity NULL NULL 34

135 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand) UNION (SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year) UNION (SELECT null, null, SUM(s.Quantity) FROM Sales s)

136 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand) UNION (SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year) UNION (SELECT null, null, SUM(s.Quantity) FROM Sales s) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL NULL 34

137 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY GROUPING SETS ( (t.year, p.brand), (t.year), () ) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL NULL 34

138 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY ROLLUP (t.year, p.brand) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL NULL 34

139 Materializing all aggregates SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand SELECT t.year, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year SELECT p.brand, SUM(s.Quantity) FROM Sales s, Product p WHERE s.product = p.name GROUP BY p.brand SELECT SUM(s.Quantity) FROM Sales s

140 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand) UNION (SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year) UNION (SELECT null, p.brand, SUM(s.Quantity) FROM Sales s, Product p WHERE s.product = p.name GROUP BY p.brand) UNION (SELECT null, null, SUM(s.Quantity) FROM Sales s)

141 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY t.year, p.brand) UNION (SELECT t.year, null, SUM(s.Quantity) FROM Sales s, Time t WHERE s.date = t.date GROUP BY t.year) UNION (SELECT null, p.brand, SUM(s.Quantity) FROM Sales s, Product p WHERE s.product = p.name GROUP BY p.brand) UNION (SELECT null, null, SUM(s.Quantity) FROM Sales s) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL Apple 20 NULL Samsung 14 NULL NULL 34

142 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY GROUPING SETS ( (t.year, p.brand), (t.year), (p.brand), () ) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL Apple 20 NULL Samsung 14 NULL NULL 34

143 Materializing all aggregates (SELECT t.year, p.brand, SUM(s.Quantity) FROM Sales s, Time t, Product p WHERE s.date = t.date AND s.product = p.name GROUP BY CUBE (t.year, p.brand) Year Brand Quantity 2017 Apple Samsung Apple Samsung Apple Samsung NULL NULL NULL 9 NULL Apple 20 NULL Samsung 14 NULL NULL 34

144 Cross tabulation Year Brand Apple Samsung Total

145 Cross tabulation (Slicer: Peter only) Year Brand Apple Samsung Total

146 Cross tabulation (Slicer: 2017) Customer Brand Apple Samsung Peter Mary Total

147 MDX stands for... Multi-Dimensional expressions

148 Measures Amount of licenses Revenues Taxes paid...

149 Dimensions Quarter Salesperson Product Country

150 In short... A cube is a list of dimensions indexing a list of measures

151 Hierarchies Dimension values are organized in hierarchies. [Location] [Geo] [Economy] i.e., slice and aggregate by geographic region, etc i.e., slice and aggregate by economic partnership, etc

152 Members Members correspond to levels in a hierarchy. [Ocenia] [Africa] [Geo] [Europe] [Asia] [America] [Switzerland] [China] [Canada] [ZH] [India] [USA] [BE]... [Brazil] [Germany]......

153 Identifying a member [Location].[Geo].[Europe].[Switzerland].[ZH].[Zurich]

154 Tuples A list of members ([Location].[Geo].[Europe].[Switzerland].[ZH].[Zurich], [Salesmen].[People].[John], [Time].[Year].[2016].[Q4]) Associated with a dimensionality (list of hierarchies) ([Location].[Geo], [Salesmen].[People] [Time].[Year])

155 Sets A set of tuples with same dimensionality { ([Location].[Geo].[Europe].[Switzerland].[ZH].[Zurich], [Salesmen].[People].[John], [Time].[Year].[2016].[Q4]), ([Location].[Geo].[Europe].[Switzerland].[BE].[Bärn], [Salesmen].[People].[Mary], [Time].[Year].[2016].[Q4]), ([Location].[Geo].[Europe].[Germany].[Berlin], [Salesmen].[People].[John], [Time].[Year].[2016].[Q3]) }

156 MDX statements: dicing SELECT [Measures].Members ON COLUMNS, [Location].[Geo].Members ON ROWS FROM [Sales]

157 MDX statements: slicing SELECT [Measures].Members ON COLUMNS, [Location].[Geo].Members ON ROWS FROM [Sales] WHERE [Products].[Line].[Laptops].[MBP]

158 Syntax

159 XBRL Architecture Linkbase (.xml) Schema (.xsd) Instance (.xml) Discoverable Taxonomy Set

160 Technologies XML XML Schema XML Names XML Link

161 Fact Dimension Value What? Assets Who? Coca Cola When? Dec 31, 2011 Of what? USD <us-gaap:assets contextref="fi2012q4" decimals="-6" id="fact fd4d06e63b4f8f6874c6e5be74" unitref="usd"> </us-gaap:assets>

162 Context <xbrli:context id="fi2011q4"> <xbrli:entity> <xbrli:identifier scheme=" </xbrli:identifier> </xbrli:entity> <xbrli:period> <xbrli:instant> </xbrli:instant> </xbrli:period> </xbrli:context> December

163 Unit <xbrli:unit id="usd"> <xbrli:measure>iso4217:usd</xbrli:measure> </xbrli:unit>

164 Concept (XML Schema) <xs:element id='us-gaap_assets' name='assets' nillable='true' substitutiongroup='xbrli:item' type='xbrli:monetaryitemtype' xbrli:balance='debit' xbrli:periodtype='instant' />

165 Graphs

166 DAGs

167 Trees

168 Node: locator <loc xlink:href=" xlink:label="loc_usgaap_assets_102d7a4d204ed45ac0deda6bbc78f38 6" xlink:type="locator" />

169 Node: resource <link:label id="lab_ko_netchangeinoperatingassetsandliabilitiesdisclosureabstrac t_a6469a522e35cbf ee_label_en-us" xlink:label="lab_ko_netchangeinoperatingassetsandliabilitiesdisclosur eabstract_a6469a522e35cbf ee" xlink:role=" xlink:type="resource" xml:lang="en-us"> NET CHANGE IN OPERATING ASSETS AND LIABILITIES DISCLOSURE [Abstract] </link:label>

170 Edge <presentationarc order="10" preferredlabel=" xlink:arcrole=" xlink:from="loc_usgaap_assetsabstract_2f55ecb2bf7c1a62009cda6bbc757094" xlink:to="loc_usgaap_assets_102d7a4d204ed45ac0deda6bbc78f386" xlink:type="arc" />

171 Summary

172 Architecture ERP Analyze CRM ETL Report OLTP Files Mine

Big Data 13. Data Warehousing

Big Data 13. Data Warehousing Ghislain Fourny Big Data 13. Data Warehousing fotoreactor / 123RF Stock Photo 2 The road to analytics Aurelio Scetta / 123RF Stock Photo 3 Another history of data management (T. Hofmann) 1970s 2000s Age

More information

IT DATA WAREHOUSING AND DATA MINING UNIT-2 BUSINESS ANALYSIS

IT DATA WAREHOUSING AND DATA MINING UNIT-2 BUSINESS ANALYSIS PART A 1. What are production reporting tools? Give examples. (May/June 2013) Production reporting tools will let companies generate regular operational reports or support high-volume batch jobs. Such

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 04-06 Data Warehouse Architecture Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology

More information

Data Mining Concepts & Techniques

Data Mining Concepts & Techniques Data Mining Concepts & Techniques Lecture No. 01 Databases, Data warehouse Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro

More information

Data Warehousing and Decision Support. Introduction. Three Complementary Trends. [R&G] Chapter 23, Part A

Data Warehousing and Decision Support. Introduction. Three Complementary Trends. [R&G] Chapter 23, Part A Data Warehousing and Decision Support [R&G] Chapter 23, Part A CS 432 1 Introduction Increasingly, organizations are analyzing current and historical data to identify useful patterns and support business

More information

Data Warehousing and Decision Support

Data Warehousing and Decision Support Data Warehousing and Decision Support Chapter 23, Part A Database Management Systems, 2 nd Edition. R. Ramakrishnan and J. Gehrke 1 Introduction Increasingly, organizations are analyzing current and historical

More information

Data Warehouses. Yanlei Diao. Slides Courtesy of R. Ramakrishnan and J. Gehrke

Data Warehouses. Yanlei Diao. Slides Courtesy of R. Ramakrishnan and J. Gehrke Data Warehouses Yanlei Diao Slides Courtesy of R. Ramakrishnan and J. Gehrke Introduction v In the late 80s and early 90s, companies began to use their DBMSs for complex, interactive, exploratory analysis

More information

CHAPTER 8 DECISION SUPPORT V2 ADVANCED DATABASE SYSTEMS. Assist. Prof. Dr. Volkan TUNALI

CHAPTER 8 DECISION SUPPORT V2 ADVANCED DATABASE SYSTEMS. Assist. Prof. Dr. Volkan TUNALI CHAPTER 8 DECISION SUPPORT V2 ADVANCED DATABASE SYSTEMS Assist. Prof. Dr. Volkan TUNALI Topics 2 Business Intelligence (BI) Decision Support System (DSS) Data Warehouse Online Analytical Processing (OLAP)

More information

Data Warehousing and Decision Support

Data Warehousing and Decision Support Data Warehousing and Decision Support [R&G] Chapter 23, Part A CS 4320 1 Introduction Increasingly, organizations are analyzing current and historical data to identify useful patterns and support business

More information

CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores

CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores Announcements Shumo office hours change See website for details HW2 due next Thurs

More information

Introduction to Data Warehousing

Introduction to Data Warehousing ICS 321 Spring 2012 Introduction to Data Warehousing Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 4/23/2012 Lipyeow Lim -- University of Hawaii at Manoa

More information

Deccansoft Software Services Microsoft Silver Learning Partner. SSAS Syllabus

Deccansoft Software Services Microsoft Silver Learning Partner. SSAS Syllabus Overview: Analysis Services enables you to analyze large quantities of data. With it, you can design, create, and manage multidimensional structures that contain detail and aggregated data from multiple

More information

Data Warehousing 2. ICS 421 Spring Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa

Data Warehousing 2. ICS 421 Spring Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa ICS 421 Spring 2010 Data Warehousing 2 Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 3/30/2010 Lipyeow Lim -- University of Hawaii at Manoa 1 Data Warehousing

More information

Evolution of Database Systems

Evolution of Database Systems Evolution of Database Systems Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Intelligent Decision Support Systems Master studies, second

More information

ETL and OLAP Systems

ETL and OLAP Systems ETL and OLAP Systems Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, first semester

More information

An Overview of Data Warehousing and OLAP Technology

An Overview of Data Warehousing and OLAP Technology An Overview of Data Warehousing and OLAP Technology CMPT 843 Karanjit Singh Tiwana 1 Intro and Architecture 2 What is Data Warehouse? Subject-oriented, integrated, time varying, non-volatile collection

More information

Acknowledgment. MTAT Data Mining. Week 7: Online Analytical Processing and Data Warehouses. Typical Data Analysis Process.

Acknowledgment. MTAT Data Mining. Week 7: Online Analytical Processing and Data Warehouses. Typical Data Analysis Process. MTAT.03.183 Data Mining Week 7: Online Analytical Processing and Data Warehouses Marlon Dumas marlon.dumas ät ut. ee Acknowledgment This slide deck is a mashup of the following publicly available slide

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 03 Architecture of DW Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro Basic

More information

Decision Support. Chapter 25. CS 286, UC Berkeley, Spring 2007, R. Ramakrishnan 1

Decision Support. Chapter 25. CS 286, UC Berkeley, Spring 2007, R. Ramakrishnan 1 Decision Support Chapter 25 CS 286, UC Berkeley, Spring 2007, R. Ramakrishnan 1 Introduction Increasingly, organizations are analyzing current and historical data to identify useful patterns and support

More information

Data Warehouse. Asst.Prof.Dr. Pattarachai Lalitrojwong

Data Warehouse. Asst.Prof.Dr. Pattarachai Lalitrojwong Data Warehouse Asst.Prof.Dr. Pattarachai Lalitrojwong Faculty of Information Technology King Mongkut s Institute of Technology Ladkrabang Bangkok 10520 pattarachai@it.kmitl.ac.th The Evolution of Data

More information

Chapter 18: Data Analysis and Mining

Chapter 18: Data Analysis and Mining Chapter 18: Data Analysis and Mining Database System Concepts See www.db-book.com for conditions on re-use Chapter 18: Data Analysis and Mining Decision Support Systems Data Analysis and OLAP 18.2 Decision

More information

Adnan YAZICI Computer Engineering Department

Adnan YAZICI Computer Engineering Department Data Warehouse Adnan YAZICI Computer Engineering Department Middle East Technical University, A.Yazici, 2010 Definition A data warehouse is a subject-oriented integrated time-variant nonvolatile collection

More information

CT75 DATA WAREHOUSING AND DATA MINING DEC 2015

CT75 DATA WAREHOUSING AND DATA MINING DEC 2015 Q.1 a. Briefly explain data granularity with the help of example Data Granularity: The single most important aspect and issue of the design of the data warehouse is the issue of granularity. It refers

More information

CS614 - Data Warehousing - Midterm Papers Solved MCQ(S) (1 TO 22 Lectures)

CS614 - Data Warehousing - Midterm Papers Solved MCQ(S) (1 TO 22 Lectures) CS614- Data Warehousing Solved MCQ(S) From Midterm Papers (1 TO 22 Lectures) BY Arslan Arshad Nov 21,2016 BS110401050 BS110401050@vu.edu.pk Arslan.arshad01@gmail.com AKMP01 CS614 - Data Warehousing - Midterm

More information

Data warehouses Decision support The multidimensional model OLAP queries

Data warehouses Decision support The multidimensional model OLAP queries Data warehouses Decision support The multidimensional model OLAP queries Traditional DBMSs are used by organizations for maintaining data to record day to day operations On-line Transaction Processing

More information

Data Warehousing and OLAP

Data Warehousing and OLAP Data Warehousing and OLAP INFO 330 Slides courtesy of Mirek Riedewald Motivation Large retailer Several databases: inventory, personnel, sales etc. High volume of updates Management requirements Efficient

More information

Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis

Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd rafal@projectbotticelli.com Objectives Explain the basics of: 1. Data

More information

Data Warehousing & OLAP

Data Warehousing & OLAP CMPUT 391 Database Management Systems Data Warehousing & OLAP Textbook: 17.1 17.5 (first edition: 19.1 19.5) Based on slides by Lewis, Bernstein and Kifer and other sources University of Alberta 1 Why

More information

SQL Server Analysis Services

SQL Server Analysis Services DataBase and Data Mining Group of DataBase and Data Mining Group of Database and data mining group, SQL Server 2005 Analysis Services SQL Server 2005 Analysis Services - 1 Analysis Services Database and

More information

DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY

DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY CHARACTERISTICS Data warehouse is a central repository for summarized and integrated data

More information

Database design View Access patterns Need for separate data warehouse:- A multidimensional data model:-

Database design View Access patterns Need for separate data warehouse:- A multidimensional data model:- UNIT III: Data Warehouse and OLAP Technology: An Overview : What Is a Data Warehouse? A Multidimensional Data Model, Data Warehouse Architecture, Data Warehouse Implementation, From Data Warehousing to

More information

collection of data that is used primarily in organizational decision making.

collection of data that is used primarily in organizational decision making. Data Warehousing A data warehouse is a special purpose database. Classic databases are generally used to model some enterprise. Most often they are used to support transactions, a process that is referred

More information

Data Modeling and Databases Ch 7: Schemas. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich

Data Modeling and Databases Ch 7: Schemas. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Data Modeling and Databases Ch 7: Schemas Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Database schema A Database Schema captures: The concepts represented Their attributes

More information

CSE 544 Principles of Database Management Systems. Fall 2016 Lecture 14 - Data Warehousing and Column Stores

CSE 544 Principles of Database Management Systems. Fall 2016 Lecture 14 - Data Warehousing and Column Stores CSE 544 Principles of Database Management Systems Fall 2016 Lecture 14 - Data Warehousing and Column Stores References Data Cube: A Relational Aggregation Operator Generalizing Group By, Cross-Tab, and

More information

Information Management course

Information Management course Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 05(b) : 23/10/2012 Data Mining: Concepts and Techniques (3 rd ed.) Chapter

More information

Data Warehousing and Decision Support (mostly using Relational Databases) CS634 Class 20

Data Warehousing and Decision Support (mostly using Relational Databases) CS634 Class 20 Data Warehousing and Decision Support (mostly using Relational Databases) CS634 Class 20 Slides based on Database Management Systems 3 rd ed, Ramakrishnan and Gehrke, Chapter 25 Introduction Increasingly,

More information

DATA MINING AND WAREHOUSING

DATA MINING AND WAREHOUSING DATA MINING AND WAREHOUSING Qno Question Answer 1 Define data warehouse? Data warehouse is a subject oriented, integrated, time-variant, and nonvolatile collection of data that supports management's decision-making

More information

Data Warehousing Conclusion. Esteban Zimányi Slides by Toon Calders

Data Warehousing Conclusion. Esteban Zimányi Slides by Toon Calders Data Warehousing Conclusion Esteban Zimányi ezimanyi@ulb.ac.be Slides by Toon Calders Motivation for the Course Database = a piece of software to handle data: Store, maintain, and query Most ideal system

More information

CS 1655 / Spring 2013! Secure Data Management and Web Applications

CS 1655 / Spring 2013! Secure Data Management and Web Applications CS 1655 / Spring 2013 Secure Data Management and Web Applications 03 Data Warehousing Alexandros Labrinidis University of Pittsburgh What is a Data Warehouse A data warehouse: archives information gathered

More information

Syllabus. Syllabus. Motivation Decision Support. Syllabus

Syllabus. Syllabus. Motivation Decision Support. Syllabus Presentation: Sophia Discussion: Tianyu Metadata Requirements and Conclusion 3 4 Decision Support Decision Making: Everyday, Everywhere Decision Support System: a class of computerized information systems

More information

Data warehouse architecture consists of the following interconnected layers:

Data warehouse architecture consists of the following interconnected layers: Architecture, in the Data warehousing world, is the concept and design of the data base and technologies that are used to load the data. A good architecture will enable scalability, high performance and

More information

Data Warehouses Chapter 12. Class 10: Data Warehouses 1

Data Warehouses Chapter 12. Class 10: Data Warehouses 1 Data Warehouses Chapter 12 Class 10: Data Warehouses 1 OLTP vs OLAP Operational Database: a database designed to support the day today transactions of an organization Data Warehouse: historical data is

More information

CHAPTER 8: ONLINE ANALYTICAL PROCESSING(OLAP)

CHAPTER 8: ONLINE ANALYTICAL PROCESSING(OLAP) CHAPTER 8: ONLINE ANALYTICAL PROCESSING(OLAP) INTRODUCTION A dimension is an attribute within a multidimensional model consisting of a list of values (called members). A fact is defined by a combination

More information

DATA MINING TRANSACTION

DATA MINING TRANSACTION DATA MINING Data Mining is the process of extracting patterns from data. Data mining is seen as an increasingly important tool by modern business to transform data into an informational advantage. It is

More information

A Multi-Dimensional Data Model

A Multi-Dimensional Data Model A Multi-Dimensional Data Model A Data Warehouse is based on a Multidimensional data model which views data in the form of a data cube A data cube, such as sales, allows data to be modeled and viewed in

More information

Processing of Very Large Data

Processing of Very Large Data Processing of Very Large Data Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, first

More information

This tutorial will help computer science graduates to understand the basic-to-advanced concepts related to data warehousing.

This tutorial will help computer science graduates to understand the basic-to-advanced concepts related to data warehousing. About the Tutorial A data warehouse is constructed by integrating data from multiple heterogeneous sources. It supports analytical reporting, structured and/or ad hoc queries and decision making. This

More information

Call: SAS BI Course Content:35-40hours

Call: SAS BI Course Content:35-40hours SAS BI Course Content:35-40hours Course Outline SAS Data Integration Studio 4.2 Introduction * to SAS DIS Studio Features of SAS DIS Studio Tasks performed by SAS DIS Studio Navigation to SAS DIS Studio

More information

Handout 12 Data Warehousing and Analytics.

Handout 12 Data Warehousing and Analytics. Handout 12 CS-605 Spring 17 Page 1 of 6 Handout 12 Data Warehousing and Analytics. Operational (aka transactional) system a system that is used to run a business in real time, based on current data; also

More information

Rocky Mountain Technology Ventures

Rocky Mountain Technology Ventures Rocky Mountain Technology Ventures Comparing and Contrasting Online Analytical Processing (OLAP) and Online Transactional Processing (OLTP) Architectures 3/19/2006 Introduction One of the most important

More information

Multidimensional Queries

Multidimensional Queries Multidimensional Queries Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, first semester

More information

Lectures for the course: Data Warehousing and Data Mining (IT 60107)

Lectures for the course: Data Warehousing and Data Mining (IT 60107) Lectures for the course: Data Warehousing and Data Mining (IT 60107) Week 1 Lecture 1 21/07/2011 Introduction to the course Pre-requisite Expectations Evaluation Guideline Term Paper and Term Project Guideline

More information

Data Warehouses and OLAP. Database and Information Systems. Data Warehouses and OLAP. Data Warehouses and OLAP

Data Warehouses and OLAP. Database and Information Systems. Data Warehouses and OLAP. Data Warehouses and OLAP Database and Information Systems 11. Deductive Databases 12. Data Warehouses and OLAP 13. Index Structures for Similarity Queries 14. Data Mining 15. Semi-Structured Data 16. Document Retrieval 17. Web

More information

Data Warehousing Introduction. Toon Calders

Data Warehousing Introduction. Toon Calders Data Warehousing Introduction Toon Calders toon.calders@ulb.ac.be Course Organization Lectures on Tuesday 14:00 and Friday 16:00 Check http://gehol.ulb.ac.be/ for room Most exercises in computer class

More information

REPORTING AND QUERY TOOLS AND APPLICATIONS

REPORTING AND QUERY TOOLS AND APPLICATIONS Tool Categories: REPORTING AND QUERY TOOLS AND APPLICATIONS There are five categories of decision support tools Reporting Managed query Executive information system OLAP Data Mining Reporting Tools Production

More information

One Size Fits All: An Idea Whose Time Has Come and Gone

One Size Fits All: An Idea Whose Time Has Come and Gone ICS 624 Spring 2013 One Size Fits All: An Idea Whose Time Has Come and Gone Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 1/9/2013 Lipyeow Lim -- University

More information

A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective

A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective B.Manivannan Research Scholar, Dept. Computer Science, Dravidian University, Kuppam, Andhra Pradesh, India

More information

Chapter 13 Business Intelligence and Data Warehouses The Need for Data Analysis Business Intelligence. Objectives

Chapter 13 Business Intelligence and Data Warehouses The Need for Data Analysis Business Intelligence. Objectives Chapter 13 Business Intelligence and Data Warehouses Objectives In this chapter, you will learn: How business intelligence is a comprehensive framework to support business decision making How operational

More information

Proceedings of the IE 2014 International Conference AGILE DATA MODELS

Proceedings of the IE 2014 International Conference  AGILE DATA MODELS AGILE DATA MODELS Mihaela MUNTEAN Academy of Economic Studies, Bucharest mun61mih@yahoo.co.uk, Mihaela.Muntean@ie.ase.ro Abstract. In last years, one of the most popular subjects related to the field of

More information

Summary of Last Chapter. Course Content. Chapter 2 Objectives. Data Warehouse and OLAP Outline. Incentive for a Data Warehouse

Summary of Last Chapter. Course Content. Chapter 2 Objectives. Data Warehouse and OLAP Outline. Incentive for a Data Warehouse Principles of Knowledge Discovery in bases Fall 1999 Chapter 2: Warehousing and Dr. Osmar R. Zaïane University of Alberta Dr. Osmar R. Zaïane, 1999 Principles of Knowledge Discovery in bases University

More information

What is a Data Warehouse?

What is a Data Warehouse? What is a Data Warehouse? COMP 465 Data Mining Data Warehousing Slides Adapted From : Jiawei Han, Micheline Kamber & Jian Pei Data Mining: Concepts and Techniques, 3 rd ed. Defined in many different ways,

More information

Data Mining & Data Warehouse

Data Mining & Data Warehouse Data Mining & Data Warehouse Associate Professor Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology (1) 2016 2017 1 Points to Cover Why Do We Need Data Warehouses?

More information

Teradata Aggregate Designer

Teradata Aggregate Designer Data Warehousing Teradata Aggregate Designer By: Sam Tawfik Product Marketing Manager Teradata Corporation Table of Contents Executive Summary 2 Introduction 3 Problem Statement 3 Implications of MOLAP

More information

Data Warehousing & OLAP

Data Warehousing & OLAP Data Warehousing & OLAP Data Mining: Concepts and Techniques Chapter 3 Jiawei Han and An Introduction to Database Systems C.J.Date, Eighth Eddition, Addidon Wesley, 4 1 What is Data Warehousing? What is

More information

CS 245: Database System Principles. Warehousing. Outline. What is a Warehouse? What is a Warehouse? Notes 13: Data Warehousing

CS 245: Database System Principles. Warehousing. Outline. What is a Warehouse? What is a Warehouse? Notes 13: Data Warehousing Recall : Database System Principles Notes 3: Data Warehousing Three approaches to information integration: Federated databases did teaser Data warehousing next Mediation Hector Garcia-Molina (Some modifications

More information

OLAP Introduction and Overview

OLAP Introduction and Overview 1 CHAPTER 1 OLAP Introduction and Overview What Is OLAP? 1 Data Storage and Access 1 Benefits of OLAP 2 What Is a Cube? 2 Understanding the Cube Structure 3 What Is SAS OLAP Server? 3 About Cube Metadata

More information

Step-by-step data transformation

Step-by-step data transformation Step-by-step data transformation Explanation of what BI4Dynamics does in a process of delivering business intelligence Contents 1. Introduction... 3 Before we start... 3 1 st. STEP: CREATING A STAGING

More information

BUSINESS INTELLIGENCE. SSAS - SQL Server Analysis Services. Business Informatics Degree

BUSINESS INTELLIGENCE. SSAS - SQL Server Analysis Services. Business Informatics Degree BUSINESS INTELLIGENCE SSAS - SQL Server Analysis Services Business Informatics Degree 2 BI Architecture SSAS: SQL Server Analysis Services 3 It is both an OLAP Server and a Data Mining Server Distinct

More information

Improving the Performance of OLAP Queries Using Families of Statistics Trees

Improving the Performance of OLAP Queries Using Families of Statistics Trees Improving the Performance of OLAP Queries Using Families of Statistics Trees Joachim Hammer Dept. of Computer and Information Science University of Florida Lixin Fu Dept. of Mathematical Sciences University

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 02 Introduction to Data Warehouse Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology

More information

Data Mining. Associate Professor Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology

Data Mining. Associate Professor Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Data Mining Associate Professor Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology (1) 2016 2017 Department of CS- DM - UHD 1 Points to Cover Why Do We Need Data

More information

1 Dulcian, Inc., 2001 All rights reserved. Oracle9i Data Warehouse Review. Agenda

1 Dulcian, Inc., 2001 All rights reserved. Oracle9i Data Warehouse Review. Agenda Agenda Oracle9i Warehouse Review Dulcian, Inc. Oracle9i Server OLAP Server Analytical SQL Mining ETL Infrastructure 9i Warehouse Builder Oracle 9i Server Overview E-Business Intelligence Platform 9i Server:

More information

Overview. Introduction to Data Warehousing and Business Intelligence. BI Is Important. What is Business Intelligence (BI)?

Overview. Introduction to Data Warehousing and Business Intelligence. BI Is Important. What is Business Intelligence (BI)? Introduction to Data Warehousing and Business Intelligence Overview Why Business Intelligence? Data analysis problems Data Warehouse (DW) introduction A tour of the coming DW lectures DW Applications Loosely

More information

1. Attempt any two of the following: 10 a. State and justify the characteristics of a Data Warehouse with suitable examples.

1. Attempt any two of the following: 10 a. State and justify the characteristics of a Data Warehouse with suitable examples. Instructions to the Examiners: 1. May the Examiners not look for exact words from the text book in the Answers. 2. May any valid example be accepted - example may or may not be from the text book 1. Attempt

More information

Data Warehousing. Data Warehousing and Mining. Lecture 8. by Hossen Asiful Mustafa

Data Warehousing. Data Warehousing and Mining. Lecture 8. by Hossen Asiful Mustafa Data Warehousing Data Warehousing and Mining Lecture 8 by Hossen Asiful Mustafa Databases Databases are developed on the IDEA that DATA is one of the critical materials of the Information Age Information,

More information

OLAP Systems and Multidimensional Expressions

OLAP Systems and Multidimensional Expressions OLAP Systems and Multidimensional Expressions Krzysztof Dembczyński Institute of Computing Science Laboratory of Intelligent Decision Support Systems Politechnika Poznańska (Poznań University of Technology)

More information

Create Cube From Star Schema Grouping Framework Manager

Create Cube From Star Schema Grouping Framework Manager Create Cube From Star Schema Grouping Framework Manager Create star schema groupings to provide authors with logical groupings of query Connect to an OLAP data source (cube) in a Framework Manager project

More information

CS377: Database Systems Data Warehouse and Data Mining. Li Xiong Department of Mathematics and Computer Science Emory University

CS377: Database Systems Data Warehouse and Data Mining. Li Xiong Department of Mathematics and Computer Science Emory University CS377: Database Systems Data Warehouse and Data Mining Li Xiong Department of Mathematics and Computer Science Emory University 1 1960s: Evolution of Database Technology Data collection, database creation,

More information

IDU0010 ERP,CRM ja DW süsteemid Loeng 5 DW concepts. Enn Õunapuu

IDU0010 ERP,CRM ja DW süsteemid Loeng 5 DW concepts. Enn Õunapuu IDU0010 ERP,CRM ja DW süsteemid Loeng 5 DW concepts Enn Õunapuu enn.ounapuu@ttu.ee Content Oveall approach Dimensional model Tabular model Overall approach Data modeling is a discipline that has been practiced

More information

CT75 (ALCCS) DATA WAREHOUSING AND DATA MINING JUN

CT75 (ALCCS) DATA WAREHOUSING AND DATA MINING JUN Q.1 a. Define a Data warehouse. Compare OLTP and OLAP systems. Data Warehouse: A data warehouse is a subject-oriented, integrated, time-variant, and 2 Non volatile collection of data in support of management

More information

Unit 7: Basics in MS Power BI for Excel 2013 M7-5: OLAP

Unit 7: Basics in MS Power BI for Excel 2013 M7-5: OLAP Unit 7: Basics in MS Power BI for Excel M7-5: OLAP Outline: Introduction Learning Objectives Content Exercise What is an OLAP Table Operations: Drill Down Operations: Roll Up Operations: Slice Operations:

More information

Decision Support Systems aka Analytical Systems

Decision Support Systems aka Analytical Systems Decision Support Systems aka Analytical Systems Decision Support Systems Systems that are used to transform data into information, to manage the organization: OLAP vs OLTP OLTP vs OLAP Transactions Analysis

More information

Data Warehousing Introduction. Esteban Zimanyi Slides from Toon Calders

Data Warehousing Introduction. Esteban Zimanyi Slides from Toon Calders Data Warehousing Introduction Esteban Zimanyi ezimanyi@ulb.ac.be Slides from Toon Calders Course Organization Lectures on Tuesday 14:00 and Thursday 16:00 Check http://gehol.ulb.ac.be/ for room Most exercises

More information

CHAPTER 3 Implementation of Data warehouse in Data Mining

CHAPTER 3 Implementation of Data warehouse in Data Mining CHAPTER 3 Implementation of Data warehouse in Data Mining 3.1 Introduction to Data Warehousing A data warehouse is storage of convenient, consistent, complete and consolidated data, which is collected

More information

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI Department of Information Technology IT6702 Data Warehousing & Data Mining Anna University 2 & 16 Mark Questions & Answers Year / Semester: IV / VII Regulation:

More information

Data Warehousing and OLAP Technologies for Decision-Making Process

Data Warehousing and OLAP Technologies for Decision-Making Process Data Warehousing and OLAP Technologies for Decision-Making Process Hiren H Darji Asst. Prof in Anand Institute of Information Science,Anand Abstract Data warehousing and on-line analytical processing (OLAP)

More information

1. Inroduction to Data Mininig

1. Inroduction to Data Mininig 1. Inroduction to Data Mininig 1.1 Introduction Universe of Data Information Technology has grown in various directions in the recent years. One natural evolutionary path has been the development of the

More information

On-Line Application Processing

On-Line Application Processing On-Line Application Processing WAREHOUSING DATA CUBES DATA MINING 1 Overview Traditional database systems are tuned to many, small, simple queries. Some new applications use fewer, more time-consuming,

More information

Course Contents: 1 Business Objects Online Training

Course Contents: 1 Business Objects Online Training IQ Online training facility offers Business Objects online training by trainers who have expert knowledge in the Business Objects and proven record of training hundreds of students Our Business Objects

More information

Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services

Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services Course 6234A: Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services Course Details Course Outline Module 1: Introduction to Microsoft SQL Server Analysis Services This module introduces

More information

Data Mining. Data warehousing. Hamid Beigy. Sharif University of Technology. Fall 1396

Data Mining. Data warehousing. Hamid Beigy. Sharif University of Technology. Fall 1396 Data Mining Data warehousing Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1396 1 / 31 Table of contents 1 Introduction 2 Data warehousing

More information

Data Warehousing and Data Mining SQL OLAP Operations

Data Warehousing and Data Mining SQL OLAP Operations Data Warehousing and Data Mining SQL OLAP Operations SQL table expression query specification query expression SQL OLAP GROUP BY extensions: rollup, cube, grouping sets Acknowledgements: I am indebted

More information

DATA WAREHOUING UNIT I

DATA WAREHOUING UNIT I BHARATHIDASAN ENGINEERING COLLEGE NATTRAMAPALLI DEPARTMENT OF COMPUTER SCIENCE SUB CODE & NAME: IT6702/DWDM DEPT: IT Staff Name : N.RAMESH DATA WAREHOUING UNIT I 1. Define data warehouse? NOV/DEC 2009

More information

Data Mining. ❸Chapter 3 Data warehouse, ETL and OLAP. Asso.Prof.Dr. Xiao-dong Zhu. Business School, University of Shanghai for Science & Technology

Data Mining. ❸Chapter 3 Data warehouse, ETL and OLAP. Asso.Prof.Dr. Xiao-dong Zhu. Business School, University of Shanghai for Science & Technology ❸Chapter 3 Data warehouse, and Business School, University of Shanghai for Science & Technology 2016-2017 2nd Semester, Spring2017 Contents of chapter 2 1 KDD Process 2 3 4 5 What is KDD? KDD Process the

More information

Data Mining. Data warehousing. Hamid Beigy. Sharif University of Technology. Fall 1394

Data Mining. Data warehousing. Hamid Beigy. Sharif University of Technology. Fall 1394 Data Mining Data warehousing Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1394 1 / 22 Table of contents 1 Introduction 2 Data warehousing

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2017/18 Unit 10 J. Gamper 1/37 Advanced Data Management Technologies Unit 10 SQL GROUP BY Extensions J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE Acknowledgements: I

More information

Decision Support, Data Warehousing, and OLAP

Decision Support, Data Warehousing, and OLAP Decision Support, Data Warehousing, and OLAP : Contents Terminology : OLAP vs. OLTP Data Warehousing Architecture Technologies References 1 Decision Support and OLAP Information technology to help knowledge

More information

Introduction to DWH / BI Concepts

Introduction to DWH / BI Concepts SAS INTELLIGENCE PLATFORM CURRICULUM SAS INTELLIGENCE PLATFORM BI TOOLS 4.2 VERSION SAS BUSINESS INTELLIGENCE TOOLS - COURSE OUTLINE Practical Project Based Training & Implementation on all the BI Tools

More information

QA Microsoft Designing Business Intelligence Solutions with Microsoft SQL Server 2012

QA Microsoft Designing Business Intelligence Solutions with Microsoft SQL Server 2012 70-467.176.QA Number: 70-467 Passing Score: 800 Time Limit: 120 min File Version: 6.7 Microsoft 70-467 Designing Business Intelligence Solutions with Microsoft SQL Server 2012 Testlet 1 Tailspin Toys Case

More information

Deccansoft Software Services. SSIS Syllabus

Deccansoft Software Services. SSIS Syllabus Overview: SQL Server Integration Services (SSIS) is a component of Microsoft SQL Server database software which can be used to perform a broad range of data migration, data integration and Data Consolidation

More information