Data Mining & Data Warehouse

Size: px
Start display at page:

Download "Data Mining & Data Warehouse"

Transcription

1 Data Mining & Data Warehouse Asso. Profe. Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of Information Technology (1)

2 Points to Cover Problem: Heterogeneous Information Sources Data integration Extract, Transform, Load (ETL) Data Warehouse With The ETL Process Problem: Data Management in Large Enterprises Goals of Data Integration Current Solutions Three-layer Architecture: Conceptual View Generic Warehouse Architecture Department of IT- DMDW - UHD 1-2

3 Problem: Heterogeneous Information Sources Heterogeneities are everywhere Personal Databases Scientific Databases Different interfaces Different data representations Duplicate and inconsistent information Digital Libraries World Wide Web Department of IT- DMDW - UHD 3

4 Data integration Data integration involves combining data residing in different sources and providing users with a unified view of these data. This process becomes significant in a variety of situations, which include both commercial (when two similar companies need to merge their databases) and scientific research results. Department of IT- DMDW - UHD 4

5 Goal: Unified Access to Data Integration System World Wide Web Digital Libraries Scientific Databases Personal Databases Collects and combines information Provides integrated view, uniform user interface Supports sharing among different applications Department of IT- DMDW - UHD 5

6 Goals of Data Integration Provide 1) Uniform (same query interface to all sources) 2) Access to (updates the database) 3) Multiple (we want many users at each time) 4) Heterogeneous (data models are different) 5) Distributed (over LAN, WAN, Internet) 6) Data Sources (not only databases). Department of IT- DMDW - UHD 6

7 Extract, Transform, Load (ETL) Extract, Transform, Load (ETL) refers to a process in database. Data extraction is where data is extracted from heterogeneous data sources; Data transformation where the data is transformed for storing in the proper format or structure. Data loading where the data is loaded into the final target more specifically, an operational data store, data mart, or data warehouse. Department of IT- DMDW - UHD 7

8 Data Warehouse With The ETL Process A simple schematic for a data warehouse with the ETL process extracts information from the source databases, transforms it and then loads it into the data warehouse. Department of IT- DMDW - UHD 8

9 Data Management in Large Enterprises Database Management system (DBMS) Result of different applications development of operational systems Examples Sales Planning Suppliers Control system Sales Administration Finance Manufacturing Department of IT- DMDW - UHD 9

10 Extract all the data into a single data source Data Warehouse Extract all the data into a single data source Query Data Warehouse Clean the data Load Periodically Data Sources Department of IT- DMDW - UHD 10

11 Data Warehouse Architectures: Conceptual View Single-layer Every data element is stored once only Virtual warehouse: is another term for a data warehouse. Operational systems Informational systems Real-time data Department of IT- DMDW - UHD 11

12 Data Warehouse Architectures: Conceptual View Two-layer Real-time + derived data Most commonly used approach in industry today Operational systems Informational systems Derived Data Real-time data Department of IT- DMDW - UHD 12

13 Three-layer Architecture: Conceptual View Transformation of real-time data to derived data really requires reconciliation step Operational systems Derived Data Reconciliation Real-time data Informational systems Reconciliation is the process of ensuring that two sets of records are in agreement. Example:- Reconciliation is used to ensure that the money leaving an account matches the actual money spent. Department of IT- DMDW - UHD 13

14 Data Warehousing: Two Distinct Issues (1) How to get information into warehouse Data warehousing (2) What to do with data once it s in warehouse Warehouse DBMS Both rich research areas Industry has focused on (2) Department of IT- DMDW - UHD 14

15 Issues in Data Warehousing Warehouse Design Extraction Integration Cleansing & merging Warehousing specification & Maintenance Optimizations Evolution Department of IT- DMDW - UHD 15

16 Generic Warehouse Architecture Client Design Phase Maintenance Query & Analysis Warehouse Integration Client Loading Metadata Extractor/ Monitor Extractor/ Monitor Extractor/ Monitor... Department of IT- DMDW - UHD 16

17 Problems with DW Approach Data has to be cleaned different formats Needs to store all the data from different data sources in single system Data needs to be updated periodically Data sources are independent content can change without notice Expensive because of the large quantities of data and data cleaning costs Department of IT- DMDW - UHD 17

18 Department of IT- DMDW - UHD Department of IT- DMDW - UHD 18

Data Mining. Asso. Profe. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of CS (1)

Data Mining. Asso. Profe. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of CS (1) Data Mining Asso. Profe. Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of CS 2016 2017 (1) Points to Cover Problem: Heterogeneous Information Sources

More information

Data Mining. Associate Professor Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology

Data Mining. Associate Professor Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Data Mining Associate Professor Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology (1) 2016 2017 Department of CS- DM - UHD 1 Points to Cover Why Do We Need Data

More information

Data Mining & Data Warehouse

Data Mining & Data Warehouse Data Mining & Data Warehouse Associate Professor Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology (1) 2016 2017 1 Points to Cover Why Do We Need Data Warehouses?

More information

Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science

Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science Data Mining Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of Computer Science 06 07 Department of CS - DM - UHD Road map Cluster Analysis: Basic

More information

STRATEGIC INFORMATION SYSTEMS IV STV401T / B BTIP05 / BTIX05 - BTECH DEPARTMENT OF INFORMATICS. By: Dr. Tendani J. Lavhengwa

STRATEGIC INFORMATION SYSTEMS IV STV401T / B BTIP05 / BTIX05 - BTECH DEPARTMENT OF INFORMATICS. By: Dr. Tendani J. Lavhengwa STRATEGIC INFORMATION SYSTEMS IV STV401T / B BTIP05 / BTIX05 - BTECH DEPARTMENT OF INFORMATICS LECTURE: 05 (A) DATA WAREHOUSING (DW) By: Dr. Tendani J. Lavhengwa lavhengwatj@tut.ac.za 1 My personal quote:

More information

INTRODUCTORY INFORMATION TECHNOLOGY ENTERPRISE DATABASES AND DATA WAREHOUSES. Faramarz Hendessi

INTRODUCTORY INFORMATION TECHNOLOGY ENTERPRISE DATABASES AND DATA WAREHOUSES. Faramarz Hendessi INTRODUCTORY INFORMATION TECHNOLOGY ENTERPRISE DATABASES AND DATA WAREHOUSES Faramarz Hendessi INTRODUCTORY INFORMATION TECHNOLOGY Lecture 7 Fall 2010 Isfahan University of technology Dr. Faramarz Hendessi

More information

Information Management course

Information Management course Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 05(b) : 23/10/2012 Data Mining: Concepts and Techniques (3 rd ed.) Chapter

More information

QM Chapter 1 Database Fundamentals Version 10 th Ed. Prepared by Dr Kamel Rouibah / Dept QM & IS

QM Chapter 1 Database Fundamentals Version 10 th Ed. Prepared by Dr Kamel Rouibah / Dept QM & IS QM 433 - Chapter 1 Database Fundamentals Version 10 th Ed Prepared by Dr Kamel Rouibah / Dept QM & IS www.cba.edu.kw/krouibah Dr K. Rouibah / dept QM & IS Chapter 1 (433) Database fundamentals 1 Objectives

More information

A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective

A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective A Novel Approach of Data Warehouse OLTP and OLAP Technology for Supporting Management prospective B.Manivannan Research Scholar, Dept. Computer Science, Dravidian University, Kuppam, Andhra Pradesh, India

More information

Question Bank. 4) It is the source of information later delivered to data marts.

Question Bank. 4) It is the source of information later delivered to data marts. Question Bank Year: 2016-2017 Subject Dept: CS Semester: First Subject Name: Data Mining. Q1) What is data warehouse? ANS. A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile

More information

Overview. Introduction to Data Warehousing and Business Intelligence. BI Is Important. What is Business Intelligence (BI)?

Overview. Introduction to Data Warehousing and Business Intelligence. BI Is Important. What is Business Intelligence (BI)? Introduction to Data Warehousing and Business Intelligence Overview Why Business Intelligence? Data analysis problems Data Warehouse (DW) introduction A tour of the coming DW lectures DW Applications Loosely

More information

DATAWAREHOUSING AND ETL PROCESSES: An Explanatory Research

DATAWAREHOUSING AND ETL PROCESSES: An Explanatory Research DATAWAREHOUSING AND ETL PROCESSES: An Explanatory Research Priyanshu Gupta ETL Software Developer United Health Group Abstract- In this paper, the author has focused on explaining Data Warehousing and

More information

Data Warehousing. Ritham Vashisht, Sukhdeep Kaur and Shobti Saini

Data Warehousing. Ritham Vashisht, Sukhdeep Kaur and Shobti Saini Advance in Electronic and Electric Engineering. ISSN 2231-1297, Volume 3, Number 6 (2013), pp. 669-674 Research India Publications http://www.ripublication.com/aeee.htm Data Warehousing Ritham Vashisht,

More information

CHAPTER 3 Implementation of Data warehouse in Data Mining

CHAPTER 3 Implementation of Data warehouse in Data Mining CHAPTER 3 Implementation of Data warehouse in Data Mining 3.1 Introduction to Data Warehousing A data warehouse is storage of convenient, consistent, complete and consolidated data, which is collected

More information

The Evolution of Data Warehousing. Data Warehousing Concepts. The Evolution of Data Warehousing. The Evolution of Data Warehousing

The Evolution of Data Warehousing. Data Warehousing Concepts. The Evolution of Data Warehousing. The Evolution of Data Warehousing The Evolution of Data Warehousing Data Warehousing Concepts Since 1970s, organizations gained competitive advantage through systems that automate business processes to offer more efficient and cost-effective

More information

Chapter 1 Database System Concepts and Architecture. Nguyen Thi Ai Thao

Chapter 1 Database System Concepts and Architecture. Nguyen Thi Ai Thao Chapter 1 Database System Concepts and Architecture Nguyen Thi Ai Thao thaonguyen@cse.hcmut.edu.vn Spring - 2016 Contents 1 -based Approach and Database Approach 2 Three-Schema Architecture and Data Independence

More information

Seminars of Software and Services for the Information Society. Data Warehousing Design Issues

Seminars of Software and Services for the Information Society. Data Warehousing Design Issues DIPARTIMENTO DI INGEGNERIA INFORMATICA AUTOMATICA E GESTIONALE ANTONIO RUBERTI Master of Science in Engineering in Computer Science (MSE-CS) Seminars in Software and Services for the Information Society

More information

Training 24x7 DBA Support Staffing. MCSA:SQL 2016 Business Intelligence Development. Implementing an SQL Data Warehouse. (40 Hours) Exam

Training 24x7 DBA Support Staffing. MCSA:SQL 2016 Business Intelligence Development. Implementing an SQL Data Warehouse. (40 Hours) Exam MCSA:SQL 2016 Business Intelligence Development Implementing an SQL Data Warehouse (40 Hours) Exam 70-767 Prerequisites At least 2 years experience of working with relational databases, including: Designing

More information

Full file at

Full file at Chapter 2 Data Warehousing True-False Questions 1. A real-time, enterprise-level data warehouse combined with a strategy for its use in decision support can leverage data to provide massive financial benefits

More information

Chapter 6. Foundations of Business Intelligence: Databases and Information Management VIDEO CASES

Chapter 6. Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Chapter 6 Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:

More information

Business Intelligence and Decision Support Systems

Business Intelligence and Decision Support Systems Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 8: Data Warehousing Learning Objectives Understand the basic definitions and concepts of data warehouses Learn different

More information

Introduction to DWML. Christian Thomsen, Aalborg University. Slides adapted from Torben Bach Pedersen and Man Lung Yiu

Introduction to DWML. Christian Thomsen, Aalborg University. Slides adapted from Torben Bach Pedersen and Man Lung Yiu Introduction to DWML Christian Thomsen, Aalborg University Slides adapted from Torben Bach Pedersen and Man Lung Yiu Course Structure Business intelligence Extract knowledge from large amounts of data

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 03 Architecture of DW Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro Basic

More information

1 Dulcian, Inc., 2001 All rights reserved. Oracle9i Data Warehouse Review. Agenda

1 Dulcian, Inc., 2001 All rights reserved. Oracle9i Data Warehouse Review. Agenda Agenda Oracle9i Warehouse Review Dulcian, Inc. Oracle9i Server OLAP Server Analytical SQL Mining ETL Infrastructure 9i Warehouse Builder Oracle 9i Server Overview E-Business Intelligence Platform 9i Server:

More information

A New Approach of Extraction Transformation Loading Using Pipelining

A New Approach of Extraction Transformation Loading Using Pipelining A New Approach of Extraction Transformation Loading Using Pipelining Dr. Rajender Singh Chhillar* (Professor, CS Department, M.D.U) Barjesh Kochar (Head(MCA,IT),GNIM) Abstract Companies have lots of valuable

More information

DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY

DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY DATA WAREHOUSE EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY CHARACTERISTICS Data warehouse is a central repository for summarized and integrated data

More information

Duration: 5 Days. EZY Intellect Pte. Ltd.,

Duration: 5 Days. EZY Intellect Pte. Ltd., Implementing a SQL Data Warehouse Duration: 5 Days Course Code: 20767A Course review About this course This 5-day instructor led course describes how to implement a data warehouse platform to support a

More information

Implementing a SQL Data Warehouse

Implementing a SQL Data Warehouse Implementing a SQL Data Warehouse 20767B; 5 days, Instructor-led Course Description This 4-day instructor led course describes how to implement a data warehouse platform to support a BI solution. Students

More information

Fig 1.2: Relationship between DW, ODS and OLTP Systems

Fig 1.2: Relationship between DW, ODS and OLTP Systems 1.4 DATA WAREHOUSES Data warehousing is a process for assembling and managing data from various sources for the purpose of gaining a single detailed view of an enterprise. Although there are several definitions

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 05 Data Modeling Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro Data Modeling

More information

Testing Masters Technologies

Testing Masters Technologies 1. What is Data warehouse ETL TESTING Q&A Ans: A Data warehouse is a subject oriented, integrated,time variant, non volatile collection of data in support of management's decision making process. Subject

More information

Implementing a Data Warehouse with Microsoft SQL Server 2014

Implementing a Data Warehouse with Microsoft SQL Server 2014 Course 20463D: Implementing a Data Warehouse with Microsoft SQL Server 2014 Page 1 of 5 Implementing a Data Warehouse with Microsoft SQL Server 2014 Course 20463D: 4 days; Instructor-Led Introduction This

More information

Data Warehousing Introduction. Toon Calders

Data Warehousing Introduction. Toon Calders Data Warehousing Introduction Toon Calders toon.calders@ulb.ac.be Course Organization Lectures on Tuesday 14:00 and Friday 16:00 Check http://gehol.ulb.ac.be/ for room Most exercises in computer class

More information

DATA WAREHOUSING APPLICATIONS IN AYURVEDA THERAPY

DATA WAREHOUSING APPLICATIONS IN AYURVEDA THERAPY DATA WAREHOUSING APPLICATIONS IN AYURVEDA THERAPY Prof. Deepa Nyayadhish Asst. Professor, V.K.K.Menon College, Mumbai Dr. Pramod Deo Asst. Professor, S.B.E.S. College, Aurangabad ABSTRACT Availability

More information

Chapter 3. Foundations of Business Intelligence: Databases and Information Management

Chapter 3. Foundations of Business Intelligence: Databases and Information Management Chapter 3 Foundations of Business Intelligence: Databases and Information Management THE DATA HIERARCHY TRADITIONAL FILE PROCESSING Organizing Data in a Traditional File Environment Problems with the traditional

More information

Data Warehouse and Data Mining

Data Warehouse and Data Mining Data Warehouse and Data Mining Lecture No. 04-06 Data Warehouse Architecture Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology

More information

DATABASE DEVELOPMENT (H4)

DATABASE DEVELOPMENT (H4) IMIS HIGHER DIPLOMA QUALIFICATIONS DATABASE DEVELOPMENT (H4) December 2017 10:00hrs 13:00hrs DURATION: 3 HOURS Candidates should answer ALL the questions in Part A and THREE of the five questions in Part

More information

Microsoft Implementing a Data Warehouse with Microsoft SQL Server 2014

Microsoft Implementing a Data Warehouse with Microsoft SQL Server 2014 1800 ULEARN (853 276) www.ddls.com.au Microsoft 20463 - Implementing a Data Warehouse with Microsoft SQL Server 2014 Length 5 days Price $4290.00 (inc GST) Version D Overview Please note: Microsoft have

More information

Information Integration

Information Integration Information Integration Part 1: Basics of Relational Database Theory Werner Nutt Faculty of Computer Science Master of Science in Computer Science A.Y. 2012/2013 Integration in Data Management: Evolution

More information

CS377: Database Systems Data Warehouse and Data Mining. Li Xiong Department of Mathematics and Computer Science Emory University

CS377: Database Systems Data Warehouse and Data Mining. Li Xiong Department of Mathematics and Computer Science Emory University CS377: Database Systems Data Warehouse and Data Mining Li Xiong Department of Mathematics and Computer Science Emory University 1 1960s: Evolution of Database Technology Data collection, database creation,

More information

Fast Innovation requires Fast IT

Fast Innovation requires Fast IT Fast Innovation requires Fast IT Cisco Data Virtualization Puneet Kumar Bhugra Business Solutions Manager 1 Challenge In Data, Big Data & Analytics Siloed, Multiple Sources Business Outcomes Business Opportunity:

More information

Fundamentals of Information Systems, Seventh Edition

Fundamentals of Information Systems, Seventh Edition Chapter 3 Data Centers, and Business Intelligence 1 Why Learn About Database Systems, Data Centers, and Business Intelligence? Database: A database is an organized collection of data. Databases also help

More information

CT75 DATA WAREHOUSING AND DATA MINING DEC 2015

CT75 DATA WAREHOUSING AND DATA MINING DEC 2015 Q.1 a. Briefly explain data granularity with the help of example Data Granularity: The single most important aspect and issue of the design of the data warehouse is the issue of granularity. It refers

More information

Data Quality Architecture and Options

Data Quality Architecture and Options Data Quality Architecture and Options Nita Khare Alliances & Technology Team - Solution Architect nita.khare@tcs.com * IBM IM Champion 2013 * December 3, 2013 0 Agenda Pain Areas / Challenges of DQ Solution

More information

Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science

Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science Data Mining Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of Computer Science 2016 201 Road map What is Cluster Analysis? Characteristics of Clustering

More information

Management Information Systems MANAGING THE DIGITAL FIRM, 12 TH EDITION FOUNDATIONS OF BUSINESS INTELLIGENCE: DATABASES AND INFORMATION MANAGEMENT

Management Information Systems MANAGING THE DIGITAL FIRM, 12 TH EDITION FOUNDATIONS OF BUSINESS INTELLIGENCE: DATABASES AND INFORMATION MANAGEMENT MANAGING THE DIGITAL FIRM, 12 TH EDITION Chapter 6 FOUNDATIONS OF BUSINESS INTELLIGENCE: DATABASES AND INFORMATION MANAGEMENT VIDEO CASES Case 1: Maruti Suzuki Business Intelligence and Enterprise Databases

More information

This tutorial will help computer science graduates to understand the basic-to-advanced concepts related to data warehousing.

This tutorial will help computer science graduates to understand the basic-to-advanced concepts related to data warehousing. About the Tutorial A data warehouse is constructed by integrating data from multiple heterogeneous sources. It supports analytical reporting, structured and/or ad hoc queries and decision making. This

More information

Chapter 3. Database Architecture and the Web

Chapter 3. Database Architecture and the Web Chapter 3 Database Architecture and the Web 1 Chapter 3 - Objectives Software components of a DBMS. Client server architecture and advantages of this type of architecture for a DBMS. Function and uses

More information

Exam /Course 20767B: Implementing a SQL Data Warehouse

Exam /Course 20767B: Implementing a SQL Data Warehouse Exam 70-767/Course 20767B: Implementing a SQL Data Warehouse Course Outline Module 1: Introduction to Data Warehousing This module describes data warehouse concepts and architecture consideration. Overview

More information

by Prentice Hall

by Prentice Hall Chapter 6 Foundations of Business Intelligence: Databases and Information Management 6.1 2010 by Prentice Hall Organizing Data in a Traditional File Environment File organization concepts Computer system

More information

This tutorial has been prepared for computer science graduates to help them understand the basic-to-advanced concepts related to data mining.

This tutorial has been prepared for computer science graduates to help them understand the basic-to-advanced concepts related to data mining. About the Tutorial Data Mining is defined as the procedure of extracting information from huge sets of data. In other words, we can say that data mining is mining knowledge from data. The tutorial starts

More information

Sql Fact Constellation Schema In Data Warehouse With Example

Sql Fact Constellation Schema In Data Warehouse With Example Sql Fact Constellation Schema In Data Warehouse With Example Data Warehouse OLAP - Learn Data Warehouse in simple and easy steps using Multidimensional OLAP (MOLAP), Hybrid OLAP (HOLAP), Specialized SQL

More information

Data-Driven Driven Business Intelligence Systems: Parts I. Lecture Outline. Learning Objectives

Data-Driven Driven Business Intelligence Systems: Parts I. Lecture Outline. Learning Objectives Data-Driven Driven Business Intelligence Systems: Parts I Week 5 Dr. Jocelyn San Pedro School of Information Management & Systems Monash University IMS3001 BUSINESS INTELLIGENCE SYSTEMS SEM 1, 2004 Lecture

More information

20767B: IMPLEMENTING A SQL DATA WAREHOUSE

20767B: IMPLEMENTING A SQL DATA WAREHOUSE ABOUT THIS COURSE This 5-day instructor led course describes how to implement a data warehouse platform to support a BI solution. Students will learn how to create a data warehouse with Microsoft SQL Server

More information

Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D)

Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D) Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D) Overview This course describes how to implement a data warehouse platform to support a BI solution. Students will learn how to create

More information

Data Mining and Warehousing

Data Mining and Warehousing Data Mining and Warehousing Sangeetha K V I st MCA Adhiyamaan College of Engineering, Hosur-635109. E-mail:veerasangee1989@gmail.com Rajeshwari P I st MCA Adhiyamaan College of Engineering, Hosur-635109.

More information

Data Warehouse and Mining

Data Warehouse and Mining Data Warehouse and Mining 1. is a subject-oriented, integrated, time-variant, nonvolatile collection of data in support of management decisions. A. Data Mining. B. Data Warehousing. C. Web Mining. D. Text

More information

IJSRD - International Journal for Scientific Research & Development Vol. 4, Issue 06, 2016 ISSN (online):

IJSRD - International Journal for Scientific Research & Development Vol. 4, Issue 06, 2016 ISSN (online): IJSRD - International Journal for Scientific Research & Development Vol. 4, Issue 06, 2016 ISSN (online): 2321-0613 Tanzeela Khanam 1 Pravin S.Metkewar 2 1 Student 2 Associate Professor 1,2 SICSR, affiliated

More information

Knowledge Modelling and Management. Part B (9)

Knowledge Modelling and Management. Part B (9) Knowledge Modelling and Management Part B (9) Yun-Heh Chen-Burger http://www.aiai.ed.ac.uk/~jessicac/project/kmm 1 A Brief Introduction to Business Intelligence 2 What is Business Intelligence? Business

More information

Data Integration and Data Warehousing Database Integration Overview

Data Integration and Data Warehousing Database Integration Overview Data Integration and Data Warehousing Database Integration Overview Sergey Stupnikov Institute of Informatics Problems, RAS ssa@ipi.ac.ru Outline Information Integration Problem Heterogeneous Information

More information

Data Warehousing & Mining. Data integration. OLTP versus OLAP. CPS 116 Introduction to Database Systems

Data Warehousing & Mining. Data integration. OLTP versus OLAP. CPS 116 Introduction to Database Systems Data Warehousing & Mining CPS 116 Introduction to Database Systems Data integration 2 Data resides in many distributed, heterogeneous OLTP (On-Line Transaction Processing) sources Sales, inventory, customer,

More information

Knowledge/Data Management. MIS 4133 Software Systems

Knowledge/Data Management. MIS 4133 Software Systems Knowledge/Data Management MIS 4133 Software Systems Outline Managing Data Technical Aspects Managerial Aspects Data Warehousing Data Mart Data Mining Knowledge Management Why Manage Data? Organizations

More information

Data Warehousing & Mining Techniques

Data Warehousing & Mining Techniques Data Warehousing & Mining Techniques Wolf-Tilo Balke Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 2. Summary Last week: What is a Data

More information

Outline. Managing Information Resources. Concepts and Definitions. Introduction. Chapter 7

Outline. Managing Information Resources. Concepts and Definitions. Introduction. Chapter 7 Outline Managing Information Resources Chapter 7 Introduction Managing Data The Three-Level Database Model Four Data Models Getting Corporate Data into Shape Managing Information Four Types of Information

More information

Chapter 6 VIDEO CASES

Chapter 6 VIDEO CASES Chapter 6 Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:

More information

Figure 1-1a Data in context. Context helps users understand data

Figure 1-1a Data in context. Context helps users understand data Chapter 1: The Database Environment Modern Database Management 9 th Edition Jeffrey A. Hoffer, Mary B. Prescott, Heikki Topi 2009 Pearson Education, Inc. Publishing as Prentice Hall 1 Definition of terms

More information

Implementing a SQL Data Warehouse

Implementing a SQL Data Warehouse Course 20767B: Implementing a SQL Data Warehouse Page 1 of 7 Implementing a SQL Data Warehouse Course 20767B: 4 days; Instructor-Led Introduction This 4-day instructor led course describes how to implement

More information

Managing Changes to Schema of Data Sources in a Data Warehouse

Managing Changes to Schema of Data Sources in a Data Warehouse Association for Information Systems AIS Electronic Library (AISeL) AMCIS 2001 Proceedings Americas Conference on Information Systems (AMCIS) December 2001 Managing Changes to Schema of Data Sources in

More information

Managing a Multi-iierData Warehousing Environment with the SAS/Warehouse Adminlstrator. Darrell Barton, SAS Institute Inc.

Managing a Multi-iierData Warehousing Environment with the SAS/Warehouse Adminlstrator. Darrell Barton, SAS Institute Inc. Managing a Multi-iierData Warehousing Environment with the SAS/Warehouse Adminlstrator Darrell Barton, SAS nstitute nc., Cary, NC ABSTRACT This paper describes a multi-tier computing environment and demonstrates

More information

1. Inroduction to Data Mininig

1. Inroduction to Data Mininig 1. Inroduction to Data Mininig 1.1 Introduction Universe of Data Information Technology has grown in various directions in the recent years. One natural evolutionary path has been the development of the

More information

IT1105 Information Systems and Technology. BIT 1 ST YEAR SEMESTER 1 University of Colombo School of Computing. Student Manual

IT1105 Information Systems and Technology. BIT 1 ST YEAR SEMESTER 1 University of Colombo School of Computing. Student Manual IT1105 Information Systems and Technology BIT 1 ST YEAR SEMESTER 1 University of Colombo School of Computing Student Manual Lesson 3: Organizing Data and Information (6 Hrs) Instructional Objectives Students

More information

High Speed ETL on Low Budget

High Speed ETL on Low Budget High Speed ETL on Low Budget Introduction Data Acquisition & populating it in a warehouse has traditionally been carried out using dedicated ETL tools available in the market. An enterprise-wide Data Warehousing

More information

2. Summary. 2.1 Basic Architecture. 2. Architecture. 2.1 Staging Area. 2.1 Operational Data Store. Last week: Architecture and Data model

2. Summary. 2.1 Basic Architecture. 2. Architecture. 2.1 Staging Area. 2.1 Operational Data Store. Last week: Architecture and Data model 2. Summary Data Warehousing & Mining Techniques Wolf-Tilo Balke Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Last week: What is a Data

More information

Data Warehousing. Seminar report. Submitted in partial fulfillment of the requirement for the award of degree Of Computer Science

Data Warehousing. Seminar report.  Submitted in partial fulfillment of the requirement for the award of degree Of Computer Science A Seminar report On Data Warehousing Submitted in partial fulfillment of the requirement for the award of degree Of Computer Science SUBMITTED TO: SUBMITTED BY: www.studymafia.org www.studymafia.org Preface

More information

Data Warehousing ETL. Esteban Zimányi Slides by Toon Calders

Data Warehousing ETL. Esteban Zimányi Slides by Toon Calders Data Warehousing ETL Esteban Zimányi ezimanyi@ulb.ac.be Slides by Toon Calders 1 Overview Picture other sources Metadata Monitor & Integrator OLAP Server Analysis Operational DBs Extract Transform Load

More information

Data Warehousing & Mining Techniques

Data Warehousing & Mining Techniques 2. Summary Data Warehousing & Mining Techniques Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Last week: What is a Data

More information

What s New in SAS/Warehouse Administrator

What s New in SAS/Warehouse Administrator What s New in SAS/Warehouse Administrator Scott Anderson, Wilbram Hazejager SAS Institute EMEA Agenda Product positioning Product history What s new since last time? Product demonstration Future plans

More information

DATA WAREHOUSE- MODEL QUESTIONS

DATA WAREHOUSE- MODEL QUESTIONS DATA WAREHOUSE- MODEL QUESTIONS 1. The generic two-level data warehouse architecture includes which of the following? a. At least one data mart b. Data that can extracted from numerous internal and external

More information

Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis

Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd rafal@projectbotticelli.com Objectives Explain the basics of: 1. Data

More information

OBJECTIVES DEFINITIONS CHAPTER 1: THE DATABASE ENVIRONMENT AND DEVELOPMENT PROCESS. Figure 1-1a Data in context

OBJECTIVES DEFINITIONS CHAPTER 1: THE DATABASE ENVIRONMENT AND DEVELOPMENT PROCESS. Figure 1-1a Data in context OBJECTIVES CHAPTER 1: THE DATABASE ENVIRONMENT AND DEVELOPMENT PROCESS Modern Database Management 11 th Edition Jeffrey A. Hoffer, V. Ramesh, Heikki Topi! Define terms! Name limitations of conventional

More information

Databases and Data Warehouses

Databases and Data Warehouses Databases and Data Warehouses Content Concept Definitions of Databases,Data Warehouses Database models History Databases Data Warehouses OLTP vs. Data Warehouse Concept Definition Database Data Warehouse

More information

1/12/2018. APPA Institute Dallas, TX Feb DATA INTEGRATION PURPOSE OF TODAY S PRESENTATION

1/12/2018. APPA Institute Dallas, TX Feb DATA INTEGRATION PURPOSE OF TODAY S PRESENTATION DATA INTEGRATION APPA Institute for Facilities Management January 23, 2018 Portland, OR PURPOSE OF TODAY S PRESENTATION To provide a broad understanding of: Data as a utility How various units of Facilities

More information

Managing Data Resources

Managing Data Resources Chapter 7 OBJECTIVES Describe basic file organization concepts and the problems of managing data resources in a traditional file environment Managing Data Resources Describe how a database management system

More information

Data Warehousing and Data Mining. Announcements (December 1) Data integration. CPS 116 Introduction to Database Systems

Data Warehousing and Data Mining. Announcements (December 1) Data integration. CPS 116 Introduction to Database Systems Data Warehousing and Data Mining CPS 116 Introduction to Database Systems Announcements (December 1) 2 Homework #4 due today Sample solution available Thursday Course project demo period has begun! Check

More information

ETL Interview Question Bank

ETL Interview Question Bank ETL Interview Question Bank Author: - Sheetal Shirke Version: - Version 0.1 ETL Architecture Diagram 1 ETL Testing Questions 1. What is Data WareHouse? A data warehouse (DW or DWH), also known as an enterprise

More information

Decision Guidance. Data Vault in Data Warehousing

Decision Guidance. Data Vault in Data Warehousing Decision Guidance Data Vault in Data Warehousing DATA VAULT IN DATA WAREHOUSING Today s business environment requires data models, which are resilient to change and enable the integration of multiple data

More information

Whitepaper. Solving Complex Hierarchical Data Integration Issues. What is Complex Data? Types of Data

Whitepaper. Solving Complex Hierarchical Data Integration Issues. What is Complex Data? Types of Data Whitepaper Solving Complex Hierarchical Data Integration Issues What is Complex Data? Historically, data integration and warehousing has consisted of flat or structured data that typically comes from structured

More information

Evolving To The Big Data Warehouse

Evolving To The Big Data Warehouse Evolving To The Big Data Warehouse Kevin Lancaster 1 Copyright Director, 2012, Oracle and/or its Engineered affiliates. All rights Insert Systems, Information Protection Policy Oracle Classification from

More information

Application software office packets, databases and data warehouses.

Application software office packets, databases and data warehouses. Introduction to Computer Systems (9) Application software office packets, databases and data warehouses. Piotr Mielecki Ph. D. http://www.wssk.wroc.pl/~mielecki piotr.mielecki@pwr.edu.pl pmielecki@gmail.com

More information

After completing this course, participants will be able to:

After completing this course, participants will be able to: Designing a Business Intelligence Solution by Using Microsoft SQL Server 2008 T h i s f i v e - d a y i n s t r u c t o r - l e d c o u r s e p r o v i d e s i n - d e p t h k n o w l e d g e o n d e s

More information

Data Mining: Approach Towards The Accuracy Using Teradata!

Data Mining: Approach Towards The Accuracy Using Teradata! Data Mining: Approach Towards The Accuracy Using Teradata! Shubhangi Pharande Department of MCA NBNSSOCS,Sinhgad Institute Simantini Nalawade Department of MCA NBNSSOCS,Sinhgad Institute Ajay Nalawade

More information

Managing Information Resources

Managing Information Resources Managing Information Resources 1 Managing Data 2 Managing Information 3 Managing Contents Concepts & Definitions Data Facts devoid of meaning or intent e.g. structured data in DB Information Data that

More information

Implement a Data Warehouse with Microsoft SQL Server

Implement a Data Warehouse with Microsoft SQL Server Implement a Data Warehouse with Microsoft SQL Server 20463D; 5 days, Instructor-led Course Description This course describes how to implement a data warehouse platform to support a BI solution. Students

More information

Summary of Last Chapter. Course Content. Chapter 2 Objectives. Data Warehouse and OLAP Outline. Incentive for a Data Warehouse

Summary of Last Chapter. Course Content. Chapter 2 Objectives. Data Warehouse and OLAP Outline. Incentive for a Data Warehouse Principles of Knowledge Discovery in bases Fall 1999 Chapter 2: Warehousing and Dr. Osmar R. Zaïane University of Alberta Dr. Osmar R. Zaïane, 1999 Principles of Knowledge Discovery in bases University

More information

CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores

CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores Announcements Shumo office hours change See website for details HW2 due next Thurs

More information

20463C-Implementing a Data Warehouse with Microsoft SQL Server. Course Content. Course ID#: W 35 Hrs. Course Description: Audience Profile

20463C-Implementing a Data Warehouse with Microsoft SQL Server. Course Content. Course ID#: W 35 Hrs. Course Description: Audience Profile Course Content Course Description: This course describes how to implement a data warehouse platform to support a BI solution. Students will learn how to create a data warehouse 2014, implement ETL with

More information

VERITAS Storage Foundation for Windows FlashSnap Option

VERITAS Storage Foundation for Windows FlashSnap Option VERITAS Storage Foundation for Windows FlashSnap Option Snapshot Technology for Microsoft Windows Server 2000 and Windows Server 2003 August 13, 2004 1 TABLE OF CONTENTS Introduction...3 Fast Data Recovery...3

More information

Copyright 2016 Datalynx Pty Ltd. All rights reserved. Datalynx Enterprise Data Management Solution Catalogue

Copyright 2016 Datalynx Pty Ltd. All rights reserved. Datalynx Enterprise Data Management Solution Catalogue Datalynx Enterprise Data Management Solution Catalogue About Datalynx Vendor of the world s most versatile Enterprise Data Management software Licence our software to clients & partners Partner-based sales

More information

SAS Data Integration Studio 3.3. User s Guide

SAS Data Integration Studio 3.3. User s Guide SAS Data Integration Studio 3.3 User s Guide The correct bibliographic citation for this manual is as follows: SAS Institute Inc. 2006. SAS Data Integration Studio 3.3: User s Guide. Cary, NC: SAS Institute

More information

DKMS Brief No. Five: Is Data Staging Relational? A Comment

DKMS Brief No. Five: Is Data Staging Relational? A Comment 1 of 6 5/24/02 3:39 PM DKMS Brief No. Five: Is Data Staging Relational? A Comment Introduction In the data warehousing process, the data staging area is composed of the data staging server application

More information