Efficient Computation of Data Cubes. Network Database Lab
|
|
- Johnathan Chase
- 5 years ago
- Views:
Transcription
1 Efficient Computation of Data Cubes Network Database Lab
2 Outlines Introduction Some CUBE Algorithms ArrayCube PartitionedCube and MemoryCube Bottom-Up Cube (BUC) Conclusions References Network Database Lab 2
3 Introduction Network Database Lab 3
4 Introduction Precompute aggregates Improve the response time of aggregation queries Datacube operator GROUP-BY, CUBE BY In OLAP, The grouping attributes dimensions The attributes that are aggregated measures One particular GROUP BY in a CUBE computation cuboid, group-by Network Database Lab 4
5 The basic CUBE problem To compute all of the aggregations as efficiently as possible. Exponential in the number of dimensions: for d dimensions, 2 d group-bys are computed. The size of each group-by depends upon the cardinality of its dimension. Fit in memory. Sparseness: Large domain size of some group-by attributes A large number of group-by attributes in the datacube query Network Database Lab 5
6 How to Simultaneously computing the aggregates Share partitioning and aggregation costs between various group-bys. Network Database Lab 6
7 Some CUBE Algorithms PipeSort, PipeHash, Overlap ArrayCube [ZDN97] PartitionCube and MemoryCube [RS97] Bottom-up Cube (BUC) [BR99] Network Database Lab 7
8 PipeSort Search the space of possible sort orders for the best set of sorts that convert the CUBE lattice into a processing tree. Network Database Lab 8
9 PipeSort Minimize the number of sorts. Compute a group-by from its smallest parent. Take advantage of common prefix. Various paths are evaluated in turn. Limitation: Does not scale well with respect to the number of group-by attributes. k At least sorts exponential in k k / 2 No partition. Network Database Lab 9
10 PipeHash Compute a group-by from its smallest parent in the lattice. Use a hash table for every simultaneously computed group-by. If all of the hash tables cannot fit in memory, PipeHash partitions the data on some attribute and processes each partition independently. Problems: Do not overlap as much computation as PipeSort. Require a significant amount of memory to store the hash tables. Network Database Lab 10
11 Overlap Try to minimize the number of disk access. Overlap as much sorting as possible by computing a group-by from a parent that shares the longest prefix. If a group-by shares a prefix with its parent, then the parent consists of a number of partitions, one for each value of the prefix. Try to fit as many partitions in memory as possible to avoid writing intermediate results. Network Database Lab 11
12 Common Problems Generate significant I/O by sorting intermediate results. Require large amounts of main memory. Do not work well on sparse CUBEs. Network Database Lab 12
13 ArrayCube Compute the CUBE for Multidimensional OLAP (MOLAP) systems. Array storage issues: Too large to fit in memory split up into chunks Many of the cells in the array are empty compress these chunks Not array format data load array from tables Network Database Lab 13
14 Basic Array cubing algorithm Construct the minimum size spanning tree for the group-bys of the Cube. Compute any group-by D i1 D i2..d ik of a Cube from the parent D i1 D i2..d ik+1, which has the minimum size. Read in each chunk of D i1 D i2..d ik+1 along the dimension D ik+1 and aggregate each chunk to a chunk of D i1 D i2..d ik. Keep only one D i1 D i2..d ik chunk in memory at any time. Network Database Lab 14
15 Example The array ABC is a array with array chunks. BC group-by: Read from 1 to 64 Aggregate each four ABC chunks to a BC chunk Output the BC chunk to disk Reuse the memory for the next BC chunk. Compute AB, AC, BC from ABC independently. Network Database Lab 15
16 Multi-Way Array Algorithm Overlap the computations of the different group-bys, thus avoiding the multiple scans required by the naive algorithm. Try to minimize the memory needed for each computation, so that we can achieve maximum overlap. Network Database Lab 16
17 Multi-Way Array Algorithm Dimension Order Row major order of the chunks with the n dimensions D 1, D 2,, D n, in some order O = (D j1, D j2,, D jn ). n 1 Memory Requirements D i Ci i= 1 i= p+ 1 Minimum Memory Spanning Tree(MMST) Optimal Dimension Order O = (D 1, D 2,, D n ), where p D 1 D... 2 D n Network Database Lab 17
18 Example The array ABC is a array with array chunks. Laid out in the dimension order ABC. Memory required: b 0 c 0 : chunk 1,2,3,4 4 4 a 0 c 0 : chunk 1,5,9, a 0 b 0 : chunk 1,17,33, Network Database Lab 18
19 Example (MMST) Network Database Lab 19
20 Example (Dimension Order) The array ABCD is a array with array chunks. Network Database Lab 20
21 PartitionCube and MemoryCube Partition the large relations into fragments that fit in memory. Perform the complex operation over each memory-sized fragment independently. Network Database Lab 21
22 Algorithm Partition-Cube Fix each possible value of a group-by attribute B j in turn and computing the tuples in the corresponding sub-datacube, followed by computing the datacube tuples with the value ALL for B j. Rather than re-reading the input relation R for the ALL datacube, we read the finest granularity cuboid. The datacube is broken up into n+1 smaller subdatacube computations. Network Database Lab 22
23 Partition Example Network Database Lab 23
24 Algorithm Memory-Cube Paths in the search lattice k At least paths (PipeSort) k k / 2 k / 2 is also upper-bound Sharing sort work Computing the cube by traversing paths Network Database Lab 24
25 Example G(1) = D ε G(2) = CD D ε D G(3) = BCD BD B ε BD D CD D G(4) = ABCD ABD AB A ε ABD AD D ACD AD D BCD BD B BD CD Network Database Lab 25
26 Bottom-up Cube (BUC) Combine the I/O efficiency of PartitionCube/MemoryCube. Take advantage of minimum support pruning like Apriori (Iceberg-CUBE problem). Proceed from the bottom of the lattice, and works its way up toward the larger, less aggregated group-bys. Network Database Lab 26
27 Example Network Database Lab 27
28 Features of BUC The elimination of the aggregation and partitioning of single tuple partitions is a key factor in the success of BUC on sparse CUBEs because many partitions have a single tuple. BUC does not try to share the computation of aggregates between parent and child group-bys, only the partitioning costs. Partitioning is the major expense, not aggregation. Network Database Lab 28
29 Conclusions What is important in computing datacube? Memory requirement partitioning Costs CPU time, I/O time Share costs partitioning, sorting, aggregation Which algorithm is best (fast)? Dense CUBE ArrayCube Sparse CUBE BUC Network Database Lab 29
30 References [ZDN97] Y. Zhao, P. M. Deshpande, and J. F. Naughton. An array-based algorithm for simultaneous multidimensional aggregates. SIGMOD 97, pages , May 1997 [RS97] K. Ross and D. Srivastava. Fast computation of sparse datacubes. VLDB 97, pages , Aug [BR99] K. Beyer and R. Ramakrishnan. Bottomup computation of sparse and iceberg cubes. SIGMOD 99, pages , June 1999 Network Database Lab 30
Data Warehousing and Data Mining
Data Warehousing and Data Mining Lecture 3 Efficient Cube Computation CITS3401 CITS5504 Wei Liu School of Computer Science and Software Engineering Faculty of Engineering, Computing and Mathematics Acknowledgement:
More informationChapter 5, Data Cube Computation
CSI 4352, Introduction to Data Mining Chapter 5, Data Cube Computation Young-Rae Cho Associate Professor Department of Computer Science Baylor University A Roadmap for Data Cube Computation Full Cube Full
More informationData Cube Technology
Data Cube Technology Erwin M. Bakker & Stefan Manegold https://homepages.cwi.nl/~manegold/dbdm/ http://liacs.leidenuniv.nl/~bakkerem2/dbdm/ s.manegold@liacs.leidenuniv.nl e.m.bakker@liacs.leidenuniv.nl
More informationData Cube Technology. Chapter 5: Data Cube Technology. Data Cube: A Lattice of Cuboids. Data Cube: A Lattice of Cuboids
Chapter 5: Data Cube Technology Data Cube Technology Data Cube Computation: Basic Concepts Data Cube Computation Methods Erwin M. Bakker & Stefan Manegold https://homepages.cwi.nl/~manegold/dbdm/ http://liacs.leidenuniv.nl/~bakkerem2/dbdm/
More information2 CONTENTS
Contents 4 Data Cube Computation and Data Generalization 3 4.1 Efficient Methods for Data Cube Computation............................. 3 4.1.1 A Road Map for Materialization of Different Kinds of Cubes.................
More informationComputing Complex Iceberg Cubes by Multiway Aggregation and Bounding
Computing Complex Iceberg Cubes by Multiway Aggregation and Bounding LienHua Pauline Chou and Xiuzhen Zhang School of Computer Science and Information Technology RMIT University, Melbourne, VIC., Australia,
More informationMulti-Cube Computation
Multi-Cube Computation Jeffrey Xu Yu Department of Sys. Eng. and Eng. Management The Chinese University of Hong Kong Hong Kong, China yu@se.cuhk.edu.hk Hongjun Lu Department of Computer Science Hong Kong
More informationLecture 2 Data Cube Basics
CompSci 590.6 Understanding Data: Theory and Applica>ons Lecture 2 Data Cube Basics Instructor: Sudeepa Roy Email: sudeepa@cs.duke.edu 1 Today s Papers 1. Gray- Chaudhuri- Bosworth- Layman- Reichart- Venkatrao-
More informationCount of Group-bys Sizes < X 400. Sum of Group-bys Sizes < X Group-by Size / Input Size
Bottom-Up Computation of Sparse and Iceberg CUBEs Kevin Beyer Computer Sciences Department University of Wisconsin { Madison beyer@cs.wisc.edu Raghu Ramakrishnan Computer Sciences Department University
More informationData Mining: Concepts and Techniques. (3 rd ed.) Chapter 5
Data Mining: Concepts and Techniques (3 rd ed.) Chapter 5 Jiawei Han, Micheline Kamber, and Jian Pei University of Illinois at Urbana-Champaign & Simon Fraser University 2013 Han, Kamber & Pei. All rights
More informationDifferent Cube Computation Approaches: Survey Paper
Different Cube Computation Approaches: Survey Paper Dhanshri S. Lad #, Rasika P. Saste * # M.Tech. Student, * M.Tech. Student Department of CSE, Rajarambapu Institute of Technology, Islampur(Sangli), MS,
More informationComputing Data Cubes Using Massively Parallel Processors
Computing Data Cubes Using Massively Parallel Processors Hongjun Lu Xiaohui Huang Zhixian Li {luhj,huangxia,lizhixia}@iscs.nus.edu.sg Department of Information Systems and Computer Science National University
More informationInternational Journal of Computer Sciences and Engineering. Research Paper Volume-6, Issue-1 E-ISSN:
International Journal of Computer Sciences and Engineering Open Access Research Paper Volume-6, Issue-1 E-ISSN: 2347-2693 Precomputing Shell Fragments for OLAP using Inverted Index Data Structure D. Datta
More informationUsing Tiling to Scale Parallel Data Cube Construction
Using Tiling to Scale Parallel Data Cube Construction Ruoming in Karthik Vaidyanathan Ge Yang Gagan Agrawal Department of Computer Science and Engineering Ohio State University, Columbus OH 43210 jinr,vaidyana,yangg,agrawal
More informationPARSIMONY: An Infrastructure for Parallel Multidimensional Analysis and Data Mining
Journal of Parallel and Distributed Computing 61, 285321 (2001) doi:10.1006jpdc.2000.1691, available online at http:www.idealibrary.com on PARSIMONY: An Infrastructure for Parallel Multidimensional Analysis
More informationPnP: Parallel And External Memory Iceberg Cube Computation
: Parallel And External Memory Iceberg Cube Computation Ying Chen Dalhousie University Halifax, Canada ychen@cs.dal.ca Frank Dehne Griffith University Brisbane, Australia www.dehne.net Todd Eavis Concordia
More informationDW Performance Optimization (II)
DW Performance Optimization (II) Overview Data Cube in ROLAP and MOLAP ROLAP Technique(s) Efficient Data Cube Computation MOLAP Technique(s) Prefix Sum Array Multiway Augmented Tree Aalborg University
More informationAn Empirical Comparison of Methods for Iceberg-CUBE Construction. Leah Findlater and Howard J. Hamilton Technical Report CS August, 2000
An Empirical Comparison of Methods for Iceberg-CUBE Construction Leah Findlater and Howard J. Hamilton Technical Report CS-2-6 August, 2 Copyright 2, Leah Findlater and Howard J. Hamilton Department of
More informationCoarse Grained Parallel On-Line Analytical Processing (OLAP) for Data Mining
Coarse Grained Parallel On-Line Analytical Processing (OLAP) for Data Mining Frank Dehne 1,ToddEavis 2, and Andrew Rau-Chaplin 2 1 Carleton University, Ottawa, Canada, frank@dehne.net, WWW home page: http://www.dehne.net
More informationEFFICIENT computation of data cubes has been one of the
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 19, NO. 1, JANUARY 2007 111 Computing Iceberg Cubes by Top-Down and Bottom-Up Integration: The StarCubing Approach Dong Xin, Student Member, IEEE,
More informationSameet Agarwal Rakesh Agrawal Prasad M. Deshpande Ashish Gupta. Jerey F. Naughton Raghu Ramakrishnan Sunita Sarawagi
On the Computation of Multidimensional Aggregates Sameet Agarwal Rakesh Agrawal Prasad M. Deshpande Ashish Gupta Jerey F. Naughton Raghu Ramakrishnan Sunita Sarawagi Abstract At the heart of all OLAP or
More informationCommunication and Memory Optimal Parallel Data Cube Construction
Communication and Memory Optimal Parallel Data Cube Construction Ruoming Jin Ge Yang Karthik Vaidyanathan Gagan Agrawal Department of Computer and Information Sciences Ohio State University, Columbus OH
More informationAn Empirical Comparison of Methods for Iceberg-CUBE Construction
From: FLAIRS-01 Proceedings. Copyright 2001, AAAI (www.aaai.org). All rights reserved. An Empirical Comparison of Methods for Iceberg-CUBE Construction Leah Findlater and Howard J. Hamilton Department
More informationFrequent Pattern Mining. Based on: Introduction to Data Mining by Tan, Steinbach, Kumar
Frequent Pattern Mining Based on: Introduction to Data Mining by Tan, Steinbach, Kumar Item sets A New Type of Data Some notation: All possible items: Database: T is a bag of transactions Transaction transaction
More informationMap-Reduce for Cube Computation
299 Map-Reduce for Cube Computation Prof. Pramod Patil 1, Prini Kotian 2, Aishwarya Gaonkar 3, Sachin Wani 4, Pramod Gaikwad 5 Department of Computer Science, Dr.D.Y.Patil Institute of Engineering and
More informationDATA MINING II - 1DL460
DATA MINING II - 1DL460 Spring 2013 " An second class in data mining http://www.it.uu.se/edu/course/homepage/infoutv2/vt13 Kjell Orsborn Uppsala Database Laboratory Department of Information Technology,
More informationChapter 4: Mining Frequent Patterns, Associations and Correlations
Chapter 4: Mining Frequent Patterns, Associations and Correlations 4.1 Basic Concepts 4.2 Frequent Itemset Mining Methods 4.3 Which Patterns Are Interesting? Pattern Evaluation Methods 4.4 Summary Frequent
More informationQuotient Cube: How to Summarize the Semantics of a Data Cube
Quotient Cube: How to Summarize the Semantics of a Data Cube Laks V.S. Lakshmanan (Univ. of British Columbia) * Jian Pei (State Univ. of New York at Buffalo) * Jiawei Han (Univ. of Illinois at Urbana-Champaign)
More informationCS 6093 Lecture 7 Spring 2011 Basic Data Mining. Cong Yu 03/21/2011
CS 6093 Lecture 7 Spring 2011 Basic Data Mining Cong Yu 03/21/2011 Announcements No regular office hour next Monday (March 28 th ) Office hour next week will be on Tuesday through Thursday by appointment
More informationChapter 4: Association analysis:
Chapter 4: Association analysis: 4.1 Introduction: Many business enterprises accumulate large quantities of data from their day-to-day operations, huge amounts of customer purchase data are collected daily
More informationMulti-Dimensional Partitioning in BUC for Data Cubes
San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research 2008 Multi-Dimensional Partitioning in BUC for Data Cubes Kenneth Yeung San Jose State University Follow
More informationMade available courtesy of Springer Verlag Germany: The original publication is available at
CubiST++: Evaluating Ad-Hoc CUBE Queries Using Statistics Trees By: Joachim Hammer, and Lixin Fu Joachim Hammer, Lixin Fu, CubiST ++ Evaluating Ad-Hoc CUBE Queries Using Statistics Trees, Distributed and
More informationBuilding Large ROLAP Data Cubes in Parallel
Building Large ROLAP Data Cubes in Parallel Ying Chen Dalhousie University Halifax, Canada ychen@cs.dal.ca Frank Dehne Carleton University Ottawa, Canada www.dehne.net A. Rau-Chaplin Dalhousie University
More informationFast Computation of Sparse Datacubes
Fast Computation of Sparse Datacubes Kenneth A. ROSS* Columbia University karqcs.columbia.edu Divesh Srivastava AT&T Labs-Research divesh@research.att.com 1 Introduction Abstract Datacube queries compute
More informationApriori Algorithm. 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke
Apriori Algorithm For a given set of transactions, the main aim of Association Rule Mining is to find rules that will predict the occurrence of an item based on the occurrences of the other items in the
More informationImproved Data Partitioning For Building Large ROLAP Data Cubes in Parallel
Improved Data Partitioning For Building Large ROLAP Data Cubes in Parallel Ying Chen Dalhousie University Halifax, Canada ychen@cs.dal.ca Frank Dehne Carleton University Ottawa, Canada www.dehne.net frank@dehne.net
More informationCube-Lifecycle Management and Applications
Cube-Lifecycle Management and Applications Konstantinos Morfonios National and Kapodistrian University of Athens, Department of Informatics and Telecommunications, University Campus, 15784 Athens, Greece
More informationInformation Sciences
Information Sciences 181 (2011) 2626 2655 Contents lists available at ScienceDirect Information Sciences journal homepage: www.elsevier.com/locate/ins Multidimensional cyclic graph approach: Representing
More informationA Simple and Efficient Method for Computing Data Cubes
A Simple and Efficient Method for Computing Data Cubes Viet Phan-Luong Université Aix-Marseille LIF - UMR CNRS 6166 Marseille, France Email: viet.phanluong@lif.univ-mrs.fr Abstract Based on a construction
More informationImpact of Data Distribution, Level of Parallelism, and Communication Frequency on Parallel Data Cube Construction
Impact of Data Distribution, Level of Parallelism, and Communication Frequency on Parallel Data Cube Construction Ge Yang Department of Computer and Information Sciences Ohio State University, Columbus
More informationProcessing of Very Large Data
Processing of Very Large Data Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, first
More informationMining Frequent Patterns without Candidate Generation
Mining Frequent Patterns without Candidate Generation Outline of the Presentation Outline Frequent Pattern Mining: Problem statement and an example Review of Apriori like Approaches FP Growth: Overview
More informationCS570 Introduction to Data Mining
CS570 Introduction to Data Mining Frequent Pattern Mining and Association Analysis Cengiz Gunay Partial slide credits: Li Xiong, Jiawei Han and Micheline Kamber George Kollios 1 Mining Frequent Patterns,
More informationPerformance and Scalability: Apriori Implementa6on
Performance and Scalability: Apriori Implementa6on Apriori R. Agrawal and R. Srikant. Fast algorithms for mining associa6on rules. VLDB, 487 499, 1994 Reducing Number of Comparisons Candidate coun6ng:
More informationData Warehousing & Mining. Data integration. OLTP versus OLAP. CPS 116 Introduction to Database Systems
Data Warehousing & Mining CPS 116 Introduction to Database Systems Data integration 2 Data resides in many distributed, heterogeneous OLTP (On-Line Transaction Processing) sources Sales, inventory, customer,
More informationData Mining Part 3. Associations Rules
Data Mining Part 3. Associations Rules 3.2 Efficient Frequent Itemset Mining Methods Fall 2009 Instructor: Dr. Masoud Yaghini Outline Apriori Algorithm Generating Association Rules from Frequent Itemsets
More informationMining Association Rules in Large Databases
Mining Association Rules in Large Databases Association rules Given a set of transactions D, find rules that will predict the occurrence of an item (or a set of items) based on the occurrences of other
More informationDwarf: Shrinking the PetaCube
Dwarf: Shrinking the PetaCube Yannis Sismanis Antonios Deligiannakis Nick Roussopoulos Yannis Kotidis Institute for Advanced Computer Studies AT&T Labs - Research University of Maryland, College Park {isis,adeli,nick}@umiacs.umd.edu
More informationCarnegie Mellon Univ. Dept. of Computer Science /615 DB Applications. Data mining - detailed outline. Problem
Faloutsos & Pavlo 15415/615 Carnegie Mellon Univ. Dept. of Computer Science 15415/615 DB Applications Lecture # 24: Data Warehousing / Data Mining (R&G, ch 25 and 26) Data mining detailed outline Problem
More informationOPTIMISING ASSOCIATION RULE ALGORITHMS USING ITEMSET ORDERING
OPTIMISING ASSOCIATION RULE ALGORITHMS USING ITEMSET ORDERING ES200 Peterhouse College, Cambridge Frans Coenen, Paul Leng and Graham Goulbourne The Department of Computer Science The University of Liverpool
More informationImplementing and Maintaining Microsoft SQL Server 2008 Analysis Services
Course 6234A: Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services Course Details Course Outline Module 1: Introduction to Microsoft SQL Server Analysis Services This module introduces
More informationAn Array-Based Algorithm for Simultaneous Multidimensional Aggregates *
An Array-Based Algorithm for Simultaneous Multidimensional Aggregates * Yihong Zhao Prasad M. Deshpande Computer Sciences Department Computer Sciences Department University of Wisconsin-Madison University
More informationData mining - detailed outline. Carnegie Mellon Univ. Dept. of Computer Science /615 DB Applications. Problem.
Faloutsos & Pavlo 15415/615 Carnegie Mellon Univ. Dept. of Computer Science 15415/615 DB Applications Data Warehousing / Data Mining (R&G, ch 25 and 26) C. Faloutsos and A. Pavlo Data mining detailed outline
More informationRange CUBE: Efficient Cube Computation by Exploiting Data Correlation
Range CUBE: Efficient Cube Computation by Exploiting Data Correlation Ying Feng Divyakant Agrawal Amr El Abbadi Ahmed Metwally Department of Computer Science University of California, Santa Barbara Email:
More informationChapter 6: Basic Concepts: Association Rules. Basic Concepts: Frequent Patterns. (absolute) support, or, support. (relative) support, s, is the
Chapter 6: What Is Frequent ent Pattern Analysis? Frequent pattern: a pattern (a set of items, subsequences, substructures, etc) that occurs frequently in a data set frequent itemsets and association rule
More informationThe cgmcube project: Optimizing parallel data cube generation for ROLAP
Distrib Parallel Databases (2006) 19: 29 62 DOI 10.1007/s10619-006-6575-6 The cgmcube project: Optimizing parallel data cube generation for ROLAP Frank Dehne Todd Eavis Andrew Rau-Chaplin C Science + Business
More informationC-Cubing: Efficient Computation of Closed Cubes by Aggregation-Based Checking
C-Cubing: Efficient Computation of Closed Cubes by Aggregation-Based Checking Dong Xin Zheng Shao Jiawei Han Hongyan Liu University of Illinois at Urbana-Champaign, Urbana, IL 6, USA Tsinghua University,
More informationMining for Data Cube and Computing Interesting Measures
International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Mining for Data Cube and Computing Interesting Measures Miss.Madhuri S. Magar Student, Department of Computer Engg.
More informationData Warehousing and Data Mining. Announcements (December 1) Data integration. CPS 116 Introduction to Database Systems
Data Warehousing and Data Mining CPS 116 Introduction to Database Systems Announcements (December 1) 2 Homework #4 due today Sample solution available Thursday Course project demo period has begun! Check
More informationC-Cubing: Efficient Computation of Closed Cubes by Aggregation-Based Checking
C-Cubing: Efficient Computation of Closed Cubes by Aggregation-Based Checking Dong Xin Zheng Shao Jiawei Han Hongyan Liu University of Illinois at Urbana-Champaign, Urbana, IL 680, USA October 6, 005 Abstract
More informationOn-Line Analytical Processing (OLAP) Traditional OLTP
On-Line Analytical Processing (OLAP) CSE 6331 / CSE 6362 Data Mining Fall 1999 Diane J. Cook Traditional OLTP DBMS used for on-line transaction processing (OLTP) order entry: pull up order xx-yy-zz and
More informationEffectiveness of Freq Pat Mining
Effectiveness of Freq Pat Mining Too many patterns! A pattern a 1 a 2 a n contains 2 n -1 subpatterns Understanding many patterns is difficult or even impossible for human users Non-focused mining A manager
More informationFrequent Pattern Mining
Frequent Pattern Mining...3 Frequent Pattern Mining Frequent Patterns The Apriori Algorithm The FP-growth Algorithm Sequential Pattern Mining Summary 44 / 193 Netflix Prize Frequent Pattern Mining Frequent
More informationA Novel, Low-latency Algorithm for Multiple Group-By Query Optimization
A Novel, Low-latency Algorithm for Multiple Group-By Query Optimization Duy-Hung Phan EURECOM phan@eurecom.fr Pietro Michiardi EURECOM michiard@eurecom.fr Abstract Data summarization is essential for users
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2009 Lecture 6 - Storage and Indexing
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2009 Lecture 6 - Storage and Indexing References Generalized Search Trees for Database Systems. J. M. Hellerstein, J. F. Naughton
More informationEcient Computation of Iceberg Cubes with Complex Measures
Ecient Computation of Iceberg Cubes with Complex Measures Jiawei Han y Jian Pei y Guozhu Dong z Ke Wang y y School of Computing Science, Simon Fraser University, B.C., Canada, fhan, peijian, wangkg@cs.sfu.ca
More informationLoglinear-Based Quasi Cubes
Journal of Intelligent Information Systems, 16, 255 276, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. Loglinear-Based Quasi Cubes DANIEL BARBARÁ XINTAO WU ISE Department, George
More informationData Mining Techniques
Data Mining Techniques CS 6220 - Section 3 - Fall 2016 Lecture 16: Association Rules Jan-Willem van de Meent (credit: Yijun Zhao, Yi Wang, Tan et al., Leskovec et al.) Apriori: Summary All items Count
More informationMarket baskets Frequent itemsets FP growth. Data mining. Frequent itemset Association&decision rule mining. University of Szeged.
Frequent itemset Association&decision rule mining University of Szeged What frequent itemsets could be used for? Features/observations frequently co-occurring in some database can gain us useful insights
More informationThe Polynomial Complexity of Fully Materialized Coalesced Cubes
The Polynomial Complexity of Fully Materialized Coalesced Cubes Yannis Sismanis Dept. of Computer Science University of Maryland isis@cs.umd.edu Nick Roussopoulos Dept. of Computer Science University of
More informationAssociation Rules. A. Bellaachia Page: 1
Association Rules 1. Objectives... 2 2. Definitions... 2 3. Type of Association Rules... 7 4. Frequent Itemset generation... 9 5. Apriori Algorithm: Mining Single-Dimension Boolean AR 13 5.1. Join Step:...
More informationThe Polynomial Complexity of Fully Materialized Coalesced Cubes
The Polynomial Complexity of Fully Materialized Coalesced Cubes Yannis Sismanis Dept. of Computer Science University of Maryland isis@cs.umd.edu Nick Roussopoulos Dept. of Computer Science University of
More informationNovel Materialized View Selection in a Multidimensional Database
Graphic Era University From the SelectedWorks of vijay singh Winter February 10, 2009 Novel Materialized View Selection in a Multidimensional Database vijay singh Available at: https://works.bepress.com/vijaysingh/5/
More informationData Warehousing & Data Mining
Data Warehousing & Data Mining Wolf-Tilo Balke Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Summary Last week: Logical Model: Cubes,
More informationRELATIONAL OPERATORS #1
RELATIONAL OPERATORS #1 CS 564- Spring 2018 ACKs: Jeff Naughton, Jignesh Patel, AnHai Doan WHAT IS THIS LECTURE ABOUT? Algorithms for relational operators: select project 2 ARCHITECTURE OF A DBMS query
More informationCompressing Relations and Indexes Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft 1 Database Talk Speaker : Jonathan Goldstein Microsoft Researc
Compressing Relations and Indexes Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft 1 Database Talk Speaker : Jonathan Goldstein Microsoft Research OVERVIEW ffl Motivation ffl Compressing relations
More information1.4 Euler Diagram Layout Techniques
1.4 Euler Diagram Layout Techniques Euler Diagram Layout Techniques: Overview Dual graph based methods Inductive methods Drawing with circles Including software demos. How is the drawing problem stated?
More informationEvaluation of Top-k OLAP Queries Using Aggregate R trees
Evaluation of Top-k OLAP Queries Using Aggregate R trees Nikos Mamoulis 1, Spiridon Bakiras 2, and Panos Kalnis 3 1 Department of Computer Science, University of Hong Kong, Pokfulam Road, Hong Kong, nikos@cs.hku.hk
More informationChapter 12: Indexing and Hashing. Basic Concepts
Chapter 12: Indexing and Hashing! Basic Concepts! Ordered Indices! B+-Tree Index Files! B-Tree Index Files! Static Hashing! Dynamic Hashing! Comparison of Ordered Indexing and Hashing! Index Definition
More informationTutorial on Assignment 3 in Data Mining 2009 Frequent Itemset and Association Rule Mining. Gyozo Gidofalvi Uppsala Database Laboratory
Tutorial on Assignment 3 in Data Mining 2009 Frequent Itemset and Association Rule Mining Gyozo Gidofalvi Uppsala Database Laboratory Announcements Updated material for assignment 3 on the lab course home
More informationIndexing. Week 14, Spring Edited by M. Naci Akkøk, , Contains slides from 8-9. April 2002 by Hector Garcia-Molina, Vera Goebel
Indexing Week 14, Spring 2005 Edited by M. Naci Akkøk, 5.3.2004, 3.3.2005 Contains slides from 8-9. April 2002 by Hector Garcia-Molina, Vera Goebel Overview Conventional indexes B-trees Hashing schemes
More informationSomething to think about. Problems. Purpose. Vocabulary. Query Evaluation Techniques for large DB. Part 1. Fact:
Query Evaluation Techniques for large DB Part 1 Fact: While data base management systems are standard tools in business data processing they are slowly being introduced to all the other emerging data base
More informationBuilding Roads. Page 2. I = {;, a, b, c, d, e, ab, ac, ad, ae, bc, bd, be, cd, ce, de, abd, abe, acd, ace, bcd, bce, bde}
Page Building Roads Page 2 2 3 4 I = {;, a, b, c, d, e, ab, ac, ad, ae, bc, bd, be, cd, ce, de, abd, abe, acd, ace, bcd, bce, bde} Building Roads Page 3 2 a d 3 c b e I = {;, a, b, c, d, e, ab, ac, ad,
More informationChapter 12: Indexing and Hashing
Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
More information6234A - Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services
6234A - Implementing and Maintaining Microsoft SQL Server 2008 Analysis Services Course Number: 6234A Course Length: 3 Days Course Overview This instructor-led course teaches students how to implement
More informationDeccansoft Software Services Microsoft Silver Learning Partner. SSAS Syllabus
Overview: Analysis Services enables you to analyze large quantities of data. With it, you can design, create, and manage multidimensional structures that contain detail and aggregated data from multiple
More informationA REVIEW DATA CUBE ANALYSIS METHOD IN BIG DATA ENVIRONMENT
A REVIEW DATA CUBE ANALYSIS METHOD IN BIG DATA ENVIRONMENT Dewi Puspa Suhana Ghazali 1, Rohaya Latip 1, 2, Masnida Hussin 1 and Mohd Helmy Abd Wahab 3 1 Department of Communication Technology and Network,
More informationSummary. 4. Indexes. 4.0 Indexes. 4.1 Tree Based Indexes. 4.0 Indexes. 19-Nov-10. Last week: This week:
Summary Data Warehousing & Data Mining Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Last week: Logical Model: Cubes,
More informationFrom SQL-query to result Have a look under the hood
From SQL-query to result Have a look under the hood Classical view on RA: sets Theory of relational databases: table is a set Practice (SQL): a relation is a bag of tuples R π B (R) π B (R) A B 1 1 2
More informationData Warehousing 資料倉儲
Data Warehousing 資料倉儲 Data Cube Computation and Data Generation 992DW05 MI4 Tue. 8,9 (15:10-17:00) L413 Min-Yuh Day 戴敏育 Assistant Professor 專任助理教授 Dept. of Information Management, Tamkang University 淡江大學資訊管理學系
More informationBCNF. Yufei Tao. Department of Computer Science and Engineering Chinese University of Hong Kong BCNF
Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong Recall A primary goal of database design is to decide what tables to create. Usually, there are two principles:
More informationCHAPTER 8. ITEMSET MINING 226
CHAPTER 8. ITEMSET MINING 226 Chapter 8 Itemset Mining In many applications one is interested in how often two or more objectsofinterest co-occur. For example, consider a popular web site, which logs all
More informationETL and OLAP Systems
ETL and OLAP Systems Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, first semester
More informationData Warehousing ETL. Esteban Zimányi Slides by Toon Calders
Data Warehousing ETL Esteban Zimányi ezimanyi@ulb.ac.be Slides by Toon Calders 1 Overview Picture other sources Metadata Monitor & Integrator OLAP Server Analysis Operational DBs Extract Transform Load
More informationMaterialized View Selection for Multidimensional Datasets*
Materialized View Selection for Multidimensional Datasets* Amit Shukla amit@cs.wisc.edu Prasad M. Deshpande pmd@cs.wisc.edu Computer Sciences Department University of Wisconsin - Madison Madison, WI 53706
More informationANU MLSS 2010: Data Mining. Part 2: Association rule mining
ANU MLSS 2010: Data Mining Part 2: Association rule mining Lecture outline What is association mining? Market basket analysis and association rule examples Basic concepts and formalism Basic rule measurements
More informationAnnouncements. References. Selinger Optimizer History. More about the Selinger Algorithm. Selinger Algorithm. CSE 444: Database Internals
Announcements CSE 444: Database Internals Lab 2 deadline EXTENDED by 24 hours Lab 2 quiz moved to Monday Lecture 12 Query Optimization (part 3) 1 2 Selinger Optimizer History References 1960 s: first database
More informationData Mining for Knowledge Management. Association Rules
1 Data Mining for Knowledge Management Association Rules Themis Palpanas University of Trento http://disi.unitn.eu/~themis 1 Thanks for slides to: Jiawei Han George Kollios Zhenyu Lu Osmar R. Zaïane Mohammad
More informationDKT 122/3 DIGITAL SYSTEM 1
Company LOGO DKT 122/3 DIGITAL SYSTEM 1 BOOLEAN ALGEBRA (PART 2) Boolean Algebra Contents Boolean Operations & Expression Laws & Rules of Boolean algebra DeMorgan s Theorems Boolean analysis of logic circuits
More informationAn Infrastructure for Scalable Parallel Multidimensional Analysis
An Infrastructure for Scalale Parallel Multidimensional Analysis Sanjay Goil Alok Choudhary Department of Electrical & Computer Engineering Northwestern University, Technological Institute, 245 Sheridan
More information