Compressed Aggregations for mobile OLAP Dissemination

Transcription

1 INTERNATIONAL WORKSHOP ON MOBILE INFORMATION SYSTEMS (WMIS 2007) Compressed Aggregations for mobile OLAP Dissemination Ilias Michalarias Arkadiy Omelchenko 1/25

2 Overview s in I. Introduction - II.mobile OLAP Research Context III. Physical Structures IV.: a Dwarf for mobile environments V.s in VI. 2/25

3 Data Warehousing s in A data warehouse is a database system using multidimensional data modeling Appropriate for decision support Accelerates queries that demand resource consuming aggregations Every multidimensional model includes a fact table dimensions Sales PK,FK1 PK,FK2 PK,FK3 Product Store Day Amount PK Products Product Category Title Stores Time PK Store PK Day Region Manager Month Year 3/25

4 Data Warehousing s in The data cube operator produces a data cube, which is the union of all possible Group-By operators applied on a fact table A data cube lattice (DCL), is a directed graph without loops, that depicts the relationships between all 2D sub-cubes, given a D-dimensional cube, and the subsequent derivation possibilities Time StoreProduct Data Cube PST PS PT ST P S T - 4/25 Lattice

5 Application Scenario s in Brokers accessing the stock market gallery data mart: At opening and closing times different stocks in different financial dimensions are analyzed by many traders using some portable device Some of these stocks are more popular than others, similarly, some analytical dimensions are more important than others In this situation, a data mart equipped with a broadcast gateway will be responsible for serving the incoming requests 5/25

6 Research Context s in Major Issues: Management of Multidimensional data in wireless networks () Providing equal/comparable functionality with desktop counterparts Cope with limited resources such as bandwidth, energy and small screen size Wireless Bandwidth the usual bottleneck of the system Transmitted items (sub-cubes) are items order of magnitude bigger than usually assumed e.g., web pages 6/25

7 Architecture s in Client-Server Network Architecture Queries are mapped to Aggregation Lattices Clients may have to locally process data Waiting queue ST PT S PT Broadcast Scheduler PST SubCube to be transmitted OLAP Server Broadcasting Gateway PS PT ST Querying P S T -- 7/25 Lattice

8 Optimization Options s in Subsumption In conjuction with wireless broadcast reduces the number of necessary transmissions [Sharaf et al., 2004] [Michalarias et al., 2005] Compression Receiving clients are served faster Indirect reduction of waiting time for pending requests Time StoreProduct Data Cube PST PS PT ST P S T - 8/25 Lattice

9 Physical Structures s in Physical Structures refer to the physical implementation of the data cube operator [Gray et al., 1997] Fundamental evaluation metrics: Storage Space Query Response Time Cube Updates Creation Time Existing proposals (among others) Cube Forests [Johnson et al., 1997] Cubetrees [Roussopoulos et al., 1998] Coalesced Cubes-Dwarfs [Sismanis et al., 2002] 9/25

10 Dwarfs [Sismanis et al., 2002] s in The Dwarf structure identifies prefix and suffix structural redundancies and factors them out by coalescing their store Prefix redundancy is high in cube dense areas Suffix redundancy is particularly high in cube sparse areas Dwarfs successfully confront the curse of dimensionality (storage size exponentially increased when adding dimensions) 10/25

11 Dwarf example Summary Table A B C D $ $ $ 20 Dwarf 1 2 Legend: Pointer Value All Value $ $ s in $ $ $10 $10 2 $40 3 $30 $70 2 $20 $20 1 $10 $10 2 $20 $20 2 $60 3 $30 $90 1 $10 2 $20 3 $10 $40 1 $10 2 $20 3 $10 4 $70 5 $10 $130 11/25

12 s in s in Summary Tables typically uses summary tables (unindexed relations which consist of all the tuples of a corresponding fact table) Dwarfs The respective subdwarf contains all (projected) views that can be subsumpted from this view Summary Tables and Dwarfs are not directly comparable A B C $ $ $ $ $ $ 20 12/25

13 Tradeoff s in The choice of a data cube physical structure for depends on the following tradeoff: The more processed (more aggregated values contained) the dataset is, the more space it occupies, and consequently less client processing for query answering is necessary. Unprocessed datasets occupy less space at the expense of increased client processing. 13/25

14 Hybrid Approaches s in Idea: For each subcube choose the physical structure which occupies less space (DV-ES [Sharaf et al., 2003] and h-fclos [Michalarias et al., 2006]) Criticism Dwarfs were not designed for a environment Reduction from hybrid approaches insufficient and highly dependent on the dataset and the specific scheduler Increased complexity since clients have to handle multiple structures 14/25

15 s in Coarse-grained Dwarf The size of the Dwarf Structure can be modified through the granularity Gmin parameter (Minimum number of addend cells, to produce a special cell ) By defining Gmin>Max(Cardinality) every aggregations are eliminated (results in exploitation of the prefix redundancy and omission of the suffix redundancy) Dwarf 1 2 Coarse-grained Dwarf 1 2 Product Dimension A $110 $110 A $50 $50 A $160 $160 A $110 A $50 15/25

16 s in Instead of pointers, uses pseudoseparators Dimension separator It separates the dimensional levels of the data cube, representing what Dwarf pointers to a lower level represent. An contains D 1 dimension separators, one for each level Node separator It separates cells of the same level. Cell values of the same Dwarf node have an ascending order. Thus, a node separator separates different sub-dwarfs. Node separators can exist in the levels [1, D-1] of the 16/25

17 fcg-dwarf vs. The ascending order of values in Dwarf nodes enables a massive saving of node separators which results in a substantial storage reduction Fully coarse-grained Dwarf Legend: Dimension Separator Node Separator s in $10 2 $20 1 $10 2 $40 3 $30 2 $20 $10 $ $10 2 $40 3 $30 2 $20 17/25 Size: 120B Size: 104B

18 Evaluation: Semi-synthetic dataset m - D w a rf v s S u m m a ry T a b le m - D w a rf v s fc g - D w a rf s in S iz e re d u c tio n [% ] D = 2 D = 3 D = 4 D = 5 D = h D C L id e n tifie r 18/25

19 Evaluation: Real dataset m - D w a rf v s S u m m a ry T a b le m - D w a rf v s fc g - D w a rf s in S iz e re d u c tio n [% ] D = 2 D = 3 D = 4 D = h D C L id e n tifie r 19/25

20 s in s can be easily incorporated in the FCLOS architecture s in Virtual Queue v1 v2 v3 v4 v5 Query Mapper FCLOS SM Metric Derivation Discoverer Cube to be transmitted OLAP Server Waiting Queue q1 q2 q3 q4 q5 BW Metric Statistics Collector Physical Structure Selector Incoming Queries To Wireless Gateway 20/25

21 Simulation Environment s in application experiment Mobile clients issue OLAP queries FCLOS md evaluated against: FCLOS (Summary tables) and h-fclos (hybrid) SBS (Summary tables) and DV-ES (hybrid) Datasets used A real data mart provided by an OLAP company Semi-synthetic 21/25

22 : Generated Traffic s in T o ta l tra ffic [M B ] F C L O S h - F C L O S F C L O S m D S B S D V E S # o f c lie n ts 22/25

23 : Query Access Time s in M e a n q u e r y a c c e s s tim e [m s ] F C L O S h -F C L O S F C L O S m D S B S D V E S # o f c lie n ts 23/25

24 s in The Conclusion achieves a considerable compression for but is by no means a general data cube physical structure; it is definitely ill-suitable for traditional desktop based data warehouses FCLOS md extends FCLOS by incorporating additional the Decoupling of scheduling decisions and physical structures Scheduling decisions consider the dimensionality and not the size of the sub-cube Any enhanced physical structure can be incorporated in the FCLOS architecture without any influence on the scheduling decisions 24/25

25 References s in (1)M. A. Sharaf, Y. Sismanis, A. Labrinidis, P. Chrysanthis, and N. Roussopoulos. Efficient Dissemination of Aggregate Data over the Wireless Web. International Workshop on the Web and Databases (WebDB), pages 93 98, June (2)Y. Sismanis, N. Roussopoulos, A. Deligianannakis, and Y. Kotidis. Dwarf: Shrinking the Petacube. ACM SIGMOD, pages , (3)I. Michalarias and H.-J. Lenz. Dissemination of Multidimensional Data Using Broadcast Clusters. In Distributed Computing and Internet Technology, volume 3816 of LNCS, pages Springer, (4)I. Michalarias, V. Boucharas, and H.-J. Lenz. Hybrid Scheduling for Aggregated Data Delivery in Wireless Networks. In Proceedings of the 1st International Conference on Communications and Networking in China, Beijing, China, IEEE. 25/25