MOIRA A Goal-Oriented Incremental Machine Learning Approach to Dynamic Resource Cost Estimation in Distributed Stream Processing Systems
|
|
- Camron Thornton
- 5 years ago
- Views:
Transcription
1 MOIRA A Goal-Oriented Incremental Machine Learning Approach to Dynamic Resource Cost Estimation in Distributed Stream Processing Systems Daniele Foroni, C. Axenie, S. Bortoli, M. Al Hajj Hassan, R. Acker, R. Tudoran, G. Brasche, Y. Velegrakis
2 Context 2
3 Context # Events Time 3
4 Context # Events Time 4
5 Context # Events Only 1 Fixed Parameter Time and NOT query specific! 5
6 Why should we consider the goal? 6
7 MOIRA Query Stream SQL MOIRA Goal Goal-centric approach Goal specified for a single query Rescheduling for improving the goal performance 7
8 MOIRA Architecture Query Stream SQL Cost Optimizer Goal Feature Extractor Monitoring System 8
9 Query Plan Map Reduce Source 1 Join Filter Join Sink Source 2 Filter 9
10 Query Plan Optimization Chain Map Reduce Source 1 Join Filter Join Sink Source 2 Filter 10
11 Query Plan Optimization Chain Map Reduce Source 1 Join Filter Filter Join Sink Source 2 Filter Filter Parallelize 11
12 Cost-Optimizer (1)!"#$ = &"'(, $#(*+&,, (h."/0h1/( c"'( + $#(*+&, + (h."/0h1/( = 100 % 12
13 Cost-Optimizer (2) window 1 Operator 1 rate Operator 1 α rate Operator 2 Operator 2 rate 1 rate 2 window 1 window 2 0 parallelism window 2 -.(/ "! " Output window #$% &' CPU usage of the window ()*+ &' RAM usage of the window, " Output Rate 13
14 Cost-Optimizer (3) Is it possible to extend the chain? Yes! Let s check if chain Source 1 chain extended Operator 1. is chainable 2. supports the parallelism of chain 3. no back-pressure 4. Cost complies the Goal )*+,,./,01)2,,h4*56h75, 14
15 Goal Moira Architecture: Before Query Stream SQL Flink 15
16 Goal Moira Architecture: Static Analysis Query Stream SQL Cost Optimizer Goal Flink Static 16
17 Goal Monitoring System Apache Flink listener through JMX CPU Network Data Rate RAM Back-pressure Window size Monitoring System Flink Static Dynamic 17
18 Goal Feature Extractor Metrics from the monitoring system Builds the operator features (input rate/window size) Invokes the cost estimator Applies the rescheduling policy Feature Extractor Flink Static Dynamic 18
19 Goal MOIRA Architecture: Dynamic Analysis Query Stream SQL Cost Optimizer Goal Feature Extractor Monitoring System Flink Static Dynamic 19
20 Latency evaluation 7 % 20 % 20 New Query Plan & Rescheduling
21 Throughput evaluation 4 % 21 New Query Plan & Rescheduling
22 Open Challenges! Machine Learning approaches to find relationships among variables ) (?!?????? ' "($)? & )! "($) then, entropy analysis to simplify the problem! ( & ' 22
23 Conclusion Goal-centric approach for each singular query MOIRA: framework for dynamic rescheduling Static and dynamic analysis Performance improvements 23
24 Thank you! Questions? 24
25 References Floratou, Avrilia, et al. "Dhalion: self-regulating stream processing in heron." Proceedings of the VLDB Endowment (2017): Han, Zheng, et al. "Elastic allocator: An adaptive task scheduler for streaming query in the cloud." Service Oriented System Engineering (SOSE), 2014 IEEE 8th International Symposium on. IEEE, Khoshkbarforoushha, Alireza, et al. "Flower: a data analytics flow elasticity manager." Proceedings of the VLDB Endowment (2017): Fu, Tom ZJ, et al. "DRS: dynamic resource scheduling for real-time analytics over fast streams." Distributed Computing Systems (ICDCS), 2015 IEEE 35th International Conference on. IEEE, Russo, Gabriele. "Towards Decentralized Auto-Scaling Policies for Data Stream Processing Applications." ZEUS 2018 (2018):
26 Experiments TPC-H queries Adapted for streaming benchmark 4k events/second Running time: 120 minutes 26
27 TPC-H Queries Query Example SELECT o_st, l_orderkey, o_shippriority, SUM(l_extendedprice) AS revenue FROM orders, lineitem WHERE l_orderkey = o_orderkey AND o_orderstatus = F AND YEAR(o_orderdate) > 1993 AND o_orderpriority LIKE '5% AND o_proctime BETWEEN l_proctime - INTERVAL '2' HOUR AND l_proctime + INTERVAL '2' HOUR GROUP BY o_st, l_orderkey, o_shippriority 27
28 TPC-H Queries Query Example SELECT o_st, l_orderkey, o_orderdate, o_shippriority, SUM(l_extendedprice*(1-l_discount)) AS revenue, FROM customer, orders, lineitem WHERE c_mktsegment = 'AUTOMOBILE AND c_custkey = o_custkey AND l_orderkey = o_orderkey AND o_orderdate < date ' AND l_shipdate > date ' AND o_proctime BETWEEN l_proctime - INTERVAL '2' HOUR AND l_proctime + INTERVAL '2' HOUR AND o_proctime BETWEEN c_proctime - INTERVAL '2' HOUR AND c_proctime + INTERVAL '2' HOUR GROUP BY o_st, l_orderkey, o_orderdate, o_shippriority 28
29 TPC-H Queries Query Example SELECT c_custkey, c_name, c_address, n_name, c_acctbal, SUM(l_extendedprice * (1 - l_discount)) AS revenue FROM customer, orders, lineitem, nation WHERE c_custkey = o_custkey AND l_orderkey = o_orderkey AND YEAR(o_orderdate) > 1990 AND l_returnflag = 'R' AND c_nationkey = n_nationkey AND o_proctime BETWEEN l_proctime - INTERVAL '2' HOUR AND l_proctime + INTERVAL '2' HOUR AND o_proctime BETWEEN c_proctime - INTERVAL '2' HOUR AND c_proctime + INTERVAL '2' HOUR AND o_proctime BETWEEN n_proctime - INTERVAL '2' HOUR AND n_proctime + INTERVAL '2' HOUR GROUP BY c_custkey, c_name, c_acctbal, n_name, c_address 29
TPC-H Benchmark Set. TPC-H Benchmark. DDL for TPC-H datasets
TPC-H Benchmark Set TPC-H Benchmark TPC-H is an ad-hoc and decision support benchmark. Some of queries are available in the current Tajo. You can download the TPC-H data generator here. DDL for TPC-H datasets
More informationOn-Disk Bitmap Index Performance in Bizgres 0.9
On-Disk Bitmap Index Performance in Bizgres 0.9 A Greenplum Whitepaper April 2, 2006 Author: Ayush Parashar Performance Engineering Lab Table of Contents 1.0 Summary...1 2.0 Introduction...1 3.0 Performance
More informationHigh Volume In-Memory Data Unification
25 March 2017 High Volume In-Memory Data Unification for UniConnect Platform powered by Intel Xeon Processor E7 Family Contents Executive Summary... 1 Background... 1 Test Environment...2 Dataset Sizes...
More informationEfficient in-memory query execution using JIT compiling. Han-Gyu Park
Efficient in-memory query execution using JIT compiling Han-Gyu Park 2012-11-16 CONTENTS Introduction How DCX works Experiment(purpose(at the beginning of this slide), environment, result, analysis & conclusion)
More informationComparison of Database Cloud Services
Comparison of Database Cloud Services Benchmark Testing Overview ORACLE WHITE PAPER SEPTEMBER 2016 Table of Contents Table of Contents 1 Disclaimer 2 Preface 3 Introduction 4 Cloud OLTP Workload 5 Cloud
More informationTechnical Report - Distributed Database Victor FERNANDES - Université de Strasbourg /2000 TECHNICAL REPORT
TECHNICAL REPORT Distributed Databases And Implementation of the TPC-H Benchmark Victor FERNANDES DESS Informatique Promotion : 1999 / 2000 Page 1 / 29 TABLE OF CONTENTS ABSTRACT... 3 INTRODUCTION... 3
More informationDeveloping a Dynamic Mapping to Manage Metadata Changes in Relational Sources
Developing a Dynamic Mapping to Manage Metadata Changes in Relational Sources 1993-2016 Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic,
More informationAvoiding Sorting and Grouping In Processing Queries
Avoiding Sorting and Grouping In Processing Queries Outline Motivation Simple Example Order Properties Grouping followed by ordering Order Property Optimization Performance Results Conclusion Motivation
More informationComparison of Database Cloud Services
Comparison of Database Cloud Services Testing Overview ORACLE WHITE PAPER SEPTEMBER 2016 Table of Contents Table of Contents 1 Disclaimer 2 Preface 3 Introduction 4 Cloud OLTP Workload 5 Cloud Analytic
More informationMidterm Review. March 27, 2017
Midterm Review March 27, 2017 1 Overview Relational Algebra & Query Evaluation Relational Algebra Rewrites Index Design / Selection Physical Layouts 2 Relational Algebra & Query Evaluation 3 Relational
More information6.830 Problem Set 2 (2017)
6.830 Problem Set 2 1 Assigned: Monday, Sep 25, 2017 6.830 Problem Set 2 (2017) Due: Monday, Oct 16, 2017, 11:59 PM Submit to Gradescope: https://gradescope.com/courses/10498 The purpose of this problem
More informationLazy Maintenance of Materialized Views
Lazy Maintenance of Materialized Views Jingren Zhou, Microsoft Research, USA Paul Larson, Microsoft Research, USA Hicham G. Elmongui, Purdue University, USA Introduction 2 Materialized views Speed up query
More informationHistogram Support in MySQL 8.0
Histogram Support in MySQL 8.0 Øystein Grøvlen Senior Principal Software Engineer MySQL Optimizer Team, Oracle February 2018 Program Agenda 1 2 3 4 5 Motivating example Quick start guide How are histograms
More informationMaterialized Views. March 28, 2018
Materialized Views March 28, 2018 1 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( 2015-03-31 ) 2 CREATE VIEW salessincelastmonth
More informationAdvanced Query Optimization
Advanced Query Optimization Andreas Meister Otto-von-Guericke University Magdeburg Summer Term 2018 Why do we need query optimization? Andreas Meister Advanced Query Optimization Last Change: April 23,
More informationCSC317/MCS9317. Database Performance Tuning. Class test
CSC317/MCS9317 Database Performance Tuning Class test 7 October 2015 Please read all instructions (including these) carefully. The test time is approximately 120 minutes. The test is close book and close
More informationEfficiency Analysis of the access method with the cascading Bloom filter to the data warehouse on the parallel computing platform
Journal of Physics: Conference Series PAPER OPEN ACCESS Efficiency Analysis of the access method with the cascading Bloom filter to the data warehouse on the parallel computing platform To cite this article:
More informationTPC BENCHMARK TM H (Decision Support) Standard Specification Revision
TPC BENCHMARK TM H (Decision Support) Standard Specification Revision 2.17.3 Transaction Processing Performance Council (TPC) Presidio of San Francisco Building 572B Ruger St. (surface) P.O. Box 29920
More informationBeyond EXPLAIN. Query Optimization From Theory To Code. Yuto Hayamizu Ryoji Kawamichi. 2016/5/20 PGCon Ottawa
Beyond EXPLAIN Query Optimization From Theory To Code Yuto Hayamizu Ryoji Kawamichi 2016/5/20 PGCon 2016 @ Ottawa Historically Before Relational Querying was physical Need to understand physical organization
More informationSimba: Towards Building Interactive Big Data Analytics Systems. Feifei Li
Simba: Towards Building Interactive Big Data Analytics Systems Feifei Li Complex Operators over Rich Data Types Integrated into System Kernel For Example: SELECT k-means from Population WHERE k=5 and feature=age
More informationOrri Erling (Program Manager, OpenLink Virtuoso), Ivan Mikhailov (Lead Developer, OpenLink Virtuoso).
Orri Erling (Program Manager, OpenLink Virtuoso), Ivan Mikhailov (Lead Developer, OpenLink Virtuoso). Business Intelligence Extensions for SPARQL Orri Erling and Ivan Mikhailov OpenLink Software, 10 Burlington
More informationTowards Comprehensive Testing Tools
Towards Comprehensive Testing Tools Redefining testing mechanisms! Kuntal Ghosh (Software Engineer) PGCon 2017 26.05.2017 1 Contents Motivation Picasso Visualizer Picasso Art Gallery for PostgreSQL 10
More informationTPC BENCHMARK TM H (Decision Support) Standard Specification Revision 2.8.0
TPC BENCHMARK TM H (Decision Support) Standard Specification Revision 2.8.0 Transaction Processing Performance Council (TPC) Presidio of San Francisco Building 572B Ruger St. (surface) P.O. Box 29920 (mail)
More informationFinding the Pitfalls in Query Performance
Finding the Pitfalls in Query Performance M.L. Kersten P. Koutsourakis Y. Zhang CWI, MonetDB Solutions EU H2020 project ACTiCLOUD The Challenge MonetDB Mar-18 Which system is relatively better? Postgres
More informationGPU ACCELERATION FOR OLAP. Tim Kaldewey, Jiri Kraus, Nikolay Sakharnykh 03/26/2018
GPU ACCELERATION FOR OLAP Tim Kaldewey, Jiri Kraus, Nikolay Sakharnykh 03/26/2018 A TYPICAL ANALYTICS QUERY From a business question to SQL Business question (TPC-H query 4) Determines how well the order
More informationRecommending Materialized Views and Indexes with the IBM DB2 Design Advisor
Recommending Materialized Views and Indexes with the IBM DB2 Design Advisor Daniel C. Zilio et al Proceedings of the International Conference on Automatic Computing (ICAC 04) Rolando Blanco CS848 - Spring
More informationSpark-GPU: An Accelerated In-Memory Data Processing Engine on Clusters
1 Spark-GPU: An Accelerated In-Memory Data Processing Engine on Clusters Yuan Yuan, Meisam Fathi Salmi, Yin Huai, Kaibo Wang, Rubao Lee and Xiaodong Zhang The Ohio State University Paypal Inc. Databricks
More informationGreenplum Database 4.0: Critical Mass Innovation. Architecture White Paper August 2010
Greenplum Database 4.0: Critical Mass Innovation Architecture White Paper August 2010 Greenplum Database 4.0: Critical Mass Innovation Table of Contents Meeting the Challenges of a Data-Driven World 2
More informationABSTRACT. GUPTA, SHALU View Selection for Query-Evaluation Efficiency using Materialized
ABSTRACT GUPTA, SHALU View Selection for Query-Evaluation Efficiency using Materialized Views (Under the direction of Dr. Rada Chirkova) The purpose of this research is to show the use of derived data
More informationSelf Regulating Stream Processing in Heron
Self Regulating Stream Processing in Heron Huijun Wu 2017.12 Huijun Wu Twitter, Inc. Infrastructure, Data Platform, Real-Time Compute Heron Overview Recent Improvements Self Regulating Challenges Dhalion
More informationA Compression Framework for Query Results
A Compression Framework for Query Results Zhiyuan Chen and Praveen Seshadri Cornell University zhychen, praveen@cs.cornell.edu, contact: (607)255-1045, fax:(607)255-4428 Decision-support applications in
More informationOptimizing Queries Using Materialized Views
Optimizing Queries Using Materialized Views Paul Larson & Jonathan Goldstein Microsoft Research 3/22/2001 Paul Larson, View matching 1 Materialized views Precomputed, stored result defined by a view expression
More informationJHive: Tool for Integrating Hive for Performance Analysis
JHive: Tool for Integrating Hive for Performance Analysis Dipti Shikha Singh 1, Garima Singh 2 1 Student, M.Tech, Computer Science Department, Babu Banarasi Das University, Lucknow, U.P, India 2 Assistant
More informationEnhanced XML Support in DB2 for LUW
extra on Demand Enhanced XML Support in DB2 for LUW Speaker Name David Owen (DOCE) Session: H12 Thursday May 26 th 2005 8:30am 1 Agenda XML support in the DB2 Family XML extender for decomposing XML documents
More informationAnorexic Plan Diagrams
Anorexic Plan Diagrams E0 261 Jayant Haritsa Computer Science and Automation Indian Institute of Science JAN 2014 Plan Diagram Reduction 1 Query Plan Selection Core technique Query (Q) Query Optimizer
More informationADMS/VLDB, August 27 th 2018, Rio de Janeiro, Brazil OPTIMIZING GROUP-BY AND AGGREGATION USING GPU-CPU CO-PROCESSING
ADMS/VLDB, August 27 th 2018, Rio de Janeiro, Brazil 1 OPTIMIZING GROUP-BY AND AGGREGATION USING GPU-CPU CO-PROCESSING OPTIMIZING GROUP-BY AND AGGREGATION USING GPU-CPU CO-PROCESSING MOTIVATION OPTIMIZING
More informationTPC Benchmark H Full Disclosure Report
HP NetServer LXr 8500 using Microsoft Windows 2000 and Microsoft SQL Server 2000 TPC Benchmark H Full Disclosure Report Second Edition Submitted for Review August 18, 2000 First Edition - August 18, 2000
More informationWhen and How to Take Advantage of New Optimizer Features in MySQL 5.6. Øystein Grøvlen Senior Principal Software Engineer, MySQL Oracle
When and How to Take Advantage of New Optimizer Features in MySQL 5.6 Øystein Grøvlen Senior Principal Software Engineer, MySQL Oracle Program Agenda Improvements for disk-bound queries Subquery improvements
More informationa linear algebra approach to olap
a linear algebra approach to olap Rogério Pontes December 14, 2015 Universidade do Minho data warehouse ETL OLTP OLAP ETL Warehouse OLTP Data Mining ETL OLTP Data Marts 2 olap Online analytical processing
More informationTwo-Phase Optimization for Selecting Materialized Views in a Data Warehouse
Two-Phase Optimization for Selecting Materialized Views in a Data Warehouse Jiratta Phuboon-ob, and Raweewan Auepanwiriyakul Abstract A data warehouse (DW) is a system which has value and role for decision-making
More informationLab Validation Report
Lab Validation Report ParAccel PADB and NetApp SAN Optimized Solution High Performance Analytics with Advanced Data Management Capabilities By Julie Lockner May 2011 Lab Validation: ParAccel PADB and NetApp
More informationTake Me to SSD: A Hybrid Block-Selection Method on HDFS based on Storage Type
Take Me to SSD: A Hybrid Block-Selection Method on HDFS based on Storage Type Minkyung Kim Yonsei University 50 Yonsei-ro, Seodaemun-gu Seoul, Korea +82 2 2123 7757 goodgail@cs.yonsei.ac.kr Mincheol Shin
More informationCPI Phoenix IQ-201 using EXASolution 2.0
TPC Benchmark TM H Full Disclosure Report CPI Phoenix IQ-201 using EXASolution 2.0 First Edition April 2, 2008 TPC-H FULL DISCLOSURE REPORT 1 First Edition April 2, 2008 CPI Phoenix IQ-201 using EXASolution
More informationCPI Phoenix IQ-201 using EXASolution 2.0
TPC Benchmark TM H Full Disclosure Report CPI Phoenix IQ-201 using EXASolution 2.0 First Edition January 14, 2008 TPC-H FULL DISCLOSURE REPORT 1 First Edition January 14, 2008 CPI Phoenix IQ-201 using
More informationJayant Haritsa. Database Systems Lab Indian Institute of Science Bangalore, India
Jayant Haritsa Database Systems Lab Indian Institute of Science Bangalore, India Query Execution Plans SQL, the standard database query interface, is a declarative language Specifies only what is wanted,
More informationTPC Benchmark H Full Disclosure Report. Sun Microsystems Sun Fire X4100 Server Using Sybase IQ 12.6 Single Application Server
TPC Benchmark H Full Disclosure Report Sun Microsystems Sun Fire X4100 Server Using Sybase IQ 12.6 Single Application Server Submitted for Review Report Date: Jun 23, 2006 TPC Benchmark H Full Disclosure
More informationCLoud computing is a service through which a
1 MAP-JOIN-REDUCE: Towards Scalable and Efficient Data Analysis on Large Clusters Dawei Jiang, Anthony K. H. TUNG, and Gang Chen Abstract Data analysis is an important functionality in cloud computing
More informationBenchmarking In PostgreSQL
Benchmarking In PostgreSQL Lessons learned Kuntal Ghosh (Senior Software Engineer) Rafia Sabih (Software Engineer) 2017 EnterpriseDB Corporation. All rights reserved. 1 Overview Why benchmarking on PostgreSQL
More informationRobust Optimization of Database Queries
Robust Optimization of Database Queries Jayant Haritsa Database Systems Lab Indian Institute of Science July 2011 Robust Query Optimization (IASc Mid-year Meeting) 1 Database Management Systems (DBMS)
More informationSchema Tuning. Tuning Schemas : Overview
Administração e Optimização de Bases de Dados 2012/2013 Schema Tuning Bruno Martins DEI@Técnico e DMIR@INESC-ID Tuning Schemas : Overview Trade-offs among normalization / denormalization Overview When
More informationApache Kudu. A Distributed, Columnar Data Store for Fast Analytics. Mike Percy Software Engineer at Cloudera Apache Kudu PMC member
Apache Kudu A Distributed, Columnar Data Store for Fast Analytics Mike Percy Software Engineer at Cloudera Apache Kudu PMC member 1 Kudu Overview 2 Pace of Data Traditional Hadoop Storage Leaves a Gap
More informationGPU-Accelerated Analytics on your Data Lake.
GPU-Accelerated Analytics on your Data Lake. Data Lake Data Swamp ETL Hell DATA LAKE 0001010100001001011010110 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>> >>>>>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>
More informationBenchmarking Polystores: the CloudMdsQL Experience
Benchmarking Polystores: the CloudMdsQL Experience Boyan Kolev, Raquel Pau, Oleksandra Levchenko, Patrick Valduriez, Ricardo Jiménez-Peris, José Pereira To cite this version: Boyan Kolev, Raquel Pau, Oleksandra
More informationCopyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12
1 MySQL : 5.6 the Next Generation Lynn Ferrante Principal Consultant, Technical Sales Engineering Northern California Oracle Users Group November 2012 2 Safe Harbor Statement The
More informationQuery Optimizer Plan Diagrams: Production, Reduction and Applications
Query Optimizer Plan Diagrams: Production, Reduction and Applications Jayant Haritsa Database Systems Lab Indian Institute of Science Bangalore, INDIA April 2011 Plan Diagrams Tutorial (ICDE 2011) 1 Cost-based
More informationCorrelated Sample Synopsis on Big Data
Correlated Sample Synopsis on Big Data by David S. Wilson A thesis submitted to Youngstown State University in partial fulfillment of the requirements for the degree of Master of Science in the Computer
More informationMaterialized Views. March 26, 2018
Materialized Views March 26, 2018 1 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( 2015-03-31 ) SELECT partkey FROM salessincelastmonth
More informationVectorized Postgres (VOPS extension) Konstantin Knizhnik Postgres Professional
Vectorized Postgres (VOPS extension) Konstantin Knizhnik Postgres Professional Why Postgres is slow on OLAP queries? 1. Unpacking tuple overhead (heap_deform_tuple) 2. Interpretation overhead (invocation
More informationDeep Dive into Concepts and Tools for Analyzing Streaming Data
Deep Dive into Concepts and Tools for Analyzing Streaming Data Dr. Steffen Hausmann Sr. Solutions Architect, Amazon Web Services Data originates in real-time Photo by mountainamoeba https://www.flickr.com/photos/mountainamoeba/2527300028/
More informationJust In Time Compilation in PostgreSQL 11 and onward
Just In Time Compilation in PostgreSQL 11 and onward Andres Freund PostgreSQL Developer & Committer Email: andres@anarazel.de Email: andres.freund@enterprisedb.com Twitter: @AndresFreundTec anarazel.de/talks/2018-09-07-pgopen-jit/jit.pdf
More informationOvertaking CPU DBMSes with a GPU in Whole-Query Analytic Processing
Overtaking CPU DBMSes with a GPU in Whole-Query Analytic Processing Adnan Agbaria David Minor Natan Peterfreund Eyal Rozenberg Ofer Rosenberg - now at Intel - now at GE Research - now a post-doc at CWI
More informationOn-Disk Bitmap Index In Bizgres
On-Disk Bitmap Index In Bizgres Ayush Parashar aparashar@greenplum.com and Jie Zhang jzhang@greenplum.com 1 Agenda Introduction to On-Disk Bitmap Index Bitmap index creation Bitmap index creation performance
More informationCIS 601 Graduate Seminar. Dr. Sunnie S. Chung Dhruv Patel ( ) Kalpesh Sharma ( )
Guide: CIS 601 Graduate Seminar Presented By: Dr. Sunnie S. Chung Dhruv Patel (2652790) Kalpesh Sharma (2660576) Introduction Background Parallel Data Warehouse (PDW) Hive MongoDB Client-side Shared SQL
More informationarxiv: v1 [cs.ai] 14 Nov 2017
DataVizard: Recommending Visual Presentations for Structured Data Rema Ananthanarayanan, Pranay K Lohia, and Srikanta Bedathur IBM Research, India November 15, 2017 arxiv:1711.04971v1 [cs.ai] 14 Nov 2017
More informationQuery processing of pre-partitioned data using Sandwich Operators
Query processing of pre-partitioned data using Sandwich Operators Stephan Baumann 1, Peter Boncz 2, and Kai-Uwe Sattler 1 1 Ilmenau University of Technology, Ilmenau, Germany, first.last@tu-ilmenau.de,
More informationSnailTrail Generalizing Critical Paths for Online Analysis of Distributed Dataflows
SnailTrail Generalizing Critical Paths for Online Analysis of Distributed Dataflows Moritz Hoffmann, Andrea Lattuada, John Liagouris, Vasiliki Kalavri, Desislava Dimitrova, Sebastian Wicki, Zaheer Chothia,
More informationVersatile XQuery Processing in MapReduce
Versatile XQuery Processing in MapReduce Caetano Sauer, Sebastian Bächle, and Theo Härder University of Kaiserslautern P.O. Box 3049, 67653 Kaiserslautern, Germany {csauer,baechle,haerder}@cs.uni-kl.de
More informationUNIVERSITY OF CALIFORNIA, IRVINE. Hyracks Console: Monitoring the Hyracks Partitioned-Parallel Runtime Platform THESIS
UNIVERSITY OF CALIFORNIA, IRVINE Hyracks Console: Monitoring the Hyracks Partitioned-Parallel Runtime Platform THESIS submitted in partial satisfaction of the requirements for the degree of MASTER OF SCIENCE
More informationTuning Relational Systems I
Tuning Relational Systems I Schema design Trade-offs among normalization, denormalization, clustering, aggregate materialization, vertical partitioning, etc Query rewriting Using indexes appropriately,
More informationMultiple query optimization in middleware using query teamwork
SOFTWARE PRACTICE AND EXPERIENCE Softw. Pract. Exper. 2005; 35:361 391 Published online 21 December 2004 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/spe.640 Multiple query optimization
More informationOracle. Professional. WITH Function-Based Indexes (FBIs), I was able to alter an execution. Avoid Costly Joins with FBIs Pedro Bizarro.
Oracle Solutions for High-End Oracle DBAs and Developers Professional Avoid Costly Joins with FBIs Pedro Bizarro In this article, Pedro Bizarro describes how to use Function-Based Indexes to avoid costly
More informationFast and Simple Relational Processing of Uncertain Data
Fast and Simple Relational Processing of Uncertain Data Lyublena Antova, Thomas Jansen, Christoph Koch, and Dan Olteanu Saarland University Database Group Saarbrücken, Germany {lublena, jansen, koch, olteanu}@infosys.uni-sb.de
More information14 Data Warehouses in a nutshell
14 Data Warehouses in a nutshell 14.1 Introduction OLTP vs. OLAP 14.2 DWH methodology 14.3 Stars and Stripes 14.4 OLAP operators: Roll up and Drill down, SQL operators ROLLUP and CUBE 14.5 ROLAP and MOLAP...
More informationCSE 135. Main Problem: Multiple languages and multiple computation servers
CSE 15 Rapid Application Development: Object-Relational Mapping & a lesson on the whys and why nots Main Problem: Multiple languages and multiple computation servers Two different computation servers with
More informationData Warehouse appliances: IBM Pure Data for Analytics
May 2017 Data Warehouse appliances: IBM Pure Data for Analytics Fabio Bresciani, Cloud & Cognitive, IBM Italia 1997 IBM ebusiness 1927 Italy 1956 Data storage industry creation 1969 IBM technology guided
More informationNewSQL Databases MemSQL and VoltDB Experimental Evaluation
NewSQL Databases MemSQL and VoltDB Experimental Evaluation João Oliveira 1 and Jorge Bernardino 1,2 1 ISEC, Polytechnic of Coimbra, Rua Pedro Nunes, Coimbra, Portugal 2 CISUC Centre for Informatics and
More information35 Database benchmarking 25/10/17 12:11 AM. Database benchmarking
Database benchmarking 1 Database benchmark? What is it? A database benchmark is a sample database and a group of database applications able to run on several different database systems in order to measure
More informationOptimizer Standof. MySQL 5.6 vs MariaDB 5.5. Peter Zaitsev, Ovais Tariq Percona Inc April 18, 2012
Optimizer Standof MySQL 5.6 vs MariaDB 5.5 Peter Zaitsev, Ovais Tariq Percona Inc April 18, 2012 Thank you Ovais Tariq Ovais Did a lot of heavy lifing for this presentation He could not come to talk together
More informationStop-and-Resume Style Execution for Long Running Decision Support Queries
Stop-and-Resume Style Execution for Long Running Decision Support Queries Surajit Chaudhuri Raghav Kaushik Abhijit Pol Ravi Ramamurthy Microsoft Research Microsoft Research University of Florida Microsoft
More informationRQL: Retrospective Computations over Snapshot Sets
Industrial and Applications Paper RQL: Retrospective Computations over Snapshot Sets Nikos Tsikoudis Brandeis University tsikudis@cs.brandeis.edu Liuba Shrira Brandeis University liuba@cs.brandeis.edu
More informationPerformance Isolation in Multi- Tenant Relational Database-asa-Service. Sudipto Das (Microsoft Research)
Performance Isolation in Multi- Tenant Relational Database-asa-Service Sudipto Das (Microsoft Research) CREATE DATABASE CREATE TABLE SELECT... INSERT UPDATE SELECT * FROM FOO WHERE App1 App2 App3 App1
More informationRun Your Own Oracle Database Benchmarks with Hammerora
Run Your Own Oracle Database Benchmarks with Hammerora Steve Shaw Database Technology Manager Software and Services Group Date: 19-NOV-09 Time: 3.00 3.45 Location: Seoul Steve Shaw Introduction Database
More informationPerformance Monitoring
Performance Monitoring Performance Monitoring Goals Monitoring should check that the performanceinfluencing database parameters are correctly set and if they are not, it should point to where the problems
More informationC-Store: A Column-oriented DBMS
C-Store: A Column-oriented DBMS Mike Stonebraker, Daniel J. Abadi, Adam Batkin +, Xuedong Chen, Mitch Cherniack +, Miguel Ferreira, Edmond Lau, Amerson Lin, Sam Madden, Elizabeth O Neil, Pat O Neil, Alex
More informationUsing MySQL, Hadoop and Spark for Data Analysis
Using MySQL, Hadoop and Spark for Data Analysis Alexander Rubin Principle Architect, Percona September 21, 2015 About Me Alexander Rubin, Principal Consultant, Percona Working with MySQL for over 10 years
More informationQuery processing for parallel languages. Brandon Myers, Mark Oskin, Bill Howe DB Day 2015
Query processing for parallel languages Brandon Myers, Mark Oskin, Bill Howe bdmyers@cs.washington.edu DB Day 2015 1 slide src: Jeff Gardner 2 How to turn astrophysics simulation output into scientific
More informationParallelism Strategies In The DB2 Optimizer
Session: A05 Parallelism Strategies In The DB2 Optimizer Calisto Zuzarte IBM Toronto Lab May 20, 2008 09:15 a.m. 10:15 a.m. Platform: DB2 on Linux, Unix and Windows The Database Partitioned Feature (DPF)
More informationOptimizing Communication for Multi- Join Query Processing in Cloud Data Warehouses
Optimizing Communication for Multi- Join Query Processing in Cloud Data Warehouses Swathi Kurunji, Tingjian Ge, Xinwen Fu, Benyuan Liu, Cindy X. Chen Computer Science Department, University of Massachusetts
More informationQuery Optimization Time: The New Bottleneck in Realtime
Query Optimization Time: The New Bottleneck in Realtime Analytics Rajkumar Sen Jack Chen Nika Jimsheleishvilli MemSQL Inc. MemSQL Inc. MemSQL Inc. 534 4 th Street, 534 4 th Street, 534 4 th Street, San
More informationHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance Øystein Grøvlen Senior Principal Software Engineer MySQL Optimizer Team, Oracle April 16, 2015 Program Agenda 1 2 3 4 5 6 Introduction to MySQL
More informationColumnstore and B+ tree. Are Hybrid Physical. Designs Important?
Columnstore and B+ tree Are Hybrid Physical Designs Important? 1 B+ tree 2 C O L B+ tree 3 B+ tree & Columnstore on same table = Hybrid design 4? C O L C O L B+ tree B+ tree ? C O L C O L B+ tree B+ tree
More informationSMOPD-C: An Autonomous Vertical Partitioning Technique for Distributed Databases on Cluster Computers
SMOPD-C: An Autonomous Vertical Partitioning Technique for Distributed Databases on Cluster Computers Liangzhe Li School of Computer Science University of Oklahoma Norman, USA lzli@ou.edu Le Gruenwald
More informationCisco Systems, Inc. First Edition April 2, 2019
Cisco Systems, Inc. TPC Benchmark H Full Disclosure Report for Cisco UCS C480 M5 Rack-Mount Server using Microsoft SQL Server 2017 Enterprise Edition And Red Hat Enterprise Linux 7.6 First Edition April
More informationCS425 Project Assignment
CS425 Project Assigmet Tue the TPC-H queries i a database of your choice What is TPC-H? Idustry stadard bechmark for testig database performace for compex SQL Decisio support Aaytics Ad-hoc TPC-H specificatio
More informationNOSQL DATABASE SYSTEMS: DECISION GUIDANCE AND TRENDS. Big Data Technologies: NoSQL DBMS (Decision Guidance) - SoSe
NOSQL DATABASE SYSTEMS: DECISION GUIDANCE AND TRENDS h_da Prof. Dr. Uta Störl Big Data Technologies: NoSQL DBMS (Decision Guidance) - SoSe 2017 163 Performance / Benchmarks Traditional database benchmarks
More informationA Comparison of Three Methods for Join View Maintenance in Parallel RDBMS
A Comparison of Three Methods for Join View Maintenance in Parallel RDBMS Gang Luo Jeffrey F. Naughton Curt J. Ellmann Michael W. Watzke Department of Computer Sciences NCR Advance Development Lab University
More informationWhitepaper. Big Data implementation: Role of Memory and SSD in Microsoft SQL Server Environment
Whitepaper Big Data implementation: Role of Memory and SSD in Microsoft SQL Server Environment Scenario Analysis of Decision Support System with Microsoft Windows Server 2012 OS & SQL Server 2012 and Samsung
More informationParallel Database Processing on a 100 Node PC Cluster: Cases for Decision Support Query Processing and Data Mining
Parallel Database Processing on a 100 Node PC Cluster: Cases for Decision Support Query Processing and Data Mining Takayuki Tamura, Masato Oguchi, Masaru Kitsuregawa Institute of Industrial Science, The
More informationarxiv: v1 [cs.db] 22 Sep 2014
Enabling Incremental Query Re-Optimization Mengmeng Liu University of Pennsylvania 333 Walnut Street Philadelphia, PA 1914 mengmeng@cis.upenn.edu Zachary G. Ives University of Pennsylvania 333 Walnut Street
More informationStop-and-Restart Style Execution for Long Running Decision Support Queries
Stop-and-Restart Style Execution for Long Running Decision Support Queries Surajit Chaudhuri Raghav Kaushik Abhijit Pol Ravi Ramamurthy Microsoft Research Microsoft Research University of Florida Microsoft
More information