SAP IQ - Business Intelligence and vertical data processing with 8 GB RAM or less
|
|
- Gerald Stewart
- 5 years ago
- Views:
Transcription
1 SAP IQ - Business Intelligence and vertical data processing with 8 GB RAM or less Dipl.- Inform. Volker Stöffler Volker.Stoeffler@DB-TecKnowledgy.info Public
2 Agenda Introduction: What is SAP IQ - in a nutshell Architecture, Idea, Background Exercise: Create a database and database objects What makes SAP IQ eligible for Big Data Scenarios (Un-) Limits, Scalability Aspects Exercise: Populate the database using bulk load Ad-hoc Queries What IQ is good at Exercise: Run predefined or own queries against your database
3 Learning Objective After completing this sesson, you will be able to: Recognize the benefits of data compression mechanisms in Big Data scenarios Describe how ad-hoc queries against raw fact data give you the flexibility to evaluate these data just along the dimensions you want NOW. Match evaluation patterns against the data structures offered by SAP IQ.
4 What is SAP IQ - in a nutshell Architecture, Idea, Background
5 Real- Time Evaluation on Very Large Tables with SAP IQ SAP IQ is a pure-bred Data Warehouse engine designed for Very Large Databases. Like SAP HANA, it utilizes a Columnar Data Store. Unlike SAP HANA, it stores data on Disk and utilizes RAM to cache parts of it. Data Compression multiplies the range of Storage Resources. Dictionary Compression for repeating column values Storage compression for all data structures Storage required for data can be 30% - 80% less than in traditional RDBMS. SAP IQ integrates seamlessly with core components of the Big Data ecosystem. SAP HANA via Smart Data Access / Extended Storage Hadoop via Component Integration Service or Table User Defined Functions
6 SAP IQ Terms Columnar Data Store: In traditional (OLTP style) RDBMS, the various column values of a data row are stored together, making access to a single complete row very efficient. In a columnar data store, the column values of many rows are stored together. A row is distributed over various column vectors Row ID: Since a row does not exist as a memory entity, it exists as a Row ID indicating the position in the various column vectors Cardinality: The number of unique / distinct column values for a column. Optimized Fast Projection: The SAP IQ term for dictionary compression Bitmap Index: Since a row exists as a Row ID only, columns of low cardinality can be reflected as (usually sparsely populated) bitmaps where each bit represents one row. There is one bitmap per unique value. A set bit indicates a row with that value.
7 SAP IQ for Big Data Scenarios What makes SAP IQ eligible for Big Data Scenarios (Un-) Limits, Scalability Aspects
8 Data Acquisition Big Data Data is acquired through bulk mechanism fast SAP IQ holds the Guinness World Record of 34.3 TB / hour (2014) scalable Parallel Processing of Load data streams cost efficient Runs on standard hardware versatile IQ can load from a wide variety of data sources including leading RDBMSs and Hadoop
9 Procedure: SAP IQ Data Acquisition Incoming data (row oriented, tabular result set or data file / data pipe) Green blocks are eligible for massive parallel execution Transformation to vertical Dictionary Compression (where applicable) Storage Compression as data is written to disk Auxiliary Indexes (incremental or non-incremental)
10 Optimized Fast Projection Dictionary Compression Eligible Columns have a metadata Lookup Table Each distinct value is represented once in the lookup table Each column value is stored as the position in the lookup table Lookup Table size depends on column data type and cardinality Number of rows in lookup table = cardinality Lookup table row size is calculated upon the column data type Up to cardinality 2^31 (2,147,483,647) Column Vector size depends on number of rows and column cardinality Each column value is represented by as few bits as required to store cardinality in binary E.g. a column with a cardinality of requires 4 bits / row, a column with a cardinality of requires 10 bits / row
11 Data Storage Big Data SAP IQ can maintain as many containers (files / raw devices) as the OS allows, each up to 4 TB in size Life Cycle SAP IQ can organize the database for different kinds of storage, reflecting data life cycle or temperature. Compression Raw data size typically reduced by 30 70% cost efficient Data compression reduces the disk footprint. integrated SAP IQ can integrate with HANA to hold no longer hot enough for inmemory and Hadoop to age out data even colder
12 SAP IQ Storage (Un-) Limitations SAP IQ maximum database size: number of files times maximum file size the OS allows. Maximum file size supported by IQ is 4 TB Organized as DBSpaces consisting of up to 2000 files Up to 2^48 1 rows per table 15-digit decimal number Table size is only limited by database size Special declarations required to extend a table beyond the size of a DBSpace Up to 2^32 indexes per table Up to columns per table (recommended limit: 10000)
13 Big Data Specific Features Very Large Database Option Semantic Partitioning Read-Only DBSpaces I/O Striping Can control data location by data values When fully populated with archive data, DBSpaces can be declared read-only and excluded from full backups Tables can be distributed over multiple devices by column and / or by partition I/O Striping Auxiliary indexes can be separated from raw data Data Aging Tables or Partitions (through semantic partitioning) with cold data can be assigned to cheaper storage
14 Background What are we doing Storage Containers Catalog Store Temp. Store System Main Store User Data Store User Data Store
15 Background What are we doing System Storage Containers First, we create Catalog Store, System Main Store and Temporary Store (0CreateDB.SQL) Catalog Store Temp. Store System Main Store User Data Store User Data Store
16 Background What are we doing System Storage Containers First, we create Catalog Store, System Main Store and Temporary Store (0CreateDB.SQL) Catalog Store: database '...\FlightStats.db Database Handle one file system file (accompanied by a.log) Holds system tables Grows on demand System Main Store: IQ path '...\FlightStatsMain.IQ One or multiple file system files or raw devices Holds system data Specified current and optionally reserved size for later extension Temp Store: temporary path '...\FlightStatsTemp_00.IQ One or multiple file system files or raw devices Holds temporary data (work tables, temporary tables, processing data) Specified current and optionally reserved size for later extension
17 Background What are we doing User Storage Containers Next, we create a User Data Store (1AdjustExtendDB.SQL) Catalog Store Temp. Store System Main Store User Data Store User Data Store
18 Background What are we doing Create Tables and Indexes Then, we create Tables and Indexes (2TablesIndexes.SQL) Table: create table FlightsOnTime Standard SQL Except iq unique clause (here to bypass dictionary compression) Indexes: Various Index Types Many apply to one column LF Low Fast for low cardinality columns HNG High Non Group for parallel calculation of totals and averages DATE for the low cardinality elements of date values more and details to follow
19 Ad-hoc Queries What IQ is good at
20 Real- Time Evaluation on Very Large Tables with SAP IQ Product Acct. Rep State Year Quarter Revenue IQ Steve TX ASE Bill OK ESP Tom MA HANA Steve AZ HANA Tom NJ IQ Tom PH ESP Greg CA HANA Steve TX IQ Bill CO ESP Steve TX HANA Bill UT HANA Tom NH
21 Real- Time Evaluation on Very Large Tables with SAP IQ Product Acct. Rep State Year Quarter Revenue IQ Steve TX ASE Bill OK ESP Tom MA HANA Steve AZ HANA Tom NJ IQ Tom PH ESP Greg CA HANA Steve TX IQ Bill CO ESP Steve TX HANA Bill UT HANA Tom NH
22 Real- Time Evaluation on Very Large Tables with SAP IQ Product Acct. Rep State Year Quarter Revenue IQ Steve TX ASE Bill OK ESP Tom MA HANA Steve AZ HANA Tom NJ IQ Tom PH ESP Greg CA HANA Steve TX IQ Bill CO ESP Steve TX HANA Bill UT HANA Tom NH
23 Data Processing Big Data Columnar Data Store allows evaluation of very large numbers of rows irrelevant columns have no impact on query performance. scalable Server or Query Workload can be distributed across multiple machines (Multiplex / PlexQ). scalable I/O Striping across all eligible disk containers. efficient Pipeline Processing Bitmap indexes allow complex aggregations through elementary binary operators. Subsequent query operators can start before completion of previous operators
24 Showcase: Grouped Average Calculation in 2 Dimensions We have a numeric fact value (like number or value of items sold) for which we want to calculate total or average values. Assumptions: Every fact row has one out of 23 status values. We re only interested in status current or historic. These two make up ~98% of the stored data. Every fact row is assigned to a geography. The geography dimension has a cardinality of ~100, but we re only interested in 8 of them (e.g. AT, BE, CH, DE, FR, IE, NL, UK). Every fact row is assigned to a product line. There s 43 of them, and we ll evaluate them all.
25 Showcase: Sample Data Excerpt Low Fast (LF) Index Status Geo PL current ES 3 current DK 5 pending UK 9 current UK 16 historic DE 29 current NL 2 historic FR 4 historic GA 5 current DE 16 current AT 31 current IT 24 current historic pending D E DK E S UK PL2 PL3 PL4 PL
26 Procedure: Showcase initial process steps Filter: Create a combined bitmap current OR historic Pipeline Execution Permutation 1: Create a combination of this bitmap (AND) with each of AT, BE, CH, DE, FR, IE, NL, UK Threads: 8 Permutation 2: Create an AND combination of each bitmap with each product line Threads: 8*43 Intermediate Result: 8*43 bitmaps each indicating the row set for a combination of Geo and PL
27 Showcase: Bit Slice Sum Calculation with HNG Index Value As an auxiliary index structure, numeric values can be stored in bit slices This is called High Non Group (HNG) Index Every bit value is represented by an own bitmap E.g. for an unsigned smallint (2 bytes; ) 16 bitmaps are stored Each represents a power of 2 (1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768)
28 Procedure: Showcase final process steps Intermediate Result: 8*43 bitmaps each indicating the row set for a combination of Geo and PL Pipeline Execution Permutation 3: AND combine each bitmap with each HNG bit slice and count resulting set bits Threads: 8*43*16 Accumulation: Multiply the number of set bits with the weight of the bit and add up for each Geo / PL Threads: 8*43 Result: (Up to) 8*43 result rows
29 Showcase: Summary Why this is efficient We re utilizing a very high number of threads These can be executed in parallel if sufficient cores are available but they don t have to They introduce no overhead and are completely independent of each other Even could be executed on different nodes in a PlexQ setup The operations executed are technically trivial and highly efficient on every hardware The intermediate results fit into hardware registers The persistent input bitmaps can be distributed over multiple disks for I/O striping The intermediate bitmaps can be expected to fit in the cache 128 Mbytes for 1G rows per bitmap (uncompressed)
30 Scalability Aspects Load time vs. number of existing rows Incremental indexes (for low cardinality data) are insensitive to the number of existing rows. Non- Incremental (B-Tree) indexes are principally sensitive to the number of existing rows, this impact is minimized using tiered B-Trees Query execution time vs. number of cores Most Analytics style queries can efficiently scale out for a high number of CPU cores. Increasing processing power can be expected to produce an adequate gain in response time. Query execution time vs. number of rows Typically, query execution time rises linear with the number of rows or slower (due to pipeline execution) Multinode Setup (Multiplex / PlexQ) Processing power and RAM is not restricted to the capabilities of a single box
31 Using SAP IQ Standard SQL SAP IQ is addressed using standard SQL easy to use for developers familiar with other RDBMS OLAP The SQL dialect is enhanced by OLAP extensions bringing analytics into the database server Standard APIs Reporting Tools Import Export ODBC, JDBC, OLE-DB, OpenClient Simply use your preferred client (unless it s proprietary) All reporting tools supporting at least one of the standard APIs can retrieve data from SAP IQ ASCII Files are the most versatile data exchange format SAP IQ reads from and writes to these
32 Consistency - Concurrency Snapshot Isolation No Blocks Full Consistency SAP IQ uses Snapshot Isolation Read operations never get into lock conflicts. This minimizes the impact of data provisioning. Data visible to a reader is always consistent nothing like dirty reads, non-repeatable reads or phantom rows Parallel If CPU cores are available, typical Analytics operations can massively utilize them
33 Integration into the SAP Big Data Landscape HANA integration Near Line Storage for SAP BW systems Smart Data Access / HANA Extended Storage Hadoop integration User defined functions in IQ to access Hadoop data and Table Parametrized Functions (TPF) Event Stream Processing SAP ESP comes with a native adapter for SAP IQ Reporting / Predictive Data Analysis Standard APIs (ODBC / JDBC / ) available for SAP and third party products OLAP in the database removes Workload from the Reporting systems
34 Thank you! Contact information: Volker Stöffler DB-TecKnowledgy Independant Consultant Germany Leinfelden-Echterdingen mailto:
Combine Native SQL Flexibility with SAP HANA Platform Performance and Tools
SAP Technical Brief Data Warehousing SAP HANA Data Warehousing Combine Native SQL Flexibility with SAP HANA Platform Performance and Tools A data warehouse for the modern age Data warehouses have been
More informationVOLTDB + HP VERTICA. page
VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics
More informationAutomating Information Lifecycle Management with
Automating Information Lifecycle Management with Oracle Database 2c The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated
More informationSAP IQ Software16, Edge Edition. The Affordable High Performance Analytical Database Engine
SAP IQ Software16, Edge Edition The Affordable High Performance Analytical Database Engine Agenda Agenda Introduction to Dobler Consulting Today s Data Challenges Overview of SAP IQ 16, Edge Edition SAP
More informationAnalyze Big Data Faster and Store It Cheaper
Analyze Big Data Faster and Store It Cheaper Dr. Steve Pratt, CenterPoint Russell Hull, SAP Public About CenterPoint Energy, Inc. Publicly traded on New York Stock Exchange Headquartered in Houston, Texas
More informationHANA Performance. Efficient Speed and Scale-out for Real-time BI
HANA Performance Efficient Speed and Scale-out for Real-time BI 1 HANA Performance: Efficient Speed and Scale-out for Real-time BI Introduction SAP HANA enables organizations to optimize their business
More informationCopyright 2014, Oracle and/or its affiliates. All rights reserved.
1 Oracle Database 12c Preview In-Memory Column Store (V12.1.0.2) Michael Künzner Principal Sales Consultant The following is intended to outline our general product direction. It is intended for information
More informationIn-Memory Data Management Jens Krueger
In-Memory Data Management Jens Krueger Enterprise Platform and Integration Concepts Hasso Plattner Intitute OLTP vs. OLAP 2 Online Transaction Processing (OLTP) Organized in rows Online Analytical Processing
More informationIntroduction to Database Services
Introduction to Database Services Shaun Pearce AWS Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Today s agenda Why managed database services? A non-relational
More informationOracle Database In-Memory
Oracle Database In-Memory Mark Weber Principal Sales Consultant November 12, 2014 Row Format Databases vs. Column Format Databases Row SALES Transactions run faster on row format Example: Insert or query
More informationdata tiering in BW/4HANA and SAP BW on HANA Update 2017
data tiering in BW/4HANA and SAP BW on HANA Update 2017 Roland Kramer, PM EDW, SAP SE June 2017 Disclaimer This presentation outlines our general product direction and should not be relied on in making
More informationSAP NLS Update Roland Kramer, SAP EDW (BW/HANA), SAP SE PBS Customer Information Day, July 1st, 2016
SAP NLS Update 2016 Roland Kramer, SAP EDW (BW/HANA), SAP SE PBS Customer Information Day, July 1st, 2016 Why SAP BW? It is all about three things to know SAPPHIRE 2016 - Quote from Hasso is there anything
More informationApril Copyright 2013 Cloudera Inc. All rights reserved.
Hadoop Beyond Batch: Real-time Workloads, SQL-on- Hadoop, and the Virtual EDW Headline Goes Here Marcel Kornacker marcel@cloudera.com Speaker Name or Subhead Goes Here April 2014 Analytic Workloads on
More informationEvolving To The Big Data Warehouse
Evolving To The Big Data Warehouse Kevin Lancaster 1 Copyright Director, 2012, Oracle and/or its Engineered affiliates. All rights Insert Systems, Information Protection Policy Oracle Classification from
More informationIn-Memory Data Management
In-Memory Data Management Martin Faust Research Assistant Research Group of Prof. Hasso Plattner Hasso Plattner Institute for Software Engineering University of Potsdam Agenda 2 1. Changed Hardware 2.
More informationPUBLIC SAP Vora Sizing Guide
SAP Vora 2.0 Document Version: 1.1 2017-11-14 PUBLIC Content 1 Introduction to SAP Vora....3 1.1 System Architecture....5 2 Factors That Influence Performance....6 3 Sizing Fundamentals and Terminology....7
More informationOracle 1Z0-515 Exam Questions & Answers
Oracle 1Z0-515 Exam Questions & Answers Number: 1Z0-515 Passing Score: 800 Time Limit: 120 min File Version: 38.7 http://www.gratisexam.com/ Oracle 1Z0-515 Exam Questions & Answers Exam Name: Data Warehousing
More informationSAP HANA SAP HANA Introduction Description:
SAP HANA SAP HANA Introduction Description: SAP HANA is a flexible, data-source-agnostic appliance that enables customers to analyze large volumes of SAP ERP data in real-time, avoiding the need to materialize
More informationEvolution of Capabilities Hunter Downey, Solution Advisor
Evolution of Capabilities Hunter Downey, Solution Advisor What is our suite? Crystal Reports Web Intelligence Dashboards Explorer Mobile Lumira Predictive 2011 SAP. All rights reserved. 2 What is our suite?
More informationIn-Memory Computing EXASOL Evaluation
In-Memory Computing EXASOL Evaluation 1. Purpose EXASOL (http://www.exasol.com/en/) provides an in-memory computing solution for data analytics. It combines inmemory, columnar storage and massively parallel
More information10/29/2013. Program Agenda. The Database Trifecta: Simplified Management, Less Capacity, Better Performance
Program Agenda The Database Trifecta: Simplified Management, Less Capacity, Better Performance Data Growth and Complexity Hybrid Columnar Compression Case Study & Real-World Experiences
More informationAgenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache
Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,
More informationMaking the Most of Hadoop with Optimized Data Compression (and Boost Performance) Mark Cusack. Chief Architect RainStor
Making the Most of Hadoop with Optimized Data Compression (and Boost Performance) Mark Cusack Chief Architect RainStor Agenda Importance of Hadoop + data compression Data compression techniques Compression,
More informationInsider s Guide on Using ADO with Database In-Memory & Storage-Based Tiering. Andy Rivenes Gregg Christman Oracle Product Management 16 November 2016
Insider s Guide on Using ADO with Database In-Memory & Storage-Based Tiering Andy Rivenes Gregg Christman Oracle Product Management 16 November 2016 Safe Harbor Statement The following is intended to outline
More informationCopyright 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12
1 Information Retention and Oracle Database Kevin Jernigan Senior Director Oracle Database Performance Product Management The following is intended to outline our general product direction. It is intended
More informationMain-Memory Databases 1 / 25
1 / 25 Motivation Hardware trends Huge main memory capacity with complex access characteristics (Caches, NUMA) Many-core CPUs SIMD support in CPUs New CPU features (HTM) Also: Graphic cards, FPGAs, low
More informationHadoop Beyond Batch: Real-time Workloads, SQL-on- Hadoop, and thevirtual EDW Headline Goes Here
Hadoop Beyond Batch: Real-time Workloads, SQL-on- Hadoop, and thevirtual EDW Headline Goes Here Marcel Kornacker marcel@cloudera.com Speaker Name or Subhead Goes Here 2013-11-12 Copyright 2013 Cloudera
More informationSAP HANA Data Warehousing Foundation Data Distribution Optimizer / Data Life Cycle Manager DWF SP03
SAP HANA Data Warehousing Foundation Data Distribution Optimizer / Data Life Cycle Manager DWF SP03 February, 2016 This is the current state of planning and may be changed by SAP at any time. Disclaimer
More informationCOLUMN-STORES VS. ROW-STORES: HOW DIFFERENT ARE THEY REALLY? DANIEL J. ABADI (YALE) SAMUEL R. MADDEN (MIT) NABIL HACHEM (AVANTGARDE)
COLUMN-STORES VS. ROW-STORES: HOW DIFFERENT ARE THEY REALLY? DANIEL J. ABADI (YALE) SAMUEL R. MADDEN (MIT) NABIL HACHEM (AVANTGARDE) PRESENTATION BY PRANAV GOEL Introduction On analytical workloads, Column
More informationDesigning dashboards for performance. Reference deck
Designing dashboards for performance Reference deck Basic principles 1. Everything in moderation 2. If it isn t fast in database, it won t be fast in Tableau 3. If it isn t fast in desktop, it won t be
More informationCopyright 2013, Oracle and/or its affiliates. All rights reserved.
2 Copyright 23, Oracle and/or its affiliates. All rights reserved. Oracle Database 2c Heat Map, Automatic Data Optimization & In-Database Archiving Platform Technology Solutions Oracle Database Server
More informationPart 1: Indexes for Big Data
JethroData Making Interactive BI for Big Data a Reality Technical White Paper This white paper explains how JethroData can help you achieve a truly interactive interactive response time for BI on big data,
More informationSAP Business Warehouse powered by SAP HANA
SAP Business Warehouse powered by SAP HANA Jürgen Hagedorn, Vice President, Head of PM for SAP HANA Europe & APJ, SAP SAP HANA Council July 30, 2013 Mumbai, India SAP Business Warehouse Widely Adopted
More informationJyotheswar Kuricheti
Jyotheswar Kuricheti 1 Agenda: 1. Performance Tuning Overview 2. Identify Bottlenecks 3. Optimizing at different levels : Target Source Mapping Session System 2 3 Performance Tuning Overview: 4 What is
More informationColumnstore and B+ tree. Are Hybrid Physical. Designs Important?
Columnstore and B+ tree Are Hybrid Physical Designs Important? 1 B+ tree 2 C O L B+ tree 3 B+ tree & Columnstore on same table = Hybrid design 4? C O L C O L B+ tree B+ tree ? C O L C O L B+ tree B+ tree
More informationOracle Database In-Memory What s New and What s Coming
Oracle Database In-Memory What s New and What s Coming Andy Rivenes Product Manager for Database In-Memory Oracle Database Systems DOAG - May 10, 2016 #DBIM12c Safe Harbor Statement The following is intended
More informationDeep Dive Into Storage Optimization When And How To Use Adaptive Compression. Thomas Fanghaenel IBM Bill Minor IBM
Deep Dive Into Storage Optimization When And How To Use Adaptive Compression Thomas Fanghaenel IBM Bill Minor IBM Agenda Recap: Compression in DB2 9 for Linux, Unix and Windows New in DB2 10 for Linux,
More informationIntroduction to SAP HANA and what you can build on it. Jan 2013 Balaji Krishna Product Management, SAP HANA Platform
Introduction to SAP HANA and what you can build on it Jan 2013 Balaji Krishna Product Management, SAP HANA Platform Safe Harbor Statement The information in this presentation is confidential and proprietary
More informationNetezza The Analytics Appliance
Software 2011 Netezza The Analytics Appliance Michael Eden Information Management Brand Executive Central & Eastern Europe Vilnius 18 October 2011 Information Management 2011IBM Corporation Thought for
More informationCloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
More informationSyllabus. Syllabus. Motivation Decision Support. Syllabus
Presentation: Sophia Discussion: Tianyu Metadata Requirements and Conclusion 3 4 Decision Support Decision Making: Everyday, Everywhere Decision Support System: a class of computerized information systems
More informationRoot Cause Analysis for SAP HANA. June, 2015
Root Cause Analysis for SAP HANA June, 2015 Process behind Application Operations Monitor Notify Analyze Optimize Proactive real-time monitoring Reactive handling of critical events Lower mean time to
More informationC-STORE: A COLUMN- ORIENTED DBMS
C-STORE: A COLUMN- ORIENTED DBMS MIT CSAIL, Brandeis University, UMass Boston And Brown University Proceedings Of The 31st VLDB Conference, Trondheim, Norway, 2005 Presented By: Udit Panchal Timeline of
More informationCapture Business Opportunities from Systems of Record and Systems of Innovation
Capture Business Opportunities from Systems of Record and Systems of Innovation Amit Satoor, SAP March Hartz, SAP PUBLIC Big Data transformation powers digital innovation system Relevant nuggets of information
More informationSafe Harbor Statement
Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment
More informationGuide to Licensed Options. SAP Sybase IQ 16.0 SP03
Guide to Licensed Options SAP Sybase IQ 16.0 SP03 DOCUMENT ID: DC01646-01-1603-01 LAST REVISED: November 2013 Copyright 2013 by SAP AG or an SAP affiliate company. All rights reserved. No part of this
More informationData Analytics using MapReduce framework for DB2's Large Scale XML Data Processing
IBM Software Group Data Analytics using MapReduce framework for DB2's Large Scale XML Data Processing George Wang Lead Software Egnineer, DB2 for z/os IBM 2014 IBM Corporation Disclaimer and Trademarks
More informationIsilon: Raising The Bar On Performance & Archive Use Cases. John Har Solutions Product Manager Unstructured Data Storage Team
Isilon: Raising The Bar On Performance & Archive Use Cases John Har Solutions Product Manager Unstructured Data Storage Team What we ll cover in this session Isilon Overview Streaming workflows High ops/s
More informationSAP HANA as an Accelerator for PLM Processes HANA Basics and Scenarios
SAP HANA as an Accelerator for PLM Processes HANA Basics and Scenarios Michael Dietz, Principal Solution Architect HANA Public Agenda SAP HANA Platform Usage Scenarios Potentials in Product Lifecycle Management
More informationSAP HANA ADMINISTRATION
IT HUNTER SOLUTIONS Contact No - +1 9099998808 Email ID ithuntersolutions@gmail.com SAP HANA ADMINISTRATION SAP HANA Technology Overview Introduction to SAP HANA SAP In-Memory Strategy HANA compare to
More informationStrategic Briefing Paper Big Data
Strategic Briefing Paper Big Data The promise of Big Data is improved competitiveness, reduced cost and minimized risk by taking better decisions. This requires affordable solution architectures which
More informationAccelerating BI on Hadoop: Full-Scan, Cubes or Indexes?
White Paper Accelerating BI on Hadoop: Full-Scan, Cubes or Indexes? How to Accelerate BI on Hadoop: Cubes or Indexes? Why not both? 1 +1(844)384-3844 INFO@JETHRO.IO Overview Organizations are storing more
More informationShabnam Watson. SQL Server Analysis Services for DBAs
Shabnam Watson SQL Server Analysis Services for DBAs Shabnam Watson BI Consultant /ShabnamWatson @shbwatson info@abicube.com https://shabnamwatson.wordpress.com Work: BI Consultant Fifteen Years of experience
More informationBeyond Relational Databases: MongoDB, Redis & ClickHouse. Marcos Albe - Principal Support Percona
Beyond Relational Databases: MongoDB, Redis & ClickHouse Marcos Albe - Principal Support Engineer @ Percona Introduction MySQL everyone? Introduction Redis? OLAP -vs- OLTP Image credits: 451 Research (https://451research.com/state-of-the-database-landscape)
More informationLazyBase: Trading freshness and performance in a scalable database
LazyBase: Trading freshness and performance in a scalable database (EuroSys 2012) Jim Cipar, Greg Ganger, *Kimberly Keeton, *Craig A. N. Soules, *Brad Morrey, *Alistair Veitch PARALLEL DATA LABORATORY
More informationEsgynDB Enterprise 2.0 Platform Reference Architecture
EsgynDB Enterprise 2.0 Platform Reference Architecture This document outlines a Platform Reference Architecture for EsgynDB Enterprise, built on Apache Trafodion (Incubating) implementation with licensed
More informationBig and Fast. Anti-Caching in OLTP Systems. Justin DeBrabant
Big and Fast Anti-Caching in OLTP Systems Justin DeBrabant Online Transaction Processing transaction-oriented small footprint write-intensive 2 A bit of history 3 OLTP Through the Years relational model
More informationData Warehouse Appliance: Main Memory Data Warehouse
Data Warehouse Appliance: Main Memory Data Warehouse Robert Wrembel Poznan University of Technology Institute of Computing Science Robert.Wrembel@cs.put.poznan.pl www.cs.put.poznan.pl/rwrembel SAP Hana
More informationExadata X3 in action: Measuring Smart Scan efficiency with AWR. Franck Pachot Senior Consultant
Exadata X3 in action: Measuring Smart Scan efficiency with AWR Franck Pachot Senior Consultant 16 March 2013 1 Exadata X3 in action: Measuring Smart Scan efficiency with AWR Exadata comes with new statistics
More informationOracle Database In-Memory
Oracle Database In-Memory A Focus On The Technology Andy Rivenes Database In-Memory Product Management Oracle Corporation Email: andy.rivenes@oracle.com Twitter: @TheInMemoryGuy Blog: blogs.oracle.com/in-memory
More informationData Warehousing in the Age of In-Memory Computing and Real-Time Analytics. Erich Schneider, Daniel Rutschmann June 2014
Data Warehousing in the Age of In-Memory Computing and Real-Time Analytics Erich Schneider, Daniel Rutschmann June 2014 Disclaimer This presentation outlines our general product direction and should not
More informationOracle CoreTech Update OASC Opening 17. November 2014
Oracle CoreTech Update OASC Opening 17. November 2014 Roger Wullschleger Senior Manager Sales Consulting CoreTech Oracle Software (Schweiz) GmbH Copyright 2014, Oracle and/or its affiliates. All rights
More informationOracle Database In-Memory
Oracle Database In-Memory Under The Hood Andy Cleverly andy.cleverly@oracle.com Director Database Technology Oracle EMEA Technology Safe Harbor Statement The following is intended to outline our general
More informationHAWQ: A Massively Parallel Processing SQL Engine in Hadoop
HAWQ: A Massively Parallel Processing SQL Engine in Hadoop Lei Chang, Zhanwei Wang, Tao Ma, Lirong Jian, Lili Ma, Alon Goldshuv Luke Lonergan, Jeffrey Cohen, Caleb Welton, Gavin Sherry, Milind Bhandarkar
More information1. Attempt any two of the following: 10 a. State and justify the characteristics of a Data Warehouse with suitable examples.
Instructions to the Examiners: 1. May the Examiners not look for exact words from the text book in the Answers. 2. May any valid example be accepted - example may or may not be from the text book 1. Attempt
More informationExadata Implementation Strategy
Exadata Implementation Strategy BY UMAIR MANSOOB 1 Who Am I Work as Senior Principle Engineer for an Oracle Partner Oracle Certified Administrator from Oracle 7 12c Exadata Certified Implementation Specialist
More informationHyPer-sonic Combined Transaction AND Query Processing
HyPer-sonic Combined Transaction AND Query Processing Thomas Neumann Technische Universität München October 26, 2011 Motivation - OLTP vs. OLAP OLTP and OLAP have very different requirements OLTP high
More informationWas ist dran an einer spezialisierten Data Warehousing platform?
Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction
More informationFlash Storage Complementing a Data Lake for Real-Time Insight
Flash Storage Complementing a Data Lake for Real-Time Insight Dr. Sanhita Sarkar Global Director, Analytics Software Development August 7, 2018 Agenda 1 2 3 4 5 Delivering insight along the entire spectrum
More informationHyPer-sonic Combined Transaction AND Query Processing
HyPer-sonic Combined Transaction AND Query Processing Thomas Neumann Technische Universität München December 2, 2011 Motivation There are different scenarios for database usage: OLTP: Online Transaction
More information5/24/ MVP SQL Server: Architecture since 2010 MCT since 2001 Consultant and trainer since 1992
2014-05-20 MVP SQL Server: Architecture since 2010 MCT since 2001 Consultant and trainer since 1992 @SoQooL http://blog.mssqlserver.se Mattias.Lind@Sogeti.se 1 The evolution of the Microsoft data platform
More informationSAP- HANA ADMIN. SAP HANA Landscape SAP HANA components, editions scenarios and guides
SAP- HANA ADMIN Prerequisites Someone who is working as a SAP BW consultant and wants to learn SAP HANA skills. Familiarity with security and administration concepts. network SAP HANA Landscape SAP HANA
More informationSAP HANA. Jake Klein/ SVP SAP HANA June, 2013
SAP HANA Jake Klein/ SVP SAP HANA June, 2013 SAP 3 YEARS AGO Middleware BI / Analytics Core ERP + Suite 2013 WHERE ARE WE NOW? Cloud Mobile Applications SAP HANA Analytics D&T Changed Reality Disruptive
More informationModern Data Warehouse The New Approach to Azure BI
Modern Data Warehouse The New Approach to Azure BI History On-Premise SQL Server Big Data Solutions Technical Barriers Modern Analytics Platform On-Premise SQL Server Big Data Solutions Modern Analytics
More informationDatabase Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
More informationPowerCenter 7 Architecture and Performance Tuning
PowerCenter 7 Architecture and Performance Tuning Erwin Dral Sales Consultant 1 Agenda PowerCenter Architecture Performance tuning step-by-step Eliminating Common bottlenecks 2 PowerCenter Architecture:
More information<Insert Picture Here> Controlling resources in an Exadata environment
Controlling resources in an Exadata environment Agenda Smart IO IO Resource Manager Compression Hands-on time Exadata Security Flash Cache Storage Indexes Parallel Execution Agenda
More informationSAP HANA ONLINE TRAINING. Modelling. Abstract This Course deals with SAP HANA Introduction, Advanced Modelling, and Data provision with SAP HANA
SAP HANA ONLINE TRAINING Modelling Abstract This Course deals with SAP HANA Introduction, Advanced Modelling, and Data provision with SAP HANA Arani Consulting Arani Consulting Email: Info@araniconsulting.com
More informationCustomer SAP BW/4HANA. Salvador Gimeno 7 December SAP SE or an SAP affiliate company. All rights reserved. Customer
SAP BW/4HANA Customer Salvador Gimeno 7 December 2016 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 1 DISCLAIMER This presentation is not subject to your license agreement or any
More informationA Comparison of Memory Usage and CPU Utilization in Column-Based Database Architecture vs. Row-Based Database Architecture
A Comparison of Memory Usage and CPU Utilization in Column-Based Database Architecture vs. Row-Based Database Architecture By Gaurav Sheoran 9-Dec-08 Abstract Most of the current enterprise data-warehouses
More information1 Dulcian, Inc., 2001 All rights reserved. Oracle9i Data Warehouse Review. Agenda
Agenda Oracle9i Warehouse Review Dulcian, Inc. Oracle9i Server OLAP Server Analytical SQL Mining ETL Infrastructure 9i Warehouse Builder Oracle 9i Server Overview E-Business Intelligence Platform 9i Server:
More informationOptimizing and Modeling SAP Business Analytics for SAP HANA. Iver van de Zand, Business Analytics
Optimizing and Modeling SAP Business Analytics for SAP HANA Iver van de Zand, Business Analytics Early data warehouse projects LIMITATIONS ISSUES RAISED Data driven by acquisition, not architecture Too
More informationData Warehousing and Decision Support (mostly using Relational Databases) CS634 Class 20
Data Warehousing and Decision Support (mostly using Relational Databases) CS634 Class 20 Slides based on Database Management Systems 3 rd ed, Ramakrishnan and Gehrke, Chapter 25 Introduction Increasingly,
More informationBig Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara
Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case
More informationConcurrency Control Goals
Lock Tuning Concurrency Control Goals Concurrency Control Goals Correctness goals Serializability: each transaction appears to execute in isolation The programmer ensures that serial execution is correct.
More information<Insert Picture Here> MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure
MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure Mario Beck (mario.beck@oracle.com) Principal Sales Consultant MySQL Session Agenda Requirements for
More informationWhat s New in SAP Sybase IQ 16 Tap Into Big Data at the Speed of Business
SAP White Paper SAP Database and Technology Solutions What s New in SAP Sybase IQ 16 Tap Into Big Data at the Speed of Business 2013 SAP AG or an SAP affiliate company. All rights reserved. The ability
More informationPostgres Plus and JBoss
Postgres Plus and JBoss A New Division of Labor for New Enterprise Applications An EnterpriseDB White Paper for DBAs, Application Developers, and Enterprise Architects October 2008 Postgres Plus and JBoss:
More informationIBM DB2 BLU Acceleration vs. SAP HANA vs. Oracle Exadata
Research Report IBM DB2 BLU Acceleration vs. SAP HANA vs. Oracle Exadata Executive Summary The problem: how to analyze vast amounts of data (Big Data) most efficiently. The solution: the solution is threefold:
More informationOLAP Introduction and Overview
1 CHAPTER 1 OLAP Introduction and Overview What Is OLAP? 1 Data Storage and Access 1 Benefits of OLAP 2 What Is a Cube? 2 Understanding the Cube Structure 3 What Is SAS OLAP Server? 3 About Cube Metadata
More informationAn Introduction to Big Data Formats
Introduction to Big Data Formats 1 An Introduction to Big Data Formats Understanding Avro, Parquet, and ORC WHITE PAPER Introduction to Big Data Formats 2 TABLE OF TABLE OF CONTENTS CONTENTS INTRODUCTION
More informationCustomer Coffee Corner for SAP IQ Using sp_iqrebuildindex()
Customer Coffee Corner for SAP IQ Using sp_iqrebuildindex() Customer SAP Product Support February, 2017 Agenda Objectives sp_iqrebuildindex() usage FAQs Useful scripts Closing remarks Open discussion 2016
More informationApache Kylin. OLAP on Hadoop
Apache Kylin OLAP on Hadoop Agenda What s Apache Kylin? Tech Highlights Performance Roadmap Q & A http://kylin.io What s Kylin kylin / ˈkiːˈlɪn / 麒麟 --n. (in Chinese art) a mythical animal of composite
More informationCISC 7610 Lecture 4 Approaches to multimedia databases. Topics: Document databases Graph databases Metadata Column databases
CISC 7610 Lecture 4 Approaches to multimedia databases Topics: Document databases Graph databases Metadata Column databases NoSQL architectures: different tradeoffs for different workloads Already seen:
More informationAppliances and DW Architecture. John O Brien President and Executive Architect Zukeran Technologies 1
Appliances and DW Architecture John O Brien President and Executive Architect Zukeran Technologies 1 OBJECTIVES To define an appliance Understand critical components of a DW appliance Learn how DW appliances
More informationScott Meder Senior Regional Sales Manager
www.raima.com Scott Meder Senior Regional Sales Manager scott.meder@raima.com Short Introduction to Raima What is Data Management What are your requirements? How do I make the right decision? - Architecture
More informationOracle Exadata: Strategy and Roadmap
Oracle Exadata: Strategy and Roadmap - New Technologies, Cloud, and On-Premises Juan Loaiza Senior Vice President, Database Systems Technologies, Oracle Safe Harbor Statement The following is intended
More informationPerformance Tuning. Chapter 25
Chapter 25 Performance Tuning This chapter covers the following topics: Overview, 618 Identifying the Performance Bottleneck, 619 Optimizing the Target Database, 624 Optimizing the Source Database, 627
More informationTHE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES
1 THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon Vincent.Garonne@cern.ch ph-adp-ddm-lab@cern.ch XLDB
More informationData-Intensive Distributed Computing
Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 5: Analyzing Relational Data (1/3) February 8, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
More information