IBM Db2 Analytics Accelerator Version 7.1 Delivering new flexible, integrated deployment options Overview Ute Baumbach (bmb@de.ibm.com) 1
IBM Z Analytics Keep your data in place a different approach to enterprise analytics Keep data in place for analytics Keep data in place, encrypted and secure Minimize latency, cost and complexity of data movement Transform data on platform Improve data quality and governance Apply the same resiliency to analytics as your operational applications Combine insight from structured & unstructured data from z and non-z data sources Leverage existing people, processes and infrastructure 2
Hybrid transaction/analytical processing Transaction Processing Analytics Workload The hybrid computing platform on IBM Z Supports transaction processing and analytics workloads concurrently, efficiently and cost-effectively Delivers industry leading performance for mixed workloads The unique heterogeneous scale-out platform in the industry Superior availability, reliability and security Db2 Analytics Accelerator and Db2 for z/os A self-managing, hybrid workload-optimized database management system that runs each query workload in the most efficient way, so that each query is executed in its optimal environment for greatest performance and cost efficiency 3
Announcing Db2 Analytics Accelerator Version 7.1 Delivering new flexible, integrated deployment options High-speed analysis of your enterprise data for real-time insight under the control and security of IBM Z Introduces new flexible, integrated deployment options Accelerator on IBM Z Unified homogeneity of service, support and operations Flexible Capacity Accelerator on IBM Integrated Analytics System Fast, simple deployment on pre-configured hardware and software Flexible and elastic data storage Based on IBM s premier analytical engine, Db2 Warehouse software Transition easily between deployment options One API One database engine 4
Db2 Analytics Accelerator Version 7.1 Deployment Options Current Technology: Appliance In Version 7.1, Db2 acceleration can be implemented within different operating environments: On an on-premises appliance On a software appliance installed on the z14 mainframe 5 Next Generation Technology: Two deployment options Both new options will offer the same functionality the same API the same implementation This provides: Coexistence and combination of deployment options, fully transparent for Db2 applications Flexibility in moving data for query acceleration as workload demands grow or change Consistency and efficiency in managing different Db2 Analytics Accelerator environments
Db2 Warehouse The new acceleration engine for Db2 Analytics Accelerator Version 7.1 Advantages of the new technology for Db2 Analytics Accelerator Improved SQL compatibility Columnar technology Improved throughput and performance POWER based hardware appliance Functionality Next Generation Current Generation Faster, easier integration with other technologies and approaches With General Availability of Version 7.1 on November 24 th, 2017, the use cases query acceleration and indatabase transformation and table types Accelerator-shadow tables and Accelerator-only tables will be supported. IBM intends to add additional use cases in continuous delivery over time. 2017 Time Graph not to scale 6
Version 7.1 - New Architecture Docker container Accelerator server Systems Manager IBM Analytics Engine Workload Monitoring Additional future functionality Authentication Infrastructure Management Docker Supported OS Virtual or physical server CPU Memory Storage (local, SAN, NAS) (Clustered) Filesystem 7
SQL Compatibility Improvements Native support for EBCDIC MBCS, GRAPHIC Converted to UTF-8 in v5 Routing all types of correlated subqueries, including table expressions with sideway references Only small subset was routed in v5 Native timestamp(12) support truncated to precision 6 in v5 Native support for "for bit data" subtype, all table encodings (EBCDIC, UNICODE, ASCII). EBCDIC only in v5 Native support for TIMESTAMP value 24:00:00 mapped to 23:59:59 in v5 8
SQL Compatibility Improvements (contd.) Improved offload for scalar functions MIN/MAX, DAY, LAST_DAY, BIT*, TIMESTAMP_ISO, VARIANCE/STDDEV/ with UNIQUE clause, not offloaded in v5 when using specific datatypes Support for HEX() function Improved support for mixed encodings Can add EBCDIC tables when UNICODE tables already added to Accelerator Improved accuracy for CURRENT_TIME, CURRENT_TIMESTAMP, CURRENT_DATE was depending on time synchronization in v5 9
Db2 Analytics Accelerator Version 7.1, Deployment on IBM Integrated Analytics System 10
Db2 Analytics Accelerator for z/os Version 7.1, deployment on IBM Integrated Analytics System (IIAS) Next generation hardware appliance A full solution that provides all components out of the box including optimized hardware and software All components provided by IBM in a balanced, performance-optimized configuration HW, which includes the rack, the physical servers and the storage SW stack including the Docker host operating system as well as the Docker container and the infrastructure management IBM Power hardware for the appliance, balanced and optimized for price/performance 11
IBM Db2 Analytics Accelerator on IBM Integrated Analytics System Product components IBM Z CLIENT Data Studio with Db2 Analytics Accelerator Studio Plug-in 12 P a t h c h OSAExpress 10 GbE OSAExpress 10 GbE Users/ Applications IBM Integrated Analytics System Dedicated highly available network connection P a n e l Data Warehouse application Db2 for z/os enabled for IBM Db2 Analytics Accelerator
Performance Optimized Hardware Power architecture Higher performance across fewer nodes CPU acceleration with multi-core and Single Instruction Multiple Data (SIMD) parallelism Increased reliability and availability Flash storage Low latency flash modules, for higher transfer speeds, reliability and operational efficiency 13
Hardware is Optimized for Big Data and Analytics Performance Processors flexible, fast execution of analytics algorithms 4X threads per core vs. x86 (up to 1536 threads per system) Memory large, fast workspace to maximize business insight 4X memory bandwidth vs. x86 1 (up to 32TB of memory) Cache ensure continuous data load for fast responses 6X more cache vs. x86 2 (>19MB cache per core) Continuous data load Massive IO bandwidth Parallel processing Flash for extreme performance Large-scale memory processing 14 1. Up to 4X depending on specific x86 and POWER8 servers being compared 2. Up to 6X more cache comparing Intel e7-8890 servers to 12 core POWER8 servers.
Hardware Architecture Overview 7 Compute Nodes in 1 rack containing IBM Power 8 S822L 24 core server 3.02GHz 512 GB of RAM (each node) 2x 600GB SAS HDD Red Hat Linux OS Up to 3 Flash Arrays in 1 rack containing IBM FlashSystem 900 Dual Flash controllers Micro Latency Flash modules 2-Dimensional RAID5 and hot swappable spares for high availability 2x Mellanox 10G Ethernet switches 48x10G ports 12x40/50G ports Dual switches form resilient network User Data Capacity: 192 TB* (Assumes 4x compression) Power Requirements: 9.4 kw Cooling Requirements: 32,000 BTU/hr Scales from: 1/3 rd Rack to 1 Rack 15
IBM Integrated Analytics System Configurations IBM Power 8 S822L 24 core server 3.02GHz, IBM FlashSystem 900 Mellanox 10G Ethernet switches M4001-003 1/3 Rack M4001-006 2/3 Rack M4001-010 Full Rack Servers 3 5 7 Cores 72 120 168 Memory 1.5 TB 2.5 TB 3.5 TB User capacity (Assumes 4x compression) 64 TB 128 TB 192 TB 16
Db2 Analytics Accelerator Version 7.1, deployment on IBM Z 17
Db2 Analytics Accelerator for z/os Version 7.1, deployment on IBM Z 18 A software appliance running on IBM Z Packages the SW stack into an IBM Secure Service Container to deliver a fully self-managed appliance running in a SSC LPAR that can be deployed in minutes Integrates seamlessly into the customer s IBM Z environment and leverages known LPAR-, memory and CPU management procedures, including call home support for enterprise hardware components. Uses customer-provided storage to hold the Accelerator-side data Scales smoothly with the assignment of available processor cores, initially addressing sizes comparable up to 1/2 rack PDA (N3001-005) appliances
IBM Db2 Analytics Accelerator on IBM Z Product components IBM z14 DB2 code including Stored Procedures Appliance UI Db2 Analytics Accelerator Studio Plug-In for Data Studio Accelerator Appliance 19
IBM Db2 Analytics Accelerator on IBM Z Hardware considerations 1 Accelerator = 1 LPAR Each LPAR requires collocated IFLs, RAM, and Storage Up to one drawer 20 IFLs on z14 Dedicated z14 Drawer Storage Use existing IFLs and memory For small production test/dev or getting-started use cases Minimum suggested configuration: 4 IFLs, 256 GB memory for test/dev 8 IFLs, 512 GB memory for production Order one drawer, comprised of 35 IFLs and up to 2.56 TB memory, with your z14 order You can convert your z14 models M01, M02, M03 servers by adding a dedicated drawer containing IFLs and memory at a very attractive price Also available on M04, M05 as a new build inclusive of the drawer, as you cannot add another drawer to M04 or M05 machines Requires up to 20TB additional customerprovided storage (actual size depends on workload) FCP or FICON attached Customer provided storage
IBM Db2 Analytics Accelerator on IBM Z IBM Secure Service Container (SSC) SE / HMC Customer s Storage Management Appliance Workload SysMgmt Operating System PR/SM LPAR CPU Memory Storage (CKD, FB) Docker Filesystem SSC Partition & Deployment Service New SSC partition mode for SW appliances SSC partition is not for general purpose usage, only appliances can be deployed Speeds up deployment with an integrated installer minutes instead of days Ensures integrity of downloaded appliance SSC Appliance Runtime Operating system incl. Docker runtime Network and storage control incl. encryption System monitoring and FFDC SSC Appliance Management Services Admin/User controls View messages, events Network management User management Disk management View Appliance status Create/obtain dumps/logs Apply service, updates All management interactions with an appliance are via Web UI / REST APIs 21 SSC services are customizable and extensible during appliance development using the SSC SDK, e.g., for the integration of WL-specific logs/traces into SSC FFDC
IBM Db2 Analytics Accelerator on IBM Z Appliance architecture Appliance container (IBM Secure Service Container) Docker container Accelerator server dashdb engine Future elements SE / HMC Systems Manager Workload Monitoring Authentication Appliance OS + management PR/SM LPAR CPU Memory Customer s Storage Management Storage (SAN, NAS) Filesystem 22 CPU, Memory, IO according to your requirements and infrastructure availability Customer-provided IBM-provided
Connection Examples and Options Only one defined TCP/IP network interface from Db2 Analytics Accelerator to Db2 subsystem(s) Hipersocket (CECinternal) OSA (internal or external) Hipersocket No wallip, no VIPA for multiple OSA interfaces OSA OSA Hipersocket Logically, M:N connectivity : multiple Db2 subsystem can share an Accelerator Multiple Accelerators can connect to a Db2 subsystem 23
Accelerator on IBM Z Optimized for Multiple Instances Usage Accelerator on IBM Z instance = 1 node = 1 LPAR Single instance typically 8 35 IFLs Single instance typically 0.5 8 TB memory Setup only symbolic Different workloads in different instances Optimize instance for workload Instance size determined by individual workload requirements not sum of all IFLs may be shared dev/test environments Isolation of independent / competing workloads 24
Summary 25
Summary: One API One implementation Two deployment options Hardware Appliance Deployment on IBM Z Uniform experience, simultaneous use, and easy transition between different implementations Common analytics engine across all the platforms: Db2 Warehouse 26
Summary: One API One implementation Two deployment options Hardware Appliance z based Appliance Uniform experience, simultaneous use, and easy transition between different implementations Common accelerator engine across all the platforms: Db2 Warehouse POWER hardware and storage integrated in a self-contained workload-optimized system for analytics Out-of-the-box experience POWER-grade QoS Scalability by scale-out HA support within appliance Analytics for largest data volumes and highest performance Software appliance deployed on customer s z hardware and storage infrastructure Download & go experience Z-grade QoS Scalability by using Z: expansion of IFLs and memory CA/DR support based on GDPS (active/passive and active/active) Flexible, smaller, elastic deployment option 27 Integrated storage and management integration into existing z environment: hardware and storage management, CA/DR infrastructure, support processes, organizational structures, no new infrastructure needed
Key Characteristics of the deployment options Db2 Analytics Accelerator on Pure Data for Analytics N3001 (Netezza Technology) Db2 Analytics Accelerator on Integrated Analytics System M4001 Db2 Analytics Accelerator on IBM Z Key advantages Out-of-the-box experience Workload Optimized System Wide set of Analytics use cases Proven technology with client references cross-industry Out-of-the-box experience Workload Optimized System Optimized for True HTAP Evolving set of Analytics use cases Download & Go experience Homogeneity within IBM Z: common resources, operation Evolving set of Analytics use cases Workload Size Very good scale-out Very good scale-out Optimized for very large query throughput and load performance Good scale-up to full drawer 28
29