DB2 Version 11.1 Overview

Transcription

1 Dean Compher DB2 Version 11.1 Overview DB2 Version 11.1 Overview This presentation covers the various features and functions of the DB2 V11 product. The presentation has been divided into several sections based on major features such as BLU Columnar technology, SQL enhancements, purescale features, and other updates. This document doesn t cover all the features that are in the DB2 V11 product and there is always a possibility that certain items may not make the release, or be deferred until a later date. The fact that a feature is found within this presentation does not guarantee it will be in the actual release. Please check the online documentation for DB2 V11 if you need to make sure that a particular feature is currently available in the release. Thanks to the various contributors to this presentation, including development, marketing, and the information development (manual) personnel. Special thanks to development and product offering: - Matt Huras - Mike Winer - Phil Downey - Michael Springgay - Jens Seifer - John Hornibrook - Kathy McKnight - Kevin Rose Apologies to anyone we missed! DB2 V11 Technical Enablement Team George Baklarz, Enzo Cialini, Kelly Schlamb, Liwen Yeow, Bob Harbus, Danny Arnold

2 Safe Harbor Statement Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON CURRENT THINKING REGARDING TRENDS AND DIRECTIONS, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. FUNCTION DESCRIBED HEREIN MY NEVER BE DELIVERED BY I BM. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at copytrade.shtml 2 2 IBM Corporation All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM's current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM's sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. IBM, and the IBM logo, are trademarks of International Business Machines Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

3 New DB2 Capabilities Petabyte Scale In-Memory Warehousing Build or expand your operational warehouse with petabyte scale BLU Technology Broader Analytical Horizons Leverage enhanced analytical functions within BLU to broaden the workloads inside your DB2 warehouse Protect your Data Enterprise class data encryption to ensure the security for all your databases Transact with Confidence Manage your business critical systems with the confidence that DB2 will scale to large data and user volumes Always there Business Additional DB2 purescale capabilities ensures your Business Systems are always up and available 3 IBM DB2 for Linux, UNIX and Windows is a next generation data platform for transactional and analytical operations. It provides continuous availability of data to keep transactional workloads and analytics operating at maximum efficiency. And, it delivers breakthrough in-memory performance enabling speed of thought analytics without the constraints of other in-memory solutions with the simplicity of load and go setup. DB2 is optimized to deliver industry-leading performance while lowering costs and improving IT productivity. DB2 is the most versatile hybrid Database Software for ultra fast, highly scalable and always available mission critical transactional, analytical and mixed workload applications, with end to end security that seamlessly integrates with other data services DB2 provides the simplest to deploy and highly scalable platform with continuous availability for transactional workloads of any size BLU In-Memory technology provides the ultimate scalable and extreme performance platform for analytics and warehousing workloads

4 Announcement Summary DB2 Version 11.1 Announced on April 12, 2016 ega occurred on June 15 th DB Announced on Dec 13, 2016 ega of DB on December 15 th End of Marketing for DB2 Version 10.5 Effective End of Marketing date of September 30 th, 2016) All DB2 Version 10.5 Editions All DB2 Business Value Offerings (BVOs) Tools and DB2 Connect Version 10.5 End of Service for both DB2 Version 9.7 and 10.1 Effective End of Service date of September 30 th, 2017 Extended support can be purchased for up to 3 additional years Direct upgrade from DB2 Version 9.7, 10.1, and 10.5 to DB End of Service date is not applicable to SAP customers with an ASL as they have a different end of service period provided by SAP 4 End of marketing means that customers will not be able to purchase DB2 Version 10.5 after the end of September, At that time they will only be able to purchase DB2 Version For those customers who purchased any of the Business Value Offerings, they will be migrated to a different license of DB2 depending on which license they had when they purchased the BVO. The Encryption Offering will bundled with every version of DB2, so customers that purchased that feature will no longer have to pay service and support on it when they migrate to DB2 Version Note: License migration only occurs when customers moves to DB2 Version Prior to that, all of the terms and conditions for DB2 Version 10.5 remain in place. Note: Customers continue to get service and support for DB End of service is not being announced at this time. One thing to highlight with this change is that all DB2 Editions (Workgroup, Enterprise, Advanced Workgroup and Advanced Enterprise) will include Encryption, Federation, and purescale. While there are limitations on the use of Federation and purescale in Workgroup and Enterprise Editions, the Encryption feature is fully supported across all editions.

5 DB2 Version 11.1 Packaging Change Summary Express Edition removed Entitled customers will be migrated to Workgroup Server Edition 3 of 4 Business Value Offerings removed Native Encryption is included in all DB2 Version 11.1 Editions Business Application Continuity is available in all DB2 Version 11.1 Editions BLU is ONLY available in Advanced/Developer Editions Workgroup Server Edition has the following changes Additional capabilities included Limited Use Virtual Socket licensing discontinued Still available for existing customers to renew S&S Introduces the new Virtual Server license metric Prices modified to adjust for the removal of Express Edition New DB2 Direct Standard and DB2 Direct Advanced Editions Attractive subscription pricing using the new Virtual Processor Core metric Same database server capabilities as WSE and AESE respectively 5 The DB2 express version will no longer be offered in DB2 V11. Existing customers will be upgraded to the DB2 Workgroup Edition. One thing to highlight with this change is that all DB2 Editions (Workgroup, Enterprise, Advanced Workgroup and Advanced Enterprise) will include Encryption, Federation, and purescale. While there are limitations on the use of Federation and purescale in Workgroup, Enterprise, and Direct Standard editions, the Encryption feature is fully supported across all editions. For those customers who purchased any of the Business Value Offerings: Native Encryption is now bundled with every version of DB2, so customers that purchased the DB2 Encryption Offering will no longer have to pay service and support on it when they migrate to DB2 Version Note: License migration only occurs when customers moves to DB2 Version Prior to that, all of the terms and conditions for DB2 Version 10.5 remain in place. It is also planned to include Encryption in DB fp8, currently due out in Q The Business Continuity Offering, providing a 2 member (1 active and 1 admin) purescale cluster for WSE and ESE, will not be offered in DB2 Version 11.1 either. However, just as the Encryption Offering is inherently bundled into the non-advanced editions, so is this distinct 2 member (active/admin) purescale cluster capability and licensing option. The BLU Acceleration Offering is no longer offered in DB2 Version Customers with this option will need to upgrade to an Advanced edition to retain entitlement to use BLU Acceleration. The DB2 Performance Management Offering continues to be available in DB2 Version There have been additional capabilities included with Workgroup Server Editions that ae talked about in the next slide. In addition a new Virtual Server licence metric has been made available for Workgroup Server Edition and a price modification due to the removal of Express Edition. We ve also included two new Direct Editions, Standard and Advanced, which introduces a new monthly subscription pricing to help with hybrid cloud deployments, more on this later.

6 Packaging: Workgroup Server Edition (WSE) Enterprise Server Edition (ESE) Workgroup and Enterprise Editions have additional capability purescale Active/Admin Member Native Encryption, Limited Federation (DB2 & Informix) New to Workgroup MDC Tables, Multi-Temp Storage Query parallelism, Scan sharing Exclusions Database Partitioning, Compression BLU Acceleration, MQ or CDC Replication, SQL Warehousing Tool (SQW) WSE: Materialized Query Tables (included with ESE) Data Server Manager Enterprise, other programs which are in Advanced Workgroup Simple Limits Authorized User (AUSI), PVU, new Virtual Server license metric 16 Cores, 128 GB memory (no database size limit) 6 Note: Pricing may change. Please refer to local country pricing since these price reflect US prices only and will not be applicable to other countries. In order to make the Workgroup Edition more appealing to customers, the price has been reduced significantly, and number of additional features have been included into the package including: purescale Active/Admin Member Option Native Encryption Federation limited to DB2 & Informix sources Workgroup has further capabilities with Multi-Dimensional Clustering, Multi-temp Storage, Query parallism and scan sharing There are some exclusions to what is included in the Warehouse product. The list includes (but it not limited to): Database Partitioning BLU Acceleration, Compression SQL Warehouse (SQW) Data Server Manager Enterprise Materialize Query Tables for Workgroup, included with Enterprise The one big change in the licensing of DB2 Warehouse Edition is that there is no longer a cap on the size of database. The core limit remains the same, but the amount of main memory has increased to 128GB. The Socket pricing metric for Warehouse Edition is no longer available in this release. However, existing customers will continue to pay S&S on this metric, but new customers will not be able to purchase using this licensing metric.

7 Packaging: Advanced Workgroup Server Edition (AWSE) Advanced Enterprise Server Edition (AESE) Complete database and packaging capabilities including purescale, Compression, Replication, Federation, BLU Acceleration, Database Partitioning Feature (DPF/MPP), Tools Restrictions DB2 Connect for using SQW to access DB2 for z and DB2 for i InfoSphere Data Architect limited to 10 authorized users Cognos Analytics limited to 5 authorized users Licensing Metrics Authorized User (AUSI), PVU, Terabyte Limits for Advanced Workgroup Server Edition 16 cores, 128 GB memory (no database size limit) With Terabyte licensing - 4 socket limit, no core limit Advanced Recovery Feature Optional 7 As with the prior release of DB2 Version 10.5, the Advanced versions of Workgroup and Enterprise Edition include all features for one single price. What is new in this release is the new Federated feature which includes connectivity to DB2, Informix, and a large number of other relational and unstructured data sources. There are three products that are included as part of the Advanced Editions that have limits on the amount of users that they can have. These features are: DB2 Connect included for using SQW tooling to access DB2 for z and DB2 for i InfoSphere Data Architect limited to 10 authorized users Cognos Analytics limited to 5 authorized users The only optional product that customers may want to consider is the Advanced Recovery Feature. DB2 Advanced Recovery Feature for Linux, UNIX, and Windows 11.1 is available in conjunction with the DB database release. This feature consists of advanced database backup, recovery, and data extraction tools that can help you improve data availability, mitigate risk, and accelerate crucial administrative tasks when time is of the essence. New features of the tools include support for DB2 11.1, including using the tools with DB2 native encryption. The Advanced Recovery Feature bundle contains the following new releases of the included tools: DB2 Merge Backup for Linux, UNIX, and Windows V3.1 DB2 Recovery Expert for Linux, UNIX, and Windows V5.1 Optim High Performance Unload for DB2 for Linux, UNIX, and Windows V6.1

8 Packaging: Direct Standard Edition Direct Advanced Edition New Delivery Mechanism for DB2 licenses New license metrics to facilitate hybrid cloud deployments Acquire the product directly online (Passport Advantage) Option to deploy either on-premises or on cloud Two Versions depending on Requirements DB2 Direct Standard Edition 11.1 Has database features of DB2 Workgroup Server Edition DB2 Direct Advanced Edition 11.1 Has database features of DB2 Advanced Enterprise Server Edition Simplified license metric, the Virtual Processor Core sold as a monthly license charge 1 VPC for every VPC available to a virtual servers Operating System, or 1 VPC for each physical core of a non-partitioned physical server If the number of VPCs is greater than the physical cores, then you only need to license the number of physical cores on the machine Minimum of 2 VPCs per deployment (1 VPC for idle/warm standby) 8 DB2 Version V11.1 includes a new delivery mechanism for licensing. The DB2 Direct Editions are intended to make it easier for a customer to acquire DB2 licenses for cloud deployments: New license metrics to facilitate hybrid cloud deployments Acquire the product directly online (Passport Advantage) Option to deploy either on-premises or on cloud It doesn't matter where the customer places their DB2 server, on premise, on cloud, etc They are licensed by the number of Virtual Processor core vcpus (explained on the next page). The license is totally flexible and can be moved between platforms without any change in license fees. Note that this license is an OPEX (Operating Expense) license. The customer is renting or leasing the DB2 license. The price includes the use of the product as well as service and support. If the license is not renewed, the customer has no valid license to run the database. There are two DB2 Direct editions, depending on what the database requirements are: DB2 Direct Standard Edition Has all of the database features of DB2 Workgroup Server Edition DB2 Direct Advanced Edition Has all of the database features of DB2 Advanced Enterprise Server Edition Both of these licenses are sold as a monthly license charge using the Virtual Processor Core (VPC) sold as a monthly license charge. Customers can choose to maintain these databases with either a fix pack stream (about every 4 months) which may contain new features and enhancements, or a stable stream (about once a year) that contains only fixes.

9 Federation Included in Advanced/Developer Packaging Extending federation capabilities to more data sources More drivers directly integrated into the DB2 installer such as Hive and Netezza Expansion to both SQL and NoSQL data stores Expansion to both on-premises and cloud data sources Set-up Simplification CREATE WRAPPER stmt and WRAPPER clause in CREATE SERVER now optional A default wrapper will be created if the clause is omitted Can create server with host, port, and service name without cataloging remote node and db Applications Federation included in each 9 This image shows only a subset of all supported data sources Where do we ship the federation server engine? See the red box. We have a common SQL engine inside of DB2, the dashdb family, and BigInsights (BigSQL) today. In this common SQL engine we have included the federation server being described here. We have also used the term Fluid Query for the broader data virtualization story which includes, federation, PDA s fluid query, data movement options and DSM for monitoring and management across the common SQL engine. Simplification of set-up process: CREATE SERVER orardb TYPE oracle version 11.0 WRAPPER net8 AUTHORIZATION "J15USER2" PASSWORD "J15USER2" OPTIONS (...); And the proposed change is that user are allowed to simply ignore the "wrapper net8" clause (i.e. the "wrapper (wrapper name)" clause should be changed from mandatory to optional). If the wrapper clause is ignored, a wrapper with default option will be created automatically. New grammar: >>-CREATE SERVER--server-name--+-- wrapper-clause > '-- type-clause ' '-- type-clause ---- wrapper-clause --' > > '-AUTHORIZATION--remote-authorization-name--PASSWORD--password-' > ><.-, V '-OPTIONS--(----server-option-name--string-constant-+--)-' wrapper-clause ---WRAPPER--wrapper-name----- type-clause TYPE--server-type '--VERSION-- server-version ' '--TYPE--server-type--VERSION-- server-version -' server-version --+-version '-.--release ' '-.--mod-' '-version-string-constant '

10 Encryption Included in Packaging GA Support for KMIP 1.1 Centralized Key Managers IBM Security Key Lifecycle Manager (ISKLM), SafeNet KeySecure, etc. DB support for Hardware Security Modules Gemalto Safenet HSM (formerly Luna) version 6.1 and above Thales nshield HSM, Security World Software version Simple Key Mgt : a local flat file used for a specific DB2 instance Enterprise Key Mgt : a centralized key manager or HSM that can be used across many databases, file systems and other uses across an enterprise 10 Native Encryption is now included and entitled in all DB2 editions (as of DB fp8). DB2 V11.1 added support for KMIP 1.1 compliant centralized key managers like IBM Security Key Lifecycle Manager (ISKLM) and SafeNet KeySecure. Use of a centralized key manager helps meet regulations and standards such as: Payment Card Industry Data Security Standard (PCI DSS) Sarbanes-Oxley Health Insurance Portability and Accountability Act (HIPAA) DB now adds direct support for Hardware Security Modules (HSMs). The HSMs currently support are: Gemalto Safenet HSM (formerly Luna) version 6.1 (firmware version ) and above Thales nshield HSM, Security World Software version 11.50

11 DB2 purescale Included in Packaging Low cost active/admin licensing where one DB2 member has minimal licensing and the other DB2 member(s) fully licensed All application workloads are directed to the primary active member(s) Sometimes referred to as the primary" member Utilities and admin tasks allowed on the secondary admin member Admin member licensed as warm standby (e.g. 100 PVUs or 1 VPC) Great for off-loading backups from primary members $ DB2 purescale active/active available in DB2 Advanced Editions, including new Direct Advanced Edition 11

12 Licensing - DB2 purescale Active/Admin Model CF CF *can have up to 127 Primary Members 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

13 Licensing - DB2 purescale Active/Admin Model Primary Member* CF CF *can have up to 127 Primary Members 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

14 Licensing - DB2 purescale Active/Admin Model Primary Member* Secondary Admin Member CF CF *can have up to 127 Primary Members 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

15 Licensing - DB2 purescale Active/Admin Model Application workloads (transactional, batch, etc.) run on the primary member Workload Workload Workload Primary Member* Secondary Admin Member Workload CF CF *can have up to 127 Primary Members 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

16 Licensing - DB2 purescale Active/Admin Model Application workloads (transactional, batch, etc.) run on the primary member Workload Backup Restore Workload Runstats Workload Primary Member* Secondary Admin Member Reorg Security Workload Monitoring Replicatio n Configuration DDL CF CF Administrative tasks/utilities allowed to run on secondary member *can have up to 127 Primary Members 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

17 Licensing - DB2 purescale Active/Admin Model Application workloads (transactional, batch, etc.) run on the primary member Workload Backup Restore Workload Runstats Workload Primary Member* Secondary Admin Member Reorg Security Workload Monitoring Replicatio n Configuration Backup DDL Backup Administrative tasks/utilities allowed, but best practice is to run them on secondary member CF CF *can have up to 127 Primary Members Administrative tasks/utilities allowed to run on secondary member 12 This slide is intended to show via a diagram how work is distributed across a three member purescale cluster. Application workloads (transactional, batch, etc.) are only allowed to run on the primary two members. Administrative tasks and utilities, however, are allowed to run on the secondary member. Nothing is preventing the same administrative work from running on the primary members as well, but the best practice is to not do this. This is an active/active cluster as work is being done on ALL members of the cluster.

18 Operating Systems - Supported New Operating System Support Power Linux LE (Little Endian) Red Hat Enterprise Linux (RHEL) 7.1+ SUSE Linux Enterprise Server (SLES) 12 Ubuntu LTS Supported Operating Systems Intel 64-bit Windows 7, 8.1, 10, Windows Server 2012 R2, Windows Server 2016 Red Hat Enterprise Linux (RHEL) 6.7+, 7.1+ SUSE Linux Enterprise Server (SLES) 11SP4+, 12 Ubuntu LTS AIX Version 7.1 TL 3 SP5+ zlinux (including Text Search support) Red Hat Enterprise Linux (RHEL) 7.1+ SUSE Linux Enterprise Server (SLES) 12 Ubuntu purescale support on AIX, Red Hat 7.2 (Power Linux LE & Intel 64), RHEL 6.8 on x86, and SLES 12 SP1 on x86 13 A customer need to be aware that there are new operating systems requirements for DB2 Version While their existing DB2 installation may be supported at an older O/S release level, they will not be able to migrate their database unless the O/S has been updated as well. DB2 Version 11.1 introduces support for Power Linux LE (Little Endian) which includes support for purescale. DB2 supports Power Linux LE with the following three operating systems: Red Hat Enterprise Linux (RHEL) 7.1+ SUSE Linux Enterprise Server (SLES) 12 Ubuntu LTS DB2 Version 11.1 has also updated the operating systems that it supports. Note the new requirements in the following list. Support for 32-bit server operating systems has been eliminated for new instances in this release, so Windows 32-bit and Linux 32-bit are no longer supported for the server editions but continue to be available as part of the developer edition. 32-bit clients continue to be supported however. Intel 64-bit Windows 7, 8.1, 10, Windows Server 2012 R2, Windows Server 2016 ( only) Red Hat Enterprise Linux (RHEL) 6.7+, 7.1+ SUSE Linux Enterprise Server (SLES) 11SP4+, 12 Ubuntu LTS AIX Version 7.1 TL 3 SP5+ zlinux Red Hat Enterprise Linux (RHEL) 7.1+ SUSE Linux Enterprise Server (SLES) 12 Ubuntu ( only)

19 Operating Systems - Discontinued In DB2 V11, the following operating systems (on any platform) are no longer supported for Client or Server: HP-UX Solaris Power Linux BE Inspur K-UX Migration Customers on these platforms will continue to be supported until the end-ofservice date for DB2 V10.5 (last release that supports these platforms) Customer can migration to AIX from HP-UX and Solaris via a DB2 backup/ restore. 14 Starting in DB2 Version 11.1, some operating systems, Linux distributions and architectures are no longer supported. Customers need to look at alternative platforms in order to upgrade to DB2 V11. The 4 operating systems mentioned in the slide are no longer being maintained. HP-UX Solaris Power Linux BE Inspur K-UX However, customers who are currently on DB2 Version 10.5 will continue to receive service for these platforms until the End of Service date for the release.

20 Operating Systems - Virtualization IBM System z IBM Processor Resource/System Manager z/vm and z/kvm on IBM System z IBM Power IBM PowerVM and PowerKVM and IBM Workload Partitions on IBM Power Systems Linux X86-64 Platforms Red Hat KVM SUSE KVM VMWare ESXi Docker container support Linux only Microsoft Hyper-V Microsoft Windows Azure on x86-64 Windows Platforms only purescale support on Power VM/KVM, VMWare, and KVM 15 DB2 Version 11.1 supports a number of virtualization environments. Virtualization can improve the utilization of processor/machine resources and also give the customer an environment to test systems without having to invest in entire systems. The following is a list of virtualized environments that DB2 supports: IBM System z IBM Power Linux X86-64 Platforms IBM Processor Resource/System Manager z/vm and z/kvm on IBM System z IBM PowerVM and PowerKVM and IBM Workload Partitions on IBM Power Systems Red Hat KVM SUSE KVM VMWare ESXi Docker container support Linux only Microsoft Hyper-V Microsoft Windows Azure on x86-64 Windows Platforms only purescale support on Power VM/KVM, VMWare, and KVM You will note that purescale is now supported in a virtualized environment and that ROCE Adapters can now be virtualized in a VMWare environment (initially) so that multiple nodes can share the same hardware adapter.

21 Streamlined Upgrade Process Upgrade directly from Version 9.7, 10.1 and 10.5 (3 releases back) Ability to roll-forward through database version upgrades Upgrading from DB2 Version 10.5 Fix Pack 7, or later Users are no longer required to perform an offline backup of existing databases before or after they upgrade A recovery procedure involving roll-forward through database upgrade now exists Applies to all editions and configurations except Database Partitioning Feature (DPF) HADR environments can now be upgraded without the need to reinitialize the standby database after performing an upgrade on the primary database Applies to all HADR deployments except DB2 purescale From DB Fix Pack 7, or later For purescale, DB fp9 will be a prereq 16 Upgrade Support Upgrading to DB2 Version 11.1 is supported from DB2 Version 10.5, DB2 Version 10.1, or DB2 Version 9.7. If you have an earlier version of DB2, you must upgrade to DB2 Version 10.5 or DB2 Version 10.1 DB2 Version 9.7 before upgrading to DB2 Version Upgrading to a DB2 Version 11.1 non-root installation is supported from a DB2 Version 10.5, DB2 Version 10.1, or DB2 Version 9.7 non-root installation. Upgrading to a DB2 Version 11.1 non-root installation from a pre-db2 Version 11.1 root installation is not supported. Instance bit size is determined by the operating system where DB2 Version 11.1 is installed, and support for 32-bit kernels and 64-bit kernels has changed. Upgrading from a system with multiple copies of DB2 Version 10.5, DB2 Version 10.1, or DB2 Version 9.7 of all levels is supported. On Windows operating systems, you must be aware of the restrictions on coexistence of previous versions of the DB2 database products. Upgrading from a partitioned database environment with multiple database partitions is supported. Restoring full database offline backups from pre-db2 Version 11.1 copies is supported. However, rolling forward of logs from a previous level is not possible. Review Backup and restore operations between different operating systems and hardware platforms for complete details about upgrade support using the RESTORE DATABASE command.

22 INPLACE Table Reorganization The manageability of large range partitioned tables is improved A single partition of a partitioned table can now be reorganized with the INPLACE option if: the table has no non-partitioned indexes ON DATA PARTITION is specified Only one data partition can be reorganized at a time INPLACE table reorganization can be run only on tables that are at least three pages in size 17

23 New options for the ADMIN_MOVE_TABLE REPORT The REPORT option can be used to monitor the progress of table moves Calculates a set of values to monitor the progress of a single or multiple table moves Focus is the COPY and REPLAY phase of a running table move To get values for all table moves, tabschema and tabname must be NULL or the empty string TERM The TERM option can be used to terminate a table move in progress Terminates a running or table move TERM will force off the application running the table move, roll back all open transactions and set the table move to a well defined operational status From here, the table move can be canceled or continued SWAP FORCE_ALL ADMIN_MOVE_TABLE(<schema>,<table>,'','FORCE_ALL','SWAP ) Helps avoid deadlocks and timeouts due to other transactions with locks Requires SYSADM, SYSCTRL, or SYSMAINT privileges 18 The ADMIN_MOVE_TABLE procedure provides, for a variety of use cases (e.g. schema change, reorg, table type/cfg change), and online method to create a new version of a table, moving the data from source-t1 to target-t2 and then switching over to target-t2 (removing source-t1 ). When the SWAP occurs, a super exclusive lock is required and can lead to extended lock waits, timeouts, and/or deadlocks. A new FORCE_ALL option is provided to allow administrators to help avoid lock issues, to finalize and complete the table move operation. When utilized, existing transactions which are concurrently accessing the table (i.e. have a table lock on source-t1 ) will be forced, similar to a FORCE APPLICATION, allowing the ADMIN_MOVE_TABLE SWAP operation to complete.

24 For Public, or Private, Cloud Environments SSL encryption between HADR primary and standby servers Integrated protection of sensitive data in the log stream Use the HADR_SSL_LABEL DB configuration parameter Initial support is for Linux on x86 platform, non-purescale Must be at Remote Storage option for Utilities Remote storage is now accessible from: INGEST, LOAD, BACKUP, and RESTORE Accessed through the use of storage access aliases Supported Storage IBM SoftLayer Object Storage Amazon Simple Storage Service (S3) 19 Although the database configuration parameters were added in DB GA, support for configuring and leveraging SSL (Secure Sockets Layer) encryption between the HADR primary and HADR standby servers is now supported in DB , to add enhanced security and protection of your enterprise data, integrated within DB2. Initial support is only for Linux on x86 (non-purescale) deployments. See the section Configuring SSL for the communication between primary and standby HADR servers in the DB2 Knowledge Center for more information on using this new security capability.

25 NO TABLESPACE Option for Backup Database DB2 History File Contains information about log file archive location, log file chain etc. If you use snapshot backups you want to have current information about log file location to perform point in time recovery (RECOVER command) For RECOVER you need a current version of the history file NO TABLESPACE backup allows you to create a current and consistent backup of the history file in a convenient way If you use COMPRESS option of the BACKUP command, the created backup image is small BACKUP with NO TABLESPACE option A NO TABLESPACE backup does not contain tablespaces A no tablespace backup is used to restore the history file by using the HISTORY FILE option with the RESTORE DATABASE command HISTORY FILE keyword is specified to restore only the history file from the backup image 20

26 Improved Serviceability and Problem Determination Client Information in db2diag.log for some events When a transaction exceeds max_log (DB Config Parm) or triggers a lock escalation, additional client diagnostic information will be included in the log CLIENT ACCTNG, CLIENT APPLNAME, CLIENT USERID, CLIENT WRKSTNNAME In MPP (DPF), information is only included if event occurs on co-ord member Identify size of backup images Currently only available in db2diag.log New column BYTES_TOTAL in the UTILSTOP logical data group 21 A couple of specific serviceability items that are included in DB In an application server environment where client applications connect to the database with the same userid/password authentication, it can be difficult from the database perspective to identify which connections the who is potentially causing some errors. DB2 will now log to the db2diag.log information from the CLIENT connection (more of a who/where beyond just authid) when a transaction triggers the max_log limit or a lock escalation. This information may be of additional help in problem determination efforts. Additionally, while the db2diag.log contains information about the size of a backup image, this is not something that is programmatically easy to obtain. DB2 now adds the BYTES_TOTAL column to be selected from the utilstop logical data group so that the backup image size can be selected and returned VIA SQL. 1. In some application environments (e.g. common when there is an application server), all connections made to the database can use a common authorization where the source application client may not be easily identifiable. However, there are client attributes that can be set by the client that make their way to the database server. These client information settings will now be reported in the db2diag.log entries for some log/lock events so assist in problem determination. In the case of MPP, only the coordinator database partition will have this client information available, so max-log and lock escalation events on other database partitions will not be able to report this added diagnostic client information. 2. The size of a backup image had not previously been made available in a programmatic way. With DB , the new column bytes_total in the UTILSTOP logical data group can included in a query to determine the size of a backup image. In the case of MPP, multiple rows will be returned for a backup event (when the table space spans multiple database partitions) so that each individual backup image on each database partition can have its size queried.

27 DB2 Support for the NX842 Accelerator DB2 backup and log archive compression now support the NX842 hardware accelerator on POWER 7+ and POWER 8 processors DB2 BACKUPs require the use of a specific NX842 library backup database <dbname> compress comprlib libdb2nx842.a Backups can be compressed by default with NX842 Registry variable DB2_BCKP_COMPRESSION has to be set to NX842 Use the following backup command format: backup database <dbname> compress Log archive compression is also supported Update the database configuration parameter LOGARCHCOMPR1 or LOGARCHCOMPR2 to NX842 update database configuration for <dbname> using LOGARCHCOMPR1 NX842 Note: These two parameters can still take different values 22 By using the nest accelerator NX842 of POWER 7+ and POWER 8 processors, you can achieve hardware compression for backup images and log archive files on AIX. Prerequisites This solution is only supported on AIX. Minimum AIX levels are AIX V7 TL3 SP3 and AIX V6 TL9 SP3. Active Memory Expansion (AME) has to be licensed but must not be enabled. This is a temporary restriction and not a technical limitation. In addition, Active Memory Sharing (AMS) has to be deactivated on the logical partition (LPAR). The CPU has to be a POWER 7+ or later. Remember: Provided that the kernel requirements are met, it is possible to recover using the backup images and log files that were compressed with NX842 on previous POWER versions. Advantages of using this solution A very fast compression can be achieved through the special hardware compression unit NX842 on POWER CPUs. The general CPU resources are not used for this compression. The NX842 compression units are typically not used for AME on database servers since deep row compression, adaptive compression and index compression can make memory compression inefficient. The compression algorithm in hardware provides faster compression than the common DB2 compression. How to use this as backup To start a backup using the hardware compression, it is necessary to specify the library: backup database databasename compress comprlib libdb2nx842.a The backups can be compressed by default with NX842. To achieve this the registry variable DB2_BCKP_COMPRESSION has to be set to NX842. Afterwards, issue the command: backup database databasename compress. The image will then be compressed using the NX842 hardware compression. Using the solution for log archive compression The NX842 hardware compression can also be used for log archive compression. To activate this, change the database configuration parameter LOGARCHCOMPR1 or LOGARCHCOMPR2 to NX842 using this command: update database configuration for databasename using LOGARCHCOMPR1 NX842 Note: These two parameters can still take different values. For example, the common DB2 compression can be used for LOGARCHCOMPR1 and NX842 compression for LOGARCHCOMPR2:update database configuration for databasename using LOGARCHCOMPR1 ON update database configuration for databasename using LOGARCHCOMPR2 NX842

28 DB2 Backup Compression Performance Results Preliminary results from early system testing About 50% DB2 backup size reduction compared to uncompressed Factor 2x less CPU consumption compared to DB2 compression Very significant reduction in CPU consumption Very significant reduction in elapsed time Maintains almost all of the compression storage benefits 100% 06:00 90% 80% 70% 60% 50% 40% 05:00 04:00 03:00 Minutes Size related to Uncompressed Backup Runtime to /dev/null in minutes Restore in minutes Not Comp- ressed Backup to /dev/null Software Compressed H/W Compressed 30% 02:00 20% 01:00 10% 0% Uncompressed Compressed NX842 Compressed 00:00 Internal Tests at IBM Germany Research & Development Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 23 When using the NX842 accelerator, DB2 backups are compressed to about 50% of what a normal backup would be. In addition, the CPU of the system is not used to do the compression, so the total CPU overhead is reduced by a factor of 2.1. Note: These benchmarks were performed on early beta versions of DB2 V11 by IBM Germany Research and Development in a laboratory environment. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

29 Improved Performance for Highly Concurrent Workloads Streamlined bufferpool latching protocol implemented in DB2 V11 Reduces contention which can develop on large systems with many threads Particularly helpful with transactional workloads 8000 Workload #1 - DB2 ESE 1.58x Workload #2 DB2 purescale x DB2 V10.5 fp5 DB2 V DB2 V10.5 fp5 DB2 V11.1 Workload 1 based on an industry benchmark standard POWER7 32c, 512 GB Workload 2 implements a warehouse-based transactional order system 4 members, 2 CFs with 16c, 256 GB Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 24 DB2 Version 11.1 adds enhancements aimed at environments with very large database (VLDB) sizes and extreme user populations. Some of the VLDB enhancements available include concurrency and scalability for recent and commonly referenced pages, features for higher transaction throughput, and the ability to perform online table reorgs at the data partition level. Highly concurrent workloads that have frequently-accessed common pages, such as small table lookups or frequent index access (especially via nested-loop joins) may see improved performance (lower transaction latency, lower latch and lock waits) and scalability (higher degree of concurrency) as result of these changes.

30 DB2 V11 purescale Feature Enhancements Installation, Upgrades and Deployments Power Linux Little-Endian (LE) support Linux Virtualization Enhancements HADR Enhancements GDPC Enhancements Performance Enhancements Unified Workload Balancing Manageability Improvements Registry, Catalog Changes & Deprecated functionality Text Search support 25

31 DB2 purescale: Simplified Install and Deployment Fast Up and Running Up and running in hours compared to competitive cluster databases Install re-engineering includes: Push-Button install for purescale clusters Socket complexity reduced by at least 40% Smarter defaults, intuitive options, parallel & quick pre-deployment host validation 30-step native GPFS setup reduced to simple 4-step DB2 install process Also easier conversion to GPFS replication post-deployment using db2cluster Increased Resiliency for aborted/partial installations Clean rollback for re-installation Additional assistance via: Simplified documentation Enhanced pre-checking of storage, tiebreaker devices, network adapters, firmware libraries Intuitive and user-friendly errors & warnings 26 The DB2 purescale Feature in Version 11.1 provides a simplified installation and deployment process. Users will be up and running within hours of starting the installation process, which gives users a tremendous advantage over other cluster databases. With DB2 Version 11.1, many of the DB2 purescale Feature installation processes have been simplified, with smarter defaults, intuitive options, and parallel and quick pre-deployment validation across hosts. Additional improvements to installation and deployment include the reduced step-by-step process of installing GPFS replication to DB2 servers, and increased resiliency for aborted, or partial installations, with support for a clean rollback for a clean restart.

32 Simplified purescale Storage Replication Deployment Today DB2 Version Takes ~8 native GPFS commands to create a replicated file system with the standard three redundancy groups. Takes ~24 native GPFS commands to convert a non-replicated FS to a replicated FS. Takes ~8 native GPFS commands to add a new disk to an existing replicated FS Takes ~7 native GPFS commands to remove a disk from existing replicated FS 1 db2cluster command 2 db2cluster commands (one for conversion and one for adding storage) 1 db2cluster command 1 db2cluster command 27

33 Linux Virtualization Enhancements RDMA over Converged Ethernet (RoCE) support added in VMWare RoCE SR-IOV for RHEL 7.2 only Allows a single adapter to be shared across multiple partitions Single-Root I/O Virtualization (SR-IOV) Standard that enables one PCI Express (PCIe) adapter to be presented as multiple separate logical devices (Virtual Functions) to virtual machines Allow the virtual machines to run native RoCE and achieve near wire speed performance. Can be enabled on Mellanox ConnectX-3/ConnectX-3 Pro/Connect X-3 VPI adapters for Ethernet 28

34 VMWare RoCE Adapter Sharing Example VMWare Partitions Shared RoCE Adapters RoCE Network X SAN storage is for KVM deployment, so the example above is a KVM deployment Example Deployment KVM Hypervisor: Red Hat Enterprise Linux (RHEL) 6.2 and higher KVM Architecture: x64 system that is supported by both RHEL 6.2 and DB2 purescale Minimum guest OS: RHEL 6 and higher Storage: SAN disks with support of SCSI-3 PR I/O fencing Tie-breaker disk: SAN disks with support of SCSI-3 PR I/O fencing Transport: SOCKETS (If you are using a network card that is less than 10GE, you must set the DB2_SD_ALLOW_SLOW_NETWORK registry variable to ON.) RDMA over Ethernet (RoCE): RDMA over Converged Ethernet (RoCE) is supported for KVM environment if the network adapter is assigned to the guest VM using the PCI device pass-through mode.

35 HADR Support for SYNC and NEARSYNC Mode Support for SYNC and NEARSYNC has been added to purescale This enhancement combines the continuous availability of DB2 purescale with the robust disaster recovery capabilities of HADR providing an integrated zero data loss (i.e. RPO=0) disaster recovery solution HADR peer window (hadr_peer_window) is not supported HADR support with purescale now includes: SYNC, NEARSYNC, ASYNC and SUPERASYNC modes Time delayed apply, Log spooling Both non-forced (role switch) and forced (failover) takeovers CF CF HADR Primary Cluster CF CF 29 Standby DR Cluster Support for SYNC and NEARSYNC for hadr_syncmode database configuration parameter has been added to purescale. This means that customers can now have another purescale cluster available in a remote location in the event the entire data center becomes unavailable. Using HADR as a disaster recovery with purescale overcomes the distance limitations associated with using GDPC (Geographically dispersed DB2 purescale cluster).

36 GDPC Support Enhancements M1 M3 CF P CF S M2 M4 Site A Site B Workload fully balanced DB2 V11 adds improved high availability for Geographically dispersed DB2 purescale clusters (GDPC) for both RoCE & TCP/IP Multiple adapter ports per member and CF to support higher bandwidth and improved redundancy at the adapter level Dual switches can be configured at each site to eliminate the switch as a sitespecific single point of failure (i.e. 4-switch configuration) Member 1 Primary CF Member 3 ro0 ro1 en2 ro0 ro1 en2 ro0 ro1 en2 Storage Switch 1 Peer 1 Switch 2 Peer 1 Site 1 GPFS replication Site 3 Storage Switch 3 Peer 2 Switch 4 Peer 2 Site 2 ro0 ro1 en2 ro0 ro1 en2 ro0 ro1 en2 30 Member 2 Secondary CF Member 4 GDPC extends its support on 10GE RoCE and TCP/IP network to cover all purescale supported operating systems: AIX, SuSE Linux Enterprise Server (SLES) and Red Hat Enterprise Linux (RHEL). Each of these configuration now supports multiple adapter ports per member and CF to support higher bandwidth and improved redundancy at the adapter level. Furthermore, dual switches can be configured at each site to eliminate the switch as a site-specific single point of failure.

37 Horizontal Scaling with DB2 purescale on POWER Linux Thousands of SQL statements/ s Scale-out Throughput DB2 purescale on LE POWER Linux Sockets RDMA member 2 members 3 members 4 members x scaling on RDMA Ethernet, 1-4 members 3.1x scaling on TCP sockets, 1-4 members % read / 20% write OLTP workload POWER8 4c/32t, 160 GB LBP 10 Gb RoCE RDMA Ethernet / 10 Gb TCP sockets Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. X

38 31

39 Unified Workload Balancing with purescale Version 11.1 extends the configuration options for member subsets allowing the user to explicitly define alternate members for a subset This allows users currently using client affinity to move their configuration to using member subsets and failover priority so that they can exploit the new benefits such as dynamic server side reconfiguration Simplification to setting up client affinity with having control at the Server vs client no need to update db2dsdriver.cfg Failover priority for member subsets added You can explicitly define the members that are part of the alternate member list by using the FAILOVER_PRIORITY attribute in the WLM_ALTER_MEMBER_SUBSET procedure Members with a failover priority of 0 (the default priority) are considered primary members and members with failover priority are considered alternative members The number of primary members in the subset defines the minimum number of members to service an application If FAILOVER_PRIORITY is not specified, default priority of 0 is used 32

40 Member Subsets : FAILOVER PRIORITY CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('BATCH', NULL,'(ADD 2 FAILOVER_PRIORITY 1)'); CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('OLTP', NULL,'(ADD 3 FAILOVER_PRIORITY 1)'); SUBSET MEMBER FAILOVER_ PRIORITY BATCH 0 0 BATCH 1 0 BATCH 2 1 OLTP 4 0 OLTP 5 0 OLTP 3 1 This information is available from SYSCAT.MEMBERSUBSETMEMBERS and db2pd membersubsetstatus -detail BATCH OLTP Member 0 Member 1 Member 2 Member 3 Member 4 Member 5 CF Database CF X DB2 Version 11.1 extends the configuration options for member subsets allowing the user to explicitly define alternate members for a subset. You can explicitly define the members that are part of the alternate member list by using the new FAILOVER_PRIORITY attribute in the WLM_ALTER_MEMBER_SUBSET procedure. Members with a failover priority of 0 (the default priority) are considered primary members (ie. where the workload runs) and members with failover priority are considered alternative members (should a failure occur to a primary member). The number of primary members in the subset defines the preferred number of members to service an application. As long as there are alternate members available, DB2 will maintain the preferred number for members for the member subset should a primary member fail. The addition of FAILOVER PRIORITY allows users currently using client affinity to move their configuration to using member subsets and failover priority so that they can exploit the new benefits such as dynamic server side reconfiguration and simplification to setting up client affinity with having control at the Server vs client no need to db2dsdriver.cfg If FAILOVER_PRIORITY is not specified, default priority of 0 is used

41 Member Subsets : FAILOVER PRIORITY CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('BATCH', NULL,'(ADD 2 FAILOVER_PRIORITY 1)'); CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('OLTP', NULL,'(ADD 3 FAILOVER_PRIORITY 1)'); SUBSET MEMBER FAILOVER_ PRIORITY BATCH 0 0 BATCH 1 0 BATCH 2 1 OLTP 4 0 OLTP 5 0 OLTP 3 1 This information is available from SYSCAT.MEMBERSUBSETMEMBERS and db2pd membersubsetstatus -detail BATCH OLTP Member 0 Member 1 Member 2 Member 3 Member 4 Member 5 CF Database CF X The application is going to fail over to one of the alternate members with the lowest failover priority when there are alternate members that are defined and one of the primary members is not available If there are multiple alternate members with the same failover priority, then the members that are chosen for the failover application are based on the lowest member number from the lowest priority grouping

42 Member Subsets : FAILOVER PRIORITY CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('BATCH', NULL,'(ADD 2 FAILOVER_PRIORITY 1)'); CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET('OLTP', NULL,'(ADD 3 FAILOVER_PRIORITY 1)'); SUBSET MEMBER FAILOVER_ PRIORITY BATCH 0 0 BATCH 1 0 BATCH 2 1 OLTP 4 0 OLTP 5 0 OLTP 3 1 This information is available from SYSCAT.MEMBERSUBSETMEMBERS and db2pd membersubsetstatus -detail BATCH OLTP Member 0 Member 1 Member 2 Member 3 Member 4 Member 5 CF Database CF X

43 DB2 purescale Health Check Unified health check tool for a DB2 purescale cluster Post-installation command: db2cluster verify Validations performed include, but are not limited to, the following: Configuration settings in peer domain and GPFS cluster Communications between members and CFs Replication setting for each file system Status of each disk in the file system 33

44 DB2 BLU Highlights Massively Parallel Processing (MPP) Performance Improvements Extended SQL Support Additional Oracle Compatibility Support SQL functions optimized for BLU Security Enhancements Improvements in Automatic Dictionary Creation 34

45 What is MPP? Which home is going to be built first? 35 There are some jobs that are just too big for one person to accomplish on their own in a reasonable amount of time. Likewise, in a database system running on a single server, there can be queries that are so complex or involve so much data that while possible to run, they aren't likely to be as fast as the business needs them to be. Think of a single person trying to build a barn. There are so many individual and repetitive tasks that need to be done as part of the construction process but with a single person all of these things are going to be done serially. The builder can only be working with one board, one hammer and one nail at a time. Throw more of the builders friends into the mix (and maybe some pizza and beverages to incent them) and now work can be done on different parts of the barn in parallel. The cows will have a comfortable place to sleep much quicker. This is the basis for MPP or Massively Parallel Processing. In MPP databases, such as IBM dashdb or IBM DB2 with the Database Partitioning Feature, data is partitioned across multiple servers or nodes with each server/node having its own dedicated memory, processors, and storage to process that data locally, and in parallel. This is often referred to as a shared-nothing architecture. The individual servers in an MPP environment are typically SMP (Symmetric Multiprocessing) servers. This is definitely the case for IBM dashdb. SMP servers have multiple processors and cores sharing the same main memory space and allow for parallelization of work within the server. However, that scalability is limited by the number of processors and cores that can be fit within a single server (which depends on the architecture of the server). Having MPP capabilities means being able to scale-out well beyond that for faster processing and handling larger data sets.

46 What is MPP? Which home is going to be built first? 35 There are some jobs that are just too big for one person to accomplish on their own in a reasonable amount of time. Likewise, in a database system running on a single server, there can be queries that are so complex or involve so much data that while possible to run, they aren't likely to be as fast as the business needs them to be. Think of a single person trying to build a barn. There are so many individual and repetitive tasks that need to be done as part of the construction process but with a single person all of these things are going to be done serially. The builder can only be working with one board, one hammer and one nail at a time. Throw more of the builders friends into the mix (and maybe some pizza and beverages to incent them) and now work can be done on different parts of the barn in parallel. The cows will have a comfortable place to sleep much quicker. This is the basis for MPP or Massively Parallel Processing. In MPP databases, such as IBM dashdb or IBM DB2 with the Database Partitioning Feature, data is partitioned across multiple servers or nodes with each server/node having its own dedicated memory, processors, and storage to process that data locally, and in parallel. This is often referred to as a shared-nothing architecture. The individual servers in an MPP environment are typically SMP (Symmetric Multiprocessing) servers. This is definitely the case for IBM dashdb. SMP servers have multiple processors and cores sharing the same main memory space and allow for parallelization of work within the server. However, that scalability is limited by the number of processors and cores that can be fit within a single server (which depends on the architecture of the server). Having MPP capabilities means being able to scale-out well beyond that for faster processing and handling larger data sets.

47 What is MPP? Which home is going to be built first? 35 There are some jobs that are just too big for one person to accomplish on their own in a reasonable amount of time. Likewise, in a database system running on a single server, there can be queries that are so complex or involve so much data that while possible to run, they aren't likely to be as fast as the business needs them to be. Think of a single person trying to build a barn. There are so many individual and repetitive tasks that need to be done as part of the construction process but with a single person all of these things are going to be done serially. The builder can only be working with one board, one hammer and one nail at a time. Throw more of the builders friends into the mix (and maybe some pizza and beverages to incent them) and now work can be done on different parts of the barn in parallel. The cows will have a comfortable place to sleep much quicker. This is the basis for MPP or Massively Parallel Processing. In MPP databases, such as IBM dashdb or IBM DB2 with the Database Partitioning Feature, data is partitioned across multiple servers or nodes with each server/node having its own dedicated memory, processors, and storage to process that data locally, and in parallel. This is often referred to as a shared-nothing architecture. The individual servers in an MPP environment are typically SMP (Symmetric Multiprocessing) servers. This is definitely the case for IBM dashdb. SMP servers have multiple processors and cores sharing the same main memory space and allow for parallelization of work within the server. However, that scalability is limited by the number of processors and cores that can be fit within a single server (which depends on the architecture of the server). Having MPP capabilities means being able to scale-out well beyond that for faster processing and handling larger data sets.

48 What is MPP? Which home is going to be built first? MPP = Massively Parallel Processing Breaking up very large, complex tasks into smaller pieces that can be worked on by multiple resources in parallel 35 There are some jobs that are just too big for one person to accomplish on their own in a reasonable amount of time. Likewise, in a database system running on a single server, there can be queries that are so complex or involve so much data that while possible to run, they aren't likely to be as fast as the business needs them to be. Think of a single person trying to build a barn. There are so many individual and repetitive tasks that need to be done as part of the construction process but with a single person all of these things are going to be done serially. The builder can only be working with one board, one hammer and one nail at a time. Throw more of the builders friends into the mix (and maybe some pizza and beverages to incent them) and now work can be done on different parts of the barn in parallel. The cows will have a comfortable place to sleep much quicker. This is the basis for MPP or Massively Parallel Processing. In MPP databases, such as IBM dashdb or IBM DB2 with the Database Partitioning Feature, data is partitioned across multiple servers or nodes with each server/node having its own dedicated memory, processors, and storage to process that data locally, and in parallel. This is often referred to as a shared-nothing architecture. The individual servers in an MPP environment are typically SMP (Symmetric Multiprocessing) servers. This is definitely the case for IBM dashdb. SMP servers have multiple processors and cores sharing the same main memory space and allow for parallelization of work within the server. However, that scalability is limited by the number of processors and cores that can be fit within a single server (which depends on the architecture of the server). Having MPP capabilities means being able to scale-out well beyond that for faster processing and handling larger data sets.

49 BLU Acceleration: MPP Scale Out Technology Pervasive SMP & MPP Query Parallelism Inter-partition query parallelism simultaneous with intra-partition- parallelized, memory-optimized, columnar, SIMD-enabled, BLU processing Value Proposition Improve Response Time All servers contribute to the processing of a query Massively Scale Data Significantly beyond current practical limits Streamline BLU Adoption Add BLU Acceleration to existing data warehouses Query #1 processing 1/3 data Hash partition (BLU Acceleration) Query #1 Query #1 processing DB BLU Capacity 1/3 data Hash partition (BLU Acceleration) Query #1 processing 1/3 data Hash partition (BLU Acceleration) DB2 V11.1 BLU Capacity 10s of TB 100s of Cores 1000s of TB 1000s of Cores 36 DB2 Version 11.1 extends compressed column-organized tables to Partitioned DB2 database environments, allowing you to leverage BLU Acceleration, a combination of innovations from IBM Research and the development labs that simplifies and speeds up reporting and analytics, at massive scale through DB2 s MPP architecture. Streamlined adoption of BLU also means that existing MPP data warehouses can easily leverage the in-memory optimized columnar technology. The following specific enhancements pertain to column-organized tables in a partitioned environment: MPP aware query planning and optimization for the column vector processing engine An optimized vector format for columnar data exchange between partitions A common table dictionary allowing data to remain compressed across the network An optimized communications infrastructure designed for multi-core parallelism

50 Optimized SQL Support for Columnar Tables SQL OLAP improvements for deeper in-database analytics with column-organized tables Additional Oracle Compatibility Support DB2 : a polyglot Wide rows database Logical character support (CODEUNITS32) DGTT support (except not logged on rollback preserve rows) Parallel insert into not-logged DGTT from BLU source IDENTITY and EXPRESSION generated columns European Language support (Codepage 819) NOT LOGGED INITIALLY support Row and Column Access Control (RCAC) ROWID Support Nested Loop Join Support 37 DB2 Version 11.1 BLU Acceleration has also added SQL advances with richer function and compatibility including SQL compatibility with IBM PureData System for Analytics. This enables native columnar online analytical processing for deep in-database analytics, the analytic capabilities of PureData System for Analytics, wide rows, new data types, logical character support, improved PostgreSQL compatibility, and a wide variety of additional SQL functions being incorporated in DB2 Version 11.1.Other enhancements include BLU Acceleration support for IDENTITY and EXPRESSION generated columns, European Language support and NOT LOGGED INITIALLY support for column-organized tables.

51 Performance Improvements BLU Acceleration includes a number of performance improvements in the columnar engine Faster SQL MERGE processing Nested Loop Join Support Synopsis Table Maintenance Industry Leading Parallel Sort Improved SORTHEAP utilization Query Rewrite Improvements Push-down of a number of SQL & OLAP functions into the BLU engine Optimized SQL support for BLU 38

52 Demonstrating BLU Single Instance Improvement DB2 V11.1 on Intel Haswell EP Query Throughput BD Insights (800GB) Queries Per Hour x Reasons for Improvement Native BLU Evaluation Query Rewrite Improvements Native Sort Native OLAP (usually combined with sort) Enables query plans to remain as much as possible within the columnar engine Find areas to improve degree determination and improve parallel use 0 DB2 V10.5 FP5 DB2 V11.1 Improved SORTHEAP Utilization Configuration Details 2 socket, 36 core Intel Xeon E GHz 192GB RAM BD Insights Internal Multiuser Workload 800GB SORTHEAP used for building hash tables for JOINs, GROUP BYs, and other runtime work Efficient use allows for more concurrent intraquery and inter-query operations to co-exist. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 39

53 BLU Sort on MPP Environment Enhancements can increase BLU Acceleration performance by as much as 1.86X 40 Row sort vs BLU sort Row Sort BLU Sort Query elapsed time (s) X faster 1.86 X faster 1.48 X faster 0 Sort query 1 Sort query 2 Sort query 3 Configuration Details BLU MPP with 1 physical node/4-mlns and 8 cores per MLN 4-socket, 40 Cores Intel Xeon platform and 1 TB RAM Used Sort Target Workload 1 TB (an internal workload based on the TPC-DS schema), with 10-Million rows to sort for each scenarios Query scenarios involving single ORDER BY concentrating on sort operation 40 Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. The industry-leading parallel sort algorithm leverages the latest innovations from the IBM TJ Watson Research division. This parallel sort features a fast radix sort with superior parallelism that is able to sort compressed and encoded data.

54 Demonstrating BLU MPP Linear Scaling DB2 Version 11.1 on an IBM Power Systems E850 Cluster Queries Per Hour Scaling Hardware at constant Data Volume 1.92x 0 3 Node (10TB) - 6 MLNS 6 Node (10TB) - 12 MLNS Queries Per Hour Scaling Hardware along with Data Volume QpH Held up! 0 3 Node (10TB) - 6 MLNS 6 Node (20TB) - 12 MLNS 41 Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. The workload being used was BD Insights (an internal workload based on the TPC-DS schema) with a multi-user apparatus through Jmeter. The 3 Node cluster had an approx 10TB raw data set loaded into it. The 6 Node also had a ~10TB database and also a ~20TB database as well. The purpose is to measure scaling on two dimensions: with constant data and scaled data. In the constant data scenario (3 node vs 6 10TB) a perfect scaling story would be that the 6 node can finish the batch in half the time of the 3 Node. It gets very close to this (213 vs 111 QpH throughput score). In the scaling data scenario (3 node at 10TB, 6 20TB), a perfect story would be both sets finishing their batches in the same amount of time. Again, it gets very close with a small percentage difference (113 vs 109 QpH throughput score).

55 Outstanding BLU MPP Performance 1.9x Higher Throughput with DB2 Version 11.1 vs. the Competition 280 Query Throughput BD Insights (4TB) 1.9x Faster!! 210 Queries Per Hour Competition on 3 Nodes of AWS DB2 V11.1 on 3 Nodes of AWS 42 Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. The workload being used was BD Insights (an internal workload based on the TPC-DS schema) with a multi-user apparatus through Jmeter. The database scale was 4TB. Workload was a 60 user multi-stream executing a set of 30 complex and intermediate queries. Both BLU and the Competition ran on 3-node MPP clusters with very close hardware specs in AWS. Specs of each BLU MPP node in AWS EC2: 32 cpu threads of Ivy Bridge (Intel Xeon E v2) virtual cores 244 GiB of RAM 640 GB of SSD Specs of each node of the Competition: 32 cpu threads of Ivy Bridge (Intel Xeon E v2) virtual cores 244 GiB of RAM 2.56 TB of SSD The competitor data was retrieved on July 25, 2015

56 The Synopsis Table Maintenance improvements now in DB addresses the use case of transactions inserting a smaller set of rows into a column-organized table. This is the new behavior and automatic, by default. Instead of updating the Synopsis Table either every 1024 rows or on transaction commit boundaries, DB2 will now cache data in memory (with algorithms to manage early DB shutdown or system crash scenarios) and only update the Synopsis Table every 1024 rows. This provides a significant performance improvement for transactions which insert a small number of rows which no longer need to update the Synopsis Table, with maximum improvements achieved with single-row insert transactions. In addition to the insert performance, having fewer entries in the Synopsis Table not only allows the table to be smaller with fewer meta-data rows, but queries will also perform faster since there will be fewer rows to check and determine which set of column values to access in the column-organized table.

57 Industry Leading Parallel Sort Leverages the latest sort innovations from IBM TJ Watson Research and DB2 Development Enhancements can increase BLU Acceleration performance by as much as 13.9X BLU Sort+OLAP on SMP Environment Query elapsed time(s) Row Sort Column1 BLU Sort Sort+OLAP query 1 Sort+OLAP query 2 Sort+OLAP query 3 Configuration Details On 4-socket Intel Xeon platform with 72 Cores and 742G RAM 1 TB TPC-DS database Query scenarios involving multiple sort and OLAP operations 44 Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. The industry-leading parallel sort algorithm leverages the latest innovations from the IBM TJ Watson Research division. This parallel sort features a fast radix sort with superior parallelism that is able to sort compressed and encoded data.

58 Columnar Engine Native Sort + OLAP Support No longer compensated on single instance DB2 V11.1 with v1 as( select i_category, i_brand, cc_name, d_year, d_moy, sm_type, sum(cs_sales_price) sum_sales, avg(sum(cs_sales_price)) over (partition by i_category, i_brand, cc_name, d_year) avg_monthly_sales, rank() over (partition by i_category, i_brand, cc_name order by d_year, d_moy) rn from BDINSIGHTS.item, BDINSIGHTS.catalog_sales BDINSIGHTS.date_dim, BDINSIGHTS.call_center, BDINSIGHTS.ship_mode where cs_item_sk = i_item_sk and cs_sold_date_sk = d_date_sk and cc_call_center_sk= cs_call_center_sk and cs_ship_mode_sk = sm_ship_mode_sk and d_year = 2000 group by i_category, i_brand, cc_name, d_year, d_moy, sm_type), v2 as( select v1.i_category, v1.i_brand, v1.cc_name, v1.d_year, v1.d_moy, v1.avg_monthly_sales, v1.sum_sales, v1.sm_type, v1_lag.sum_sales psum, v1_lead.sum_sales nsum from v1, v1 v1_lag, v1 v1_lead where v1.i_category = v1_lag.i_category and v1.i_category = v1_lead.i_category and v1.i_brand = v1_lag.i_brand and v1.i_brand = v1_lead.i_brand and v1. cc_name = v1_lag.cc_name and v1. cc_name = v1_lead.cc_name and v1.rn = v1_lag.rn + 1 and v1.rn = v1_lead.rn - 1) select * from v2 where d_year = 2000 and avg_monthly_sales > 0 and case when avg_monthly_sales > 0 then abs(sum_sales - avg_monthly_sales) / avg_monthly_sales else null end > 0.1 order by sum_sales - avg_monthly_sales, cc_name fetch first 100 rows only Elapsed Time in Seconds OLAP Functions rank, avg OLAP Query Elapsed Time (s) (lower is better) V10.5 V x Faster!! 45 This is BD Insights Complex 4 a query with 2 OLAP functions (rank() and avg()). This one is much improved in DB2 V11.1 because SORT and OLAP evaluators are pushed down into BLU.

59 Columnar Engine Native Sort + OLAP Support Access Plan Difference with Native Evaluator support DB2 V10.5 Row Data Engine Columnar Data Engine DB2 V11.1 Row Data Engine Columnar Data Engine 46 The Access plans will show what part of the processing is done in the row engine vs the BLU engine. All parts below the CTQ evaluator are done in the BLU engine. Here we can see that by pushing down the SORT evaluator we keep the data processing in BLU only which contributes to the 4x speedup observed for this query.

60 BLU Acceleration: Massive Gains for ELT & ISV Apps BLU Declared Global Temporary Table (not-logged DGTT) Parallelism 16x Faster! Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 47 There are a couple of enhancements targeted at loading tables and using temporary objects. The NOT LOGGED INITIALLY clause can now be used in CREATE TABLE and ALTER TABLE statements that involve column-organized tables. Logging is temporarily disabled when The NOT LOGGED INITIALLY is specified. With logging disabled, operations such as INSERT, UPDATE, or DELETE run faster. Loading into DGTT (Declared Global Temporary Tables) is now be done in parallel and can also be done to a columnar table, resulting in space savings and faster performance.

61 Even More BLU Enhancements BLU Automatic Dictionary Creation on uncommitted data Earlier dictionary availability for Ingest/Insert/Import for large insert jobs such as INSERT with subselect Higher PCTENCODED values in SYSCAT.COLUMNS Improved query performance following insert BLU aggregation enhancements, improving memory utilization and performance Better memory and performance for single/unique group results Improved decisions when to pre-partition the input stream More MPP GROUP BY improvements New and improved Hashing algorithm 48 In previous deliverables of DB2, Automatic Dictionary Creation for BLU tables could only be completed on committed data. Uncommitted data would not be applicable to ADC and thus could prevent the BLU dictionary from being created or not include some valuable data to be part of the resulting dictionary. Now in DB , ADC can be completed and include uncommitted data, which can be quite valuable in a variety of scenarios, especially an INSERT with subselect style of statement. The end result is earlier compression, possibly compression of small tables when previously no compression may have occurred, and queries which follow get to leverage a higher percentage of compressed column values than earlier deliverables would have enabled. Focusing on BLU and aggregation, a number of improvements have been made to provide better memory utilization and increase performance, including If a group contains a single row, then the column value itself is the aggregation result. The result is sent directly out and the hash table uses up less memory, This saving is significant when there is a nearly unique GROUP BY column. Pre-partitioning of the stream done on the GROUP BY columns allows each agent to process a subset of groups and uses significantly less memory Better decisions to avoid a "Partial-Final" Group by when the partial group by does not reduce the stream by much A New 64 bit hash function is used to reduce the number of collisions in the hash table and better utilize the memory required for aggregation

62 Additional BLU Performance Improvements BLU SIMD enhancements Intel AVX2, SSE2, SSSE3, SSE4 z13, Power8, and PPC LE More functions and bit operations, SIMD comparisons Improved BLU performance in some cases for queries with OLAP and ORDER BY through sort elimination DECIMAL data type improvements (not specific to BLU tables) Optimized conversions between row and column organized formats Intel, PPCLE, z Systems (previously optimized for AIX) SUM optimizations for Linux on z Systems 49 Some additional performance work in DB offer increased performance to a variety of workloads that execute code-paths which were included in the targeted improvements. One of the secret sauces in DB2 s BLU technology for column-organized tables is the adoption and leverage of SIMD processing available in various modern processor technology. Additional developer has been done in this deliverable to (further) leverage SIMD capabilities across all supported hardware platforms, providing additional improvements ( you mileage may vary depending on workload and platform) over and above DB GA. plan optimizations are an ongoing effort in DB2. While we may never be done and complete in the optimizations (they can be complex and take time to develop!), the DB2 development team has addressed some scenarios with BLU tables where some sort operations can be eliminated. You may find examples of these improvements by comparing plans to 11.1 or 10.5 plans. The DECIMAL data type, which can be quite common in many financial and other classes of applications, has had a number of processing improvements. For those applications which utilize and have significant processing of DECIMAL columns, there should be improvements to the application workloads.

63 Support for New Data Types BINARY and VARBINARY Binary data to be stored and manipulated without overhead of BLOB type A binary string is a sequence of bytes that are used to store pictures, sound, etc BINARY and VARBINARY data types are compatible with each other and are compatible with the BLOB data type They are not compatible with character string data types, except those character strings that are defined as FOR BIT DATA Support for BINARY and VARBINARY data types enhances compatibility with other relational database management systems BINARY can contain bytes VARBINARY can be up to bytes Column support for BOOLEAN data type True, False, Unknown (NULL) Restrictions include Replication (SQL / Q Replication, CDC) Embedded SQL support specifically host variable support ALTER NICKNAME column type change to/from BOOLEAN ANALYZE_TABLE expression 50 BINARY and VARBINARY data types are added (DB GA). Column support for the BOOLEAN data type is now in DB Valid values for BOOLEAN include TRUE, true, t, yes, y, on, 1, FALSE, false, f, no, n, 0 Restrictions in this first delivery of BOOLEAN include Replication of a BOOLEAN column Since IMPORT using CREATE mode is deprecated, it is blocked for tables containing BOOLEAN columns ALTER NICKNAME supports column name change for BOOLEAN type, not to/from BOOLEAN data type BOOLEAN data type will not be supported in ANALYZE_TABLE External functions with Language C, CLR and OLE won t support BOOLEAN input/return External procedures with Language C, COBOL, CLR and OLE will not support BOOLEAN input/return

64 Workload Manager Dispatcher WLM Dispatcher: CPU control at the database level Database level CPU Shares and Limits allows prioritized allocation of CPU in multi-database environments wlm_cpu_shares, wlm_cpu_share_mode, and wlm_cpu_limit DB CFG parameters Previously only supported at the service superclass and subclass levels Database X: High Priority (700 SOFT CPU shares) Database Y: Low Priority (300 HARD CPU shares) Service Class A Service Class B Service Class C 51 CPU Share 500 CPU Share 200 CPU Share 300 Several new WLM improvements are being introduced in : - You can now assign CPU Shares and Limits at the database level with the WLM Dispatcher, simplifying the task of managing resources in multi-database environments. A single database level share value now controls the division of CPU resources between databases, while the CPU share value at the service superclass level controls the division of the database level CPU resources. You can also place a CPU limit at the database level which will limit the total CPU consumption of all service classes within the database.

65 Workload Manager Activity Thresholds WLM: New activity thresholds ACTIVITYTOTALRUNTIME & ACTIVITYTOTALRUNTIMEINALLSC enable thresholds based on actual execution time of an activity Unlike ACTIVITYTOTALTIME, ACTIVITYTOTALRUNTIME does not include time queued by WLM ACTIVITYTOTALRUNTIMEINALLSC is similar to ACTIVITYTOTALRUNTIME, but only defined on a service subclass and supports the REMAP action CREATE THRESHOLD MAXDBACTIVITYTIME FOR DATABASE ACTIVITIES ENFORCEMENT DATABASE WHEN ACTIVITYTOTALRUNTIME > 2 HOURS STOP EXECUTION Queue wait time Runtime CREATE THRESHOLD MAXDBACTIVITYTIME FOR DATABASE ACTIVITIES ENFORCEMENT DATABASE WHEN ACTIVITYTOTALRUNTIME > 1 HOUR COLLECT ACTIVITY DATA WITH DETAILS AND VALUES 52 We have introduced new ACTIVITYTOTALRUNTIME and ACTIVITYTOTALRUNTIMEINSC thresholds which operate based on the amount of time an activity has actually been executing (excluding queueing) as opposed to the time it has spent in the system. This allows more granular control and flexibility for managing your workload behavior.

66 New CREATE FUNCTION statement for aggregate UDFs The new CREATE FUNCTION (aggregate interface) statement allows you to create your own aggregate functions An aggregate function returns a single value that is the result of an evaluation of a set of like values, such as those in a column within a set of rows Use your choice of programming language Four sections within the function are defined based on the stage of the aggregation process INITIATE ACCUMULATE MERGE FINALIZE 53

67 Support for the (+) Outer Join Operator Support for the outer join operator enhances cross-vendor support Queries can use the outer join operator (+) as alternative syntax within predicates of a WHERE clause The use of the Oracle compatibility mode is not required to enable this feature 54

68 COLLATION_KEY Function COLLATION_KEY The COLLATION_KEY function returns a VARBINARY string that represents the collation key of the expression argument, in the specified collation This function is used in SQL for ordering and comparison operations Format COLLATION_KEY( expression, collation_name, [length] ) Arguments Expression - An expression for which the collation key is determined Collation_Name - An expression that specifies the collation to use when the collation key is determined Length - An expression that specifies the length attribute of the result in bytes Example: The following query orders employees by their surnames by using the language-aware collation for German in code page 923 SELECT FIRSTNME, LASTNAME FROM EMPLOYEE ORDER BY COLLATION_KEY (LASTNAME, 'SYSTEM_923_DE') 55

69 New Functions, Data Types and Columnar Optimization Date/Time Date/Time Statistics Bit Manipulation Data Types Strings OLAP Pushdown OLAP Pushdown DATE_PART ADD_YEAR COVARIANCE_SAMP HASH INT2 STRPOS RANK FIRST_VALUE DATE_TRUNC ADD_MONTHS STDDEV_SAMP HASH4 INT4 STRLEFT DENSE_RANK RATIO_TO_REPORT AGE ADD_DAYS VARIANCE_SAMP HASH8 INT8 STRRIGHT ROW_NUMBER EXP LOCALTIMESTAMP ADD_HOURS CUME_DIST TO_HEX FLOAT4 REGEXP_COUNT LPAD LOG10 NOW Function ADD_MINUTES PERCENT_RANK RAWTOHEX FLOAT8 REGEXP_EXTRACT RPAD COLLATION_KEY THIS_QUARTER ADD_SECONDS PERCENTILE_DISC INT2AND BPCHAR REGEXP_INSTR TO_CHAR LN THIS_WEEK DAYOFMONTH PERCENTILE_CONT INT2OR BINARY REGEXP_LIKE INITCAP TO_NUMBER THIS_YEAR FIRST_DAY MEDIAN INT2XOR VARBINARY REGEXP_MATCH_COUNT TO_DATE MOD THIS_MONTH DAYS_TO_END_OF_MONTH WIDTH_BUCKET INT2NOT LOG REGEXP_REPLACE MONTHNAME SIN NEXT_QUARTER HOURS_BETWEEN COVAR_POP INT4AND RANDOM REGEXP_SUBSTR DAYNAME COS NEXT_WEEK MINUTES_BETWEEN STDDEV_POP INT4OR BOOLEAN BTRIM POWER TAN NEXT_YEAR SECONDS_BETWEEN VAR_POP INT4XOR AVG COT NEXT_MONTH DAYS_BETWEEN VAR_SAMP INT4NOT COUNT ASIN NEXT_DAY WEEKS_BETWEEN INT8AND COUNT_BIG ACOS EXTRACT INT8OR MIN ATAN INT8XOR MAX TRUNCATE INT8NOT SUM 56 This is a summary list of all of the new, enhanced, and optimized SQL that is being included in the D

70 Compatibility features for Netezza Performance Server Use the SQL_COMPAT global variable to activate the following optional NPS compatibility features SET SQL_COMPAT='NPS' Compatibility Features Double-dot notation You can use double-dot notation to specify a database object TRANSLATE parameter syntax TRANSLATE (char-string-exp, from-string-exp,to-string-exp) NPS Operator Symbols The operators ^ and ** are both interpreted as the exponential operator, and the operator # is interpreted as bitwise XOR Grouping by SELECT clause columns You can specify the ordinal position or exposed name of a SELECT clause column when grouping the results of a query Routines written in NZPLSQL The NZPLSQL language can be used in addition to the SQL PL language. X DB2 provides features that enable applications that were written for a Netezza Performance Server (NPS) database to use a DB2 database without having to be rewritten. In addition to the Netezza Performance Server (NPS) compatibility features that are always active, you can use the SQL_COMPAT global variable to activate the following optional NPS compatibility features: Double-dot notation TRANSLATE parameter syntax NPS Operator Symbols Grouping by SELECT clause columns Routines written in NZPLSQL

71 Removal of Restrictions and Limitations 32K system temp tablespace not required for extended row sizes Alter VARCHAR/VARGRAPH length in column-organized tables Size increase only, cannot modify string units (e.g. CODEUNIT32) Log file size limit increased to 64GB (from 4GB) New maximum of 64GB = 16,777,152 4k pages db2convert command now supports tables in MPP and with generated columns Allow system maintained MQT in MPP that references a nickname Enable WITH clause to be used in SELECT INTO statements for Static SQL and SQL Stored Procedures or Functions WITH CTE_1(c1, c2) as (select c1, c2 from t1) SELECT sum(c2) into :outvar FROM CTE_1 WHERE c1 between :lowval and :highval 57 The items on this chart are limitations/restrictions which are now lifted in DB These should be self describing and need no additional information.

72 Data Server Manager What s New for DB Manage your IBM data Enterprise Enterprise view of DB2 and dashdb KPIs and alerts from one dashboard Data Server Manager Login authentication through LDAP New enterprise grid view shows hundreds DB2 and dashdb servers at a glance Monitor In Depth New powerful database performance overview page for rapid problem determination Database time breakdown 18 KPIs in one screen for realtime or historical analysis Breakdown by workload For DB2 and dashdb Ensure DB2 High Availability See the availability of all of your HADR and purescale clusters at a glance Dashboard, SNMP and alerts that highlight potential availability problems before they happen Making remote data act like local data Make data in remote DB2, BigSQL or dashdb databases look and act like local tables with and Fluid Query Explain and monitor queries with calls to remote data sources with There are many improvements you will find with Data Server Manager The biggest enhancements are in enterprise management, in depth database monitoring, high availability monitoring and federation (Fluid Query) support for DB Data Server Manager can now act as the administration and monitoring console for a number of dashdb products. By establishing a connection to a dashdb Local, dashdb for Transactions, or dashdb for Analytics database, Data Server Manager automatically reconfigures its feature set to compliment the selected database type. IBM Data Server Manager can now be configured to use LDAP authentication. Credentials validation and DSM user privilege assignment can be delegated to an Open LDAP or Apache Directory server. DSM Administrators can configure DSM to use LDAP for user management. In contrast to the tile view that users of IBM Data Server Manager are used to navigating, the new grid view displays each monitored database as a row in a table. Displayed properties include the database name, current status, alerts, and performance metrics. You can now easily manage hundreds of databases at a glance. The Data Server section of the Overview page is now more intuitive, providing users with an easy-to use, yet comprehensive, database monitoring experience: The Database time breakdown chart now combines five metrics into one line graph, allowing users to see how the peaks and valleys of each metric. You can also view key performance metrics for the whole database or by individual workload. IBM Data Server Manager now provides monitoring and alert services for purescale with HADR configurations of IBM DB2 V10.1 (and above). Users can now add connections to other purescale members and CF nodes as well as all HADR standby members and CF nodes automatically, when adding a connection to a member node in a purescale with HADR cluster. You can: * Easily confirm the status of your high availability service. * Monitor databases within both Primary purescale cluster and Standby purescale cluster nodes in a purescale HADR cluster. * Display tiles by cluster on the Home page, using the primary member tile sorting, shown items, and Sort by controls. * Drill down to view the health status of nodes in your PureScale HADR cluster. * See alerts of issues occurring in your PureScale/HADR cluster, as they happen. Data Server Manager now supports DB2 Fluid Queries (Federation) for DB , a technology that allows users to connect to remote tables that look and act like local tables. You can connect to remote DB2, BigSQL or dashdb servers, browse available tables and make federate them in DB2 so they look like local tables Data Server Manager also supports monitoring federated queries through the Database Monitoring page for DB so that you can diagnose even complex queries that span multiple systems. Visual explain now includes detailed Federated query information. New metrics charts are now available through the new Metrics filter drop-down list box and several new metrics have been added to the list of available columns on the Monitor > Database > Statements pages.

73 DS Driver and DB2 Connect - Focus for DB Administration: Extending deployments across the Enterprise Who What Enterprise DBA and Infrastructure Teams Manage the IBM DS Driver easily Capability Improve monitoring and problem determination Growing Core Mission Critical workloads Who What Enterprise DBA, Infrastructure, LOB Identify and optimize expensive workloads, extend application and data store usage, and direct resources to the right users or applications. Capability Keep dynamic Timestamp conversions WLB with multi function Blocking for scrollable cursors ODBC register with descriptions In-memory keydb for SSL connection Bringing Cloud Workloads to IBM Data Stores Who What Data Analyst, Developer, Data Scientist Deploy modern applications to Cloud accessing IBM s DB s Capability Net driver simplification supporting Cloud deployment (ADO.Net) New Hybrid, Cloud, and Mobile model Rest Drivers service access to DB2 and dashdb (New Data Server Gateway for ODATA) Expanding New Workloads to IBM Data Stores Who What Capability Enterprise DBA, Developer, LOB (app owners) Support DB2 and critical platforms MS EF7 support DB2 for z/os Downloadable DS Drivers form the basis for connectivity from clients and applications to DB2. The same client technology supports both DB2 and dashdb LUW and Cloud access and with a license key applied also supports DB2z and DB2 I access. New platform enhancements: ADO.Net Package to help simplify deployment Security enhancements In-Memory keydb for SSL connection Performance enhancements: Blocking support for scrollable cursors Keep dynamic support for DB2 for z/os 12 Usability enhancement Microsoft ODBC Data Source Administrator GUI enhancements for dsdriver Manageability enhancements: Register ODBC DSN with Description Mobile and Rest service access: OData access and gateway in support of mobile application access to DB2. With this support Odata Rest interfaces are converted to SQL with results then returned back to the requester.

74 DB2 Version 11.1 Highlights Core Mission Critical Workloads : Extending DB2 Leadership Comprehensive Enterprise Security Enterprise Encryption Centralized Key Managers (KMIP) Availability 2 nd only to DB2 for zos Warehousing Workloads : Most Consumable, Most Scalable In-Memory Warehousing Platform Massive Scale Warehousing at In-Memory Performance MPP BLU Scalability PB scale in-memory warehousing 60 Simple Fast Deployment Up and running in hours Even Greater Availability Zero data loss DR with HADR More online management More Platforms Supported Power Linux (LE) Virtualization for RDMA (x86) Significant Core Database Advances Very Large Database Performance Higher user throughput Simpler, Faster, More Online Upgrades Faster, no need for offline backup Streamlined HADR upgrade DB2 Version 9.7 direct to 11.1 Next Gen In-Memory Performance, Function & Workloads Faster ELT/ETL performance More Query Workloads Optimised More Function supported Generated Columns RCAC OLAP + BLU Perf Enhanced Compatibility Multi-Lingual SQL Advances PostgresSQL Support for European Languages Codepage 819 This chart shows the high-level features that are being delivered as part of the DB2 V11 release. The major features include: - Improved enterprise encryption support - Faster up and running for purescale installation - New Sync and Near-Sync support for HADR in a purescale environment - Additional support for Power Linux Little-endian (LE) and purescale - Virtualization support of RDMA adapters in a VMWare environment - Higher throughput in large database systems - Easier upgrade process and direct upgrade from 9.7 systems - BLU support on MPP clusters - Faster query processing for BLU queries and support for generated columns, row and column access control and additional functions pushed down into the columnar engine - Additional SQL support in DB2 including some Postgres syntax, new data types and many new functions - A new code page for BLU systems (Codepage 819 for European languages)

75 DB Highlights 61 Comprehensive Enterprise Security Enterprise Encryption PKCS#11 HSM support SSL Encryption for HADR Initially only on Linux/x86 Higher Availability and Core Capabilities Even Greater Availability Seamless HADR purescale upgrades ADMIN_MOVE_TABLE new FORCE_ALL Asynchronous UNDO (technical preview) Additional Core Function Workload Manager (WLM) enhancements Federation simplification and integration Improved DECIMAL and DECFLOAT perf More Compatibility and Serviceability Even Greater SQL Compatibility Column support for BOOLEAN data type Common table expression (DB2for z/os) WITH and SELECT INTO support Increased Serviceability Client info in db2diag for lock and lock errors Backup image size in event monitor history Raise maximum log file size to 64 GB Column-Organized (BLU) Tables Performance Improvements Synopsis table enhancements (HTAP) Additional SIMD exploitation Aggregation enhancements, sort elimination INSERT from sub-select performance improvements Additional advantages for SAP Added Function Removing Limitations Automatic Dictionary Creation on uncommitted data ALTER VARCHAR/VARGRAPHIC length support Additional Operating System Support PureScale SLES 12, RHEL 6.8 on x86 Ubuntu on z Systems Non-pureScale on Ubuntu Text Search support Windows Support for Windows Server 2016 This chart shows at a very high level the highlights of new capabilities, compatibilities, performance, serviceability, and support improvements and extensions being delivered in DB

76 Technical Preview: Online Crash Recovery (Async UNDO) During Crash Recovery (or HADR Takeover by Force), connections are not allowed to the database until all recovery is complete A REDO phase to redo all transaction activity to the end of log An UNDO phase to rollback transactions that had not committed As a technical preview in DB , non-production environments can be configured to allow connections and activity into the database after the REDO phase and while the UNDO phase is executing, using a simple registry variable db2set DB2_ASYNC_UNDO=yes Technote describing this in further detail will be published, titled Database accessibility during Backward phase of crash-recovery or 'HADR Takeover by Force' 62 The online crash recovery with asynchronous UNDO discussed on this slide is not supported in DB and should not be used in production environments (i.e. do not set the undocumented registry variable DB2_ASYNC_UNDO other than to have early access to test this capability in your non-production environments). There will be a detailed technote published that describes the capability, locking behavior (when UR transactions will or will not have access during the Async UNDO), and monitoring. At the time of writing these notes the draft technote was here (may be in a different location when becomes available)

77 Technical Preview: Online Crash Recovery (Async UNDO) ps No Change prior to DB down DB up DB down DB up (*) DB up DB up REDO UNDO Total Transactions Per Second purescale New Connections allowed to DB after REDO, just as asynchronous UNDO begins Current (*) Some data may be locked while UNDO proceeds concurrently Time This graph shows the availability characteristics of a DB2 database (with purescale, non-purescale prior to , and now non-purescale in ) following a crash when DB2 is performing crash recovery. We ll focus on the non-purescale example here, as purescale had already been designed to offer maximum database and data availability in case of a crash. DB2 has a REDO and an UNDO phase of crash recovery. REDO ensures that all activities that had occurred prior to the crash (for log records that were written to the log files for example, committed transaction activity) are replayed and in the database as if the crash had not occurred. The UNDO pass then rolls back all uncommitted transactions that were in flight at the time of the crash. Connections to the database are not allowed until both the REDO and UNDO phases of crash recovery are completed. Now in DB , DB2 will allow connections to the database after the REDO phase and during/while the UNDO phase executes. Data (tables) that have uncommitted activity to be undone will be locked, but access to all other tables will be allowed. This is of most benefit in batch/etl processing environments where a large batch job which inserted/updated/deleted many many rows will take some time to undo this should not prevent applications from connecting to the database while the large and uncommitted transaction is being rolled back in the crash recovery UNDO phase.

78 64 For questions regarding the content of this deck, please contact Rick Buglio at