What's new and exciting in Tools for DB2 for z/os

Transcription

1 DB2 User Group Italia What's new and exciting in Tools for DB2 for z/os Elisabetta Curci e_curci@it.ibm.com IBM Analytics on System Z Platform Sales Representative Italy

2 Please Note: IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 2

3 Agenda IBM DB2 z/os Tools Today IBM DB2 z/os Tools Directions 2016 Some specific new tools Data Server Manager What is DSM Integrating Products and Timelines Directions Features Infosphere System Z Connector for Hadoop 3

4 IBM DB2 z/os Tools and z Analytics today DB2 Administration tool DB2 Administration toolkit SAP Edition DB2 Automation Tool DB2 Automation Tookit SAP Edition DB2 Bind Manager DB2 Buffer Pool Analyzer DB2 Change Accumulation Tool DB2 Cloning Tool Db2 High Performance Unload DB2 Log Analysis Tool Db2 Object Comparison Tool DB2 Object Restore DB2 Path Checker DB2 Perform Toolkit SAP Edition QMF z/os DB2 Query Monitor DB2 Query Workload Tuner...and more! DB2 Recovery Expert DB2 Sort DB2 SQL Performance Analyzer DB2 Table Editor DB2 Utilities Enhancement Tool DB2 Utilities Suite DB2 Query Workload Tuner Infosphere Classic Federation Infosphere Classic Replication Server Infosphere family for data governance IS Data Architect IS Information Server IS Optim Archive IS Optim Test Data Management IS Data Replication Tivoli Omegamon XE for DB2 PM/PE DB2 Analytics Accelerator DB2 Analytics Accelerator Loader 4

5 DB2 Tools Delivered Q32015 Q12016 DB2 Administration Tool/Object Comparison Tool v11.2 More support for IBM DB2 Analytics Accelerator Enhanced Change Management and usability features DB2 Cloning Tool v3.2 Ability to clone a subsystem at the data set level instead of the volume level, reducing the footprint of the clone Clone table spaces from image copies NO DOWNTIME! No fast-replication storage requirement Consistent cloning across z/os LPARS allows you to easily clone large applications to a new LPAR at a consistent point-in-time DB2 Recovery Expert v3.2 Mass application recovery: Prioritize mass recovery based on application priorities (no fast-replication storage requirement) Redirected recovery can backup object from source (table, table space, database) and restore to different target (database, LPAR NO DOWNTIME IMPACT to SOURCE!!! Backout errant transactions to ensure errors are removed from applications 5

6 IBM DB2 z/os Solution Packs Today DB2 Utilities Solution Pack DB2 Automation Tool DB2 High Performance Unload DB2 Sort DB2 Utilities Enhancement Tool Automate and enrich the IBM utilities DB2 Performance Solution Pack DB2 SQL Performance Analyzer Tivoli Omegamon XE for DB2 PE Optim Query Workload Tuner DB2 Query Monitor Master the performance lifecycle Complete solutions for all needs Simplified install and maintenance Build intelligence into when and how actions are performed Optimize performance and resource utilization associated with DBA activities to reduce TCO DB2 Tools Solution Packs 6 DB2 Fast Copy Solution Pack DB2 Cloning Tool DB2 Recovery Expert Superior avail & cost for copy, backup & recovery operations DB2 Administration Solution Pack DB2 Administration Tool DB2 Object Comparison Tool DB2 Table Editor Optim Configuration Manager Manage objects & schema

7 Directions for DB2 Tools 7

8 Strategic Directions for DB2 Tools Self-managing DB2 systems Extensive set of alert thresholds & recommended actions Advanced action resolution (including calendaring) Integration of performance tools for true intelligent decision making Move from automation to true autonomic capability Simplification & Modernization of Administration Autonomics Advanced graphical interfaces Convergence of tools Cloud & Database Provisioning Install, migrate, decommission subsystems Simple, automatic provisioning of tools in support of new systems Exploit z/osmf support for workflows RESTful APIs to support marketplaces Analytics (including Enterprise Spark) Enable access to all z data for the purpose of analytics Identify opportunities to support Data Scientist & other use cases Leverage streams & Spark for improved efficiency (e.g. continuous distribution statistics RUNSTATS elimination) 8

9 DB2 Tools Portfolio Focus in 2016 All DB2 12 Toleration & Exploitation! Portfolio refresh coincident with DB2 VNEXT (Versions, Releases, PTFs) Administration Solution Change Management enhancements (CM Batch, Multi-target Change) Improved Accelerator support Integration of provisioning capability into Administration Pack Expose Admin functions via REST APIs (for UCD or DSM integration) Utilities Solution New version of HPU for performance improvements and resource optimization DB2 Sort: Support for variable length block level data exchange Extend autonomics (additional data sources, REORG management, statistics management) Analytics Solution Extend Loader data sources based on client demand QMF analytics capabilities enhancements TCz GUI enhancements including workflows to facilitate configuration Performance Solution Populate autonomics model with performance data (OMPE & QM) Populate IDAA with performance data to facilitate analytics (QM) New DSM-based QWT and other products 9

10 IBM Data Server Manager 10

11 IBM Data Server Manager Deliver a Simplified User Experience Single installer and integrated repository Common integrated web console Provides enterprise view of your environment Guided workflow and analysis Deliver familiar capabilities from the Database Tools Database Administration, Query Tuning, Configuration Management 11 7

12 Key development directions mapping Key clients requirements Streamline tools, installs, repositories Make it easier to use; provide workflow, diagnostic smarts, actionable recommendations Provide better performance and scalability Manage databases across the enterprise one stop shop tooling 12

13 Data Server Manager: Overall Timeline Mar 30, 2015 Jun 30, 2015 Dec 11, 2015 ega: IBM Data Server Manager V1.1.0 Next Generation Core Tooling for DB2 (OPM, OQWT, OCM, Data Studio) - Initial DB2 for LUW Performance Management and DB Administration BigInsights-Simplicity in up and running for BigSQL- - Initial BigInsights BigSQL Administrationa and Management - Good customer reception IBM Data Server Manager V1.1.1 Extending the value of Next Generation Tooling -Second drop for DB2 for LUW, closed most gaps in differences between old tools and new tools BigInsights expanding platforms - - Updates for Linux Platforms IBM Data Server Manager V1.1.2 Expanding the value of Next Generation of DB2 Tooling to z -Third drop with support for DB2 for z/os -DB2 Query Workload Tuner for z/os V5.1 based on DSM standalone or in Performance Solutions Pack for z/os -Admin Solutions Packs for z/os updates with DB2 Configuration Manager for z/os V4.1 based on DSM -- Platform adds for DB2 LUW

14 Data Server Manager: Overall Timeline March IBM Data Server Manager V DB2 Configuration Manager z/os: DB2 Query Tuner for z/os Migration from Eclipse based OQWT to DSM Based Query Tuner Capture Query environment Integration Query Monitor. Performance Solution Pack Participating in DB2 Cypress for z/os ESP Beta OMPE V540 OPM based EI Management Console functions for DB2 for z/os (including Utility Solution Pack Autonomics) integrated into DSM 2Q Q Q 2016

15 Where can you find Data Server Manager today Data Server Manager Base Edition DB2 Performance Solution Pack for z/os DB2 Utility Solution Pack for z/os DB2 Admin Solution Pack for z/os FREE!- Use to upsell to paid products 15

16 How can I get DB2 Configuration Manager? OCM 3.1.x is part of previous releases of the z Admin Solution Pack Integrated with z Admin Tool OCM 3.1.x is part of DB2 Connect Unlimited Advanced Edition for z/os DSM 3.1.x makes DB2 Connect management easy Great value for our customers 16

17 Features of the Data Server Manager Based Tools At A Glance Data Server Manager Base Connect to DB2 for z/os V10 and V11. Database object navigation, viewing object detail, and linking to related objects. Database object dependency display. Data browsing and editing. Basic database object operations, such as creation of tables, indexes, constraints, and tablespaces; dropping of tables, indexes and constraints; altering tables. Showing system privilege from the perspective of Group/User, Role, or SQL object. Choose: -"Group/User" to see the role and the relative object privilege for a user account; - "Role" to see the role a user account belongs to and its relative object privilege; - "SQL object" to see a specific object and users or roles that have the relative authority. DB2 Query Workload Tuner for z/os V5.1 (standalone, DB2 Perf Soln Pack) Launching of visual explain and tune query on the SQL editor A new tuning wizard to capture SQL statements from multiple sources Several Tuning advisors provide recommendations for: Statistics Advisor Index Advisor IDAA Advisor Understand problem query or workload Access plan graph Query formatting and annotation Tuning Report Test Candidate Index Access Plan Comparison Index Impact Analysis DB2 Configuration Manager for z/os V5.1 (inside DB2 Admin Soln Pack only ) Track Configuration Changes Configure zparm Compare and Clone Configurations Manage Application Profile Manage alias Manage clients DB2 Utility Solution Pack V2.2 Customizable profiles for identifying object situations and generating resolving JCL (reorg, copy, runstats, etc) Automatic prioritization of object situations and their resolving actions Ability to define maintenance windows for enabling active autonomics, allowing DB2 to self manage utility runs Notification support ( , text, WTO) on selected autonomic events Graphical trend analysis of historical RTS Capture of utility history, recording utility output, time, duration, etc. 17

18 Introducing Data Server Manager Supporting DB2 for z/os Easy Database Administration Quick and simple way to navigate database objects, view data and edit, write SQL and get results back Modernize Query Tuning / Increase IDAA value Improve performance of applications leveraging DB2z Simplified, modern look and feel to database query tuning and also carried through in the DB2z Performance Pack Get more value out of your IDAA investments through evaluating IDAA eligible workloads New features such as index impact analysis Simplify Best Practices Configuration Easier to manage database clients connecting to DB2 for z/os systems, and manage what they are doing Increase value of the DB2z Administration Pack Integrated Tool for DB2 for z/os - Management Console and DSM Integration Common strategy for DB2 for z/os customers who want simple access - one tool - to manage the health of the enterprise in a modernized fashion via web browser 18

19 Data Server Manager - Key Features 19

20 3-step setup 20

21 Basic Navigation : New Tuning Wizard Abstract Help users navigate between panels more conveniently Launch point Optimize -> Query Tuning Select New to choose the Database for Tuning 21

22 Manage Databases Using the Database Explorer Manage database objects Explore the catalog Explore database object properties 22

23 Develop and Run SQL Scripts Manage scripts Explain SQL Tune SQL Validate SQL 23 Customize and filter result output Save execution results

24 Create and schedule jobs Schedule a Create Table job from Explore Databases Scheduled commands from Explore Databases Alternatively, create a script and schedule the job 2 24

25 Analytics Query workload tuner for DB2 for z/os overview Web-based user interface DSM common infrastructure Query Tools and Advisors Overview Page Index Advisor Access Plan Graph Summary Report Formatter Annotation Service SQL Statistics Advisor Access Plan Comparison Workload Tools and Advisors Summary Report Test Candidate Index Statistics Advisor Index Impact Analytics Access Plan Comparison Index Advisor IDAA Advisor Configuration Report Repository database Monitored Monitored DB2 Monitored DB2 for for z/os z/os DB2 for z/os

26 Expert-Enabled Query Tuning Out of the Box Identify query candidates from numerous sources DB2 catalog Dynamic statement cache User defined repository Facilitate analysis Query formatting Query annotation Access path visualization and annotation Get expert tuning advice Improve statistics quality Improve database design Improve accelerated queries

27 Tune SQL or Workload Review access plan graph and annotations Advisors streamline tuning process Single Query Statistics Advisor Workload Statistics advisor Index advisor IDAA advisor New! Use what-if analysis to determine what impact indexes have on a workload Use what-if analysis to determine what statements are eligible for acceleration Run scripts to implement advise or share results using tuning report 27

28 Analytics Access Plan Graph Recommendation Integration

29 Analytics Workload Index impact analysis Abstract Analyze the impact of the recommended indexes to other SQL statements Launch point Select a tuning job -> View Results -> Index script -> Test Candidate Indexes

30 Analytics Workload Index impact analysis cont. Analysis scope Statement Cache Packages Existing jobs

31 Analytics Index impact analysis cont.

32 Migrating workloads from QWT 4.1.x to DSM 32

33 Data Server Manager : Useful web sites Download Youtube APn-vF 33

34 IBM InfoSphere System z Connector for Hadoop 34

35 Whether you run Hadoop on z or off z z/os Data is important Transaction and Log Data are consistently PART OF the analysis! Evidence of the value of combining traditional data with Hadoop analytics Source : Gartner Inc, Research Note G Survey Analysis: Big Data Investment Grows but Deployments Remain Scarce in Nick Heudecker, Lisa Kart, Published 09/09/2014

36 Big Data Integration Is Critical For Success With Hadoop Most Hadoop initiatives involve - collecting, - moving, - transforming, - cleansing, - integrating, - exploring, and - analysing volumes of disparate data sources and types. By most 80% accounts of the development effort in a big data project goes into data integration goes towards and only20% data analysis. Extract, Transform, and Load Big Data With Apache Hadoop - Whitepaper:

37 For most, Big Data Integration Must Reach Back to z Systems 80% of transaction data is stored or originates on IBM z Systems Much of this is because of the scale of the 3,000+ mainframe systems Average annual downtime for System z applications is 5 minutes! Latest System z hardware capable of running 8,000 virtual linux images EAL5+ encryption & cryptographic hardware securing data in motion & at rest Empowering System z Data Integration with Hadoop & Big Data Initiatives Bring together data from both realms Preserve core transactional data security Provide up-to-date information Minimize overhead and TCO 37

38 InfoSphere System z Connector for Hadoop (Passport PID 5725S33) Z Systems IBM IOP & BigInsights Big Data NoSQL DB2 VSAM Logs IMS IDMS Datacom Linux file system & ETL Processes Open Systems DB2 Oracle SQL Server Teradata IBM System z Connector for Hadoop Network endpoint Cloud

39 IBM InfoSphere System z Connector for Hadoop Key Facts Easy to Setup and Use: Installs from Linux Drag-and-drop interface no programming required COBOL/PL1 copybook parsing & presentation to automate data reformatting Automated Code Page Conversion Stellar Performance: Expect to see dramatic z/os MIPS usage reduction over home grown & ETL Users report three and four times throughput increases Broadening Support: Multiple Hadoop distributions on and off the mainframe Multiple sources: DB2, VSAM/QSAM, IMS, IDMS, Datacom, JDBC, Log files Multiple destinations: Hadoop, Linux File Systems, MongoDB, streaming endpoints Feature Rich: Define multiple data transfer profiles Filter and translate data & columns on the fly, on the target Secure pipe (SSL) RACF and TopSecret integration Preserves metadata, landing mainframe data into targets 39

40 Behind the Scenes Real life client flow semplification 10 TB daily Transactional data Staging MQFTE Staging Code Before Mainframe processing (storage on disc and tape) Unacceptable delay with data volume increase Exceed MIPS consumption & batch window Security Challenges (unnecessary copies) Outsourcing labor cost After Transactional data System z Connector Mainframe Point & Click Solution (no programming required) Integrate with RACF mainframe security & scheduling MIPS consumption 1/10 vs previous/traditional approach 4X Speed increase moving detailed data for analysis No staging + cost saving on storage & software 40

41 41 InfoSphere System z Connector for Hadoop Key Benefits Simplicity, Speed of deployment (no z/os install), Instant value through point and click. Security (RACF & Top Secret access method so no duplicated profiles), End to end data movement & conversion with no staging. The product does not require multiple steps to make z/os data available for analytics (vs. traditional ETL or manual processes). No Cobol, no SQL and no associated cost to maintain those file transfer jobs Minimal z/os CPU usage (MIPS) as all data conversions are happening on Linux processors Integration with existing infrastructure (schedulers,...) 41

42 THANK YOU 42