Saving ETL Costs Through Data Virtualization Across The Enterprise
|
|
- Sherman Matthews
- 5 years ago
- Views:
Transcription
1 Saving ETL Costs Through Virtualization Across The Enterprise IBM Virtualization Manager for z/os Marcos Caurim z Analytics Technical Sales Specialist 2017 IBM Corporation
2 What is Wrong with Status Quo? There is not enough time in the day to move all the data. My mobile users expect to see current data, not yesterday s data.
3 Current Integration Limitations Movement Using ETL Tools System of Record Staging Server OLTP Files Files Files ETL Server Staging Server Staging Server Warehouse S Q L Reportin g Ad-hoc OLAP Represents ETL inconsistency High latency Complex, high mainframe costs
4 ETL Drives Up Mainframe Costs ETL costs found in three areas Additional hardware, storage and networking costs Labor involved in managing file transfers Wasted systems cycles (MIPS) IBM study found that to move one terabyte of data, with three derivative copies each day, amortized over a four year period added up to $8,269,335 ETL responsible for consuming 16-18% of total MIPS Clabby Analytics The ETL Problem, October 2013
5 Virtualizing Movement Cloud Mainframe virtualization enables data structures that were designed independently to be leveraged together, from a single source, in real time, and without complex, costly data movement RDBMS Web/ Mobile Logical Source Big Unstructured
6 Virtualization Use Cases Faster, easier delivery of modern systems of engagement Need for immediate insight into your customer or business Reduce the cost/complexity of accessing mainframe data Modernization Real-time Analytics Optimization
7 IBM Virtualization Manager for z/os
8 Cost Efficient Information Processing Mainframes have multiple processors General purpose processor all processing counts against capacity Specialty Engines Eligible workloads don t count against GPU capacity IBM Virtualization Manager can run 99% of its own processing in the ziip engine Enables mainframe data to be integrated in-place without processing penalty Eligible Workloads Can Run Outside of GPP within ziip GPP ziip
9 Typical ETL Process Issues with data inconsistency not timely Complex process Prone to errors Costly - high MIPS usage Analytics, Search Warehouse Staging Warehouse Load into Warehouse Transformation of data into compatible formats Adaba s IDMS Natura l IMS CICS Sequenti al Db2 for z/os VSAM extracts from mainframe and non-mainframe data sources Db2 LUW Informi x dashd B Oracle IBM Federation Server SQL Server
10 Augmenting ETL with Virtualization All data transformations run on ziip specialty engine for significantly reduced MIPS capacity usage SQL JDBC/ODBC/DRDA NoSQL JSON Services SOAP Analytics, Search z/os Connect REST/APIs Design Information delivered in right format, in realtime IBM Virtualization Server for z/os IBM ziip Specialty Engine Combined data delivered to Mapping Caching analytics Map/Reduce Join Query mainframe and Parallel/IO nonmainframe Optimization data Security Monitoring Metadata Adaba s IDMS Natura l IMS CICS Sequenti al Db2 for z/os VSAM Db2 LUW Informi x Derby Oracle IBM Federation Server SQL Server
11 Augmenting Warehouse via DVS Analytics, Search SQL JDBC/ODBC/DRDA NoSQL JSON Services SOAP z/os Connect REST/APIs Design Warehouse IBM Virtualization Server for z/os IBM ziip Specialty Engine Combined data delivered to Mapping Caching analytics Map/Reduce Query Parallel/IO Optimization Join VSAM with DW data Security Monitoring Metadata Adaba s IDMS Natura l IMS CICS Sequenti al Db2 for z/os VSAM
12 Complex ETL Script ETL Environment Source System Extract Program Pre-Landing ETL (Flow 1) Landing ETL (Flow2) Staging ETL (Flow 3) Vendor Extract ETL (Flow 4) Vendor Landing ETL (Flow 5) Vendor Updates (Flow 6) Source System Services Environment Hub Key Generation Services Hub Key Generation Services base Environment Vendor Systems Cross-Ref Pre-Landing Landing Staging Landing Staging Cross-Ref Enterprise Exchange Interface
13 SQL Insert Into Select Statement Web/Mobi le ESB, ETL Analytics, Search Transactional Can replace complex and hard to manage ETL scripts with SQL statement SQL JDBC/ODBC/DR DA NoSQL MongoDB API IBM Virtualization Server for z/os Services SOAP/REST/HT ML Web HTTP Events CDC/ Streams Mapping Caching SQL Insert Into Select Statement Map/Reduce Query Parallel/IO Optimization Design Security Monitoring IBM ziip Specialty Engine Metadata SMF Sys Logs Tape Adaba s IDMS Natura l IMS CICS Sequenti al Db2 for z/os VSAM Big SQL Hadoop Mongo DB Db2 LUW Informi x dashd B Oracle IBM Federation Server SQL Server
14 Functional Architecture Input Ingest/Transform Persist Analyse Visualise/Interact Transaction ETL Landing Zone Engineering ETL Enterprise Warehouse Visualization Reporting Dashboarding ETL Hadoop Cluster Exploratory Analytics
15 Functional Architecture Input Ingest/Transform Persist Analyse Visualise/Interact Mainframe Transaction ETL/ELT (if necessary) Analytical LPAR (if necessary) Engineering ETL Enterprise Warehouse Visualization Reporting Dashboarding ETL Hadoop Cluster Exploratory Analytics
16 Analytics LPAR Architecture Mainframe Analytics LPAR QMF Cognos XXX Distributed Visualization Transactional LPAR SparkSQL IDVM BigSQL Access DB2 IMS DB2 IMS Sharing Distributed DBs Hadoop DB2 Dash PDA IDAA Stores
17 Ingestion and Integration Component Description: The Integration component focuses on the processes and environments that deal with the capture, qualification, processing, and movement of data in order to prepare it for storage in the Repository Layer, which is subsequently shared with the Analytical and Access applications and systems OLTP LPAR Lake LPAR Integration & Ingestion Distributed Environment DB2 IMS DB2 IMS ETL Tool Hadoop IDAA Sharing IDAA Loader IDVM Spark Integration & Ingestion Existing Cobol CDC apps Stage DB2 Dash PDA IDAA Loader: Load directly into IDAA non DB2 for z/os (IMS, VSAM, Logs, etc). Can accelerate exploration and discovery CDC: Update, if needed, from OLTP DB2 Schema to an OLAP DB2 Schema and also to IDAA (both, OLTP and OLAP) Existing Cobol apps: Several cobol programs already deployed. Leverage to new Lake LPAR to control costs of data movement. Invest on exploration and discovery to reduce total number of those programs Stage and other ETL tools: leverage IDVM or SparkSQL to connect mainframe data when needed, reducing inhouse cobol development dependency. Can be deployed on Linux on mainframe to reduce latency and footprint Load into Hadoop or into DWH, Mart (depend on use case) Z Connector for Hadoop: Accelerate know mainframe data movement to the Hadoop environment
18 IBM Analytics Banking Student Loan Processing Optimizing ETL to enabling faster loan review and approval Mountains of data to process Poor data quality, complicated by millions of records to process took 12 hours to load Faster Time to Insight accessing more than 7 million records went from 12 hours to less than 13 minutes Improved TCO Complex joins in-memory were performed on the mainframe, with 93% on ziip engine Software IBM Virtualization Manager for z/os The challenge: Student loan processing was taking too long due to poor data quality and huge volumes of student data stored in IMS DB on the customer s z12 mainframe. With IBM Virtualization Manager for z/os accessing more than 7 million IMS records, the cycle went from 12 hours (via ETL) to less than 13 minutes. Complex joins in-memory were performed on the mainframe, with 93% of related processing running on the System z Integrated Information Processor (ziip). The lending institution was able to use real-time insight to processing student loans faster and more accurately improving business efficiency and avoiding regulatory fines.
19 Unlocking Z for Real-time Business Insight Simple Get transactional access, no data movement Open to all Apps Modern APIs enable access Secure Avoid risk by reducing moving data off Z Systems IBM Virtualization Manager for z/os Fast Exploits Z architecture, including parallelism and in-memory processing Cost Effective Keeps Z costs down with up to 99% ziip offload Non z/os data
20 IBM Analytics Insurance North American Insurance Firm Modernization to accelerate adding new online customers From days to milliseconds Online account origination went from 3 days to 200 milliseconds, Improved operational efficiency Overcame time delays associated with inefficient batch processes API-enabled IBM Z apps/data Enhanced developer productivity with APIs to actuarial data in IMS DB Software IBM Virtualization Manager for z/os IBM z/os Connect Enterprise Edition The challenge: New online customers at major insurance company had to wait days for confirmation of coverage when adding a new insurance product (motorcycle, boat, RV, etc.). Batch processes associated with their policy management system running on their z13 mainframe contributed to the new product request taking approximately 3 business days to complete. Actuarial data in IMS DB was API-enabled using IBM Virtualization Manager, which allowed developers to incorporated risk calculation and cost estimates into new online service. Online policy origination went from 3 days to 200 milliseconds, and registered 400+ new policies in the first 2 weeks of going live.
21 IBM Analytics Financial Services Global Financial Services - Real-time, self service analytics for faster insight into customer investment needs 17 million VSAM records Huge data volumes 15 VSAM files concatenated together brought back 17 million records Faster time to insight Enabling portfolio managers to provide timely investment advice Real-time information For business analysts who no longer waited for data to be loaded Software IBM Virtualization Manager for z/os The challenge: Prior to doing analytics, business analysts had to enlist database programmers to create reports from VSAM data residing on the IBM z13 mainframe. Getting mainframe data into the data warehouse involved a complicated, multi-step extraction process that created delays for business analysts. IBM Virtualization Manager enabled real-time access to IMS DB and VSAM data from the online dashboard of the business intelligence application. Analysts can respond faster to business requests for customer insights enabling portfolio managers to use the intelligence to make more relevant, timely investment suggestions to their clients.
22 Thank You
23 Backup slides
24 Runtime flow Sources Transaction Transaction (Mainframe) Integration Analytical Lake Storage Discovery & 2.1 Exploration ETL 2 Landing Zone 3 4 Enterprise Warehouse (and Marts) Archive 5 Engineering 6 7 Stewardship Discovery Actionable Insight Interactive Workloads Long-Running Workloads 1. Transaction data is extracted on a periodic basis or from operational systems. 1. Mainframe data can be directly access for Discovery & Exploration 2. Mainframe data is extracted based on needs and use case (not all data needed or should be moved) 2. is ingested into the analytics environment using an ETL engine (Stage or BigIntegrate) which generates the technical and operational metadata and stores it in the metadata repository for access during Engineering, Stewardship and Discovery. 3. is placed initially, when needed, in a Landing Zone (Hortonworks) where it can be staged, transformed and integrated. 4. is then loaded into an Enterprise Warehouse (DB2, dashdb, PDA, IDAA) and possibly to downstream marts (IDAA) for reporting, dashboarding and other interactive workloads. 5. As data ages, it is extracted from the Enterprise Warehouse (again using the ETL engine) and loaded into the Archive repository (Hadoop) where it can be accessed for long-running workloads such as exploratory analytics 1. DB2 for z/os transaction historical data can leverage IDAA capabilities to archive data 6. in either location (and its associated metadata) can be accessed for Engineering, modeling etc., using InfoSphere Architect. I can be accessed for Stewardship (curation, adding business metadata etc) and Discovery using the Information Governance Catalog UI. 7. Business Users and Scientists can access data either directly or through virtualization/federation tools such as BigSQL, IDVM. The users then visualize and analyze the data using their favorite tools (Cognos, SPSS, R Studio, QMF, etc.)
25 Mainframe data sources Traditional sources: The original corporate data sources are still very valuable resources. They are made up of application data (CRM, HR, and other customer data systems), transactional data (sales, events, claims, etc.), systems of record (historical data, reference data, etc.) and third-party data (provided by 3 rd part organizations e.g. census data). DB2 IMS VSAM Other 3 rd party DB Log (SMF,RMF, Midleware) DB2: High performance RDBMS In memory capabilities to even fast performance Exploration of NoSQL capabilities with native support of XML, JSON (up to 540 million transactions per hour arriving through a RESTful web API into DB2) Together, with IDAA, delivers real hybrid transactional analytical processing IMS: High performance NoSQL database (hierarchical). Fast Path High Volume Transaction Processing reaching a sustained average transaction rate of over 117,000 transactions per second on a single IMS instance. VSAM: Virtual Storage Access Method, another NoSQL database on mainframe, extreme performance. DB2 and IMS are based on VSAM. VISA process up to 145k transactions per second Logs: Another very important source data. The logs on mainframe has well defined data structure which can and should be used for analytics
26 Mainframe Analytical data lake storage Component Description: The overall purpose of the Mainframe Analytical Lake Storage component is for it to be a set of secure data repositories allowing for Discovery and Exploration of real time data, performing Actionable Insight, and utilizing Enhanced Applications, without a need to physically move from it source. Although is not mandatory, it can be use to control mainframe costs and have fine tune workload management DB2 IMS IDAA Sharing allows applications running on more than one DB2 or IMS subsystem to read and write to the same set of data concurrently. Possible architectures includes one DB2 member for transactional workload and one DB2 member as for analytical workload. Avoid unnecessary ETL, start the exploration and discovery right on transactional data without impacting applications DB2: HTAP (Hybrid Transactional/Analytical Processing) Leverage the same infrastructure to run any kind of workload. bases on DB2 are logical objects. It gives the possibility to have a transactional and analytical data model controlled by the same RDBMS. One OLTP application can access analytical data One OLAP application can access transactional data IDAA: Can be used to deploy a data warehouse and or specific data marts directly on the mainframe. IMS, VSAM, and other mainframe data can be loaded directly to be used in temporal data marts. Archive historical DB2 data to free up mainframe storage keeping it accessible
27 access Component Description: The overall purpose of the Access component is to express the various capabilities needed to interact with the Lake Repository component. The capabilities serve the access needs of data scientists, business analytics, developers, and others that need access to valuable data. virtualization: Describes any approach to data management that allows a user or application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted or where it is physically located. SparkSQL IDVM BigSQL SparkSQL Securely Integrate OLTP and Business Critical, can access almost all type of mainframe data Same distribution, no need for mainframe skills. Same set of applications language can be used: Scala / Python / Java / R / SQL Can be called from BigSQL IBM Virtualization Manager for z/os: The base for several IBM products such as QMF, Spark for z/os, IDAA Loader, etc Virtualized almost all data on mainframe, including 3 rd parties DBs, like Adabas, IDMS Can virtualized BigSQL objects to easier integration with hadoop environments Can also virtualized other distributed data stores BigSQL: Hadoop query engine derived from decades of IBM R&D investment in RDBMS technology, including database parallelism and query optimization. Can access DB2 for z/os directly thru DRDA connection Can access mainframe data thru IDVM Can access mainframe data thru SparkSQL
28 Access thru SparkSQL, BigSQL, and IDVM Application SparkSQL BigSQL IDVM DB2 IDAA IMS VSAM Other Distributed DBs Hadoop DB2 Dash PDA
29 Access thru BigSQL Application Distributed application accessing data from several sources eg.: Hadoop, DB2 and VSAM SparkSQL Big SQL calls Spark (using UDF) Big SQL native connection BigSQL IDVM Big SQL JDBC Connection DB2 IDAA IMS VSAM Other Distributed DBs Hadoop DB2 Dash PDA
30 Access thru IDVM Application Any application (distributed or mainframe) accessing and joining data from several sources eg.: Hadoop, DB2 and IMS SparkSQL JDBC connection to all data on mainframe BigSQL objects can be declared on IDVM to simplify access BigSQL IDVM DB2 IDAA IMS VSAM Other Distributed DBs Hadoop DB2 Dash PDA
31 Access thru SparkSQL on mainframe Application scientist tasks, leveraging mainframe data: Scala / Python / Java / R / SQL Spark z/os BigSQL IDVM DB2 IDAA IMS VSAM Other Distributed DBs Hadoop DB2 Dash PDA
32 Ingestion and Integration Component Description: The Integration component focuses on the processes and environments that deal with the capture, qualification, processing, and movement of data in order to prepare it for storage in the Repository Layer, which is subsequently shared with the Analytical and Access applications and systems OLTP LPAR Lake LPAR Integration & Ingestion Distributed Environment DB2 IMS DB2 IMS ETL Tool Hadoop IDAA Sharing IDAA Loader IDVM Spark Integration & Ingestion Existing Cobol CDC apps Stage DB2 Dash PDA IDAA Loader: Load directly into IDAA non DB2 for z/os (IMS, VSAM, Logs, etc). Can accelerate exploration and discovery CDC: Update, if needed, from OLTP DB2 Schema to an OLAP DB2 Schema and also to IDAA (both, OLTP and OLAP) Existing Cobol apps: Several cobol programs already deployed. Leverage to new Lake LPAR to control costs of data movement. Invest on exploration and discovery to reduce total number of those programs Stage and other ETL tools: leverage IDVM or SparkSQL to connect mainframe data when needed, reducing inhouse cobol development dependency. Can be deployed on Linux on mainframe to reduce latency and footprint Load into Hadoop or into DWH, Mart (depend on use case) Z Connector for Hadoop: Accelerate know mainframe data movement to the Hadoop environment
ANY Data for ANY Application Exploring IBM Data Virtualization Manager for z/os in the era of API Economy
ANY Data for ANY Application Exploring IBM for z/os in the era of API Economy Francesco Borrello francesco.borrello@it.ibm.com IBM z Analytics Traditional Data Integration Inadequate No longer Viable to
More informationIBM Data Virtualization Manager for z/os Leverage data virtualization synergy with API economy to evolve the information architecture on IBM Z
IBM for z/os Leverage data virtualization synergy with API economy to evolve the information architecture on IBM Z IBM z Analytics Agenda Big Data vs. Dark Data Traditional Data Integration Mainframe Data
More informationIBM DATA VIRTUALIZATION MANAGER FOR z/os
IBM DATA VIRTUALIZATION MANAGER FOR z/os Any Data to Any App John Casey Senior Solutions Advisor jcasey@rocketsoftware.com IBM z Analytics A New Era of Digital Business To Remain Competitive You must deliver
More informationData Virtualization for the Enterprise
Data Virtualization for the Enterprise New England Db2 Users Group Meeting Old Sturbridge Village, 1 Old Sturbridge Village Road, Sturbridge, MA 01566, USA September 27, 2018 Milan Babiak Client Technical
More informationIBM Data Virtualization Manager in Detail + Demo Atlanta DB2 User Group Meeting December 7, 2018
IBM Data Virtualization Manager in Detail + Demo Atlanta DB2 User Group Meeting December 7, 2018 Milan Babiak Client Technical Professional, Analytics on Z Systems North America IBM Canada Milan.Babiak@ca.ibm.com
More informationADABAS & NATURAL 2050+
ADABAS & NATURAL 2050+ Guido Falkenberg SVP Global Customer Innovation DIGITAL TRANSFORMATION #WITHOUTCOMPROMISE 2017 Software AG. All rights reserved. ADABAS & NATURAL 2050+ GLOBAL INITIATIVE INNOVATION
More informationIBM DB2 Analytics Accelerator Trends and Directions
March, 2017 IBM DB2 Analytics Accelerator Trends and Directions DB2 Analytics Accelerator for z/os on Cloud Namik Hrle IBM Fellow Peter Bendel IBM STSM Disclaimer IBM s statements regarding its plans,
More informationIBM DB2 Analytics Accelerator
June, 2017 IBM DB2 Analytics Accelerator DB2 Analytics Accelerator for z/os on Cloud for z/os Update Peter Bendel IBM STSM Disclaimer IBM s statements regarding its plans, directions, and intent are subject
More informationKhadija Souissi. Auf z Systems November IBM z Systems Mainframe Event 2016
Khadija Souissi Auf z Systems 07. 08. November 2016 @ IBM z Systems Mainframe Event 2016 Acknowledgements Apache Spark, Spark, Apache, and the Spark logo are trademarks of The Apache Software Foundation.
More informationCONSOLIDATING RISK MANAGEMENT AND REGULATORY COMPLIANCE APPLICATIONS USING A UNIFIED DATA PLATFORM
CONSOLIDATING RISK MANAGEMENT AND REGULATORY COMPLIANCE APPLICATIONS USING A UNIFIED PLATFORM Executive Summary Financial institutions have implemented and continue to implement many disparate applications
More informationAnalytics with IMS and QMF
Analytics with IMS and QMF Steve Mink Worldwide z System Analytics Client Success mink@us.ibm.com March 2015 * IMS Technical Symposium 2015 QMF for z/os 11 QMF for z/os is a visual business intelligence
More informationBest practices for building a Hadoop Data Lake Solution CHARLOTTE HADOOP USER GROUP
Best practices for building a Hadoop Data Lake Solution CHARLOTTE HADOOP USER GROUP 07.29.2015 LANDING STAGING DW Let s start with something basic Is Data Lake a new concept? What is the closest we can
More informationRickard Linck Client Technical Professional Core Database and Lifecycle Management Common Analytic Engine Cloud Data Servers On-Premise Data Servers
Rickard Linck Client Technical Professional Core Database and Lifecycle Management Common Analytic Engine Cloud Data Servers On-Premise Data Servers Watson Data Platform Reference Architecture Business
More informationBigInsights and Cognos Stefan Hubertus, Principal Solution Specialist Cognos Wilfried Hoge, IT Architect Big Data IBM Corporation
BigInsights and Cognos Stefan Hubertus, Principal Solution Specialist Cognos Wilfried Hoge, IT Architect Big Data 2013 IBM Corporation A Big Data architecture evolves from a traditional BI architecture
More information@Pentaho #BigDataWebSeries
Enterprise Data Warehouse Optimization with Hadoop Big Data @Pentaho #BigDataWebSeries Your Hosts Today Dave Henry SVP Enterprise Solutions Davy Nys VP EMEA & APAC 2 Source/copyright: The Human Face of
More informationTHINK DIGITAL RETHINK LEGACY
THINK DIGITAL RETHINK LEGACY Adabas & 2050+ Platform Strategy & Roadmap Bruce Beddoe VP Adabas Systems 1 % BUSINESS & MISSION-CRITICAL 2 For internal use only Billions invested in DIFFERENTIATING business
More informationBig Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara
Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case
More informationzspotlight: Spark on z/os
zspotlight: Spark on z/os Avijit Chatterjee, Ph.D. achatter@us.ibm.com, @ChatterAvijit STSM, IBM Competitive Project Office 1 CEOs are increasingly focused on customers as individuals leveraging contextual
More informationIBM Data Replication for Big Data
IBM Data Replication for Big Data Highlights Stream changes in realtime in Hadoop or Kafka data lakes or hubs Provide agility to data in data warehouses and data lakes Achieve minimum impact on source
More informationModern Data Warehouse The New Approach to Azure BI
Modern Data Warehouse The New Approach to Azure BI History On-Premise SQL Server Big Data Solutions Technical Barriers Modern Analytics Platform On-Premise SQL Server Big Data Solutions Modern Analytics
More informationFast Innovation requires Fast IT
Fast Innovation requires Fast IT Cisco Data Virtualization Puneet Kumar Bhugra Business Solutions Manager 1 Challenge In Data, Big Data & Analytics Siloed, Multiple Sources Business Outcomes Business Opportunity:
More informationCertkiller.P questions
Certkiller.P2140-020.59 questions Number: P2140-020 Passing Score: 800 Time Limit: 120 min File Version: 4.8 http://www.gratisexam.com/ P2140-020 IBM Rational Enterprise Modernization Technical Sales Mastery
More informationThe Evolution of Big Data Platforms and Data Science
IBM Analytics The Evolution of Big Data Platforms and Data Science ECC Conference 2016 Brandon MacKenzie June 13, 2016 2016 IBM Corporation Hello, I m Brandon MacKenzie. I work at IBM. Data Science - Offering
More informationTECHED USER CONFERENCE MAY 3-4, 2016
TECHED USER CONFERENCE MAY 3-4, 2016 Bruce Beaman, Senior Director Adabas and Natural Product Marketing Software AG Software AG s Future Directions for Adabas and Natural WHAT CUSTOMERS ARE TELLING US
More informationSyncsort DMX-h. Simplifying Big Data Integration. Goals of the Modern Data Architecture SOLUTION SHEET
SOLUTION SHEET Syncsort DMX-h Simplifying Big Data Integration Goals of the Modern Data Architecture Data warehouses and mainframes are mainstays of traditional data architectures and still play a vital
More informationData Analytics using MapReduce framework for DB2's Large Scale XML Data Processing
IBM Software Group Data Analytics using MapReduce framework for DB2's Large Scale XML Data Processing George Wang Lead Software Egnineer, DB2 for z/os IBM 2014 IBM Corporation Disclaimer and Trademarks
More informationAccelerating Digital Transformation on Z Using Data Virtualization
Front cover Accelerating Digital Transformation on Z Using Data Virtualization Blanca Borden Calvin Fudge Jen Nelson Jim Porell Redpaper IBM Redbooks Accelerating Digital Transformation on Z Using Data
More informationData 101 Which DB, When Joe Yong Sr. Program Manager Microsoft Corp.
17-18 March, 2018 Beijing Data 101 Which DB, When Joe Yong Sr. Program Manager Microsoft Corp. The world is changing AI increased by 300% in 2017 Data will grow to 44 ZB in 2020 Today, 80% of organizations
More informationBIG DATA COURSE CONTENT
BIG DATA COURSE CONTENT [I] Get Started with Big Data Microsoft Professional Orientation: Big Data Duration: 12 hrs Course Content: Introduction Course Introduction Data Fundamentals Introduction to Data
More informationUnderstanding the latent value in all content
Understanding the latent value in all content John F. Kennedy (JFK) November 22, 1963 INGEST ENRICH EXPLORE Cognitive skills Data in any format, any Azure store Search Annotations Data Cloud Intelligence
More informationData Architectures in Azure for Analytics & Big Data
Data Architectures in for Analytics & Big Data October 20, 2018 Melissa Coates Solution Architect, BlueGranite Microsoft Data Platform MVP Blog: www.sqlchick.com Twitter: @sqlchick Data Architecture A
More informationReliability and Performance with IBM DB2 Analytics Accelerator Version 4.1 IBM Redbooks Solution Guide
Reliability and Performance with IBM DB2 Analytics Accelerator Version 4.1 IBM Redbooks Solution Guide The IBM DB2 Analytics Accelerator for IBM z/os (simply called DB2 Accelerator or just Accelerator
More informationData 101 Which DB, When. Joe Yong Azure SQL Data Warehouse, Program Management Microsoft Corp.
Data 101 Which DB, When Joe Yong (joeyong@microsoft.com) Azure SQL Data Warehouse, Program Management Microsoft Corp. The world is changing AI increased by 300% in 2017 Data will grow to 44 ZB in 2020
More informationAvailability Digest. Attunity Integration Suite December 2010
the Availability Digest Attunity Integration Suite December 2010 Though not focused primarily on high availability in the uptime sense, the Attunity Integration Suite (www.attunity.com) provides extensive
More informationPřehled novinek v SQL Server 2016
Přehled novinek v SQL Server 2016 Martin Rys, BI Competency Leader martin.rys@adastragrp.com https://www.linkedin.com/in/martinrys 20.4.2016 1 BI Competency development 2 Trends, modern data warehousing
More informationStages of Data Processing
Data processing can be understood as the conversion of raw data into a meaningful and desired form. Basically, producing information that can be understood by the end user. So then, the question arises,
More informationComposite Software Data Virtualization The Five Most Popular Uses of Data Virtualization
Composite Software Data Virtualization The Five Most Popular Uses of Data Virtualization Composite Software, Inc. June 2011 TABLE OF CONTENTS INTRODUCTION... 3 DATA FEDERATION... 4 PROBLEM DATA CONSOLIDATION
More informationIBM DB2 Analytics Accelerator: Real-Life Use Cases
Patric Becker IBM BoeblingenLaboratory IBM DB2 Analytics Accelerator: Real-Life Use Cases Legal Disclaimer IBM Corporation 2016. All Rights Reserved. The information contained in this publication is provided
More informationWhat's new and exciting in Tools for DB2 for z/os
DB2 User Group Italia - 2016 What's new and exciting in Tools for DB2 for z/os Elisabetta Curci e_curci@it.ibm.com IBM Analytics on System Z Platform Sales Representative Italy Please Note: IBM s statements
More informationIncrease Value from Big Data with Real-Time Data Integration and Streaming Analytics
Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Cy Erbay Senior Director Striim Executive Summary Striim is Uniquely Qualified to Solve the Challenges of Real-Time
More informationMaking Data Integration Easy For Multiplatform Data Architectures With Diyotta 4.0. WEBINAR MAY 15 th, PM EST 10AM PST
Making Data Integration Easy For Multiplatform Data Architectures With Diyotta 4.0 WEBINAR MAY 15 th, 2018 1PM EST 10AM PST Welcome and Logistics If you have problems with the sound on your computer, switch
More informationIBM IMS Tools Keynote
IBM IMS TECHNICAL SYMPOSIUM 2016 IBM IMS Tools Keynote Janet LeBlanc IMS Tools Offering Manager 2016 IBM Corporation Agenda Our journey where we have been A couple of products you should see this week:
More informationIBM DB2 Analytics Accelerator use cases
IBM DB2 Analytics Accelerator use cases Ciro Puglisi Netezza Europe +41 79 770 5713 cpug@ch.ibm.com 1 Traditional systems landscape Applications OLTP Staging Area ODS EDW Data Marts ETL ETL ETL ETL Historical
More informationBuilding a Data Strategy for a Digital World
Building a Data Strategy for a Digital World Jason Hunter, CTO, APAC Data Challenge: Pushing the Limits of What's Possible The Art of the Possible Multiple Government Agencies Data Hub 100 s of Service
More informationMicrosoft Azure Databricks for data engineering. Building production data pipelines with Apache Spark in the cloud
Microsoft Azure Databricks for data engineering Building production data pipelines with Apache Spark in the cloud Azure Databricks As companies continue to set their sights on making data-driven decisions
More informationIBM Db2 Event Store Simplifying and Accelerating Storage and Analysis of Fast Data. IBM Db2 Event Store
IBM Db2 Event Store Simplifying and Accelerating Storage and Analysis of Fast Data IBM Db2 Event Store Disclaimer The information contained in this presentation is provided for informational purposes only.
More informationVOLTDB + HP VERTICA. page
VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics
More informationDB2 for z/os Tools Overview & Strategy
Information Management for System z DB2 for z/os Tools Overview & Strategy Haakon Roberts DE, DB2 for z/os & Tools Development haakon@us.ibm.com 1 Disclaimer Information regarding potential future products
More informationBig data easily, efficiently, affordably. UniConnect 2.1
Connecting Data. Delivering Intelligence Big data easily, efficiently, affordably UniConnect 2.1 The UniConnect platform is designed to unify data in a highly scalable and seamless manner, by building
More informationThe Never Ending Value of z Systems Focus on Analytics & Big Data
The Never Ending Value of z Systems Focus on Analytics & Big Data Hélène Lyon Distinguished Engineer & CTO, Analytics on z Systems for Europe Europe IMS SWAT Technical Executive IBM Systems, zsoftware
More informationOracle Big Data Discovery
Oracle Big Data Discovery Turning Data into Business Value Harald Erb Oracle Business Analytics & Big Data 1 Safe Harbor Statement The following is intended to outline our general product direction. It
More informationOptimizing Data Transformation with Db2 for z/os and Db2 Analytics Accelerator
Optimizing Data Transformation with Db2 for z/os and Db2 Analytics Accelerator Maryela Weihrauch, IBM Distinguished Engineer, WW Analytics on System z March, 2017 Please note IBM s statements regarding
More informationOracle Big Data Connectors
Oracle Big Data Connectors Oracle Big Data Connectors is a software suite that integrates processing in Apache Hadoop distributions with operations in Oracle Database. It enables the use of Hadoop to process
More informationDQpowersuite. Superior Architecture. A Complete Data Integration Package
DQpowersuite Superior Architecture Since its first release in 1995, DQpowersuite has made it easy to access and join distributed enterprise data. DQpowersuite provides an easy-toimplement architecture
More informationVirtuoso Infotech Pvt. Ltd.
Virtuoso Infotech Pvt. Ltd. About Virtuoso Infotech Fastest growing IT firm; Offers the flexibility of a small firm and robustness of over 30 years experience collectively within the leadership team Technology
More informationActivator Library. Focus on maximizing the value of your data, gain business insights, increase your team s productivity, and achieve success.
Focus on maximizing the value of your data, gain business insights, increase your team s productivity, and achieve success. ACTIVATORS Designed to give your team assistance when you need it most without
More informationDrawing the Big Picture
Drawing the Big Picture Multi-Platform Data Architectures, Queries, and Analytics Philip Russom TDWI Research Director for Data Management August 26, 2015 Sponsor 2 Speakers Philip Russom TDWI Research
More informationFrom Single Purpose to Multi Purpose Data Lakes. Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019
From Single Purpose to Multi Purpose Data Lakes Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019 Agenda Data Lakes Multiple Purpose Data Lakes Customer Example Demo Takeaways
More informationAnalytics on z Systems, what s new?
Khadija Souissi Analytics on z Systems, what s new? IBM Architektentage 16.11.2016 Agenda Spark on z Systems What is Spark Spark Details The ecosystem for Spark on z/os Use Cases New Dimension for the
More informationApplying Analytics to IMS Data Helps Achieve Competitive Advantage
Front cover Applying Analytics to IMS Data Helps Achieve Competitive Advantage Kyle Charlet Deepak Kohli Point-of-View The challenge to performing analytics on enterprise data Highlights Business intelligence
More informationThe age of Big Data Big Data for Oracle Database Professionals
The age of Big Data Big Data for Oracle Database Professionals Oracle OpenWorld 2017 #OOW17 SessionID: SUN5698 Tom S. Reddy tom.reddy@datareddy.com About the Speaker COLLABORATE & OpenWorld Speaker IOUG
More informationData-Intensive Distributed Computing
Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 5: Analyzing Relational Data (1/3) February 8, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
More informationEvolving To The Big Data Warehouse
Evolving To The Big Data Warehouse Kevin Lancaster 1 Copyright Director, 2012, Oracle and/or its Engineered affiliates. All rights Insert Systems, Information Protection Policy Oracle Classification from
More informationNetezza The Analytics Appliance
Software 2011 Netezza The Analytics Appliance Michael Eden Information Management Brand Executive Central & Eastern Europe Vilnius 18 October 2011 Information Management 2011IBM Corporation Thought for
More informationIntroduction to Federation Server
Introduction to Federation Server Alex Lee IBM Information Integration Solutions Manager of Technical Presales Asia Pacific 2006 IBM Corporation WebSphere Federation Server Federation overview Tooling
More informationTop Five Reasons for Data Warehouse Modernization Philip Russom
Top Five Reasons for Data Warehouse Modernization Philip Russom TDWI Research Director for Data Management May 28, 2014 Sponsor Speakers Philip Russom TDWI Research Director, Data Management Steve Sarsfield
More informationIBM C IBM z Systems Technical Support V6.
IBM C9030-634 IBM z Systems Technical Support V6 http://killexams.com/exam-detail/c9030-634 QUESTION: 66 The value of the MobileFirst Platform is that it: A. Provides a platform to build, test, run and
More informationDesigning a Modern Data Warehouse + Data Lake
Designing a Modern Warehouse + Lake Strategies & architecture options for implementing a modern data warehousing environment Melissa Coates Analytics Architect, SentryOne Blog: sqlchick.com Twitter: @sqlchick
More informationAyush Ganeriwal Senior Principal Product Manager, Oracle. Benjamin Perez-Goytia Principal Solution Architect A-Team, Oracle
Oracle Data Integration Platform A Cornerstone for Big Data Ayush Ganeriwal Senior Principal Product Manager, Oracle Benjamin Perez-Goytia Principal Solution Architect A-Team, Oracle Pencho Tzonev Head
More informationIBM Replication Updates: 4+ in 45 The Fillmore Group February A Premier IBM Business Partner
IBM Replication Updates: 4+ in 45 The Fillmore Group February 2019 A Premier IBM Business Partner Poll 1: Is your organization currently using replication in a production environment? 2 Agenda Replication
More informationAbstract. The Challenges. ESG Lab Review InterSystems IRIS Data Platform: A Unified, Efficient Data Platform for Fast Business Insight
ESG Lab Review InterSystems Data Platform: A Unified, Efficient Data Platform for Fast Business Insight Date: April 218 Author: Kerry Dolan, Senior IT Validation Analyst Abstract Enterprise Strategy Group
More informationAn InterSystems Guide to the Data Galaxy. Benjamin De Boe Product Manager
An InterSystems Guide to the Data Galaxy Benjamin De Boe Product Manager Analytics 3 InterSystems Corporation. All rights reserved. 4 InterSystems Corporation. All rights reserved. 5 InterSystems Corporation.
More informationLambda Architecture for Batch and Stream Processing. October 2018
Lambda Architecture for Batch and Stream Processing October 2018 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided for informational purposes only.
More information2014 年 3 月 13 日星期四. From Big Data to Big Value Infrastructure Needs and Huawei Best Practice
2014 年 3 月 13 日星期四 From Big Data to Big Value Infrastructure Needs and Huawei Best Practice Data-driven insight Making better, more informed decisions, faster Raw Data Capture Store Process Insight 1 Data
More informationIBM dashdb Local. Using a software-defined environment in a private cloud to enable hybrid data warehousing. Evolving the data warehouse
IBM dashdb Local Using a software-defined environment in a private cloud to enable hybrid data warehousing Evolving the data warehouse Managing a large-scale, on-premises data warehouse environments to
More informationEMC Documentum xdb. High-performance native XML database optimized for storing and querying large volumes of XML content
DATA SHEET EMC Documentum xdb High-performance native XML database optimized for storing and querying large volumes of XML content The Big Picture Ideal for content-oriented applications like dynamic publishing
More informationHow to Modernize the IMS Queries Landscape with IDAA
How to Modernize the IMS Queries Landscape with IDAA Session C12 Deepak Kohli IBM Senior Software Engineer deepakk@us.ibm.com * IMS Technical Symposium Acknowledgements and Disclaimers Availability. References
More informationHOW TO ACHIEVE REAL-TIME ANALYTICS ON A DATA LAKE USING GPUS. Mark Brooks - Principal System Kinetica May 09, 2017
HOW TO ACHIEVE REAL-TIME ANALYTICS ON A DATA LAKE USING GPUS Mark Brooks - Principal System Engineer @ Kinetica May 09, 2017 The Challenge: How to maintain analytic performance while dealing with: Larger
More informationCloud Analytics and Business Intelligence on AWS
Cloud Analytics and Business Intelligence on AWS Enterprise Applications Virtual Desktops Sharing & Collaboration Platform Services Analytics Hadoop Real-time Streaming Data Machine Learning Data Warehouse
More informationPrices in Japan (Yen) Oracle Technology Global Price List December 8, 2017
Oracle Technology Global Price List December 8, 2017 This document is the property of Oracle Corporation. Any reproduction of this document in part or in whole is strictly prohibited. For educational purposes
More informationOracle Big Data SQL. Release 3.2. Rich SQL Processing on All Data
Oracle Big Data SQL Release 3.2 The unprecedented explosion in data that can be made useful to enterprises from the Internet of Things, to the social streams of global customer bases has created a tremendous
More informationCapturing Your Changed Data
Capturing Your Changed Data with the CONNX Data Synchronization Tool Table of Contents Executive Summary 1 Fulfilling a Need with Minimal Investment 2 Departmental Reporting Servers 3 Data Migration 4
More informationComparison of SmartData Fabric with Cloudera and Hortonworks Revision 2.1
Comparison of SmartData Fabric with Cloudera and Hortonworks Revision 2.1 Page 1 of 11 www.whamtech.com (972) 991-5700 info@whamtech.com August 2018 Page 2 of 11 www.whamtech.com (972) 991-5700 info@whamtech.com
More informationDb2 for z/os Gets Agile
New England Db2 Users Group September 28, 2017 Db2 for z/os Gets Agile Robert Catterall IBM Senior Consulting Db2 for z/os Specialist 2017 IBM Corporation Agenda The distinction between data-as-a-service
More informationDATACENTER SERVICES DATACENTER
SERVICES SOLUTION SUMMARY ALL CHANGE React, grow and innovate faster with Computacenter s agile infrastructure services Customers expect an always-on, superfast response. Businesses need to release new
More informationData Warehousing on System z What is available & How to implement
Data Warehousing on System z What is available & How to implement Expanding System z s Role in Data Warehouse & Business Intelligence Implementations 2008 IBM Corporation IBM Systems Agenda Agenda Market
More informationOracle GoldenGate for Big Data
Oracle GoldenGate for Big Data The Oracle GoldenGate for Big Data 12c product streams transactional data into big data systems in real time, without impacting the performance of source systems. It streamlines
More informationRealizing the Full Potential of MDM 1
Realizing the Full Potential of MDM SOLUTION MDM Augmented with Data Virtualization INDUSTRY Applicable to all Industries EBSITE www.denodo.com PRODUCT OVERVIE The Denodo Platform offers the broadest access
More informationDatabase Management Systems
Database Management Systems Trends and Directions Namik Hrle IBM Fellow CTO Private Cloud and z Analytics March 2017 Please note: IBM s statements regarding its plans, directions, and intent are subject
More informationData Analytics at Logitech Snowflake + Tableau = #Winning
Welcome # T C 1 8 Data Analytics at Logitech Snowflake + Tableau = #Winning Avinash Deshpande I am a futurist, scientist, engineer, designer, data evangelist at heart Find me at Avinash Deshpande Chief
More informationIOTA ARCHITECTURE: DATA VIRTUALIZATION AND PROCESSING MEDIUM DR. KONSTANTIN BOUDNIK DR. ALEXANDRE BOUDNIK
IOTA ARCHITECTURE: DATA VIRTUALIZATION AND PROCESSING MEDIUM DR. KONSTANTIN BOUDNIK DR. ALEXANDRE BOUDNIK DR. KONSTANTIN BOUDNIK DR.KONSTANTIN BOUDNIK EPAM SYSTEMS CHIEF TECHNOLOGIST BIGDATA, OPEN SOURCE
More informationHow Insurers are Realising the Promise of Big Data
How Insurers are Realising the Promise of Big Data Jason Hunter, CTO Asia-Pacific, MarkLogic A Big Data Challenge: Pushing the Limits of What's Possible The Art of the Possible Multiple Government Agencies
More information1. Which programming language is used in approximately 80 percent of legacy mainframe applications?
Volume: 59 Questions 1. Which programming language is used in approximately 80 percent of legacy mainframe applications? A. Visual Basic B. C/C++ C. COBOL D. Java Answer: C 2. An enterprise customer's
More informationMetaMatrix Enterprise Data Services Platform
MetaMatrix Enterprise Data Services Platform MetaMatrix Overview Agenda Background What it does Where it fits How it works Demo Q/A 2 Product Review: Problem Data Challenges Difficult to implement new
More informationQMF Analytics v11: Not Your Green Screen QMF
QMF Analytics v11: Not Your Green Screen QMF Central Ohio Db2 Users Group CODUG December 5, 2017 Roger Midgette The Fillmore Group Frank Fillmore The Fillmore Group Doug Anderson Rocket Software roger.midgette@thefillmoregroup.com
More informationBIG DATA ANALYTICS A PRACTICAL GUIDE
BIG DATA ANALYTICS A PRACTICAL GUIDE STEP 1: GETTING YOUR DATA PLATFORM IN ORDER Big Data Analytics A Practical Guide / Step 1: Getting your Data Platform in Order 1 INTRODUCTION Everybody keeps extolling
More informationImproving Your Business with Oracle Data Integration See How Oracle Enterprise Metadata Management Can Help You
Improving Your Business with Oracle Data Integration See How Oracle Enterprise Metadata Management Can Help You Özgür Yiğit Oracle Data Integration, Senior Manager, ECEMEA Safe Harbor Statement The following
More informationREGULATORY REPORTING FOR FINANCIAL SERVICES
REGULATORY REPORTING FOR FINANCIAL SERVICES Gordon Hughes, Global Sales Director, Intel Corporation Sinan Baskan, Solutions Director, Financial Services, MarkLogic Corporation Many regulators and regulations
More informationMAPR DATA GOVERNANCE WITHOUT COMPROMISE
MAPR TECHNOLOGIES, INC. WHITE PAPER JANUARY 2018 MAPR DATA GOVERNANCE TABLE OF CONTENTS EXECUTIVE SUMMARY 3 BACKGROUND 4 MAPR DATA GOVERNANCE 5 CONCLUSION 7 EXECUTIVE SUMMARY The MapR DataOps Governance
More informationData Warehousing in the Age of In-Memory Computing and Real-Time Analytics. Erich Schneider, Daniel Rutschmann June 2014
Data Warehousing in the Age of In-Memory Computing and Real-Time Analytics Erich Schneider, Daniel Rutschmann June 2014 Disclaimer This presentation outlines our general product direction and should not
More information