Combine Native SQL Flexibility with SAP HANA Platform Performance and Tools

Transcription

1 SAP Technical Brief Data Warehousing SAP HANA Data Warehousing Combine Native SQL Flexibility with SAP HANA Platform Performance and Tools

2 A data warehouse for the modern age Data warehouses have been a mainstay in enterprises for more than 20 years, and they still play a critical role in business today. In the age of digital transformation, businesses are flooded with unstructured data from the Internet of Things (IoT) and social networks, as well as more traditional structured data from ERP, HR, and inventory systems, customers, and other sources. By 2020, databases, analytics, and IoT workloads will account for 22% of total business workloads. 1 Organizations still need a single source of truth for data management, processing, and modeling. Modern data warehouses fill that role by helping to consolidate data into a comprehensive pool of current and historical information that supports planning and forecasting through analytics and visualizations. Recently, the way data warehouses are used has shifted. In the past, the focus was purely on volume. Today, it s on complexity and variety of data, along with the need for real-time analysis. The SAP HANA modern data platform provides an effective and powerful combination for storing and modeling modern data flows in real time. Customers want and need the Universal Data Element Framework (UDEF) to support rapid application development for digital innovation, alongside maintenance for existing projects. But what if your business requires native SQL functionality and tools? What if you want to make better use of your existing SQL development expertise? 2 / 12

3 Introducing SAP HANA Data Warehousing SAP HANA Data Warehousing lets you combine your SQL environment, tools, and expertise with the capabilities provided by the SAP HANA platform, including: High performance Multi-petabyte scalability Support for high end-user counts Data management and tools In a modern data warehouse environment, data modeling needs to be able to support diverse data types and data semantics from schema-less sources, and then combine that data with structured or natively relational data. SAP HANA Data Warehousing gives you the freedom to build custom data-management processes and employ different data modeling approaches, such as a virtual data modeling (VDM) or data vault, instead of a standard star schema. In addition, the solution lets you work with tools that support native models built directly on the database, including best-of-breed tools from SAP or third-party SQL tools. That way, you have the flexibility to replace individual tools instead of an entire software stack in response to technology changes. In addition, the modeling for SAP HANA Data Warehousing applies to both on-premise and cloud-based solutions, which is a strong difference from other technologies. The solution provides multi-temperature data tiering at several layers with automatic data movement. SAP also provides SQL integration with the data tiering layers, including SAP IQ, Apache Hadoop, and Apache Spark SQL through SAP Vora, in order to optimize analytical performance and scalability for Big Data up to the multi-petabyte range. This document focuses on an SAP HANA Data Warehousing solution based on the SAP HANA platform, which can be used independently or in a hybrid approach with SAP Business Warehouse. For information on using SAP Business Warehouse or SAP BW/4HANA Advanced Business Application Programming (ABAP) implementations, see: business-warehouse.html bw4hana-data-warehousing.html 3 / 12

4 The SAP HANA platform hosts key services and provides an in-memory database, extensive datamanagement capabilities, Hadoop integration through the SAP Vora engine, and integrated data stores through dynamic tiering. Specifically, the SAP HANA platform offers the ability to: Build next-generation applications Analyze large quantities of data from virtually any source in real time Gain insights from both structured and unstructured data Reduce or eliminate data aggregation, indexing, mapping, and extract-transfer-load (ETL) functions typically needed in complex data warehouses and data marts Scale your data warehouse to petabytes of data Improve query performance by making use of built-in aggregates Column-store technology accelerates processing Maximize the query efficiency of raw source data by using high-performance column-store technology, which delivers high-speed compression and ad-hoc analysis without complex tuning. Column-store technology accelerates analytics because the query engine accesses only the columns that contain relevant data. In addition, the columnar structure allows data to be processed in parallel, with separate CPU cores using hardware acceleration. The columnar architecture of the SAP HANA platform also allows you to modify the database schema without interrupting currently running applications. 4 / 12

5 SAP HANA Data Warehousing Foundation includes a data lifecycle manager (DLM), which enables automatic data movement for SAP HANA dynamic tiering, SAP IQ, and Hadoop with SAP Vora based on rules and data aging, table hierarchies, and storage destinations. The modern data platform (MDP) also includes features for Data Distribution Optimizer (DDO) and for Native DataStore Objects (NDSO). SAP HANA dynamic tiering allows less frequently used (warm) data to be moved from the main (hot) in-memory SAP HANA database into extended storage. For example, you can offload the storage and processing of aged data to a less-expensive dynamic tiering disk store, while keeping critical business data in memory for best performance. Warm data on disk can still be queried, updated, and combined with hot data in SAP HANA memory, all with transactional consistency and with pruning and filtering pushed down to the next data tier. After you determine how to divide data based on temperature, you can either build your own SQL commands to move data from SAP HANA memory to dynamic tiering, or you can automate the task by using the SAP DLM tool. SAP DLM part of the SAP HANA Data Warehousing Foundation provides a graphical interface for creating rules to move less frequently used data to alternate storage tiers, such as dynamic tiering, Hadoop, or SAP IQ. 5 / 12

6 GENERATED SAP HANA VIEW (PRUNING/UNION) SAP HANA Node (Hot Store) Dynamic Tiering (Warm Store) DATA MOVEMENT DATA LIFECYCLE MANAGER SAP HANA Extension Node (Warm Store) Figure 1. SAP HANA data lifecycle manager lets you move aged data to alternate storage tiers Multi-Petabyte Scale (Cold Store) 6 / 12

7 The MapR file system (MapR-XD) provides significant scale-out capacity on enterprise hardware for a database cluster, along with substantial cost savings over traditional networkattached storage (NAS) or storage-area-network (SAN) systems especially for large SAP IQ data volumes in SAP HANA Data Warehousing. When you use a multiplex grid based on the SAP IQ software running on top of a MapR data platform to store your data files, you achieve near-linear scalability in storage input/output throughput as more server nodes are added for SAP IQ. The result is high performance with a lower total cost of ownership compared to typical file systems. In addition to the reduced total cost of ownership (TCO) offered by MapR-XD, you can take advantage of the significant data compression offered by SAP IQ. For example, one SAP customer was able to compress 24 petabytes of raw data down to about 2 petabytes by running SAP IQ on its MapR-XD implementation. The MapR file system also adds disaster recovery capabilities to the SAP IQ landscape. MapR-XD is supported at the following data volumes for each data tier: Data Tier SAP HANA Dynamic Tiering SAP IQ Apache Hadoop with SAP Vora Data Volume 1 60 TB compressed data 100 TB 2 PB compressed data Unlimited compressed data 7 / 12

8 The SAP HANA modern data platform provides advanced in-memory analytical processing to help you build intelligent applications that provide deeper insights. SAP HANA in-database predictive analytics integrates with Hadoop, R, and SAS solutions, so you can analyze Big Data to predict business outcomes and make faster, smarter decisions across your business. For example, you can use the SAP HANA Predictive Analysis Library (PAL) to perform data mining and statistical calculations on large data sets within real-time applications. You can also use the SAP HANA platform to process R code as part of an overall query for statistical computing and graphics. The SAP HANA platform also lets you process massive amounts of data from a wide variety of sources and data types by supporting: Spatial analysis, provided by a multilayered engine that supports spatial columns, access methods, and reference systems Text analysis, text mining, and search to discover new insights from unstructured documents Series-data processing, to efficiently process large volumes of series data in conjunction with business data Event-stream processing, to capture, analyze, and act on real-time streaming information Graph-data processing, to model and analyze complex, connected data using a familiar relational database, intuitive modeling and visualization tools, and built-in graph algorithms 8 / 12

9 and development tools The SAP HANA platform lets you build complex business logic using graphical modeling tools. Graphical calculation tools provide built-in union, join, projection, and aggregation nodes, in addition to other features like distinct, count, calculation, and dynamic joins. You can even perform operations without any SQL or SQL script knowledge, or you can use SQL scripts to create script-based calculation views. SAP Enterprise Architecture Designer (EA Designer) SAP EA Designer is the next generation, web-based SAP HANA extension of SAP PowerDesigner. The HTML5-based software lets you build a blueprint of your current enterprise architecture, and then use impact analysis and data-lineage features to see the impact of changes before implementation. The software provides visualizations and dashboards with a drag-and-drop interface that eliminates the need to drill down directly to detailed database structures. The SAP EA Designer software stores all models and metadata in a GitHub repository or a variety of relational databases. SAP Web IDE SAP Web IDE for the SAP HANA platform is a web-based integrated development tool for technical users who work closely with the database, performing detailed administration or creation of apps. The tool lets you build, debug, test, and deploy apps using wizards, templates, code, or graphical editors. Like SAP EA Designer, SAP Web IDE stores all models and metadata in a GitHub repository. 9 / 12

10 The SAP HANA platform provides a rich set of tools to help you manage your entire SAP HANA workflow, from model and app development, to versioning, impact analysis, design, implementation, scheduling, and monitoring of data and processes. 7 App Lifecycle Management 8 Job Scheduling 6 Modeling 2 1 Modeling 1 SAP HANA Platform WITH SAP HANA Extended Application Services SAP HANA Data Warehousing Foundation GitHub Integration 5 Design and Implement Data Flows 2 Impact Analysis 1 4 Impact Analysis 2 3 Versioning 1. Modeling 1: Create conceptual, logical, physical, business process, and enterprise architecture models for data warehousing using SAP PowerDesigner software and SAP EA Designer. 2. Impact analysis 1: Determine how your models affect your data structures and other models. 3. Versioning: Keep an inventory of all your models and other artifacts. 4. Impact analysis 2: Examine and edit your models, as needed, on the SQL level, using SAP Web IDE for the SAP HANA platform. 5. Design and implement data flows: The smart data-integration services enable access to SAP data and other data through ETL mechanisms or through streaming, and they let you move data between different tiers. 6. Modeling 2: Improve models, as needed, using SAP Web IDE for the SAP HANA platform. 7. App lifecycle management: SAP HANA application lifecycle management keeps track of artifacts and keeps multiple instances of the SAP HANA database in sync. 8. Job scheduling: Schedule data acquisition, update processes, and monitor data quality with built-in data-lineage processes. 10 / 12

11 With SAP, you can select a data warehouse approach that meets your company s needs today and into the future: SAP BW/4HANA is the best option for businesses that prefer a pre-built data warehousing solution that will predominantly interact with SAP software systems. SAP HANA Data Warehousing provides powerful tools integrated with the SAP HANA platform. It is a great fit for organizations looking to maximize their existing or third-party SQL tools and expertise and aiming to create custom models and transformations with SQL. Because SAP BW is integrated with and optimized for the SAP HANA platform, you can even choose a mixed approach that combines the benefits of both SAP BW and SAP HANA Data Warehousing. This approach lets you make use of your familiarity with SAP BW and SAP data semantics, while also making use of the agility and high degree of freedom associated with the SQL approach. Pre-built data warehouse approach SAP BW/4HANA SAP HANA platform Mixed approach SAP HANA Data Warehousing approach SAP tools Third-party SQL tools SAP HANA platform Regardless of the approach you use, you still receive all the benefits and features of the SAP HANA platform, including multi-tiering, data aging, query distribution, code pushdown, and deep integration with Big-Data solutions such as dynamic tiering, SAP IQ, Hadoop, and SAP Vora. 11 / 12

12 Summary SAP HANA Data Warehousing offers the freedom to build custom data-management processes while working with tools that support native models built directly on the database, including best-of-breed tools from SAP and third-party SQL tools. The solution offers the performance, modeling, provided by the SAP HANA platform, with the flexibility to run on premise or in the cloud. Provide a single source of truth for data management, processing, and modeling Support native SQL functionality and tools Optimize the use of existing SQL development expertise Support for SAP and third-party tools that support native models built directly on the database Multi-temperature data tiering at several layers with automatic data movement SQL integration with data tiering layers optimizes performance and scalability up to petabytes Advanced analytical modeling and processing options Reduced total cost of ownership through support for lower-cost MapR-XD data storage Graphical modeling tools for building complex business logic Workflow-management tools from modeling, versioning, analysis, and design to implementation Learn more To find out more, call your SAP representative today or visit us online at 12 / 12

13 1 Cisco Mobile Visual Networking Index (VNI) Forecast. VNI Global Fixed and Mobile Internet Traffic Forecasts. visual-networking-index-vni/index.html. (17/07) No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE s or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they should not be relied upon in making purchasing decisions. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies. See for additional trademark information and notices.