Lower Maintenance Costs and Sustain System Performance with the SAP NetWeaver BI Data Aging Toolkit. Bryan Katis SAP Labs

Similar documents
Hardware Requirements

The test has been performed using a 64 Bit SAP NetWeaver Application Server Java 7.1 Enhancement Package 1 SP4 or greater system.

SAP Learning Solution RKT ERP 2005 LSO 6.00

Do Exception Broadcasting

Extract Archived data from R3

Configure TREX 6.1 for Efficient Indexing. Document Version 1.00 January Applicable Releases: SAP NetWeaver 04

SAP NetWeaver MDM MDM Import and Syndication Server & Port Concept

Visual Composer - Task Management Application

Install TREX for CAF Version 1.00 March 2006

configure an anonymous access to KM

How to Upgr a d e We b Dynpro Them e s from SP S 9 to SP S 1 0

Create Partitions in SSAS of BPC Version 1.00 Feb 2009

How To Extend User Details

TABLE DISTRIBUTION IN HANA HANA. SAP Active Global Support, June 2012

SAP GRC Access Control: Configuring compliant user provisioning (formerly Virsa Access Enforcer) into CUA Systems

MDM Syndicator Create Flat Syndication File

Integrate a Forum into a Collaboration Room

MODULE 2: CREATE A DECISION TABLE USING RULES COMPOSER (BRM)

What s New / Release Notes SAP Strategy Management 10.1

Create Monitor Entries from a Transformation routine

Create Monitor Entries from an update routine

How To Configure IDoc Adapters

SAP NetWeaver How-To Guide

How To Configure the Websocket Integration with SAP PCo in SAP MII Self Service Composition Environment Tool

A Step-By-Step Guide on File to File Scenario Using Xslt Mapping

link SAP BPC Excel from an enterprise portal Version th of March 2009

Configure UD Connect on the J2EE Server for JDBC Access to External Databases

How To... Configure Integrated Configurations in the Advanced Adapter Engine

Process Control 2.5 Implementation Checklist

Send Multiple IDocs Within One XI Message

How To Set up NWDI for Creating Handheld Applications in SAP NetWeaver Mobile 7.1

Web Page Composer anonymous user access

How To Develop a Simple Web Service Application Using SAP NetWeaver Developer Studio & SAP XI 3.0

Quick Reference Guide SAP GRC Access Control Compliant User Provisioning (formerly Virsa Access Enforcer): HR Triggers

SAP NetWeaver Process Integration 7.1

Cache Settings in Web Page Composer

How to Create Your Very Own Master Data Object Types. Richard LeBlanc SAP Labs Canada

How to Set Up and Use the SAP OEE Custom UI Feature

Visual Composer Build Process

Optimize the RFC performance for mysap CRM Mobile Sales Version 1.00 April Applicable Releases: SAP CRM Mobile Sales and Service 4.0 and 5.

Business Objects Integration Scenario 2

Line Items in BI Integrated Planning

Introducing SAP Enterprise Services Explorer for Microsoft.NET

Enhancement Framework

How To... Promote Reports and Input Schedules Through Your System Landscape

Multiple Components in One Database

Configure SSO in an SAP NetWeaver 2004s Dual Stack

How to Use Function Keys in Mobile Applications for Handhelds

Work with Variables in SAP NetWeaver Visual Composer Version 1.00 May 2006

How to Set Up and Use the SAP OEE Custom KPI Andons Feature

Transport in GP. How-to Guide Beginning with SAP NetWeaver 2004s SPS06. Version 2.00 January 2006

SAP MII: Leveraging the Data Buffering Feature for Connection Error Handling.

How To... Master Data Governance for Material: File Down- and Upload

Consuming Web Dynpro components in Visual Composer.

Installation Guide Business Explorer

How To... Master Data Governance for Material: BADI USMD_SSW_SYSTEM_METHOD_CALLER to create successor change request

SAP Plant Connectivity 2.2

How to Create a New SAPUI5 Development Component

Data Validation in Visual Composer for SAP NetWeaver Composition Environment

How To Troubleshoot SSL with BPC Version 1.01 May 2009

WDA - Custom themes for Web Dynpro ABAP applications without SAP Enterprise Portal integration

Building a Tax Calculation Application

How To...Use a Debugging Script to Easily Create a Test Environment for a SQL-Script Planning Function in PAK

View Time Security for crystalreports.com

How To... Reuse Business Objects and Override Operations of a Business Object

Setup an NWDI Track for Composition Environment Developments

How To... Master Data Governance for Material: BADI USMD_SSW_PARA_RESULT_HANDLER to merge result of parallel workflow tasks

Sizing for Guided Procedures, SAP NetWeaver 7.0

Best Practices Using KMC Capabilities in an External Facing Portal Version 1.00 October 2006

BW310. BW - Enterprise Data Warehousing COURSE OUTLINE. Course Version: 10 Course Duration: 5 Day(s)

How To Use Surveys with Mobile Sales for handheld Version 1.00 January 2007

How To...Configure Integration of CUP with SPM

Monitoring System Landscapes Using the DBA Cockpit

What's New in the DBA Cockpit with SAP NetWeaver 7.0

Preview of Web Services Reliable Messaging in SAP NetWeaver Process Integration 7.1

How to receive and convert PDF-documents with SAP XI

Building Custom Applications Using IBOM

Backup & Restore with SAP BPC (MS SQL 2000)

How to Translate a Visual Composer Model Part I

Access Control 5.3 Implementation Considerations for Superuser Privilege Management ID-Based Firefighting versus Role-Based Firefighting Applies to:

and Adapt ERP Roles and Their Content to SAP Enterprise Portal

How-to Guide SAP EPM. How To Use Comments. Version 1.00 March Applicable Releases: EPM BPC 5.1 Microsoft & EPM BPC 7 Microsoft

SAP NetWeaver How-To Guide How to use Process Execution Manager Using SAP Test Data Migration Server

How to View Dashboards in the Self Service Composition Environment with Additional Metadata

TBW30 SAP BW Modeling & Implementation

SAP BW 3.3 April 2004 English. General Ledger Analysis. Business Process Procedure. SAP AG Neurottstr Walldorf Germany

Enterprise Portal Logon Page Branding

SAP NetWeaver How-To Guide

R e l ea se 6. 20/

Reconcile Data Between SAP Source Systems and SAP NetWeaver BI

SAP NetWeaver 04. Unification Terminology

BusinessObjects Enterprise XI 3.0 with Complex NAT Networks

SAP Composite Application Framework. Creating a Content Package Object

How To Build the Carry Forward and Account Transformation Business Rules

Automatic Deletion of Similar/Identical Requests from InfoCube after Update

Visual Composer for NetWeaver CE: Getting Started with a Typical Workflow

How To Recover Login Module Stack when login to NWA or Visual Administrator is impossible

Handle. How-to Guide SAP NetWeaver 2004s. Version 1.00 Sept 2006

TBW60. BW: Operations and Performance COURSE OUTLINE. Course Version: 10 Course Duration: 5 Day(s)

How To Generate XSD Schemas from Existing MDM Repositories

Transcription:

Lower Maintenance Costs and Sustain System Performance with the SAP NetWeaver BI Data Aging Toolkit Bryan Katis SAP Labs

Learning Objectives As a result of this session, you will be able to: Explain data aging and Information Lifecycle Management (ILM) Understand how SAP NetWeaver BI supports ILM Know the most important aspects of implementing a data aging strategy SAP AG 2006, 2

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

What is Data Aging? Data warehousing is a very powerful concept for creating a unified and consistent view of the business In a data warehousing environment, it is typical that: Data is amassed and analyzed at an increasing rate As time progresses, companies face the dilemma of storing more and more historical data Over time, data tends to lose its day-to-day relevance and is therefore accessed less frequently The costs associated with maintaining historical data are high Data aging is a strategy for managing data over time, balancing data access requirements with TCO Each data aging strategy is uniquely determined by the customer s data and the business value of accessing the data The SAP BI solution provides alternatives for the typical cost vs. business data availability conundrum SAP AG 2006, 5

Challenges Facing the Information Lifecycle Data growth Emails, attachments, Web sites, audio and video content, voice recordings Constantly increasing data volumes in BI Direct access capabilities Predictable residence time for ERP data Long-term direct accessibility appreciated, ad-hoc analysis needs Legal requirements SEC, FDA, HIPPA, SOA, GDPdU, Basel II for ERP data Data value during lifecycle Constantly decreasing in ERP environments More long lasting in BI environments Costs Personal costs, technology costs, process costs Technological innovations ATA disks, blue laser, etc. Write-once file system, NLS, etc. The challenges cannot only be addressed by purchase of additional memory. An SAP effective AG 2006, 6 administration of the data is necessary.

Information Lifecycle Management (ILM) Everyone s talking about ILM ILM is not only a product ILM is not only software ILM is not only a tool ILM is a storage management concept that actively manages all information objects during their entire lifecycle ILM is a combination of processes and technologies with a goal of providing the right information at the right time in the right place with the lowest possible costs SAP AG 2006, 7

Archiving Requirements of the Various Layers in a BI Environment BI Data Mining Business Planning Persistent BI analysis horizon High-volume data sets Direct availability required Data archiving process ERP Strategic Analysis Operational Reporting EDW Layer Extracts Persistent BI inbound layer High-volume, highly granular data Direct availability not required Data archiving processes Description of extraction formats DART (Data Retention Tool) Service API for BI Extraction - Aggregation + + Granularity - Relationships Entities Classic archiving Archiving objects Legal storage regulations Cost reduction SAP AG 2006, 8

Information Lifecycle Management at SAP One View Analyze business objects, determine data categories and layers, define attributes, identify parameters Automation, archiving, migration, deletion, accessibility Realize Analyze Categorize ILM Define Policy Determine policies for attributes and categories such as retention period, legal compliance, data extracts Apply Policy Apply policies to business objects and data categories SAP AG 2006, 9

Data Aging: The Corporation Information Factory Departmental Data Marts Acctg Finance Marketing ETL Sales SAP BW Powered by SAP NetWeaver DSS Applications ERP ERP ERP CRM Changed Data Staging Area EDW ecomm. Bus. Int. ERP Corporate Applications Local ODS Global ODS Oper. Mart Granularity Manager Cross media Storage mgr Exploration warehouse/ data mining Session Analysis Near-line Storage Internet Dialogue Manager Cookie Cognition Preformatted dialogues Web Logs Archives Source: Bill Inmon SAP AG 2006, 10

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

Typical Data Warehouse Problems End-User Challenges Making timely, informed business decisions Users cannot wait for historical data to be restored Transparent access to data for regular reporting and ad-hoc analysis IT Management Challenges Meeting end-user data demand while managing cost High costs of adding/managing online disk storage High costs of backup and recovery especially when data is accessed infrequently Data protection and availability SAP AG 2006, 12

Constantly Increasing Database Volumes Meta Group 70% of all corporate data is currently located in databases Gartner Group Databases with multiple terabytes are already a reality in the near future these will have hundreds of TBs Business Applications such as those from SAP play a significant role in these growth in memory needs Growth rate is 64% SAP AG 2006, 13

Distribution of Memory Costs The costs for data media don t even make up a quarter of memory costs (Giga Information Group) Misc (Purchasing, training) 10% Personel 45% Hard disk 23% Environment (Electricity, Space) 3% Storage- Management (Soft & Hardware) 19% The administrative expense for 1 terabyte of memory is five to seven times as high as the memory cost itself (Dataquest/Gartner) SAP AG 2006, 14

Motivation for a Data Aging Strategy: Benefits Performance Faster data load times Faster query execution times Cost Storage costs: High availability, high IO disks, etc. Resource and Administration overhead System: CPU, Memory, etc. Headcount: Number of full-time employees, etc. Control of system growth Availability Data availability faster rollups, change runs, etc. System availability less downtime for backups, upgrades, etc. SAP AG 2006, 15

How to Avoid High Data Volumes in a DW Environment? Is data still needed? Yes Can the data be aggregated? Yes Aggregate data! Maybe No Yes Can the data be archived? No Yes We need to address the no longer needed AND the maybe scenario! Data remains in the database! Archive data! SAP AG 2006, 16

Data Aging Strategy Implementation Initial Steps Data aging is a strategy for managing data over time to balance data access requirements with TCO Each data aging strategy is uniquely determined by the customer s data and the business value of accessing the data Which tools should I consider using? Online Database Storage Near-line Storage BW data archiving Frequently read /updated data Infrequently read data Very rarely read data SAP AG 2006, 17

Benefits of a Fundamental ILM Strategy for BI 1 Information available for any time frame for ad-hoc analyses and rebuilds 2 Main memory and CPU as well as costs for system administration 3 Increase Volume Manage and use even larger amounts of information more effectively Reduce Resource Consumption Reduction of hardware costs for hard drive on the BI side Increase Availability Quicker, simpler software and release management in BI Reduced backup and recovery times Intelligent data access 4 Optimize Performance Speed up loading processes in SAP NetWeaver BI SAP NetWeaver BI Query answer times in the dialog SAP AG 2006, 18

Data Aging and Information Lifecycle Management Motivation to develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

Data Model Design and Strategies: Definition Include data aging early on in the blueprinting phase Data retention should be determined during requirements gathering Determine retention for all data layers including transactional and master data Transactional data should be evaluated from both a DataSource and an individual DataTarget perspective Evaluate legal reporting requirements Evaluate regulatory reporting and retention requirements Future business data analysis requirements should also be considered Observation: SAP typically observes data retention of three to five years in InfoCubes and DataStore objects Data model sizing should be included in overall BW capacity planning Data volume and growth should be determined Data change activity profiles should also be determined Frequency of data deletion and data updates should be included BW capacity plans and TCO should be revisited regularly! SAP AG 2006, 20

Data Model Design and Strategies: Definition (cont.) Data warehouse data retention should be integrated with your OLTP system s archiving plans Data warehouses normally retain more historical data than operational transaction processing systems Do you need to archive BW data if your ERP system also archives it? Does your OLTP archiving strategy limit future data warehouse developments? SAP R/3 archives can be read from BW via SAP Archive Information System functionality Keys to a successful data aging strategy development: Define data retention for all data Profile your data activity and access Determine the capacity impact of ongoing data storage If possible, determine a cost model for your data storage/access Choose and implement technology that does not limit your business SAP AG 2006, 21

ILM in SAP NetWeaver BI 1. Online Data persistent in the database Data modeling aspects important Use multiple layers to control data growth Frequent cleanup necessary 2. Near-line Near-line Storage (NLS) ADK (Archiving Development Kit) and non-adk-based Set up proper archiving policy Transparent access for reporting 3. Offline Classic archiving as known in SAP BW 3.x Very cheap storage medium can be used ADK-based No access for reporting SAP AG 2006, 22

Online: Data Volume Minimization Strategies PSA deletion: Temporary inbound data load layer, exists for recovery or reload purposes Retain daily loads one week, monthly retain 60 days Customer example: DB growing at 450 GB/month without PSA deletion, 150 GB/month with deletion ODS object data deletion Optimize ODS Object activation performance Change log deletion: It s like a PSA When data consolidated in subsequent data target When no delta at all is provided by this ODS Active table When data is consolidated in active table SAP AG 2006, 23

Online: Data Volume Minimization Strategies (cont.) Data consolidation: Eliminate redundant data Merge data from similar data marts Optimize aggregates Aggressively archive and delete from InfoCube and ODS object layers Summarize at the InfoCube and aggregate level (RRI) Remote access: Use RRI, Virtual InfoProviders, UDI, etc.: Operational processing possibly might remain in OLTP system SAP AG 2006, 24

Use Offline Archiving with MultiProvider MultiProviders are used BEx or Web Reporting To combine actual data with reloaded data from archive Defining data targetindependent queries 2005 2004 2004 2003 2003 Multi Provider 2006 2005 ADK Archive 2005 2004 2004 2003 2003 Data has to be reloaded before availability in reporting SAP AG 2006, 25

Use NLS Archiving MultiProviders are used for Defining data target-independent queries BEx or Web Reporting Multi Provider 2006 2005 NLS NLS Archive Archive 2005 2004 2004 2003 2003 SAP AG 2006, 26

Use Offline Archiving with MultiProvider MultiProviders are used for Splitting data from one business scenario into separate basic InfoCubes ( logical partitioning ) Possible partitioning criteria: Year, plan/actual, regions, business area Use MultiProviders to cut large amounts of data into chunks BEx or Web Reporting Multi Provider 2002 2002 2003 2003 2004 2004 2005 2006 2005 ADK Archive SAP AG 2006, 27

Use Offline Archiving with MultiProvider (cont.) Benefits Single InfoProviders smaller, less complex, and less sparsely filled than one big InfoProvider (Parallel) data load into individual InfoProviders Better performance in administration on basic InfoProviders In most cases, queries are split automatically and distributed to InfoProviders (for more details on parallel execution see SAP Service Marketplace Note 629541) Dedicated query split for archive-related data portion as well Central interface for querying even if underlying data models change Disadvantages Administration (with aggregates) Data has to be reloaded before available in reporting Increasing number of InfoProviders to be administrated Remodeling at year end (are you using a new InfoCube in the new year?) New process design at year end (unload, new archive object) SAP AG 2006, 28

Single Basic InfoCube Approach with NLS BI accelerator 2005 2004 2003 2006 2005 2004 Archive 2002 2001 Year Change Data Archiving Process Archive Type Near-line storage Archive 2003 2002 2001 Schedule: January, 01 Selection Schema Time slices (Archive everything older than 2 years) Zero Administration SAP AG 2006, 29

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

Data Aging Strategy Implementation Toolkit SAP BI s data aging toolkit: Near-line data storage The relocation of data from online storage to a secondary storage system Secondary storage is typically utilizing more cost-effective online storage (e.g., IDE disks) or removable media (e.g., tape, optical) Seamless online BW Query access to data via direct SQL access BW data archiving The relocation of data from online storage to offline storage No direct access to archived data for online querying Integrated with SAP s Archive Development Kit Online database storage Near-line storage BW data archiving Frequently read/updated data Infrequently read data Very rarely read data SAP AG 2006, 31

Where Is Archiving and Near-line Storage Applicable? Access frequency reads Decreasing TCO Archiving with SAP BW Archiving (BW 3.x) For analysis, archived data must be reloaded first again into the BW database Reduction in costs of data retention on alternative media Access frequency reads Decreasing TCO Near-line storage Data age Near-line storage (> BW 3.x*) Direct access to data in alternative storage media for queries Performance and data retention costs to access aged data can be minimized SAP AG 2006, 32 Data age * Pilot project possible for BW 3.x

Near-line Storage (NLS): Basics Access Frequency Online Memory Near-line Storage/Archives Direct SQL Access Offline Archive Memory Management Days Months Years Costs RAM DASD Virtual Tape Central Access Tape Central Capacity Tape Time Performance SAP AG 2006, 33

Modern NLS Memory Management System Performance Characteristics Performance- and cost-optimized management to keep data in multilayer, transparent storage systems Combined storage systems of disk, tape, and optical storage media in different variations Direct row-oriented data access also to compress archive data using all types of storage media Access strategies and aging pattern for logical grouping of data Dynamic migration on cost-effective storage media for data files with regressive access frequency Automated process for backup, shadowing, mirroring, recovery, etc. SAP AG 2006, 34

Classic Archiving in SAP NetWeaver BI Access Frequency Online Online Reload Previously Archiving Archiving Age of Data Archiving (SAP BW 3.x) ADK-based archiving solution for InfoCubes and ODS objects Cost reduction due to storing data on alternative storage media ADK indexes for BI Query access ineffective, index structures too inflexible Archived data must be reloaded into the SAP NetWeaver BI database for analysis purposes SAP AG 2006, 35

SAP NetWeaver BI: Near-line Storage Strategy Access Frequency Online From now on Near-line Storage Age of Data Near-line Storage (SAP NetWeaver 2004s BI) SAP NetWeaver BI analyses have direct access to archived data in various storage media Availability of historic data while reducing costs Physical decoupling of frequently, less frequently, or rarely used data Reloading of data into the InfoCube or DataStore object only necessary in exceptional cases SAP AG 2006, 36

The Near-line Storage Solution for SAP NetWeaver BI Separation of frequently used (kept in database) and rarely used (in NLS) data NLS support for InfoCubes and DataStore objects Transparent access to non-archived and archived data for queries Intelligent data access Analysis/feedback data selection High-level index in SAP NetWeaver BI Low-level index in NLS Open interface for certified partners Present development partners PBS Software CBW FileTek StorHouse OuterBay LiveArchive SAND-Technologies Searchable Archive SAP AG 2006, 37

SAP NetWeaver 2004s BI Generic NLS Interface Analysis Data Flow Control Flow InfoCube/DataStore with NLS Combination Distribution and Sending High-Level Index DB Interface Archiving/ Restoring Near-line Storage Adapter BI Database Near-line Storage Partner Solution Low-Level Index Data Manager Robot. Tape Library Optical Libraries Network-Attached Storage or Cost-Effective Data Medium SAP AG 2006, 38

SAP NetWeaver 2004s BI Generic NLS Interface (cont.) Analysis Data Flow Control Flow InfoCube/DataStore with NLS Distribution and Sending High-Level Index DB Interface Archiving/ Restoring Near-line Storage Adapter BI Database Near-line Storage Partner Solution Low-Level Index Data Manager Robot. Tape Library Optical Libraries Network-Attached Storage or Cost-Effective Data Medium SAP AG 2006, 39

SAP NetWeaver 2004s BI: NLS-Based Archiving Query Properties Near-line storage should be read as well Scheduling via Process Chain InfoProvider InfoCubes DataStore Objects Data Archiving Process Archive Type Offline, ADK-based Near-line, ADK-based Near-line storage, standalone Selection Schema Time slices (Relative archiving periods, delta oriented, compressed data only) Request-oriented Free selection criteria Online DB Archive SAP AG 2006, 40

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving Summary

BW Data Archiving Technology SAP BW 3.0B/3.1 Content provides the following functionality: BW objects data archiving Selection of data based on any criteria (time- or data-specific) Automatically scheduled on a periodic basis Data deletion (without archiving) Restoring (reloading) of archived data Consistent archiving processes for BW objects Future proof data archives for reuse Archive objects can be defined for the following DataTargets: InfoCubes ODS objects It is planned to include PSA and master data archiving in future BW releases IDoc archiving has been available since BW 2.0B SAP AG 2006, 42

BW Data Archiving Technology: ADK Features SAP BI data archiving is built on the Archive Development Kit (ADK). The ADK is part of the underlying WebAS technology on which mysap applications such as SAP BW, R/3, CRM, etc., are built. ADK Features: Open to CSP archiving and storage systems (example: IXOS) Widely used by mysap application components Handles structural changes Platform independency (code-page, number format) Data compression Statistics and indexing support Administration of archiving via transaction SARA Provide logging function for archive administration Supports indexing with Archive Information System (SAP AS) (BW does not use this functionality) Archiving objects are transport-enabled SAP AG 2006, 43

Technical Architecture: BW and ADK Integration InfoCubes BW database Archiving Object Data Objects File system External Storage Systems Optical storage system ODS object Archiving Object Data Objects Archive Files Archive Files Storage system HSM System SAP Functionality ADK Third-party products SAP AG 2006, 44

BW Data Archiving Technology: Process Overview activates BW Repository BW Repository generates InfoCube ODS object DataManager DataManager read delete Archiving Object Archiving Object Write Verification Delete Scheduled Read Archive Archive Administration Administration (SARA) (SARA) Datamart Datamart Extractor Extractor ADK ADK File system, CMS, HSM SAP AG 2006, 45

BW Data Archiving Technology: Archive Admin Archiving Object InfoCubes 1:1 Relationship InfoCube : File Archiving Object ODS objects Archiving Object Archiving object meta data is maintained at the DataTarget level Generated on DataTarget activation Archiving object name is generated from DataTarget name Administration and execution Accessed via link from Admin Workbench or transaction code SARA Maintain variants for archive execution Data selection criteria, Archive processing and scheduling options Scheduling and monitoring of archiving sessions Management of archive runs Information systems/statistics SAP AG 2006, 46

BW Data Archiving Technology: Archiving Process Archiving Process: Select DataTarget to be archived Configure archiving process Configure and generate archive object Test archiving process Archive data Yes Successful? No Validate archive? Yes Delete Data No Move archive data to storage subsystem (optional) SAP AG 2006, 47

BW Data Archiving Technology: Object Definition Populated after activation! SAP AG 2006, 48 InfoCube Example

BW Data Archiving Technology: Object Definition (cont.) Time Slots option: Time characteristics to be used for selection criteria for archiving. Relative time archiving can be achieved using Time Slots. Protected Archived Areas option: Ability to lock archived time selections from further data load activations. Archive data would have to be reloaded first before new data is activated. Note: Only available for Time Slots archiving of ODS objects. Record selection option: Possible to choose fields used for selection criteria for archiving. All DataTarget fields are available to be chosen. SAP AG 2006, 49 ODS Object Examples

BW Data Archiving Technology: Object Definition (cont.) Archive will create a new file when max file size is reached Number of data records written to a file before a new file is created Defines field layout for archive file Available (unassigned) fields for record SAP AG 2006, 50

BW Data Archiving Technology: Object Definition (cont.) Logical path for archive file defined via transaction FILE Only required if you have an archive storage subsystem (e.g., IXOS, Filenet, Storhouse, etc.) Determines how the subsequent data deletion job will be scheduled and executed SAP AG 2006, 51

BW Data Archiving Technology: Menu Access SAP AG 2006, 52

BW Data Archiving Technology: Data Deletion Write job W1 W2 Archiving Object InfoCubes Delete job 1 V1 File Deletion Wn Vn D Archiving Object ODS objects Delete job 2 V2 Write to archive file n Verify archive file n Selective deletion with selection criteria of archiving session Write to archive file and Data deletion are separated Archived data files are read for verification Completeness Accessibility One-step deletion if all session files are verified (different from OLTP archiving) Using selection criteria of archiving session Note: NOT the contents of the archive file like SAP R/3 archives ODS objects: Selective deletion from active table InfoCube: Selective deletion from fact tables Optimized deletion strategies Adjust or rebuild aggregates determined by data volume Note: DataTarget is locked for data loads until data is deleted or the archive run is invalidated SAP AG 2006, 53

BW Data Archiving Technology: Write Process The write process archiving parameters Relative or absolute time-slot archiving Time granularity for data selection for relative archiving Ability to restrict archiving to previous closed time periods from relative archiving, e.g., closed fiscal years, posting periods Ability to exclude additional specified time periods from relative archiving Constant time period archive specification (hard-coding) Archive write process control SAP AG 2006, 54

Administration of the Archiving Process The write process archiving parameters Selective criteria archiving (non-time slot) Ability to restrict archiving via selection criteria for all fields selected during archive object definition; individual values, ranges, exclude values, etc. Archive write process control SAP AG 2006, 55

BW Data Archiving Technology: Extraction/Reloading InfoCube ODS object InfoCube ODS object Update Rules InfoPackage Export DataSource Archive Files InfoPackage is extended by an option archive selection Selection options for available archive sessions and files Only full extraction supported Archive files are scanned with selection criteria of the request Reload to original DataTarget is possible but not recommended Reload recommendation: Extract to a copy of original DataTarget instead Use MultiProvider to combine remaining data with reloaded data SAP AG 2006, 56

Data Aging and Information Lifecycle Management Motivation to Develop a Data Aging Strategy Data Aging Scenarios Near-line Storage and SAP NetWeaver 2004s BI Data Archiving and SAP NetWeaver 2004s BI Summary

Summary 1The Healthy System Don t start thinking about data archiving when your system is about to crash! 2Timely Planning Proactive preparation for sustainable system performance 3Interdisciplinary Process Data archiving requires a large amount of coordination between IT and those who are responsible for applications SAP AG 2006, 58

For more information: Access the SAP Developer Network www.sdn.sap.com The central hub for the SAP technology community Everyone can connect, contribute and collaborate- consultants, administrators and developers Focus around SAP NetWeaver and SAP xapps High quality of technical resources Articles, how-to guides, weblogs, collaborative areas, discussion forums and downloads, toolkits and code-samples A collaboration platform, not a one-way street SAP experts from customers, partners and SAP SDN is powered by SAP NetWeaver Built on the SAP Enterprise Portal Featuring collaboration capabilities of SAP Knowledge Management SAP AG 2006, 59

7 Key Points to Take Home TCO can be dramatically decreased for your EDW by implementing a data aging strategy Successful data warehouse implementations are cost-effective solutions that continue to drive the business It is imperative that you consider data retention early on during the blueprinting phase of your project Data retention requirements should be routinely assessed and adjusted based on business demands and costs SAP NetWeaver BI provides a rich set of tools for implementing your data- aging strategy SAP NetWeaver BI s Near-line Storage solution provides a compelling, cost-effective alternative to online database storage for less frequently accessed data Data archiving is a very flexible and powerful tool for managing infrequently accessed data in your data warehouse SAP AG 2006, 60

Questions? Q&A Bryan Katis bryan.katis@sap.com SAP AG 2006, 61

Copyright 2006 SAP AG. All Rights Reserved No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iseries, pseries, xseries, zseries, z/os, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MaxDB is a trademark of MySQL AB, Sweden. SAP, R/3, mysap, mysap.com, xapps, xapp, SAP NetWeaver and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages SAP AG 2006, 62