SAP BW Archiving with Nearline Storage at Esprit

Transcription

1 SAP BW Archiving with Nearline Storage at Esprit Claudia Ottilige, Esprit Europe GmbH Dr. Michael Hahne, Hahne Consulting GmbH 27. Februar 2013

2 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 2

3 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 3

4 Company overview Esprit Company Overview One of the most well-known apparel brands in the world. Founded in San Francisco in 1968 by Susie and Doug Tompkins Offering style and quality made to last. Effortless style, authentic and easygoing living are at the heart of the brand philosophy. Company is represented in more than 40 countries More than 1,000 directly managed retail stores Distributes via more than 10,800 controlled space wholesale point-of-sale Total selling space over 1 million square meters. Since 1993 listed on the Hongkong stock exchange. Group turnover: 30,165 m HK$ (FY 2011/2012) Product lines: Casual, Collection, edc, others: accessoires, bodywear, shoes, kids, sports, de.corp Distribution channels: Wholesale, Retail, Licences SAP BW Archiving with Nearline Storage at Esprit 4

5 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 5

6 Background / initial situation (I) Esprit s SAP BW has currently 16 TB of data, data volume still growing nightly data loading times too long à reports are often available too late for the users potential recovery of the system would take several days backup time very long à one night can become too short for a backup in case of further growth of data volume à new backup infrastructure needed makes copy of whole production system to test-system inacceptable database reorganisation takes very long data storage cost could be reduced assumption: SAP BW is not meant to archive data relevant for audits, audit relevant data are stored in source systems 6 SAP BW Archiving with Nearline Storage at Esprit 6

7 Background / initial situation (II) During the last 12 months the used dabase size increased from 14,5 to 16 TB. Average growth per month: 125 GB Several housekeeping activities had positive impact on the data volume. SAP BW Archiving with Nearline Storage at Esprit 7

8 Data volume reduction by DVM Data volume management (DVM) is a concept which tries to keep the data volume of the SAP BW system as small as possible. This includes: deletion of unused data deletion of temporary data data archiving (ADK) near-line storage database re-organisation aggregation of data data compression (e.g. Infocubes) processes and guidelines to keep the amount of data as small as possible... 8 SAP BW Archiving with Nearline Storage at Esprit 8

9 Archive Development Kit (ADK) versus Nearline Storage (NLS) ADK (SAP BW 3.X) NLS (SAP NW 7.0 BI) Classical Archiving solution for Infocubes and ODS objects based on Archive development kit (ADK) Archived data must be reloaded into the SAP BW system when needed Cost reduction by using cheaper storage media and through data compression read-only Archiving solution for Infocubes and DS objects Direct access to NLS data for reporting and progagation Archived data does not need to be reloaded into the SAP BW system when needed Cost reduction possible through compression of data NLS data can be stored on file system, DBMS, also in ADK files read-only SAP BW Archiving with Nearline Storage at Esprit 9

10 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 10

11 Data access vs. Data growth Typical Data Growth from year to year 0.5 TB 1.5 TB Typical Data Access vs. Data Growth Frequently read Infrequently read Rarely read 2.5 TB 4 TB 30% 12 TB 10% 100% 90% 75% With increasing Data Growth access likelihood changes significantly SAP BW Archiving with Nearline Storage at Esprit 11

12 Information Lifecycle Management Online: Persistent in the Data Warehouse Data Modelling and Layered Architectures are relevant Housekeeping Nearline: Transparent Access BW on a diet Offline: Classical Archiving Frequently read/updated data Online Database Storage Near Line Storage Data Archiving Infrequently read data Very rarely read data SAP BW Archiving with Nearline Storage at Esprit 12

13 Data Management Best Practice: Enterprise Data Warehousing Data Marts Analytical layer è Mainly aggregated data Roll-Up & Transformation Process Archiving monthly to yearly Roll-Up & Transformation Process Data Load Process Data Integration Layer Data Acquisition Layer 50-70% of overall data volume Ready to use Data è Feeding the Data Marts Archiving weekly or monthly Granular, untransformed Data è 1:1 mapping to ERP System Archiving mostly daily SAP BW Archiving with Nearline Storage at Esprit 13

14 Look-Ups in data flows Reporting Layer Look up of historical data in Update Rules Data Warehouse Layer History Objects Staging ODS Acquisition Layer Adhoc reporting, Analysis Process Designer Look- Up Nearline - Object SAP BW Archiving with Nearline Storage at Esprit 14

15 Look-Up API Analysis Process Data Access API Single Point of access to all data archived and non-archived DB Interface NLS - API SAP BW Archiving with Nearline Storage at Esprit 15

16 DSO Look-up in Transformations with BW 7.30 With BW 7.30 there s a DSO look-up as Transformation Rule available Reads also NLS partitions automatically 7.30 SAP BW Archiving with Nearline Storage at Esprit 16

17 NLS Access in Analysis Processes with BW 7.30 With BW 7.30 the Source-Typ DSO in Analysis Processes reads the NLS Partitions automatically if required 7.30 Data from NLS can be included as option SAP BW Archiving with Nearline Storage at Esprit 17

18 Data Management Best Practice: Integration of NLS in Reporting Layer SAP BW Archiving with Nearline Storage at Esprit 18

19 Data Management Best Practice: Transparent Access in Queries with NLS SAP BW Archiving with Nearline Storage at Esprit 19

20 NLS Support für MultiProvider with NearLine Readmode disabled at all enabled at all InfoProvider settings SAP BW Archiving with Nearline Storage at Esprit 20

21 NLS Properties in a Query with Fixed NLS Settings read NLS do not read NLS see InfoProvider settings SAP BW Archiving with Nearline Storage at Esprit 21

22 NLS Query Variables with BW 7.30 Variable NLS Settings read NLS do not read NLS see InfoProvider settings SAP BW Archiving with Nearline Storage at Esprit 22

23 Advantages of a Data Volume Management Strategy Performance Faster Data Loads Faster Queries Costs Less storage costs Less administrative effort controlled system growth Availability of data: faster rollups and change runs etc. of system: less downtime for backups, upgrades etc. Enterprise Data Warehousing Data Marts on Demand : more flexibility, faster rollout SAP BW Archiving with Nearline Storage at Esprit 23

24 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 24

25 SAP Defined Integration Scenario NW-BI-NLS 04S - Nearline Storage 2004s for SAP NetWeaver BW Certified Partners (June 2012) (see also )

26 Informatica ILM Nearline for SAP BW (Former SAND Nearline) Reduce production storage costs by moving infrequently accessed data to a nearline data store. Does not require any database administration. Data is stored in encrypted files on standard file system, simplifying backup and restore processes as well as replication to other environments. Reduce non-compliance risk with continued access to data in immutable, compressed files - compression rates as high as 98%, with average of 90%+ compression rate. Contact: rsobral@informatica.com

27 SAP NetWeaver BW Nearline Storage on DataVard OutBoard DataVard s OutBoard is the all-in-sap nearline storage solution Built in ABAP, runs in your existing SAP landscape, no additional database or hardware required OutBoard is priced per productive system landscape. No matter how much data you load onto it. Compresses all InfoProviders, PSA and Changelogs up to 95% Analytical Engine Contact: outboard@datavard.com BW Accelerator Database OutBoard Near-Line Storage SAP NLS Interface Data Store Object (volatile) Enterprise Query, Reporting & Analysis Enterprise Data Warehouse (Architected) Data Marts Data Warehouse Layer (historical) DataSource / PSA Change Logs InfoObjects / Master Data Meta Data Repository / Documents Data Flow Control / Process Chains

28 SAP NetWeaver BW Nearline Storage on PBS CBW With CBW PBS provides an add-on solution for the data retrieval of archived/ nearlined BW data. On the basis of the SAP ADK files, the CBW solution creates aggregates on the archive data and (optionally, if needed) index structures to enable a fast retrieval option for the archived data directly from the end-user's original query. PBS CBW NLS IQ for Sybase Analytics Server IQ utilizes the Sybase Analytics Server IQ as a nearline database and offers extremely fast query response times. Contact: klaus.zimmer@pbs-software.com

29 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 29

30 Conclusions (I) vs. PY: Storage: In addition to NLS data, ~ 3 GB could be stored via ADK files, as data are older than requested retention times. During the next 5 years, approx. 4 7 TB of data can be archived only for the integration layer. (data vary because of co-existance phase of two ERP systems) SAP BW Archiving with Nearline Storage at Esprit 30

31 Conclusions (II) Integration layer is the main focus for archiving. Reporting layer is already supported by BWA to increase performance, archiving of data is not a must (alternative: keep only aggregated data for historical data). PSA and extraction layer are deleted very promptly. SAP BW Archiving with Nearline Storage at Esprit 31

32 Esprit s Evaluation (criteria) To compare different providers the following evaluation criteria have been applied: Is separate hardware necessary? How is the Query performance in comparison to BWA? How large is the compression rate? How well does the software assists with the archiving process? Reference customers Support via OSS Compatibility with HANA Architectural fit with other archiving tools already available at Esprit Is it possible to access productive data from the test system? Company background 32 SAP BW Archiving with Nearline Storage at Esprit

33 Datavard OutBoard SAP NLS Interface OutBoard Features 1. Data archiving / storing DSO & Cubes in NLS 2. Loading of data via DTP 3. Reporting on NLS data 4. Reloading of requests 5. Usage in process chains 1. OutBoard Cockpit OutBoard Browser OutBoard Analyzer Statistics 2. Performance-optimization for fast access to NLS Data 3. Mass archiving / Automation 4. Storage Management (via separate Tablespaces a.o.) 5. Archiving Request deletion 6. Archiving PSA & Change log

34 ADK vs. NLS To have 2 archiving systems available increases complexity and cost therefore we decided to only implemented NLS solution. Motivation: NLS is necessary anyway: fast response to new reporting requirements quick reload of cubes quick reload of data for realignments look-ups on archived data during data load possible In case amount of data needs to be further reduced also in Reporting layer (e.g. because BWA is at it s limits) there are 2 possible strategies how to deal with it using the Datavard NLS solution: à either data model can be adapted (aggregation of data) OR à data are saved in NLS, but are not made available for reporting (can be quickly retrieved if absolutely necessary à but probably performance issue) SAP BW Archiving with Nearline Storage at Esprit 34

35 Agenda Company Esprit Initial situation NLS Best Practices Solution options Conclusions Project results so far SAP BW Archiving with Nearline Storage at Esprit 35

36 Project results up to now (I) Several housekeeping activities implemented in 2011 Several workshops regarding archiving concept in 2011 Data volume scoping by SAP in October 2011 Analysis of SAP BW system by SAP in February 2012 Data volume management workshop with SAP (1 day) in Feb Defined & prioritized workpackages to reduce SAP BW data volume Implemented some workpackages (e.g. deleted some large unused DSO s) 36 SAP BW Archiving with Nearline Storage at Esprit 36

37 Project results up to now (II) defined all objects to be archived, distinguishing between: Archiving development kit (ADK) or Nearline storage defined business requirements for data retention times Near-line Storage presentations by different nearline storage solution providers received offers from solution providers workshop with nearline storage expert telephone call with reference customer checked already available SAP archiving solutions at Esprit Europe (PBS solution for current FI/CO system) Calculated business case for NLS solution Installed Outboard Software Started with archiving of first DSO s in October SAP BW Archiving with Nearline Storage at Esprit 37

38 Summary Near-line storage is state-of-the-art alternative to classical archiving NLS can solve various problems: Load run time Query performance Data volume and storage costs Backup and recovery SAP data volume management workshop identifies the biggest potential issues and provides recommendations Quick return on investment (often less than a year) 38 SAP BW Archiving with Nearline Storage at Esprit 38

39 How to get started Analysis Workshop / Fitnesstest Objective: identify Quick Wins / low hanging fruits Benchmarking and Recommendations in areas of System Performance and Robustness Data Management Housekeeping ROI in a couple of months Proof of Concept Test installation Proof of Concept Project and usage Analysis and design Implementation Operation Check-up with Fitnesstest Copyright Hahne Consulting GmbH

40 Questions? 40 SAP BW Archiving with Nearline Storage at Esprit 40