UniCredit Global Backup Infrastructure. Mirco Lissandrini, Team Leader GCC Open Storage

Transcription

1 UniCredit Global Backup Infrastructure Mirco Lissandrini, Team Leader GCC Open Storage Milan, 28 May 2013

2 AGENDA UniCredit at a glance UniCredit Business Integrated Solutions: Identity Card Global Backup Area Network Infrastructure The Key Elements of Global Backup Area Network Facts and Figures of Global Backup Area Network Local / Remote Cluster DG Configuration and Replication Backup and Replication Data Flow Policy Distribution By Type Case Study: Catalog Disk Layout Case Study: Synthetic vs. Optimized Synthetic Questions & Answers 2

3 UniCredit at a glance Employees: more than 156,000 * Branches: 9,322 * Banking operations in 22 countries International network spanning: ~ 50 countries Global player in asset management: bn in managed assets * Market leader in Central end Eastern Europe leveraging on the region's structural strengths 3 * Source: UniCredit Company Profile, data as at December 31, 2012

4 UniCredit Business Integrated Solutions: Identity Card UK GERMANY CZ REP. POLAND ~ Fte s ( y/y pro forma) 11 Countries* SLOVAKIA 4 Legal Entities HUNGARY ITALY ROMANIA ~ 2,6 mld Revenues by 2012 AUSTRIA ~ 550 mln IT Investments UniCredit Business Integrated Solutions is the first concrete milestone within the Group Strategic Plan 2012 announced in November 2011 to be achieved. Owned by UniCredit, is created from the integration and consolidation of 16 Group companies (among them UGIS, UCBP, URE, UC) and is dedicated to providing services in the sectors of Information and Communication Technology (ICT), Back Office and Middle Office, Real Estate, Security and Procurement. A new business model, unique in the European banking sector, focused on Business needs (i.e. Commercial Banking, Global Markets, CEE), not only on providing services. * UniCredit Business Integrated Solutions operates also in 2 branches, one located in New York and one in Singapore.

5 Global Backup Area Network Infrastructure Unified Infrastructure where all resources are shared from 4 Main Data Centers (Masters, Media Servers, Libraries, Deduplication Appliances, Policies, Clients, Retentions, Configurations, SLPs, etc.) 5

6 The Key Elements of Global Backup Area Network Master Servers: 4 HP DL580 G7-2 local MetroClusters (Italy & Germany) joined into a GeoCluster with Volume Replication of Shared Disk Groups between sites. Current version is on Windows 2008 R2 SP1 Media Servers: 24 HP DL380 G8 spread across 4 Data Centers. Current version is on Windows 2008 R2 SP1 and RHEL 6.1 NBU Catalog: 900 GB Catalog Images compressed (2300 GB flat) Global Data Protected: 22.3 PB overall (18.1 PB on DataDomain vs. 4.2 on Tape 82% vs 18%) Tape Libraries: 4 Oracle SL8500 with 48 T10K model A and B drives, around 8200 cartridges whose 4900 are scratch Deduplication Appliances: 18 Data Domain Appliances installed (DD880, DD890 and DD990), 4 PB of total space, 18.1 PB of total pre-compression, 2.0 PB of total postcompression, 9.1x (89.1%) global deduplication ratio and 15.8x (93.7%) best ratio NetStorage: every Master, Media Server and Data Domain Appliance is connected to private, not-firewalled and Nexus 5000/7000 based 10GbE network. NetBackup client link is still a work in progress 6

7 Facts and Figures of Global Backup Area Network Monthly Jobs: Monthly Restores: ( daily) 4000 (mostly DB cloning) Monthly Successful Rate: 99,6% 99,7% Monthly Data Backed Up: Number of NB Clients: Number of Policies: 1612 Number of SLP: PB of primary copies ( TB daily) 6532 (not unique) Retention Levels: weekly (1,2 and 3) montly (1,2,3,6,9 and 18) yearly (1,2,3,5,7,10 and 11) Data Distribution: 67% 70% of backup data are addresses to Deduplication Appliances, the remaining to Tape Libraries Storage Unit Reference: Deduplication Appliances all backups with retention less 18 months are addresses to DBs vs. FileSystems: 60% 65% of daily backup data belong to Database systems (DB2, Oracle, SAP, SQL, Sybase and Teradata) 7

8 Local / Remote Cluster DG Configuration and Replication 2 Distinct Clusters (Local and Remote) with Global Cluster Option enabled Each volume is mirrored locally and remotely by Storage Foundation to get full-redundancy Disk Groups are asyncronously replicated from local to remote sites for Disaster Recovery reasons Target Scenario is to enable Active Master entity rotation from site to site 8

9 Backup and Replication Data Flows Every Backup is duplicated on a secondary site indipendently from the destination unit First copy is written locally (Tape or Disk), then it is duplicated on the next metropolitan Data Center Some strategic backups require to be replicated on a cross-country site for legal or critical demands 9

10 Policy Distribution By Type Most of the Policies are of FileSystem type (54,2%+36,7%) DB policies are globally less than 3,7% (DB2, Oracle, SAP, SQL and Teradata) FileSystem backups are usually Master initiated, DBs instead are client initiated Policies are separated by Accounting Code, Backup Type, Window and Retention Policy Distribution by Type Vault ; 0,13% VMware ; 3,39% FlashBackup-Windows ; 0,75% DB2 ; 0,25% Teradata ; 1,70% MS-SQL-Server ; 0,19% Standard ; 54,21% MS-Windows ; 36,75% MS-Exchange- Server ; 0,57% NBU-Catalog ; 0,06% NDMP ; 0,57% Oracle ; 1,32% SAP ; 0,13% 10

11 Case Study: Catalog Disk Layout - 1 Master Server suffered of a severe queuing issue in the scheduling rush hours A lot of tuning changes on EMM, NBRB, NBPEM and NBDB were applied to reduce the impact of queuing issue The problem has gotten worse after NetBackup 7.5 upgrade (Catalog info moved into NBDB) The queuing was solved splitting NBDB and Catalog flat files into two distinct volumes F:\ - BlkSize 64K NBUCatalog NBDB Volume Group: USNBUCatalog 2500 GB G:\ - BS 64K 500 GB NBUlog Volume Group: USNBULog SPLITTED UNCHANGED Volume Group: USNBUCatalog Volume Group: USNBULog E:\ - BS 64K F:\ - BS 4K G:\ - BS 64K 1300 GB NBDB 200 GB NBUCatalog NBUlog 500 GB 11

12 Case Study: Catalog Disk Layout - 2 Some other tuning had been applied to NetBackup Relational Database server.conf -m truncates the transaction log when a checkpoint is done. -gn 40 number of requests the database server can handle at one time. -cl 2G minimum cache size, as a limit to automatic cache resizing. -ch 32G maximum cache size, as a limit to automatic cache growth. Some EEBs had been installed to improve DB queries from bpdbm and nbpem (i.e ) 12

13 Case Study: Synthetic vs. Optimized Synthetic - 1 NetBackup Synthetic Backup is a feature where a full backup can be synthesized on Media Server by making use of previous full and subsequent incremental backups reading those component images and writing a new image. Because processing takes place on Master and Media Servers instead of the client, synthetic backups help to reduce the network traffic eliminating or reducing frequency of traditional full backups The policy type must be either Standard or MS-Windows. The Collect True Image Restore Information With Move Detection option must be selected on the Policy Attributes tab. The schedule that is created for a synthetic backup must have Synthetic Backup selected. 13

14 Case Study: Synthetic vs. Optimized Synthetic - 2 NetBackup Optimized Synthetic Backup is a feature where a full backup can be synthesized on Storage Server by making use of previous full and subsequent incremental backups without reading and writing those component images. Virtual Synthetics use DataDomain s metadata for synthesizing a full backup; The big benefit is no data is moved back onto the NetBackup Media Server, therefore an increased performance will be raised as well as an important server load reduction Policy requirements are the same of NetBackup Synthetic Backup DDOS 5.2.x is required and feature must be activated with command: ddboost option set virtual-synthetic enabled OptimizedImage flag must be present for StorageServer and DiskPool (nbdevconfig changests and nbdevconfig changedp with -setattribute OptimizedImage) 1 F DD Boost 2 I I I I I I 3 I VF NetBackup Server 1. Sunday Traditional full backup 2. Monday Saturday Incremental backups 3. Sunday Incremental, then Virtual Full Virtual Full = regions from traditional full and Incrementals Data Domain System 14

15 Case Study: Synthetic vs. Optimized Synthetic - 3 In Synthetic Backup: - Full-image traverses from the storage (disk or tape) to the Media Server via whatever connection. - Incremental-image(s) also traverses from the storage to the Media Server. - Media Server uses its resources (processor, memory, network, storage I/O, etc) to combine previous full and incremental(s) to create a new "full" backup. - Media Server then sends this new synthesized image back to the storage via whatever connection. In Optimized Synthetic Backup: - Media Server 'instructs' the Storage Server (via the OST API) to synthesize a new image. - Without sending anything to the Media Server, the Storage Server synthesizes the new image out of existing images. This is all done inside the storage itself. The Media Server resources are not used. - When the new synthetic image is successfully created, the Storage Server 'informs' (via the OST API) the Media Server mission accomplished. 15

16 Questions & Answers 16