How to health-check your TSM environment

Transcription

1 How to health-check your TSM environment Holger Speh Consulting IT Specialist

2 Motivation Why is my backup so slow? How much data is transferred every day? How are my tape drives utilized? It s the network!? Where is my bottleneck? Why is my restore so slow? Healthcheck your environment! 2 Why do I run out of scratch tapes all the time?

3 What is a TSM Healthcheck? A short engagement to evaluate an existing TSM solution. Typically focused on optimisation potential TSM server health Overall performance Aligning current configuration with best practices Can also be focused on helping with strategic planning for future needs. Must be clear and honest, even if it has bad news 3 3

4 What is a TSM Healthcheck not? It is not just an overview dashboard lights tool. It is not deployment. It is not an excuse to sell/buy unneeded software. It is not a long term engagement. It is not about fixing specific current problems. It is not an excuse for the customer to find an IBM engineer to be on the hook forever for recommendations resulting from the assessment. It does not include Disaster Recovery analysis 4 4

5 Methodology overview 1. Prepare an assessment workshop 2. Perform assessment workshop 3. Perform analysis and generate diagrams and tables 4. Create report (doc/presentation) 5. Present analysis results and recommendations 5 5

6 Prepare yourself to understand the situation Understand why this has been started Understand the customer s expectations Understand your own expectations Prepare a customer assessment workshop and send customer an assessment worksheet 6 6

7 Perform a detailed client workshop Set customer expectations of the assessment Gather general information about processes, organisation, strategy, etc. Understand your client s requirements! Gather as much data as possible about customer s environment TSM configuration Network topology SAN topology Systems and Software used Disk layout Tape infrastructure Setup data gathering/monitoring mechanisms Remember to monitor OS cpu, memory, disk and network Monitor the same time period as you will analyse from within TSM 7 7

8 Know your tools and understand the data This TSM Healthcheck is a pretty technical based tool! It does not replace a general architectural review! It does not provide all answers at hand! You need to be able to interpret the data! Use latest available reference literature! 8 8

9 Workflow and Tools used TSM Administrative Commandline MySQL Perl Bash Shell Scripts Gnuplot Actlog Archives Backups Contents Spacemgfiles Volumeusage 9 6-May-09

10 TSM performance measurement areas Not all layers can be monitored through TSM Client performance for LANfree needs special server parameter OS monitoring for TSM host needed 10 10

11 The situation: Consolidation of very large TSM environment Global Technology Services Customer asked for proposal to simplify and consolidate his TSM environment Following points had be be respected: LAN-free ( Clients), LAN-based ( Clients) Central Tape Management (ermm / RMM) Dynamic loadbalancing due to heavy data growth Simple and leight weighted Use optimal tape performance Use TSM mirroring Use available techniques Don t consider disaster recovery Encryption PnP (Plug n Play of new systems) Migrate... finally 11 11

12 Quick overview ordered by sections Server i0 i1 s1 s2 s3 s4 s5 s6 s7 s8 w1 w2 x1 x2 x3 x4 x5 x6 x7 x8 xa xb xc xd Kategorie Kriterien HP-UX zos Win AIX client activity 24h traffic daytime gaps backup archive restores/retrieves client volume avg hourly nighttime volume avg hourly daytime volume daily file-level daily tdp client performance avg mb/sec max mb/sec client sessions avg parallel count max parallel count server activity 24h traffic nighttime gaps migration reclamation expiration server performance db backup expiration migration server volume migration mount wait migration 12 6-May-09

13 Section: client activity zos and HP-UX environment Actvity around the clock Nearly no free time windoes for administrative activities during the day Moderate to frequent restore activites Windows and AIX environment Backup operation only during the night Adequate free time windows for administrative activities during the day No restore activities tsmgroup platform_name count gb HPUX HPUX HPUX TDP Oracle HP HPUX TDP R3 HP 52 3, MVS AIX MVS CE Archive 3, MVS HPUX 3, MVS IRIX MVS Linux MVS Linux MVS LinuxIA MVS LinuxPPC MVS SUN SOLARIS MVS TDP Oracle HP MVS TDP Oracle SUN MVS TDP R3 HP 31 1, MVS WinNT Windows WinNT AIX AIX AIX TDP MSExchg AIX WinNT SUM 7, May-09

14 Section: client volume zos environment Moderate to high data volume per hour during the night Low data volume per hour during the day File-level as TDP traffic HP-UX environment Moderate data volume per hour during the night as during the day Few file-level Traffic, much TDP Windows environment Low utilization per hour Low data traffic AIX environment Fresh environment with yet low utilization Partition of day and night recognizable 14 14

15 Section: client performance All environments Low thruput during file-level activities Only TDP Nodes show higher thruput 15 15

16 Section: client sessions zos environment Moderate to many parallel sessions Note: Every active session allocates memory and generates locks in the TSM DB HP-UX environment Few to moderate parallel sessions Windows environment Few parallel sessions AIX environment Very few parallel sessions on freshly setup systems Moderate utilization on all other systems 16 16

17 Section: server activity zos environment Activity round the clock with no free time windows Main activity is migration Sometimes very long running expirations HP-UX environment Different behaviour Adsmi0 only shows server activities during the day Adsmi1 show activity round the clock with a lot of migration Windows environment Few nightly activities Sometimes long expiration AIX environment Currently server activity only during the day Exception: Expiration, which is scheduled with short intervals 17 17

18 Section: server performance zos environment Very good DB performance, exception is s7 Migration only moderate HP-UX environment Very good DB performance Migration only moderate to bad Windows environment Mixed behaviour Moderate (w1) and very good (w2) performance AIX environment Mixed behaviour Sometimes very good DB Performance (x1-x4) 18 18

19 Section: server volume Only migration shows significant volume zos environment Low to very high migration volume Frequent disk pool overflows HP-UX environment Low migration volume Frequent disk pool overflows Windows environment Low migration volume AIX environment Nearly no migration volume 19 6-May-09

20 Section: mount wait zos environment Low waiting times HP-UX environment Low to moderate waiting times Windows environment Low to moderate waiting times High waiting times for client actions AIX environment Low waiting times High waiting times for client actions 20 20

21 Potential Storage Pool Overflows Daily migration volume compared to disk pool size Assumption: only 1x daily migration Potential small disk pool if daily migration volume exceeds storage pool size stgpool_name overflows stgpool_name overflows adsmi0 adsms5 BACKUP_DISK_LAN1 5 ARCHIVE_DISK 7 BACKUP_DISK_REDO1 10 BACKUP_DISKJ 16 BACKUP_DISK_REDO2 10 adsms6 adsmi1 ARCHIVE_DISK 14 BACKUP_DISK_REDO1 25 BACKUP_DISK 5 BACKUP_DISK_REDO2 15 BACKUP_DISK2 4 adsms1 BACKUP_DISK3 3 ARCHIVE_DISK 9 BACKUP_DISKJ 1 BACKUP_DISKJ 21 BACKUP_DISKJ_GR 4 adsms2 adsms7 ARCHIVE_DISK 1 ARCHIVE_DISK 31 BACKUP_DISK 12 adsms8 BACKUP_DISKJ 18 ARCHIVE_DISK 1 adsms3 BACKUP_DISKJ 22 ARCHIVE_DISK 2 adsmx7 BACKUP_DISK 17 BACKUP_3592_J1_FS 4 adsms4 adsmx8 BACKUP_DISKJ 30 BACKUP_3592_J1_FS

22 LANfree considerations by volume 161 LANfree Clients Activities by Volume Volume by date Plattform GB Count platform_name Total HPUX <1 820 HPUX 55 HPUX TDP MSExchg 39 HPUX TDP Oracle HP 9 HPUX TDP R3 HP 58 SUM 1,047 Grand Total 161 TDP MSExchg <1 4 TDP MSExchg LANfree Clients platform_name Total HPUX 1 TDP MSExchg 37 TDP R3 HP 36 Grand Total 74 TDP MSExchg TDP MSExchg TDP MSExchg > SUM 221 TDP Oracle HP <1 9,392 TDP Oracle HP ,042 SUM 10,434 TDP R3 HP <1 3,519 TDP R3 HP ,240 TDP R3 HP TDP R3 HP TDP R3 HP > SUM 7,046 SUM 17,639 SUM 1,109 Overall SUM 18,748 Small operations no longer LANfree Massive reduction of LANfree tape mounts New Old date LAN LANFREE LAN LANFREE Total , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Median 19, , , , , Max 24, , , , , Min 4, , , , , May-09

23 LANfree considerations by performance 161 LANfree Clients Activities by Thruput and GB Volume by Thruput platform_name Total HPUX 55 TDP MSExchg 39 TDP Oracle HP 9 TDP R3 HP 58 Grand Total LANfree Clients platform_name Total TDP MSExchg 39 TDP Oracle HP 1 TDP R3 HP 45 Grand Total 85 Platform MB/sec Count Avg GB/Op HPUX HPUX HPUX HPUX SUM 1,047 TDP MSExchg TDP MSExchg TDP MSExchg TDP MSExchg TDP MSExchg TDP MSExchg > SUM 221 TDP Oracle HP 0-1 2, TDP Oracle HP 1-5 7, TDP Oracle HP TDP Oracle HP TDP Oracle HP SUM 10,434 TDP R3 HP 0-1 1, TDP R3 HP 1-5 2, TDP R3 HP TDP R3 HP , TDP R3 HP , SUM 7,046 New Old date LAN LANFREE LAN LANFREE Total , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Median 20, , , , , Max 24, , , , , Min 4, , , , , May-09

24 Solution strategy: Equalize potential bottlenecks by introducing a layered architecture which accounts all included components Each layer is setup by following defined standards and will be easily expandable in a horizontal manner by adding additional components. Through standardized usage of included components a vertical efficiency can process data by its business value. By introducing planning mechanisms this highly scalable environment can be administered very effectively and efficiently

25 Findings require infrastructure alignment to implement solution strategy Global Technology Services TSM Software Consolidate environment by focussing on only two platforms Adjust config to Best Practices Define standard mechanisms for client backup procedures according to their SLAs Establish central configuration manager for TSM objects Disk Hardware Reduce complexity coming from LANfree backups and force Disk2Disk by enlarging disk pools by 10-15TB Establish shared disk environment by establishing GPFS Establish standardized Disk environment with DS8300 for Open Systems and Mainframe Tape Hardware Establish standardized Tape environment with TS1120 for Open Systems and Mainframe, which also supports encryption SAN Hardware New dual fabric SAN with >200 ports per fabric 25 25

26 What about TSM 6.1? DB/2 included DB/2 instance is hidden Many tables can be seen but not all can be accessed Don t do number crunching direct on TSM DB Include DB/2 parameters into assessment TSM Reporting available Offers limited view on available data Aggregates data Presented method is still valid for TSM 6.1 Little Adjustment might be needed 26

27 27 27

28 28

29 29 Global Technology Services

30 Disclaimer No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or program(s) at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customer experiences described herein are based upon information and opinions provided by the customer. The same results may not be obtained by every user. Reference in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectual property rights, may be used instead. It is the user's responsibility to evaluate and verify the operation on any non-ibm product, program or service. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR INFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-ibm products discussed herein. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products May-09