Redbooks Paper. IBM Tivoli Storage Manager: A Technical Introduction. Introduction / Overview. The author of this redpaper

Transcription

1 Redbooks Paper Andy Wharton Roland Leins Charlotte Brooks IBM Tivoli Storage Manager: A Technical Introduction Introduction / Overview IBM Tivoli Storage Manager is one of the core products of the IBM Tivoli Storage Management product set. It provides a solution for distributed data and storage management in an enterprise network environment. IBM Tivoli Storage Manager supports a wide variety of platforms for mobile, small and large systems, and, together with complementary products, delivers many data management functions, including data protection for file and application data, record retention, space management, and disaster recovery. This paper gives a high-level technical introduction to IBM Tivoli Storage Manager. It positions IBM Tivoli Storage Manager within the IBM Tivoli Storage Management solution and provides an overview of its architecture. It also describes the base concepts, the interfaces, and supported environments, recommends some design approaches, and shows IBM Tivoli Storage Manager s interaction with other products in the IBM Tivoli Storage Management product set. The author of this redpaper Andy Wharton is an Accredited IT Specialist in the EMEA North Region Strategy and Design Authority. He is a Storage Subject Matter Expert, specializing in backup and restore, and has been designing and implementing IBM Tivoli Storage Manager and ADSTAR Distributed Storage Manager solutions since 1995 for IBM internal, Strategic Outsourcing, and ebusiness Hosting. Andy is a previous author of Using ADSM to Back Up Databases and was part of the team which set up the original ADSM Certification tests. Copyright IBM Corp All rights reserved. ibm.com/redbooks 1

2 Thanks to the following people for their invaluable contributions to this and the previous edition of this Redpaper: Roland Leins Storage Solution Sales, Central Europe and Emerging Markets Charlotte Brooks Yvonne Lyon Emma Jacobs International Technical Support Organization, San Jose Center Freddy Saldana Tivoli Systems, San Jose Mike Pousson Tivoli Storage Management Marketing, San Jose Patrick Keyes EMEA North Region Strategy and Design Authority, UK Lionel Roquain Strategic Outsourcing Service Delivery France Stephen Wharton Storage Services, UK Chris Zaremba TDP Client Architect Betsy Colby Tivoli Storage Manager API Client Development 2 IBM Tivoli Storage Manager: A Technical Introduction

3 The IBM Tivoli Storage Management solution In today s connected world, data has become the key asset of companies and one of its most important competitive differentiating factors. Temporary inaccessibility or the complete loss of data has a huge financial impact, and can drive companies out of business. The inability to manage data can have a negative impact on a company s profitability and can limit their ability to grow. Storing, protecting, and managing data growth are now among the major challenges of today s businesses. Today's storage management needs to go beyond traditional backup and recovery solutions. Data is the currency of today's e-business economy, and planning to store this data needs to encompass data reliability, solution scalability, disaster planning, and recovery. This will impact the overall infrastructure as well as individual mission-critical applications. For the latest Tivoli Storage News, go to the Tivoli Storage Management Web page: Then click on the News and Events link on the left-hand side of the page. Figure 1 shows the structure of the IBM Tivoli Storage Management Solution and how it fits into the Tivoli Enterprise. IBM Tivoli Storage Management consists of five major solution components: Enterprise protection Application protection SAN management Storage resource management System backup and restore Tivoli Storage Management Asset Management Operations Management Enterprise Protection Application Protection Tivoli Enterprise Change Management SAN Management Storage Resource Management E-Business Management IT Service Level Management Security Management System Backup & Restore Figure 1 IBM Tivoli Storage Management and Tivoli Enterprise IBM Tivoli Storage Manager: A Technical Introduction 3

4 Enterprise protection implements an enterprise-wide solution for data protection, disaster recovery, space management, and record retention. It covers all types of heterogeneous system platforms ranging from mobile systems up to large scale enterprise servers, and supports all types of storage resources, including locally attached as well as network or SAN attached storage. Flexible storage management policies support business needs and provide powerful automation features thus eliminating labor intensive and cost intensive manual storage management tasks. Strategic business applications are typically complex collections of interdependent components from both commercial and proprietary software, and span desktop, distributed, and mainframe computing environments. Application protection is concerned with the data availability, performance, and recoverability, and integrates the application data management into enterprise data protection. Storage Area Network (SAN) architectures add a new dimension to storage and data management by providing a separate, dedicated network to allow businesses of all sizes to provide access to and share data, regardless of operating systems. IBM Tivoli Storage Area Network Manager is a significant step towards helping customers cope with the explosive growth of information in the ebusiness age. SAN management is concerned with the efficient management of the Fibre Channel based SAN environment. Physical connectivity mapping, switch zoning, performance monitoring, and error monitoring are among the most important features. Given the growth in enterprise storage, there is a consequential increase in the number of devices and logical units that would benefit from central management. IBM Tivoli Storage Resource Manager provides logical management, reporting, and predictive capacity planning for these storage resources in order to ensure that the enterprise makes the best possible use of the resources that it owns. IBM Tivoli Storage Resource Manager for Databases provides the added capability to monitor and manage storage usage within various database products again, with the objective of efficiently managing the available storage resources. IBM Tivoli Storage Resource Manager for Chargeback uses the information gathered by IBM Tivoli Storage Resource Manager and IBM Tivoli Storage Resource Manager for Databases to calculate chargeback information and generate invoices for storage usage. 4 IBM Tivoli Storage Manager: A Technical Introduction

5 IBM Tivoli Storage Manager IBM Tivoli Storage Manager is the core product of the IBM Tivoli Storage Management product set. It provides a solution for distributed data and storage management in an enterprise network environment. IBM Tivoli Storage Manager protects and manages data from more than 30 operating platforms, covering mobile, desktop, and server systems over the entire distributed world. It supports hundreds of storage devices (including disk, tape, and optical) as well as LAN, WAN and SAN infrastructures. These are the base functions provided by IBM Tivoli Storage Manager and its complementary products: Data protection, including: Operational backup and restore of data: The backup process creates a copy of the data to protect against the operational loss or destruction of file or application data. The customer defines how often to back up (frequency) and how many numbers of copies (versions) to hold. The restore process places the backup copy of the data back into a customer-designated system or workstation. Disaster recovery: All activities required to organize, manage, and automate the recovery process from a major loss of IT infrastructure and data across the enterprise. This includes processes to move data offsite into a secure vault location, to rebuild IT infrastructure, and to reload data successfully in an acceptable time frame. Data resource management, including: Vital record retention, archive and retrieval: The archive process creates a copy of a file or a set of files representing an end point of a process for long term storage. Files can remain on the local storage media or can be deleted. The customer controls how long (via the retention period) an archive copy is to be retained. The retrieval process locates the copies within the archival storage and places them back into a customer-designated system or workstation. Space management (or hierarchical storage management): This process provides the automatic and transparent movement of operational data from the user system disk space to a central storage repository. If the user accesses this data, it is dynamically and transparently restored to the client storage. The solution is network based, which means that these functions are available to the whole network environment. All the functions can be automated to run in a 24X7 lights-out environment. Administration costs are minimized by centralization of all of the management of IBM Tivoli Storage Manager components. IBM Tivoli Storage Manager architecture Good backup and restore requires thought! This statement may seem obvious, but there is a big difference between solutions implemented well, with careful thought and effective testing, and solutions implemented as an afterthought with little or no planning and with no testing. It is an unfortunate fact that many people only discover that their backup processes are inadequate when they need to restore data. IBM Tivoli Storage Manager: A Technical Introduction 5

6 Backup design Any backup and restore tool is only as good as its implementation. Cars are extremely safe as long as no-one crashes them, but people still manage to do so. Many storage tools are designed to provide excellent backup capability, but the key to making them work effectively relies on a good understanding of what data needs to be restored, and when, including the testing of that restore, thus ensuring that the tools have been implemented in the right way. IBM Tivoli Storage Manager is designed with restore in mind. Its functions and features do provide an efficient and easy to use backup capability and will manage data storage, but the primary objective is to be able to restore data when needed, since that is the only real reason to take a backup. Typical key factors for restore are: What data would I need to restore? When might I need to restore it? In what time scale would I need to restore (recovery time objective)? How much data can I afford to lose (recovery point objective)? What type of data is it files, databases, raw volumes? What post processing will be needed to make the data usable? Perhaps this is an obvious statement, but the most important factor when thinking about restore is to obtain clear and well described requirements. This is easy to say, but how often have people said I want to back up everything and keep it forever? This is both unhelpful and potentially extremely expensive. Any backup, as for many aspects of IT, will be a compromise between speed of recovery and cost of solution. Almost any volume of data can be restored quickly provided that the backup was taken in the correct way and that appropriate equipment is available; however, would such a solution be cost justified? Backup designers should also be wary of mixed backup and archive requirements. A very common request is for weekly, monthly, and yearly backups to be taken, with appropriate retention periods. The restore requirements for these backups are different: the first (weekly) are typically for operational recovery, as described above; while the monthly and yearly backups are more likely to be for audit requirements and are, therefore, archives, not backups. Different processes may be required to meet all these requirements and the data to which they apply may also be different for example, why retain operating system files for 7 years? For this reason, requirements like these should be separated. IBM Tivoli Storage Manager overview IBM Tivoli Storage Manager is implemented as a client server software application, consisting of a IBM Tivoli Storage Manager server software component, IBM Tivoli Storage Manager Backup/Archive client, the IBM Tivoli Storage Manager storage agent, and other complementary Tivoli and vendor software products. Figure 2 shows the main components of IBM Tivoli Storage Manager. 6 IBM Tivoli Storage Manager: A Technical Introduction

7 Administration Client DB Remote ITSM Managed Storage Local Area Network IBM Tivoli Storage Manager Servers DB Local ITSM Managed Storage Storage Area Network Client Servers/ Applications Figure 2 IBM Tivoli Storage Manager architecture The IBM Tivoli Storage Manager server provides a secure environment, including automation, reporting and monitoring functions, for the storage of client data. It also provides the storage management policies and maintains all object inventory information to allow it to manage this data in the most effective way. The IBM Tivoli Storage Manager Backup/Archive client, IBM Tivoli Storage Manager storage agent, and complementary products implement data management functions such as data backup and recovery, archival, space management, or disaster recovery. The client software can run on different systems, including laptop computers, PCs, workstations, or server systems. The client and server software can also be installed on the same system for a local backup solution. The storage agent software in conjunction with the server software enables the implementation of LAN-free backup solutions exploiting the SAN infrastructure. It is also possible to define server hierarchies or multiple peer-to-peer servers in order to provide a multi-layer storage management solution or an electronic vaulting solution. IBM Tivoli Storage Manager: A Technical Introduction 7

8 IBM Tivoli Storage Manager server One of the principal architectural components of the IBM Tivoli Storage Manager server is its in-built relational database. The Tivoli Storage Manager database was especially designed for the task of managing data, and it implements zero-touch administration. All policy information, logging, authentication and security, media management and object inventory is managed through this database. Most of the fields are externalized through Tivoli Storage Manager high level administration commands, SQL SELECT statements or for reporting purposes, by using an ODBC driver. Obviously, this database is fully protected with software mirroring, roll-forward capability and with its own management and online backup and restore functions. For storing the managed data, the Tivoli Storage Manager server manages a storage repository. The storage repository can be implemented using any combination of supported media magnetic or optical disk, tape, and robotic storage devices, which are locally connected to the server system or which are accessible through a SAN. To exploit SAN technology, the Tivoli Storage Manager server has features implemented to dynamically share SAN connected automated tape library systems among multiple Tivoli Storage Manager servers, as well as provide (as an option), LAN-free and server-free backup. The Tivoli Storage Manager server provides built-in device drivers for more than 300 different device types from every major manufacturer. It is also able to utilize operating system device drivers and external library manager software such as the Windows 2000 Removable Storage Manager (RSM). Within the storage repository the devices can operate stand-alone or can be linked together to form one or more storage hierarchies. The storage hierarchy is not limited in the number of levels and can also span over multiple servers using so-called virtual volumes. See Storage and device concepts on page 19 for storage management functions available for the storage repository. IBM Tivoli Storage Manager Backup/Archive client Data is sent to the IBM Tivoli Storage Manager server using the IBM Tivoli Storage Manager Backup/Archive client and complementary Tivoli and non-ibm/tivoli products. These products work together with the Tivoli Storage Manager server base product to ensure that the data you need to store is managed as defined.the Tivoli Storage Manager Backup/Archive client, included with the server, provides the operational backup and archival function. The client implements the patented progressive backup methodology, adaptive sub-file backup technology and unique record retention methods, as described in Backup and archival concepts on page 15. The Backup/Archive clients are implemented as multi-session clients, which means that they are able to exploit the multi-threading capabilities of modern operating systems. This enables the running of backup and archive operations in parallel to maximize the throughput to the server system. Depending on the client platform, the Backup/Archive client may provide a graphical, command line or Web user interface (see Figure 3). 8 IBM Tivoli Storage Manager: A Technical Introduction

9 Figure 3 Backup/Archive client user interfaces IBM Tivoli Storage Manager: A Technical Introduction 9

10 Many platforms provide all three interfaces. The command line interface is useful for experienced users and allows generation of backup or restore scripts for scheduled execution. The graphical interface is designed for ease of use for the end user for ad hoc backups and restores. The Web client is especially useful for those clients, such as NetWare, where no native GUI is available, or for performing remote backup/restore operations, for example in a helpdesk environment. Some clients (including some UNIX variants and Microsoft platforms) use a new plug-in architecture to implement an image backup feature for raw device backup. This allows you to back up and recover data stored in raw (that is, not a filesystem) volumes. It also provides an additional method to make point-in-time backups of entire filesystems as single objects (image backup) and recover them in conjunction with data backed up by using the progressive backup methodology. IBM Tivoli Storage Manager storage agent The IBM Tivoli Storage Manager storage agent supports LAN-free backup solutions using a SAN infrastructure. The storage agent dynamically shares SAN connected tape libraries and disks with the Tivoli Storage Manager server, and it has the ability to write and read client data directly to and from server-owned storage media. The storage agent receives data objects via the Tivoli Storage Manager API (as discussed in IBM Tivoli Storage Manager externalized interfaces on page 12) and communicates with the Tivoli Storage Manager server over the LAN using TCP/IP to exchange control information and meta-data about the objects being backed up. The data movement itself utilizes the LAN-free path over the SAN to write directly to the storage media. Thus the data movement is removed from both the LAN and the Tivoli Storage Manager server processor for potentially greater scalability. The storage agent is available for selected Backup/Archive clients as well as for backing up popular databases and applications such as MS SQL-Server, MS Exchange, Oracle, DB2, R/3 and Lotus Domino. Server-free data movement IBM Tivoli Storage Manager allows clients to directly back up and restore filesystem images between disk storage and tape devices accessible over a SAN. This server-free data movement is handled on behalf of the Tivoli Storage Manager server by an outboard data mover, such as the IBM SAN Data Gateway. The data mover must be able to execute the SCSI-3 extended copy command. Server-free data movers must have addressability to all the devices involved, which include disk and tape drives. The devices may be attached to the SAN through either direct Fibre-Attach or SCSI devices connected to a data mover. Server-free data movement is supported for Windows platforms (server and client) at the time of writing. Unlike traditional LAN and LAN-free backups and restores, neither the Tivoli Storage Manager server or client is in the data path. Instead, the data mover handles and moves the data for backup and restore. In this way, backup and restore operations do not require resources from client or server processors or from the LAN. Because it will not be copying data, the Tivoli Storage Manager server can potentially handle more concurrent client connections and server operations. In addition, the Tivoli Storage Manager client, since it does not have to read and send data to the server, can handle a greater application load. 10 IBM Tivoli Storage Manager: A Technical Introduction

11 IBM Tivoli Storage Manager administration For the central administration of one or more server instances, IBM Tivoli Storage Manager provides command line or java-based administration interfaces (see Figure 4), also called administration clients. Figure 4 IBM Tivoli Storage Manager administration interfaces Using the unique enterprise administration feature it is possible to configure, monitor and manage all server and client instances from one administrative interface, known as the enterprise console. It includes: Enterprise configuration Administrative command routing Central event logging functions IBM Tivoli Storage Manager: A Technical Introduction 11

12 The enterprise configuration allows Tivoli Storage Manager server configurations to be defined centrally by an administrator and then propagated to other servers. This significantly simplifies the configuration and management of multiple servers in an enterprise. Administrative command routing allows administrators to issue commands from one Tivoli Storage Manager server and route them to other target servers. The commands are executed on the target servers, and the command output is returned and formatted on the server where the command was issued. In an enterprise environment with multiple Tivoli Storage Manager servers, client and server events can be logged to a central management server through server-to-server communications, thereby enabling centralized event management and automation. IBM Tivoli Storage Manager externalized interfaces IBM Tivoli Storage Manager provides a data management Application Programming Interface (API), which can be used to implement application clients to integrate popular business applications, such as databases or groupware applications. The API also adheres to an open standard (XBSA) and is published to allow customers or vendors to implement specialized or custom clients for particular data management needs or non-standard computing environments. In general, we distinguish between IBM Tivoli Storage Manager products for Applications software products and the API exploitation through vendor applications. The IBM Tivoli Storage Manager products are separate program products delivered by IBM to connect business applications, via the Tivoli Storage Manager API to the server. These products are IBM Tivoli Storage Manager for Databases, Mail, ERP, Hardware, and Application servers. Applications such as Oracle, Lotus Notes and Domino, Microsoft Exchange, and Microsoft SQL server, provide their own backup and restore or storage management tools. IBM Tivoli Data Protection product strategy is to interface to these tools to provide backup and restore facilities and functions for these applications. Tivoli Data Protection program products can also exploit the special features of IBM's Enterprise Storage Server within an IBM Tivoli Storage Manager based storage management solution. For more information, see Additional application and hardware support on page 30. Alternatively, some vendor applications exploit the Tivoli Storage Manager data management API by integrating it into their software product itself to implement new data management functions, or to provide backup and archival functionality on additional system platforms. Some examples are IBM CommonStore for R/3, Lotus Domino, and Microsoft Exchange data archival, IBM BRMS/400 to provide an AS/400 backup solution, and SSSI Archive Backup Client for OpenVMS data backup and recovery. IBM's DB2/UDB also works with a direct connection to the IBM Tivoli Storage Manager API. Non-IBM/Tivoli products such as Veritas Backup Exec also can use the API to manage backup data with an IBM Tivoli Storage Manager server. In addition to the externalized interfaces to the server database as described in IBM Tivoli Storage Manager server on page 8, IBM Tivoli Storage Manager offers multiple interfaces for event logging, reporting and monitoring the data management environment. All of the activities of the Tivoli Storage Manager server and client are logged in the server database, and they can also be sent for reporting and monitoring purposes to external event receivers using the event filter mechanism. Potential event receivers are the Tivoli Enterprise framework, SNMP based systems management software packages (such as IBM Tivoli NetView), the Windows event log, and user written applications. To integrate IBM Tivoli Storage Manager storage management with external library management applications, IBM Tivoli Storage Manager offers an external library manager interface. Using this interface, it is possible to integrate the Tivoli Storage Manager server into 12 IBM Tivoli Storage Manager: A Technical Introduction

13 third-party storage management environments. One example of this is for sharing STK tape libraries using the ACSLS program. You can dynamically share tape drives installed in STK libraries between multiple Tivoli Storage Manager servers using the EDT-DistribuTAPE software from Gresham software. More information on this product is available at the Web site: IBM Tivoli Storage Manager supported environments IBM Tivoli Storage Manager server and client software is available on many different operating system platforms and can exploit different communication protocols. Figure 5 summarizes the supported environments at the time of writing. INFORMIX LOTUS DOMINO DB2 MICROSOFT Exchange Server SQL Server APPLE Macintosh ORACLE ITSM for Mail Lotus Notes on AIX Lotus Notes on Windows NT Domino on AIX Domino on Windows NT Microsoft Exchange Server ITSM for Databases Microsoft SQL Server Oracle Backup on AIX Oracle Backup on HP-UX Oracle Backup on Sun Solaris Informix IBM DB2 (included in DB2) BULL DPX/2 300 B.O.S. DPX/20 NAS NDMP DIGITAL OpenVMS TANDEM (SSSI) Guardian Tru64 (ETI) SAP R/3 SYBASE WebSphere (BMC) Application Server SUN Solaris SunOS IBM AIX AS/400 Linux zseries HEWLETT- OpenEdition MVS PACKARD zos HP-UX Linux pseries Linux iseries Tivoli Storage Manager Client Platforms MICROSOFT Windows 98 Windows ME Windows XP Windows NT Server/WS Windows 2000 Windows 2003 LINUX Red Hat NUMA-Q SuSE IBM ESS DYNIX/PTX TurboLinux SILICON DB2 GRAPHICS DB2/R3 IRIX with FlashCopy ITSM for ERP Backup Restore Interface (BACKINT) for Oracle IBM CommonStore Support for Informix, MS SQL Server, DB2 ITSM for Hardware IBM ESS ITSM for Application Servers Websphere Application Server NOVELL NetWare z/vm Linux OS/400 z/os AIX Solaris HP-UX Windows 2000/2003 Tivoi Storage Manager Servers Supported Networks APPC IPX/SPX Named Pipes Shared Memory TCP/IP 3270 Emulation Disk Optical Tape Storage Hierarchy Figure 5 IBM Tivoli Storage Manager supported environments The IBM Tivoli Storage Manager server is available on a wide range of major operating systems. Table 1 shows the current list of server platforms, including versions for Tivoli Storage Manager servers. Check the IBM Software Support Web site for the most up-to-date information on supported levels and fixes. Note that for brevity, only the operating systems supported at the latest Tivoli Storage Manager server level (5.2 at the time of writing, exceptions where specifically noted) are shown in this table. Many earlier OS levels are also supported with earlier server levels. Check the Web site for details. Table 1 IBM Tivoli Storage Manager most recent server platforms Server platforms Operating system level Server version AIX 5.1 or HP-UX 11.0 or (11i) 5.2 IBM Tivoli Storage Manager: A Technical Introduction 13

14 Server platforms Operating system level Server version z/os, OS/390 OS/390 V2R10or later z/os V1R Sun Solaris 8, or Windows Linux x86 Windows 2000 Pro, Server, Advanced, DataCenter (all service packs) Windows 2003 Standard, Enterprise, Datacenter Red Hat Advanced Server 2.1 SuSE 7, 8, Linux zseries SuSE Enterprise Server 8 for zseries 5.2 Linux pseries SuSE OS/400 PASE V5R1 or V5R2 with PASE option (33) 5.2 VM/ESA and z/vm z/vm V3R1, V4R2 or later 5.2, running at V3.1.2 server level Table 2 and Table 3 provide an overview of all available clients at the time of publishing this paper. Other operating system clients may be updated in the future. Check the product information on the IBM Tivoli Storage Manager home page for the latest complete client availability information: There are several variations of UNIX clients. Table 2 details the UNIX clients and the operating system levels that are supported. Note that for brevity, only the operating systems supported at the latest Tivoli Storage Manager server level (5.2 at the time of writing, exceptions where specifically noted) are shown in this table. Many earlier OS levels are also supported with earlier server levels. Check the Web site for details. Table 2 UNIX clients Client platforms Version Operating system AIX or 5.2 (including HSM) HP-UX or 11.1 (11i) Sun Solaris 5.2 7, 8, or 9 (including HSM) Linux x Red Hat 7.2, 7.3, 8, and Advanced Server 2.1 SuSE 7.3, 8, 8.1 and SLES 7,8 TurboLinux 7.5,8.0 OS/390 UNIX System Services 5.2 z/os V1R1 or higher OS/390 V2R8 or higher Linuz zseries and OS/ SuSE Enterprise Server 7 and 8 for S/390 and zseries Linux pseries/iseries 5.2 SuSE IBM Tivoli Storage Manager: A Technical Introduction

15 Client platforms Version Operating system Tru64 UNIX 5.2 (at 5.1 functional level) SGI IRIX 5.2 (at 5.1 functional level) 5.1A 6.5 IBM NUMA-Q (formerly Sequent) 4.2 IBM NUMA-Q PTX Version Table 3 gives the PC and other operating systems (non-unix) that are supported as clients. Note that for brevity, only the operating systems supported at the latest Tivoli Storage Manager server level (5.2 at the time of writing, exceptions where specifically noted) are shown in this table. Many earlier OS levels are also supported with earlier server levels. Check the Web site for details. Table 3 PC and other clients PC Clients Platforms Version Operating Systems Novell NetWare or 6 (with current Novell patches) Microsoft Windows (Intel) 5.2 Windows NT 4.0 with SP5 or SP6a Windows XP Windows 2000 Pro, Server, Advanced, DataCenter (all service packs) Windows 2003 Apple Macintosh 5.2 Macintosh OS X, V OS/ V4R4, V4R5 - via BRMS API client Base concepts This section gives a high level introduction to the base data and storage management paradigms used by IBM Tivoli Storage Manager to implement its functionality. We will cover data protection or backup, record retention or archival, storage management, policy, and security. Backup and archival concepts Backup, in IBM Tivoli Storage Manager terms, means the creation of an additional copy of a data object to be used for operational recovery. As already mentioned, the selection of data objects to be backed-up needs to be done carefully to ensure that, when restored, the data is still usable. A data object can be a file, a part of a file, a directory or a user defined data object like a database table. The backup version of this data object is stored separately in the IBM Tivoli Storage Manager server storage repository. Potentially, you can make several backup versions of the data, each version at a different point-in-time. These versions are closely tied together and related to the original object as a group of backups, and Tivoli Storage Manager manages the retention of these objects in a consistent way. If the original data object is corrupted or lost on the client system, restore is the process of sending a backup version of the data from the server back to the client. Typically, the most current version of the data is normally restored, so Tivoli Storage Manager selects this as the default, but you can choose to restore from any of the existing backup versions. The number and retention period of backup versions is controlled by server policy definitions. Old (extra) IBM Tivoli Storage Manager: A Technical Introduction 15

16 backup versions are automatically deleted as new versions are created, if the number of versions stored exceeds the defined limit, or may be deleted after a certain period of time. Figure 6 shows how policy definitions work with Tivoli Storage Manager. In this case, we have specified to keep a maximum of 6 backup versions of a particular file. The files is backed up via normal daily backup operations each day that it changes. The most recently backed up file version is designated the active backup. All other versions are inactive backups. Tivoli Storage Manager automatically deletes inactive backups when the total number of backup versions stored exceeds the policy limit. In this case, on day 8, the oldest inactive file version (which is actually the file as backed up on day 1) is expired so that at all times, a maximum of 6 backup versions is retained. Tivoli Storage Manager policy definitions also include these parameters separately for files which are deleted from the client. Sample Policy: Retain 5 extra copies plus most recent backup Version Deleted ie expired backup Inactive Copy ie previous backups Active Copy ie Last backup File changes Day Figure 6 IBM Tivoli Storage Manager sample policy For file level based backup, the main difference from many other backup applications is that Tivoli Storage Manager uses the progressive backup methodology. As shown in Figure 7, after the first complete backup, IBM Tivoli Storage Manager then operates with incremental backups only. As a consequence, only those files that are new or that have changed since the last backup will be backed up. IBM Tivoli Storage Manager's file level progressive backup methodology, in comparison with other methods like Full+Incremental or Full+Differential backup schemes, significantly reduces the amount of data being copied and managed, and prevents unnecessary backups of unchanged data to reduce and consolidate the recovery tape-set. As a result, IBM Tivoli Storage Manager offers faster recovery by not restoring multiple versions of the same file, only the data that is actually needed. 16 IBM Tivoli Storage Manager: A Technical Introduction

17 ITSM Progressive Backup Methodology Standard Incremental Standard Differential Day 1 10GB 10GB 10GB Day 2 1GB 1GB 1GB Day 3 1GB 1GB 2GB Day 4 1GB 10GB 10GB Day 5 1GB 1GB 1GB Day 6 1GB 1GB 2GB Day 7 1GB 10GB 10GB Total 16GB Total 34GB Total 36GB Figure 7 Progressive Backup Methodology vs. other backup schemes The reorganization of the physical storage media to store each client s data physically together on a small number of media in order to provide faster access in the case of a complete system recovery is done transparently to the client, and is completely automated on the server using data meta information stored in the server database. IBM Tivoli Storage Manager s adaptive sub-file backup technology implements another powerful method to further reduce the amount of data transferred from the client to the server system. This method enables the backup-archive client (Web client, command line, and GUI) to back up only the changed portion of a file, either on a byte or block level, instead of transferring the whole file to the server every time.this feature helps to overcome bandwidth limitations of the network link, especially for mobile or remote client systems. Figure 8 shows how this feature works. IBM Tivoli Storage Manager: A Technical Introduction 17

18 Client Machine Master File Figure 8 Adaptive sub-file backup Sub Files Only change bytes or blocks are sent. Restore requires master file plus sub-files. At any point in time, IBM Tivoli Storage Manager allows the creation of a complete set of client files (backup set) on the server system using the most recent backup versions stored in the server storage repository. Backup sets, as shown in Figure 9, can be used to retain a snapshot of all client files for a longer period of time (Instant Archive) or for LAN-free recovery of a client system by copying this backup set onto portable media and restoring them locally (Rapid Recovery). 18 IBM Tivoli Storage Manager: A Technical Introduction

19 A Storage Pool A I I A tsm> generate backupset Backup Set Snap shot of active backed up files from one client Stored and managed as a single object via volume history On specific media or server storage (but not within a storage pool) Granularity is file space level It is not a file system image End User client A A A Backup Set IBM QIC-5010 Data Cartridge Figure 9 Tivoli Storage Manager Backup set Archive with IBM Tivoli Storage Manager means creating a copy of a file as a separate object in the storage repository to be retained for a specific period of time. Typically you would use this function to create an additional copy of data to be saved for historical purposes, and therefore, special consideration should be given to ensure that the data format is not dependent on anything. Vital records (data that must be kept for legal or other business reasons) are likely candidates for the archive process. You can specify to delete the original copy of the data on the source system once the archive copy is created on the server. In this way, you can use archive to make additional space available on the Tivoli Storage Manager client system. However, archive should not be thought of as a complete space management function, because transparent automatic recall is not available. You can gain access to archived data by using retrieve to return it to the Backup/Archive client. To locate the archived data within the storage repository, Tivoli Storage Manager allows you to add a description to the data and to form archive packages of related files. You can then use this description field to search the server database for matching packages, to determine which data to retrieve. Therefore, the difference between backup and archive is that backup creates and controls multiple backup versions that are directly attached to the original client file; whereas archive creates an additional stored object that is normally kept for a specific period of time, as in the case of vital records. Storage and device concepts All IBM Tivoli Storage Manager-managed client data is stored in the IBM Tivoli Storage Manager storage repository, which consists of pools of like storage devices, such as disk, tape, or optical devices. The storage repository is controlled by the server, which uses its own model of storage to view, classify, and control these storage devices, and to implement the storage management functionality (see Figure 10). IBM Tivoli Storage Manager: A Technical Introduction 19

20 TSM Client WAN, LAN, SAN Storage Repository Storage Pool Volume Storage Pool Storage Pool TSM Server Migrate Copy Data Objects Storage Pool Relocate Storage Hierarchy Figure 10 IBM Tivoli Storage Manager storage management concept The main difference between the storage management approach of IBM Tivoli Storage Manager and other commonly used systems is that IBM Tivoli Storage Manager storage management concentrates on managing data objects as they exist in the storage pools, rather than just the backup tapes as a whole. Data objects can be sub-file components, files, directories or raw logical volumes that are backed up from the client systems; they can be objects like tables or records from database applications, or simply a block of data that a client system wants to store on the server storage. Each object has an associated management policy bound to it which defines what IBM Tivoli Storage Manager does with that object. To store these data objects on storage devices and to implement storage management functions, IBM Tivoli Storage Manager uses logical definitions to classify the available physical storage resources. Most important is the logical entity called a storage pool which describes a storage resource for one single type of media; for example, a disk partition or a set of tape cartridges. Storage pools are the place where data objects are stored. A storage pool is built up from one or more storage pool volumes. For example, in the case of a tape storage pool, this would be a single physical tape cartridge. To describe how IBM Tivoli Storage Manager can access those physical volumes to place the data objects on them, IBM Tivoli Storage Manager uses a logical entity called a device class. A device class is connected to a storage pool and specifies how the server gains access to volumes of this storage pool. IBM Tivoli Storage Manager organizes storage pools in one or more hierarchical structures. This storage hierarchy can span over multiple server instances and is used to implement management functions to migrate data objects, automatically and transparently to the client, from one storage hierarchy level to another; or in other words, from one storage device to another. This function may be used, for example, to store backup data (for performance reasons) onto an IBM Tivoli Storage Manager server disk space before moving the data to tape cartridges. The actual location of all data objects at all times is automatically tracked within the server database. 20 IBM Tivoli Storage Manager: A Technical Introduction

21 IBM Tivoli Storage Manager has implemented additional storage management functions for moving data objects from one storage volume to another. As discussed in the previous section, IBM Tivoli Storage Manager uses the progressive backup methodology to back up client files to the storage repository. Once in the repository, functions are provided within the server to reorganize data and storage media to facilitate fast and effective recovery. These Tivoli Storage Manager functions relocate data objects from one volume to another, and collocate data objects that belong together, either at the client system level or at the data group level. Collocation is shown in Figure 11. A A B B C C A A B B C C A B C A B C A C B A B A C B C A A A B B B C C C Collocation off Collocation on Figure 11 Tivoli Storage Manager collocation Another important storage management function implemented within the server is the ability to copy client data objects (either asynchronously or concurrently with the client backup operation) and to store them in different storage pools. These copy storage pools can be created on local tape drives and taken off-site, on remotely accessible tape drives, or on another server completely. This provides additional copies of the stored data in a secure place, which is available to be recovered in the event of losing individual storage media or even the whole storage repository. This function is fully transparent to the client, and is performed and tracked automatically within the IBM Tivoli Storage Manager server. Figure 12 illustrates the copy storage pool function. IBM Tivoli Storage Manager: A Technical Introduction 21

22 C B A C B A C B A C B A Figure 12 Tivoli Storage Manager copy storage pools Policy concepts A data storage management environment consists of three basic types of resources: client systems, rules, and data. The client systems contain the data to be managed, and the rules specify how the management must occur; for example, in the case of backup, how many versions should be kept, where they should be stored, and so on. IBM Tivoli Storage Manager policies define the relationships between these three resources. Figure 13 illustrates this policy relationship. Depending on your actual needs for managing your enterprise data, these policies can be very simple or very complex. 22 IBM Tivoli Storage Manager: A Technical Introduction

23 Policy Domain Client Nodes Policy Set Mgmt Class (default) Backup, Archive and HSM Data Management Rules Data Mgmt Class Backup, Archive and HSM Data Management Rules Mgmt Class Backup, Archive and HSM Data Management Rules Figure 13 Tivoli Storage Manager policy relationships and resources IBM Tivoli Storage Manager has certain logical entities that group and organize the storage resources and define relationships between them. Client systems, or nodes in Tivoli Storage Manager terminology, are grouped together with other nodes with common storage management requirements, into a Policy Domain. The Policy Domain contains a logical structure called a Policy Set. A Policy Set contains and helps to manage a collection of storage management rules for different storage management activities. The rules are stored, within the Policy Set, in one or more Management Classes. A Management Class contains the rule descriptions (actually stored in entities called Copy Groups), which are linked to the stored data objects. The rules are really just a set of storage management parameters, such as number of stored copies, retention period, storage media, and so on. When a data object is linked to particular rules, it is said to be bound to the management class that contains those rules. Another way to look at the components that make up a policy is to consider them in the hierarchical fashion in which they are defined. Consider the policy domain at the top, containing at least one policy set which contains many management classes. The management classes contain the copy groups and the storage management parameters and it is the management classes that the Tivoli Storage Manager client can use to select how particular data objects are to be stored. IBM Tivoli Storage Manager: A Technical Introduction 23

24 Security concepts A Policy Domain contains: One Active Policy Set, which contains: One default Management Class, which contains: Some HSM Definitions A Backup Copy Group, specifying: - Where to store data objects - How many versions to store - How long to store them An Archive Copy Group, specifying: - Where to store data objects - How long to store them Many other Management Classes, each containing: Some HSM Definitions A Backup Copy Group An Archive Copy Group A good practice, when designing a Tivoli Storage Manager backup policy, is to consider using either the number of versions or the retention period. As mentioned above, Tivoli Storage Manager s backups are expired based on whichever parameter is matched first (versions or time) so, in order to achieve a consistent restore capability (that is, restore to one of X previous versions or restore to any point of backup within Y days) think carefully about the values you use. The storage repository of IBM Tivoli Storage Manager is the place where all the data of an enterprise is stored and managed. Clearly therefore, security is a key aspect of Tivoli Storage Manager. To ensure that only the owning client or an authorized party can gain access to data objects, Tivoli Storage Manager implements, for authentication purposes, a mutual suspicion algorithm, which is similar to the methods used by Kerberos authentication. Whenever a client (backup/archive or administrative) communicates with the server, an authentication has to take place. This authentication contains both-sides verification, which means that the client has to authenticate itself to the server, and the server has to authenticate itself to the client before any data objects are exchanged. To do this, all clients have a password and a userid, which is stored at the server side as well as at the client side. In the authentication dialog these passwords are used to encrypt the communication. The passwords are not sent over the network, to prevent hackers from intercepting them, and a new key is used for each encryption. A communication session will be established only if both sides are able to decrypt the dialog. If the communication has ended, or if a timeout period without activity is passed, the session will be automatically terminated and a new authentication will be necessary. In mobile computing environments, files are often sent to the Tivoli Storage Manager server system using a modem connection, and so they are exposed to the security hazards of public telephone lines. The Backup/Archive client optionally provides (in addition to the end-point security concept outlined above) a data encryption function, which allows for encrypting data before it is sent to the server, and which protects the data while it is being transferred to the server and also while it resides in the storage repository. 24 IBM Tivoli Storage Manager: A Technical Introduction

25 IBM Tivoli Storage Manager complementary products IBM Tivoli Storage Manager complementary products use the Tivoli Storage Manager server as a backbone product to implement additional data and storage management functions. In the following section we introduce IBM Tivoli Space Manager for hierarchical space management, IBM Tivoli Disaster Recovery Manager as an enterprise-wide solution for disaster recovery, and IBM Tivoli Decision Support for Storage Management Analysis for a comprehensive reporting and monitoring solution to plan the growth and collect vital management information for an efficient enterprise data management deployment. IBM Tivoli Space Manager IBM Tivoli Space Manager uses the Tivoli Storage Manager framework services together with the industry standard Data Management Application Programming Interface (DMAPI) to deliver a fully integrated solution for open systems Hierarchical Space Management (HSM). IBM Tivoli Space Manager provides an HSM client, which interfaces with DMAPI and implements the functionality outlined in Figure 14 for certain Tivoli Storage Manager clients. HSM Client Server Migrate IBM Tivoli Storage Manager HSM Server End User Recall Integrated Backup Storage Usage Figure 14 IBM Tivoli Space Manager for HSM IBM Tivoli Space Manager maximizes usage of existing storage resources by transparently migrating data from client hard drives to the Tivoli Storage Manager storage repository based on size and age criteria. When the migrated data is accessed, IBM Tivoli Space Manager transparently recalls it back onto the local disk. The migration of files and the management of migrated files is controlled by policies. Manual (user controlled) migration and recall is also possible. IBM Tivoli Space Manager s HSM function is fully integrated with Tivoli Storage Manager operational backup, so that if a client requests to back up a file which has already been migrated to the server, it will be copied to a backup storage pool internally, thereby avoiding a client data recall. The HSM function through IBM Tivoli Space Manager is available for AIX and Solaris clients. IBM Tivoli Storage Manager: A Technical Introduction 25

26 Disaster Recovery Manager Disaster Recovery Manager (DRM) is a feature of the IBM Tivoli Storage Manager Extended Edition. and coordinates and automates the process of recovering from a disaster. It provides for offsite media management, automated restore of the Tivoli Storage Manager server and managed client recovery. It complements the already implemented robust protection features of Tivoli Storage Manager and automates many already facilitated protection functions. DRM automatically captures information required to recover the Tivoli Storage Manager server after a disaster. It assists in preparing a plan that allows recovery in the most expedient manner. This disaster recovery plan contains information, scripts, and procedures needed to automate server restoration, and helps ensure quick recovery of your data after a disaster. DRM also manages and tracks the movement of off-site media to reduce the time required to recover in the event of a disaster. It is able to track media that are stored on-site, in-transit, or off-site in a vault, no matter whether it is a manual or electronic vault, so your data can be easily located if disaster strikes. The DRM media cycle is shown in Figure 15. Scratch Pool Mountable Check-in Not Mountable Courier Courier Retrieve Vault Vault Retrieve Figure 15 Disaster Recovery Manager offsite media tracking cycle Client recovery information can also be captured by DRM. This information can be used to assist with identifying what clients need to be recovered, in what order, and what is required to recover them, including data and media that is not managed by Tivoli Storage Manager. NDMP IBM Tivoli Storage Manager Extended Edition provides support for certain Network Attached Storage (NAS) appliances using Network Data Management Protocol (NDMP) to perform high-performance, scalable backups and restores at the volume and file level. These backups and restores minimize network traffic by transferring data independently of the Tivoli Storage 26 IBM Tivoli Storage Manager: A Technical Introduction

27 Manager client and server where available. NDMP support enables both full and differential file system image backups and restores of Network Appliance file servers with operating system Data ONTAP or higher. Multiple backup and restore operations can be performed simultaneously. NDMP support is implemented through a combination of the Tivoli Storage Manager Extended Edition server and Backup/Archive client code. No additional code need be installed on the server, client, or NAS appliance. during backup and restore operations, data flows directly between the NAS appliance and the tape drive. Either a SCSI-attached tape device local to the NAS appliance, or a SAN-attached SCSI or ACSLS device, shared with the Tivoli Storage Manager server can be used. Bare metal recovery Bare metal recovery (BMR) is the ability to restore a system completely, assuming that all data and configuration information has been erased from the hard disk. There are a number of products and techniques which can be used to provide BMR in association with IBM Tivoli Storage Manager product capabilities. For Windows clients, BMR can be achieved using the regular Backup/Archive client. The basic technique is to re-install the operating system and then restore all additional files and configuration information using the Tivoli Storage Manager client. An alternative method is to use a third-party volume imaging product to capture the complete system information in a single object. Various third-party products can or will offer the capability to store the captured volume images on the Tivoli Storage Manager server. For UNIX clients, a re-install of the OS together with Backup/Archive client restore of additional data provides a solution. In the case of AIX, IBM Tivoli Storage Manager for System Backup and Recovery (also known as SysBack) provides a complete backup and restore solution including full system (installation) images, volume groups, raw logical volumes and filesystems. See IBM Tivoli Storage Manager for System Backup and Restore on page 34 for more information. A more detailed discussion of bare metal recovery techniques on various platforms is contained in the redbook Disaster Recovery Strategies with Tivoli Storage Management, SG IBM Tivoli Decision Support for Storage Management Analysis IBM Tivoli Decision Support for Storage Management Analysis uses the framework services of IBM Tivoli Decision Support to deliver important decision-making information about your enterprise data management deployment. IBM Tivoli Decision Support is a stand-alone product that provides a ready-to-use view into the wealth of data gathered by Tivoli enterprise products. The product consolidates this data and transforms it into accessible IT business-relevant information. This information, presented in a variety of graphical formats, can be viewed interactively (slice, dice, drill down, drill through) and posted on a URL. IBM Tivoli Decision Support provides insight and the ability to better answer IT business-relevant questions. IBM Tivoli Decision Support is available on Windows NT and Windows The IBM Tivoli Decision Support Discovery Guides are a set of best practices guides provided for various applications. To use these guides, IBM Tivoli Decision Support has to be installed IBM Tivoli Storage Manager: A Technical Introduction 27

28 and available. The Tivoli Decision Support for Storage Management Analysis is the guide used to produce the following analyses: Storage Event Analysis Storage Performance Analysis Storage Capacity Analysis A typical IBM Tivoli Decision Support for Storage Management Analysis graph is shown in Figure 16. Figure 16 IBM Tivoli Decision Support for Storage Management Analysis The information used by the guide is obtained directly from the Tivoli Storage Manager server with the use of the ODBC interface. The information is then transferred to a relational database, as required by Tivoli Decision Support for Storage Management Analysis, such as DB2, MS SQL-Server, and Oracle. The database can reside on the same system as IBM Tivoli Storage Manager or IBM Tivoli Decision Support or on a separate system. The database is used for queries to generate the IBM Tivoli Decision Support reports. Tivoli SANergy Tivoli SANergy is a product which allows filesystems to be shared across a Storage Area Network. SANergy provides a function, known as a MetaData Controller (MDC), which owns the filesystem. Any SANergy client may gain access to that storage across the SAN by first requesting the MDC for permission, and for the location of the data. After the MDC 28 IBM Tivoli Storage Manager: A Technical Introduction