Brendan Lelieveld-Amiro, Director of Product Development StorageQuest Inc. December 2012

Transcription

1 Automated archiving using IBM Tape Libraries and StorageQuest Archive Manager Automated archiving made easy using volume spanning with StorageQuest Archive Manager and an IBM Tape Library Brendan Lelieveld-Amiro, Director of Product Development StorageQuest Inc. December 2012 Copyright IBM Corporation, 2012

2 Table of contents Abstract...1 Introduction...1 Comparing data archiving with data backups...1 Why do organizations need to archive?...2 Benefits of archiving to a tape library...3 LTO roadmap...3 How the StorageQuest Archive Manager works with IBM Tape Libraries?...4 SAM and the Linear Tape File System...5 Tape archiving system overview...5 Summary...6 Resources...7 About the author...7 Trademarks and special notices...8

3 Abstract In this white paper, Brendan Lelieveld-Amiro, Director of Product Development at StorageQuest discusses about using the StorageQuest Archive Manager volume spanning feature across Linear Tape-Open ( LTO) media in providing a scalable archiving solution. Introduction Many organizations are faced with growing amounts of information. There is an increasing need to cost effectively retain data for longer periods of time, for the purposes of both legal requirements such as Sarbanes-Oxley or practical requirements such as maintaining customer records for long periods of time. StorageQuest Archive Manager (SAM) provides automated management and archiving of data to tape libraries using volume spanning among many LTO media. IBM Tape Libraries provide a cost-effective method to store data as these tape libraries have a very high capacity and consume lesser energy compared to other technologies. This white paper explains about archiving and the benefits of archiving with the StorageQuest Archive Manager software, IBM Tape Libraries, the IBM Linear Tape File System and LTO media. StorageQuest Inc., a global leader in data and archive management software and hardware in collaboration with IBM has developed a powerful tape-archiving management software. StorageQuest has certified the IBM Tape Libraries as a back-end storage device with StorageQuest Archive Manager. Comparing data archiving with data backups Archiving and backup are terms that are sometimes interchanged, although they represent two very different methods of storing data. Archiving is the process of moving data that is no longer actively used to a secondary storage location, whereas a backup creates a copy of data that is currently being used, for the sake of recovery. Archives contain data that might not need to be maintained on costly online storage, such as spinning platter Redundant Array of Independent Disks (RAID) systems, but might need to be retrieved at a later time. Typically, archives residing on a less-expensive disk or a passive tape are used to save on energy costs by not requiring an active storage medium for legal requirements, or even to free up space on production systems. Archives are typically indexed manually for easy retrieval. Indexing enables faster retrieval by human operators. The StorageQuest Archive Manager software tracks all cartridges, folders, files, and metadata written to the archive so that they can be easily retrieved by using the advance search feature or by browsing the virtual drive. Data backups are used to restore data, typically for the purpose of disaster recovery. As the backup instances are actively changing, the data is not removed from the source system and the backup is used only if needed. The StorageQuest Archive Manager software is capable of performing backups, although is designed specifically for archiving. 1

4 Why do organizations need to archive? Most organizations archive data because regulations require that they retain specific types of data for a specific period of time or have a need to reclaim space on their more-expensive primary storage medium, such as a RAID server. There are a growing number of regulations, and the need for organizations to be in compliance, is driving data retention. Examples include Security and Intelligence, Government, Healthcare, Finance, Insurance, and Legal. In the United States, regulations such as the Sarbanes-Oxley Act, HIPAA, and Basel III are some of the laws governing how long data must be retained. Until recently, most organizations dealt with mandatory retention periods of only a few years for important business data. Some organizations now are required to maintain data for upwards of 30 years. One of the challenges that organizations face is the sheer size of data growth annually. Organizations must now have policies in place to move data off of their higher cost storage primary storage devices to long-term retention storage devices, such as IBM Tape Libraries to reclaim that space. Internal corporation policies provide a reasonable time to store s, files, and other business data and archiving frees up valuable company IT resources. With less data on their primary storage devices, business systems and applications perform better. An obvious reason also is to keep cost down. Archiving is a cost-effective way to manage data rather than having to constantly increase the storage capacity of their primary storage medium. 2

5 Benefits of archiving to a tape library The IBM Tape Library can store large amounts of data for long-term archiving. Irrespective of whether the data is near-line or off-line, it can be easily retrieved. LTO tape cartridges have a life span of 15 to 30 years, as compared to the current disk lifespan of approximately 3 to 5 years. Archiving to a tape library provides a low total cost of ownership (TCO) solution. The cost per gigabyte is low and future expansion is considerably lower compared to other archiving storage methods. Energy consumption for power and cooling is anywhere from 20 to 100 times less expensive than disk storage, said Bruce Master, Senior Program Manager, Worldwide Tape Storage Systems Marketing at IBM providing a Green Solution. IBM developed Linear Tape-Open (LTO) cartridges offer high capacity storage and speeds. The LTO cartridge roadmap shows substantial gains in capacity and speed LTO roadmap Generation Attribute LTO-5 LTO-6 LTO-7 LTO-8 Release date 2010 December 2012 TBA TBA Native data capacity 1.5 TB 2.5 TB 6.4 TB * 12.8 TB * Maximum speed (MBps) * 472 * Table 1: LTO tape specification (Source Ultrium LTO) * Formal specification has not been finalized at the time of publishing this paper. 3

6 How the StorageQuest Archive Manager works with IBM Tape Libraries? The StorageQuest Archive Manager software allows administrators to create virtual volumes representing one or many tape cartridges in one or multiple IBM Tape Libraries. Files and folders copied into each volume are automatically written out to the tape media in the background and spanned across as many physical media cartridges as necessary. This is completely transparent to the client and application. The volumes show a concatenated view of all the tape cartridges in one file hierarchy structure. The SAM software routes all read and write requests to the corresponding cartridge giving the illusion that the client application is accessing a very large storage device even though the read and write requests are targeted to individual cartridges. StorageQuest Archive Manager creates a virtual drive letter for each of the tape libraries connected to a network. Files can be written to the media in various ways. Virtual drive letters allow clients to drag important files and folders from local hard drives or networks to the tape library. Alternatively, a user can synchronize the data of specified folder locations on the network or a local server by having the SAM software to pull the new or modified files from the primary storage and write to the tape library in the background. Applications can also access this drive letter as it appears as a regular hard drive. Files and folders added to the drive letter are sent to the write cache. The write cache allows for acceptance of data destined for more tapes and volumes than drives available in the library. This writing (flushing) process is controlled by several trigger conditions, where the software waits to write files until it has either a certain number or size of files total, before writing. The same is true with the number of files. This is done to prevent tapes from being exchanged between drives too frequently. SAM also uses specialized provisions to accommodate temporary files and files that are modified frequently to be filtered from writing. SAM contains an easy-to-read GUI with a powerful search capability. Therefore, users can locate files written to any tape across any volume created with the system, whether the tape cartridge is nearline or offline. Searching is accomplished using a number of useful criteria, such as which tape the files are on; the status of the files; whether they have been written or not, and using typical search criteria, such as date modified, size, and file name. When the SAM service requires user intervention for events, such as media to be added or offline media to be reinserted for data retrieval, SAM provides a friendly mail-style balloon message to the client administrator locally or as a standalone application across the network. The software has the ability to replicate data copied to any volume to any other volume, or network drive. This replication is controlled through volume properties, allowing customers to mirror files to another SAM volume, or to any other network location. This replication also moves files to an off-site storage device for disaster recovery or creates duplicates and sends them offsite. 4

7 SAM and the Linear Tape File System The IBM developed Linear Tape File System is an open format file system specifically written for use with tape drives, libraries and LTO-5 media. As this file system is not proprietary, the tapes written can be read back in virtually using any modern operating system with an LTO drive, without using any third-party software. StorageQuest uses the standard medium changer command specification for controlling and managing the IBM Library robotics to move cartridges between slots, drives, and mailslots. After the tapes are loaded in to the tape drives, StorageQuest uses IBM Linear Tape File System utilities and drivers for identification, erasing, high-level formatting, and reading and writing of files and folders. As the Linear Tape File System software and drivers are integrated into the SAM service, all the operations are performed in the background without requiring any user intervention. The IBM Linear Tape File System utilities and drivers provide a high-performance, reliable, and portable base for the archiving system. Tape archiving system overview Figure 1 provides an overview of the tape archiving system. Figure 1: Tape archiving system overview 5

8 Summary Organizations continue to retain more data over longer periods of time. An organization s data is not only a valuable asset, but also required to be maintained for compliancy reasons. The combination of StorageQuest Archive Manager, Linear Tape File System, and IBM Tape Libraries provide a scalable long-term archive solution that can protect customers data for a long time. 6

9 Resources The following websites provide useful references to supplement the information contained in this paper: IBM Systems on PartnerWorld ibm.com/partnerworld/systems IBM Redbooks ibm.com/redbooks StorageQuest Inc. IBM ISV Solutions Resource Library ibm.com/systems/storage/solutions/isv/index.html About the author Brendan Lelieveld-Amiro is the Director of Product Development at StorageQuest Inc. and has spent over 15 years in the Storage and Archiving industry. 7

10 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. 8

11 Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 9