Alternative Storage Technologies. Whitepaper

Transcription

1 Alternative Storage Technologies

2 What does «digital» really mean? When a file is stored digitally, the data is stored as binary codes, i.e. sequences of numbers (0 and 1,) which are written on to a storage medium. By decoding these numbers, a perfect copy of the original file is brought back. Digital storage, including the Cloud, normally means digital application of analogue storage medium. THE COMPLEX NEEDS OF PRESERVATION The world is overflowing with digital data; in fact the digital universe is doubling every two years according to IDC 1. A share of this digital universe has value in the long term. This is for instance data that is considered part of our cultural heritage, strategic business information or data that must be preserved for legal reasons. In order to ensure long term survival, these data need to be treated differently than short-lived information. Do the existing storage technologies meet the complex requirements involved in long-term preservation of data? What are the unique characteristics of the available alternatives? These are some of the issues we will look into. In our evaluation of the alternatives we will in particular focus on the following aspects: How well is data security and integrity maintained? How easy will it be to access the data in the future? How cost-efficient is the technology in a long term perspective? 1 IDC, The Digital Universe of Opportunities, April

3 MAGNETIC TAPE What it is: Magnetic tape is an analogue medium for magnetic recording, made of a thin magnetizable coating on a long, narrow strip of plastic film. The plastic film is coated with magnetic material, i.e. ferric oxide powder which is mixed with a binder to attach it to the plastic. It includes a dry lubricant to avoid damaging the tape drive. The magnetic coating makes it possible to retain electronically encrypted data in digital format. The tape is normally packaged in a plastic cartridge for protection and tape drives are used to read and write the data. Cheap mass storage: Magnetic tape allows large amounts of digital data to be stored at a relatively low cost, making it well suited for a mass storage purpose. Handle with care: When using tape for archiving purposes one has to bear in mind that tape is a sensitive material that requires proper handling and environmental conditions. For instance, magnetic fields are obviously a threat to any magnetic storage, and smoke and small particles can cause damage and loss of data. It is therefore imperative to have clean operating conditions, with the right temperature and humidity levels. With minimal usage, handled properly and stored in optimal conditions, magnetic tapes have a lifetime expectancy ranging between 10 to 30 years. 2 However, due to the delicate material and its high density of storage, the risk of damage and failure increases each time the tape is used. Consequently, the usage should be Linear Tape Open (LTO) is a magnetic tape storage open technology that was developed in the late 1990s by HP, IBM and Certance (now Quantum). LTO is widely used for mass storage, and the latest generations (LTO 7) can store up to 6 terabyte. For long-term preservation purposes, it is important to be aware that data capacity increases for each generation, but this has a negative impact on the expected lifetime. As for other magnetic tapes, LTO requires a migration-based archiving strategy in order to ensure data safety and accessibility. kept to a minimum, and it is considered best practice to migrate the data at least every 5-10 years to newer tape formats. Migrations and vendor lock-in: A drawback with magnetic tape is that data retrieval is dependent on specific reading devices, i.e. tape drives, which need to be maintained. New tape drives are normally not able to bring back data from older generations of tapes 3. This means that when using magnetic tape for long term preservation, regular and endless investments in migration is needed. 2 Fujifilm, LTO Ultrium Technology 2012, 3 LTO Ultrium readers are two generations backward compatible

4 OPTICAL DISCS What it is: Optical discs are flat circular disks that encode digital data on one of its flat surfaces. They are usually made of aluminium, but for preservation purposes it is preferred that the reflective layer of the optical disk is gold 4. Optical disc drives (ODD) with laser beams are used for engraving the data onto the disc, and the laser light is also used for reading back the data whilst the disk spins at high speeds. CDs and DVDs: The two main types of optical discs used by archivists are CDs and DVDs. These can be further split into: CD-R and DVD-R: Write once, read many CD-RW and DVD-RW: Rewritable DVD-RAM: Rewritable discs formatted for random access (like a computer hard disc) Blu-Ray is an optical disc designed to supersede the DVD format with a higher capacity. However, it is more popular among households than professional archivists. M-Disk is an optical disc which was introduced in 2013, claiming a lifetime of 1000 years. However, a Blu-Ray reader is needed for data retrieval, and the future accessibility of these is a challenge the manufacturer does not give an answer to. Ease of use: Optical discs such as CDs and DVDs are easy to use, and both the media and the hardware are relatively inexpensive to purchase. In addition, the laser-based device reader is not in contact with the disc which makes mechanical failure less likely, and most of the readers are backward compatible. These factors have made optical discs widely used by archivists. High failure rates: Originally developed primarily as a mass consumer product, elaborate measures are required when using optical discs for long term preservation. Optical discs have a relatively high risk of failure 5, and it is strongly recommended to always do thorough testing to ensure required standards are met. Scratching and environmental factors such as dust, heat or UV light can cause severe damage to the disks. Optical discs also offer low data capacity compared to alternative technologies. Frequent migrations: It is challenging to reliably estimate the real lifetime of optical discs. They are fragile, and the life span depends on factors such as manufactured quality, how well it is recorded and its physical handling and storage. To lower the risk of failure, best practice is to frequently migrate onto newer formats. This is in sharp contrast to manufacturers who tend to claim an expected lifetime of up to several hundred years. 4 DPC; Handbook in Digital Preservation ; 5 Risks Associated with the Use of Recordable CDs and DVDs as a Reliable Storage Medium in Archival Collections Strategies and Alternatives, Memory of the World Programme, UNESCO, 2006;

5 HARD DISK DRIVE (HDD) What it is: A hard disk is a rapidly rotating disk used for storing digital information. The data is recorded by magnetizing a thin film of ferromagnetic material. A read/write head on an arm accesses the data when the disk is spinning. Instant access: Hard disks provide instant access to the data and the reading and writing device is an integrated part of the storage medium. They enable cost-efficient storage of large amounts of data, with a storage capacity of up to 10 terabytes per disk (2016). Short durability: High failure rates make hard disks inappropriate for long-term preservation. A recent study projects an average lifespan of six years. 6 It is therefore common to combine a number of disks for redundancy, also known as a RAID. From a mechanical point-of-view, a frequent issue is that the disk will not spin, and the failure rates increase with age. Another common reason for failure is Solid-State Drives (SSD) use flash memory for digital storage, unlike HDD s magnetic technology. They are more mechanically robust than HDDs and require less power, but have less data capacity and are more expensive in use. SSD is well suited for portable devices and applications requiring fast data access, but are not considered to have the attributes required for long-term digital preservation. that the read/write head scrapes the rotating platter and thereby causes damage and data loss. Furthermore, a hard drive is a magnetic storage technology and its magnetic strength will naturally deteriorate over time when placed offline for archiving upon a shelf. 6 retrieved November 21st, 2013

6 THE CLOUD What it is: The Cloud is a term used for storage services where the data is stored in virtualized pools operated by third parties. These hosting companies operate large data centres, filled with servers, which could be located anywhere in the world. They are normally subject to strict security measures such as restricted access, environmental control and emergency backup power supply. Store and ignore: For the end user, the Cloud offers clear advantages such as instant access, ease of use with no need for maintenance, and it is inexpensive to buy or lease storage space. This makes it well suited for basic storage needs. However, for the valuable data that requires secure, Amazon Glacier is an online file storage service developed for long-term storage of data that does not require instant access. Like other Cloud services, Glacier is based on hard drives, but the data retrieval time is 3 to 5 hours according to Amazon. Storage is cheap, but users are charged for retrieving the data hence the risk of vendor lock-in should be considered. The privacy, security and ownership concerns related to cloud storage also apply to Amazon Glacier. long-term preservation the Cloud involves some major concerns related to security, privacy and ownership. Ownership and privacy concerns: The Cloud involves entrusting a third party to store the data. This means the service provider has access to the data and may accidentally or deliberately disclose or alter the information. The service provider may also go out of business, posing questions as to who owns the servers and the data on them. The data owner will in this situation have limited control, and the data is inaccessible from the moment the network connection or power supply is switched off. Security issues: Data accessible through the web is vulnerable to hacking. Although the service providers strive to increase the security level, there is no guarantee that the data is out of reach of individual hackers, companies or national security agencies that might have an interest in accessing the information. Some data centres offer offline back-up, normally in the form of magnetic tape, in addition to cloud storage. This makes the data more secure, but a migration-based archiving strategy is then needed when the time perspective is long. Data centre with servers

7 MICROFILM What it is: Microfilm is 16mm or 35mm film stored on open reels or in cassettes. It contains small images, usually in black and white, although some microfilm formats also support digital data. The two main types of microfilms are: Silver halide film: The recommended alternative for long-term preservation. The image is transferred to the film by using silver emulsion on a polyester strip. Vesicular film: Creates the image on the polyester strip by using microscopic bubbles instead of silver emulsion, making it a less expensive but also a less durable solution. Special cameras capable of photographing at reduced size are used in order to transfer an image to microfilm. The image is then printed on the film and chemically processed in a laboratory. The film processing makes the recording process more complex compared with some of the alternative technologies. In spite of this, microfilm is widely used for long-term preservation of data, for a number of reasons. A microfiche has many of the same attributes as microfilm, but is formatted as a card rather than as a film reel. They are normally stored in open top envelopes in drawers or boxes. An ultrafiche is a compact version of a microfiche, storing images at much higher densities and often made directly from computers. A migration-free WORM: The durability and secure read-back of information written as images make microfilm a migration-free storage alternative, potentially creating large cost savings and eliminating the risk of migration-related data loss. By being an analogue medium stored offline, microfilm is also a secure solution. As a true WORM (Write Once, Read Many) it is literally impossible to edit or delete the information that is written to the film. Hence microfilm meet many of the criteria for long-term preservation. Manual handling: The major drawback with microfilm is that the workflow for reproducing data is time-consuming and largely manual. Microfilm was designed for storing analogue images, and in a world overflowing with digital data, traditional microfilm is simply not an efficient alternative. Long durability and no need for decoding: Microfilm has a lifetime expectancy of up to several hundred years when stored properly at the recommended temperature and humidity levels. Additionally, as the data is normally analogue, data stored on microfilm can be read back by simply using a magnifying glass. Hence data retrieval is independent of specific reading devices, and the images on the film require no software decoding. Desktop readers with large screens and zooming lenses are normally used for data retrieval. The downside is that it is a manual and cumbersome process to access and reproduce data stored on traditional microfilm. Microfilm reader

8 PIQL S PRESERVATION TECHNOLOGY What it is: Piql offers Preservation Services based on a turnkey system designed to comply with the needs of long-term digital preservation. The result is a technology that allows digital data to be preserved safely and efficiently, and easily retrievable independent of future access to specific technologies or vendors. Piql uses high-resolution photosensitive film as a digital storage medium. Data is written to film as large QR-codes, each containing 8.8 million pixels. This allows any kind of data to be preserved offline, on a storage medium with a documented lifetime of 500 years. Piql has developed all software and hardware needed to write and read back data. Yet there is no vendor lock-in as the decoding software is open source and the data can be retrieved by using any digital camera and computer available in the future. Future-proof WORM: Piql s solution offers many of the same preservation qualities as microfilm, but applied to digital data. As a true WORM stored offline, it is impossible to manipulate or delete data once it is written. Digital data written as 0 and 1 obviously need decoding to get back the original file. To make data retrieval future-proof, explanations of how to decode and retrieve the information is written as human readable text on Data preserved in digital and visual formats

9 the storage medium. By being self-contained with a documented lifetime of 500 years, valuable data can be preserved for the unforeseeable future. The technology is also migration-free, both in terms of storage medium, reading device and file formats (provided that archival formats are used). The latter is due to the fact that file format specifications can be written in readable text on the storage medium. For digital and visual formats: Piql s technology differs from traditional microfilm in enabling digital data to be stored alongside visual images, making it suitable for the digital era we have entered. The flexibility in combining digital and visual formats opens up for new possibilities and additional security, such as visual previews of digital files. (retrieval time will vary according to each client s preferences). Migration-free long-term preservation: Piql s preservation technology is not recommended for data that needs to be instantly accessible to ensure business continuity. It is rather a secure, migrationfree and future-proof option for valuable data that needs to be preserved long-term. Piql s film and box are tested to ensure the preserved data will be kept intact for hundreds of years. Search and access files: Piql s technology allows for metadata searches and accessibility on a different level than traditional microfilm. As a seamless element within a standard IT infrastructure, users can search for the requested file and get it back in original format within minutes

10 DIFFERENT TECHNOLOGIES FOR DIFFERENT NEEDS We have seen that each of the storage technologies have different characteristics, making them suitable for different needs. Data owners need to thoroughly consider their requirements in terms of e.g. acceptable data retrieval time, security level and budget limitations during the archival period. The following table provides an overview of some of the attributes of each of the storage technologies. It is important to keep in mind that this is a general overview of technologies constantly in development. Within each category, there are differences between vendors and products.

11 Magnetic Tape Optical Disks Hard Disk Drive Cloud Microfilm Piql Preservation Services Projected lifetime N/A Digital data Analogue data 8 True WORM 9 10 Offline format 11 Instant retrieval Vendor-neutral reading device Migration-free storage medium True archival format 12 7 General figures, there are variations from vendor to vendor 8 Human readable text or pictures 9 Write Once, Read Many, meaning the data is unalterable after being recorded on the storage medium 10 CD-R and DVD-R only 11 Does not require any maintenance of the medium to ensure data integrity 12 Self-contained format: descriptions of how the data is stored, file formats, information on how to decode the data, etc.

12 Piql secures long-term future access to valuable digital data. Head-quartered in Norway since 2002, Piql Preservation Services are offered to clients around the world by the use of technology that converts high-resolution film into a digital preservation medium for the future. Piql AS Grønland 56, N-3045 Drammen, Norway tel office@piql.com