DNA Evolution 3.5 Administrator Guide 2014 StorageDNA, Inc. All rights reserved. The owner or authorized user of a valid copy of DNA Evolution may reproduce this publication for the purpose of learning to use such software. No part of this publication may be reproduced or transmitted for commercial purposes, such as selling copies. StorageDNA, DNA Evolution and DNA Sync are trademarks of StorageDNA, Inc. Third-party trademarks are the property of their respective owners. Page 1 of 63
TABLE OF CONTENTS Introduction... 6 The Architecture... 6 DNA Evolution Software Components... 6 DNA Evolution Controller... 6 DNA Evolution Archive Manager... 6 DNA Evolution Client... 6 DNA Evolution Hardware Components... 7 DNA Evolution Linux Appliance... 7 Tape Hardware... 7 Understanding DNA Evolution Archiving, Backup, and Nearline Workflows... 7 Understanding How DNA Evolution Writes Data and Metadata... 7 Understanding DNA Evolution Data and Command Flow... 8 Step 1: Creating and Saving a Job... 9 Step 2: Job Start... 9 Archiving Via Folders or AAF/XML... 9 Step 3: Archive Job Run...10 Step 4: Web Browse, Search, and Preview...10 Step 5: Web Restore...10 Fuzzy Logic-based AAF/XML/EDL Restores...10 DNA Evolution Deployment Options...11 Deployment 1: Linux Server, Network Clients Deployment (Mac, Windows, Linux)...12 Deployment 2: Direct-Mount Linux Server Deployment...13 Deployment 3: Direct-Mount Mac Server Deployment...14 Deployment 4: Direct-Mount Mac Virtual Server Deployment...15 Pre-Install Checklist...16 Rack space...16 Page 2 of 63
Static IP address...16 Bar code labeled LTO tapes...16 X Series Linux archive manager considerations...16 N Series Linux archive manager considerations...16 Mac archive manager considerations:...17 Hardware Install and Configuration...18 X100, X200 Linux Controller/Linux Archive Manager Hardware Setup (for Deployments 1 and 2)...18 X100, X200 Linux Controller/Mac Archive Manager Hardware Setup (for Deployment 3)...19 X100-24, X200-24, X200-48, X400-28 Linux Controller/Linux Archive Manager Hardware Setup (for Deployments 1 and 2)...20 X100-24, X200-24, X200-48, X400-28 Linux Controller/Mac Archive Manager Hardware Setup (for Deployment 3)...21 Mac Virtual Server, Mac Archive Manager Setup (for Deployment 4)...22 Software Install, Power-On...23 DNA Evolution Controller Setup...23 Log In...23 Check IP Address...24 Set Up Static IP Address...24 Ensure All Processes Are Running...24 Determining DNA Evolution Version...25 DNA Evolution Linux Archive Manager Setup...25 DNA Evolution Mac Virtual Controller Setup...26 DNA Evolution Mac Archive Manager Setup...26 Install the Atto Drivers...26 Install LTFS...26 DNA Evolution Mac Archive Manager Installation...26 DNA Evolution Remote Client Setup...27 Before You Set Up a DNA Evolution Client...28 DNA Evolution Client Installation...28 Page 3 of 63
The Administrator Dropbox...30 AccessDNA...31 Getting to AccessDNA...31 Enable Folder Access...32 Enable UNC Drives...32 View and Restart Services...32 Enable Client Logging...32 Configuration and System Settings...33 Configuring Previews...33 Preview Naming and Folder Requirements...33 Generating Previews...33 Configuring the Previews Folder...33 Configuring Catalog Backups...34 Configuring Email...35 Create, Edit, Preview Metadata Tagging Templates...35 Normal and limited user Metadata Tagging...36 Setup Automated Preview Creation...37 Manually Indexing Previews...37 LTO/LTFS Tape Management...39 MOUNT, UNMOUNT...39 MOVE (Autoloaders only)...39 Format Tapes...39 Verify Tapes...41 Duplicate Tapes...43 Viewing Tapes and Utilization, Adding Notes...45 Tape Online/Offline Indicator...46 Tape Listing...46 Page 4 of 63
Deleting Tapes...46 Command Line LTFS TapE IMPORT...47 Location of Command Line Tape Tools...47 LTFS TAPE Import...47 Reports...50 Archive Clients and Servers Management...52 The Clients View...52 Point-to-Point Connectivity...53 Tape Devices...53 Users and Groups Tabs Reference Guide...54 Adding, Editing, Deleting Users...54 Adding, Editing Groups...55 Normal User Workflow...55 Group Workflow...55 License Management...57 License Usage and Policy...57 License Listing...57 To Add a User to a Machine...57 To Remove/Uninstall a Client Machine and Reclaim the License...57 Upgrading DNA Evolution Controller, Archive Manager, Archive Client...58 Upgrading DNA Evolution Controller and Linux Archive Manager...58 Upgrading Archive Manager and Archive Client...60 Catalog Management...61 Importing and Exporting Archives via XML...61 How to Export an Archive...61 How to Import an Archive...62 Page 5 of 63
INTRODUCTION Built on Linear Tape-Open (LTO) and Linear Tape File System (LTFS) technologies, DNA Evolution is a nearline and archive workflow solution designed to manage digital content. THE ARCHITECTURE DNA EVOLUTION SOFTWARE COMPONENTS DNA EVOLUTION CONTROLLER The DNA Evolution Controller is software that runs on a networked Linux appliance. This software can run on a physical server (X Series, N Series) or on a virtual server (M Series). This core software component is included in all DNA Evolution products and is the core of the solution. The appliance running the controller resides on the network and provides overall management and policy functions to the rest of the DNA Evolution environment. The controller itself does not transfer data, but instead manages the traffic of all data transfers. It provides an interface for: Setup and administration of archives and jobs User and group management Client machine management Additionally, it stores all policy information regarding a DNA Evolution environment (archive policies, user policies, group policies, and reporting). DNA EVOLUTION ARCHIVE MANAGER The DNA Evolution Archive Manager is software that runs either on the Linux appliance or on a Mac system, based on customer environment and workflow requirements. (Currently, there is no Archive Manager for Microsoft Windows. However, archiving from Windows machines is supported via the archive client described in the next section). The Archive Manager is the module responsible for interfacing with the LTO tape hardware and negotiating access to it for the different client software components. It can archive and restore data over directly connected sources on the machine it is installed on (e.g., local volumes mounted via direct attached NAS or SAN storage), or it can run in a networked mode, (e.g., on the Linux appliance), where it receives data from multiple networked archive clients. DNA EVOLUTION CLIENT The DNA Evolution Client is a lightweight background application that is installed directly onto a laptop, desktop, or server. It is designed to perform network-based transfers and runs natively on a number of host operating systems, including Mac, Linux, and Windows. The client is responsible for performing transfers and can perform archive and restores to and from an Archive Manager. The workstation that the client software is installed on connects to various storage systems (local, SAN, NAS, etc.) and acts as the gateway for archive and restore. It transfers data to LTO tape via the archive manager over an intermediate network connection. All information about how to perform the transfer is fetched from the controller, and subsequent information about the transfer, such as, job progress and status, is stored on the controller. Page 6 of 63
DNA EVOLUTION HARDWARE COMPONENTS DNA EVOLUTION LINUX APPLIANCE The DNA Evolution Linux appliance is a hardware appliance that is included in almost all DNA Evolution installs (X and N Series). For the M series, the DNA Evolution Controller Software can exist on a customer-provided Mac. As an option, the Linux hardware appliance included in the X and N Series can be configured to host the DNA Evolution archive manager in certain deployments. TAPE HARDWARE Most DNA Evolution solutions also include LTO tape hardware (X and M series). The tape hardware is hosted on a system that hosts the archive manager. Therefore, in certain deployments, it can be hosted on the Linux appliance or connected to a Mac archive manager. UNDERSTANDING DNA EVOLUTION ARCHIVING, BACKUP, AND NEARLINE WORKFLOWS DNA Evolution allows you to move data from your primary storage to an LTO/LTFS storage tier. The following list highlights the key differences between archiving, backup, and nearline workflows: Archiving: Archiving refers to the long-term archival of media. Media that is archived in today s modern file-based workflows includes raw footage from file-based cameras, media related to finished projects, and final exports. Backup: Backup, on the other hand, is performed to protect your active work-in-progress media. This is done largely to recover quickly from a data loss event, such as a storage failure, operator error, or a virus. Backed-up data may often not be archived at all after a project is completed. Nearline: As media pipelines deal with higher resolution data, it is not feasible to keep all data online. DNA Evolution enables seamless migration by which content can be migrated and deleted safely from your online storage and retrieved or accessed directly (via LTFS) when needed. UNDERSTANDING HOW DNA EVOLUTION WRITES DATA AND METADATA DNA Evolution follows a policy of open data and open metadata. All data is written to LTO (via LTFS) in an open, readily accessible format. Metadata is written to an XML file system, enabling an end user to easily update or extract this information. DNA Evolution maintains two XML databases. These can be found in Shared Folders (under Catalogs) for the Macbased controller (M Series) or in /sdna_fs/proxies/ for the Linux controller (X and N Series). For each archive, you will see a folder that matches the archive name. Under each archive name, you will see two folders named 1 and metadata. The 1 folder maintains a file-system view that matches the Browse Files view in the web interface. This serves as the file-system metadata index and catalog. Each filename and directory structure is preserved. However each file (though retaining the same name as the source) is an XML file representing the original file. (See screenshot below for an example) Page 7 of 63
As seen in the screenshot above, the details with the location on LTO, LTO bar code, and checksum are stored in an open XML format. DNA Evolution includes a media asset management (MAM) engine that extracts metadata and derives clip information for over 180 file formats. The clip and extracted metadata are stored per archive in an open XML file system A sample is the following: DNA Evolution can derive single file/self-contained clips (e.g..mov,.mp4) as well as multi-file clips such as Avid Mediafiles, P2, XDCAM, ARRI, and more. For a complete list of supported clips and formats, refer to document DNA Evolution v3.5: Format Support. UNDERSTANDING DNA EVOLUTION DATA AND COMMAND FLOW This section includes information about how the controller, archive managers, and archive clients interact to move data. In addition, details include how data and metadata travel across the various components. An example of a network archive and a local archive are featured in the diagram pictured in the next screenshot. Page 8 of 63
STEP 1: CREATING AND SAVING A JOB The first step is for you to create and save a job via the web interface. When a job is saved, it is stored on the controller in its policy database. The process of creating a job involves choosing a source and target, which can vary depending on the specific configuration and environment. In general, users need to pick the <archive client machine OR archive manager w/ source directories> AND <archive manager machine/target archive> pair. If an Archive Client is picked as the source and Archive Manager as the target, it is defined as a network transfer job. If an Archive Manager is picked as source and target, then it is a local transfer job. Additional options can be selected, including policies and filters, among others. DNA Evolution automatically manages the target hardware without end user intervention. STEP 2: JOB START An archive job can be scheduled to start automatically or run manually via clicking the start button in the web interface, or may be executed by a third-party through scripting. When an archive job starts, a process is spawned. The process first contacts the controller s web services, as it has no information locally stored about the job. The process retrieves the source directory, target archive and job run policies, from the controller s web services. After this step, the process can start archiving or restoring content. ARCHIVING VIA FOLDERS OR AAF/XML DNA Evolution allows you to either select folders for archiving, or archive via an exported AAF or XML representation of content that is converted into a DNA Evolution ticket. When an AAF or XML ticket is used for archiving, only media referenced within the exported AAF or XML is archived. Page 9 of 63
STEP 3: ARCHIVE JOB RUN The process will perform a local archive if an <archive manager, archive manager> pair has been specified and will perform a network archive if an <archive client, archive manager> pair has been specified during job creation. The process then will contact the Archive Manager, (locally or over the network), that was selected for archiving. The Archive Manager first looks for available tape hardware. The system may use standalone tape drives or an autoloader. In both cases, the archive manager will look for a free tape drive. If a free tape drive is not found, the job will fail and return with a device not available error. The job will retry when it is manually run again. On the other hand, if it finds a free drive, it will check to see if a valid tape is loaded (a valid tape is one that has been assigned to the archive). If a valid tape is not loaded, it will attempt to load a valid tape for the archive (if it is an autoloader). If a valid tape is not found in the slots of the autoloader, or if the standalone tape drive does not contain a valid tape, the client sends an error to the controller, requesting the end user to load the appropriate tape (NOTE: Email notification must be enabled to receive notifications for tape changes). If the archive manager finds a free drive that can load an appropriate tape, it makes the tape available for writing. At this point, the process is ready to start archiving data. Source data can be local or accessed via a network share. Data can be archived locally to available hardware, if the archive is run directly on the archive manager, or archived over the network to an archive manager. In either case, data is archived direct-to-tape without any intermediate caching. DNA Evolution utilizes the ability of LTO technology to checksum and perform writes at the same time in one operation. Before files are written to tape, the process compares the list of files it needs to archive with the files that have already been archived, as part of this archive. To do so, it compares the source file name, size, and modified time against the target. NOTE: DNA Evolution compares files in the source and target and only makes incremental copies. It performs this check based on the location of the file and the file size, modified date. In all cases, the archiving process on the archive client or manager will send progress and other information to the controller s web services. Information it sends to the controller includes XML files describing the file system (enabling web browse and search). During archiving, the archiving process can extract additional metadata/file information. In addition to XML information, the archiving process can provide a low-res previews folder. The archiving process also correlates file names, enabling web-based preview play. Additionally, DNA Evolution can create links of files successfully archived for purposes of preview creation and other tasks. STEP 4: WEB BROWSE, SEARCH, AND PREVIEW After a job has run, and related metadata has been synced to the controller, you can browse, search, and preview your archived data via the web interface. DNA Evolution presents a single global view of all data that is archived. STEP 5: WEB RESTORE You can select files to restore directly via browsing or searching within the DNA Evolution web-based interface. Files selected for restore can be restored back to their original location or to a new location. When a restore is initiated, the archiving process retrieves job information from the Controller, much like an archive job, and processes files in the restore list. FUZZY LOGIC-BASED AAF/XML/EDL RESTORES In addition to browse and search-based restores, DNA Evolution enables restores via AAF/XML/EDL. An exported AAF/XML/EDL can be provided to DNA Evolution as a restore list. DNA Evolution interprets the restore list and performs a search against its archive and finds the tapes that the referenced files reside on. It then gathers the original media and Page 10 of 63
restores it to a desired location based on job definition. In addition, DNA Evolution can also be used to perform a fuzzy match, in which it uses the original online files referenced in the exported AAF/XML/EDL as a clue to find the source media residing on tape. This is relevant in an offline/online scenario in which media needs to be conformed. DNA EVOLUTION DEPLOYMENT OPTIONS With DNA Evolution systems, there is flexibility in how they can be deployed. This chapter provides information about deployment options and covers the pros and cons of each option. For every configuration, any highlights in the configuration are shown in red. Page 11 of 63
DEPLOYMENT 1: LINUX SERVER, NETWORK CLIENTS DEPLOYMENT (MAC, WINDOWS, LINUX) Description Valid DNA Evolution Systems Performance/Stream (1 Gb LAN) Performance/Stream (10 Gb LAN) Known Limitations In this deployment, the Linux appliance hosts both the controller and the archive manager. The Linux appliance also hosts the tape hardware. Client software is installed on any system that needs to be archived from or restored to. All data moves between the client systems and the Linux server over an intermediate LAN connection. Valid for all systems except M Series Win Client Mac Client Linux Client 500 700 Mb/s 250 550 Mb/s 500 700 Mb/s Win Client Mac Client Linux Client Not Available Not Available Not Available Win Client Mac Client Linux Client None None None Client systems: System Requirements 1 Gb or 10 Gb connection Windows XP, 2000, 2003, 7; Linux; Mac OS X Tiger, Leopard, Snow Leopard, Lion, Mountain Lion, Mavericks Recommended private network for performance Page 12 of 63
DEPLOYMENT 2: DIRECT-MOUNT LINUX SERVER DEPLOYMENT Description Valid DNA Evolution Systems In this deployment, the Linux server hosts both the controller and the archive manager. However, unlike the deployment highlighted in Section: Linux Server, Network Clients Deployment (Mac, Windows, Linux), no client software is installed on any other system. Instead, data is pulled directly from the Linux archive manager by mounting network shares (NFS/Samba) or by installing a SAN client on the Linux archive manager (e.g., Quantum StorNext client). Valid for all DNA Evolution systems except M Series Performance/Stream NFS Samba Proprietary SAN Client 500 700 Mb/s 500 700 Mb/s Up to 1.2 Gb/s/stream Known Limitations System Requirements Not supported for Avid Unity, or any storage that does not support a Linux client or ability to mount NFS/SAMBA. We are currently testing with new Linux ISIS client. Please contact us for more details. None Page 13 of 63
DEPLOYMENT 3: DIRECT-MOUNT MAC SERVER DEPLOYMENT Description In this deployment, the Linux appliance hosts the controller software, and the Mac acts as the archive manager. The Mac also connects to end-user storage, and data transfers happen directly over SAS without any network limitations. Valid DNA Evolution Systems Performance/Stream Valid for all DNA Evolution systems except M Series NFS Samba SAN Client/Direct Connect Not Available Not Available Up to 1.2 Gb/s Known Limitations Only supported for limited DNA Evolution hardware. Mac System Requirements: System Requirements Mac OS X Leopard (32 bit/64 bit), Snow Leopard (32 bit/64 bit), Lion and Mountain Lion (32 bit/64 bit), Mavericks (32 bit/64 bit) Open PCIE slot for Atto H644 (OR) Atto Thunderlink 1068 Thunderbolt to SAS adapter (For new Mac Pro, Mac Mini, MacBook) Relevant SAS and Thunderbolt cables Atto drivers HP LTFS drivers Mac Archive Manager can be configured with FC and other third-party hardware. Check with StorageDNA support. Page 14 of 63
DEPLOYMENT 4: DIRECT-MOUNT MAC VIRTUAL SERVER DEPLOYMENT Description Valid DNA Evolution Systems Performance/ Stream Known Limitations In this deployment, a Mac-based system hosts the controller software (as a virtual machine), and the Mac also acts as the archive manager. The Mac also connects to end-user storage, and data transfers happen directly over SAS without any network limitations. Valid only for M Series NFS Samba SAN Client/Direct Connect NA NA Up to 1.2 Gb/s Additional DNA Evolution clients not supported. Mac System Requirements: System Requirements Mac OS X Leopard (32 bit/64 bit), Snow Leopard (32 bit/64 bit), Lion and Mountain Lion (32 bit/64 bit), Mavericks (32 bit/ 64 bit) Open PCIE slot for Atto H644 (OR) Atto Thunderlink 1068 Thunderbolt to SAS adapter (For new Mac Pro, Mac Mini, Macbook) Relevant SAS and Thunderbolt cables Atto drivers HP LTFS drivers HP LTFS drivers Mac Archive Manager can be configured with FC and other third-party hardware. Check with StorageDNA support. Page 15 of 63
PRE-INSTALL CHECKLIST The following checklist applies to the following DNA Evolution products: X100, X200, X100-8, X100-24, X200-24, X200-48, X400-48, N100. Refer to the DNA Evolution M100/VM: Install and Setup Guide for Mac-based configurations. Before your hardware and software is deployed, please ensure the following: RACK SPACE X100 (2u) + Additional rack space for Mac archive manager (if configuring) X200 (2u) + Additional rack space for Mac archive manager (if configuring) X100-8 (2u) + Additional rack space for Mac archive manager (if configuring) X100-24 (3u) + Additional rack space for Mac archive manager (if configuring) X200-24 (3u) + Additional rack space for Mac archive manager (if configuring) X200-48 (5u) + Additional rack space for Mac archive manager (if configuring) X400-48 (5u) + Additional rack space for Mac archive manager (if configuring) N100 (1u for N100) + Additional rack space for Mac archive manager (if configuring) + Additional rack space for customer LTO hardware STATIC IP ADDRESS The DNA Controller needs a static IP address and must be allocated prior to setup. BAR CODE LABELED LTO TAPES Ensure that prior to setup you have at least one bar code labeled LTO tape. LTO tapes must be bar code labeled in both single drive and autoloader deployments. All bar codes must be professionally printed with no duplicate ID/Names. X SERIES LINUX ARCHIVE MANAGER CONSIDERATIONS The Linux archive manager is pre-installed on the Linux server that also hosts the controller. In this case, the Linux server will be connected via SAS to the LTO hardware. All LTO hardware ships with 2m SAS cables and rack mount placement should be accounted for. N SERIES LINUX ARCHIVE MANAGER CONSIDERATIONS The Linux archive manager is pre-installed on the Linux server that also hosts the controller. In this case, the Linux server will be connected via SAS or FC to the customer provided LTO hardware. Please ensure that the appropriate and supported HBA cards, connectivity and cables are provided for connecting the N100 server to the existing LTO hardware. Please contact StorageDNA for latest information on supported third-party libraries. Page 16 of 63
MAC ARCHIVE MANAGER CONSIDERATIONS: The Archive Manager can also be hosted on a separate Mac while the Linux controller acts as a standalone controller. In this case, ensure that the customer provided Mac is racked and is equipped with the proper cards, cables and connectivity to the LTO hardware. Page 17 of 63
HARDWARE INSTALL AND CONFIGURATION The following section includes hardware installation guides for each DNA Evolution model. X100, X200 LINUX CONTROLLER/LINUX ARCHIVE MANAGER HARDWARE SETUP (FOR DEPLOYMENTS 1 AND 2) This section assumes you have completed the pre-install steps before hardware setup. Step 1: Review the checklist of hardware components: (A) HP DL120/320/380 Rack-Mount Server (Box 1) (B) 2 m mini-sas to mini-sas connector OR 4 m external mini-sas fan-out cable (Box 1) (C) HP 1U Rack-Mount SAS Enclosure w/1 or 2 x HH LTO Tape Drive (Box 2) (D) Rack-Mount Kit for SAS Enclosure (Box 2) (E) StorageDNA USB Key for Auto-Backups Step 2: Unpack your HP DL120/320/380 server (A). Use the HP DL120/320/380 instructions to rack your server. Step 3: Unpack your 1U SAS tape rack-mount kit (C). The rack-mount kit will be prepopulated with your HP LTO halfheight tape drive. Use the SAS tape rack-mount enclosure instructions to rack the enclosure. Step 4: Connect the SAS cable (B) from the HP DL120/320/380 (A) to the internal tape drive in the HP 1U rack-mount kit (C). Step 5: Connect power cords to both the server (DL120/320/380) (A) and the tape rack-mount enclosure (C). Step 6: Connect the network to the DL120/320/380 (A). You will also need to connect a keyboard, monitor, and mouse to your system for setup. Remember to plug the USB key (E) into an open USB slot. Step 7: Power the system on in the following sequence: Tape rack-mount enclosure (C) DL120/320/380 (A) Step 8: Upon boot-up, log in using the following credentials: Username: root Password: dna180 Step 9: Refer to Chapter 6, Software Install, Power-On, of this user guide for next steps. Page 18 of 63
X100, X200 LINUX CONTROLLER/MAC ARCHIVE MANAGER HARDWARE SETUP (FOR DEPLOYMENT 3) This section assumes you have completed the pre-install steps before hardware setup. Step 1: Review the checklist of hardware components: (A) HP DL120/320/380 Rack-Mount Server (Box 1) (B) 2 m mini-sas to mini-sas connector OR 4 m external mini-sas fan-out cable (Box 1) (C) HP 1U Rack-Mount SAS Enclosure w/1 or 2 x HH LTO Tape Drive (Box 2) (D) Rack-Mount Kit for SAS Enclosure (Box 2) (E) StorageDNA USB Key for Auto-Backups (F) Atto H644 SAS Card Additionally, you must have a Mac Pro to host the Atto H644 card. (G) Mac Pro with free PCI-E slot for Atto H644 card Step 2: Unpack your HP DL120/320/380 server (A). Use the HP DL120/320/380 instructions to rack your server. Step 3: Unpack your 1U SAS tape rack-mount kit (C). The rack-mount kit will be prepopulated with your HP LTO halfheight tape drive. Use the SAS tape rack-mount enclosure instructions to rack the enclosure. Step 4: Power down the Mac Pro (G). Install the Atto H644 card in the Mac Pro s free PCI-E slot. Step 5: Connect the SAS cable (B) from the Mac Pro (G) to the internal tape drive in the HP 1U rack-mount kit (C). Step 6: Connect power cords to both the server (DL120/320/380) (A) and the tape rack-mount enclosure (C). Step 7: Connect the network to the DL120/320/380 (A). You also will need to connect a keyboard, monitor, and mouse to your system for setup. Remember to plug the USB key (E) into an open USB slot. Step 8: Power the system on in the following sequence: Tape rack-mount enclosure (C) Mac Pro (G) DL120/320/380 (A) Step 9: Upon boot-up, log in using the following credentials: Username: root Page 19 of 63
Password: dna180 Step 10: Refer to Chapter 6, Software Install, Power-On, of this user guide for next steps. X100-24, X200-24, X200-48, X400-28 LINUX CONTROLLER/LINUX ARCHIVE MANAGER HARDWARE SETUP (FOR DEPLOYMENTS 1 AND 2) This section assumes you have completed the pre-install steps before hardware setup. Step 1: Review the checklist of hardware components: (A) HP DL120/320/380 Rack-Mount Server (Box 1) (B) 2 m mini-sas to mini-sas connector OR 4 m external mini-sas fan-out cable (Box 2) (C) HP 1U/2U/4U Rack-Mount Autoloader w/1/2/4 x HH LTO Tape Drive (Box 2) (D) Rack-Mount Kit for LTO Autoloader (Box 3) (E) StorageDNA USB Key for Auto-Backups (Box 1) Step 2: Unpack your HP DL120/320/380 server (A). Use the HP DL120/320/380 instructions to rack your server. Step 3: Unpack your 1U/2U/4U SAS LTO Autoloader Kit (C). The autoloader will be prepopulated with your HP LTO halfheight tape drive(s). Use the SAS tape rack-mount instructions to rack the enclosure. Step 4: Connect the SAS cable (B) from the HP DL120/320/380 (A) to the tape drive(s) in the HP Autoloader (C). The SAS connections are in the rear. For X200/X400 Series systems with fan-out cables, ensure that the cable with label SAS0 is connected to the bottommost drive. Step 5: Connect power cords to both the server (DL120/320/380) (A) and the tape autoloader (C). Step 6: Connect the network to the DL120/320/380 (A). You also will need to connect a keyboard, monitor, and mouse to your system for setup. Remember to plug the USB key (E) into an open USB slot. Step 7: Power the system on in the following sequence: Tape autoloader (C) The display should state that all drives are RDY. DL120/320/380 (A) Step 8: Upon boot-up, log in using the following credentials: Username: root Password: dna180 Step 9: Refer to Chapter 6, Software Install, Power-On, of this user guide for next steps. Page 20 of 63
X100-24, X200-24, X200-48, X400-28 LINUX CONTROLLER/MAC ARCHIVE MANAGER HARDWARE SETUP (FOR DEPLOYMENT 3) This section assumes you have completed the pre-install steps before hardware setup. Step 1: Review the checklist of hardware components: (A) HP DL120/320/380 Rack-Mount Server (Box 1) (B) 2 m mini-sas to mini-sas connector OR 4 m external mini-sas fan-out cable (Box 2) (C) HP 1U/2U/4U Rack-Mount Autoloader w/1/2/4 x HH LTO Tape Drive (Box 2) (D) Rack-Mount Kit for LTO Autoloader (Box 3) (E) StorageDNA USB Key for Auto-Backups (Box 1) (F) Atto H644 SAS Card (If hosting on old Mac Pro) OR Atto Thunderlink 1068 Thunderbolt to SAS adapter (if hosting on new Mac Pro, Mac Mini or MacBook) (G) Mac Pro with free PCI-E slot for Atto H644 card OR new Mac Pro, Mac Mini, MacBook with a free Thunderbolt port. Step 2: Unpack your HP DL120/320/380 server (A). Use the HP DL120/320/380 instructions to rack your server. Step 3: Unpack your 1U/2U/4U SAS LTO Autoloader Kit (C). The autoloader will be prepopulated with your HP LTO halfheight tape drive(s). Use the SAS tape rack-mount instructions to rack the enclosure. Step 4: Power down the Mac (G). Install the Atto H644 card in the old Mac Pro s free PCI-E slot OR Connect the Thunderbolt cable between the new Mac Pro, Mac Mini, MacBook and the Atto SH1068 Box. Step 5: Connect the SAS cable (B) from the old Mac Pro (G) to the tape drive(s) in the HP Autoloader (C) OR connect the SAS cable from the Atto SH1068 Box to the tape drive(s) in the HP Autoloader (C). The SAS connections are in the rear. For X200/X400 Series systems with fan-out cables, ensure that the cable with label SAS0 is connected to the bottommost drive. Step 6: Connect power cords to both the server (DL120/320/380) (A) and the tape autoloader (C). Step 7: Connect the network to the DL120/320/380 (A). You also will need to connect a keyboard, monitor, and mouse to your system for setup. Remember to plug the USB key (E) into an open USB slot. Step 8: Power the system on in the following sequence: Tape autoloader (C) The display should state that all drives are RDY. Mac Pro (G) DL120/320/380 (A) Step 9: Upon boot-up, log in using the following credentials: Page 21 of 63
Username: root Password: dna180 Step 10: Refer to Chapter 6, Software Install, Power-On, of this user guide for next steps. MAC VIRTUAL SERVER, MAC ARCHIVE MANAGER SETUP (FOR DEPLOYMENT 4) The setup for the Mac Virtual System is covered in a separate document DNA Evolution M100: Install and Setup Guide. Page 22 of 63
SOFTWARE INSTALL, POWER-ON This section includes the steps for configuring, installing software modules into, and powering on a DNA Evolution system. DNA EVOLUTION CONTROLLER SETUP The DNA Evolution Linux appliance (HP DL120/320/380) is included with all DNA Evolution systems: X100, X100-8, etc. In this section, we guide you through the steps needed to power on and initial setup of the HP Linux server and controller. Please refer to the hardware setup guides (racking, power, etc.) for each system in the previous section before powering on the system. Power on and setup steps for the Mac Virtual Server are similar but details are covered in document DNA Evolution M100 Install and Setup Guide. After first power-on, the following section will guide you through the steps necessary to ensure your system is ready to go: Check your IP address. If needed, set a static address. Ensure all tape hardware is configured and visible. Ensure all processes are running. LOG IN After power-on, you can log in to the server using the following default credentials: Username: root Password: dna180 You then will be guided to the CentOS desktop. On the desktop, you will find a number of scripts designed to help you get started (see next screenshot). Page 23 of 63
CHECK IP ADDRESS The script named display-network will display all Ethernet ports and their IP addresses. SET UP STATIC IP ADDRESS To configure a static IP address, run the script configure network. This will invoke the Centos Ethernet settings screen. Follow these steps: Click on the adapter you want to configure (Default should be eth0). Click on Edit. Select Statically set IP address. Provide IP address, gateway, and subnet mask. Remember to save (File->Save). Also, select the adapter and inactivate and reactivate the network. ENSURE ALL PROCESSES ARE RUNNING Run script check-services to ensure all DNA Evolution processes are running successfully. Normally, you should see the following: GOOD: Web Server is running GOOD: Database is running GOOD: LDAP is running GOOD: Samba shares are running GOOD: SDNA Daemon is running GOOD: DNA Client Service Provider is running GOOD: DNA Client Proxy is running GOOD: Fuse module is loaded GOOD: Logger is running and listening on 9094 Page 24 of 63
GOOD: SDNAServerProxy is running and listening on 4545 GOOD: SDNAServerProxy is running and listening on 4545 DNA Evolution 3.5 Administrator Guide DETERMINING DNA EVOLUTION VERSION To determine the current version of the software, refer to the version and build number in the footer of any of the pages including My Archives screen. (See next screenshot) DNA EVOLUTION LINUX ARCHIVE MANAGER SETUP If you are connecting the tape hardware directly to a Mac, then skip this step. Since the tape hardware can be connected to the supplied HP Linux server, this section guides you through the steps needed to ensure the hardware is connected and visible. On the Linux controller desktop, run the script check-tapehardware to ensure all expected hardware is connected. For all systems, based on the number of tape drives, you should see the following for each drive: Vendor: HP Model: Ultrium 6-SCSI Rev: Z39W Type: Sequential-Access ANSI SCSI revision: 06 For automation units (X100-8, X200-24, X200-48, X400-48), you also should see the automation component shown as: Page 25 of 63
Vendor: HP Model: MSL G3 Series Rev: E.02 Type: Medium Changer ANSI SCSI revision: 05 DNA Evolution 3.5 Administrator Guide You can t see the tape hardware? Check if your tape drive is powered on, and make sure all cables are properly connected. DNA EVOLUTION MAC VIRTUAL CONTROLLER SETUP Software install and power-on steps for the Mac Virtual Controller are included in another document DNA Evolution M100: Install and Setup Guide. DNA EVOLUTION MAC ARCHIVE MANAGER SETUP It is possible to configure a Mac Pro (pre-trash Can version) as an archive manager as an alternative to using the Linux appliance for this purpose. This section will guide you through the software setup required prior to installation of the Mac archive manager and the installation steps involved for the Mac archive manager software components. INSTALL THE ATTO DRIVERS Install the proper ATTO drivers for the associated SAS or Fibre Channel HBA to ensure you can see the attached tape hardware. Please refer to the ATTO website for downloading and instructions on firmware and driver setup. Once the driver is installed, you should ensure that you have powered on the tape hardware and that the tape hardware is visible. INSTALL LTFS Install the HP or IBM Mac OS X LTFS drivers included in the package. DNA EVOLUTION MAC ARCHIVE MANAGER INSTALLATION After following the previous steps, you can begin your DNA Evolution archive manager installation and setup. Step 1: Download the DNA Evolution client/archive manager. The client can be downloaded directly from the DNA Evolution web interface. In the Job Listing pane under My Archives, select the Client Installers icon. Download the Mac version to your local desktop. Step 2a: Installation After you agree to the End User License Agreement, you will need to provide the following information: 1. DNA Evolution Server IP address: Provide the IP address of the DNA Evolution Server configured in Section: DNA Evolution Controller Setup. OR Page 26 of 63
Hostname: Alternatively, you can provide dnaserver as the hostname. Tip: If you would like to configure your own hostname, you will need to deploy your own SSL certificate. Please contact StorageDNA Support for more information. 2. A valid license key Provide a valid archive manager license key, and, if accurate, it will proceed to the next screen of the wizard. In this screen, you can provide a client machine identifier. This defaults to the hostname of the machine, but you can change that to an alternate name at this time. Note that this name is not used for network connectivity and is for identification and labeling purposes only. You can click through the next few screens, and the installer will complete the process. Tip: Where can you find your license keys? Go to the Clients tab->license Management, and pick any license highlighted in green. Please ensure you pick a Mac archive manager license key with the appropriate number of tape drives. Step 3: Set up AccessDNA The installer will launch AccessDNA automatically after installation is complete. AccessDNA is an application that allows you to set rules on which folders you want exposed to the DNA Evolution web interface for user access. These rules can be set up for global users or based on individual users. Once a folder is exposed, a user with access to the web console can create jobs to and from the client system automatically. Please refer to the AccessDNA section for information on how to properly configure AccessDNA. Tip: If you do not save your settings in AccessDNA, your settings will not take effect. DNA EVOLUTION REMOTE CLIENT SETUP There is no need to install client software on a Mac that is designated as an archive manager. Network (Remote) Clients can only archive and restore to/from Linux archive managers. If you are deploying with clients and have set up a Linux controller and archive manager, you are now ready to set up and install your DNA Evolution clients. A network client can be set up on any system (Win, Mac, Linux). Once installed, the DNA Evolution web interface allows you to archive and restore to/from the network client system. Page 27 of 63
Step 1: Check the client operating system and resources. DNA Evolution 3.5 Administrator Guide BEFORE YOU SET UP A DNA EVOLUTION CLIENT Currently, you can install a DNA Evolution client on the following operating systems: Mac OS X 10.4 and above Windows XP, 2003 Server, Vista, 7 Linux (all flavors) Step 2: Ensure the DNA Evolution controller is visible to the client over the network. This can be done using ping. DNA EVOLUTION CLIENT INSTALLATION Once you have qualified your environment, you can begin your DNA Evolution client installation and setup. Step 1: Download the DNA Evolution client. The client can be downloaded directly from the DNA Evolution web interface. In the Job Listing pane under My Archives, select the Client Installers icon. Download the Mac or Windows version to your local desktop. Step 2a: Installation After you agree to the End User License Agreement, you will need to provide the following information: 1. DNA Evolution Controller IP address: Provide the IP address of the DNA Evolution Controller. OR Hostname: Alternatively, you can provide dnaserver as the hostname. 2. A valid license key Provide a valid license key, and, if accurate, it will proceed to the next screen of the wizard. In this screen, you can provide a client machine identifier. This defaults to the hostname of the machine, but you can change that to another name. Note that this name is not used for network connectivity and is for labeling purposes only. You can click through the next few screens, and the installer will complete the process. Tip: Where can you find your license keys? Go to the Clients tab->license Management, and pick any license highlighted in green. Step 3: Set up AccessDNA. The installer will launch AccessDNA automatically after installation is complete. AccessDNA is an application that allows you to set rules on which folders you want exposed to the DNA Evolution web interface for user access. These rules can Page 28 of 63
be set up for global users or based on individual users. Once a folder is exposed, a user with access to the web console can create jobs to and from the client system automatically. Please refer to the next section on how to properly configure AccessDNA. Once you have configured your folders in AccessDNA, you are ready to start archiving to and from this client. Page 29 of 63
THE ADMINISTRATOR DROPBOX The administrator dropbox is a useful location on the controller that stores logs, reports and other detailed information. It can be accessed via the web interface but can also be accessed over the network. Any system with connectivity to the controller can connect to the administrator dropbox directly. WINDOWS: To access the admininstrator dropbox over the network, follow these steps from a Windows system: From a Windows system, right click on My Computer, and click on Map network drive Provide the dna server IP address and share name. (See next screenshot) Mac: To access the admininistrator dropbox over the network, follow these steps from a Mac: From a Mac system on the network, open Finder. Click on Go->Connect to server (see next screenshot). Login as Guest Connect to the admindropbox volume ONLY Page 30 of 63
ACCESSDNA AccessDNA is an application that is installed automatically when you install a DNA Evolution client or a Mac based archive manager. AccessDNA provides a DNA Evolution client access to certain key attributes, such as: Which specific folders should be exposed to the web interface Which users are allowed to access these folders from the online web GUI Which UNC drives should be accessible If local hostnames and IPs should be reported to the online web GUI GETTING TO ACCESSDNA On a Mac, AccessDNA can be found under Applications->StorageDNA->accessDNA.app. On a Windows PC, AccessDNA can be found under Programs->StorageDNA->AccessDNA.exe. Page 31 of 63
ENABLE FOLDER ACCESS Only a user with physical access to the machine has ultimate authority over which folders to expose and to which users those folders are accessible in a DNA Evolution environment. If Restrict StorageDNA access to selected folders is checked, then DNA Evolution will restrict folders according to the rules set up by the user. If this checkbox is not enabled, all folders (including items hidden by operating systems) will be visible to the DNA Evolution environment. Tip: Folder restriction is also enforced when the job runs. So if a user does not have access, the job will be denied access to the folder. Once restriction is enabled, you can make folders visible to the DNA Evolution environment by adding them to the list. When you click the Add button, you can select the folder and decide if you would like to expose this folder to all DNA Evolution users or to just select users and groups. You can edit and delete these entries when required. ENABLE UNC DRIVES AccessDNA requires you to explicitly add networked machines that you will access over UNC. This check is put in place to prevent unauthorized access to other computers in your networked environment. Check the Enable UNC Support checkbox to enable UNC access for this machine. To configure UNC, click on Configure. After the configuration dialog is open, you can add UNC machines. To add a new machine, you can browse the network for machines or type in the UNC machine name. Optionally, you can add the network provider. Once you are done adding the UNC machines, you can add UNC paths in the folder access list. Tip: (For Avid Unity/ISIS) While setting up Unity/ISIS Connection Manager, the following information can be helpful: The username/password of a Windows box must exactly match the password provided for the username under the Connection Manager, OR the username should be entirely different. To find the Unity/ISIS system name that you will need to provide in AccessDNA: In Windows, under My Computer, you will see the workspace listed as projects under a Unity/ISIS server name. Use that name as the server name for AccessDNA. VIEW AND RESTART SERVICES You can view and restart essential DNA Evolution client services from this menu. If your client machine is not browsable or is unable to sync jobs, it is a good idea to restart client services via this menu option. ENABLE CLIENT LOGGING You can enable or disable client logging for debugging purposes. This is an advanced option and should be used under guidance from a StorageDNA support professional. Page 32 of 63
CONFIGURATION AND SYSTEM SETTINGS CONFIGURING PREVIEWS Previews (often referred to as Proxies) are web-playable low-resolution video files that correlate directly to the original full resolution files that have been archived to LTO using DNA Evolution. DNA Evolution can automatically link the preview files to their corresponding high-resolution clips archived on LTO. DNA Evolution DOES NOT provide functionality for creating previews, but can link to previews created by a third-party transcoder. The DNA Evolution controller can be configured to point to a folder (local or networked) where the previews are located. DNA Evolution automatically indexes the previews folder at a scheduled interval to find any newly added previews. PREVIEW NAMING AND FOLDER REQUIREMENTS DNA Evolution uses the following heuristic for linking high-resolution media with low-resolution clips in the Previews folder. (NOTE: Previews are available only in the Search Clip tab via the Clip Info button) When you select a clip in the Clip view, DNA Evolution looks at the high-resolution filename on the LTO tape. (For clips that have multiple files, (e.g. P2, XDCAM), DNA Evolution looks for a preview file with the name of the video track of that clip. The preview filename must have a partial name match with the clip s filename. For example, if the high-resolution file is called Demo-Movie.MXF, then DNA Evolution will match that with any preview filename with the format <filename>-<low res name>.extension. So in this example, DNA Evolution will match on any of the following Demo-Movie-web.mpeg4, Demo-Movie-preview.MOV, etc. The preview folder can have any hierarchical structure. DNA Evolution indexes the entire structure and finds previews. GENERATING PREVIEWS Previews should be generated by a third-party transcoder. They are played using the embedded HTML-5 player in the browser. As such, any preview created must be playable by HTML-5 FlowPlayer. Refer to the document DNA Evolution 3.5: Workflow Guide for Automated Third-party Preview Generation for steps on configuring a third-party transcoder for automated preview generation. CONFIGURING THE PREVIEWS FOLDER Previews can be configured from the System Settings -> Setup Previews tab. Page 33 of 63
Previews can be created on any network share as long as they follow the Preview Naming and Folder Requirements. To connect the DNA Evolution Controller Server to the Preview folder, click on Connect. Here you can provide an IP/hostname and other details to connect to the Preview share. Once connected, DNA Evolution Controller will periodically index the Previews. At the bottom of the page, you can see the last indexed time (shown in next screenshot). CONFIGURING CATALOG BACKUPS DNA Evolution maintains an XML-based catalog of all file metadata. This catalog is backed up to a USB key attached to the DNA Evolution server. However, this catalog can be automatically backed up to a second location. This second location can be any network share. The backup share can be configured from the System Settings->Manage Backups tab. To connect the DNA Evolution Controller Server to a network folder for backups, click on Connect. Page 34 of 63
Here you can provide an IP/hostname and other details to connect to the Backup share. Once connected, DNA Evolution Controller will periodically backup the catalog. At the bottom of the page, you can see the last backup time. CONFIGURING EMAIL DNA Evolution allows an administrator to set an email server for outgoing emails for alerts. To configure the email server, browse to System Settings -> Email. Configure the email server using the provided fields (see next screenshot). Please refer to your internal IT department or StorageDNA support for setting up your mail server. CREATE, EDIT, PREVIEW METADATA TAGGING TEMPLATES DNA Evolution allows for the creation of advanced metadata tagging templates. Tagging templates in DNA Evolution have the following features: Ability to create unlimited key-value pairs Ability to create unlimited headings Ability to create fields of types date, description, string, dropdown, numeric To create, edit or preview the metadata tagging templates, perform the following: Login as an Admin User (template creation and editing is only available to Admin Users) In the Archive Settings Tab of any archive, click on the Create/Manage template button. The Add/Edit/Preview metadata page will open. Create/Add new template: Click on the Add tab. Type your template name. Page 35 of 63
You can select from a number of different fields to add (See next screenshot). After adding the fields, click on Create Template. Update existing template: Click on the Update tab. Select the template from the dropdown. You can add new fields by clicking on the different types OR you can update existing fields by selecting them from the row. Remove existing template: Click on the Remove tab. Select the template from the dropdown. A preview will be generated. You can then review and delete your template. Preview existing template: Click on the Preview tab. Select the template from the dropdown. A preview will be generated. NORMAL AND LIMITED USER METADATA TAGGING Normal and Limited users of DNA Evolution can add metadata based on the custom templates created by Administrators of the system. Log in as Normal or Limited user When in the Clip Info view, choose the + Metadata button The next popup will allow you to select from a dropdown of templates. Select the template to be used (See next screenshot) Fill in the appropriate information into the template and click on Save Data The new metadata is added to the clip Page 36 of 63
Multiple items can be selected for metadata tagging by multi-selecting items in the Clip Search tab, then choosing the Tag button SETUP AUTOMATED PREVIEW CREATION DNA Evolution can be configured to auto-generate previews. DNA Evolution relies on a third-party transcoder to create previews and auto-indexes the previews for display in the web. Please refer to the document DNA Evolution 3.5: Workflow Guide for Automated Third-party Preview Generation. MANUALLY INDEXING PREVIEWS Often times it may be necessary to have the preview indexer run right away rather than wait to index files on the 1 hour interval (e.g. during a demo). In such cases, it is possible to run the preview indexer on-demand. To run the preview indexer, follow these steps: Go to the DNA Controller Desktop (see next screenshot). Page 37 of 63
Run the script gen_previews.sh by double-clicking on the desktop shortcut and choose -> Run in terminal. This will index all the previews. Page 38 of 63
LTO/LTFS TAPE MANAGEMENT DNA Evolution enables the management of LTO tapes. Operations supported are Format, Mount, Unmount, Move, Verify, Import and Duplicate. NOTE: All operations can be performed directly using the web interface except the Import function. MOUNT, UNMOUNT Mount and unmounts ltfs mounts a tape. The tape must be loaded in the drive for the mount to succeed. To load a tape into a drive, either manually load the tape (single drive) or move the tape into a drive (autoloader). Steps on how to move a tape into the autoloader drive are covered in the Move section. MOVE (AUTOLOADERS ONLY) The Move operation allows you to use the autoloader robot to manually move a tape into a drive or move a tape from a drive back to its slot. To move a tape, follow these steps: Browse to Clients. Select the Archive Manager that the loader is connected to. Select the Tape Devices tab. Select one of the drives and click on View Loader. The autoloader screen will appear where you will be able to move tapes. To move a tape from a slot to a drive, select the tape from the slot. Then select the drive from the drop down (See screenshot below). Then click on Load Tape. To move a tape from a drive to a slot, select the drive from the drop down (See screenshot above) and click on Unload Tape. The autoloader keeps track of which tape resides in which slot and automatically moves the tape back to its original slot. FORMAT TAPES DNA Evolution allows you to format one or more tapes (in case you have an autoloader) in a single job. Formatting a tape performs the following operations: Loads the tape into a drive (in case you have an autoloader) Ensures the tape does not already exist in the DNA Evolution catalog Performs an LTFS format on the tape Registers the tape into the DNA Evolution system Page 39 of 63
Do not use HP or IBM-provided mkltfs commands directly (ie at the desktop level), as this will not register the LTO tape with the DNA Evolution tapes To format 1 or more tapes, perform the following: Step 1: Browse to My Clients -> Select the archive manager -> Click on Tape Devices tab. Step 2: Select the drive you want to use to perform the formatting. Step 3: Click on the Format button. The format screen will open (See next screenshot). Step 4: Click on Add. You will see a candidate list of tapes that are eligible for formatting. You can use the tape filter to narrow the list of tapes. (See next screenshot) Eligible tapes for formatting should be in the autoloader and should not already exist in DNA Evolution. If you don t see a tape you wish to format, ensure you have deleted it from the MyTapes list. Step 5: Select the tapes you wish to format. Check the Force Format box if the tapes were previously formatted in DNA Evolution (or other solution using LTFS) Force format will result in completely overwriting the previous LTFS index and all data on that tape. DNA Evolution will load each tape into the drive (in case of an autoloader) and format each tape sequentially. Tape format results can be viewed in the AdminDropbox (see next screenshot) Page 40 of 63
VERIFY TAPES You must verify your tapes before you delete the original source files residing on disk. Verification is critical to ensuring your content is safe. DNA Evolution enables you to verify media on one or more tapes. Verification is performed at a number of levels to ensure media is safe. Verification performs the following checks: LTFSCK of your tape to ensure that the LTFS volume is consistent LTFS mount to ensure it mounts successfully File listing on the LTO tape to ensure it matches with the catalog (DNA Evolution keeps a list of all files on tape) Reads every file back and runs a full MD5 checksum. Compares it to the original MD5 checksum recorded in the catalog (DNA Evolution computes the checksum of every file archived to LTO) Follow these steps to verify your tape: Step 1: Make sure all tapes you are verifying are write-protected. (See next image) Page 41 of 63
Step 2: Load the tape into a single drive or load them into the autoloader slots. Ensure that NO TAPES are loaded in the autoloader drives and are residing in their original slots. Loading tapes into a slot that was previously occupied by a tape in a drive can cause the autoloader to malfunction. Step 3: Browse to the My Clients tab. Select the archive manager and the Tape Devices tab. Select the drive that you want to use to verify your tapes. Step 4: Click on Verify Step 5: The verify screen will open (See next screenshot). Step 6: If using an autoloader, the verification screen allows for multiple tapes in the autoloader to be verified sequentially. If using a drive, your tape and drive will be pre-filled. Step 7: To add multiple tapes in an autoloader, click on Add. Step 8: The tapes selection screen will open (See next screenshot). Here you will see a list of all tapes that are eligible for verification. You can use the tape filter to further narrow down your tapes list. Page 42 of 63
Step 9: After you have added your tapes, click Start Tape Verify. DNA Evolution will start verifying your first tape and then automatically load the other tapes (if an autoloader). The results can be found in AdminDropbox/Validate Tape Reports/Tape ID. A number of files will be present (See next screenshot). Step 10: DNA Evolution creates a log for the ltfsck check and checksum logs for each archive and the files on the tape. In this example, tapevalidate-dna603-20-may-utf-report is the verification report for the archive may-utf on tape DNA603 (See next screenshot). Additionally, you will find a foundfiles.out (in this sample, the file is called 20-may-UTFfoundfiles.out ) which contains a list of all files found for this archive on the tape. The file tapevalidate-dna603-20-may- UTF-failed.log is a list of all files that failed verification. The xml file is for display purposes internal to DNA Evolution. To ensure your tapes are verified correctly, the ltfsck log should be successful and there should be no Failed logs. If there are failures, please save the logs and contact StorageDNA support for assistance. DUPLICATE TAPES Page 43 of 63
Duplication is a critical addition to ensuring your content is safe. You need a minimum of 2 LTO drives to create a duplicate tape. Tip: Before duplicating a tape, ensure the source tape is (a) Marked as full (b) Successfully verified and (c) Write lock is enabled. Ensure the target tape is (a) In the autoloader/drive (b) Has a barcode (should be a unique barcode and not the same as the source tape) (c) Formatted LTFS and part of the DNA Evolution system. DNA Evolution enables you to make duplicate (second) copies of your LTO tapes. Duplication allows you to keep a second copy at an offsite location ensuring that two copies of your data exist in two locations. Duplication will perform the following steps: Load source tape and target tape into separate LTO drives. To duplicate a tape, perform the following: Step 1: Browse to My Clients -> Select the archive manager -> Click on Tape Devices tab. Step 2: Select the source drive you want to use to perform the duplication. For Autoloader configs, ensure there are no tapes loaded in LTO drives. If you have a standalone drive, then ensure your source tape (that you want to duplicate) is loaded in the source drive. Step 4: After selecting the source drive, click on Duplicate Step 5: The Duplication Settings screen will open (See screenshot below). Page 44 of 63
Step 6: The Duplication Settings screen contains a grid of two columns for source and target tapes. You must first select your source tapes. To select source tapes, click on Add Step 7: The Tape List screen will open. The screen will show all tapes that are a candidate for duplication. (See screenshot below) Step 8: Select the tapes you wish to duplicate. Ensure there are available (formatted, part of the system and have zero bytes used) tapes in the autoloader. There should be as many tapes as you have selected to duplicate. Step 9: Once you have selected all the source tapes, you must select a target for each source tape. To select a target, click on target cell for each source tape and select the target tape from the dropdown. (See next screenshot) Step 10: Click on Start to begin the automated duplication process. Step 11: To view the duplication progress, you can reselect the drive from the Devices page and then click on Duplicate to return to the progress screen. VIEWING TAPES AND UTILIZATION, ADDING NOTES All tapes in your environment are listed in the Tapes tab (See next screenshot). Page 45 of 63
TAPE ONLINE/OFFLINE INDICATOR Every tape in the MyTapes page will appear in bold green if the tape is present in an auto-loader. If not present, it will appear in italicized grey. Tape online/offline indicator is supported only in autoloaders. Also you MUST initiate a tape move either via a job or manually to have the tape online/offline list refresh. Adding new tapes in the loader without causing a tape movement from the DNA Evolution will not refresh the list. TAPE LISTING For every tape, you can select the tape and view, and, for certain options, edit the following: Tape Serial: This cannot be edited and is the LTFS volume name, tape serial number, and bar code of the tape. Tape Location and Notes: Here you can specify where your tape is currently physically located. Tape Capacity: Every tape provides a visual clue as to how much space is left on it. Upon selecting the tape, you get a more detailed view of the total space on the tape and how much is remaining. Full capacity of LTO Tapes will not be available for use. This is due to LTFS metadata and an extra buffer maintained by DNA Evolution to ensure smooth LTFS file-system operation. Tape History Log: This table allows you to add log notes on a per-tape basis. DNA Evolution allows you to delete a tape. To delete a tape: Step 1: Browse to My Tapes DELETING TAPES Step 2: Select the tape you wish to delete and click on the icon. Page 46 of 63
Deleting a tape simply removes it from the list of tapes in the DNA Evolution tapes list. It does not remove the archive catalog entries (which must be manually deleted). And it does not remove any data from the tape. The tape must be Force Formatted for all data to be removed. COMMAND LINE LTFS TAPE IMPORT DNA Evolution supports importing LTO/LTFS tapes created by third-party applications. Tapes MUST BE formatted LTFS in order for DNA Evolution to be able to import them. LOCATION OF COMMAND LINE TAPE TOOLS Tape tools are accessible via command line (Terminal access) on each archive manager. Tape tools cannot be run from an archive client. On Mac (Must be logged in as SUDO user): Open Terminal cd /Applications/StorageDNA/accessDNA.app/Contents/MacOS/ Here you will find tape_validate (bulk verify tool), format_changer (bulk format tool), ltfs_import (bulk import tool), ltfs_copy (bulk duplicate tool) NOTE: All operations (except tape import) can be run directly from the web interface. On Linux: Open Terminal on desktop or ssh to server cd /opt/sdna/bin Here you will find tape_validate (bulk verify tool), format_changer (bulk format tool), ltfs_import (bulk import tool), ltfs_copy (bulk duplicate tool) NOTE: All operations (except tape import) can be run directly from the web interface. LTFS TAPE IMPORT The bulk import script enables LTO/LTFS tapes written by third-party software (OR) LTO/LTFS tapes written by another DNA Evolution system to be imported (please see the tips below). Tape import works in the following manner: LTO/LTFS tapes written by third-party and other DNA Evolution systems can now be used within DNA Evolution. When a third-party LTO/LTFS tape is imported into an archive, its file-system structure is read from the tape and a catalog structure is created under its tape ID. After this process, the data on tape can be searched as if the tape was archived using DNA Evolution. Page 47 of 63
You MUST import using the checksum option to be able to run a verification on these tapes, which could take substantially longer than importing without the checksum option, so please plan accordingly. When an LTO/LTFS tape written by another DNA Evolution system is imported into an archive, its file-system structure is read from the tape and a catalog structure is created exactly as if it was created on that system. After this process, the data on tape can be searched as if the tape was archived on that system. However, as in the previous case, you MUST import using the checksum option to be able to run verification on these tapes. To run tape import on a tape written by DNA Evolution, follow these steps: Locate the bulk import script based on the instructions in Section: Location of Command Line Tape Tools. Once in the scripts directory, type./ltfs_import for instructions on how to run the bulk import Examples: o./ltfs_import -archive "Nov13-sdna-tape-import-test" -source st1 -serials DNA603 checksum The above command will import a DNA Evolution tape into the archive named Nov13-sdna-tapeimport-test and will also compute the checksums of the files. (See next screenshot: archive after import.) Omit the checksum flag if you do not want checksums computed. To run tape import on a tape written by open LTFS tools (drag n drop) or third-party LTFS archiving software, follow these steps: Locate the bulk import script based on the instructions in Section: Location of Command Line Tape Tools. Once in the scripts directory, type./ltfs_import for instructions on how to run the bulk import Examples: o./ltfs_import -archive "Nov13-nonsdna-import-test" -source st1 -serials DNA603 checksum The above command will import a third-party tape into the archive named Nov13-nonsdna-importtest and will also compute the checksums of the files. (See screenshot: archive after import.) Omit the checksum flag if you do not want checksums computed. Page 48 of 63
Checksum during import will dramatically increase time to import (as every file has to be read from tape) versus non-checksum where only the file-system index from LTFS is read. However, if you do not enable checksum, you cannot verify these tapes. Page 49 of 63
REPORTS DNA Evolution allows multiple reports to be automatically generated. These reports can be accessed via the Reports tab on the main screen. (See next screenshot) DNA Evolution allows a number of different types of reports to be generated. All reports are generated into the administrator dropbox in a folder called reports. This is accessible via the web or by connection over a network share. The following is a list and description of each report type. Archive Files Reports o List files on Archive: This report generates a list of all files in the archive along with their checksums. Tape Reports o o o List of Tapes not Verified: This report allows you to determine a list of tapes that have not been verified in a particular period. Once the report is received, the tape list can be fed to the bulk verification tool. List of Tapes not Duplicated: This report allows you to determine a list of tapes that have not been duplicated in a particular period. Once the report is received, the tape list can be fed to the bulk duplication tool. List of files on Tape: Often times it becomes necessary to have a list of files on tape. This report creates a list of files on each tape by archive. Job Reports: Page 50 of 63
o o o Jobs failed: This report gives a list of all jobs that have failed in a particular time period. Jobs succeeded: This report gives a list of all jobs that have succeeded in a particular time period. Jobs with warnings: This report gives a list of all jobs that have failed in a particular time period. The reports can be run automatically every night at mid-night or instantly. The report notification can also be sent via email. Page 51 of 63
ARCHIVE CLIENTS AND SERVERS MANAGEMENT All clients and servers are managed via the Clients tab. You can perform a number of operations on clients and servers. THE CLIENTS VIEW On the Clients main tab, you will see a list of all clients and servers in your environment. You can perform the following actions via the action toolbar to the left of the clients listing: For each client, you can edit and view information in the following sub-tabs: General Information Point-to-Point Connectivity Tape Devices (if it is a server) Each of the sections above is highlighted in more detail below. You can view but cannot edit the information under the License Information sub-tab. (The information in this sub-tab is discussed in detail in section: License Management. General Information In this section, you can see information about each client and server. Specifically, the following items need further explanation: Client Machine Color: Black signifies a healthy archive client or archive manager machine that has an active connection to the DNA Server. Red signifies an archive client or archive manager machine that has lost connection due to networking issues or is not turned on. A Red machine cannot run jobs or be used for configuring jobs. Client Name (Editable): This is purely used for visual identification purposes and has no effect on hostnames. Default run as user (Editable): This is the default local username under whose credentials jobs will be created. Page 52 of 63
License Type: This signifies whether the system is an archive client or server. An archive client has the license type (LIC_CLIENT_ARCHIVECLIENT), and an archive manager has a license type (LIC_CLIENT_ARCHIVESERVER). User/Group Access List: As an admin, you can add users/groups to the Authorized Users list for this client or server. If a user is not on this list, that user will not be able to view this system for archive or restore jobs. POINT-TO-POINT CONNECTIVITY This section is dedicated largely to configured network transfers for DNA Sync. It is not relevant to a DNA Evolution configuration. TAPE DEVICES In this section, you can see a list of all tape hardware connected to the client or server. You will see only tape devices connected to archive managers. For standalone environments, you will see one or more tape drives connected. You can perform LTFS operations directly on the tape currently inserted in the tape drive. To perform an operation on a tape, insert a tape in the drive. A list of allowable operations includes Mount/Unmount, Format, Verify, Duplicate. These are covered in the LTO/LTFS Tapes Management section of this guide. Page 53 of 63
USERS AND GROUPS TABS REFERENCE GUIDE DNA Evolution allows you to create multiple users and groups in the web interface. As an admin, you can use users and groups to give different levels of access to different users. You can decide which clients and servers a user or group has access to and which folders on a client machine are accessible to a user. ADDING, EDITING, DELETING USERS The Users tab allows you to add and edit users. To add a user, click on the Add/Plus icon in the action toolbar. Fill in the required information about the user. You also will be required to select the role type. An admin user will be able to see all aspects of the system. A normal user will be able to see only the client machines, servers, archives, jobs that they created, and any groups to which they belong. A limited user will only be able to see the client machines, servers and archives that they have been included in, but they cannot create, run or perform any function beyond browsing and searching within the assigned archive(s). To delete a user, simply select the user and click on the Delete/Minus icon. Additionally, as an admin, you can add a user to, or remove one from, any groups from this screen. Page 54 of 63
ADDING, EDITING GROUPS The Groups tab allows you to add and edit groups. To add a group, click on the Add icon in the action toolbar. Fill in the required information about the group (name and description). To delete a group, simply select the group and click on the Minus icon. Additionally, as an admin, you can add a user to, or remove one from, any groups from this screen. NORMAL USER WORKFLOW Normal user workflows allow an organization to create users that can perform all the operations of archive, search and restore but are limited to certain archives, client machines (Archive clients or managers) and also to certain folders and volumes on the client machines. A typical workflow would be: Step 1: Create a normal user Step 2: Assign an archive manager and/or client to the normal user Step 3: Normal user can now login and create their own archives GROUP WORKFLOW Group Workflows allow an organization to create archives and client machines (archive managers and clients) under a group, which enables any user who belongs to that group to have access to those archives and clients. A typical workflow would be: Step 1: Create a group Step 2: Assign one or more users to that group Step 3: Assign an archive manager or client to that group Page 55 of 63
Step 4: Create an archive with group access Step 5: Now all users in that group will have access to both the archive and client machines Page 56 of 63
LICENSE MANAGEMENT Machines that have DNA Evolution client software installed are administered via the Clients -> License Information subtab on the web management console. DNA Evolution enables you to manage licenses if you are the assigned owner of the license. If you do not see any licenses, it means you have not been assigned any licenses. LICENSE USAGE AND POLICY All DNA Evolution licenses are assigned by default and owned by the admin user. Only the user who is the owner of a license ( admin user by default) can see the licenses in the Clients -> License Information sub-tab. If you need licenses assigned to another user, you need to advise your reseller and/or StorageDNA at time of purchase. Upon client software installation using a license key, the client machine by default will appear only for the admin. The admin user will need to assign the machine to a normal user before the normal user can create syncs to and from the installed machine. LICENSE LISTING All licenses are listed under the Clients -> License Information sub-tab. Use the following key for licenses: Black: Assigned valid license Green: Available and unassigned license Red: Expired license TO ADD A USER TO A MACHINE After installation, you may want to give another user access to use that client for archive and restore operations. First, browse to the Clients tab. In the General Information section, you can add users to the authorized users list. Add the user you would like to give access and save. Now the normal user will be able to use the client machine for DNA Evolution functions such as archive and restore. TO REMOVE/UNINSTALL A CLIENT MACHINE AND RECLAIM THE LICENSE When a client machine is uninstalled, the associated license key will be reclaimed. Additionally, you can simply delete a machine from the client list. This has the same effect in returning the license to the available pool. Page 57 of 63
UPGRADING DNA EVOLUTION CONTROLLER, ARCHIVE MANAGER, ARCHIVE CLIENT The following section guides you through the process of upgrading the DNA Evolution Controller, Archive Manager and Archive Client. UPGRADING DNA EVOLUTION CONTROLLER AND LINUX ARCHIVE MANAGER The following section has instructions on how to perform a controller and archive manager upgrade on the Linux server. Step 1: Download and copy the upgrade package. The upgrade package can be downloaded from the upgrade email sent out with every release. Download the upgrade package to a machine that can mount the ADMINSTRATOR DROPBOX network share. (See section: Network Connecting to the DNA Server Administrator Dropbox.) The upgrade file will be a.run file (e.g. DNAEvolution-v3.5-8XXX.run) where the updater has the following format <PRODUCT NAME>-RELEASE-BUILD o Copy the.run upgrader package to the following directory: ADMINDROPBOX/Updates (see screenshot) Step 2: Run the updater Login to the DNA Controller via terminal OR login via a keyboard, monitor and mouse On the desktop, run the script update-server by double-clicking on its desktop shortcut Upon running the script, you will see a menu to update the server (see next screenshot) Page 58 of 63
Select Update Evolution Server You should see the updater package you copied to the Updates folder. Select the package using space bar (see next screenshot). Select OK to start the upgrade process. The upgrade process will take some time to run. After completion, exit the menu. Page 59 of 63
UPGRADING ARCHIVE MANAGER AND ARCHIVE CLIENT Tip: The Linux archive manager running on the central controller is auto-updated as part of the controller upgrade process, while configurations with Mac-based Archive Managers will require extra steps. To upgrade archive clients or archive managers NOT running on the controller machine, you must download the installers from the DNA Evolution web interface (similar to running an install from scratch). Follow the same steps in section: DNA Evolution Mac Archive Manager Setup and section: DNA Evolution Archive Client Setup. Choose the Upgrade option when running the client installers. Page 60 of 63
CATALOG MANAGEMENT IMPORTING AND EXPORTING ARCHIVES VIA XML DNA Evolution can export metadata from an archive. An archive s metadata export is a zip file. Upon opening the zip file, you will find a file-system that looks exactly like the archive. Every filename matches the filename archived to LTO. However, in this case, the file is an XML file. Each file contains all the metadata that is extracted and indexed by DNA Evolution. Here is a sample metadata XML file: <file type="file" > <filename><![cdata[avid_mob.0a01.4fd8b4fd8b4b3.mxf]]></filename> <proxy-path><![cdata[000 - Avid MediaFiles/1/Volumes/JupiterEnt p2 test/avid MediaFiles/MXF/1]]></proxy-path> <source-path><![cdata[/volumes/avid MediaFiles/MXF/1]]></source-path> <archive-name><![cdata[000 - Avid MediaFiles]]></archive-name> <clip-name><![cdata[so-20120118-a02.23 0024FR.new.01]]></clip-name> <actions> <action version="1" size="365153" mod-time="1339602100" run="20120628-093651" checksum="08db823c9f941a6d83a5a82ab32b0111" write-time="1340901430" write-index="8" type="upd" tapeid="ayx407" /> </actions> </file> Currently, the XML contains basic file information including the checksum and tapeid it is stored on. The exported XML file serves as a metadata backup that can be used in the following ways: Way to offload an old archive from a DNA Evolution server to make space for newer archives Way to move an archive from one DNA Evolution server to another (NOTE: Archives can be exported and imported) Way to move metadata to another third-party system, such as an asset management system HOW TO EXPORT AN ARCHIVE To export an archive, select the archive you wish to export. Then click the Export icon in the main action area (shown in next image). Page 61 of 63
Upon exporting the archive, a success message should appear (shown below). The archive export process runs at specific timed intervals and can take a few minutes to complete. You can then browse to the zip file in the Admin Dropbox and download the zip file (shown below). Zip files are named with the following convention sdnaexport<archive name>-<timestamp>.zip. Download the zip file by selecting the file and clicking the download button (shown below). HOW TO IMPORT AN ARCHIVE An archive can be imported back into the same DNA Evolution controller or it can be imported into another DNA Evolution controller. The import will bring in the relevant tapes, create and populate the archive. To import, first you will need to copy the zip file to the DNA Evolution controller Dropbox. You can do this by connecting to the Admin dropbox network share. (refer to previous Instructions) The file can be copied into the base directory or you can create a sub-directory. After you have copied the zip file, follow these steps: Page 62 of 63