StorageGRID Webscale 10.2

Transcription

1 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments NetApp, Inc. 495 East Java Drive Sunnyvale, CA U.S. Telephone: +1 (408) Fax: +1 (408) Support telephone: +1 (888) Web: Feedback: Part number: _A0 December 2015

2

3 Table of Contents 3 Contents Understanding the StorageGRID Webscale expansion procedure... 4 Downloading the StorageGRID Webscale installation files... 4 Required materials... 5 Planning an expansion of your StorageGRID Webscale system... 6 Supported expansions... 6 Limitations of the expansion procedure... 7 Verifying hardware and networking... 8 Noting the StorageGRID Webscale software version... 8 Expanding storage on Storage Nodes What storage volumes are Adding direct-attached or SAN storage volumes in OpenStack Dashboard Expanding your StorageGRID Webscale system Preparing to expand an existing data center site Downloading the expansion SAID package Deploying the StorageGRID Webscale Installer in OpenStack Dashboard Deploying grid nodes Uploading the SAID package and preparing the deployment Deploying grid nodes in OpenStack Dashboard Deploying StorageGRID Webscale appliance Storage Nodes Running the Grid Expansion: Initial grid task Initializing storage volumes for Storage Nodes Setting the installation state Activating the nonprimary Admin Node services Starting the StorageGRID Webscale services Running the Grid Expansion: Add Server grid task Applying hotfixes and maintenance releases Running the Clean Up Unused Cassandra Keys grid task Reverting to the default boot mode Configuring your expanded StorageGRID Webscale system Verifying that the Storage Node is active Copying audit logs Copying the NMS database Troubleshooting StorageGRID Webscale software installation fails Contacting technical support Glossary Copyright information Trademark information How to send comments about documentation and receive update notifications Index... 48

4 4 Understanding the StorageGRID Webscale expansion procedure You must understand all of the steps required to successfully expand a StorageGRID Webscale system without interrupting system operations. Read this guide in full before you begin an expansion. It is assumed that you are familiar with the StorageGRID Webscale system and that the expansion has been planned in advance. It is also assumed that all required hardware is installed, connected, and configured to specifications. This guide is intended for technical personnel trained to install and support the StorageGRID Webscale system. An advanced level of computer literacy is required, including knowledge of Linux/ UNIX command shells, networking, virtualization, and server hardware setup and configuration. Note: Before you initiate an expansion, you must ensure that any grid tasks from previous provisioning revisions are complete. You cannot start an expansion while previous expansion, upgrade, or decommissioning procedure is in progress. Related information StorageGRID Webscale 10.2 Grid Primer StorageGRID Webscale 10.2 Administrator Guide Downloading the StorageGRID Webscale installation files Before you can use StorageGRID Webscale features, you must download the software from the NetApp Support Site. 1. Access the NetApp Support Site at mysupport.netapp.com. 2. Download the StorageGRID Webscale Installer (SGI) archive file (SGWS-<version>- <SHA>.tgz). 3. Extract the files from the archive file and use the information in the README file to select the correct SGI deployment template file (.yaml) for your environment.

5 Understanding the StorageGRID Webscale expansion procedure 5 Required materials Gather the materials listed in the following table to prepare for expansion. Item StorageGRID Webscale OpenStack Heat Template Notes Select the appropriate OpenStack Heat template for your OpenStack deployment: SGI_Public_Routing_Template.yaml StorageGRID Installer Heat template for deployments to externally routable tenant networks that do not require the use of floating IP addresses. SGI_Private_Routing_Template.yaml StorageGRID Installer Heat template for deployments to private tenant networks. This template creates floating IP addresses and associations. These files must be downloaded from the NetApp Support Site at mysupport.netapp.com as part of the StorageGRID Webscale Installer archive file (.tgz). StorageGRID Webscale Installer Virtual Machine Disk file Hotfix and/or Maintenance Release ISO image OpenStack software and documentation Passwords.txt file Provisioning passphrase The Virtual Machine Disk (.vmdk) file used to deploy the StorageGRID Webscale Installer. This file must be downloaded from the NetApp Support Site at mysupport.netapp.com as part of the StorageGRID Webscale Installer archive file (.tgz). Determine whether or not a hotfix and/or maintenance release has been applied to the grid node type that is being added. The expansion grid node must be updated to the same hotfix or maintenance release as the other installed grid nodes of the same type. See the storage-gridrelease version number listed on the Grid Topology > grid node > SSM > Services > Overview > Main page. To acquire hotfixes and maintenance releases, contact technical support. For supported versions, see the Interoperability Matrix. NetApp Interoperability Matrix Tool. The file is located in the SAID package generated during provisioning, and contains the required system passwords. The passphrase is created and documented when the StorageGRID Webscale system is first installed. The provisioning passphrase is not in the Passwords.txt file.

6 6 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments Item Service laptop Notes The StorageGRID Webscale system is installed through a service laptop. The service laptop must have: Microsoft Windows operating system Network port Telnet and SSH client (for example, PuTTY) SCP tool (for example, WinSCP) to transfer files to and from the primary Admin Node. Supported browser The following browsers have been tested with StorageGRID Webscale to verify compatibility: Google Chrome 43 Microsoft Internet Explorer 11.0 Mozilla Firefox Documentation Administrator Guide Release Notes Related information StorageGRID Webscale 10.2 Administrator Guide StorageGRID Webscale 10.2 Grid Designer User Guide Planning an expansion of your StorageGRID Webscale system You must have a clear understanding of the procedures you need to complete to successfully expand your StorageGRID Webscale system. For information on the limits placed on the expansion of your StorageGRID Webscale system, see Limitations of the expansion procedure on page 7. To expand you StorageGRID Webscale system by adding additional storage volumes (rangedbs) to Storage Nodes, see Expanding storage on Storage Nodes on page 10. To expand your StorageGRID Webscale system by adding grid nodes to existing data center sites, you must complete the expansion process in the order listed: 1. Expanding your StorageGRID Webscale system on page 13 You need to configure the expansion in the NMS MI, and then use the StorageGRID Webscale Installer and OpenStack Dashboard to deploy the grid nodes. 2. Configuring your expanded StorageGRID Webscale system on page 32 Perform any additional configuration procedures that you need to complete, which depend on the type of grid node added. Supported expansions You can expand your StorageGRID Webscale system to meet your storage requirements by adding storage to Storage Nodes, or by adding additional grid nodes to an existing site. The procedures that

7 Understanding the StorageGRID Webscale expansion procedure 7 you run vary depending on whether you are adding storage to a Storage Node, or adding grid nodes to a data center site. Note: You cannot add data center sites to StorageGRID Webscale grids deployed in the OpenStack environment. Adding storage volumes to Storage Nodes You can add storage volumes (LUN) to a Storage Node to increase its storage capacity. In most cases, you will need to add equivalent storage capacity to more than one Storage Node at a time. The content placement instructions for active information lifecycle management (ILM) rules determine the number of storage volumes you must add. For example, if an ILM rule's content placement instructions are configured to make one replicated copy at every data center site, you must add an equivalent amount of storage to Storage Nodes at each site. Note: You cannot add storage volumes to a Storage Node during upgrade. Adding grid nodes to data center sites You can expand your StorageGRID Webscale system by adding new grid nodes to existing data center sites: Admin Nodes: Adding Admin Nodes is only supported for sites that do not already include an Admin Node. You can only have one Admin Node per data center site. Storage Nodes: In most cases, you will need to add more than one Storage Node at a time. The content placement instructions for active information lifecycle management (ILM) rules determine the number of Storage Nodes you must add. For example, if an ILM rule's content placement instructions are configured to make one replicated copy at every data center site, you must add an equivalent number of Storage Nodes at each site. However, if an ILM rule's content placement instructions are configured to make one replicated copy at two of three data center sites, you must only add an equivalent number of Storage Nodes at two data center sites and not at the third. Archive Nodes: Each Archive Node supports archiving to the cloud through the S3 API. API Gateway Nodes Related tasks Expanding your StorageGRID Webscale system on page 13 Expanding storage on Storage Nodes on page 10 Related information StorageGRID Webscale 10.2 Administrator Guide Limitations of the expansion procedure Before you expand your StorageGRID Webscale system, you should be aware of the limitations when adding Admin Nodes and Storage Nodes. Note: You cannot add data center sites to StorageGRID Webscale grids deployed in the OpenStack environment. These limitations are enforced by the Grid Expansion maintenance procedure in the NMS MI. Admin Nodes You cannot add multiple Admin Nodes to a single data center site.

8 8 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments You can add an Admin Node to a data center site if the site was deployed without an Admin Node. The primary Admin Node is always deployed during the initial implementation, and cannot be added using the expansion procedure. Storage Nodes While you can add Storage Nodes to your StorageGRID Webscale system at any time to increase the storage capacity of the system, you cannot decommission a Storage Node, or change the IP address of a previously installed Storage Node during the expansion process. Storage expansion requirements depend on the StorageGRID Webscale system s information lifecycle management (ILM) rules, the type of copies made (erasure-coded or replicated), and the StorageGRID Webscale system s topology. You must always add enough storage to take copies (and the type of copies) into account. If an ILM rule creates replicated copies, you must consider the number of copies created and their distribution through storage pools. For example, if an ILM rule requires one replicated copy at data center site 1 (DC1) and one copy at data center site 2 (DC2), you must add an equivalent amount of storage at each site to increase the overall capacity of the StorageGRID Webscale system. If an ILM rule creates erasure-coded copies, you must consider the Erasure Coding profile in use and the reason you are adding Storage Nodes. If you are adding Storage Nodes because the system's existing Storage Nodes are nearing capacity, it is not enough to just add new Storage Nodes to the system. You must add enough Storage Nodes to support the erasure-coding scheme in use. For example, to support an Erasure Coding profile using a 6+3 scheme, you must add at least nine Storage Nodes to the system. Verifying hardware and networking Before you begin the expansion of your StorageGRID Webscale system, you must ensure that you have installed and configured the necessary hardware to support the StorageGRID Webscale system. You must also verify network connectivity between servers at the data center site, and confirm that the primary Admin Node can communicate with all expansion servers that are intended to host the StorageGRID Webscale system. Noting the StorageGRID Webscale software version At the end of the expansion process, all grid nodes of the same type must be running the same version of the StorageGRID Webscale software. This includes applied hotfixes and maintenance releases. About this task Before performing an expansion, note the current version number of the StorageGRID Webscale software running on the grid node type that is to be added and then use it at the end of the expansion process to compare against expansion grid nodes to determine if hotfixes, maintenance releases, or both must be applied to expansion grid nodes. 1. In the NMS Management Interface (MI), go to Grid Topology > grid node > SSM > Services > Main. 2. Scroll to the Packages section at the bottom of the page and note the version number listed under the Version attribute.

9 Understanding the StorageGRID Webscale expansion procedure 9 After you finish Use this version numbers at the end of the expansion process to compare against expansion grid nodes to determine if hotfixes, maintenance releases, or both must be applied to expansion grid nodes. See Applying hotfixes and maintenance releases on page 29.

10 10 Expanding storage on Storage Nodes You can expand the storage capacity of a Storage Node with less than 16 storage volumes by adding additional storage volumes. Storage Nodes support a maximum of 16 storage volumes. About this task Storage volumes are added to a Storage Node to increase the storage capacity of the Storage Node. When adding storage volumes to a Storage Node, confirm that this is not the only Storage Node with less than the maximum number of storage volumes. Typically you will need to add storage volumes to more than one Storage Node in order to support the requirements of the StorageGRID Webscale system's ILM rules. For example, if the StorageGRID Webscale system's ILM rules make replicated copies across multiple data center sites, you may be required to add an equivalent number of storage volumes to Storage Nodes at each data center site. Note: You cannot add storage volumes to a Storage Node during upgrade. What storage volumes are The underlying storage of a Storage Node is divided into a fixed number of storage volumes. Storage volumes are drives or partitions that act as mount points for storage. Storage volumes are also referred to as object stores. You can view the list of storage volumes on a Storage Node through the NMS MI. Each item in the LDR > Storage > Overview > Object Stores table corresponds to a mount point listed in the Volumes table on the SSM > Resources > Overview page of the same Storage Node. That is, the object store with an ID of 0000 corresponds to /var/local/rangedb/0 in the SSM > Resources > Overview > Volumes table. Object metadata is always stored to object store 0. Adding direct-attached or SAN storage volumes in OpenStack Dashboard If a Storage Node includes fewer than 16 storage volumes, you can add additional volumes to that Storage Node to increase its storage capacity. Each Storage Node can support up to 16 storage volumes. Before you begin The existing Storage Node must be integrated with direct-attached or SAN storage. It is not integrated with NFS mounted storage volumes. You must have the Passwords.txt file. About this task To increase the total usable storage capacity of the StorageGRID Webscale system, generally you must add storage capacity to more than one Storage Node. 1. Install the new storage hardware. For more information, see the documentation provided by your hardware vendor.

11 Expanding storage on Storage Nodes Add one or more hard disks to the virtual machine for use as storage (rangedbs): a. Log in to the appropriate tenant and project in OpenStack Dashboard. b. Select Project > Compute > Volumes. c. Click Create Volume. d. In the Create Volume dialog box, enter the volume details: Volume Name: Enter a new name for the storage volume that follows the naming convention for existing storage volumes, for example, DC1-S1 RangeDB4. Description: Enter a meaningful description for the expansion storage volume, for example, StorageGRID Node DC1-S1 RangeDB 4. Volume Source: Select No source, empty volume from the drop-down list. Type: Select No volume type from the drop-down list. Size (GB): Enter the appropriate size for the storage volume (rangedb) in gigabytes. Availability Zone: If Availability Zones are used by your StorageGRID Webscale system, select the appropriate zone, otherwise, select Any Availability Zone from the drop-down list. e. Click Create Volume. The storage volume is created and the new entry is added to the Volumes table. 3. In the Actions column for the new storage volume, select Manage Attachment from the dropdown list. 4. In the Manage Volume Attachments dialog box, select the grid node instance to attach the storage volume to from the drop-down list. 5. Click Attach Volume. The storage volume is attached the grid node virtual machine. The name of the new volume in the virtual machine is listed in the Attached To column, for example, /dev/sde. You can compare this value with the output of the add_rangedbs.rb script to confirm that the correct device is being added. 6. From the service laptop, log in to the Storage Node as root using the password for listed in the Passwords.txt file. 7. Configure the new storage for use by the Storage Node: a. Configure the new storage volumes: add_rangedbs.rb This script finds any new storage volumes and prompts you to format them. b. Accept the reformatting: y c. If any of the drives have previously been formatted, you are asked if you want to reformat them. To proceed: y d. If you do not want to format the drives: n. 8. Check that the services start correctly; a. View a listing of the status of all services on the server: storagegrid-status

12 12 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments The status is updated automatically. Wait until all services are Running or Verified. b. Exit the status screen: Ctrl+C 9. Verify that the Storage Node is online: a. Log in to the NMS MI using the Vendor account or an account with Grid Management permissions. b. Go to Grid Topology > Storage Node > LDR > Storage > Configuration > Main. c. If the Storage State Desired drop-down list is set to Read-only or Offline, select Online. d. Click Apply Changes. 10. If you want to view information about the new storage volume in the NMS MI, go to Grid Topology > Storage Node > LDR > Storage > Overview > Main and view the details in the Object Stores table.

13 13 Expanding your StorageGRID Webscale system Before you can add grid nodes or a data center site to your StorageGRID Webscale system, you must use the NMS MI to prepare the system, and then you must complete a specific series of steps to add grid nodes to your deployed and operational StorageGRID Webscale system. 1. Preparing to expand an existing data center site on page 14 You can add grid nodes to a deployed data center site to add additional storage or archiving capacity, load balancing, or to add an Admin Node to a site that does not already have one. 2. Downloading the expansion SAID package on page 15 You must download the latest version of the SAID package, which contains deployment information for the expansion grid nodes. 3. Deploying the StorageGRID Webscale Installer in OpenStack Dashboard on page 16 You must deploy the StorageGRID Webscale Installer (SGI) using the OpenStack Dashboard. The SGI is then accessed through a web browser and used to deploy grid nodes. 4. Deploying grid nodes on page 17 You must deploy each expansion grid node defined in your grid specification file. You deploy Admin Nodes, API Gateway Nodes, and Archive Nodes on virtual machines. You deploy Storage Nodes either on virtual machines or on physical servers if you are installing the StorageGRID Webscale appliance. 5. Running the Grid Expansion: Initial grid task on page 24 Running the Grid Expansion: Initial grid task adds configuration information to the StorageGRID Webscale system for all expansion grid nodes at all data center sites. The initial configuration grid task is run once per expansion. 6. Initializing storage volumes for Storage Nodes on page 26 You need to initialize the storage volumes (rangedbs) on each Storage Node to make the storage available to the StorageGRID Webscale system. 7. Setting the installation state on page 26 You need to activate the installation state for each expanded grid node. 8. Activating the nonprimary Admin Node services on page 27 If your expansion includes adding a nonprimary Admin Node to an existing data center site, you need to run a script to activate services. 9. Starting the StorageGRID Webscale services on page 27 You need to start StorageGRID Webscale software on all newly added grid nodes by enabling services for each expansion grid node. 10. Running the Grid Expansion: Add Server grid task on page 28 You must run the Grid Expansion: Add Server grid task for each grid node added in the expansion. This prepares the StorageGRID Webscale system for the new grid node. 11. Applying hotfixes and maintenance releases on page 29 You must confirm that the expansion grid node is running the same version of the StorageGRID Webscale software as the other grid nodes of the same type. If it is not, you must apply the necessary hotfixes or maintenance releases to update the expansion grid node. 12. Running the Clean Up Unused Cassandra Keys grid task on page 30 When you add one or more Storage Nodes to a previously deployed data center site, you must run the Clean Up Unused Cassandra Keys grid task generated by the system. You must run the grid task generated for each data center site. 13. Reverting to the default boot mode on page 31

14 14 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments After you have completed the procedure to expand a grid node in maintenance mode, you need to reset the grid node to boot the operating system normally. Preparing to expand an existing data center site You can add grid nodes to a deployed data center site to add additional storage or archiving capacity, load balancing, or to add an Admin Node to a site that does not already have one. Before you begin Any previously started expansion, upgrade, or decommissioning procedures must be completed before starting a new expansion. 1. Sign in to the NMS MI using the Vendor account. 2. Go to Grid Management > Grid Maintenance > Expansion. 3. Enter your provisioning passphrase and click Verify. 4. Click Start. 5. In the Site Selection dialog box, select the data center site to add grid nodes to from the dropdown list and click Save. 6. You must create an entry for each grid node you want to add: a. Click Create. b. In the Grid Node Configuration dialog box, specify the settings for the grid node: Type: The type of grid node to create. You can choose from the four grid node types, plus Storage Node with ADC, which creates a Storage Node that hosts the Administrative Domain Controller (ADC) service. The Storage Node with ADC grid node type must be selected for the first three Storage Nodes at each data center site. Name: The name that will be displayed in the NMS MI. The text box is auto-populated with the suggested name based on the standard naming conventions. NTP Role: The grid node's Network Time Protocol (NTP) role. The options are Automatic, Primary, and Client. Automatic assigns the Primary option to Admin Nodes and Storage Nodes with ADC, and the Client option to all other grid node types. IPv4 address: The IP address for the grid network interface. For example: Network Mask: The network mask for the grid network interface. For example: Default Gateway: The default gateway of the grid node. For example: Optional. You can add additional devices to specify routing for other networks: a. Click Create at the bottom of the Devices list. b. Select the new device entry. c. In the IPv4 Address text box, enter the IP address (v4) for the grid node. Each grid node must have an unique IP address. For example:

15 Expanding your StorageGRID Webscale system 15 d. In the Network Mask text box, enter the network mask for the grid network interface. For example: e. Click Create above the Routes table and enter the required information: Network: The route's destination network. Network Mask: The network mask for the routing network address. Gateway: The gateway's IP address for this route. f. Click Save. 8. Click Save. 9. If necessary, you can add additional grid nodes of the same type: a. Select the existing grid node you want to clone. b. Click Clone. A new entry is created with the suggested grid node name and Type, Network Mask, and Default Gateway values auto-populated based on the information for the grid node you are cloning. c. Enter the IPv4 Address value, and make any necessary adjustments to the cloned grid node entry. d. Click OK. 10. Review the configuration settings for the grid nodes you are adding, and click Edit to update the values for the selected grid node if you notice any errors, or click Remove to remove a grid node entry entirely. 11. Click Expand. 12. Click OK in the confirmation dialog box. Result The preparation for the expansion is complete. You need to download the latest copy of the SAID package, and then deploy the grid nodes using the StorageGRID Webscale Installer to complete the physical expansion of your StorageGRID Webscale system. Related tasks Downloading the expansion SAID package on page 15 You must download the latest version of the SAID package, which contains deployment information for the expansion grid nodes. Expanding your StorageGRID Webscale system on page 13 Downloading the expansion SAID package You must download the latest version of the SAID package, which contains deployment information for the expansion grid nodes. Before you begin You must have the provisioning passphrase.

16 16 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments About this task The SAID package is included in the provisioning backup archive file (.zip). 1. Log in to the NMS MI using the Admin or Vendor account. 2. Select Grid Management > Grid Maintenance > Provisioning Backup. 3. Enter the provisioning passphrase and click Start Download. The download starts immediately. Depending on your browser version, you may be prompted to select the location to save the file, or it may be saved automatically to the default download location. 4. Extract the SAID file (.zip) from the downloaded archive file (.zip) to a location that is accessible to the StorageGRID Webscale Installer. Attention: The Grid Provisioning archive file contains encryption keys and passwords that you can use to obtain data from the StorageGRID Webscale system. You must download the file to a secure location. Deploying the StorageGRID Webscale Installer in OpenStack Dashboard You must deploy the StorageGRID Webscale Installer (SGI) using the OpenStack Dashboard. The SGI is then accessed through a web browser and used to deploy grid nodes. Before you begin OpenStack software must be installed and correctly configured. You must have the SGI virtual machine disk file (.vmdk) and the correct Heat template file (.yaml) for your deployment. These files must be extracted from the StorageGRID Webscale installaton file (.tgz) downloaded from the NetApp Support Site at mysupport.netapp.com. You must have network configuration information for the SGI (IP address, network mask, default gateway). About this task You must deploy the SGI on the same network as, or a network that is accessible to, the grid nodes being deployed for the StorageGRID Webscale system. An additional, unique IP address is required for the SGI, one that is separate from the IP addresses assigned to grid nodes in the grid specification file. The best practice is to remove the SGI virtual machine after you have deployed all grid nodes and verified that they have started successfully and joined the grid. This helps to ensure that future changes to the grid topology are not competed with the incorrect version of the SGI. Each time you make a change to the grid topology, you must ensure that the SGI version matches the version of your StorageGRID Webscale system. 1. Log in to the OpenStack Dashboard. 2. Select Project > Orchestration > Stacks. 3. Click Launch Stack.

17 Expanding your StorageGRID Webscale system In the Select Template dialog box enter the configuration information for the StorageGRID Webscale Installer stack: Template Source: Select URL from the drop-down list. Template URL: Enter, or copy and paste, the location of the SGI heat template, for example: Leave Environment Source and Environment File at their default values. 5. Click Next. 6. In the Launch Stack dialog box enter the SGI deployment information: Stack Name: Enter a meaningful name for the stack, for example, NetApp-SGI. Creation Timeout (minutes): The time to allow for the stack creation before a timeout in minutes. Rollback On Failure: Select this option to enable rollback to the starting state upon failure. This option should be selected with caution, because it will prevent error messages from being displayed in OpenStack Dashboard if the deployment fails. Password for user "username": Enter the password for the specified OpenStack project user account. StorageGRID Network: Select the network the grid nodes are being deployed in from the drop-down list. StorageGRID Network Netmask: Enter the network mask for the SGI on the StorageGRID Network. StorageGRID Network Gateway: Enter the default gateway for the SGI on the StorageGRID Network. StorageGRID Installer IP Address: Enter the IP address for the SGI on the StorageGRID Network. Public Network: Select the public network the SGI will use from the drop-down list. This is the network that Floating IP addresses are allocated from. This field is only applicable, and is only displayed, if you are using the SGI template for private tenant networks (SGI_Private_Routing_Template.yaml). StorageGRID Installer Image URL: Enter, or copy and paste, the URL for the SGI virtual machine disk (.vmdk) file, for example: SGI a323.vmdk 7. Click Launch. You must wait for the deployment to complete, and then you can access the SGI in your web browser at the IP address you specified in the StorageGRID Installer IP Address text box. Deploying grid nodes You must deploy each expansion grid node defined in your grid specification file. You deploy Admin Nodes, API Gateway Nodes, and Archive Nodes on virtual machines. You deploy Storage Nodes either on virtual machines or on physical servers if you are installing the StorageGRID Webscale appliance.

18 18 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments Choices Uploading the SAID package and preparing the deployment on page 18 During the expansion of your StorageGRID Webscale system, you must upload the SAID package, which defines the changes to your grid topology. Deploying grid nodes in OpenStack Dashboard on page 19 You need to deploy virtual machine based grid nodes in OpenStack Dashboard. Deploying StorageGRID Webscale appliance Storage Nodes on page 20 When you install a StorageGRID Webscale appliance, you pause from using the StorageGRID Webscale Installer and access the StorageGRID Webscale Appliance Installer web page to complete the appliance installation. Uploading the SAID package and preparing the deployment During the expansion of your StorageGRID Webscale system, you must upload the SAID package, which defines the changes to your grid topology. 1. In a supported web browser, navigate the StorageGRID Webscale Installer using the IP address configured when deploying the StorageGRID Webscale Installer. 2. On the Welcome page, select Modify an existing StorageGRID Webscale System. 3. In the Upload the SAID package page, click Upload, locate and select the correct SAID package file (.zip) for the expansion and click Open. The SAID file is named with GIDgridIDNumber_REVrevisionNumber_SAID.zip 4. Click Next. 5. In the Grid Configuration page, enter the required information for Storage Nodes: Number of RangeDBs: Specify the number of storage volumes (RangeDBs) to attach to each Storage Node. Size of RangeDBs (GBs): Specify the size of each individual storage volume (RangeDB) in gigabytes. You must enter a value between 50GB and 20000GB (20 terabytes). The minimum value for production systems is 4000 GB (4 terabytes). These values are defined in the grid specification file, and should not be modified unless you verify that the specified values are incorrect. If you are only deploying StorageGRID Webscale appliance Storage Nodes, these values are not used because the full storage capacity on the appliance is always used. 6. Click Save. 7. Click Next. 8. In the Deploy Grid Nodes page, select the expansion grid nodes to deploy: If your expansion does not include StorageGRID Webscale appliance (SGA) Storage Nodes, select the Deploy only expanded nodes checkbox, click Generate Configuration, and copy the generated URL from the Grid Node URL text box. If your expansion includes SGA Storage Nodes and other types of virtual machine based grid nodes, you need to select the grid nodes you are deploying on virtual machines, deselect the SGA Storage Nodes you are expanding, and deselect all nodes that are not part of the expansion. When you have verified that only the virtual machine grid nodes that are being expanded are selected, select the Deploy only expanded nodes checkbox, click Generate Configuration, and copy the generated URL from the Grid Node URL text box.

19 Expanding your StorageGRID Webscale system 19 If your expansion only includes SGA Storage Nodes, deselect all of the grid nodes n the grid. After you finish You need to use the Grid Node URL value to deploy the virtual machine based expansion grid nodes in OpenStack Dashboard. SGA Storage Nodes are deployed using the StorageGRID Webscale Appliance Installer. Deploying grid nodes in OpenStack Dashboard You need to deploy virtual machine based grid nodes in OpenStack Dashboard. 1. In a supported web browser, navigate to the OpenStack Dashboard and sign in. 2. Click Project > Orchestration > Stacks. 3. Click Launch Stack. 4. In the Select Template dialog box enter the grid deployment file information: Template Source: Select URL from the drop-down list. Template URL: Paste in, or enter, the Grid Nodes URL value from the Deploy the Grid page in the SGI. Leave Environment Source and Environment File at their default values. 5. Click Next. 6. In the Launch Stack dialog box enter the grid deployment information: Stack Name: Enter a meaningful name for the stack, for example, DC1-S6. Creation Timeout (minutes): The time to allow for the stack creation before a timeout in minutes. Rollback On Failure: Select this option to enable rollback to the starting state upon failure. This option should be selected with caution, because it will prevent error messages from being displayed in OpenStack Dashboard if the deployment fails. Password for user "username": Enter the password for the specified OpenStack project user account. StorageGRID node security group: The Neutron security group to use for the grid node. The default value, StorageGRID Node Firewall should be used in most cases. You can ensure that the correct value is listed by verifying that the release number is listed followed by the deployment date and a unique ID, for example, unique_id. Maintenance Mode: Select True from the drop-down list. StorageGRID node server flavor: The type of server to use for the grid node. You need to select the node server flavour that corresponds to the stack for your StorageGRID Webscale grid deployment. For example, if you named the StorageGRID Webscale stack NetApp_SGW, look for an entry named NetApp_SGW_node_flavor-uniqueID StorageGRID node root image: Select "StorageGRID root image" as the root disk for the grid node. You can ensure that you are selecting the correct entry by verifying that the release number is listed followed by the deployment date and a unique ID, for example, unique_id.

20 20 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments 7. Click Launch. 8. Return to the SGI and monitor the progress of the grid node installation. Wait until the status bar for each expansion grid node you are adding is yellow and the status is Stopped in Maintenance Mode before continuing with the expansion procedure. If you are installing StorageGRID Webscale appliance Storage Nodes, complete the software installation of the Storage Nodes before running the Grid Expansion: Initial grid task. Otherwise, run the Grid Expansion: Initial grid task next. Deploying StorageGRID Webscale appliance Storage Nodes When you install a StorageGRID Webscale appliance, you pause from using the StorageGRID Webscale Installer and access the StorageGRID Webscale Appliance Installer web page to complete the appliance installation. Connecting to the appliance configuration web page To start the appliance software installation, you connect to the appliance configuration web page. Using this page enables you to configure the management network, configure the data network connection, enter the StorageGRID Webscale Installer IP address, and monitor the installation progress. Step 1. Open a browser and enter the E5600SG controller Management Port 1 IP address that was provisioned during the physical installation: The StorageGRID Webscale Appliance Installer web page appears: When you are first installing the appliance, the status for each of the procedures on the web page indicates that the procedure is not complete. Configuring the data network connections Using the StorageGRID Webscale Appliance Installer web page, you enter the IP address of the data network. Additionally, you enter the subnet mask for the network and at least a default gateway. Entering the IP address, subnet mask, and gateway enables you to configure the data network connection. Before you begin You must already have the IP address of the data network. 1. On the StorageGRID Webscale Appliance Installer web page, click Configure StorageGRID data network connection:

21 Expanding your StorageGRID Webscale system To edit the data network IP address, in the StorageGRID data network connection section, click Update IP/netmask. The button name changes to Save Changes and a pop-up appears. 3. Enter the IP address of the data network and click Save Changes. Route information based on the specified IP displays. 4. In the pop-up, click OK. 5. If needed, edit the route and click Save route. Setting the StorageGRID Webscale Installer IP address You can use the StorageGRID Webscale Appliance Installer web page to set the IP address of the StorageGRID Webscale software installer. Setting this enables installer connectivity. Before you begin You must know the IP address of the StorageGRID Webscale Installer. 1. Click Home to navigate to the main StorageGRID Webscale Appliance Installer web page. 2. In the Set StorageGRID Webscale Installer IP text box, enter the IP address of the StorageGRID Webscale Installer and click Update.

22 22 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments 3. Click OK in the confirmation dialog box. Installing StorageGRID Webscale software on the appliance You install the StorageGRID Webscale software and the operating system by using the appliance installation web page. You can also use the web page to monitor the installation. Installing this software enables you to monitor the appliance information in the StorageGRID Webscale system. Before you begin You must have already configured the management and data networks and entered the StorageGRID Webscale Installer IP address. You must have access to the Passwords.txt file. You must have an SSH client, such as PuTTY, to use to connect the StorageGRID Webscale appliance. About this task Attention: You must monitor the progress of the installation and put the grid node in maintenance mode at the appropriate time. After the operating system is installed, you are prompted to run a script to put the grid node in maintenance mode. You must run this script within 5 minutes. When you install the software, the web interface initiates the following operations: Establishes a connection to the storage array. Checks the operational status of all drives. Creates a primary disk pool. Calculates volume sizes. Creates volumes. Creates the LUN mappings. Renames the array. Creates a configuration file. Rescans SCSI ports and reloads devices. 1. From the StorageGRID Webscale Appliance Installer web page, next to the Set StorageGRID Webscale Installer IP option, click Begin StorageGRID node_name node install :

23 Expanding your StorageGRID Webscale system 23 A list of install operations appears. You can review the installation progress as each operation status changes from Not started to Completed. The status refreshes every five seconds. 2. To monitor progress, return to the StorageGRID Webscale installation web page. The Deploy Grid Nodes section shows the installation progress for the appliance Storage Node. 3. Review the appliance web page. The following occurs: When the operating system installation is in progress, a thumbnail image of the installation appears next to the list of operations. The web page displays the last 10 lines of the installation log, which updates every five seconds. During the installation of StorageGRID Webscale software, you are prompted to start the grid node in maintenance mode. Since this is a maintenance procedure, follow the instructions on screen to put the grid node into maintenance mode.

24 24 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments The StorageGRID Webscale Installer Deploy Grid Nodes web page status bar changes to blue, indicating a job in progress, and then to yellow, indicating that the grid node is in maintenance mode. 4. Open the StorageGRID Webscale Installer in your web browser and monitor the progress of the grid node installation. Wait until the status bar for each expansion grid node you are adding is yellow and the status is "Stopped in Maintenance Mode" before continuing with the expansion procedure. If you are installing additional StorageGRID Webscale appliance Storage Nodes, complete the software installation of the Storage Nodes before running the "Grid Expansion: Initial" grid task. Otherwise, run the "Grid Expansion: Initial" grid task next. Running the Grid Expansion: Initial grid task Running the Grid Expansion: Initial grid task adds configuration information to the StorageGRID Webscale system for all expansion grid nodes at all data center sites. The initial configuration grid task is run once per expansion. About this task Only the following grid tasks can be run concurrently: Clean Up Unused Cassandra Keys (CCLE) ILM Evaluation (Volume Lost) (ILME) ILM Re-evaluation (User Triggered) (ILME) LDR Foreground Verification (VFGV), except if adding a Storage Node If any other grid tasks are running, wait for them to complete or release their lock, or abort them as appropriate. A previously started CCLE grid task can be aborted if your expansion includes a Storage Node. The expansion generates a new CCLE grid task that you need to run during the expansion process. For more information about grid tasks and resource locking, see the Administrator Guide. 1. In the NMS MI, go to Grid Topology > primary Admin Node > CMN > Grid Tasks > Configuration > Main. 2. In the Pending table, locate the Grid Expansion: Initial grid task, and then select Start from the Actions drop-down list. If the Pending table includes grid tasks from multiple provisioning revisions, always run grid tasks from the earliest revision (lowest revision number) first.

25 Expanding your StorageGRID Webscale system Click Apply Changes. The grid task moves from the Pending table to the Active table and continues to execute until it completes or fails. When it completes or fails, it moves to the Historical table with a Status description. If the grid task fails, contact technical support. You must wait for the NMS MI page to automatically refresh before the change is visible. Do not submit the change again. You can resume Grid tasks that are Paused. If you Abort a grid task you cannot restart it while it is in the Historical table. After you finish You must wait until the grid task completes successfully before continuing to the next step in the expansion procedure. Related information StorageGRID Webscale 10.2 Administrator Guide

26 26 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments Initializing storage volumes for Storage Nodes You need to initialize the storage volumes (rangedbs) on each Storage Node to make the storage available to the StorageGRID Webscale system. Before you begin You must have the access to the Passwords.txt file. 1. From the service laptop, log in to the new Storage Node as root using the password listed in the Passwords.txt file. 2. Initialize the storage volume (rangedb) hard disks: ruby /tmp/ldrinit.rb a. When unallocated drives are detected, you are asked to use the volume as an LDR rangedb and Accept proposal [y/n], enter: y. b. For each rangedb drive on the Storage Node, when you are asked to Reformat the rangedb drive <name>? [y/n]?, enter y. In the following example, the rangedbs associated with the DC1-S4 Storage Node are initialized: DC1-S4:~ $ ruby /tmp/ldrinit.rb Unallocated drives detected. Is the following proposed action correct? Use /dev/sdb (300.0GiB) as an LDR rangedb Use /dev/sdc (300.0GiB) as an LDR rangedb Use /dev/sdd (300.0GiB) as an LDR rangedb Accept proposal [Y/n]? y Determining drive partitioning WARNING: drive /dev/sdb exists Reformat the rangedb drive /dev/sdb? [Y/n]? y WARNING: drive /dev/sdc exists Reformat the rangedb drive /dev/sdc? [Y/n]? y WARNING: drive /dev/sdd exists Reformat the rangedb drive /dev/sdd? [Y/n]? y Formatting the following: /dev/sdb /dev/sdc /dev/sdd Formatting Storage Drives Finalizing device tuning Disabling rpcbind service Creating Object Stores for LDR Generating Grid Interface Configuration file LDR initialization complete DC1-S4:~ $ Setting the installation state You need to activate the installation state for each expanded grid node. Before you begin You must have the access to the Passwords.txt file.

27 Expanding your StorageGRID Webscale system From the service laptop, log in to the expanded grid node as root using the password listed in the Passwords.txt file. 2. Set the installation state: echo ACTIVATE > /var/local/run/install.state Activating the nonprimary Admin Node services If your expansion includes adding a nonprimary Admin Node to an existing data center site, you need to run a script to activate services. 1. Log in to the primary Admin Node as root using the password listed in the passwords.txt file. 2. Activate the primary Admin Node services: activate-admin-services.sh Starting the StorageGRID Webscale services You need to start StorageGRID Webscale software on all newly added grid nodes by enabling services for each expansion grid node. Before you begin You must have the access to the Passwords.txt file. About this task When adding more than one type of grid node in a single expansion, you must enables services in the following order: 1. Admin Nodes 2. Storage Nodes and Archive Nodes If you are adding multiple Storage Nodes to an existing site, only enable services for a maximum of four Storage Nodes at the same time. Wait at least five minutes after starting to enable services on the first set of grid nodes the before enabling services for the next Storage Node. The time it takes for enabling services to complete depends on the amount of data that needs to be copied to each Storage Node. If billions of objects are being copied, it will take several hours for the enable services process to complete. You do not need to enable services for StorageGRID Webscale appliance Storage Nodes. 3. API Gateway Nodes Enabling services for grid nodes in this order prevents queue buildups, particularly in larger expansions where the length of time to complete the process can be longer. 1. From the service laptop, log in to the primary Admin Node as root using the password listed in the Passwords.txt file. 2. Add your private key identity to the SSH authentication agent: ssh-add

28 28 StorageGRID Webscale 10.2 Expansion Guide for OpenStack Deployments 3. Enter the SSH Access Password for the primary Admin Node listed in the Passwords.txt file. 4. Start the Grid Deployment Utility (GDU): gdu-console 5. Type the provisioning passphrase, press Tab to select OK, and then press Enter. 6. Complete the following steps for each new expansion grid node: a. Ensure that the appropriate grid node is selected and that the current state is Available. b. Press Tab to move through the options to the Tasks list. c. Press the down arrow key to highlight Enable Services, and then press the Spacebar to select it. d. Press Tab to move through the options, and then select the Start Task action. e. Press Enter to run the grid task. The status of the grid node returns to Available when the grid task completes. If you are adding multiple Storage Nodes to an existing site, wait at least five minutes before enabling services for the next Storage Node. 7. When the grid task completes for the last expansion grid node you are enabling services for, press the right arrow key to move to the Quit action, and then press Enter. As each grid node joins the StorageGRID Webscale, you can monitor it through the NMS MI. Alarms are normal until the services establish connectivity within the system. The alarms are cleared automatically as connections are established. Note: Archive Nodes show a major alarm when they are first started. This is normal. You must configure the Archive Node's connection to the targeted external archival storage system to clear these alarms. 8. End the SSH session: ssh-add -D Running the Grid Expansion: Add Server grid task You must run the Grid Expansion: Add Server grid task for each grid node added in the expansion. This prepares the StorageGRID Webscale system for the new grid node. Before you begin You must have run the Grid Expansion: Initial grid task. You have determined the correct order for running the grid tasks if you are adding more than one type of grid node. About this task Only the following grid task can be run concurrently: Clean Up Unused Cassandra Keys (CCLE) LDR Foreground Verification (VFGV), except if adding a Storage Node If any other grid tasks are running, you must wait for them to complete, release their lock, or abort them as appropriate. A previously started CCLE grid task can be aborted if your expansion includes a Storage Node. The expansion generates a new CCLE grid task that you need to run during the