MAX Data 1.1 Installation and Setup Guide November 2018 215-13646_A0 doccomments@netapp.com
2 Contents Deciding whether to use this guide... 4 Understanding the MAX Data components... 5 Setup overview... 6 Verifying your configuration is supported... 7 Setting up the NetApp AFF storage system... 7 Setting up the application servers... 9 Installing or upgrading the server operating system...10 Verifying the NVDIMMs...10 Memory sizing...10 Installing the RDMA network...10 Installing the SAN network...10 Installing the Host Utilities...10 Installing the MAX Data software package...11 Setting up the MAX Recovery Server...12 Installing or upgrading the server operating system...12 Verifying the NVDIMMs...13 Sizing the MAX Recovery non-volatile memory...13 Installing the RDMA network...13 Installing the MAX Data software package...13 Connecting the MAX Data servers to the MAX Recovery server...13 Configuring the MAX Data environment...14 Planning backup of MAX Data server data on AFF...14 Scheduling Snapshot copies...15 Running the configuration wizard...15 Recovering data from a Snapshot copy...16 Recovering data from the MAX Recovery Server...17 Provision the replacement application server...17 Add the new server using the MAX Data GUI...17 Monitoring the MAX Data environment...18 Removing MAX Data...20 Removing applications from the MAX Data servers...21 Removing MAX Data application servers from the cluster...21 Removing mirrored data from the MAX Recovery server...21
3 Removing the MAX Recovery server from the cluster...21 Uninstalling the MAX Data and MAX Recovery software...21 Remove the SVM from the AFF storage system...22
4 Deciding whether to use this guide This guide describes how to install and configure NetApp Memory Accelerated Data (MAX Data). MAX Data software provides ultra-low latency storage with automatic tiering and enterprise-level storage management. You should use this guide if you want to complete the following tasks: Install the MAX Data components on one or more application servers Install the MAX Data components on a recovery server Cable all components together Configure the connection to a NetApp AFF storage system Verify the system is set up correctly Monitor the system Configure backups Recover from a storage failure If you want a high-level installation overview, see the MAX Data 1.1 Quick Install Guide. Provisioning storage for an application, configuring applications, and ongoing optimization of the system is covered in the application-specific solution content available from the Resource Page. To learn more about MAX Data, see the MAX Data Resource Page.
5 Understanding the MAX Data components A MAX Data system includes the application servers with a supported operating system, nonvolatile memory (NVDIMMs), high-speed networks, an optional MAX Recovery server, a NetApp AFF storage system with a supported ONTAP software version, and a supported application. Instead of using NVDIMMs, you can use regular DRAM memory to test MAX Data. You can also use DRAM in a production environment if you also use the optional MAX Recovery server to mirror your data, or if you are using an application that manages its own high availability and can handle the loss of the in-memory data.
6 Setup overview There are five main steps to setting up the MAX Data environment. The following figure shows the steps and the order in which you complete them.
7 Verifying your configuration is supported Use the NetApp Interoperability Matrix Tool (IMT) to verify that you have a supported combination of all of the following components: Server model for MAX Data and MAX Recovery servers Server operating system version and any required packages and drivers RDMA adapters for connecting MAX Data to MAX Recovery servers Switch for connecting MAX Data to MAX Recovery servers MAX Data software version NetApp AFF storage system model ONTAP software version For Fibre Channel connections from the MAX Data server to the AFF system: o FC initiator cards and firmware o FC switch models and firmware o Host Utilities For iscsi connections from the MAX Data server to the AFF system: o iscsi initiator software version or iscsi hardware initiator card model and firmware o Host Utilities Setting up the NetApp AFF storage system The NetApp AFF storage system provides high-speed storage to offload less-used data from the MAX Data server. You must connect the systems using FC or iscsi and configure the ONTAP software. The data connection from the AFF storage cluster to the MAX Data servers is a standard SAN configuration. The ONTAP SAN documentation describes supported configurations. Complete the following steps on the AFF storage system using OnCommand System Manager or the ONTAP CLI: 1. Ensure there is an aggregate with adequate space. The MAX Data configuration wizard will create one or more LUNs that total 25 times the size of the persistent memory (or DRAM allocated to act in place of persistent memory) for each server. This ratio cannot be changed. 2. Create a Storage Virtual Machine (SVM) a. Record the SVM management IP address, administrator name, and password for use by the MAX Data server. b. Specify the login security method as cert to use SSL certification authentication. c. Configure DNS for the SVM. d. Verify you have configured NTP for the AFF nodes. The time on the AFF node must match the time on the MAX Data servers to within one minute. e. Enable the FC or iscsi protocol on the SVM. f. Create at least two data LIFs on each node in the AFF cluster for use by the MAX Data server.
8 3. To protect the data stored on the AFF nodes, configure SnapMirror SVM replication with the identity discard option. Detailed steps are available from the ONTAP Documentation Center or the System Manager online help.
9 Setting up the application servers Setting up the application server includes installing the hardware and software components. The following diagram provides an overview of the setup process.
10 Installing or upgrading the server operating system If you are not running a supported operating system version, install or upgrade as needed. Only the versions listed in the IMT are supported. Configure DNS on the server. Configure NTP on the server. The time on the MAX Data servers must match the time on the AFF nodes to within one minute. Verifying the NVDIMMs Verify that supported non-volatile memory DIMMs are installed in the server following the NVDIMM and server manufacturers standards. If you are using DRAM instead of NVDIMMs, the MAX Data configuration wizard will partition the memory needed for MAX Data. Memory sizing The MAX Data configuration wizard will create one or more LUNs that total 25 times the size of the persistent memory (or DRAM allocated to act in place of persistent memory) for each server. This ratio cannot be changed. The goal is to have sufficient non-volatile memory to hold the working data for the application. At a minimum, you must have enough memory so that the total expected storage for the application does not exceed 25 times the memory. If you are using DRAM, you must have enough DRAM to partition in place of non-volatile memory plus the DRAM needed for the operating system and application needs. Installing the RDMA network Mirroring data from a MAX Data server to the MAX Recovery server requires a dedicated RDMA network. Install the supported cards and switch following the instructions provided by the manufacturers and cable the components. The MAX Data Release Notes specify the number of MAX Data servers supported by one MAX Recovery server. For security, use an isolated network or VLAN to restrict access to only the MAX Data and MAX Recovery servers. Installing the SAN network MAX Data servers connect to the AFF storage server using either a Fibre Channel or an iscsi network. You should use two FC or Ethernet switches for high availability. Follow the instructions in the ONTAP SAN Configuration Guide to set up and cable the SAN components. Installing the Host Utilities The host utilities include recommended settings and software to assist configuring the SAN connection to the AFF storage system. Be sure to download the supported version from the NetApp Support Site. Follow the installation instructions provided with the package. The MAX Data configuration wizard sets host OS values. To see the values set by the wizard, use the multipath -t command on the host.
11 Installing the MAX Data software package The MAX Data software includes an installation script that installs the MAX FS file system and the Cockpit management UI, installs all required support packages, opens the necessary firewall ports, and configures NetApp AutoSupport reporting. Complete the following steps on the application server: 1. Obtain the appropriate MAX Data license for the number of servers you are deploying. 2. Download the software package from the NetApp Support Site. 3. Copy the installation package to each server and extract the files to a working directory. 4. Log into the server with root privileges. 5. In the working directory, run the following command to install the software and open the required firewall ports:./max_install.sh --enable-ports 6. Accept the license agreement. 7. Repeat for each MAX Data server in your configuration.
12 Setting up the MAX Recovery Server The MAX Recovery server mirrors the data from the MAX Data servers. If a MAX Data server ever fails, you can quickly recover the data from the MAX Recovery server. Setting up the recovery server includes installing the hardware and software components. The MAX Recovery server uses the same software installation package as the MAX Data server. The following diagram provides an overview of the setup process. Installing or upgrading the server operating system If you are not running a supported operating system version, install or upgrade as needed. Only the versions listed in the IMT are supported.
13 Verifying the NVDIMMs Verify that supported non-volatile memory DIMMs are installed in the server following the NVDIMM and server manufacturers standards. Sizing the MAX Recovery non-volatile memory The MAX Recovery server non-volatile memory requirement is based on the amount of nonvolatile memory in the servers it is backing up. 1. Calculate the total amount of non-volatile memory in all of the MAX Data servers that will connect to the recovery server. 2. Ensure that the recovery server has at least this much non-volatile memory installed. Installing the RDMA network Mirroring data from a MAX Data server to the MAX Recovery server requires a dedicated RDMA network. Install the supported cards and switch following the instructions provided by the manufacturers and cable the components. The MAX Data Release Notes specify the number of MAX Data servers supported by one MAX Recovery server. Ensure that jumbo frames are enabled on the network. For security, use an isolated network or VLAN to restrict access to only the MAX Data and MAX Recovery servers. Installing the MAX Data software package The MAX Recovery server uses the same software installation package as the MAX Data server. The MAX Data software includes an installation script that installs the MAX FS file system and the Cockpit management UI, installs all required support packages, opens the necessary firewall ports, and configures NetApp AutoSupport reporting. Complete the following steps on the application server: 1. Verify that your MAX Data license covers the MAX Recovery server you are deploying. 2. Download the MAX Data software package from the NetApp Support Site. 3. Copy the installation package to the server and extract the files to a working directory. 4. Log into the server with root privileges. 5. In the working directory, run the following command to install the software and open firewall ports:./max_install.sh --enable-ports 6. Accept the license agreement. 7. When the installation is complete, copy the license file you received from your sales representative into the /opt/netapp/max/license directory to activate the product. Connecting the MAX Data servers to the MAX Recovery server The network that connects the data servers to the recovery server must be a dedicated, highspeed network with RDMA capability. The following diagram shows how the components are connected.
14 1. Install a supported NIC in each data server 2. Install a supported NIC in the recovery server 3. Install a supported switch 4. Cable each NIC to the switch as shown in the diagram 5. Log into the switch and verify that the supported firmware version is installed See the NIC and switch documentation for instructions on updating the firmware if needed. 6. Verify RDMA networking is correctly installed and is using jumbo frames. a. For Mellanox RNICs use the ibv_devinfo command to check the MTU setting b. Use the ibv_rc_pingpong command to verify end-to-end support Example using Mellanox ConnectX4 RNICs: MAX Data server: ibv_rc_pingpong -g 0 -d NIC_id -m 4096 -i 1 IP_address MAX Recovery server: ibv_rc_pingpong -d NIC_id -g 0 -i 1 -m 4096 Configuring the MAX Data environment You use the MAX Data GUI configuration wizard to configure the entire MAX Data environment, including the AFF storage system, application servers running MAX Data software, and optional recovery server running MAX Data software. Planning backup of MAX Data server data on AFF The best practice is to mirror the MAX Data file system to a MAX Recovery server and store Snapshot copies of the MAX Data file system on the AFF storage system. Restoring from the
15 MAX Recovery server enables having no data loss (recovery point objective zero), while the Snapshot copies enable you to restore previous versions of the file system data. Scheduling Snapshot copies You should schedule Snapshot copies of the application files when you configure the application in the MAX Data configuration wizard. You can schedule Snapshot copies in the MAX Data configuration wizard when you enter the application data. The MAX Data software coordinates the Snapshot copies. First a local copy is made, and then the local Snapshot copy is transferred to the AFF storage system as an ONTAP Snapshot copy. You can choose the copy frequency, and you can make local copies more frequently than the remote copies. Running the configuration wizard If you have more than one application server running the MAX Data software and using the same SVM credentials, you can run the configuration wizard from any of those servers. NOTE: When regular DRAM is used in place of non-volatile memory DIMMs, the configuration wizard reboots the operating system during configuration. Be sure rebooting will not disrupt any other workloads on the servers. Reconnect to the server after the reboot completes. To run the wizard, complete the following steps: 1. Open a web browser on a server where you installed MAX Data and point it to: https://localhost:9090 2. Log in with root credentials. 3. Following the configuration wizard prompts, enter the requested information including ONTAP credentials, network addresses, desired memory configuration, and details about your application. Details on configuring specific applications is included in the MAX Data Technical Reports or Solution Briefs for supported applications. See the MAX Data Resource Page for a list of available documents. 4. If desired, specify the optional MAX Recovery server to configure.
16 Recovering data from a Snapshot copy If the application server running MAX Data is still functional, you can recover lost files from a Snapshot copy. 1. Point a web browser to a MAX Data server: https://hostname IP_address:9090 2. Click the Snapshots tab. 3. Select the desired copy and click action button on the far right. 4. Click Restore. 5. The file system is restored to the state at the time the selected Snapshot copy was made.
17 Recovering data from the MAX Recovery Server If you are using the optional MAX Recovery server to mirror your MAX Data file system, you can quickly recover data with a recovery point objective (RPO) of zero data loss. The following diagram shows the overall recovery process: Provision the replacement application server Create a replacement application server and install MAX Data following the instructions in this guide. The replacement server should be set up just the like server it replaces. Add the new server using the MAX Data GUI Open the MAX Data GUI on a surviving application server in the cluster. If there is no application server available, you can open the GUI on the MAX Recovery server. 1. Point a web browser to a MAX Data server: https://hostname IP_address:9090 2. Log in with root credentials. 3. On the Cluster tab, click Configure. 4. Click Skip to continue to the Add hosts page. 5. Enter the information for the new server being added.
18 6. On the Provision Application page, select the recovery source and follow the instructions on the screen to recover data to the replacement server. Monitoring the MAX Data environment You can use the MAX Data GUI to monitor storage and performance metrics. You can also view logs. 7. Point a web browser to a MAX Data server: https://hostname IP_address:9090 8. Log in with root credentials. 9. Click the localhost tab on the top left and then click one of the following tabs: Storage Monitoring System Monitoring Utilization System Logs The following is an example of the Storage Monitoring tab. Your system might look slightly different.
19
20 Removing MAX Data You can remove some or all of the MAX Data components from your environment. The following diagram shows the process for removing MAX Data.
21 Removing applications from the MAX Data servers Before you can remove the server from the MAX Data cluster, you must remove the applications using MAX Data. 1. Stop all I/O with the MAX FS file system. 2. Point a web browser to a MAX Data server: https://hostname IP_address:9090 3. Log in with root credentials. 3. Click the Cluster tab. 4. For each server listed, click the menu icon at the end of the row and select Remove Applications from Host. Removing MAX Data application servers from the cluster Before you can remove the MAX Data software, you must remove the server from the cluster. 1. Point a web browser to a MAX Data server: https://hostname IP_address:9090 2. Log in with root credentials. 3. Click the Cluster tab. 4. For each server listed, click the menu icon at the end of the row and select Remove Host from Cluster. Removing mirrored data from the MAX Recovery server Before you can remove a MAX Recovery server from the cluster, you must remove the mirrored data. Remove the mirrored data after removing the MAX Data application servers. 1. Point a web browser to a MAX Data server: https://hostname IP_address:9090 2. Log in with root credentials. 3. Click the MAX Recovery Servers tab. 4. Click the menu icon at the end of the data row and select Clean Mirrored Data. Removing the MAX Recovery server from the cluster Before you can remove the MAX Recovery software, you must remove the server from the cluster. 1. Point a web browser to a MAX Data server: https://hostname IP_address:9090 2. Log in with root credentials. 3. Click the MAX Recovery Servers tab. 4. Click the menu icon at the end of the data row and select Remove Host from Cluster. Uninstalling the MAX Data and MAX Recovery software After the servers have been removed from the cluster, you can remove the MAX Data and MAX Recovery software. 1. Log into the server with root privileges.
22 2. In the working directory where you copied the software package, run the following command to uninstall the software:./max_uninstall.sh 3. Repeat for each MAX Data and MAX Recovery server in your configuration. Remove the SVM from the AFF storage system Removing the MAX Data components removes the data stored on the AFF storage system, but you still need to remove the SVM. Complete the following step on the AFF storage system using OnCommand System Manager or the ONTAP CLI: Delete the SVM.