NEC Storage M series for SAP HANA Tailored Datacenter Integration Configuration and Best Practice Guide

Transcription

1 NEC Storage M series for SAP HANA Tailored Datacenter Integration Configuration and Best Practice Guide (M120/M320/M320F/M110/M310/M310F/M510/M710/M710F) August, 2018 NEC

2 Copyright 2018 NEC Corporation. All Rights Reserved. The information in this document is for informational purposes only and may contain typographical errors and technical inaccuracies. This document is provided as is and NEC Corporation makes no representations or warranties of any kind. Product, brands, and trade names used in this document are trademark or registered trademarks of their respective holders. 2 NEC Corporation 2018

3 Table of contents Introduction... 4 Overview of NEC Storage M Series... 5 Selection of NEC Storage M series configuration... 6 Selection of NEC Storage M series model... 6 Connection to the host server... 6 Disk drives... 7 RAID level... 7 Cache memory... 7 Estimation of required disk size... 7 SAP HANA data... 7 SAP HANA log... 8 OS, /usr/sap, swap... 8 SAP HANA shared... 8 Backup area... 8 Disk drive configuration... 8 Installation and Configuration Access Control Virtual Cache Partitioning Path redundancy Create udev rules File system and mount option SAP HANA Storage Connector API HANA configuration Performance check References NEC Corporation 2018

4 Introduction This document is the configuration and best practice guide to use NEC Storage M series with SAP HANA Tailored Datacenter Integration (TDI). SAP HANA was initially provided as an appliance server and storage for SAP HANA DB was included in an appliance configuration. With TDI SAP offers an additional approach to use SAP certified Enterprise Storage as SAP HANA storage, providing a more flexible system. This guide is applied for productive SAP HANA systems. For non-productive SAP HANA systems, you are recommended to follow this document. But also you can choose other storage and disk drive configuration, if performance for storage system is not required in such non-productive SAP HANA systems. 4 NEC Corporation 2018

5 Overview of NEC Storage M Series NEC Storage M series is a series of optimum disk array systems for open systems and suitable for use in SAN environments. NEC Storage M series is a lineup of high performance and high reliability storage products that implement the latest virtualization technologies and focus on reducing the burden and the cost of managing storages. NEC Storage M series enable quick response to diverse needs of corporate and social information systems, help drive the creation of new value and solutions, and contribute to the development of next generation ICT businesses. The next figures show line up of SAP HANA Enterprise Storage certified storages. You can use these storages as SAP HANA s storage by following this guide. M*20 Series M*10 Series 5 NEC Corporation 2018

6 Selection of NEC Storage M series configuration To select and configure NEC Storage M series for SAP HANA TDI you need to follow the guide described here. Selection of NEC Storage M series model You can use only SAP certified enterprise storage. You can find certified storages at The SAP HANA Hardware Directory At the time of this document is written, M110, M310/M310F, M510, M710/M710F, M120 and M320/M320F are certified. M110 and M120 have single controller and dual controller model, and only dual controller model can be used for this configuration. You need to decide which model of storage you use from following points. Number of SAP HANA and non-hana systems connected to. There are upper limits of supported SAP HANA nodes for each model. These limits are decided to keep good performance for SAP HANA. If non-hana DB systems share the storage, those systems are also counted. You must take care that the workload of non-hana system do not affect the workload of SAP HANA, for example, by separating disk pool and FC ports for SAP HANA and non-hana systems. Number of disk drives Disk space required for one HANA system is determined from the memory size and data size of HANA system as the result of sizing. Number of disk drives are decided from - Disk space requirements - RAID level - Minimum number of disk drives to satisfy disk throughput and latency KPI of TDI system defined by SAP. The total number of disk drives for each HANA system must not be over the maximum drives of each storage model. The table below shows the supported maximum number of HANA nodes for each Storage model. Max. Max. Max Max. Model Drives FC port Cache(GB) HANA node M GB 3 M310/M310F GB 8 M GB 16 M710/M710F GB 32 M GB 10 M320/M320F GB 30 The maximum HANA node in above table assumes maximum cache memory is installed. If a non-hana DB system is connected, it must be counted as one HANA node. Connection to the host server NEC Storage M Series has Fibre channel (FC), iscsi and SAS interfaces to connect to the host server, but in TDI environment, you can use only FC interface. iscsi and SAS are not allowed. FC 8Gbps interface is not allowed, you must use FC 16Gbps or FC 32Gbps interface if it is supported. 6 NEC Corporation 2018

7 Server and storage must be connected with 2 or more FC ports for redundancy. It is recommended to connect storage and server directly to separate work load of other HANA system and non-hana system. You can use FC switch, but in that case, one 16Gbps FC port can be shared with up to 3 or 4 HANA systems, because the KPI for disk performance requires throughput of 400MB/s. Disk drives The types of disk drives you can use for HANA data and log volume are SAS HDD/10000rpm SAS HDD/15000rpm SAS SSD You can select any size of disk drive from supported drives on each storage model. RAID level You must configure disk drives with redundancy. RAID10, RAID5/50 or RAID6/60 is recommended. Cache memory It is recommended to use maximum available cache for each model. Estimation of required disk size In SAP HANA systems, disk spaces are required for: OS, swap, /usr/sap SAP HANA shared SAP HANA data SAP HANA log backup Among these, performance of SAP HANA data and SAP HANA log areas are very important and you must follow the guide of this document. You may need additional disk space. For example, if an ABAP instance is also installed on the SAP HANA server, more space will be required. The size of these disk areas are calculated from the memory size (RAM) of SAP HANA. The value used for calculation is the memory size used by SAP HANA, not physical memory size of SAP HANA server. For example, your sizing requires 1.3TB memory for SAP HANA, and you want to deploy SAP HANA on 2TB memory server, the disk size should be calculated by using 1.3TB. SAP HANA data The size of data volume is 1.2 x net disk space for data(including additional free space of 20%) or 1 x RAM. net disk space for data is equivalent to the value calculated using the SAP Quick Sizer or one of the sizing reports(sap note , , ). If SAP HANA requires more data area, you can add disks later. Note: SAP HANA appliance has 3 x RAM for SAP HANA data. 7 NEC Corporation 2018

8 SAP HANA log The size of the log volume depends on the RAM size. RAM size is 512GB or less: the log volume is 1/2 of RAM size. RAM size is more than 512GB: the log volume is minimum 512GB. OS, /usr/sap, swap These are local to each server and can be put on local disks of the server or storage. The minimum requirements for each area are: OS installation: 10GB /usr/sap: 50GB A small swap space is recommended in order to avoid performance regressions. A good value for the swap space is 2 GB (SAP note FAQ: SAP HANA Memory). If you enable kdump, disk space for dump is also need to be considered. SAP HANA shared This is mounted on /hana/shared/<sid> and SAP HANA is installed here. In a single node system, this area can be put on a server local disk or on the storage. In a scale out system, this area is accessed from all HANA nodes and implemented using a shared file system such as OCFS2 or NFSv4. For single-node SAP HANA systems, the recommended disk space for /hana/shared/<sid> is: RAM is 1024GB or less: size of shared is 1 x RAM RAM is 1024GB or more: size of shared is 1024GB For scale-out SAP HANA systems, the recommended disk space for /hana/shared/<sid> depends on the number of worker nodes. Per each four worker nodes of a given scale-out system, a disk space of 1x RAM of one worker is recommended: size of shared is 1 x RAM_of_worker per 4 worker nodes Examples: 1 ~ 4 x 1.5TB worker node: size of /hana/shared/<sid> is 1 x 1.5TB = 1.5TB 5 ~ 8 x 1.5TB worker node: size of /hana/shared/<sid> is 2 x 1.5TB = 3TB 9 ~ 12 x 1.5TB worker node: size of /hana/shared/<sid> is 3 x 1.5TB = 4.5GB 1 ~ 4 x 3TB worker node: size of /hana/shared/<sid> is 1 x 3TB = 3TB 5 ~ 8 x 3TB worker node: size of /hana/shared/<sid> is 2 x 3TB = 6TB 9 ~ 12 x 3TB worker node: size of /hana/shared/<sid> is 3 x 3TB = 9TB Backup area This is a disk space to keep backup files of HANA DB. Prepare necessary space according to your backup policy. It is recommended to use another disk pool from HANA data and log disk pool. Disk drive configuration At first, RAM size of HANA is determined using SAP Quick Sizer or sizing report. Then necessary disk spaces are calculated from RAM size and additional requirements such as back up. Please consult NEC SAP competence center for recommended disk drive configuration. The next table is sample of it. 8 NEC Corporation 2018

9 Sample disk configuration on M120: All in a pool HANA RAM Total Disk Size Sample configuration 256 GB 1,708 GB 6 x SSD 400GB, RAID 5/50(4+P) 512 GB 3,116 GB 4 x SSD 1.6TB, RAID 5/50(2+P) 1,024 GB 5,932 GB 32 x HDD 600GB, RAID 10 3,072 GB 14,124 GB 40 x HDD 600GB, RAID 5/50(4+P) 6,144 GB 26,412 GB 65 x HDD 600GB, RAID 5/50(4+P) 9 NEC Corporation 2018

10 Installation and Configuration Please consider the following when you install SAP HANA on server and storage. SAP HANA installation in TDI must be done by who is certified as SAP Certified Technology Specialist SAP HANA Installation (E_HANAINS141). Access Control The istorage AccessControl is a function of istorage. It associates logical disks on storage and server. The logical disks are only visible from associated servers. Virtual Cache Partitioning istorage VirtualCachePartitioning virtually divides primary cache of disk arrays to multiple areas and assigns each divided area to a virtual machine or business application. This ensures stable performance by restricting the cache memory used by each virtual machine or business application and excluding intervention. Path redundancy The storage and server must be connected with multiple FC paths and setup of path redundancy is required. You can use the native Linux multipathd functionality. The settings of multipathd are in the file /etc/multipath.conf. This is an example used in a RHEL 7.4 OS environment. defaults { find_multipaths yes # polling_interval 10 # path_selector "round-robin 0" # path_grouping_policy multibus # uid_attribute ID_SERIAL # prio alua # path_checker readsector0 # rr_min_io 100 # max_fds 8192 # rr_weight priorities # failback immediate no_path_retry 0 features "0" user_friendly_names yes} blacklist { device { vendor "*" product "*" } } blacklist_exceptions { device { vendor "NEC" product "*" } } multipaths { multipath { wwid 258c2323c1e alias hana_log1 } multipath { 10 NEC Corporation 2018

11 } } wwid 258c2323c1e alias hana_data1 Note: value of wwid is specific to each logical disk. istorage StoragePathSavior is the software product to enable path redundancy and workload balancing. You can also use this instead of native Linux multipathd. When you use SAP HANA Storage Connector API described later, you must use the native Linux multipathd. Create udev rules Create the file /etc/udev/rules.d/10-local.rules and add the following two lines: ACTION=="add", SUBSYSTEM=="block", ENV{ID_VENDOR}="NEC", ENV{ID_MODEL}="DISK*", RUN+="/bin/sh -c '/bin/echo > /sys%p/queue/max_sectors_kb'" ACTION=="add", SUBSYSTEM=="block", ENV{ID_VENDOR}="NEC", ENV{ID_MODEL}="DISK*", RUN+="/bin/sh -c '/bin/echo 4096 > /sys%p/queue/nr_requests'" These parameters must be set for better performance. To verify your configuration will work at the next boot you can use the commands: udevadm control --reload udevadm test $(udevadm info q path n /dev/sdg) # /dev/sdg is device of hana data or log The second command produces a long output and close to the end you should see the following two lines: run: '/bin/sh -c 'echo 4096 > /sys/block/sdg/queue/nr_requests'' run: '/bin/sh -c 'echo > /sys/block/sdg/queue/max_sectors_kb'' File system and mount option Though xfs and ext3 are supported for HANA data, log and shared volumes, xfs must be used. In a SAP HANA scale up system, you can format disk by mkfs.xfs command. mkfs.xfs <HANA shared device> mkfs.xfs <HANA log device> mkfs.xfs <HANA data device> Then, add the following mount options to /etc/fstab <HANA shared device> /hana/shared xfs inode64,largeio,swalloc 2 1 <HANA data device> /hana/data xfs async,inode64,nobarrier,logbsize=256k,largeio,swalloc 2 1 <HANA log device> /hana/log xfs async,inode64,nobarrier,logbsize=256k,largeio,swalloc 2 1 Please use async,inode64,nobarrier,logbsize=256k,largeio,swalloc to mount file system. In a SAP HANA scale out system, HANA data and log volumes also need to be xfs file system and are mounted by HANA using SAP HANA Storage Connector API. HANA shared volume is shared between worker and standby nodes. It should be NFS exported xfs file system. SAP HANA Storage Connector API In a SAP HANA scale out system, which has more than one worker and standby nodes, SAP HANA data and log disks are mounted and unmounted by the SAP HANA storage connector API for Fibre Channel (fcclient). While the disk is mounted, a SCSI-3 Persistent Reservation is written to the device and it ensures that other nodes do not access those disks (I/O fencing). The storage connector API is controlled in the storage section of SAP HANA global.ini file. This is an example of it. 11 NEC Corporation 2018

12 [storage] ha_provider = hdb_ha.fcclient partition_*_* prtype = 5 partition_*_data mountoptions = -o async,logbsize=256k,largeio,swalloc,nobarrier,inode64 partition_*_log mountoptions = -o async,logbsize=256k,largeio,swalloc,nobarrier,inode64 partition_1_data alias = hana_data_1 partition_1_log alias = hana_log_1 partition_2_data alias = hana_data_2 partition_2_log alias = hana_log_2 When you install SAP HANA, create a global.ini file contains above lines and start the SAP HANA installation with --storage-cfg=/some/path option. /some/path is a directory where global.ini file exists. HANA configuration After SAP HANA is installed, SAP HANA fileio layer needs to be configured for best performance. On HANA 2.0, add following lines to /hana/shared/${sid}/global/hdb/custom/config/global.ini [fileio] async_read_submit[log] = on async_write_submit_blocks[log] = all max_parallel_io_requests[log] = 256 async_read_submit[data] = on async_write_submit_blocks[data] = all max_parallel_io_requests[data] = 256 Then restart HANA. On HANA 1.0, start HANA and execute: su - <sid>adm hdbparam --paramset fileio[log].async_read_submit=on hdbparam --paramset fileio[log].async_write_submit_blocks=all hdbparam --paramset fileio[log].max_parallel_io_requests=256 hdbparam --paramset fileio[data].async_read_submit=on hdbparam --paramset fileio[data].async_write_submit_blocks=all hdbparam --paramset fileio[data].max_parallel_io_requests= NEC Corporation 2018

13 Performance check After you completed the installation of a productive system, you must check that performance of HANA data and log disks fulfills TDI KPIs defined by SAP. This is done by using HWCCT (Hardware Configuration Check tool). The official tool documentation is contained in the PDF attachment of SAP note Hardware Configuration Check Tool - Central Note. The documentation contains the latest KPIs for data throughput and latency. Additionally, SAP Active Global Support offers the HANA Going-Live Check which among other tests conducts a data throughput test using the very same SAP HANA Hardware Configuration Check Tool. As of August 2018, NEC Storage M110/M310/M310F/M510/M710/M710F, M120/M320/M320F are certified with scenario HANA-HWC-ES 1.1. Note: NEC Storage M110/M310/M510/M710/M5000 shipped before July 2018 were certified with scenario HANA-HWC-ES 1.0. Read SAP Note to check which version of HWCCT you should use. 13 NEC Corporation 2018

14 References NEC Storage(Global, Japan) NEC SAP HANA Offering(Global, Japan) Certified and Supported SAP HANA Hardware Directory SAP HANA Tailored Data Center Integration FAQ SAP HANA TDI - Storage Requirements SAP HANA Administration Guide SAP HANA Administration Tools -> SAP HANA Hardware Configuration Check Tool for Tailored Data Center Integration SAP note Hardware Configuration Check Tool - Central Note SAP note SAP HANA Storage Connector API SAP Note FAQ: SAP HANA Memory Sizing 14 NEC Corporation 2018