Technical Paper. Performance and Tuning Considerations on Amazon Web Services with SAS 9.4 Using IBM Spectrum Scale

Transcription

1 Technical Paper Performance and Tuning Considerations on Amazon Web Services with SAS 9.4 Using IBM Spectrum Scale

2 Release Information Content Version: 1.0 May Trademarks and Patents SAS Institute Inc., SAS Campus Drive, Cary, North Carolina SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. Statement of Usage This document is provided for informational purposes. This document may contain approaches, techniques and other information proprietary to SAS.

3 Contents Contents... i Introduction...2 Test Bed Description... 2 Data and IO Throughput... 2 SAS File Systems... 3 Hardware Description... 4 AWS EC2 Instance Compute Node Description... 4 Host Tuning... 4 Additional Settings Used... 4 AWS EC2 Instance Storage Node Description... 4 Host Tuning... 5 Additional Settings Used... 5 AWS EC2 Storage Test Description... 5 AWS EC2 Networking Description... 6 Test Results... 7 Conclusion...8 Acknowledgments...9 Contact Information...9 Appendix IBM Spectrum Scale i

4 AWS Quick Start Script Using IBM Spectrum Scale IBM Spectrum Scale Tunables Red Hat Tuning of the AWS Instances Provisioning the Ephemeral Storage ii

5 Introduction Shared file systems are a necessity w hen implementing multiple SAS 9.4 compute nodes in an Amazon Web Services (AWS) environment. There are many options and questions about shared file systems. This paper presents recent testing and research performed by SAS and IBM using SAS 9.4 w ith IBM Spectrum Scale as the shared file system. It covers a range of topics from performance implications to best practices and tunings to ideal AWS infrastructure configurations. This paper is a basis for an informal round-table discussion that w ill provide some guidance, solicit customer experiences, and create group talk about trends and feedback that w ill be helpful to SAS, the IBM Spectrum Scale team, and the Amazon Web Services (AWS) offering teams. This paper represents test results for SAS 9 w orkloads using AWS Elastic Cloud Computing (EC2) instances. The testing w as conducted using SAS 9.4 and the IBM Spectrum Scale version file system. The configuration included four i3.8xlarge AWS EC2 instances for the compute nodes and four r4.8xlarge AWS EC2 instances for the IBM Spectrum Scale Netw ork Shared Disk (NSD) nodes. Before starting, it should be noted that the decisions made and the conclusions draw n about AWS EC2 design points w ere determined w hen this paper w as w ritten in the first quarter of Be aw are that AWS EC2 offerings are constantly changing. It is in your best interest to understand the rationale used in the selection process. Consider w hat w as done for these specific results as a point-in-time design. Future improvements in AWS offerings might change the selections that w ere made here. Test Bed Description This effort used a scaled test bed of one and four compute nodes running a SAS mixed analytics w orkload. Of interest w as w hether the EC2 instances using IBM Spectrum Scale as the clustered file system (CFS) w ould yield benefits for SAS large-block sequential IO patterns. The test bed chosen for SAS 9.4 in AWS w ith IBM Spectrum Scale testing w as a SAS mixed analytics w orkload. This w as a scaled w orkload of computation and IO-oriented tests to measure concurrent, mixed job performance. The actual w orkload w as composed of 19 individual SAS tests: 10 computation, tw o memory, and seven IO-intensive tests. Each test w as composed of multiple steps. Some tests relied on existing data stores, and other tests (primarily, computation tests) relied on generated data. The tests w ere chosen as a matrix of long-running and shorter-running tests, ranging in duration from approximately five minutes to one hour and 20 minutes. Actual test times vary by hardw are-provisioning differences. In some instances, the same test (running against replicated data streams) w as run concurrently and/or back-to-back in a serial fashion to achieve an average of *20 simultaneous streams of heavy IO, computation (fed by significant IO in many cases), and memory stress. In all, to achieve the 20-concurrent test matrix, 77 tests w ere launched. Data and IO Throughput The IO tests input an aggregate of approximately 300GB of data, and the computation tests input more than 120GB of data for a single instance of each SAS mixed analytics w orkload w ithin 20 simultaneous tests on each node. Much more data is generated as a result of test-step activity and threaded kernel procedures such as the SAS SORT routines (e.g., SORT makes three copies of the incoming file to be sorted). As stated, some of the same tests run concurrently using different data. Some of the same tests are run back-to-back to produce a total average of 20 tests running concurrently. This raises the total IO throughput of the w orkload significantly. In Figure 1, the aggregate SASDATA I/O bandw idth quickly exceeds 2.5GB/sec and achieves a peak of about 4GB/sec w ith the w orkload. This is a good, average, SAS-shop-throughput characteristic for a four-node cluster. This throughput is cumulative f rom all three primary SAS file systems shared by all four nodes. Note that no SASWORK 2

6 I/O is reflected in this figure. SASDATA I/O Bandwidth Total Read GB/sec Total Write GB/sec :16 21:18 21:19 21:21 21:23 21:24 21:26 21:28 21:29 GB/sec 21:31 21:33 21:34 21:36 21:38 21:39 21:41 21:43 21:44 21:46 21:48 21:49 21:51 21:53 21:54 21:56 21:58 21:59 Time Figure 1. I/O Bandwidth of SASDATA in the AWS Cloud SAS File Systems There are three primary file systems involved in the testing. SAS permanent data file space SASDATA SAS w orking data file space SASWORK SAS utility data file space UTILLOC The SASDATA file system resides on IBM Spectrum Scale. The SASWORK and UTILLOC file systems reside on Red Hat Enterprise Linux XFS. The XFS file systems w ere configured on each compute node using local ephemeral storage w ith multiple drives. Ephemeral storage is unique in that storage is deleted and all data is lost every time an AWS EC2 instance is stopped. Therefore, it requires the file system to be re-created and restriped on each reboot or launch of the instance. For this w orkload s code set, data, result space, w orking space, and utility space, the follow ing space allocations w ere made: SASDATA 100TB of EBS st1 storage SASWORK 7.6TB of local ephemeral storage (NVMe) UTILLOC is combined w ith SASWORK space This gives you a general size of the w orkload s on-storage footprint. It is important to note that throughput, not capacity, is the key factor in configuring storage for SAS performance. 3

7 With regard to the enormous amount of file space (100TB) reserved for SASDATA, the input data for tests is only a few TBs in size. How ever, the over-provisioning of storage is required to achieve better throughput. To achieve a sustained 500MB/sec per EBS volume, w e had to create volumes that w ere greater in size than 12TB. For more information about w hy this w as done, see AWS EBS documentation at s.amazon.com/ebs/. Hardware Description This test bed w as run against one and four compute nodes using the SAS mixed analytics w orkload w ith 20 simultaneous tests. The storage nodes w ere configured as four IBM Spectrum Scale NSD nodes. All compute nodes w ere identical in configuration. All storage nodes w ere identical in configuration. AWS EC2 Instance Compute Node Description AWS offers a variety of systems to choose from for the compute nodes. The i3.8xlarge system w as chosen. The selection process considered the number of CPU cores, memory per core, onboard disk drives, netw ork speed, maximum throughput, and cost per hour. Here are the specifications for the four AWS EC2 compute nodes: Host: AWS EC2 i3.8xlarge instances Kernel: Linux el7.x86_64 (RHEL 7.4 HVM) CPU: 16 cores, Genuine Intel Xeon CPU E v4, Family 6 Model 79, Stepping 1, 2 Socket, x86_64, 2.3GHz AWS Available: 1 socket, 16 cores Mem ory: 244GB Disks: 1x10GB EBS virtual disk for OS and system usage, 4x1.9TB NVMe SSD onboard (ephemeral storage) striped together Netw ork: 10Gigabit Maximum Throughput: 875MB/sec from system NIC to EBS storage As of this w riting, tw o metrics w orth noting are the 16-core system and the 875MB/sec throughput. This implies a throughput-per-core ratio of 875/16=54.6MB/sec/core. SAS applications can operate at higher ratios typically at or above 125MB/sec/core. Because of the throughput limitations, the performance scores w ill be limited in comparison to similar systems w ithout this restriction. Host Tuning Host tuning is based on the Red Hat throughput-performance profile w ith modifications performed and saved as sasperformance. See the Appendix for information about the actual modifications and other pertinent information. Additional Settings Used The four NVMe disks w ere formatted and striped as an XFS file system and used for SASWORK and UTILLOC. The file systems w ere created as a logical volume consisting of four LUNs striped at a 64KB-block size to match SASBUFSIZ E. See the Appendix for additional instructions on the procedures used to create the XFS file system and provisions for the directories. AWS EC2 Instance Storage Node Description AWS offers a variety of systems to choose from for the storage nodes. The r4.8xlarge system w as chosen. The selection process started w ith know ing the compute node w as the i3.8xlarge and trying to match the netw ork speed and throughput performance. The selection process w as further narrow ed by considering the number of CPU cores, memory per core, and cost per hour. Note that a storage node w ith a large amount of memory can allow for a larger 4

8 IBM Spectrum Scale pagepool size setting. Pagepool can help the effective throughput of the system, but care must be taken w hen determining this setting because too much can have a negative performance impact. Here are the specifications for the four AWS EC2 storage nodes: Host: AWS EC2 r4.8xlarge instance Kernel: Linux el7.x86_64 (RHEL 7.4 HVM) CPU: 16 cores, Genuine Intel Xeon CPU E v4, Family 6 Model 79, Stepping 1, 2 Socket, x86_64, 2.3GHz AWS Available: 1 socket, 16 cores Mem ory: 244GiB Disks: 1x10GiB EBS virtual disk for OS and system usage Netw ork: 10Gigabit Maximum Throughput: 875MB/sec from system NIC to EBS storage Host Tuning Host tuning is identical to the compute node. It is based on the Red Hat throughput-performance profile w ith modifications performed and saved as sas-performance. See the Appendix for information about the actual modifications and other pertinent information. Additional Settings Used Storage is configured as IBM Spectrum Scale NSD nodes. See the Appendix for additional information about IBM Spectrum Scale and its settings. AWS EC2 Storage Test Description AWS provides several types of storage. Amazon EBS w as chosen due to its highly available, consistent, low -latency block storage attributes. st1 large-block IO w as chosen from the EBS storage sub-categories for its large-block IO capabilities. Within EBS, there is general-purpose SSD (gp2), sustained IOPS SSD (io1), and cold storage (sc1). In general, SAS Foundation requires large-block sequential IO for best performance. We considered w hether throughput w as more important than IOPS for our w orkload, and it w as. The other storage possibilities w ere ruled out for the follow ing primary reasons: cold storage (sc1) is not intended for high performance, sustained IOPS SSD (io1) is cost prohibitive, and low -latency SSD (gp2) is not designed for large-block IO. st1 storage is based on hard disk drive (HDD) technology. Historically, HDD systems that are designed for high throughput require over-provisioning of drives to attain high rates of sustained IO performance. st1 storage is no exception. st1 is designed so that, based on capacity, there is a base IO rate and a burst IO rate w ith flexibility in performance, capacity, and cost. For more information, see AWS documentation on EBS storage at s.amazon.com/ebs/. Because of the test environment, it w as necessary to select a capacity that insured the burst IO rate equaled the base IO rate so that the storage w ould not run out of burst credits. When burst credits are depleted, the result is throttling or reduced IO. Exploring burst budgets w as not in the scope of this testing. Documentation show s st1 storage has a maximum throughput of 500MB/sec. The minimum capacity needed to ensure that burst IO=base IO w as 12.5TiB at the time of this w riting. The compute nodes and storage nodes w ere selected and matched for throughput at 875MB/sec per node. Therefore, each storage node required tw o volumes of 12.5TiB to supply the necessary IO for the 875MB/sec maximum throughput of the storage nodes. Doing the math reveals each storage node w as provisioned w ith tw o 12.5TiB of st1 volumes, for a total of 25TiB per node. The four storage nodes had a total capacity of 100TiB. This might seem to be an extreme amount of storage. How ever, it w as necessary to ensure that long-sustained 5

9 periods of large-block sequential IO not be affected by the expiration of the burst IO budget credits. Preliminary testing of smaller volumes revealed the IO rates did reduce after long periods of use. See the AWS w ebsite for more information. AWS EC2 Networking Description SAS mixed analytics w orkload testing w as performed in the AWS cloud using an AWS virtual private cloud (VPC). The VPC provides a logically isolated virtual netw ork in the AWS cloud. It w as configured in the us-east-1a availability zone. Some of the essential components w ithin the VPC include a public subnet, netw ork address translation (NAT) gatew ay, and a private subnet. Figure 2 show s (in AWS notation) the system components used in the testing environment. Internet Gateway /20 l i ( bli AWS Cloud For SAS Testing VPC NAT gateway Bastion Host Public subnet /24 Amazon Private Subnet xx/24 Amazon EBS st1 volumes 10 Gb netw ork IBM Spectrum Scale NSDs 10 Gb netw ork Placement Group SAS Compute Instances AMIs Availability Zone us-east-1a Virtual Private Cloud Figure 2. AWS VPC Configuration for SAS Testing The test setup w as initially deployed using an AWS Quick Start Script developed by the IBM Spectrum Scale team. Working in collaboration w ith this team, the script w as improved during testing. At the time of this w riting, the Quick Start Script can be accessed at s.amazon.com/quickstart/architecture/ibm-spectrum-scale/. The definitions of the VPC created using the Quick Start Script are listed in the AWS Quick Start Script Using IBM Spectrum Scale section. 6

10 The compute nodes w ere i3.8xlarge systems w ith a netw ork bandw idth of 10Gbit/sec. The storage nodes (NSDs) w ere sized as r4.8xlarge and had a 10Gbit/sec netw ork bandw idth. It is important to note that w hen the VPC w as created, the instances w ere defined using an AWS placement group in one availability zone only. This is important for tw o performance reasons: 1. Placement groups ensure that all nodes are in close proximity to each other. In the case w here instances are not close to each other, the 10Gbit instances could operate at a low er netw ork bandw idth of 5Gbit/sec. 2. A placement group can span availability zones w ithin a VPC. AWS recommends doing so for high availability (HA). How ever, it w as decided not to span availability zones for this testing because doing so automatically limits the netw ork performance from 10Gbit/sec to 5Gbit/sec. More information about placement groups can be found at s.amazon.com/awsec2/latest/user Guide/placementgroups.html. The AWS VPC had a public subnet that used an AWS elastic IP of and w as the route to the internet. In this type of setup, outbound traffic is routed through the NAT instance, w hich provides isolation from public internet traffic w hile allow ing outbound traffic from the public subnet ( /20). The private subnet ( xx/24) is used to route traffic w ithin the internal netw ork. The compute nodes and storage nodes are in the private subnet. Security groups are a part of the netw ork fabric and are important for privacy and security. They act as firew alls and can restrict outbound and inbound traffic. For the purpose of this testing, the security groups w ere minimally configured and are not covered in this paper. Test Results The SAS mixed analytics w orkload w as run in a quiet setting. There w as no competing activity on the server or storage. Multiple runs w ere performed to standardize results. Table 1 show s the performance of a single node run w ith SASDATA on XFS, a single node run w ith SASDATA on IBM Spectrum Scale, and the per-node average of a four-node run w ith SASDATA on IBM Spectrum Scale. SASWORK/UTILLOC resided on local NVMe SSDs (ephemeral storage) for all of these runs. This table show s a frequency mean value of the CPU/real-time ratio, summed f rom all of the 77 tests submitted. It show s the summed User CPU Time and summed System CPU Time in minutes. Number of Nodes SASDATA File System Type Mean Value of CPU/Real Time Ratio Elapsed Real Time in Minutes Workload Aggregate User CPU Time in Minutes Workload Aggregate System CPU Time in Minutes Workload Aggregate 1-Node XFS Node Spectrum Scale Node (avg) Spectrum Scale Table 1. Performance The third column show s the ratio of total CPU Time (User + System CPU) to total Real Time. If the ratio is less than 1, then the CPU is spending time w aiting on resources, usually IO. The AWS test configurations delivered excellent ratios ranging from 1.08 to 1.11 of Real Time to CPU. The natural question is, How can I get above a ratio of 1.0? Because some SAS procedures are threaded, you can actually use more CPU cycles than w all-clock time (Real 7

11 Time). The fourth column show s the total elapsed Real Time in minutes, summed from each of the jobs in the w orkload. The AWS-configured single node run w ith SASDATA on XFS executes the aggregate run time of the w orkload in approximately 804 minutes of total execution time. The one- and four-node runs w ith SASDATA on IBM Spectrum Scale executes the aggregate run time of the w orkload in 817 minutes and 824 minutes of real time per node, respectively. The primary take-aw ay from these tests is that the one- and four-node AWS configurations using IBM Spectrum Scale NSDs easily provided enough throughput to fully exploit this host environment. The four-node AWS configuration using IBM Spectrum Scale could meet the accelerated and scaled throughput demand, w hile providing a very healthy CPU/Real Time ratio per node. Conclusion These conclusions come w ith a caveat. The results are good relative to the environment and the envelope of performance. The governance of the 10Gbit netw ork fabric in AWS constrains the overall IO potential. This is due to the NIC-to- EBS limit of 875MB/sec throughput of the best performing systems today in 10Gbit netw ork fabric. This equates to a potential IO of 54.6MB/sec/core of the 16 core systems tested. As of this w riting, SAS recommends at least 125MB/sec/core of throughput, or more than 2.28 times the current AWS limits. Understanding this point is key and essential to reading the conclusions the results are good relative to the potential performance. You might have questions about the 25Gbit systems offered in AWS that are 2.5 times faster in fabric and have full duplex (bi-directional) capability in the netw ork. These systems w ere tested and found to have little improvement in performance and w ere more expensive. This is due to the 25Gbit systems having double the cores (i3.16xlarge-32 cores) and double the throughput (1,750MB/sec) to equal the same throughput-per-core ratio of 54.6MB/sec/core (1750MB/sec/32 cores). Note that the i3.8xlarge w as not available in 25Gbit fabric as of this w riting. Had it been available, there might have been significantly higher scores. This information intends to condition you to take the proverbial grain of salt w ith the conclusions. Given the limits of the environment and having an understanding of the performance impacts, the systems w ere designed and tuned to their best potential. That being considered, the results are good. In summary, the results are good. The test demonstrates that the AWS i3.8xlarge server instance has a capability of providing reasonable performance for a SAS mixed analytics w orkload in this performance environment. Also, the r4.8xlarge server instance w as a good choice for the storage nodes because it matched the performance metrics of the compute nodes for throughput, netw ork bandw idth, and memory. As AWS is constantly evolving, if the throughput limits per core w ere to increase, the resulting scores w ould likely have a corresponding benefit. The IBM Spectrum Scale shared file system proved to be of value in using the large, high-throughput EBS st1 spinning-array block storage supporting four compute nodes. Using the XFS file system for comparison and considering the purpose of the shared file system, the scores demonstrate the performance benefits of IBM Spectrum Scale. A final note: You should consider that the w orkload used in this testing attempts to model a mixed representation of w hat an average SAS shop might be executing at any given time. Because your results might vary from those presented in this paper, it is crucial to plan and perform your ow n testing to conf irm that this is a viable solution for your particular needs. AWS might be able to satisfy a business need for your company in this domain of high- 8

12 performance analytics. Be sure to consider all of the factors involved w ith regard to performance and cost so that you can set the proper expectations. Acknowledgments This project w as a collaboration betw een IBM and SAS. Special thanks go to the extended IBM Spectrum Scale AWS team (Gautam Shah, John Lew ars, and Dheren Singh), the SAS Technical Solutions support team, and the AWS support team, w ith a special nod to Joe Baron from Amazon for his architecture-enablement training and support during this project. Contact Information Your comments and questions are valued and encouraged. Contact the authors at: Ben Smith IBM Softw are Defined Computing +1 (919) smithbe1@us.ibm.com Jim Kuell SAS Institute Inc. +1 (919) Jim.Kuell@sas.com Brian Porter IBM ISV Enablement/Spectrum Scale +1 (720) bporter1@us.ibm.com 9

13 Appendix IBM Spectrum Scale IBM Spectrum Scale is a pow erful shared file system that provides w orld-class reliability, scalability, and availability for cloud, big data analytics, and high-performance computing environments. IBM Spectrum Scale simplifies data management w ith integrated tools designed to help manage from gigabytes to petabytes of data and thousands to billions of files. Key features of IBM Spectrum Scale include: Unified block, file, and object storage. Massively parallel data access for high performance. True softw are-defined storage deployment using either cloud storage on-prem commodity hardw are or the pow erful Elastic Storage Server grid architecture. Transparent cloud-tiering for hybrid cloud storage in minutes. Integrated user interface w ith the entire IBM Spectrum storage family for simplified administration. Comprehensive data lifecycle management tools, including transparent policy-driven data migration, highspeed metadata scanning, and native data compression and encryption. The SAS testing team described IBM Spectrum Scale as a mature and scalable clustering product that has been tested and proven with SAS workloads to have specific advantages compared to competing products. AWS Quick Start Script Using IBM Spectrum Scale This w eb link provides the AWS Quick Start Script used to create the VPC, security groups, placement groups, NAT, bastion box, and other components of the test environment: s.amazon.com/quickstart/architecture/ibmspectrum-scale/. The AWS VPC cluster w as built w ith the follow ing parameters: Big5-Redux-ClusterStack-B5LW AmiID BastionSecurityGroupID BlockSize ComputeInstanceType ami-de8aeea4 sg-c11d68b4 2M i3.8xlarge ComputeNodeCount 4 CreatePlacementGroup TRUE DataReplica 1 DiskPerNode 2 DiskSize EBSType GpfsMountPoint KeyPairName st1 /gpfs/fs1 sasaw s-va 10

14 Big5-Redux-ClusterStack-B5LW LicenseAgreementTerms Operator PrivateSubnet1ID PrivateSubnet2ID Accept subnet-1b subnet-1b RootVolume 100 ServerInstanceType r4.8xlarge ServerNodeCount 4 SpectrumS3Bucket spectrum-scale-bucket VpcId vpc-bfa0d5c7 IBM Spectrum Scale Tunables The IBM Spectrum Scale clustered parallel file system has many tunable parameters. There are typically only a few of these that are changed to benefit the performance for SAS. The follow ing commands w ere used to configure the file system for AWS testing: mmchconfig workerthreads=1024 mmchconfig prefetchpct=40 mmchconfig maxfilestocache=1m mmchconfig maxmbps=2048 mmchconfig pagepool=64g Red Hat Tuning of the AWS Instances The follow ing tunings that w ere used for the i3.8xlarge AMI compute nodes are based on the Red Hat Enterprise Linux 7.4 operating system. Any other version of the operating system might require different settings. See the follow ing document for more information about configuring Red Hat systems for SAS w orkloads: Here are the steps for tuning: 1. Navigate to the tuned directory and create a sas-performance profile as a copy of the throughputperformance profile. Edit its tuned.conf file. cd /usr/lib/tuned/ cp -r throughput-performance/ sas-performance vi sas-performance/tuned.conf 2. Edit the tuned.conf file to appear as follow s: [cpu] force_latency=1 governor=performance energy_perf_bias=performance min_perf_pct=100 [vm] 11

15 transparent_huge_pages=never [sysctl] kernel.sched_min_granularity_ns = kernel.sched_wakeup_granularity_ns = vm.dirty_ratio = 30 vm.dirty_background_ratio = 10 vm.swappiness=30 3. Set the new tuned profile as the active profile. This can be validated w ith tuned-adm active. tuned-adm profile sas-performance 4. Change the ec2-user's max open files to 500,000 and max user processes to 131,072: vi /etc/security/limits.conf 5. Add the follow ing tw o lines: ec2-user - nofile ec2-user - nproc Log off and back on after performing these steps. Verify changes w ith ulimit -a. Provisioning the Ephemeral Storage The SAS compute nodes are i3.8xlarge systems. They each include four 1.9TiB NVMe disk drives suitable for use as a combined SASWORK/UTILLOC directory. The NVMe drives are called ephemeral because all current information is lost every time the instance is started or rebooted. They are temporary by design. To use these ephemeral drives to their best advantage, the follow ing commands w ere executed every time the instance w as started or rebooted. These commands created the physical volumes, volume group, logical volume, and XFS file system and mounted the drives for use in the testing performed in this paper. Use cases and environments vary, so consider this as just a reference that can be used to model similar tasks in creating local file systems for use w ith SAS. In this specific case, the four drives w ere striped as a RAID 0 system w ith a 64KB block size to match the setting of SAS BUFSIZE and a 16MB read ahead. # ===================================================================== # ---- Commands to Re-Create Volumes for Striped /saswork Directory # # ====================================================================== # # st Time setup: # Need to run as root # Need lvm2 package installed on a New Instance # Need to create /saswork directory # # yum install lvm2 # mkdir /saswork # # ---- Rebuild the Physical Volumes, Volume Group, Logical Volume, xfs 12

16 # filesystem and mount /saswork # # List devices blockdev report # Create physical volumes from four ephemeral devices pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1 # View physical volumes pvdisplay # Create a volume group with the four physical volumes vgcreate vg_sas_ephemeral /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 dev/nvme3n1 # View volume groups vgdisplay # Create a logical volume from the volume group with a 64 KB block size # and a 16 MB read ahead lvcreate -l100%free vg_sas_ephemeral -n lv_sas_ephemeral -i4 -I64 -r32768 # View detailed info about logical volumes lvs -o name,vg_name,size,attr,lv_size,stripes,stripesize,lv_read_ahead # Create the xfs file system mkfs.xfs /dev/vg_sas_ephemeral/lv_sas_ephemeral # Mount the saswork file system mount /dev/vg_sas_ephemeral/lv_sas_ephemeral /saswork # View file systems df ht 13

17 To contact your local SAS office, please visit: sas.com/offices SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. Copyright 2018, SAS Institute Inc. All rights reserved.