64K page size enablement of Active Memory Expansion

Size: px

Start display at page:

Download "64K page size enablement of Active Memory Expansion"

Aron Phillips
6 years ago
Views:

1 64K page size enablement of Active Memory Expansion

2 Table of Contents 1.Overview Prerequisites to enable all page sizes in an AME environment Enabling all AIX supported page sizes in an AME environment VMO tunable ame_mpsize_support Modeling for expansion factor in mixed page size Workloads Live Partition Mobility (LPM) and Suspend/Resume Deactivating accelerator in a 64K page AME environment Best Practices Performance of a 64K page enabled AME environment...5

3 1. Overview Active Memory Expansion (AME) is an existing technology. Memory expansion is achieved by compressing in-memory data. Until now, an AME environment only supported 4K and 16MB page sizes. This document details all the enhancements to existing AME technology. Prior to going through this document, please refer to Active Memory Expansion: Overview and Usage Guide for an indepth understanding of the AME technology. This feature, 64K page size enablement of AME, is available in AIX 7.2 TL1 and requires Power8 native mode, compression accelerators, and firmware FW860. Enhanced Power8 memory bandwidth has enabled its compression accelerators to efficiently compress/decompress 64K pages. FW860 exploits this capability by providing 64K compression services to guest operating systems. All AIX supported page sizes are enabled in an AME environment, with the exception of 16MB Mixed Page Size Segment (MPSS). The AME environments will now have support for the following page sizes - 4K 64K MPSS 4K_64K 16MB 16GB Power8 will now have support for 64K pageable page size in an AME environment. When running with an expansion factor of 1.0, applications will not experience any performance degradation, which previously was the case when enabling AME, due to the loss of 64K page size support. Now, that the 64K pageable page size is supported in an AME environment, administrators can set AME ON as the default policy, with an expansion factor of 1.0, for all the deployed LPARs (on a Power8). With the expansion factor of 1.0, administrators don't have to worry about tracking individual LPARs and workloads to manage when AME can be used and when it cannot. Later, when it is desired and convenient, AME modeling can be done for workloads and an expansion factor set as appropriate. In an AME enabled environment, both 16MB and 16GB pages will remain in an uncompressed form. The expansion of memory will be achieved by compressing 4K and 64K pages. For the rest of the document, references to all page sizes means all AIX supported page sizes except 16MB MPSS. 2. Prerequisites to enable all page sizes in an AME environment Until now, AME environments only had 4K and 16MB pages enabled. 4K pages would use an accelerator for compression and decompression, if available. It was advisable to have an accelerator but not necessary. 64K page size requires an accelerator for compression and decompression in an AME environment. Power8 systems with firmware FW860 have accelerators that support 64K page compression and decompression as a single unit. AIX 7.2 TL01 will exploit this Power8 accelerator support of 64K page size. On Power8 with firmware FW860 and AIX 7.2 TL01 all page sizes can be enabled in an AME

4 environment. For the rest of the document, unless otherwise specified, references to Power8 means Power8 with firmware FW860 and compression accelerators. 3. Enabling all AIX supported page sizes in an AME environment An Upgrade of an existing Power8 AME environment to AIX 7.2 TL01 will not automatically get all page sizes enabled. A new install of AIX 7.2 TL01 on a Power8 machine will not automatically get all page sizes enabled in an AME environment. In both these scenarios, by default, legacy behavior will be preserved in an AME environment. Only 4K and 16MB page sizes will be enabled. There will be no change in the environment. To enable all page sizes in an AME environment, the user would need to explicitly specify the intent to do so. The user will use a new vmo tunable to enable all page sizes in an AME environment VMO tunable ame_mpsize_support A new, unrestricted vmo tunable is added to facilitate enabling of all page sizes in an AME environment. This will require bosboot followed by a reboot of the LPAR. Steps to enable all page sizes in an AME environment - 1. On the HMC profile for the LPAR, enable AME with an expansion factor of 1.0 or with an expansion factor as recommended by the amepat command. 2. On the LPAR, enable all page sizes in the AME environment by setting the new vmo tunable vmo -ro ame_mpsize_support=1 3. bosboot and reboot the LPAR 4. Modeling for expansion factor in mixed page size Workloads The amepat command is used to model for expansion factors for an AME environment. Since all true AME workloads are 4K pages only today, the amepat command model assumes 4K data compression. When it encounters a 64K page, it will break it down into 16 4K separate compresses. In AIX 7.2 TL01, the amepat command has been enhanced to work well in a multiple page size environment with both 4K and 64K page workloads. A single unit of 64K page data compresses better than 16 4K units. The amepat command has been enhanced to understand 64K pages. A new flag, -M, has been added to the amepat command. This flag indicates to amepat to compress a 64K page as a single unit when determining compressibility of the workload. This flag can only be used when modeling on or for a Power8 machine. The amepat command will give more accurate expansion factor recommendations when specified with the -M flag in 64K page workload environments. The default behavior for amepat will remain unchanged unless the new flag is specified. Now, with both 4K and 64K pageable page sizes being supported in an AME environment, it is advised to model for expansion factors in an AME enabled environment with an expansion factor set to 1.0. With the expansion factor of 1.0, AME will be enabled but not used (since there is no memory

5 expansion achieved at an expansion factor of 1.0). When modeled in an AME enabled environment with an expansion factor of 1.0, the amepat command will give more accurate expansion factor recommendations. 5. Live Partition Mobility (LPM) and Suspend/Resume 64K page size accelerator support is required to enable the 64K page size in an AME environment. Power8 with firmware FW860 enables accelerators to compress/decompress 64K pages. Thus when performing LPM or Suspend/Resume of a '64K page enabled AME' LPAR, the destination LPAR is required to have 64K page accelerator support as well. LPM will fail in the initial stage if it is determined that the destination LPAR doesn't have 64K accelerator capabilities and the source LPAR has AME with 64K page size enabled. In the case of suspend/resume, suspend will succeed but resume will fail if resume is initiated on an LPAR that doesn't have 64K accelerator capabilities. 6. Deactivating accelerator in a 64K page AME environment A hardware accelerator may be deactivated on an LPAR via the HMC. If 64K page size is enabled in an AME environment and the accelerator is deactivated, an error log entry will be created by the system log facility. After the accelerator is deactivated, both 4K and 64K pages will be compressed/decompressed using software methods. Memory already compressed by the accelerator, prior to deactivating of the accelerator, will be decompressed by software methods. Once the accelerator is activated again for the LPAR, memory will be compressed/decompressed using the accelerator. Memory already compressed using software methods, while the accelerator was deactivated, will be decompressed using software methods. It should be noted that deactivating of the accelerator in a 64K page enabled AME environment is not supported. If the accelerator is deactivated, AME will fall back to software compression/decompression methods to keep the system alive. It is advised not to deactivate the accelerator if 64K page size is enabled in an AME environment. 7. Best Practices 1. Use the amepat command to model for expansion factor recommendations once the workload has stabilized. 2. Workloads that are predominantly file-cache will not benefit much from AME technology. 3. When modeling in an environment with a 64K page workload, specify '-M' flag to the amepat command. 4. To get more accurate expansion factor recommendations from the amepat command, model in an AME enabled environment with the expansion factor of Performance of a 64K page enabled AME environment Now that the 64K page size can be enabled in an AME environment, applications will not see any performance degradation with a 1.0 expansion factor. Until now, when AME was activated on an LPAR, 64K page workloads were converted to 4K workloads which resulted in performance

6 degradation. Here is a comparison of a predominantly 64K page workload in an AME enabled environment. The comparison is made with AME enabled in the legacy mode, with only 4K page support, and also with the enhanced AME, with both 4K and 64K page size support. The workload is a java based application which simulates warehouse transaction processing. The Java environment version was 1.8. First the workload was run with AME disabled. The LPAR has 60GB of memory assigned and 42 dedicated cores running in SMT-4. After stabilizing, the workload was using 98% of the physical memory at 80% of CPU utilization. Throughput of the workload was recorded after a steady state of 1 hour. Subsequently, with AME enabled both in legacy and enhanced mode, the same workload was run with different memory configurations and throughput recorded. There will be no compression/decompression activity when AME is enabled with an expansion factor of 1.0. The results are shown in the following table: Memory configuration (in GB) Memory Expansion 0.00% 25.00% 50.00% 75.00% % Relative Performance (legacy mode) Relative Performance (enhanced mode) AME CPU utilization % (legacy mode) AME CPU utilization % (enhanced mode) The Memory Expansion metric in the above table is calculated by: ((baseline physical memory new physical memory) / (new physical memory)) * 100

7 1.2 Relative Performance % 25% 50% 75% 100% Memory Expansion % Fig 1 - AME Relative Throughput Performace enhanced mode legacy mode The graph above, in Fig 1, shows the relative performance of the workload in an AME enabled environment, at different expansion factors, with respect to AME disabled environment. With AME enabled in legacy mode, with only 4K page support, the performance of the workload went down significantly even at an expansion factor of 1.0. This is due to no 64K page size support in legacy mode. In enhanced AME mode, there is no loss of performance at an expansion factor of AME CPU Utilization % % 25% 50% 75% 100% Memory Expansion % enhanced mode legacy mode Fig 2 - AME CPU Utilization

8 The graph above, in Fig 2, shows the percentage of the CPU utilized by AME at different expansion factors. The graph demonstrates that AME will use considerably less CPU when enabled in enhanced mode. Both the graphs above show that a 64K page focused workload will benefit immensely from enhanced AME. Generally, a 64K chunk of data will achieve a higher compression ratio than 16 individual 4K chunks. For predominantly 64K page focused workloads, when using AME in enhanced mode, a more aggressive expansion factor can be set while still consuming considerably less CPU as compared to legacy AME.

June IBM Power Academy. IBM PowerVM memory virtualization. Luca Comparini STG Lab Services Europe IBM FR. June,13 th Dubai

June IBM Power Academy. IBM PowerVM memory virtualization. Luca Comparini STG Lab Services Europe IBM FR. June,13 th Dubai June 2012 @Dubai IBM Power Academy IBM PowerVM memory virtualization Luca Comparini STG Lab Services Europe IBM FR June,13 th 2012 @IBM Dubai Agenda How paging works Active Memory Sharing Active Memory