LPARs revisited. LPARs revisited. Jan Tits - STG. Thanks to Harv Emery and Jerry Moody IBM Corporation

Transcription

1 LPARs revisited LPARs revisited Jan Tits - STG jantits@be.ibm.com Thanks to Harv Emery and Jerry Moody

2 Agenda 2

3 What are logical partitions Divides physical resources of a single zseries machine amongst multiple logical machine images (partitions) Each partition looks and operates like its own physical machine Independent of and without knowledge of other partitions Potentially different configurations Processors, Storage, I/O, Local Time Its own operating system 3

4 What are logical partitions Usually used for hardware consolidation and workload balancing Partitions are managed by PR/SM Hypervisor which runs native on the machine Work is dispatched on a logical processor basis, not on partition as whole Tasks from different partitions can be operating in parallel on different physical processors At least one partition is required 4

5 What are logical partitions Up to 60 partitions can be defined/active at any given time Defined via IOCDS RESOURCE PARTITION=(CSS(0),(MICKEY,1),(MINNIE,2),(GOOFY,3),(*, 4)(*,5)) Reserved partition (*,id) no longer required for dynamic definition Activation profile contains configuration details Number/type of processors, amount of storage, etc Activation done manually via HMC or automatically at POR 5

6 What are logical partitions Amount of physical processor time given to a partition is based on the partition's logical processor Weight relative to the rest of the Active partitions. If MICKEY has weight 200, MINNIE has weight 100 MICKEY gets up to 200/300 processing power of machine MINNIE gets up to 100/300 processing power of machine 6

7 What are logical partitions Amount of physical processor time given to a partition is based on the partition's logical processor Weight relative to the rest of the Active partitions. If MICKEY has weight 200, MINNIE has weight 100 MICKEY gets up to 200/300 processing power of machine MINNIE gets up to 100/300 processing power of machine If GOOFY is Activated with weight 100 MICKEY gets up to 200/400 processing power of machine MINNIE gets up to 100/400 processing power of machine GOOFY gets up to 100/400 processing power of machine 7

8 Configuration Options Partition Types ESA/390 ESA/390 TPF Coupling Facility Linux VM 8

9 System z10 z/vm-mode partitions z/vm 5.4 New LPAR type for IBM System z10: z/vm-mode Allows z/vm V5.4 users to configure all CPU types in a System z10 LPAR Offers added flexibility for hosting mainframe workloads Add IFLs to an existing standard-engine z/vm LPAR to host Linux workloads Add CPs to an existing IFL z/vm LPAR to host z/os, z/vse, or traditional CMS workloads Add zaaps and ziips to host eligible z/os specialty-engine workloads Test integrated Linux and z/os solutions in the same LPAR No change to software licensing Software continues to be licensed according to CPU type z/vm-mode LPAR z/os Production z/os Dev/Test and Optional Failover z/os CFCC CMS Linux Linux Linux Production Linux Linux Linux z/os z/os z/os CFCC z/vm z/vm LPAR LPAR LPAR LPAR LPAR LPAR 9 CP CP CP CP CP zaap zaap zaap ziip ziip ICF ICF IFL IFL IFL IFL IBM System z10 IFL

10 Configuration Options Processor Types General Processors zaap Application Assist Processors (java) ziip Integrated Information Processors (databases) IFL Integrated Facility for Linux ICF Internal Coupling Facility Each processor type has its own weight per partition 10

11 Configuration Options Physical processors are either... Shared amongst all logical processors in any partition Best way to maximize machine utilization Excess processor resource from one partition can be used by another Can be limited via per partition capping Dedicated to a specific logical processor in a specific partition Provides least LPAR management time Does not allow excess processor time to be used by other logical processors 11

12 PR/SM Hypervisor PU Dispatching Pools PU Pool Physical PUs to dispatch to online logical PUs z10 or z9 EC with 10 CPs, 1 ICF, 2 IFLs, 1 ziip and 3 zaaps CP pool contains 10 CP engines ICF pool contains 1 ICF IFL pool contains 2 IFLs zaap pool contains 3 zaaps ziip pool contains 1 ziip z/os LPAR can have different CP, zaap and ziip weights z/vm-mode LPAR (z10 only) can have different CP, zaap, ziip, IFL and ICF weights z990 with 11 CPs, 1 ICF, 2 IFLs, and 3 zaaps CP pool contains 11 CP engines Specialty pool contains 6 engines ICFs, IFLs, zaaps z/os LPAR zaap weight is set equal to the initial CP weight 12

13 PR/SM Hypervisor PU Pool Rules Logical PUs dispatched from supporting pool only Logical CPs from CP pool only, for example Pool width Width equals the number of physical PUs in the pool Limits an LPAR s maximum number of shared logical PUs brought online PUs placed in pools by Activate (POR) Concurrent Upgrade OnDemand or Concurrent MES Dedicated LPAR deactivation Dedicated LPAR configure logical PU OFF PUs removed from pools by Concurrent Downgrade - On/Off CoD, CBU, CPE, PU Conversion MES Dedicated LPAR activation ( width permitting) Dedicated LPAR configure logical PU ON ( width permitting) 13

14 System z10 Coupling Facility Processors SoD: To be removed on System z future Dynamic ICF Expansion Dedicated ICF and Shared CP in the same CF partition Dynamic ICF Expansion Across ICFs Dedicated ICF and Shared ICF in the same CF partition CF partition processor options Recommended for production Dedicated ICFs (or dedicated CPs Expensive!) Not recommended (Exception: Backup or function test CF only) Shared ICFs or shared CPs Dynamic ICF Expansion Read and follow PR/SM Planning, SB , recommendations on weights, Dynamic Dispatch, and capping VERY carefully to avoid performance problems, wasted resource, link checkstops, etc. Big ICFs on System z10 do NOT address these issues. 14

15 Managing Partitions Dynamically add/remove partition processors/storage as workload requires Requires specifying Reserve resources at partition activation to add Adjust a partition's weight by processor type Cap a partition to the specified weight Limit groups of partitions as an entity 15

16 Hardcapping Enforce the relative weight Never allow the LPAR to use more than its share of resources Aka hard cap or PR/SM hardcap 16

17 Hardcapping 17

18 Hardcapping 18

19 Defined Capacity limit enforcement Set a defined capacity limit in support of Workload License Charges Aka soft cap Measured in MSU (millions of service units) per hour Enforce defined capacity limit using 4 hour average by WLM when 4-hour average goes over the defined capacity limit, WLM caps the partition at IPL WLM defaults to a 4-hour time interval that contains no partition cpu usage Software charges based upon highest observed rolling 4-hour average utilization Available to LPARs that meet these criteria: zseries hardware z/os in 64-bit mode Shared general purpose engines (no dedicated engines) Relative weight NOT enforced (no PR/SM hardcap) 19

20 Defined Capacity limit enforcement 20

21 Defined Capacity limit enforcement 21

22 Defined Capacity limit enforcement 3 possible situations depending on relative sizes of defined capacity limit and the weight of the partition capacity share based on weight = defined capacity limit WLM instructs PR/SM to fully cap LPAR at its weight capacity share based on weight < defined capacity limit not possible to cap LPAR at its weight all the time cap LPAR at its weight part of the time capacity share based on weight > defined capacity limit WLM causes PR/SM to define a phantom weight pretends utilization for an LPAR to make it possible to cap LPAR 22

23 Defined Capacity limit enforcement Phantom weight calculation Phantom Weight(P)= CEC Capacity Defined Capacity Limit(P) All active partitions x Partition Weight(P) - Σ Partition Weight(i) i=1 Value can amount to a maximum of up to 1000 times the number of active LPARs not possible to specify a capacity limit that is very small compared to the capacity based on the weight definition 23

24 Defined Capacity limit enforcement 24

25 LPAR group capacity limit Adds capability to define a z/os LPAR as a member of a group of LPARs Group can cross sysplex boundaries Group can include LPARs not participating in a sysplex Adds capability to specify capacity of the group of LPARs in MSUs per hour Synergy with LPAR defined capacity PR/SM and WLM work together to help: Enforce the capacity defined for the group Enforce the capacity optionally defined for each individual LPAR May provide better control of CP resource consumed for WLC pricing Exclusive to System z9/z10 Requires at a minimum: z/os or z/os.e Version 1 Release 8 (1.8) 25

26 LPAR group capacity limit May help reduce the amount of capping For more productive use of white space and higher utilization Individual LPAR capacity limits Group capacity limit Cappe d Capacity Limit Capacity Limit Capacity Limit Cappe d LPAR2 LPAR2 No cap No cap LPAR3 LPAR3 LPAR1 LPAR1 LPAR1 LPAR2 LPAR3 LPAR1 LPAR2 LPAR3 26

27 LPAR group capacity limit 27

28 LPAR group capacity limit 28

29 LPAR group capacity limit 1 G R O U P C A P A C I T Y R E P O R T - PAGE 3 z/os V1R8 SYSTEM ID ESA7 DATE 03/02/2009 INTERVAL CYCLE SECONDS RPT VERSION V1R8 RMF TIME GROUP-CAPACITY PARTITION SYSTEM -- MSU -- WGT -CAPPING-- - ENTITLEMENT - NAME LIMIT DEF ACT DEF WLM% MINIMUM MAXIMUM -ALDGRP 35 EURO ESA NO EY2 ESA NO IS2 ESA NO TOTAL GROUP-CAPACITY PARTITION SYSTEM -- MSU -- WGT -CAPPING-- - ENTITLEMENT - NAME LIMIT DEF ACT DEF WLM% MINIMUM MAXIMUM -BERGRP 174 ACPT ESA NO PROD ESA NO TOTAL

30 Intelligent Resource Director Automatic adjustments can be made by Workload Manager (WLM) Add/remove partition processors (Vary CPU management) Up to Reserve amount Adjust individual partition weights (CPU weight management) Within specified range Shift weight between members of a sysplex on the same machine (a cluster) 2 other components: Dynamic CHPID management Channel subsystem I/O priority management Partition capping is mutually exclusive with WLM CPU Management Allows system to dynamically move resources to the work Introduces the concept of the LPAR cluster 30

31 CPU weight management WLM manages physical CPU resources across z/os images within an LPAR cluster based on service class goals Dynamic changes to the LPAR weight Sum of LPAR weights can be redistributed within the LPAR cluster Partition(s) outside the cluster are not affected Move CP resource to the partition which needs it LPAR Cluster 2084 z/os SYSPLEX1 z/os SYSPLEX1 31

32 VARY CPU management Dynamic management of online CPs to each partition in the LPAR cluster Optimizes the number of CPs for the partition's current weight Prevents 'short' engines Maximizes the effectiveness of the MVS dispatcher Dynamic Config On/Off z/os SYSPLEX1 LCP LCP LCP LCP z/os SYSPLEX1 LCP LCP Dynamic Config On/Off PCP PCP PCP PCP 32

33 RMF: LPAR Cluster Report 33

34 Intelligent Resource Director 34

35 What Are 'Short CP's? Term created by the WSC performance staff Performance phenomenon created by LPAR hypervisor enforcing LPAR weights on busy processors or capped partitions LPAR ensures each partition has access to the amount of processor specified by the LPAR weights This can reduce the MIPS delivered by the logical CPs in the partition Controlled by a combination of LPAR weights and number of Logical CPs Potential Performance Problems In a processor migration short CPs are not a problem as long as the partition on the new CEC has access to an equal or greater number of MIPS per CP Techdocs Item: WP Performance Considerations when moving to Fewer, Faster CPUs 35

36 Logical to Physical CP Ratio Strive to keep logical to physical ratio in the 2:1 or 3:1 area A higher ratio will work but will cause an increased cost which needs to be factored into the capacity plan Biggest issue to reducing the logical to physical CP ratio is the requirement to run small LPARs as z/os uni-processors Availability issues of running z/os as a uni-processor Places greater emphasis on doing LPAR consolidation to make fewer LPARs which need more than 1 CP of capacity Virtual storage constraints need to be reviewed 36

37 HiperDispatch - Hardware and Hypervisor View Hypervisor (PR/SM) Virtualization layer at Operating System image level Distributes physical resources Memory Channels EMIF Processors Logical processors dispatched on physical processors Dedicated / Shared Affinities Share distribution based on weights Logical View of 2 Book System Memory L2 Cache L1.5 L1.5 PR/SM L1.5 L1.5 L1 L1 L1 L1 L1 Memory L2 Cache L1.5 CPU CPU CPU CPU CPU L1.5 L1 CPU 37

38 The motivation for HiperDispatch Hardware caches are most efficiently used when each unit of work is consistently dispatched on the same physical CPU (or related set of CPUs) In the past, System z hardware, firmware, and software have remained relatively independent of each other But, modern processor and memory designs make a closer cooperation appropriate. Topology is important: Different CPUs in the complex have different distances to the various sections of memory and cache (here, distance is measured in CPU cycles.) Memory access times can vary from less than 10 cycles to several hundred cycles depending upon cache level and whether the access is local or remote. 38

39 Horizontal CPU management PR/SM guarantees an amount of CPU service to a partition based on weights PR/SM distributes a partition s share evenly across the logical processors Additional logicals are required to receive extra service which is left by other partitions. The extra service is also distributed evenly across the logicals. The OS must run on all logicals to gather all its share [z/os Alternate Wait Management] GP GP GP GP GP GP GP GP GP GP GP GP GP GP GP GP ziip ziip LP Red: 16 GPs [weight 500] + 2 zaaps [weight 50] LP Blue: 16 GPs [weight 500] + 2 zaaps [weight 50] Book 0 Book 1 39

40 Vertical CPU Management Logical processors are classified as vertical high, medium or low PR/SM quasi-dedicates vertical high logicals to physical processors The remainder of the share to distributed to the vertical medium processors Vertical low processors are only given service when other partitions do not use their entire share. Vertical low processors are parked by z/os when no extra service is available GP GP GP GP GP GP GP GP GP GP GP GP GP GP GP GP ziip ziip Red: 7 Vertical High GPs Blue: 7 Vertical High GPs M M LP Blue: 8 Vertical Low GPs LP Red: 8 Vertical Low GPs L L Book 0 Book 1 LP Red: 16 GPs [weight 500] + 2 zaaps [weight 50] LP Blue: 16 GPs [weight 500] + 2 zaaps [weight 50] 40

41 HiperDispatch mode PR/SM Supplies topology information/updates to z/os Ties high priority logicals to physicals (gives 100% share) Distributes remaining share to medium priority logicals Distributes any additional service to unparked low priority logicals z/os Ties tasks to small subsets of logical processors Dispatches work to high priority subset of logicals Parks low priority processors that are not need or will not get service The combination provides the processor affinity that maximized the efficiency of the hardware caches 41

42 Addressing Workload Variability SRM Balancer stripes workload across Affinity Nodes by priority in an attempt to keep the work evenly distributed Historic Address space utilization statistics are collected every 2 sec. in an effort to predict future requirements Supervisor implements needs-help algorithm to address transient spikes in utilization Maintains priority-based Affinity Node utilization statistics Responsively acts on statistics by asking other LPs for Help SRM tracks PR/SM white space attributes to dynamically address longer term workload requirements Adds / removes Logical Processors to / from existing Affinity Nodes when both the zos workload warrants it, and the partner LPARs allow it Parks / unparks low priority LPs based on available excess capacity 42

43 Special processing for SYSSTC Work classified to SYSSTC typically contains lots of short-running local SRBs required for transaction flow. Examples of address spaces recommended to be classified into SYSSTC are VTAM, TCP/IP and IRLM. SRBs classified into SYSSTC can execute on any available logical processor even HiperDispatch mode. WLM service policies should be reviewed with this in mind. 43

44 Controlling HiperDispatch HiperDispatch mode is enabled by specifying HIPERDISPATCH=YES in IEAOPTxx The default is HIPERDISPATCH=NO for compatibility HIPERDISPATCH=YES is recommended There is a HealthChecker routine to remind if HIPERDISPATCH=NO Control authority for global performance data must be enabled for proper operation in HiperDispatch mode This option is selected in the logical partition security controls on the Hardware Management Console This is the default selection. 44

45 Hiperdispatch: RMF Report Example z/os V1R9 SYSTEM ID R71 DATE 01/28/2009 INTERVAL CONVERTED TO z/os V1R10 RMF TIME CPU 2097 MODEL 716 H/W MODEL E26 SEQUENCE CODE A73A2 HIPERDISPATCH=YES 0---CPU TIME % LOG PROC --I/O INTERRUPTS-- NUM TYPE ONLINE LPAR BUSY MVS BUSY PARKED SHARE % RATE % VIA TPI 0 CP CP CP CP CP CP CP CP CP CP A CP B CP C CP D CP E CP F CP TOTAL/AVERAGE Medium LCPs Low Un-parked LCPs 45

46 z990 HMC Reset Profile General Page (OS2) Logical partition is the only mode supported, basic mode is not available (HCD also provides only the LPAR mode option) Logical Partition 'Suffix' Naming Convention LPnameXX where LPname is the first 6 characters of the customer required name where xx = LPname suffix 1st character = LCSSid (0 = LCSS.0, 1 = LCSS.1) 2nd character = same as MIFid of 1 to F 46

47 System z10 - Reset Profile - General (CEC TSYS z10 E64) Logical partition is the only mode supported, basic mode is not available (HCD also provides only the LPAR mode option) Logical Partition 'Suffix' Naming Convention LPnameXX where LPname is the first 6 characters of the customer required name where xx = LPname suffix 1st character = CSSid (0 = CSS 0 to 3 = CSS 3) 2nd character = same as MIFid of 1 to F 47

48 System z10 - Reset Profile - Storage Shows customer available storage only on System z10. Fixed HSA is separate on System z10. HSA is included on System z9 and earlier. 48

49 System z10 - Reset Profile - Dynamic Global enable/disable of Dynamic I/O REMOVED on System z10. - Dynamic I/O always enabled at POR on System z10. - Dynamic I/O can be disabled/enabled later on System z10 globally or by partition on the HMC. S/390, z900 and z800 dynamic I/O expansion setting is removed. For z990 to z9, dynamic I/O expansion requirement is supported within HCD (IODF) by the MAXDEV option when defining a subchannel set in an LCSS. This impacts HSA size. On System z10, HSA is fixed with every supported LCSS defined with 15 partitions, both subchannel sets, and maximum devices in both subchannel sets. Dynamic LPAR add/delete is supported by renaming from * to add, to * to delete. Subchannel Set 0 Up to 65,280 subchannels (63.75k) Subchannel Set 1 Up to 65,535 subchannels (64k - 1) HSA Subchannels = MAXDEV times number of LPARs in LCSS 49

50 System z10 Reset Profile - Options No functional change 50

51 System z10 Reset Profile - CP/SAP New on System z10 EC: POR option to convert some purchased CPs to SAPs for the duration of the POR removed. Number of CPs, SAPs, zaaps and ziips is now a comment on System z10. New on z9 EC: Option to view Fenced Book page for Enhanced Book Availability 51

52 System z10 Reset Profile Fenced Book Page Shows purchased processors and available PUs if a 17 PU book is removed. Allows selection of processors to be available after POR with a fenced 17 PU book if all purchased processors do not fit. LIC Processors 75 at top: SAPs + 2 spares = 77 total on E64 With 17 fenced, 59 useable plus 1 spare = 60 remaining on E64 This case: All processors remain available with a 17 PU book fenced. 52

53 System z10 Reset Profile Fenced Book Page Shows purchased processors and available PUs if a 20 PU book is removed. Allows selection of processors to be available after POR with a fenced 20 PU book if all purchased processors do not fit. LIC Processors 75 at top: SAPs + 2 spares = 77 total on E64 With 20 fenced, 57 available plus 0 spares = 57 remaining on E64 Note default removal of ziips, which can be changed. 53

54 System z10 Reset Profile - Partitions Remove partitions not to be activated automatically at POR. Change Order of activation as desired. Remember to activate Internal CFs before supported z/os partitions. 54

55 System z10 Image Profile - General Page (Partition TOSP1) New on System z10: z/vm-mode partition for z/vm 5.4 supports 5 different processor types. The Logical partition 'Partition Identifier' is a 1 or 2 digit unique hexadecimal value from 0 to 3F. Recommended convention: Assign first digit to partition s LCSS ID 0 to 3 Assign second digit to Partition Number 1 to F 55

56 System z10 Image Profile - General Page Partition Time Offset from STP or ETR time Time Offset Page 56

57 System z10 Image Profile Time Offset Typically used with STP or the ETR set to local time when a sysplex that is required to use LOCAL=GMT needs to be set to a different time zone than the CEC. Different sysplexes can also use different offsets to operate on local time in multiple different time zones. Note: This is somewhat unusual. Setting STP or the ETR to GMT, not local time, is recommended. 57

58 System z10 Image Profile - Processor Page Select Capacity Group Profile Name Example: System z10 EC ESA/390 mode partition: CPs, zaaps, ziips Note: z10 EC allows Initial and Reserved to add up to 64 PUs. The operating system sees all those PUs (31 in this case). Don t specify more engines than the operating system supports or engine types it doesn t support. 58

59 System z10 Customize Group Capacity Profile Group Capacity in MSUs Use Customize/Delete Activation Profiles task to copy the Default Group Profile, customize it and save it with the new Group name. This creates a new Capacity Group. Operating System Support z/os 1.8 and above 59

60 System z10 Image Profile - Processor Page 60 Example: System z10 EC z/vm-mode partition (z/vm 5.4 and later) which supports all PU types. New October, 2008: z10 allows creation and save of an LPAR Image profile with an Initial processor definition that can t be activated. Why? Allows creation of profiles to be used when CBU is active.

61 System z10 Image Profile - Security Global performance data = Partition Data Report information on other partitions I/O configuration control = Write IOCDS and HCD dynamic hardware change. Counter Facility Security Options NEW System z10 October, 2008 Sampling Facility Security Options NEW System z10 October,

62 System z10 Image Profile - Storage Recommended! Central storage : z10 EC supports up to 1 TB Central Storage ( Initial + Reserved) maximum in an LPAR Check OS level for supported amounts. Recommended! Expanded Storage: Some OSs do not support. One example is z/os (64-bit) running on a System z10. Initial Storage: Brought ON at LPAR activation. Reserved Storage: Specify like reserved processors to add storage to a running partition. z/os supports configuring storage ON and, if RSU specified, OFF. z/vm 5.4 supports configuring reserved central ON. Storage origin (Central and Expanded storage) It is recommended that you use the 'Determined by the system' option 62

63 System z10 Image Profile - Options I/O Priority Range MSUs - WLC Note: Sysplex cluster name is not required for a z/os member of a parallel sysplex it is required for a non-member partition to be managed as a workload. 63

64 System z10 Image Profile Load Classic Load during activation = IPL when activated Check box and provide IPL parameters (Recommended) Allow dynamic change to IPL address/parameter if wanted. Supports classic IPL or SCSI IPL (e.g. for Linux) 64

65 System z10 Image Profile - Crypto for Crypto Express2 Note candidate list for concurrent addition of Crypto Coprocessors. 65

66 Changing Running Partitions 66

67 System z10 EC Change LPAR Controls CP Tab Enter changes, Change Running System or Save to Profiles or do both. 67

68 System z10 Logical Processor Add New on System z10! Change is concurrent to the running partition. Operating System Support: Concurrent change to the running OS z/ OS 1.10 z/vm

69 System z10 Change LPAR Cryptographic Controls New on System z10! Change is concurrent to the running partition. Operating System Supprt: z/os ICSF: Concurrent change to the running z/os image. 69

70 System z10 Change I/O Priority Queuing 70

71 System z10 Change Logical Partition Security Counter Facility Security Options NEW System z10 October, 2008 Sampling Facility Security Options NEW System z10 October,

72 System z10 Change LPAR Group Controls Select Edit to reassign LPAR Group Membership 72

73 System z10 Change LPAR Group Controls Edit LPAR Group Membership 73

74 System z10 Memory and Addressability 74

75 System z9, z990, and z890 Memory Granularity Memory Granularity = Increment Size Storage assignments/reconfiguration and HSA must be an even multiple Physical increment size fixed at 64 MB Expanded memory granularity always 64 MB Central memory granularity is virtualized for each LP LP central memory increment is determined according to the size of the larger of the two central memory elements defined in the activation profile: Initial central memory or Reserved central memory Single Storage Pool - All central storage ES configured as needed from CS - No POR needed Review MVS RSU parameter. Large z990 increment size may result in too much memory being reserved for reconfiguration after migration unless the new RSU options introduced in OS/ are used. Large Element Size Granularity 64 MB to 32 GB 64 MB >32 GB to 64 GB 128 MB >64 GB to 128 GB 256 MB >128 GB to 256 GB 512 MB Rare to exceed today 75

76 System z10 Memory Granularity Memory Granularity = Increment Size Storage assignments/reconfiguration must be an even multiple System z10 physical increment size is fixed at 256 MB on z10 EC and 128 MB on z10 BC (Was 64 MB on all z9, z990, z890) Expanded memory granularity always 256 MB (64 MB on z9, z990, z890) Central memory granularity is virtualized for each LP LP central memory increment is determined according to the size of the larger of the two central memory elements defined in the activation profile: Initial central memory or Reserved central memory Virtualization determined by the limit of 512 increments per CS element Single Storage Pool - All central storage ES configured as needed from CS - No POR needed Review MVS RSU parameter. Large System z10 increment size may result in too much memory being reserved for reconfiguration after migration unless the new RSU options introduced in OS/ are used. Large Element Size Granularity 128 MB up to 64 GB (BC only) 128 MB 256 MB EC or > 64 GB BC up to 128 GB 256 MB >128 GB up to 256 GB (240 GB BC) 512 MB >256 GB to 512 GB (EC only) 1 GB >512 GB to 1 TB (EC only) 2 GB 76

77 MVS RSU Parameter for System z In IEASYSxx. Specifies the number of central storage increments to be made available for central storage reconfiguration MVS attempts to keep this area free of long term fixed pages calculated as follows for RSU = number CS amount to be reconfigured! RSU = ! storage increment size! Or: Storage to be kept free = RSU * increment If element size is upgraded, check the RSU parameter! z/os - Recommended RSU coding instead of RSU = number to allow z/os to calculate the number of increments RSU = % of storage, MBs or GBs RSU = OFFLINE Amount is Reserved Storage amount. No storage is kept free of long term fixed pages except Reserved Storage that has been configured ON. 77

78 System z10 SE Base System Storage Allocation Note: Memory only adds up to 98% because this machine is in TEST mode running an IBM internal tool that steals 32 GB of memory. 78

79 System z10 Central Storage Addressability Addressability for HSA is allocated top down from the highest supported address System z10 Highest address = 8 TB System z9/990/890 Highest address = 1 TB Partition Addressability is assigned below the fixed 64 GB HSA addressability when the LPAR is activated Initial and Reserved CS are assigned contiguous CS addressability at activation. Initial and Reserved ES (e.g. for z/vm) are assigned contiguous CS addressability at activation. This does NOT have to be contiguous with the partition s CS addressability for CS. In addition they have contiguous ES addressability. Origin addresses are assigned top/down by default but a specific origin can be requested. Allowing default allocation is recommended in almost all cases. An addressability gap created, for example, by LPAR deactivation, can be filled in by a subsequent activation LPAR/HSA Addressability Partition Storage Addressability CS/ES Storage Pool 64 GB (z10 EC) 8,192 GB 8,128 GB Note: Gaps may exist due to storage reconfiguration or LPAR deactivation 79 Physical Memory HSA starts at book/memory physical address 0 (in book 0) Physical memory is assigned for Initial ES and CS at LPAR activation. It is assigned to Reserved ES and CS if and when that memory is Configured ON. PR/SM is allowed to assign an LPAR any physical memory anywhere in the machine. Limit is purchased memory size. There is no requirement for LPAR physical memory to be contiguous Enhanced Book Availability is designed to be able to move partition memory and HSA to different book to enable removing a book from a running machine without disruption. Unused Addressability 0 GB

80 System z10 SE - Logical Partition Storage Allocation CS Addressability Math in MB: 8,388,608 Top End = 8TB - 65,536 HSA = 64 GB - 32,768 Gap = 32 GB - 16,384 TOSPF CS = 16 GB = 8,273,920 TOSPF Origin - 4,096 TOSP1 CS = 4 GB = 8,269,824 TOSP1 Origin HSA is allocated 64 GB of addressability but only 16 GB (8 on z10 BC) of storage. Origin = Start of LPAR s addressability Initial = Initial CS Maximum Initial = Reserved CS Current = Physical Memory Assigned Gap = Unused addressability above Expanded Storage Elements (eg z/vm) Take BOTH CS Addressability below the LPAR s CS origin address (not shown) AND separate ES Addressability (shown). 80

81 Additional Information SB a PR/SM Planning Guide SG IBM System z10 Enterprise Class Technical Guide SC Hardware Management Console Operations Guide SA z/os Planning: Workload Management SG z/os Intelligent Resource Director Available at search on full manual number 81