z/os V1R10 64-bit Architecture z/os v1r10 Enhancements GSE z/os Workgroup 18/03/ IBM Corporation

Transcription

1 64-bit Architecture z/os v1r10 Enhancements GSE z/os Workgroup 18/03/2009

2 64-bit Common Area Virtual Storage Common area VSCR Common area virtual storage use has been a critical constraints issue over the years and will continue to become a critical limiting factor in application growth 64-bit common virtual storage overview Release-to-Release VSCR savings goals Things you can do to alleviate common area constraints Alleviating common area VSCR What R10 provides to alleviate the problem RMF 64-bit common support overview How to monitor 64-bit common virtual storage usage 2

3 64-bit Common Area Virtual Storage IBM recognizes common area virtual storage constraint relief as limiting application growth on System z and has formally committed to alleviate the problem This will continue to require extensive efforts across the z brand from all major z/os components and middleware involved: z/os, DB2, Comm Server (TCPIP/VTAM), IMS, CICS, and MQ 64-bit common support is introduced to provide the needed infrastructure for common area VSCR Common area VSCR is an effort in every z/os release 3

4 Agenda 64- Bit VSCR Overview 64-Bit Common Storage Overview Large Page support z10 RMF Support 4

5 Agenda 64- Bit VSCR Overview 64-Bit Common Storage Overview Large Page support z10 RMF Support 5

6 31-Bit Address Space Memory Map 2 Gig = Extended Private 16 Meg = Extended CSA Extended LPA Extended SQA Extended Nucleus Nucleus SQA LPA CSA Private System Region PSA Global storage includes Nucleus, Extended Nucleus, SQA, ESQA, LPA, ELPA, CSA, and ECSA Global storage managed by VSM includes SQA, ESQA, CSA, and ECSA Upper boundary of ECSA storage and lower boundary of CSA storage are always on a Megabyte boundary Sizes of SQA, ESQA, CSA, and ECSA storage are specified at IPL time via the IEASYSxx parmlib member 6

7 Common Area Below 2 GB 24 bit (below 16M) common storage use continues to grow 31 bit (16M-2G) common storage use continues to grow at about 5%- 20% each release Middleware/applications require larger private areas for optimal operation Example: DB2 s DBM1 private area LPA areas growing as well (50M-100M) Common Squeezing Private is a well-known issue 7

8 z/os V1R9 Addressability x (High non-shared) User Private Area Region 1st table (R1T) Data spaces TB Area reserved for Memory Sharing (z/os V1/R5) (Low non-shared) User Private Area Region 2nd table (R2T) 2 TB Region 3rd table (R3T) Below 2GB The bar 16 M - Line 8

9 Necessary VSCR Move data out of common, when possible Where to move data depends on attributes of the required storage 64-bit common is the direction IBM lab has decided to focus on exploiting Data traditionally in the common area needs to move elsewhere: Private Common addressed dataspace (CADS), still a strategic direction 64-bit shared 64-bit common with z/os V1R10 9

10 Potential Candidates for Storage Relief I/O Supervisor of z/os DB2 Communication storage manager used by IP Services IMS Potential future areas of interest: MQ, CICS, and WAS ISV cooperative initiative IBM will assist ISVs in identifying common storage relief opportunities in ISV code Have profiled customers with VSCR issues to help us identify key VSCR areas 10

11 Storage Use must be Analyzed Why is the block in common storage? How is the block accessed? By one address space By a set of selected address spaces By every and any address space in the system Does the block need fixed storage? Is block used for I/O? Does the block need disabled reference storage? Do you need to split the block? Must block remain in 31-bit storage? 11

12 What are Alternatives Move to private Need to be in common storage? no, move to private Move to 31-bit private if small amount of data Move to 64-bit private if large amount of data Other address spaces can access the data Pages can be DREF, fixed in central storage Storage is task/address space owned Move to 64-bit shared memory CADS dataspace 64-bit common - z/os V1R10 solution 12

13 DREF Storage All virtual pages in z/os are either: Fixed in central storage, pageable, or DREF DREF meant fixed in expanded storage DREF pages are not backed by central storage frames until they are referenced Note: Expanded storage no longer exists To keep compatibility, DREF pages are now fixed in central storage IBM now recommends that vendors use pageable pages instead of DREF storage, if possible or consider using the PGSER macro to fix pages in central storage 13

14 Summary of Alternatives Storage Attributes Private 64-Bit Shared (z/os V1R5) 64-bit Common (z/os V1R10) CADS Accessed by one space Ideal Not best solution Not best solution Not best solution Accessed by a set of spaces Accessed by every space Possible Ideal Possible but - potential for overlays Possible but cumbersome Possible but cumbersome Yes Possible but - need multiple dataspaces if data > 2GB - MAXCAD limit Yes but - see above DREF Storage supported Yes for 31-Bit No for 64-Bit No Yes Yes Fixed storage supported Storage Ownership Yes No Yes No (for external callers) Yes (for internal components) Task/Address Space System Storage must be explicitly freed System Storage must be explicitly freed Storage managed by a application storage manager 14

15 Virtual Storage Constraint Relief (VSCR) Without VSCR, zos images will be capped, limiting customer growth and resulting in outages 15

16 Agenda 64- Bit VSCR Overview 64-Bit Common Storage Overview Large Page support z10 RMF Support 16

17 64-bit Common Storage Area The 64-bit common area size can be specified via the HVCOMMON keyword in IEASYSxx parmlib member: HVCOMMON ={xxg} {xxt} HVCOMMON value can also been specified by responding to message IEA101A Value Range: 2 Gigabytes (2G) to 1 Terabyte (which is 1024 gigabytes) and resides on a 2 GB boundary Default Value: 64 Gigabytes (64G) When reserving the actual amount of common storage consumed is the amount specified with the HVCOMMON parameter plus 2 GB for operating system purposes 17

18 z/os V1R10 64-bit Common Allows sharing above the bar across every address space Any address space can access storage without explicitly having to request access Only allows authorized code to obtain storage and storage can only be assigned system keys 0-7 (to avoid system integrity issues by unauthorized code) Storage can be DREF or Fixed (as well as Pageable) An owner is associated with the storage for diagnostic purposes Storage needs to be explicitly freed 18

19 z/os V1R10 Addressability with HVCOMMON in IEASYSxx Member x (High non-shared) User Private Area Region 1st table (R1T) Data spaces TB HVCOMMON Default value: 64G Area reserved for Memory Sharing (z/os V1/R5) 64-bit Common User Private Area Region 2nd table (R2T) 2 TB 2 TB 66GB Region 3rd table (R3T) The bar 16 M - Line GB Below 2GB Segment table Page table 19

20 Operator Commands HVCOMMON parameter plus 2 GB for operating system purposes Default if not specified in IEASYSxx is 64 GB D VIRTSTOR,HVCOMMON IAR019I DISPLAY VIRTSTOR 470 SOURCE = DEFAULT TOTAL 64-BIT COMMON = 66G 64-BIT COMMON RANGE = 1982G-2048G 64-BIT COMMON ALLOCATED = 4M Abbreviate: D VS,HVCOMMON 20

21 64-bit Common Storage Monitoring SRM monitors how much of the specified 64-bit Common Storage (HVCOMMON=) is currently in use and alerts the operator when the usage exceeds certain thresholds The following messages get issued by SRM, When 80% or 95% of the 64-bit common storage is in use: IRA130E HIGH COMMON SHORTAGE IRA131E CRITICAL HIGH COMMON SHORTAGE IRA132I HIGH COMMON SHORTAGE RELIEVED 21

22 Storage Map with z/os V1R User Private area R1T Shared area Shared area R2T bit COMMON Data spaces R3T User Private area User Private area User Private area The bar M line 0 22

23 64-bit Common Compared with Other Areas Like Common Below 2G: Like Common Data Spaces: Like 64-Bit Shared: Are visible at the same address in every address space Every address space has access to the memory object once it is created Storage Tracker capabilities are provided for diagnostic purposes Can be pageable, fixed, or DREF Must be explicitly freed Same RAS level Must be explicitly allocated Can be pageable, fixed, or DREF At system-scope, number of 1M pages or CADS cannot be changed within an IPL. Are allocated in 1MB multiples on a 1MB boundary Memory objects managed by sys functions Can be grouped as a set of related objects by specifying a memory object token Must be explicitly freed Unlike Common Below 2G: Memory objects managed by sys functions Can not be implicitly fixed at allocation No subpools Unlike Common Data Spaces: Virtual storage is addressable by all address spaces at allocation time No need to explicitly ask for access to the virtual storage Can be grouped as a set of related objects by specifying a memory object token Must be explicitly freed No capability-based addressing Lower RAS level Unlike 64-Bit Shared: Virtual storage is addressable by all address spaces at allocation time No need to explicitly ask for access to the virtual storage Can be explicitly fixed Can be referenced while disabled (DREF) Lower RAS level 23

24 Using 64-bit common storage IARV64 macro 1 MB memory objects on 1MB boundary IARST64 macro like GETMAIN and STORAGE OBTAIN IARCP64 manage pools of storage cells 24

25 IARV64 REQUEST=GETCOMMON syntax IARV64 REQUEST=GETCOMMON SEGMENTS=xsegments ORIGIN=xorigin KEY=xkey CALLERKEY MOTKNSOURCE=USER MOTKN=xmotkn NO_MOTKN MOTKNSOURCE=SYSTEM OUTMOTKN=xoutmotkn PAGEFRAMESIZE=4K 1MEG MAX TYPE=PAGEABLE DREF 25

26 IARV64 REQUEST=PAGEFIX syntax IARV64 REQUEST=PAGEFIX RANGLIST=xranglist NUMRANGE=xnumrage 1 LONG=YES NO RANGLIST=ranglist Range List Format Virtual Address Number of pages Virtual Address Number of pages 1 to 16 pairs Virtual Address Virtual Address Number of pages Number of pages 26

27 IARV64 REQUEST=GETSTOR syntax IARV64 REQUEST=GETSTOR SEGMENTS=xsegments ORIGIN=xorigin KEY=xkey CALLERKEY FPROT=YES NO DUMP=LIKERGN DUMPPRIORITY=xdumppriority 99 PAGEFRAMESIZE=4K 1MEG MAX TYPE=PAGEABLE DREF 27

28 IARV64 REQUEST=SHAREMEMOBJ syntax IARV64 REQUEST=SHAREMEMOBJ USERTKN=xusertkn RANGELIST=xrangelist NUMRANGE=xnumrange 1 ALETVALUE=xaletvalue 0 SVCDUMPRGN=YES DUMPPRIORITY=xdumppriority 99 COND=YES NO 28

29 IARV64 REQUEST=LIST syntax IARV64 REQUEST=LIST V64LISTPTR=xv64listptrxusertkn V64LISTLENGTH=xv64listlength V64SELECT=YES DUMP=LIKERGN ORDER=ASCENDING DUMPPRIORITY 29

30 Agenda 64- Bit VSCR Overview 64-Bit Common Storage Overview Large Page support z10 RMF Support 30

31 64-bit Common customization 16 Meg-line the Bar 64-bit Common Address spaces 2TB HVCOMMON Large Pages LFAREA Real Storage 31

32 Large Pages Memory architecture to support large (1 MB) pages When large pages are used in addition to the existing 4 KB page size, they are expected to reduce memory management overhead for exploiting applications With the z10 models, page frames can be allocated with a 4K size The z10 additionally supports a larger page frame size of 1 MB It is expected that long running, memory access intensive applications will benefit from large page frames Large pages are treated as fixed pages and are never paged out 32

33 Translation Look-aside Buffers (TLB) DAT is hardware contained in each PU in charge of translating virtual address into real address During such translation, DAT may access up to 5 CS tables (third region, second region, first region, segment and page) managed by z/os To avoid DAT accesses to CS, TLBs were introduced TLBs are a fast array memory within each PU After translating using CS tables, DAT keeps the relation page/frame in a TLB entry and the next time before going through a CS table translation, DAT inspects TLBs looking for the referred page In case of a hit the translation process is much faster 33

34 TLB Performance and 1M Address Pages z10 has 512 TLB entries per PU Due to the explosion of virtual addresses because of 64-bit capability, 512 entries is not enough to generate good performance (a high TLB hit ratio) The introduction of 1M addresses pages decreases the number of pages in virtual storage (fewer larger pages instead of many small pages), thus improving DAT performance To avoid the reformatting of page data sets and high data rate during ASM paging operations those pages must be fixed in central storage (CS) 34

35 Large Page Support Memory reserved for large page support can be defined with a new parameter in the IEASYSxx parmlib member: LFAREA=xx% xxxxxxm xxxxxxg xx% Indicates that the amount of real storage to be used for large pages is specified as a percentage of all online real storage available at IPL time. The maximum amount of real storage that can be used to back large pages is 80% of the online storage available at IPL minus 2 GB 35

36 Large Page Support xxxxxxg - Indicates the amount of online real storage available at IPL time in gigabytes that is to be used for large pages. The maximum amount of real storage that can be used to back large pages is 80% of the online storage available at IPL minus 2 GB. If the amount specified exceeds this value, the specification is rejected and the system issues a prompt for a new value to be specified on the LFAREA parameter. For calculations in gigabytes, x = online real storage at IPL in GB. MAX_LFAREA = (x - 2G) *.8 36

37 Large Page Support xxxxxxm - Indicates the amount of online real storage at IPL time in megabytes that is to be used to back large pages. The maximum amount of real storage that can be used to back large pages is 80% of the online storage available at IPL minus 2 GB. If the amount specified exceeds this value, the specification is rejected and the system issues a prompt for a new value to be specified on the LFAREA parameter. For calculations in megabytes, x = online real storage at IPL in MB. MAX_LFAREA = (x M) *.8 37

38 Large Page Potential Exploiters Future release of DB2 to support for buffer pools Java 6.0 SR1 for z/os planned to support With z/os V1R10, following applies to application use of large pages: Must run on a z10 Must issue the IARV64 macro Must not be dependent upon LE environment Must explicitly run in 64-bit mode Can take benefit of this large page support 38

39 z10 Exploitation New System z10 functions you can exploit z/os V1R8 z/os V1R9 z/os V1R10 Hiperdispatch YES YES YES >128 GB Support YES YES YES >54 CPs for a single LPAR YES YES Capacity Provisioning YES YES Large Page (1 MB) Support YES YES HiperSockets Multiple Write Facility ziip-assisted z/os Global Mirror (XRC) YES YES YES YES ARCH(8), TUNE(8) YES*** YES YES 39

40 Agenda 64- Bit VSCR Overview 64-Bit Common Storage Overview Large Page support z10 RMF Support 40

41 RMF Support for 64-bit Common RMF provides new overview conditions for the Postprocessor based on SMF record 71 RMF provides support for 64-bit common memory in the RMF Postprocessor and Monitor III on a per address space and system basis: By collecting additional data in SMF record 71 By collecting additional data in SMF record 78 subtype 2 By extending the RMF Postprocessor Paging Activity Report By presenting system-wide and address space related information in the new RMF Monitor III Storage Memory Objects report 41

42 SMF Record 71 The following fields are added to the Paging Activity Paging Data Section of SMF Record 71 Offsets Name Len Format Description 1164 x48c SMF71COM 8 float Min.number of 64-bit common memory objects allocated in the system 1172 x494 SMF71COX 8 float Max. number of 64-bit common memory objects allocated in the system 1180 x49c SMF71COA 8 float Average number 64-bit common memory objects allocated in the system 1188 x4a4 SMF71CRM 8 float Minimum number 64-bit common memory frames backed in real storage 1196 x4ac SMF71CRX 8 float Maximum number 64-bit common memory frames backed in real storage 1204 x4b4 SMF71CRA 8 float Average number 64-bit common memory frames backed in real storage 1212 x4bc SMF71CFM 8 float Minimum number of 64-bit common memory frames fixed in real storage 1220 x4c4 SMF71CFX 8 float Maximum number of 64-bit common memory frames fixed in real storage 1228 x4cc SMF71CFA 8 float Average number of 64-bit common memory frames fixed in real storage 1236 x4d4 SMF71CSM 8 float Minimum number of 64-bit common memory auxiliary storage slots 1244 x4dc SMF71CSM 8 float Maximum number of 64-bit common memory auxiliary storage slots 1252 x4e4 SMF71CSA 8 float Average number of 64-bit common memory auxiliary storage slots 42

43 SMF Record 71 Continued The following fields are added to the Paging Activity Paging Data Section of SMF Record 71 Offsets Name Len Format Description 1260 x4ec SMF71COM 8 float Min number of shared memory objects allocated in the system 1268 x504 SMF71COX 8 float Max number of shared memory objects allocated in the system 1276 x50c SMF71COA 8 float Average number shared memory objects allocated in the system 1284 x514 SMF71CRM 8 float Minimum number of high virtual shared memory frames backed in real storage 1292 x51c SMF71CRX 8 float Maximum number of high virtual shared memory frames backed in real storage 1300 x524 SMF71CRA 8 float Average number of high virtual shared memory frames backed in real storage 43

44 New overview/exception conditions are available in SMF type 71 Condition Name Source Average number of 64-bit common memory objects allocated in the system CMOA SMF71COA Average number of 64-bit common memory frames backed in real storage CFRA SMF71CRA Average number of 64-bit common memory frames fixed in real storage CFFRA SMF71CFA Average number of 64-bit common memory auxiliary storage slots CAUXSA SMF71CAA Average number of shared memory objects allocated in the system SMOA SMF71SOA Average number of high virtual shared memory frames backed in real storage SFRA SMF71SRA 44

45 SMF Record Type 78 Subtype 2 Offsets Name Len Format Description 440 x1b8 R782SHBY 48 mixed1 Number of shared bytes allocated in storage above the 2-GB-line. 488 x1e8 R782COBY 48 mixed1 Number of 64-bit common bytes allocated in storage above the 2-GB-line. 536 x218 R782TOMO 40 mixed2 Total number of 64-bit private memory objects allocated. 576 x240 R782SHMO 40 mixed2 Number of shared memory objects. 616 x268 R782COMO 40 mixed2 Number of 64-bit common memory objects. 656 x290 R782LGMO 40 mixed2 Number of large memory objects 696 x2b8 R782TOFR 40 mixed2 Number of 1 MB frames. 736 x2e0 R782MEML 8 binary Address space memory limit in MB 45

46 SMF Record Type 78 Subtype 2 Table1 Table2 Offsets Name Len Format Description 0 x0 VSDGMIN 8 floating Minimum value 8 x8 VSDGNTME 4 binary Time stamp for minimum value. 12 xc 4 reserved 16 x10 VSDGMAX 8 floating Maximum value 24 x18 VSDGXTME 4 binary Time stamp for maximum value. 28 x1c 4 reserved 32 x20 VSDGTOTL 8 floating Total all samples (used to calculate the average). 40 x28 VSDGHWM 8 floating High watermark Offsets Name Len Format Description 0 x0 VSDGMIN 8 floating Minimum number of memory objects / frames 8 x8 VSDGNTME 4 binary Time stamp for minimum value. 12 xc 4 reserved 16 x10 VSDGMAX 8 floating Maximum number of memory objects / frames 24 x18 VSDGXTME 4 binary Time stamp for maximum value. 28 x1c 4 reserved 32 x20 VSDGTOTL 8 floating Total all samples (used to calculate the average). 46

47 Postprocessor Paging Activity Report FRAME AND SLOT COUNTS CENTRAL STORAGE LOCAL PAGE DATA SET SLOT COUNTS (91 SAMPLES) MIN MAX AVG MIN MAX AVG AVAILABLE AVAILABLE SLOTS 1,951,191 1,951,191 1,951,191 SQA 19,220 19,295 19,263 VIO SLOTS LPA 4,751 4,759 4,757 CSA 16,588 16,609 16,601 NON-VIO SLOTS LSQA 49,851 50,763 50,165 REGIONS+SWA 454, , ,700 BAD SLOTS TOTAL FRAMES TOTAL SLOTS 1,951,192 1,951,192 1,951,192 FIXED FRAMES SHARED FRAMES AND SLOTS NUCLEUS 2,385 2,385 2,385 CENTRAL STORAGE 9,119 9,185 9,136 SQA 15,622 15,697 15,665 LPA FIXED TOTAL CSA 8,583 8,599 8,594 FIXED BELOW 16 M LSQA 14,466 14,569 14,516 AUXILIARY SLOTS REGIONS+SWA 36,129 36,693 36,538 BELOW 16 MEG TOTAL 9,119 9,185 9,136 BETWEEN 16M-2G 24,058 24,630 24,467 MEMORY OBJECTS AND FRAMES TOTAL FRAMES 77,388 77,928 77, OBJECTS COMMON SHARED LARGE FRAMES COMMON COMMON FIXED SHARED MB

48 New Monitor III Storage Memory Objects report RMF V1R10 Storage Memory Objects Line 1 of 10 Command ===> Scroll ===> CSR Samples: 60 System: TRX1 Date: 11/28/08 Time: Range: 60 Sec System Summary Memory Objects Frames Area Used % ---- Common Shared Large Common Fixed Shared 1 MB Common Shared 1 MB Service ---- Memory Objects --- Frames Bytes Jobname C Class ASID Total Comm Shr Large 1 MB Total Comm Shr SMSPDSE S SYSTEM M 0 0 COMM644K B DISCRETN G 1.00G 0 COMM641M B DISCRETN M 150M 0 TRACE S SYSTEM M 0 0 GRS S SYSTEM G 0 0 STOR641M B DISCRETN M 300M 0 SMSVSAM S SYSTEM M 0 0 THR64EDT B VEL G G OMVS S SYSTEM M 0 0 STOR644K B DISCRETN M

49 Postprocessor VSTOR Report V I R T U A L S T O R A G E A C T I V I T Y PAGE 6 z/os V1R10 SYSTEM ID TRX1 DATE 11/28/2008 INTERVAL RPT VERSION V1R10 RMF TIME CYCLE SECONDS PRIVATE AREA DETAIL JOB NAME - COMM644K MEMORY LIMIT M NUMBER OF BYTES OF ALLOCATED BLOCKS BY AREA (BELOW 16 MEG) SUBPOOL (AREA) MIN MAX AVG K K K 236 (SWA) 84K K K 237 (SWA) 108K K K 255 (LSQA) 28K K K USER REGION 0 4K K K 121 4K K K 252 (REENTRANT) 36K K K HIGH VIRTUAL MEMORY USAGE (ABOVE 2GB) BYTES MIN MAX AVG PEAK PRIVATE SHARED COMMON 1.016G G G 1.016G MEMORY OBJECTS PRIVATE SHARED COMMON LARGE FRAMES (1MB) TOTAL

50 Thank you! SG z/os Version 1 Release 10 Implementation 50