z10 CPU MF Overview and WSC Experiences

Transcription

1 Advanced Technical Skills (ATS) North America z10 MF Overview and WSC Experiences SHARE Session 2113 March 16, 2010 John Burg IBM Washington Systems Center 1

2 2 Advanced Technical Support Washington Systems Center Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. AlphaBlox* APPN* CICS* CICS/VSE* Cool Blue DB2* DFSMS DFSMShsm DFSMSrmm DirMaint DRDA* DS6000 DS8000 ECKD ESCON* FICON* FlashCopy* GDPS* HiperSockets HyperSwap IBM* IBM eserver IBM logo* IMS Language Environment* Lotus* Large System Performance Reference (LSPR ) Multiprise* MVS OMEGAMON* Parallel Sysplex* Performance Toolkit for VM PowerPC* PR/SM * Registered trademarks of IBM Corporation Processor Resource/Systems Manager The following are trademarks or registered trademarks of other companies. * All other products may be trademarks or registered trademarks of their respective companies. RACF* Redbooks* Resource Link RETAIN* REXX RMF S/390* Scalable Architecture for Financial Reporting Sysplex Timer* Systems Director Active Energy Manager System/370 System p* System Storage System x* System z System z9* System z10 Tivoli* Tivoli Storage Manager TotalStorage* VSE/ESA VTAM* WebSphere* xseries* z9* z10 z10 BC z10 EC z/architecture* z/os* z/vm* z/vse zseries* Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

3 Topics Advanced Technical Support Washington Systems Center MF Introduction What it is and how to enable Lessons Learned Key Performance Metrics CPI, Problem State, z10 Cache / Memory Hierarchy Formulas WSC Experiences with SMF 113s HiperDispatch Theory and Measurement Aligning SMF 113s with Activity by Logical z/os 1.11 Dynamic Channel Path Management (DCM) Measurement Future Direction Workload Characterization Summary 3

4 z10 Measurement Facility Introduction 4

5 What is the z10 Measurement Facility New hardware instrumentation facility Measurement Facility ( MF) Available on System z10 EC GA2 and z10 BC Supported by a new z/os component (Instrumentation), Hardware Instrumentation Services (HIS) Potential Uses for this new cool virtualization technology COUNTERS Supplement Current Performance Metrics Workload characterization SAMPLING ISV product improvement Application Tuning 5

6 High Level Description Data collection done by System z10 hardware Low overhead Little/No skew in sampling Access to information which is not available from software Information about how software and hardware interact 2 Basic Modes COUNTERS SAMPLING This presentation will focus on COUNTERS New IBM Research article IBM System z10 performance improvements with software & hardware synergy Contact IBM team for copy of the article 6

7 Requirements and Steps to utilize z10 MF Requirements for MF System z10 machine must be at GA2 Driver 76D Bundle #20 or higher z10 BC sub-capacity models must be at Bundle #20 or higher z/os LPAR being measured must be at z/os 1.8 or higher with APARs: OA25755, OA25750, and OA25773 OA27623 also recommended to add Speed to SMF 113s and HIS COUNTERS output Not supported for z/os running as a z/vm guest Steps to utilize MF 1. Configure the z10 Server to collect MF Data Update LPAR Security Tabs (See appendix) 2. Configure HIS on z/os to collect MF Data * Set up HIS Proc Set up OMVS Directory Collect SMF 113s via SMFPRMxx //HIS PROC //HIS EXEC PGM=HISINIT,REGION=0K,TIME=NOLIMIT //SYSPRINT DD SYSOUT=* 3. Collect MF Data Start HIS Modify with Begin/End for COUNTERS or SAMPLING F HIS,B,TT= Text',PATH='/his/',CTRONLY,CTR=ALL 4. Analyze the MF Data SMF 113s Remember CTR=ALL to get Extended Counters! 7

8 MF (COUNTERS) for Performance and Capacity Sizing What is MF (COUNTERS)? A new z10 capability to measure z10 cache / memory hierarchy characteristics How can it be used today? To supplement current performance metrics (e.g. from SMF, RMF, DB2, CICS) As a secondary data source to understand why performance may have changed What it may be used for in the Future? To help characterize workload for Capacity Sizing with zpcr What MF is not It is Not a substitute for traditional performance nor capacity metrics It does Not indicate the capacity being achieved by the LPAR or processor You can calculate MIPS, but it has no relationship to commonly used capacity ratings 8

9 MF Lessons Learned since August 2009 MF Performance Metrics can be used to help understand why performance changed Customers are successfully running MF COUNTERS collecting SMF 113s Over days and months without any reported performance impact Feedback from Volunteers is this is very easy to enable, with a minimal time investment SMF 113 Logical IDs are equal to the SMF 70 Logical IDs Can match up SMF 113s & SMF 70s to identify GCPs, ziips or zaaps Can see the unique Vertical Polarity Logical CPs cache/memory characteristics E.G. Vertical Mediums may have higher L2 Remote activity In multi-book z10 ECs there can be L2 Remote Activity even if <=12 GCPs Because of I/O activity from SAPs as the data is initially stored in the Remote L2 Utilize the Counter Version Number fields to map to technology Number is increased for a change in meaning or number of counters SMF113_2_CTRVN1 Basic or Problem-State counter sets SMF113_2_CTRVN2 Crypto or Extended counter sets 9

10 Key Performance Metrics 10

11 MF and HIS provide a z/os logical view of z10 Resource Usage and Cache Hierarchy Sourcing L 1.5 L 1 Memory L2 Cache L 1.5 L 1 HIS LP0 LP1 LP4 z/os LPAR L 1.5 L 1 Book PR/SM MF L 1.5 L 1 Memory L2 Cache L 1.5 L 1 L 1.5 L 1 LPAR / Logical CP view: Memory Accesses Cache L 2 Accesses (local and remote) L1.5 Accesses L1 Sourced from Hierarchy Instructions and Cycles Crypto function 11

12 Introducing MF Key Performance Metrics: SYSID HOUR CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR SYSD 9 AM Hour SYSD 8 AM to 5 PM CPI Cycles per Instruction PRBSTATE - % Problem State L1MP Level 1 Miss % L15P % sourced from L1.5 cache L2LP % sourced from Level 2 Local cache (on same book) L2RP % sourced from Level 2 Remote cache (on different book) MEMP - % sourced from Memory LPAR - APPL% (GCPs, zaaps, ziips) captured and uncaptured Potential W orkload Characterization z10 L1 sourcing from cache/memory hierarcy These numbers come from a synthetic Benchmark and do not represent a production workload 12

13 MF can help provide z10 cache/memory resource change insights Decreased Contention Increased Residency Time = CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR L1 Hit % Increased Contention = Decreased Residency Time CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR L1 Hit % 13

14 Formulas Potential Workload Characterization z10 L1 sourcing from cache/memory hierarcy Metric CPI Calculation note all fields are deltas between intervals B0 / B1 PRBSTATE (P33 / B1) * 100 L1MP ((B2+B4) / B1) * 100 L15P ((E128+E129) / (B2+B4)) * 100 L2LP ((E130+E131) / (B2+B4)) * 100 L2RP ((E132+E133) / (B2+B4)) * 100 MEMP (((E134+E135) + (B2+B4-E128-E129-E130-E131-E132- E133-E134-E135)) / (B2+B4)) * 100 LPAR ( ((1/CPSP/1,000,000) * B0) / Interval in Seconds) * 100 CPI Cycles per Instruction PRBSTATE - % Problem State L1MP Level 1 Miss % L15P % sourced from L1.5 cache L2LP % sourced from Level 2 Local cache (on same book) L2RP % sourced from Level 2 Remote cache (on different book) MEMP - % sourced from Memory LPAR - APPL% (GCPs, zaaps, ziips) captured and uncaptured B* - Basic Counter Set - Counter Number P* - Problem-State Counter Set - Counter Number See The Set-Program-Parameter and -Measurement Facilities SA for full description E* - Extended Counters - Counter Number See IBM The -Measurement Facility Extended Counters Definition for z10 SA for full description CPSP is SMF113_2_CPSP Speed 14

15 WSC Experiences HiperDispatch Theory and Measurement Aligning SMF 113s with Activity 15

16 HiperDispatch attempts to align Logical CPs with PUs in the same Book HD=NO HD=YES Memory Book Memory Memory Book Memory L2 Cache L2 Cache L2 Cache L2 Cache L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1.5 L 1 L 1 L 1 L 1 L 1 L 1 L 1 L 1 L 1 L 1 L 1 L 1 PR/SM MF PR/SM MF HIS LP0 LP1 LP4 z/os LPAR HIS LP0 LP1 LP4 z/os LPAR LPAR / Logical CP view: Memory Accesses Cache L 2 Accesses (local and remote) L1.5 Accesses L1 Sourced from Hierarchy Instructions and Cycles Crypto function 16

17 From MF, HiperDispatch=YES May Decrease the L2 Remote % = CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR L1 Hit % Potential W orkload Characterization z10 L1 sourcing from cache/memory hierarcy CPI Cycles per Instruction PRBSTATE - % Problem State L1MP Level 1 Miss % L15P % sourced from L1.5 cache L2LP % sourced from Level 2 Local cache (on same book) L2RP % sourced from Level 2 Remote cache (on different book) MEMP - % sourced from Memory LPAR - APPL% (GCPs, zaaps, ziips) captured and uncaptured 17

18 HiperDispatch Measurement from an Internal Benchmark HD? 2097 E40 CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR NO 1x32w YES 1x32w L1 Hit % CPI Cycles per Instruction PRBSTATE - % Problem State L1MP Level 1 Miss % L15P % sourced from L1.5 cache L2LP % sourced from Level 2 Local cache (on same book) L2RP % sourced from Level 2 Remote cache (on different book) MEMP - % sourced from Memory LPAR - APPL% (GCPs, zaaps, ziips) captured and uncaptured HiperDispatch=YES resulted in a 12% improvement as measured by the traditional /Transaction metric These numbers come from a synthetic Benchmark and do not represent a production workload 18

19 Traditional Activity - SMF 70s 19

20 SMF 113s Can Be Aligned with SMF 70s - Activity From RMF 70 Records - Activity From MF SMF 113 Records MONTDAY SYSID HOUR POOL PRID POLAR WEIGHT LOGPSHR VPOL CPI PRB L1MP L15P L2LP L2RP MEMP LPAR STATE OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS HIGH OCT 13 ATS MED OCT 13 ATS LOW ***** OCT 13 ATS LOW ***** OCT 13 ATS MED OCT 13 ATS MED ziips Vertical Medium and Remote L2 Since SMF 113 IDs = SMF 70 Processor IDs, this enables Identifying SMF 113 ID types: GCPs, zaaps, and ziips Identifying MF characteristics by Logical e.g. Vertical Medium Polarity Remote L2 Activity 20

21 MF HiperDispatch Summary MF can be used To explain why we saw the improvement from HiperDispatch=YES from the traditional metrics To identify the Logical cache / memory characteristics by Engine Type and/or Vertical Polarity 21

22 WSC Experiences z/os 1.11 FICON Dynamic Channel Path Management (DCM) 22

23 Advanced Technical Skills (ATS) North America New with z/os 1.11 Dynamic Channel Path Management (DCM) WLM can automatically manage ESCON, FICON Bridge (FCV), and FICON I/O paths connected to DASD subsystems in response to changing workload demands. The objectives for FICON DCM are: 1) Simplify I/O configuration planning and definition 2) Reduce the customer skills required to manage z/os 3) Dynamically balance I/O channel resources 4) More efficient use of channel resources 5) Enhance availability 23

24 Advanced Technical Skills (ATS) North America ATS Configuration IBM Systems & Technology Group FICON DCM Evaluation Topology Our Test utilized CHPIDS B8 (Static), B9 (Managed), and BA (Managed) to LCU1 68xx S1 B8 LPA1 SYSC LPAR Cluster A LCSS0 S1 BC M1 B9 LPA2 SYSD M2 BA LPAR Cluster B LPA3 LPB1 LPB2 LCSS1 M3 BB LCSS2 Switch C4 Switch C5 Switch C6 Switch C7 LCU1-68xx LCU2-69xx LCU3-6Axx LCU4-6Bxx 24

25 Advanced Technical Skills (ATS) North America What we did Defined 2 CU Groups CU Group and 6900 addresses - Static Channel B8, DCM Managed Channels B9, BA, BB CU Group 2 6A00 and 6B00 addresses - Static Channel BC, DCM Managed Channels B9, BA, BB DCM will always look to see that a CU Group has 2 CHPIDs for Availability, including Static CHPIDS If less than 2 then it will permanently assign Managed channels to ensure there are 2 In out case since we only have 1 static CHPID assigned to each CU Group, it gave 1 Managed CHIPD Drove Channel load to 8 Devices in CU Group 1 over 2 CHPIDS (B8 and B9) ~ 61% Busy each We utilized CHDRVR internal tool to drive I/O to the devices via job JPBURGC8 with DCM disabled Once DCM enabled, the load caused the Free Managed Channel (BA) to be moved to CU Group 1 Evidenced by messages IEE303I and IEE302I messages We saw the individual Vary Paths (OFF) to each address first (from the Source CU Group) Followed by the Individual Vary Paths (ON) to each address next (to the Target CU Group) With the 3 rd Channel coming over, the Channels Utilization dropped to ~56% Busy each but now 3 of them Overall Channel Load increased ~37% This led to A decrease in I/O response time and an increase in I/Os per Second ~36% For JPBURGC8 an increase in throughput and a decrease in /EXCP 25

26 z/os 1.11 DCM Channel Busy DCM ON SMF 73 Channel Busy 26

27 z/os 1.11 DCM I/O Response and Rate Job Improvements DCM ON SMF 74 DASD Subsytem I/O Response SMF 74 DASD Subsytem I/O Rate JPBURGC8 DCM? CPSEC EXCPS / EXCPs DASD I/Os per sec No , ,720 Yes , ,339 DCM Improvement ==> JPBURGC8 SMF 30 Characteristics 27

28 z/os 1.11 DCM - MF Characteristics SYSID MON DAY SH Hour CPI PRBSTATE L1MP L15P L2LP L2RP MEMP LPAR DCM SYSC FEB 17 P OFF SYSC FEB 17 P OFF SYSC FEB 17 P OFF SYSC FEB 17 P OFF DCM OFF OFF SYSC FEB 17 P ON SYSC FEB 17 P ON SYSC FEB 17 P ON SYSC FEB 17 P ON SYSC FEB 17 P ON SYSC FEB 17 P ON SYSC FEB 17 P ON DCM ON ON With DCM ON CPI is down and LPAR is up (due to load), indicative of improvement Problem State is up, indicative of LPAR Busy increase Conclusion: The best performance view is at Job Level from SMF 30 statistics 28

29 Summary With DCM=ON The I/O driver job saw a 36% throughput increase from traditional SMF 30 statistics Along with a reduced /EXCP (-3%) and improved DASD I/O response times MF provided a secondary view of the job improvement environment DCM can help: Simplify I/O configuration planning and definition Enhance availability See Techdoc WP for more information about DCM 29

30 Future Direction Workload Characterization 30

31 Workload Characterization Future Vision Future vision is to potentially combine SMF 23s and SMF113s To help identify workload characteristics and to provide better input for capacity planning and performance 80 Customer/Partitions thru 3/12 Workload Characterization SMF 23s SMF 113s Looking for Volunteers (2 days, 24 hours/day, SMF 23s, 70s, 71s, 72s, 113s per LPAR) If interested send note to No deliverable will be returned Benefit: Opportunity to ensure your data is used to influence clustering analysis 31

32 z10 Measurement Facility COUNTERS Summary Traditional metrics continue to provide the best view of Performance MF can help explain why a change occurred In the future MF may help characterize workloads for Capacity Sizing MF has a very low overhead to run and is easy to implement Less than 1/100 of a second for HIS address space in 15 minute interval Customers are successfully running MF in Production Today Volunteers are still needed for our Workload Characterization study Feedback from Volunteers is this is very easy to enable, with a minimal time investment 32

33 Acknowledgements Many people contributed to this presentation including: Riaz Ahmad Greg Boyd Harv Emery Gary King Frank Kyne Dennis Ng Steve Olenik Dale Riedy Bill Rooney Brian Smith Bob St John Kathy Walsh 33

34 Thank You for attending! 34

35 Appendix 35

36 Documentation MVS Commands SA Setting up hardware event data collection 1-39 The Set-Program-Parameter and -Measurement Facilities SA Full description of Basic, Problem-State and Crypto Counter Sets IBM The -Measurement Facility Extended Counters Definition for z10 SA Full description of Extended Counter Set WSC Short Stories and Tall Tales SHARE Summer 2009 Denver - Session 2136 John Burg Techdoc in late 1QT 2010 This presentation and a detailed write up for enabling MF John Burg ITSO Red Book reference Planned for 4QT 2010 Exploiting System z LPAR Capacity Controls - SG Part Book: Part 1 - MF Part 2 - HiperDispatch, Group Capacity Controls, hard/soft capping Draft available ~April

37 MF Update Lessons Learned since March 2009 L1 Miss % can be determined from MF COUNTERS z10 EC must be at bundle #20 or higher for MF COUNTERS IRD considerations If goes offline, only activity within internal is recorded in an Intermediate record, then If no activity in follow on 15 minute interval(s), Intermediate record is not cut for the ID No Final record when HIS is ended When activity resumes, Intermediate record is written for ID New APAR OA27623 to add Speed to SMF 113 and to HIS COUNTERS output Processor speed for which the hardware event counters are recorded. Speed is in cycles / microsecond for z10 EC SMF 113 new field: SMF113_2_CPSP - 4 byte binary Simplifies conversion of Cycles into Time Customers are successfully running MF COUNTERS (and collecting SMF 113s) over 24 hours Analyze the major LPARs on a z10 at the same time 37

38 APAR OA27623 Speed HIS COUNTERS Output These numbers come from a synthetic Benchmark and do not represent a production workload 38

39 How it works Hardware Instrumentation Counters HW 0 1 n Modify HIS,Begin OS <15 min intervals> Modify HIS,End.CNT Delta counters, (End- Begin), to.cnt file One file for all s SMF 113s Raw counters to SMF One record per 39

40 What data is in the MF per Logical CP Basic Counters (and Problem) per (1) Cycles Instructions L1 Cache Sourcing basic information Crypto Counters per (1) Counts and Cycles by Crypto function Extended Counters per (Model Dependent) (2) z10 L1 Cache Hierarchy Sourcing Cache Hierarchy Information and more z10 L1 Sourcing detailed information 1 - See The Set-Program-Parameter and -Measurement Facilities SA for full description 2 - See IBM The -Measurement Facility Extended Counters Definition for z10 SA for full description See Appendix for Basic, Problem and Crypto Counters 40

41 What we did Set up MF on WSC z10 and z/os 1.10 Started/Modified HIS and collected SMF 113s and *.CNT Data Ran COUNTERS mode, COUNTERS=ALL (Basic, Problem, Crypto, Extended) via: F HIS,B,TT='EncrypCounters2',PATH='/his/',CTRONLY,CTR=ALL Ran DASD dumps DASD dumps sequentially over 20 minute duration With option: ENCRYPT(CLRTDES) - Built sample reports with a REXX exec Used *.CNT output to as input Validated with SMF 113s Reports Basic Counters Basic / Extended Counters - z10 L1 Cache Hierarchy Sourcing Report Crypto Counters 41

42 SYSHISyyyymmdd,hhmmss.CNT Output unformatted HIS019I EVENT COUNTERS INFORMATION FILE NAME: SYSHIS CNT COMMAND: MODIFY HIS,B,TT='EncrypCounters2',PATH='/his/',CTRONLY,CTR=ALL COUNTER VERSION NUMBER 1: 1 COUNTER VERSION NUMBER 2: 1 COUNTER SET= BASIC COUNTER IDENTIFIERS: 0: CYCLE COUNT 1: INSTRUCTION COUNT 2: L1 I-CACHE DIRECTORY-WRITE COUNT 3: L1 I-CACHE PENALTY CYCLE COUNT 4: L1 D-CACHE DIRECTORY-WRITE COUNT 5: L1 D-CACHE PENALTY CYCLE COUNT START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7AD83D26 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FC45A5 COUNTER VALUES (HEXADECIMAL) FOR 00: BEBF CF B57E0D A620C B25C43DBC Description Start / End time Counters per - 00 START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7AD95826 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FD3625 COUNTER VALUES (HEXADECIMAL) FOR 01: CFB22F1D D23D49A D89E B662EA C1F067B B8087F6A Counters per - 01 START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7ADABCA6 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FE1525 COUNTER VALUES (HEXADECIMAL) FOR 04: DE76A A8F16E5E AC8F B92F07B E926CFD Counters per - 04 COUNTER SET= PROBLEM-STATE COUNTER IDENTIFIERS: 32: PROBLEM-STATE CYCLE COUNT 33: PROBLEM-STATE INSTRUCTION COUNT 34: PROBLEM-STATE L1 I-CACHE DIRECTORY-WRITE COUNT 35: PROBLEM-STATE L1 I-CACHE PENALTY CYCLE COUNT 36: PROBLEM-STATE L1 D-CACHE DIRECTORY-WRITE COUNT 37: PROBLEM-STATE L1 D-CACHE PENALTY CYCLE COUNT 42

43 Sample Report Basic Counters Normalized Basic Counters to per Second L1 Index and Directory Write Counts used In Cache Hierarchy Sourcing L1 Miss % can be derived from MF information Instruction Count is the base. If instructions are not in z10 L1 Cache, then they must be Sourced from the z10 hierarchy. The Total Sourced is the Total Write Count, the Misses L1 Miss % = Directory Write Counts (I+D) / Instruction Counts 3.2% = (580, ,572,649.35) / 68,15, These numbers come from a synthetic Benchmark and do not represent a production workload 43

44 Sample Report Basic / Extended Counters z10 L1 Cache Hierarchy Sourcing Various Sources from Extended Counters Total L1 Sourcing from Basic Counters These numbers come from a synthetic Benchmark and do not represent a production workload 44

45 MF and HIS provide a z/os logical view of z10 Resource Usage and Cache Hierarchy Sourcing L 1.5 L 1 Memory L2 Cache L 1.5 L 1 HIS LP0 LP1 LP4 L 1.5 L 1 Book PR/SM MF L 1.5 L 1 Memory L2 Cache L 1.5 L 1 L 1.5 L 1 LPAR / Logical CP view: Memory.27% Cache L 2 (local and remote) 20.92% L % L1 Sourced from above Hierarchy 2.15 Million / Sec 3.2% L1 Miss % 96.8% L1 Hit % z/os LPAR These numbers come from a synthetic Benchmark and do not represent a production workload 45

46 Sample Report Crypto Counters This information may be useful in determining: When and What encryption function is occurring (Count)? How many cycles are being used? The encryption facility executed both SHA functions and TDES functions for this specific test. Since MF is new, this information is not available from RMF today Need to analyze more Customer data These numbers come from a synthetic Benchmark and do not represent a production workload 46

47 Image Profile Security Customization for HIS 47

48 Counter Data Basic Counter Set Cycle count Instruction count Level-1 I-cache directory write count Level-1 I-cache penalty cycle count Level-1 D-cache directory write count Level-1 D-cache penalty cycle count Problem State Counter Set Problem state cycle count Problem state instruction count Problem state level-1 I-cache directory write count Problem state level-1 I-cache penalty cycle count Problem state level-1 D-cache directory write count Problem state level-1 D-cache penalty cycle count Extended Counter Set Number and meaning of counters are model dependant 48

49 Counter Data Crypto Activity Counter Set (CPACF activity) PRNG function count PRNG cycle count PRNG blocked function count PRNG blocked cycle count SHA function count SHA cycle count SHA blocked function count SHA blocked cycle count DES function count DES cycle count DES blocked function count DES blocked cycle count AES function count AES cycle count AES blocked function count AES blocked cycle count 49

50 SMF Record type 113, subtype 2 Layout: (SMF manual, HISYSMFR macro) Standard SMF record header ( 1C x bytes) SMF record control information TOD when SMF record is written, etc. Offset, length, and number of data sections Data section TOD when counter data was captured number Offset, length, and number of Counter Set Sections Offset, length, and number of Counter Sections Counter Set Sections Counter Set type (1=BASIC, 2=PROB, 3=CRYPTO, 4=EXT) Bit mask identifying the counters being recorded in array e.g. FC x => counters 0-5 are valid Counter Sections 8-byte counter values (contiguous) 50

51 z10 versus z9 hardware comparison z9 EC 1.7 Ghz superscalar Caches L1 private 256k i, 256k d L2 shared 40 mbs / book book interconnect: ring L1 Memory L2 Cache L1... L1 z10 EC 4.4 Ghz redesigned pipeline superscalar Caches L1 private 64k i, 128k d L1.5 private 3 mbs L2 shared 48 mbs / book book interconnect: star L1.5 L1 Memory L2 Cache L1.5 L1... L1.5 L1 51

52 DCM Requirements Software Dependencies - OA28321 All systems in LPAR cluster must be z/os 1.11 Hardware Dependencies Processor All currently supported processors (z900 and up) Channels All currently supported FICON channels Coupling Facility required if running multi-system Switches Must have control unit port (CUP) function Must be defined in the IODF Control Unit No special microcode needed 52

53 DCM Management Strategy Channel Utilization * Three ranges Good (0 25) Average (26 50) Poor (51 and above) Port I/O Intensity Three ranges Good (0 1000) Average ( ) Poor (3001 and above) * From RMF Channel Path Activity Report From RMF ESS Link Statistics Report 53

54 FICON DCM Process WLM Collect data from all systems in LPAR Cluster Return data averaged over period of time Data Collection Collect performance data Possible Possible changes changes Add Add A0 A0 to to Add Add A1 A1 to to Add Add A0 A0 to to Add Add A1 A1 to to Link Port Statistics Control Unit Information Switch Information Channel Statistics Dynamic I/O Add, replace or delete CHPID from control unit DCM Change DCM Algorithms Check for bottlenecks, performance issues Determine what changes can be made Select the best change Change the I/O configuration Dynamic I/O New 53% 53% 53% Old Managed Managed CHPID CHPID (A1) (A1) 80% Static Static CHPID CHPID Managed Managed CHPID CHPID (A0) (A0) Static Static 80% CHPID CHPID Control Control Unit Unit Control Control Unit Unit