WSC Short Stories and Tall Tales. Session IBM Advanced Technical Support. March 5, John Burg. IBM Washington Systems Center

Transcription

1 IBM Advanced Technical Support WSC Short Stories and Tall Tales Session 2536 March 5, 2009 John Burg IBM Washington Systems Center 1

2 2 Advanced Technical Support Washington Systems Center Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. AlphaBlox* APPN* CICS* CICS/VSE* Cool Blue DB2* DFSMS DFSMShsm DFSMSrmm DirMaint DRDA* DS6000 DS8000 ECKD ESCON* FICON* FlashCopy* GDPS* HiperSockets HyperSwap IBM* IBM eserver IBM logo* IMS Language Environment* Lotus* Multiprise* MVS OMEGAMON* Parallel Sysplex* Performance Toolkit for VM PowerPC* PR/SM Processor Resource/Systems Manager * Registered trademarks of IBM Corporation The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce. * All other products may be trademarks or registered trademarks of their respective companies. RACF* Redbooks* Resource Link RETAIN* REXX RMF S/390* Scalable Architecture for Financial Reporting Sysplex Timer* Systems Director Active Energy Manager System/370 System p* System Storage System x* System z System z9* System z10 Tivoli* Tivoli Storage Manager TotalStorage* VSE/ESA VTAM* WebSphere* xseries* z9* z10 z10 BC z10 EC z/architecture* z/os* z/vm* z/vse zseries* Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

3 Topics Advanced Technical Support Washington Systems Center z10 Compression Considerations SMF 23 Record Update z10 CPU Measurement Facility Update Summary Workload Characterization Future Vision 3

4 z10 Compression Considerations 4

5 Compression History Some History Compression is an instruction: CMPSC, op code B263 Performance varies depending on machine design implementation Also data content and usage Two types of Compression Generic and Tailored Generic compression uses dictionary building blocks to create a dictionary Tailored compression samples the data to be compressed and creates a fully custom dictionary from this data. The dictionary is kept with the data. Generic and Tailored compression paths may have different performance characteristics Always has been a tradeoff between DASD savings and CPU time Tailored typically has higher DASD savings, but may incur higher CPU cost 5

6 6 Advanced Technical Support Washington Systems Center LSPR Mixed workload and Compression (B263) Behavior over Time LSPR B Compression Performance Expansion Performance G G G G4(3.3ns) 9672 G4(3.1ns) 9672 G4(2.7ns) 9672 G5 (2.6ns 9672 G5(2.4ns) 9672 G5(2.0ns) 9672 G6(1.8ns) 9672G6(1.65ns 2066 (1.6ns) 2064 (1.3ns) 2064 (1.09ns) 2084 (0.8ns) 2094 (.579ns) 2097 (.227ns) Relative Performance Range G G G G4(3.3ns) 9672 G4(3.1ns) 9672 G4(2.7ns) 9672 G5 (2.6ns 9672 G5(2.4ns) 9672 G5(2.0ns) 9672 G6(1.8ns) 9672G6(1.65ns 2066 (1.6ns) 2064 (1.3ns) 2064 (1.09ns) 2084 (0.8ns) 2094 (.579ns) 2097 (.227ns) Relative Performance Range Over time Compression has kept pace with the LSPR Mixed Workload

7 Processor Hardware Cache Characteristics which affect compression z Compression dictionary is cached in standard L1 processor cache Thus interfering with all other work on the system z Each CP has a compression engine The compression engine has a 16k cache to hold dictionary references This removes the impact from other workload But will concentrate any negative impact to the compression work units Resulting cache misses can result in performance below the default mix LSPR ratio DFSMS Tailored compression tests 8k, 32k and 128k dictionary sizes for highest DASD savings The working set of the larger dictionary sizes may not totally fit in the z k cache, resulting in high cache miss rates and higher CPU costs 7

8 Identifying the size of a Tailored Dictionary for a data set LISTCAT If you do a LISTCAT of a compressed data set, you will get the dictionary token. For example: ACT-DIC-TOKEN---- X' ' The last nibble in the forth byte contains a "9" which indicates the number of bits representing a single character. The dictionary sizes are: SMF 9 = 8K A = 16K B = 32K C = 64K D = 128K Note that you will only see 9, B and D as DFSMS only builds 8K, 32K and 128K dictionaries. The dictionary token is available in SMF records. Type 14 for sequential has a field SMF14TKN Type 64 for VSAM has SMF64DTK 8

9 What we did Ran Compression Tests of z9 Vs z Vs 2097 Dedicated LPAR with 1 logical processor Only 1 data set and 1 dictionary was accessed per test The data set size in all cases was 2016 MB Tests Compression Type No compression Generic compression Tailored compression Write Vs Read 9

10 Results 2094 Vs 2097 CPU Differences Compressed Datasets Function Compressed 2094 CPU 2097 CPU ITR Ratio Compression % Write No % Write Generic % Write Tailored % Read No % Read Generic % Read Tailored % 10

11 Compression Tests Observations Tailored Compression may have a higher CPU cost than Generic when compressing the same data Thus ITR (work per CPU) of 2097 is less than 2094 for tailored than generic Differences in Generic and Tailored Dictionaries Tailored is 4x size Vs Generic Tailored is more fully populated and more heavily used, resulting in substantially more DASD space savings Differences can result in Higher cache miss rate for Tailored 2097 has separate 16k Cache and with 2094 there is no separate cache but shares L1 Tests ignore benefit to other workloads running Because of no L1 interference from compression activity going to 16k cache 11

12 Compression Considerations Summary Compression continues to be a trade off Between DASD Savings and CPU consumption Compression can influence the ITRR of jobs as you migrate to new processor technology Relative to the LSPR Mixed workload Compression is influenced by many factors Number of concurrent jobs, symbol set, data content 12

13 SMF 23 Record Update 13

14 What are SMF 23 Records? SMF 23 The SMF record New extensions to the SMF 23 record Provide information related to Dispatching, Storage and I/O Available on z/os 1.8 and above Why you d want to collect them? They may provided a way to help characterize your workload to improve your capacity planning LoIO Mix zpcr is simply an estimate of your actual workload pattern Record Size and Interval Small record bytes (258 bytes with deltas ) per System per interval Interval set in SMFPRMxx by STATUS(hhmmss) SMF[,SYNC NOSYNC] 14

15 What is in the SMF 23s? - New Fields via APAR OA22414 Storage Total Number of Getmain requests (NGR) Total Pages backed during Getmain requests (PBG) Total Number of Fixed requests for Storage below 2 GB (NFR) Total number of Frames for Fixed requests for Storage below 2 GB (PFX) Faults Total number of first reference faults (1RF) Total number of non first reference faults (NRF) I/Os Total Number of I/Os (NIO) Dispatches (Dispatch) Number of unlocked TCB Dispatches (TCB) Number of SRB Dispatches (SRB) APAR OA27161 Closed 1/19/2009 To provide delta counters for above fields Otherwise cumulative counters 15

16 What are we doing? Collecting SMF 23s from variety of customers Only 13 Customers so far 2084, 2086, 2094, 2097 Normalizing data so we can compare across customers Converted to deltas between intervals Converted to per Second Converted to per PCI used from SMF 70s LPAR busy * Machine LSPR PCI rating 16

17 SMF 23 Summary We are gathering a larger sample size before drawing conclusions More work to be done Looking for Volunteers (up to 24 hours of SMF 23s, 70s, 72s per LPAR) If interested send note to jpburg@us.ibm.com No deliverable will be returned IBM will be performing clustering analysis Goal: Identify if SMF 23s can help determine workload characterization Potential input to zpcr? Benefit: Opportunity to ensure your data is used to influence clustering analysis Recommendation Start collecting SMF 23s with the new information 17

18 z10 CPU Measurement Facility 18

19 What is the z10 CPU Measurement Facility New hardware instrumentation facility CPU Measurement Facility (CPU MF) Available on System z10 EC GA2 and z10 BC Supported by a new z/os component (Instrumentation), Hardware Instrumentation Services (HIS) Potential Future Uses for this new cool virtualization technology CPU MF provides support built into the processor hardware So exploiting mechanism allows the observation of performance behavior with nearly no impact to the system being observed Potential Uses Future workload characterization ISV product improvement Application Tuning 19

20 High level description Data collection done by System z hardware Low overhead Little/No skew in sampling Access to information which is not available from software Information about how software and hardware interact 2 Basic Modes COUNTERS SAMPLING SAMPFREQ= is the default (samples per minute), = 13,333 / sec 8M samples in 10 minutes is the default > (DURATION=10 is the default, 10 minutes) Recommendation Start with a small frequency, e.g. SAMPFREQ=320, and increase after early experiences e.g. ensure enough disk space for output > Smaller z10 BCs should increase only up to SAMPFREQ= (for DURATION=60) New IBM Research article IBM System z10 performance improvements with software and hardware synergy 20

21 Requirements and Steps to utilize z10 CPU MF Requirements for CPU MF System z10 machine must be at GA2 Driver 76D Bundle #11 or higher z10 BC sub-capacity models must be at Bundle #20 or higher z/os LPAR being measured must be at z/os 1.8 or higher with APARs: OA25755, OA25750, and OA25773 Not supported for z/os running as a z/vm guest Steps to utilize CPU MF 1. Configure the z10 Server to collect CPU MF Data Update LPAR Security Tabs (See appendix) 2. Configure HIS on z/os to collect CPU MF Data * Set up HIS Proc Set up OMVS Directory Collect SMF 113s via SMFPRMxx //HIS PROC //HIS EXEC PGM=HISINIT,REGION=0K,TIME=NOLIMIT //SYSPRINT DD SYSOUT=* 3. Collect CPU MF Data Start HIS Modify with Begin/End for COUNTERS or SAMPLING 4. Analyze the CPU MF Data SMF 113s or OMVS files (e.g. *.CNT, etc.) * See MVS Commands SA Setting up hardware event data collection

22 How it works Hardware Instrumentation Counters HW CPU 0 CPU 1 CPU n Modify HIS,Begin OS <15 min intervals> Modify HIS,End.CNT Delta counters, (End- Begin), to.cnt file One file for all CPUs SMF 113s Raw counters to SMF One record per CPU 22

23 What data is in the CPU MF per Logical CP Basic Counters (and Problem) per CPU (1) Cycles Instructions L1 Cache Sourcing basic information Crypto Counters per CPU (1) Counts and Cycles by Crypto function Extended Counters per CPU (Model Dependent) (2) Cache Hierarchy Information and more z10 L1 Sourcing detailed information z10 L1 Cache Hierarchy Sourcing 1 - See The Set-Program-Parameter and CPU-Measurement Facilities SA for full description 2 - See IBM The CPU-Measurement Facility Extended Counters Definition for z10 SA for full description See Appendix for Basic, Problem and Crypto Counters 23

24 CPU MF and HIS provide a z/os logical view of z10 Resource Usage and Cache Hierarchy Sourcing L 1.5 L 1 CPU Memory L2 Cache L 1.5 L 1 CPU HIS LP0 LP1 LP4 z/os LPAR L 1.5 L 1 CPU Book PR/SM CPU MF L 1.5 L 1 CPU Memory L2 Cache L 1.5 L 1 CPU L 1.5 L 1 CPU LPAR / Logical CP view: Memory Accesses Cache L 2 Accesses L1.5 Accesses L1 Sourced from Hierarchy Instructions and Cycles Crypto function 24

25 What we did Set up CPU MF on WSC z10 and z/os 1.10 Started/Modified HIS and collected SMF 113s and *.CNT Data Ran COUNTERS mode, COUNTERS=ALL (Basic, Problem, Crypto, Extended) via: F HIS,B,TT='EncrypCounters2',PATH='/his/',CTRONLY,CTR=ALL Ran DASD dumps DASD dumps sequentially over 20 minute duration With option: ENCRYPT(CLRTDES) - Built sample reports with a REXX exec Used *.CNT output to as input Validated with SMF 113s Reports Basic Counters Crypto Counters Basic / Extended Counters - z10 L1 Cache Hierarchy Sourcing Report 25

26 SYSHISyyyymmdd,hhmmss.CNT Output unformatted HIS019I EVENT COUNTERS INFORMATION FILE NAME: SYSHIS CNT COMMAND: MODIFY HIS,B,TT='EncrypCounters2',PATH='/his/',CTRONLY,CTR=ALL COUNTER VERSION NUMBER 1: 1 COUNTER VERSION NUMBER 2: 1 COUNTER SET= BASIC COUNTER IDENTIFIERS: 0: CYCLE COUNT 1: INSTRUCTION COUNT 2: L1 I-CACHE DIRECTORY-WRITE COUNT 3: L1 I-CACHE PENALTY CYCLE COUNT 4: L1 D-CACHE DIRECTORY-WRITE COUNT 5: L1 D-CACHE PENALTY CYCLE COUNT START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7AD83D26 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FC45A5 COUNTER VALUES (HEXADECIMAL) FOR CPU 00: BEBF CF B57E0D A620C B25C43DBC Description Start / End time Counters per CPU - 00 START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7AD95826 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FD3625 COUNTER VALUES (HEXADECIMAL) FOR CPU 01: CFB22F1D D23D49A D89E B662EA C1F067B B8087F6A Counters per CPU - 01 START TIME: 2009/02/07 16:11:02 START TOD: C3B6ADBE7ADABCA6 END TIME: 2009/02/07 16:31:19 END TOD: C3B6B24700FE1525 COUNTER VALUES (HEXADECIMAL) FOR CPU 04: DE76A A8F16E5E AC8F B92F07B E926CFD Counters per CPU - 04 COUNTER SET= PROBLEM-STATE COUNTER IDENTIFIERS: 32: PROBLEM-STATE CYCLE COUNT 33: PROBLEM-STATE INSTRUCTION COUNT 34: PROBLEM-STATE L1 I-CACHE DIRECTORY-WRITE COUNT 35: PROBLEM-STATE L1 I-CACHE PENALTY CYCLE COUNT 36: PROBLEM-STATE L1 D-CACHE DIRECTORY-WRITE COUNT 37: PROBLEM-STATE L1 D-CACHE PENALTY CYCLE COUNT 26

27 Sample Report Basic Counters Normalized Basic Counters to per Second L1 Index and Directory Write Counts used In Cache Hierarchy Sourcing These numbers come from a synthetic Benchmark and do not represent a production workload 27

28 Sample Report Crypto Counters This information may be useful in determining: When and What encryption function is occurring (Count)? How many cycles are being used? Since CPU MF is new, this information is not available from RMF today The encryption facility executed both SHA functions and TDES functions for this specific test. These numbers come from a synthetic Benchmark and do not represent a production workload 28

29 Sample Report Basic / Extended Counters z10 L1 Cache Hierarchy Sourcing Various Sources from Extended Counters Total L1 Sourcing from Basic Counters These numbers come from a synthetic Benchmark and do not represent a production workload 29

30 CPU MF and HIS provide a z/os logical view of z10 Resource Usage and Cache Hierarchy Sourcing L 1.5 L 1 CPU Memory L2 Cache L 1.5 L 1 CPU HIS LP0 LP1 LP4 L 1.5 L 1 CPU Book PR/SM CPU MF L 1.5 L 1 CPU Memory L2 Cache L 1.5 L 1 CPU L 1.5 L 1 CPU LPAR / Logical CP view: Memory.27% Cache L % L % L1 Sourced from above Hierarchy 2.15 Million / Sec z/os LPAR These numbers come from a synthetic Benchmark and do not represent a production workload 30

31 z10 CPU Measurement Facility Summary COUNTERS mode took very low overhead to run Less than 1/100 of a second for HIS address space in 15 minute interval CTRONLY option ensures that delta counters are provided in the *.CNT output Useful for determining rates from the Start and End times CPU MF provides unique z10 performance measurement capabilities Future potential uses may include Workload characterization, ISV product improvement, and application tuning 31

32 Summary - Workload Characterization Future Vision Future vision is to potentially combine SMF 23s and SMF113s To help identify workload characteristics and to provide better input for capacity planning and performance Workload Characterization SMF 23s SMF 113s Looking for Volunteers (up to 24 hours of SMF 23s, 70s, 72s, 113s *.CNT per LPAR) If interested send note to jpburg@us.ibm.com No deliverable will be returned Benefit: Opportunity to ensure your data is used to influence clustering analysis 32

33 Acknowledgements Many people contributed to this presentation including: Riaz Ahmad Mary Astley Greg Boyd Harv Emery Gary King Marty Moroch Steve Olenik Bob St John Debbie St. Onge Kathy Walsh 33

34 Thank You for attending! 34

35 Appendix 35

36 Image Profile Security Customization for HIS 36

37 Counter Data Basic Counter Set Cycle count Instruction count Level-1 I-cache directory write count Level-1 I-cache penalty cycle count Level-1 D-cache directory write count Level-1 D-cache penalty cycle count Problem State Counter Set Problem state cycle count Problem state instruction count Problem state level-1 I-cache directory write count Problem state level-1 I-cache penalty cycle count Problem state level-1 D-cache directory write count Problem state level-1 D-cache penalty cycle count Extended Counter Set Number and meaning of counters are model dependant 37

38 Counter Data Crypto Activity Counter Set (CPACF activity) PRNG function count PRNG cycle count PRNG blocked function count PRNG blocked cycle count SHA function count SHA cycle count SHA blocked function count SHA blocked cycle count DES function count DES cycle count DES blocked function count DES blocked cycle count AES function count AES cycle count AES blocked function count AES blocked cycle count 38

39 SMF Record type 113, subtype 2 Layout: (SMF manual, HISYSMFR macro) Standard SMF record header ( 1C x bytes) SMF record control information TOD when SMF record is written, etc. Offset, length, and number of data sections Data section TOD when counter data was captured CPU number Offset, length, and number of Counter Set Sections Offset, length, and number of Counter Sections Counter Set Sections Counter Set type (1=BASIC, 2=PROB, 3=CRYPTO, 4=EXT) Bit mask identifying the counters being recorded in array e.g. FC x => counters 0-5 are valid Counter Sections 8-byte counter values (contiguous) 39

40 z10 versus z9 hardware comparison z9 EC CPU 1.7 Ghz superscalar Caches L1 private 256k i, 256k d L2 shared 40 mbs / book book interconnect: ring L1 CPU Memory L2 Cache L1 CPU... L1 CPU z10 EC CPU 4.4 Ghz redesigned pipeline superscalar Caches L1 private 64k i, 128k d L1.5 private 3 mbs L2 shared 48 mbs / book book interconnect: star L1.5 L1 CPU Memory L2 Cache L1.5 L1 CPU... L1.5 L1 CPU 40