z/architecture Overview and z/vm 64-bit Support Internals

Transcription

1 z/architecture Overview and z/vm 64-bit Support Internals Damian Osisek IBM z/vm Development BayBunch May 30, 2003

2 Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. e-business logo* Enterprise Systems Architecure/390 ESCON* FICON IBM* IBM eserver IBM logo* Multiprise* S/390* VM/ESA* z/architecture z/os z/vm* zseries* * Registered trademarks of IBM Corporation The following are trademarks or registered trademarks of other companies. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both Penguin (Tux) compliments of Larry Ewing Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries UNIX is a registered trademark of The Open Group in the United States and other countries. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notic Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibili or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. 64BIT_TM1

3 Agenda Introduction 64-bit Architecture Implications for VM VM 64-bit Support Design Goals Constraints Implementation Restrictions Related Changes Summary 64BIT_0010

4 64-bit Architecture Architecture mode selected by software ESA/390 z/architecture z/architecture major characteristics 64-bit registers 64-bit addresses and addressing mode New instructions Five-level DAT structure 128-bit PSWs Two-page prefix area 64BIT_0020

5 64-bit Registers bit GRs 64-bit addressing 64-bit integer arithmetic bit CRs Few architectural changes from ESA/390 Some 64-bit exploitation 1, 7, 10, 11, 12, 13 and 15 support 64-bit addresses 64BIT_0030

6 64-bit Addresses Larger real and virtual storage sizes Gigabytes (GB) = 2**30 Terabytes (TB) = 2**40 Petabytes (PB) = 2**50 Exabytes (EB) = 2**60 Additional addressing mode (64-bit) 64BIT_0040

7 Instruction set New instructions To deal with new architectural components AGR, LPSWE To convert between old and new formats LLGTR, SLGF, STMH To do things we've always wanted RLLG, OILL ESA/390 instructions continue to operate Some are sensitive to new 64-bit addressing mode LA Some new 32-bit instructions RLL 64BIT_0050

8 Sample new instructions Instruction Syntax Result Load (32) L 1,FIELD uuuuuuuu FEDCBA98 Load (64) LG 1,FIELD FEDCBA Load (64 from 32) LGF 1,FIELD FFFFFFFF FEDCBA98 Load Logical (64 from 32) LLGF 1,FIELD FEDCBA98 Load Logical Thirty-one Bits LLGT 1,FIELD EDCBA98 Insert Characters under Mask (high) uuuuuuuu = unchanged FIELD = FEDCBA ICMH 1,X'F',FIELD Load Multiple (32) LM 1,2,FIELD FEDCBA98 uuuuuuuu uuuuuuuu FEDCBA98 uuuuuuuu Load Multiple (64) LMG 1,1,FIELD FEDCBA Load Multiple High LMH 1,2,FIELD FEDCBA98 uuuuuuuu uuuuuuuu 64BIT_0060

9 Five-Level DAT Structure Region Table Entry (RTE) 64-bit entry holding Region-Table Origin or Segment-Table Origin Region Table One to four pages (512 to 2048 entries) Region First Table Each entry points to Region Second Table Region Second Table Each entry points to Region Third Table Region Third Table Each entry points to Segment Table 64BIT_0070

10 Five-Level DAT Structure (continued) Segment-Table Entry (STE) 64-bit entry holding Page-Table Origin Page Table is 256 entries (maps 1M) Page-Table Entry (PTE) 64-bit entry holding Page-Frame Real Address 64-bit virtual address structure: RFX RSX RTX SX PX BX (11) (11) (11) (11) (8) (12) BIT_0080

11 Five-Level DAT Structure (continued) Address-Space-Control Element (ASCE) Generic name for 64-bit designator; may be: Segment-Table Designation (STD) Region-Table Designation (RTD) -- First, Second, or Third Real-Space Designation (RSD) Segment Table maps M segments (one region, 2G) Region Third Table maps 2048 regions (2K x 2G = 4T) Region Second Table maps 2048 RTTs (4M x 2G = 8P) Region First Table maps 2048 RSTs (8G x 2G = 16E) Self-distinguishing (type) and self-defining (length) 64BIT_0090

12 128-bit PSWs High-order word largely unchanged from ESA/390 Bit 12: must be 0 (cf. must be 1 in ESA/390) Bit 31: Extended-Addressing mode bit Second word contains Basic-Addressing mode bit Third and fourth words contain 64-bit instruction address Addressing modes: PSW.31 PSW.32 Addressing mode (EA) (BA) bit bit 1 0 invalid bit 64BIT_0100

13 128-Bit Program Status Word IE Prog E 0R000TOX Key 0MWPASCCMask A B A Instruction Address Instruction Address (continued) BIT_0110

14 Two-Page Prefix Area Prefix Area doubled in size Two frames (8K) on 8K boundary Logout area moved to second frame Made room for expanded interruption old/new PSWs Separate locations for ESA/390 and z/architecture PSWs Note: Software needs base register to address second frame 64BIT_0120

15 Implications for VM Guest can switch architecture dynamically No MACHINE statement support Prepare in advance for potential guest z/architecture switch Guests need 64-bit real and 64-bit virtual storage Major upheaval in storage management Architectural concepts must be reflected in VM command language Concern for compatibility (e.g., of scripted use) 64BIT_0130

16 Implications for VM (continued) A single image could be huge 256 system-owned devices probably inadequate A single guest could be huge Scheduler algorithms may require some attention Need to support more than 14 CPUs Need to continue to deliver support for ESA/390 architecture processors 64BIT_0140

17 VM 64-bit Support Design Goals Constraints Implementation Restrictions Related Changes 64BIT_0150

18 Design Goals Add support for z/architecture-mode guests Minimize change to CP Run on both ESA/390 and z/architecture hardware Maintain compatibility where appropriate Maintain performance parity where feasible Eliminate deprecated function where feasible Deliver with hardware 64BIT_0160

19 Constraints IBM eserver Small team Some large real exploitation deferred Other features and functions required Aggressive schedule Tied to hardware Needed early hardware to test 64BIT_0170

20 Implementation Significant changes in Storage management Guest simulation Virtual SIE (vsie) Minidisk Cache TRACE DISPLAY/DUMP/STORE I/O Dump format Major control blocks 64BIT_0180

21 Implementation (continued) Little or no change in Spool XC-mode support Most console functions Device support Inter-machine communication User Directory Most CP APIs (still 31-bit) 64BIT_0190

22 Implementation (continued) Most CP code relies only on ESA/390 functionality Selected modules exploit z/architecture capabilities 64-bit registers and instructions Storage above 2G 64-bit addressing mode Linkage changed to facilitate different register use styles Two builds, one source base Conditional assembly Variable assembly Macro encapsulation Module clones Build-specific modules 64BIT_0200

23 Major Control Block Changes Significant changes to PFXPG VMDBK PGMBK SIEBK SAVBK Storage-related architectural structures 64BIT_0210

24 z/architecture Exploitation Storage above 2G used for Minidisk cache Pageable guest pages VDISK pages (as of z/vm 4.4.0) Not used for CP free storage CP pages Preferred guest pages 65BIT_0220

25 z/architecture Exploitation (continued) Some use of 64-bit addressing mode 64-bit registers used fairly extensively Architectural level set allows use of Immediate and Relative instructions in both builds zseries 9672 G5-G6 Multiprise 3000 NOT 3090, 4381, 9221, 9121, 9021, 9672 R1/R2/R3/G3/G4, MP2000, P/390, R/390 64BIT_0230

26 Above-2G Storage Management Page faults resolved using above-2g frames if available CP translation services (HCPTRANS) move page below 2G line to return 31-bit address All locked pages must reside below 2G All CP data structures (including DAT tables, PGMBKs) must reside below 2G "Low" and "high" storage managed separately based on demand When storage below 2G constrained: Existing mechanisms would page out to expanded storage first (if available), then to DASD If Xstore constrained or not available, pages are migrated from below to above 2G (new in z/vm 4.3.0) If PGMBKs consume excessive storage below 2G, above-2g steal task driven to increase chance that PGMBKs can be paged out (new in 4.3.0) 64BIT_0240

27 Module Linkage Module register use style may be of three types: SHORTREG - 32-bit only LONGREG - 32-bit and 64-bit FULLREG - build-dependent Caller's registers must be saved/restored across call Callee has access to caller's registers in SAVBK Linkage must preserve caller's registers in callee-sensitive fashion 64BIT_0250

28 Module Linkage (continued) Module generally uses one register style Declared in MDLATENT macro for module or entry point Most modules are SHORTREG (default) Might have entry points that use other styles E.g., HCPSTK is SHORTREG, but entry points HCPSTKCG and HCPSTKFG are LONGREG In this case Must declare module register style Assume majority of module runs in declared register style Need to exercise some care 64BIT_0260

29 Module Linkage (continued) 64-bit build SAVBK doubled in size to accommodate changes Flags in SAVBK define its format New SVGBK maps save area for FULLREG and LONGREG modules Linkage macros require SAVBK/SVGBK Three cases to consider (in 64-bit build): SHORTREG => SHORTREG LONGREG => SHORTREG anyreg => LONGREG (or FULLREG) 64BIT_0270

30 SHORTREG => SHORTREG SAVBK Header Low-order Registers Workarea SAVELARG X'80' Reserved X'100' 64BIT_0280

31 LONGREG => SHORTREG SAVBK Header Low-order Registers Workarea Reserved High-order Registers X'80' X'C0' SAVELARG+ SAVERG64 X'100' 64BIT_0290

32 AnyREG => LONGREG SVGBK Header 64-bit Registers Reserved SVGLARG+ SVGCREG+ SVGRG64 Reserved Workarea X'100' 64BIT_0300

33 Conditional Assembly New HCPAIF, HCPAELSE, HCPAEND macros Global HCPOPTNS symbols define target build &HCPES390-1 for 32-bit, 0 for 64-bit &HCPESAME - 0 for 32-bit, 1 for 64-bit New maclib holds 64-bit build HCPOPTNS HCPAIF (&HCPESAME) STMG R0,R5,PFXTRCSV Save registers used by trace HCPAELSE (&HCPES390) STM R0,R5,PFXTRCSV Save registers used by trace HCPAEND 64BIT_0310

34 Conditional Assembly (continued) Listing for either build shows all source code HCPAIF (&HCPESAME) 0000D4 EB05 0C C STMG R0,R5,PFXTRCSV Save HCPAELSE (&HCPES390) * Start of Excluded Block STM R0,R5,PFXTRCSV Save HCPAEND * End of Excluded Block BIT_0320

35 Variable Assembly Designed primarily for FULLREG modules Small set of variable symbols defined in HCPFULSY COPY Allow build-dependent symbols to be created &G &G SETC 'G' (64-bit) - or - SETC '' (32-bit) L&G R1,WORKADDR 64BIT_0330

36 Variable Assembly (continued) Some use of &HCPES390/&HCPESAME in open code and macros TM SAVEFORM,SAVELARG BC 1*&HCPES390+8*&HCPESAME,DIE LA R5,GPRSIZE(,R3) Next reg... GPRSIZE EQU 4*&HCPES390+8*&HCPESAME AIF (NOT &HCPESAME).NOTME....NOTME ANOP, 64BIT_0340

37 Macro Encapsulation Many macros generate build-dependent code Often parameter-driven New parameter notations introduced (e.g., for pointer to 64-bit value) Some new macros introduced to ease build dependencies E.g., HCPGREG General-purpose macro to locate, extract, or set virtual machine general register contents Accommodates guests with 32-bit or 64-bit registers Used extensively in Diagnose support 64BIT_0350

38 Module Clones Some functions are architecturally different in fundamental ways between two builds E.g., address translation Rather than use conditional assembly, created parallel modules ("clones") Modules included in Nucleus build determined by MDLATENT NUCLEUS= parameter MDLATENT HCPPAS, MODATTR=(RES,MP,DYN), NUCLEUS=ESA390 64BIT_0360

39 Build-Specific Modules Some functions required in only one build E.g., >2G storage initialization Target nucleus specified using MDLATENT macro MDLATENT HCPISG, MODATTR=(RES,MP,STAT,LONGREG), NUCLEUS=ESAME 64BIT_0370

40 Other CP Changes for 64-bit Support New STORAGE/MAXSTORAGE statements in User Directory (and in profiles) Indirect addressing constructs on DISPLAY Page Table Invalidation Lock use improved New DEFINE STORAGE CONFIGURATION command added Eliminated IPLable CP Nucleus Eliminated RIO370 area Extensions to HCPUSING and HCPDROP Allow use of named, dependent, and ranged USINGs Exit-related variable symbols in System Config file Allow build-dependent exit file selection Monitor record extensions 64BIT_0380

41 HCPUSING/HCPDROP CP's versions of USING and DROP Extended to allow dependent, labeled, and ranged USINGs Also allow "named" DROPs Originally because of SIEBK changes but generally useful 64BIT_0390

42 Dependent USINGs HCPUSING VMDBK,R11 HCPUSING SIEBK,VMDSDSC,VMDBK TM SIEPSW1,PSWWAIT HCPDROP SIEBK... SIEBK DSECT... VMDBK DSECT... VMDSDSC DS CL BIT_0400

43 Labeled USINGs NEW HCPUSING VMDBK,R11 HCPUSING VMDBK,R7 CLC VMDUSER,NEW.VMDUSER L R7,NEW.VMDCYCLE HCPDROP NEW 64BIT_0410

44 Ranged USINGs HCPUSING (CACXSIDE,CACYSIDE),R7... CACBK DSECT... CACXSIDE DS 0X... CACYSIDE DS 0X... 64BIT_0420

45 Restrictions IBM eserver No DCSS or NSS above 2G Most Diagnose functions not supported in guest 64-bit addressing mode; operands must be below 2G in guest storage. Exceptions: Diagnose 8, 44, 4C, DC (as of z/vm 4.3.0) New 64-bit version of page-fault handshaking (PFAULT), exploited by Linux Diagnose 4 supports host addresses above 2G CP and preferred guests must fit in 2G Preferred guests may not exceed 2G-x total <<2G concurrently locked pages 64BIT_0430

46 Summary IBM eserver CP in z/vm has evolved to accommodate z/architecture Hosts Guests There is much more to do Your requirements help us select and prioritize our efforts 64BIT_0440