z/os Communications Server TCP/IP VIPA (Virtual IP Address) Trademarks

Transcription

1 Communications Server TCP/IP VIPA (Virtual IP Address) Linda Harrison IBM Advanced Technical Skills Trademarks The following are Registered Trademarks of the International Business Machines Corporation in the United States and/or other countries. IBM The following are trademarks or registered trademarks of other companies. Microsoft is a registered trademark of Microsoft Corporation in the United States and other countries. All other products may be trademarks or registered trademarks of their respective companies. Refer to for further legal information. Page 2 1

2 Agenda Introduction Inbound Connections Without VIPA Two Types of VIPA and Routing Background VIPA Requirements Which VIPA to Use DVIPA Recoveries and Failovers Source VIPA DVIPA Details DVIPA Route and Alternate Distribution Options Sysplex Distributor Distribution Options Sysplex Distributor TN3270 LU Support Fast Response Cache Acclerator (FRCA) QDIO Acclerator Multi-tier Applications and non- Targets Sysplex Autonomics Sysplex-Wide Security Associations (SWSA) Syntax, Commands, and References VIPA Syntax VIPA Commands Documents Appendices Appendix A: Sysplex and Subplex Support Appendix B: Details of Static VIPA Backup and Load Balance Appendix C: Details of DVIPA Backup and Load Balance Page 3 Introduction: Inbound Connections Without VIPA Page 4 2

3 Original with Inbound Connections Router TCP/IP is single point of failure Inbound Connection Request Remote Client and Server Remote client sends in a connection request. Source in IP Header = Remote client IP address Destination in IP Header = server IP address Response is sent back to the client. Source in IP Header = server IP address Destination in IP Header = Remote client IP address Destination IP Address Connection Request Source= and Destination= Response Source= and Destination= All Inbound packets are received over the destination (unless failure). An outage may cause a connection drop. For a TCP connection, the same client/server IP addresses are used for the life of the connection. TCP/IP Routing Table is used to determine which the outbound packets are sent over. Static Routing and OSPF support multiple concurrent parallel routes if IPCONFIG MULTIPATH is configured. Policy Agent and NetAccess can effect the decision. Page 5 QDIO Gratuitous ARP Fail-over Router TCP/IP Router Router TCP/IP Router backup capability. Connections are not dropped. When devices are started by the TCP/IP stack the stack determines if there is already a parallel connection to the same network. If at a later time one of the connections goes down the other will "take-over" for it. The other will send out a gratuitous ARP with the IP address of the failed and "own" the IP address of the failed until its recovery. Note that this does require that both s did originally come up so that the stack marked them as parallel to the same network. If one of the s fails (any failure that causes the LINK to go down), then QDIO Gratuitous ARP Fail-over occurs. The failed IP address is taken over by one of the working s. A gratuitous ARP will be sent out to associate that IP address with the working s MAC (Media Access Card) address. QDIO Gratuitous ARP Fail-over does not require any configuration. The same failover support exists for IPv6 where gratuitous neighbor advertisements are sent rather than gratuitous ARPs. Page 6 3

4 Introduction: Two Types of VIPA Page 7 Static VIPA for Inbound Connections TCP/IP Router Remote Client and Server Remote client sends in a connection request. Source in IP Header = Remote client IP address Destination in IP Header = server IP address Response is sent back to the client. Source in IP Header = server IP address Destination in IP Header = Remote client IP address Destination VIPA IP Address Connection Request Source= and Destination= Response Source= and Destination= Inbound packets are received over either (depending on routing protocol). An outage may NOT cause a connection drop. For a TCP connection, the same client/server IP addresses are used for the life of the connection. TCP/IP Routing Table is used to determine which the outbound packets are sent over. Static Routing and OSPF support multiple concurrent parallel routes. Policy Agent and NetAccess can effect the decision. Page 8 4

5 Dynamic VIPA (DVIPA) TCP/IP TCP/IP VIPA TCP/IP TCP/IP VIPA Router Router Router Router DVIPA dynamically moves to a different TCP/IP stack in the sysplex when the first stack fails. VIPA still addresses connection resilience but now also addresses application recovery. If an image is taken down, DVIPA backup policies can be used to define where the associated DVIPAs move to within the sysplex. DVIPA support also allows for manual movement of applications with associated DVIPA addresses. Only one stack at any point in time owns a specific DVIPA address. There is no maximum number of static VIPA interfaces, but the maximum number of dynamic VIPA interfaces is Page 9 Sysplex Distributor (Distributed DVIPA) Target Stack TCP/IP Target Stack Target Stack TCP/IP Target Stack TCP/IP Distributor Stack TCP/IP TCP/IP TCP/IP VIPA TCP/IP TCP/IP TCP/IP Distributor Stack TCP/IP VIPA Router Router A distributed DVIPA exists on several stacks, but is advertised outside the sysplex by only one stack. This stack receives all incoming connection requests and routes them to all the stacks in the distribution list for processing Router Router A Distributed DVIPA (Sysplex Distributor) is just a special kind of DVIPA. One Stack Performs Routing Functions (Distributor Stack) Owns Sysplex-Wide VIPA And Advertises To Routers Routes Connection Requests To Application Hosts (Target Stacks) All inbound packets travel through the Distributing Stack. All outbound packets from the Target Stacks flow according to the Target Stack routing table so they do not necessarily flow through the Distributor Stack. Page 10 5

6 VIPA Benefits of Sysplex System z Administration Services System Cloning Application Cloning Reduced Definitions via System Symbolics Sysplex Horizontal Growth. Discovery of new sysplex members. Dynamic connectivity via XCF connectivity. Sysplex Single System Image (SSI). Multiple images of the same application on different images may appear as a single application to end users. Balance workload across Sysplex members. Minimize application failure impact. Transparent application workload movement between images. Automatic Recovery with Automatic Restart Manager (ARM). VIPA Independence from Physical Interfaces Interface failure does not cause session failures. As long as one network interface is operational, sessions do not fail. VIPA addressing may be independent of the LAN addressing. Changes to LAN addresses may be transparent to end users because the VIPA address does not change. No DNS and firewall changes required. Page 11 Introduction: and Routing Background Page 12 6

7 Modes Mode is configured in the CHPID definition. Some cards have only one port. One port and one CHPID. Many cards have two ports. Two ports and two separate CHPIDs. Each port on a two port card is configured independently in only one Mode. New -Express3 have four ports. Two pairs of ports. Each pair has a separate CHPID. Each pair of ports on a four port card is configured together in only one Mode, independently from the other pair of ports on the card. Two Modes for LAN attachment: OSE Mode - non-qdio LCS and LSA Protocols Only s with copper LAN attachments supports SNA traffic using LSA (Link Station Architecture) protocol and TCP/IP traffic using LCS (LAN Channel Station) protocol Linux supports TCP/IP and CCL SNA traffic using LCS protocol OSD Mode - QDIO Protocol s with copper or fiber LAN attachments supports TCP/IP traffic Linux supports TCP/IP traffic Linux supports any protocol with QDIO Layer 2 configured Non-QDIO versus QDIO OSE Mode - non-qdio LCS and LSA Protocols Requires /SF (unless only used for TCP/IP and not shared between LPARs) Every IP address, including VIPA, must be manually defined. OSD Mode - QDIO Protocol Does not require /SF All IP addresses are dynamically downloaded to the from the TCP/IP stack. Any VIPA movement/changes are dynamically downloaded to the from the TCP/IP stack. See Techdocs item PRS3169 for details about defining to Communications Server. Page 13 IPCONFIG MULTIPATH Router TCP/IP Router BEGINROUTES ROUTE /24 = LNK11 MTU 1492 ROUTE /24 = LNK12 MTU 1492 ROUTE DEFAULT LNK11 MTU 1492 ROUTE DEFAULT LNK11 MTU 1492 ROUTE DEFAULT LNK12 MTU 1492 ROUTE DEFAULT LNK12 MTU 1492 ENDROUTES IPCONFIG MULTIPATH "load balances" outbound packets Static Routing and OMPROUTE OSPF support IPCONFIG MULTIPATH Default Multipath routing is per connection as opposed to per packet Failed first hop router Static Routing Dead Gateway Detection TCP connection will eventually timeout (3 to 10 minutes) and TCP will redrive the route selection algorithm and hopefully get a successful connection UDP and RAW packets are lost With Static Routing HSRP/VRRP should be used between first hop routers See for HSRP and VRRP details. Dynamic Routing detects failures OSPF defaults to 40 seconds and RIP takes up to 3 minutes Page 14 7

8 Multipath Consideration Outbound Load Balance Multipath can be configured per connection or per packet. A connection is a unique combination of source IP address, source port, destination IP address, and destination port. The per connection option is recommended because the per packet option may cause additional overhead (and network traffic) if packets are received out of sequence due to different paths through the network. However, if numerous connections appear to be coming from a single end point (ie. a firewall) then traffic will not be truly load balanced. The per packet option uses the s alternating between them evenly for outbound packets. The per connection option uses the s alternating been them evenly on a connection basis, therefore the s are all utilized but the outbound traffic is not as evenly distributed as the per packet option. In this presentation it is still indicated that the per connection option provides outbound load balance, even though the real comparison of the outbound traffic may not appear equal between the s. Additionally see per packet APAR PK Inbound Load Balance Inbound load balance is really determined by the first hop router. If the first hop router is capable of load balancing traffic across multiple s when the destination is a VIPA address, then inbound traffic will be truly load balanced. Routers have static and OSPF load balancing capability similar to outbound Multipath. See Page 15 PRIRouter, SECRouter, NONRouter All IP addresses in a TCP/IP HOME list are registered (dynamically downloaded) with the QDIO adapters. HOME changes are automatically sent to QDIO adapters. If the receives any packets with its MAC as the destination and a destination IP address that is "unknown" (meaning not an IP address in the HOME list), then does the following: If PRIRouter is defined (assuming the is started to that stack) then all "unknown" packets are sent to the PRIRouter stack. If PRIRouter is not defined (the is not started to any stack with PRIRouter)(could be that PRIRouter is coded but that connection is down due to failure or other outage) then if SECRouter is coded all "unknown" packets are sent to the SECRouter stack. If multiple SECRouters then a random (unpredictable) stack with SECRouter coded will be sent the "unknown" packets. There is no way to set the order of precedence for the secondary routers. Multiple Secondary Routers are supported on Ethernet Only If only NONRouter is defined (or any PRIRouter and SECRouter connection are down due to failure or other outage) then all "unknown" packets are discarded by the. NONROUTER is the default. Non-QDIO (OSE mode) may define PRIROUTER and SECROUTER via /SF. IPCONFIG DATAGRAMFWD PRIRouter is used when traffic is routed through the stack to another stack. Keep in mind that if one stack is used to route to other stacks IPCONFIG DATAGRAMFWD is required. V1.6+ DATAGRAMFWD not required for Sysplex Distributor. If target TCP/IP stacks only have XCF connectivity, datagram forwarding still needs to be configured on the distributor as all packets originating from the target will be forwarded to the distributor. PRIRouter and VLAN ID Packets with unknown IP addresses are passed to the router stack. If VLANID is specified then packets with unknown IP addresses are only passed to the router stack if the packets have a matching VLAN ID tag. Without VLANID Across one hardware box (System z central processor complex (CPC)), PRIROUTER can only be specified in the profile of one TCP/IP stack for the same port. With VLANID Across one hardware box (System z central processor complex (CPC)), PRIROUTER can only be specified in the profile of one TCP/IP stack for the same port for the same VLANID. PriRouter, SecRouter, NonRouter definition is ignored when VMAC parameter is defined. Recommendation: Use VMAC for shared ports rather than PRIROUTER/SECROUTER. Page 16 8

9 Considerations VIPA Limitations devices have a limit on the number of IP addresses (both IPv4 and IPv6 addresses) that can be registered to the device. The limit is dependent on the microcode level of the -Express device. This limit applies across all TCP/IP stacks that share the -Express device. When defining a large number of VIPAs, take care not to exceed this limit. If the limit is exceeded, IP addresses beyond the limit will not be registered with the -Express devices, and incoming packets with those IP addresses will not be routed to the correct stack unless that stack is designated as the primary router. RIP Routing and VIPA in Same (Sub)Network as s If using the RIP routing protocol and host route broadcasting is not supported by adjacent routers (that is, adjacent routers are unable to learn host routes), the following restrictions for VIPA addresses must be applied in order to benefit from fault tolerance support: If you use subnetting and VIPA addresses are in the same network as the physical IP addresses, the subnetwork portion of any VIPA addresses must not be the subnetwork portion of any physical IP addresses in the network. In this case, assign a new subnetwork for the VIPA address. If subnetting is not used on any physical interface, the network portion of any VIPA address must not be the network portion of any physical IP address in the network. In this case, assign a new network for the VIPA address, preferably a class C network address. If using the RIP routing protocol and host route broadcasting is supported by adjacent routers (that is, adjacent routers are able to learn host routes), the network or subnetwork portions of VIPA addresses can be the same across multiple TCP/IP stacks in the network. Spanning Tree Protocol If using a DVIPA when connecting an -Express Gigabit Ethernet QDIO device to a intelligent bridge or switch, ensure that the Spanning Tree Protocol (STP) on the intelligent bridge or switch is configured properly for DVIPA giveback and takeover operations. See the IP Configuration Guide for more details on STP problems. Port Fast Mode If using VIPA along with an intelligent bridge or switch, ensure that Port fast mode (Cisco) is enabled. This helps to decrease the amount of time the VIPA is unreachable in scenarios where there is dynamic movement of VIPA (dynamic or static). For more information, see your bridge or switch manual. Page 17 Introduction: VIPA Requirements Page 18 9

10 Base vs. Parallel Sysplex Base Parallel Sysplex Sysplex In Base Sysplex XCF signaling or messaging XCF signaling or messaging Yes Yes flows over ESCON CTCs, or ICP (Internal Coupling Facility installed No Yes Coupling Peer Channel) in CEC (COUPLExx). Structures defined No Yes In Parallel Sysplex XCF signaling or messaging DVIPA Yes Yes flows over Coupling Facility (COUPLExx). Coupling Facility is not required for Base Sysplex. Coupling Facility is required for Parallel Sysplex. Structures are not defined in a Base Sysplex. Structures are defined in COUPLExx and stored in Coupling Facility in a Parallel Sysplex. DVIPA requires Parallel Sysplex or Base Sysplex. TCP/IP only requires Parallel Sysplex coupling facility structures for Sysplex-wide Security Association (SWSA), SysplexPorts, and Explicit Bind Port Range structures (covered later in this presentation). DVIPA requires DynamicXCF. Each stack must have an IPv4 or IPv6 DYNAMICXCF address, or both. This address is used by distributing stacks to determine which destination targets are in the sysplex. When using DVIPAs, do not define an IUTSAMEH link. All Dynamic XCF Interfaces are created automatically if the DYNAMICXCF statement is configured. VIPAROUTE may be defined in the VIPADYNAMIC block to send DVIPA data traffic over an alternate IP path. XCF communications about DVIPAs, target stack server availability, etc. still use XCF communications. Page 19 Static VIPA Summary VIPA in same subnet as s? HSRP/VRRP required on first hop router? Requires PRIROUTER? Inbound load balance of s? Dynamic backup? Non-QDIO s with static routing No Yes One of the following is required: PRIROUTER must be defined on all the s. s must not be shared with other LPARs. Yes, if router supports multipath. No Non-QDIO s with dynamic routing No No One of the following is required: PRIROUTER must be defined on all the s. s must not be shared with other LPARs. Yes, with OSPF. No, with RIP. Yes QDIO s with static routing Yes Yes No Yes, if router supports multipath. Yes QDIO s with dynamic routing No No No Yes, with OSPF. No, with RIP. Yes Page 20 10

11 DVIPA Summary VIPA in same subnet as s? HSRP/VRRP required on first hop router? Requires PRIROUTER? Inbound load balance of s? Dynamic backup? Dynamic TCP/IP Stack Backup? Non-QDIO s with static routing No Yes s must be shared between the VIPA primary stack and VIPA backup stack and PRIROUTER/SECROUTER must be defined. Yes, if router supports multipath. No No Non-QDIO s with dynamic routing No No One of the following is required: PRIROUTER must be defined on all the s. s must not be shared with other LPARs. Yes, with OSPF. No, with RIP. Yes Yes QDIO s with static routing Yes Yes No Yes, if router supports multipath. Yes Yes QDIO s with dynamic routing No No No Yes, with OSPF. No, with RIP. Yes Yes Page 21 OMPROUTE OSPF Coding OMPROUTE OSPF Routing is not required but is highly recommended for VIPA. Define all the DVIPAs which the host might own. If OSPF is being used VIPAs should be defined as OSPF_Interface rather than INTERFACE. OMPROUTE supports a maximum of 255 physical IPv4 interfaces. There is no theoretical limit on how many VIPAs can be configured. Ranges of DVIPA interfaces can be defined using the subnet mask parameter on the OSPF_Interface or Interface statement. The range defined will be all the IP addresses that fall within the subnet defined by the mask and the IP address. The IP address parameter must be the subnet number of the range being defined, not a host address within that range. The following example defines a range of DVIPA addresses from to : OSPF_Interface IP_address = Name = dummy_name Subnet_mask = ; The Name parameter is not actually used for DVIPAs. The names of DVIPA interfaces are assigned dynamically by the stack when a DVIPA interface is created. Therefore, the Name field set on the Interface or OSPF_Interface statement for a DVIPA will be ignored. Optimize Performance To minimize routing table size and advertisements that have to be processed, try to put and the sysplex into a stub or totally stubby area or isolate areas with BGP or EIGRP. To mimimize OSPF adjacencies, try to avoid OMPROUTE becoming the designated router. May be unavoidable on Hipersockets LAN since no routers attached. Only use debug tracing when necessary. Use CTRACE tracing whenever possible. Consider taking XCF links out of the OSPF domain, if not used for data traffic. Define them as Interface rather than OSPF_Interface - Cuts down significantly on adjacencies. OSPF uses multicast packets Disable multicast snooping on switches with shared s attached to them. Page 22 11

12 Introduction: Which VIPA to Use Page 23 What type of VIPA to use? Static VIPA When you only want the IP address to be used on one TCP/IP stack. DVIPA When you only want the IP address to be used on one TCP/IP stack at any given time but you want the dynamic backup capability of moving the IP address to a different stack at failure. If you only ever want the application to run on one system at a time, but you need the application to backup to another system when the primary stack or app goes down, then DVIPA without Sysplex Distributor is what you need. There is no maximum for static VIPA interfaces, but the maximum number of dynamic VIPA interfaces is Sysplex Distributor When you have multiple Servers that are identical to your clients, you want all clients to use the same Server identity, but one instance of a server cannot handle the total number of clients that you have. If you want the application to run "identically" on multiple systems concurrently and have connections distributed (or load balanced) to those target systems, then Sysplex Distributor is what you need. If an application running on only one system does not have the capacity to support all the clients you want to support, and the application may run "identically" on multiple different systems, then Sysplex Distributor is what you need. Page 24 12

13 Application Specific DVIPA TCP/IP Application APPL TCP/IP TCP/IP Application APPL TCP/IP Application APPL Router Router Router Router A DVIPA may be defined such that the VIPA is not brought up by the TCP/IP stack until an application does a Bind(). This is known as an application specific DVIPA. If the application goes down, the DVIPA is deleted. If using MODDVIPA to create the DVIPA, then MODDVIPA would be needed to delete the DVIPA from the stack. The application may be restarted on a different in the Sysplex via Automatic Restart Manager (ARM), etc. At that time the application may issue a Bind() and the DVIPA would be brought up by the second TCP/IP stack. Only one stack at any point in time owns a specific DVIPA address. Page 25 Multiple Application-instance vs. Unique Application-instance Multiple Application-instance Scenario Some applications will accept client requests on any IP addr by binding to INADDR_ANY or IN6ADDR_ANY (i.e.. TN3270 & FTP). If the appl is identical on multiple images in sysplex, a DVIPA may move between stacks. The stacks in the sysplex do all the work of activating a DVIPA in the event of a failure. A Multiple Application-instance DVIPA is defined with a profile statement: VIPADYNAMIC VIPADEFINE ENDVIPADYNAMIC or VIPADYNAMIC VIPABACKUP ENDVIPADYNAMIC Backup allows ranking to determine order. VIPABACKUP does not require VIPADEFINE. The DVIPA is active when the stack starts and processes the profile statement. Multiple Application-instance DVIPA may be distributed (Sysplex Distributor) with a profile statement: VIPADYNAMIC VIPADISTRIBUTE DEFINE... ENDVIPADYNAMIC Unique Application-instance Scenario For some applications, each application instance must have a unique IP address. Each application instance is assigned a unique IP address as its DVIPA. Before defining individual DVIPAs, one or more blocks of IP addresses must be defined for these DVIPAs, and the individual DVIPAs must be defined from within the blocks. Each block should be represented as a subnet, so that a VIPARANGE statement can be defined for it. A Unique Application-instance DVIPA is defined with a profile statement: VIPADYNAMIC VIPARANGE ENDVIPADYNAMIC The DVIPA is not active though until it is started in one of three different ways. Defining a single block of VIPA IP addresses makes the definition process easier, but provides less individual control. Since the smallest subnet consists of four IP addresses, defining a unique subnet for each DVIPA wastes three other IP addresses that could have been used for DVIPAs. Page 26 13

14 Activate Unique Application-instance DVIPA Unique Application-instance DVIPA may be activated in one of three different ways: Bind() Function Call DVIPA is created when the application instance issues a bind() function call and specifies an IP address that is not active. DVIPA is deleted when the application instance closes the socket. BIND Parameter on the PORT Statement DVIPA is created when the application instance uses the BIND parameter on the PORT reservation statement. DVIPA is deleted when the application closes the socket. If the BIND parameter and an IP address are specified on the PORT reservation statement for a server application, and the application binds a socket to that port and INADDR_ANY, TCP/IP will convert the bind to be specific to the IP address specified on the PORT reservation statement. Utility MODDVIPA (which issues SIOCSVIPA or SIOCSVIPA6 ioctl()) DVIPA is created when the application instance uses the utility MODDVIPAIPA to activate the DVIPA. DVIPA is deleted when the application instance uses the utility MODDVIPAIPA to deactivate the DVIPA. MODDVIPA can be included in a JCL procedure or OMVS script to activate the DVIPA before initiating the application. As long as the same JCL or script is used to restart the application on another node in the event of a failure, the same DVIPA will be activated on the new stack. Page 27 VIPADYNAMIC VIPADEFINE, VIPARANGE bind(), or VIPARANGE MODDVIPA? As per the IP Configuration Guide: When should VIPADEFINE and VIPABACKUP be used to define a DVIPA? One or more applications bind to INADDR_ANY or IN6ADDR_ANY, and the application exist on multiple TCP/IPs. DVIPA takeover is desired. The DVIPA does not need to be deleted when the application is stopped. When should VIPARANGE and bind() or PORT BIND be used to define a DVIPA? The application cannot bind to INADDR_ANY or IN6ADDR_ANY, or TCP/IP stack initiated DVIPA takeover is not desired. For the bind(), the IP address to which the application binds can be controlled by the user. The application s first explicit bind causes the listening socket to remain for the life of the application. Automatic deletion of the DVIPA when the application is stopped is acceptable. A specific DVIPA address must be associated with a specific application. The application does not need to be APF authorized, or run under a user ID with superuser authority. When should VIPARANGE and the MODDVIPA utility (ioctl command SIOCSVIPA or SIOCSVIPA6) be used to define a DVIPA? The application cannot bind to INADDR_ANY or IN6ADDR_ANY, or TCP/IP stack initiated DVIPA takeover is not desired. The IP address to which the application binds is known but cannot be controlled by the user. Automatic deletion of the DVIPA when the application is stopped is not acceptable. The MODDVIPA utility (or application issuing ioctl command) will be run from an APF authorized library and under a userid with superuser authority. Utility can be initiated from JCL or an OMVS script. You can restrict access to the MODDVIPA (EZBXFDVP) program by defining a RACF profile under the SERVAUTH class and specifying the user IDs that are authorized to execute the ioctl or the MODDVIPA utility program. Page 28 14

15 Introduction: DVIPA Recoveries and Failovers Page 29 DVIPA Recovery TN3270A 1 TCP/IP VIPA TN3270B 2 TCP/IP TN3270A 1 TCP/IP VIPA TN3270B 2 TCP/IP VIPA TN3270A 1 TCP/IP VIPA TN3270B 2 TCP/IP VIPA Session1 Router Router Session1 Router Session2 Router Session3 Session2 Router Router 1. VIPA1 comes up on primary stack fails. All current connections are dropped. 3. VIPA1 is moved to backup stack 2. Clients can immediately log back in recovers and VIPA1 moves back to 1. New sessions are setup on 1. Data connections that are still active to 2 are routed by 1 to 2 until those connections terminate. Page 30 15

16 VIPARANGE 1 TCP/IP VIPA Appl TCP/IP TCP/IP VIPA Appl TCP/IP VIPA Appl Router Router Router Router The application causes the VIPA to be brought up on one and only one stack at a time. 1. Appl1 is brought up on Appl1 issues Bind() on VIPA1 is brought up on Appl1 goes down on VIPA1 is deleted on Appl1 is brought up on Appl1 issues Bind() on VIPA1 is brought up on 2. MODDVIPA(EZBXFDVP) utility to issue SIOCSVIPA IOCTL - Bind-Specific Application - Bind-Inaddr_any Application - If using MODDVIPA to create a DVIPA, then MODDVIPA would be needed to delete the DVIPA from the stack. See DVIPA Creation Results table in IP Configuration Guide. Page 31 Sysplex Distributor Failure (Distributor) TN3270A TN3270B TN3270C TN3270A TN3270B TN3270C TN3270A TN3270B TN3270C 1 TCP/IP 2 TCP/IP 1 TCP/IP 2 TCP/IP 1 TCP/IP 2 TCP/IP Sess1 Sess2 Sess3 Sess1 Sess2 Sess3 Sess1 Sess2 Sess3 When the Distributing stack fails, the backup Distributor receives routing tables from the target stacks of all the current sessions so none of the sessions to the Target systems fail. When the primary Distributor stack recovers the inbound session paths are returned to the primary Distributor stack. Page 32 16

17 Sysplex Distributor Failure (Target) TN3270A TN3270B TN3270C TN3270A TN3270B TN3270C TN3270A TN3270B TN3270C 1 TCP/IP 2 TCP/IP 1 TCP/IP 2 TCP/IP 1 TCP/IP 2 TCP/IP Sess1 Sess2 Sess3 Sess1 Sess2 Sess3 Sess4 Sess2 Sess3 When a Sysplex Distributor Target stack fails, any sessions currently setup to that Target fail. No more connection requests are sent to that failed Target system. Page 33 Introduction: Source VIPA Page 34 17

18 Source IP Address Selection As per the IP Configuration Guide TCP/IP determines the source IP address for a TCP outbound connection, or for a UDP or RAW outbound packet, using the following sequence, listed in descending order of priority. 1. Sendmsg( ) using the IPV6_PKTINFO ancillary option specifying a nonzero source address (RAW and UDP sockets only) 2. Setsockopt( ) IPV6_PKTINFO option specifying a nonzero source address (RAW and UDP sockets only) 3. Explicit bind to a specific local IP address 4. PORT profile statement with the BIND parameter 5. SRCIP profile statement (TCP connections only) 6. TCPSTACKSOURCEVIPA parameter on the IPCONFIG or IPCONFIG6 profile statement (TCP connections only) 7. SOURCEVIPA: static VIPA address from the HOME list or from the SOURCEVIPAINTERFACE parameter 8. HOME IP address of the link over which the packet is sent For a TCP connection, the source address is selected for the initial outbound packet, and the same source IP address is used for the life of the connection. For the UDP and RAW protocols, a source IP address selection is made for each outbound packet. Page 35 Outbound Connection Using IP Address of LINK TCP/IP Router is single point of failure Local Client and Remote Server Local client sends out a connection request. Source in IP Header = client IP address Destination in IP Header = Remote server IP address Response is sent back to the client. Source in IP Header = Remote server IP address Destination in IP Header = client IP address Source IP Address Connection Request Source= and Destination= Response Source= and Destination= An outage may cause a connection drop. Resolver may be used to determine remote server IP address. For a TCP connection, the same client/server IP addresses are used for the life of the connection. TCP/IP Routing Table is used to determine which the outbound packets are sent over. Static Routing and OSPF support multiple concurrent parallel routes. Policy Agent and NetAccess can effect the decision. Page 36 18

19 Outbound Connection Using Source VIPA TCP/IP Router Local Client and Remote Server Local client sends out a connection request. Source in IP Header = client IP address Destination in IP Header = Remote server IP address Response is sent back to the client. Source in IP Header = Remote server IP address Destination in IP Header = client IP address Source VIPA IP Address Connection Request Source= and Destination= Response Source= and Destination= An outage may NOT cause a connection drop. For all the same reasons documented previously for Inbound Connections Resolver may be used to determine remote server IP address. TCP/IP Routing Table is used to determine which the outbound packets are sent over. Static Routing and OSPF support multiple concurrent parallel routes. Policy Agent and NetAccess can effect the decision. Page 37 Source VIPA Configurations IPCONFIG SOURCEVIPA and IPCONFIG6 SOURCEVIPA Enables interface fault tolerance for clients that establish outbound connections. Static VIPAs can be used as the source IP address for outbound datagrams for TCP, RAW, UDP (except routing protocols), and ICMP requests. IPCONFIG SOURCEVIPA defines that an IPv4 static VIPA that precedes a physical IPv4 address in the HOME list is used as the source IP address, if not overridden. IPCONFIG6 SOURCEVIPA defines that an INTERFACE SOURCEVIPAINT keyword defines the IPv6 static VIPA to be used as the source IP address, if not overridden. IPCONFIG TCPSTACKSOURCEVIPA and IPCONFIG6 TCPSTACKSOURCEVIPA Enables a single sysplex-wide source IP address for clients that establish outbound connections. The source IP address is defined on the TCPSTACKSOURCEVIPA statement. Any valid IP address (real interface, VIPA, DVIPA, or distributed DVIPA) in the HOME list may be defined. Outbound TCP datagrams only (no RAW or UDP support). Either specify single distributed DVIPA on all stacks in sysplex (SYSPLEXPORTS required), or define unique source IP address on each stack. Requires SOURCEVIPA. The recommendation is to use SRCIP instead. SRCIP/ENDSRCIP Enables a source IP address for a job or destination address (destination new in V1.8). The source IP address is defined on the SRCIP/ENDSRCIP statement. Any valid IP address (real interface, VIPA, DVIPA, or distributed DVIPA) in the HOME list may be defined. Outbound TCP datagrams only (no RAW or UDP support). If a Distributed DVIPA is specified in the SRCIP block, then GLOBALCONFIG EXPLICITBINDPORTRANGE is required on all target stacks to guarantee that unique sysplex wide ephemeral ports are assigned. See Explicit Bind Port Range on a later page in this presentation. SOURCEVIPA not required. Order of precedence: 1. SRCIP 2. TCPSTACKSOURCEVIPA 3. SOURCEVIPA SRCIP is Recommended Rather than TCPSTACKSOURCEVIPA Because SRCIP provides all of the functionality of TCPSTACKSOURCEVIPA and additional granularity. Specifying JOBNAME * in a SRCIP profile statement provides the same result as specifying the TCPSTACKSOURCEVIPA parameter for implicit bind scenarios, and also applies to applications that issue a bind to the INADDR_ANY or IN6ADDR_ANY IP address. Sysplex Distributor If a distributed DVIPA is used as the source IP address for outbound connections there is no load balancing being performed like what happens for inbound connections. Page 38 19

20 Target Stack TCP/IP Distributor Stack TCP/IP Router Outbound Connection Using TCP/IP Stack Source VIPA TCP/IP Router TCP/IP VIPA Target Stack TCP/IP Local Client and Remote Server Local client sends out a connection request. Source in IP Header = client IP address Destination in IP Header = Remote server IP address Response is sent back to the client. Source in IP Header = Remote server IP address Destination in IP Header = client IP address Source VIPA IP Address Connection Request Source= and Destination= Response Source= and Destination= An outage may NOT cause a connection drop. For all the same reasons documented previously for Inbound Connections Resolver may be used to determine remote server IP address. TCP/IP Routing Table is used to determine which the outbound packets are sent over. Static Routing and OSPF support multiple concurrent parallel routes. Policy Agent and NetAccess can effect the decision. If a Distributed DVIPA is being used as a source IP address then SYSPLEXPORTS must be used to guarantee that unique sysplex wide ephemeral ports are assigned. (EZBEPORT CF structure) Page 39 TCP Stack Source VIPA Distributed DVIPA and Sysplex Ports If you specify the same distributed DVIPA interface for TCPSTACKSOURCEVIPA on multiple target stacks, you should specify SYSPLEXPORTS on the VIPADISTRIBUTE statement. Otherwise connections might be disrupted because identical connections could be created from more than one stack. Multiple clients on Sysplex Distributor target stacks may be connecting to remote servers. The FTP servers on Sysplex Distributor target stacks may support passive FTP connections. If SYSPLEXPORTS is defined, the coupling facility structure EZBEPORT coordinates sysplexwide ephemeral port assignment for each distributed DVIPA when they are used for TCP connection requests. The stack maintains a list of allowable ephemeral ports Any port number above 1023 that has not been reserved for TCP by a PORT or PORTRANGE statement. The coupling facility structure contains sysplex port assignment information. The name of this structure is in the format EZBEPORTvvtt. vv is the 2-character VTAM group ID suffix specified on the XCFGRPID start option. tt is the TCP group ID suffix specified on the GLOBALCONFIG statement in the TCP/IP profile. If no VTAM group ID suffix is specified, but a TCP/IP group ID suffix is specified, vv is 01. If no TCP/IP group ID suffix is specified, but a VTAM group ID suffix is specified, tt is not present. If neither group ID suffix is specified, both vv and tt are not present. See the Communications Server: SNA Network Implementation Guide, SC for more information about the EZBEPORT structure. If your FTP server accepts connections on a distributed DVIPA, SYSPLEXPORTS must be specified for the distributed DVIPA if either of the following are true: The DVIPA is an IPv6 address. Passive mode FTP is used. Page 40 20

21 Source IP Distributed DVIPA Based on Destination IP and Explicit Bind Port Range BATCHJOB V1.8 SRCIP DESTIP Connect to Selects source IP address based on the customer A ( nnn) destination IP address of the connection. and transfer data. Does not support distributed DVIPA. Connect to Distributed DVIPA as source IP address customer B for SRCIP V1.9 Enhancement ( nnn) and transfer Extends the destination-based source IP data. address selection to include a distributed DVIPA. Participating stacks will reserve a coordinated range of port numbers for this use with GLOBALCONFIG EXPLICITBINDPORTRANGE. If an application issues an explicit bind to INADDR_ANY or INADDR6_ANY and port 0, the stack has GLOBALCONFIG EXPLICITBINDPORTRANGE defined, and the stack has SRCIP rules, a port from this new range will be requested. Source IP address Source IP address Allow source IP , deny all others! Allow source IP , deny all others! /24 Customer A network Customer B network /24 SRCIP Jobname CUSTAJOB Jobname CUSTBJOB Jobname User1* 555:555::222 DESTIP / DESTIP / ENDSRCIP CINET: Supported if only one stack configured or multiple stacks configured but all applications have stack affinity All TCP/IP stacks in the sysplex that participate in EXPLICITBINDPORTRANGE processing should have the same port range specified. To ensure this, specify the GLOBALCONFIG EXPLICITBINDPORTRANGE statement in a file that is specified in an INCLUDE statement in the TCP profile data sets of all the participating stacks. The port range defined on the EXPLICITBINDPORTRANGE parameter should not overlap any existing port reservations of any TCP/IP stacks in the sysplex. Any reserved ports that are within the range are excluded. The EXPLICITBINDPORTRANGE port range must be large enough, but if the size is too large, there are fewer ports available for local ephemeral port allocation. Page 41 DVIPA Details: DVIPA Route and Alternate Distribution Options Page 42 21

22 DVIPA Route TN3270A TN3270B TN3270C Shared Session Request 1 TCP/IP TCP/IP Profile VIPADYNAMIC VIPAROUTE dynxcf_ip target_ip VIPAROUTE on the VIPADYNAMIC statement is used to select a route from a distributing stack or a backup distributing stack to a target stack. This allows a path other than the XCF link to be used. Note that GRE (Generic Routing Encapsulation) is used for IPv4. IPv6 does not need GRE to have an outer IP header for encapsulation. The GRE encapsulation process increases the size of the forwarded packet by 28 bytes. As a result, if the size of the encapsulated GRE packets are larger than the maximum transmission unit (MTU) of the network interface that will be used for forwarding the packet, the TCP/IP stack might need to perform fragmentation. Page 43 Multi-Node Load Balance (MNLB) Target Servers WLM R1, R2, R3 Access Routers WLM L2 Switch Forwarding Agent IP Network R1 R2 R3 WLM L2 Switch Forwarding Agent Client Workstation WLM Local Director Services Manager Cisco s Original MNLB MNLB defines multiple real Server addresses as one or more server cluster addresses. Client traffic specifies the server cluster address as its destination address. Service Manager informs Forwarding Agents of best suited Server based on input from WorkLoad Manager. Current Sysplex Distributor (SD) supported MNLB SD may become the Service Manager for MNLB. Page 44 22

23 Load Balancing Advisor (LBA) Load Balancing Advisor Load Balancing Agent Load Balancer Load Balancing Agent Load Balancing Agent IBM s Load Balancing Advisor Alternative load balancing solution to Sysplex Distributor Does not use DVIPAs. Requires at least one external load balancer that supports the Server/Application State Protocol (SASP). Client traffic specifies the server cluster address as its destination address. LBA Advisor informs LBA Balancer of best suited Server based on input from WorkLoad Manager. Different from sysplex distributor and MNLB because the actual decision of where to route work is made outside of the sysplex. UDP is supported as well as TCP. Page 45 DVIPA Details: Sysplex Distributor Distribution Options See the IP Configuration Reference manual for syntax rules. Page 46 23

24 Distribution Options There are several things that influence the Sysplex Distributor Distribution: Target Server Responsiveness (TSR) Distribution Method Weighted Distribution Local Target Preference Processor Type Importance Level and Crossover Timed (Target) Affinity QoS Policy Outbound Interface QoS Policy Performance Monitoring QoS Policy Total Connections Limit QoS Policy Maximum Connections Limit Share Port WLM Page 47 Target Server Responsiveness (TSR) Distributor uses several health metrics to monitor Target stacks and Servers: TSR (Target Server Responsiveness) fraction, is a compound health-metric per Target Server (range from 0 (bad) to 100 (good)): TCSR (Target Connectivity Success Rate) indicates the connectivity between the Distributing stack and the Target stack. Are new connection requests reaching the Target? (0 is bad, 100 is good) SEF (Server accept Efficiency Fraction) indicates the Target Server accept efficiency. Is the Server accepting new work? (0 is bad, 100 is good) QoS (Quality of Service) fractions take retransmits and packet loss into consideration. (0 is good, 100 is bad) Number of half-open connections also impact the TSR value. No configuration is required for TSR usage. TSR is automatically calculated and used when Sysplex Distributor is defined. CER (Connection Establishment Rate) indicates the Client network connectivity between Server and Client. Are new connections being established? (0 is bad, 100 is good) New in V1.11 CER no longer impacts the TSR value, but is still calculated and included in netstat displays CER Distributor TCSR Target Stack SEF Server Page 48 24

25 WLM Weights and DISTMETHOD WLM-based forwarding requires WLM goal mode on Distributing and Target systems. WLM-based forwarding requires TCP/IP Profile IPCONFIG SYSPLEXROUTING. If you have not made the changes needed to enable WLM-based forwarding (SYSPLEXROUTING has not been specified for all participating stacks, or all participating stacks are not configured for WLM GOAL mode), even if BASEWLM or SERVERWLM is specified, the distributing stack will use round-robin forwarding to distribute connections. TCPDATA HOSTNAME The weights received from WLM are returned based on the value of HOSTNAME. The HOSTNAME value is determined by the search path at initialization time. Verify that all systems have a unique HOSTNAME value to ensure proper distribution. WLM-based forwarding requires BASEWLM or SERVERWLM. TCP/IP Profile VIPADYNAMIC VIPADISTRIBUTE DISTMETHOD BASEWLM Distributing stack forwards connections based upon the workload of each of the target stacks. WLM-based forwarding based on a comparison of available/displaceable capacity on each system. Relative WLM System Weight Recommendations are used. TCP/IP Profile VIPADYNAMIC VIPADISTRIBUTE DISTMETHOD SERVERWLM Distributing stack forwards connections based on a WLM recommendation indicating how well each server is executing on its system. WLM-based forwarding based on: How well each server is meeting the goals of its service class A comparison of available/displaceable capacity on each system given the importance level of the service class. Server-specific WLM Weight Recommendations are used. TCP/IP Profile GLOBALCONFIG SYSPLEXWLMPOLL seconds (new in V1.8) Determines how quickly the sysplex queries to WLM are done, which causes quicker reactions to workload changes. Page 49 Round Robin TN3270A 1 Session Request TN3270B TCP/IP TN3270C TCP/IP Profile VIPADYNAMIC VIPADISTRIBUTE DISTMETHOD ROUNDROBIN Causes round robin distribution to target systems rather than using WLM or Policy information. When Round Robin is defined WLM and Policy do not influence distribution. Page 50 25

26 Sysplex Distributor (SD) Distribution Prior to V1.9 LPAR Capacity LPAR1 LPAR1 LPAR2 LPAR1 LPAR2 SHAREPORT Fence off spare capacity LPAR2 Target systems vary significantly in terms of capacity Small systems versus larger systems WLM recommendations may favor the larger systems significantly. The target application may not scale well to larger systems, being unable to take full advantage of the additional capacity on the larger systems. The result can be that these types of servers when running on larger systems get inflated WLM recommendations and as a result they get overloaded with work. SHAREPORT may have a different number of server instances One system may have three the other may have only one The current Round Robin or WLM recommendations do not change distribution based on the number of server instances on each target. Round Robin distributes 1 connection per target stack regardless of the number of shareport server instances on that stack. WLM Server-specific weights from a target stack with multiple server instances reflects the average weight. Customers may want to reserve some capacity on certain systems for batch type of workloads that get injected into the system during specific time periods and which have specific time window completion requirements. If that system is also a target for long running DDVIPA connections WLM recommendations will allow that available capacity to be consumed and thereby potentially impact the completion of the batch jobs Or vice versa; the connections on that system may suffer from a performance perspective when those jobs are running Page 51 Weighted Distribution LPAR3 Target Stack TCP/IP Profile IPCONFIG DYNAMICXCF dynxcf_ipaddr3 LPAR3 Fence off spare capacity LPAR4 SHAREPORT LPAR4 Target Stack TCP/IP Profile IPCONFIG DYNAMICXCF dynxcf_ipaddr4 LPAR2 LPAR2 Target Stack TCP/IP Profile IPCONFIG DYNAMICXCF dynxcf_ipaddr2 LPAR1 weight1 weight1 weight3 Sysplex Distributor (SD) Weighted Distribution V1.9 Enhancement Based on currently active connections Weights to balance active connections, not incoming connections. Weights configured on the VIPADISTRIBUTE statement per destination IP address Objective is to keep the number of active connections distributed according to the configured weights. More optimal than traditional round robin or weighted round robin algorithms. Case 2 Configured weights Current number of active connections Status LPAR1 Target Stack TCP/IP Profile IPCONFIG DYNAMICXCF dynxcf_ipaddr1 weight2 LPAR1 LPAR2 LPAR above on target on target LPAR0 LPAR below Case 1 LPAR1 LPAR2 LPAR3 LPAR4 Configured weights Current number of active connections Status below above above above LPAR0 Distributing Stack TCP/IP Profile VIPADYNAMIC VIPADISTRIBUTE DISTMETHOD WEIGHTEDACTIVE DESTIP dynxcf_ipaddr1 WEIGHT value2 dynxcf_ipaddr2 WEIGHT value1 dynxcf_ipaddr3 WEIGHT value1 dynxcf_ipaddr4 WEIGHT value4 Note: WLM and Policy information are not used. Page 52 26

27 Local Target Preference (new in V1.8) OPTLOCAL appl Client appl Server appl Server appl Client appl Server NOOPTLOCAL 1 TCP/IP OPTLOCAL (new in V.18) on the VIPADYNAMIC VIPADISTRIBUTE statement causes target stacks to optimize sysplex connections for which both endpoints reside on the same stack. When this value is specified, target stacks should bypass sending connection requests to the sysplex distributor stack for connections to a distributed DVIPA and port pair that reside locally, and instead process the connection locally using local optimizations. The local target stack continues to favor the local stack unless conditions on the local stack become unfavorable as defined by the value specified. NOOPTLOCAL causes target stacks to send locally originating connection requests to distributor stack even when both endpoints reside on same target stack. A value of 0 indicates that connections originating from a target stack within sysplex should always bypass sending connection request to the sysplex distributor. A value of 1 indicates that connections originating from a target stack within the sysplex should always bypass sending the connection request to the sysplex distributor as long as the WLM weight for the server on the local stack is not 0. If a value in the range 2-16 is specified, the value is used as a multiplier against local target stack s raw WLM weight to cause it to be favored over other targets. Regardless of the value specified on the OPTLOCAL parameter, if no local server is available, or the SEF is less than 75 or the abnormal transaction completions is greater than 250, or the health indicator is less than 75, connections are sent to the distributing stack. If round-robin distribution is configured, the OPTLOCAL value is forced to a value of 0. TIMEDAFFINITY and OPTLOCAL are mutually exclusive. TIMEDAFFINITY and OPTLOCAL take precedence over DISTMETHOD. As part of the OPTLOCAL feature, regardless of how a target is chosen (OPTLOCAL or NOOPTLOCAL configured), if the target and client are using the same stack, then the data traffic will stay local (distributor will not be part of the datapath). Page 53 Processor Type Prior to V1.9 capacity of specialty processors, such as zaap and ziip, has not been factored in by WLM when calculating weights for: Sysplex Distributor Load Balancing Advisor (DNS/WLM) WLM (Work Load Monitor) V1.9 Enhancements System Processor Capacity System z9 Allows Communications Server (CS) access to improved weight information in two different ways SERVERWLM - server-specific weights (Sysplex Distributor and Load Balancing Advisor) Uses composite WLM weight that factors in capacity of specialty processors based on the portion they are used by the specific server address space as determined by WLM. No configuration changes needed for SERVERWLM. BASEWLM - system weights (Sysplex Distributor and Load Balancing Advisor) Uses composite WLM weight that factors in capacity of specialty processors based on the portion they are used on each system (whole system rather than specific server address space) as determined by WLM. Uses separate WLM weights from all processor types and combines them based on CS configuration of how each processor type should impact the combined weights. An option to configure the mix of processor types that is needed by the application workload is distributed to will be added to both Sysplex Distributor and Load Balancing Advicer. VIPADISTRIBUTE... BASEWLM PROCTYPE(CP=3,zAAP=1,zIIP=0) ziip zaap General CPs zseries z990 zaap General CPs zseries z900 General CPs Page 54 27

28 LPAR2 will be preferred LPAR1 New workload at IL=2 (can displace IL=3 to IL=7 workload) IL 0: High IL 7: Low IL Importance Level and Crossover New in V1.11 Importance Level (IL) of workload influences distribution target choice. For example, if new workload at IL 2 arrives for distribution Both targets have 500 service units of displaceable workload Prior to V1.11, they would be equal V1.11 takes IL of displaceable workload into consideration SUs LPAR2 IL SUs New in V1.11 Crossover Cost of workload being run on a particular type of processor influences distribution target choice. For example, if new workload at IL 2 arrives for distribution on 90% zaap and 10% CP Both targets have equal amount of displaceable service units Prior to V1.11, they would be equal V1R11 takes crossover cost of workload being run on a CP instead of a zaap LPAR2 will be preferred since it has the least amount of crossover cost New workload designed to use 90% zaap and 10% CP New workload at IL=2 (can displace IL=3 to IL=7 workload) LPAR1 IL CP SUs zaap SUs LPAR2 IL CP SUs zaap SUs Page 55 Timed Affinity TN3270A TN3270B TN3270C 1 Session Request TCP/IP TIMEDAFFINITY on the VIPADYNAMIC VIPADISTRIBUTE statement specifies whether or not a connection from a client to a particular server instance of several served by sysplex distributor shall establish an affinity for future connections from the same client (IP address) to the same Distributed DVIPA and ports. A value of 0 means that no affinity is established when a new connection request is distributed to a particular server application instance by sysplex distributor. A nonzero value means that when a connection from a client is routed to a particular server instance, any subsequent connections from the same client to the same Distributed DVIPA and ports are routed to the same server instance until the specified number of seconds have elapsed after the last such connection was closed. Timed Affinity is required when using TN3270E printer associations. TIMEDAFFINITY and OPTLOCAL are mutually exclusive. TIMEDAFFINITY and OPTLOCAL take precedence over DISTMETHOD. Page 56 28

29 Quality of Service (QoS) Policy Outbound Interface PROFILE.TCPIP VIPADYNAMIC VIPADEFINE VIPADISTRIBUTE PORT 2000 DESTIP ALL ENDVIPADYNAMIC Policy Agent Definitions OutboundInterface ForLoadDistribution TRUE Sysplex Distributor Target Stack Subnet 1 Subnet 1 Sysplex Distributor Distributor Stack Sysplex Distributor Target Stack ip@2 Subnet 2 Subnet 2 Sysplex Distributor Target Stack ip@3 Policy Agent Definitions OutboundInterface ip@1 ForLoadDistribution TRUE Incoming connection requests can be distributed to different target stacks within the sysplex by the sysplex distributor distributing stack. Page 57 Outbound Interface Configuration QoS Outbound Interface routing requires: WLM based routing Distributing stack Policy Agent configuration: PolicyAction PolicyScope DataTraffic (or Both) OutboundInterface ip_addr PolicyRule ForLoadDistribution TRUE Where ip_addr specifies the DynamicXCF addresses of the Sysplex Distributor Targets to receive the connections. A value of 0 can be specified for the interface, which indicates to the Sysplex Distributor Distributing stack that if it cannot distribute the request to a target stack on one of the specified interfaces, then the request can be distributed to any of the other eligible target stacks. Page 58 29

30 QoS Policy Performance Monitoring Policy Agent configuration statement PolicyPerfMonitorForSDR Enable is required for Policy Performance Monitoring. Assigns cumulative and discrete weight fractions to the monitored policy performance data and sends them to the Sysplex Distributor Distributing stack as the monitored data crosses defined thresholds. PolicyPerfMonitorForSDR parameters: Enable Disable Enables or disables the policy performance monitor function. The weight fractions determined for the loss ratio and timeout ratio are added together to form a single weight fraction before being sent to the Sysplex Distributor Distributing stack. One weight fraction is generated for each DVIPA/port pair on a Sysplex Distributor Target stack. SamplingInterval Specifies the interval in seconds for sampling policy performance data. LossRatioAndWeightFr Specifies two numbers. The first is the unit ratio of retransmitted bytes (loss) over transmitted bytes, in tenths of a percent ( ). The second number is the weight fraction to be returned to the Sysplex Distributor Distributing stack, in percentage (1-100). When present, this parameter results in creation of a threshold table. LossMaxWeightFr Specifies the max weight fraction to be assigned for the loss ratio factor. TimeoutRatioAndWeightFr Specifies two numbers. The first number is the unit ratio of the number of timeouts over transmitted packets, in tenths of a percent ( ). The second number is the weight fraction to be returned to the Sysplex Distributor Distributing stack, in percentage (1-100). TimeoutMaxWeightFr Maximum weight fraction to be assigned for the timeout ratio factor. MaxConnWeightFr Specifies three percentages that are used in calculating the connection limit portion of the policy action (service level) weight fractions. Each percentage must be in the range 1-100, and each value must be greater than or equal to the preceding value. When calculating policy action weight fraction, the number of active connections to a target DVIPA/Port is compared with maximum connections allowed for the associated policy action as follows: When the number of active connections reaches the percentage of maximum connections specified by the first number, the policy action weight fraction is set to MAX (50%, current calculated value). When the number of active connections reaches the percentage of maximum connections specified by the second number, the policy action weight fraction is set to MAX (85%, current calculated value). When the number of active connections reaches the percentage of maximum connections specified by the third number, the policy action weight fraction is set to 100%. Page 59 QoS Cumulative and Discrete Weight Fractions Discrete Weight Fractions requires: Distributing Stack WLM Distribution Method (BASEWLM or SERVERWLM) Distributing and Target Stacks Policy Agent configuration PolicyPerfMonitorForSDR Enable PolicyAction name PolicyRule ForLoadDistribution TRUE Note: PolicyAction name must be the same on the distributing and target stacks Target Stacks Policy Agent configuration PolicyRule ForLoadDistribution FALSE Sent over TCP connections Separate for each Service Level Cumulative Weight Fractions are sent when: Distributing Stack WLM Distribution Method (BASEWLM or SERVERWLM) Distributing and Target Stacks Policy Agent configuration PolicyPerfMonitorForSDR Disable Sent over XCF connections Not separate by Service Level Service Level configured by PolicyRule ApplicationPriority serv_lev Sysplex Distributor Target Stack Sysplex Distributor Stack QoS Weight Fraction Policy Agent Statistics Stack QoS Weight Fraction Sys Dist / WLM Interface Coupling Facility Sysplex Distributor Stack Policy Agent Policies Sysplex Distributor Target Stack QoS Weight Fraction Policy Agent Statistics Sysplex Distributor Distributing Stack Page 60 30

31 Total Connections and Maximum Connections Limits QoS Policy Performance Monitoring may be used with a Total Connections Limit policy and/or a Maximum Connections Limit. Total Connections Limit PolicyAction PolicyScope TR TotalConnections tot_num Where tot_num is the number of concurrent connections allowed TR policy is considered CONSTRAINED when Active number of connections > 90% tot_num When TR policy is CONSTRAINED All policy action fractions, including the default, will be forced to 100% Maximum Connections Limits PolicyAction PolicyScope DataTraffic MaxConnections max_num Where max_num is the number of concurrent connections allowed Using PolicyPerfMonitorForSDR default MaxConnWeightFr When active connections reaches 70% (first number on MaxConnWeightFr parameter) then the weight fraction is set to 50%. When active connections reaches 85% (first number on MaxConnWeightFr parameter) then the weight fraction is set to 85%. When active connections reaches 95% (first number on MaxConnWeightFr parameter) then the weight fraction is set to 100%. Recommendations Use MaxConnections rather than PolicyScope TR because it has multiple throttling percentages and more granularity. Avoid having multiple applications map to the same QoS policy action. Page 61 Share Port WLM PORT statement in the TCP/IP Profile is used to reserve a port for a specific job name. Parameter SHAREPORT or SHAREPORTWLM may be specified on the PORT statement. When SHAREPORT is specified, TCP/IP allows multiple listeners to listen on the same combination of port and interface. Specification of this keyword causes incoming connection requests for the port to be distributed among the listeners using a weighted round-robin distribution method based on the servers accept Efficiency Fractions (SEFs) of the listeners sharing the port. By specifying SHAREPORTWLM on the PORT statement, connections are distributed in a weighted round-robin fashion based on the WLM server-specific recommendations, as modified by the Server accept Efficiency Fraction (SEF). SHAREPORT and SHAREPORTWLM are only valid for TCP ports. If the shared port is a Sysplex Distributed port and SERVERWLM is the distribution method that is being used, then SHAREPORTWLM should be coded on each target s PORT statement to take advantage of the WLM server-specific recommendations when connections are received at the target; if it is not, new connections continue to be distributed using the existing SHAREPORT algorithm when they are received at the target. Server s Accept Efficiency Fractions (SEF) The server s accept Efficiency Fractions (SEF) is a measure of the efficiency of the server application in accepting new connection requests and managing its backlog queue. Page 62 31

32 DVIPA Details: Sysplex Distributor TN3270 LU Support Page 63 TN3270 Coordinated LU Names Active LU names must be unique across all LPARs in the Sysplex. New in V1.10 TN3270 LU names may be coordinated across the Sysplex. One telnet server may be configured as a LU Name Server (LUNS) for multiple LU Name Requesters (LUNR) in the sysplex so that the telnet port may be configured for Sysplex Distributor. A shared set of LU names are defined as a shared name space. Generic LU requests pick an available LU name in the shared name space. Specific LU requests verify that the requested LU is not already in use in the Sysplex. Reconnect with specific LU name requests after temporary network failure is supported. Manage timemark processing on existing connection(s) across sysplexed TN3270 servers: Terminate "stale" TN3270 connections Clean up and terminate associated SNA session Free LU name(s) for terminated connections Process reconnect on any TN3270 server in the Sysplex Telnet A LUNR Local LUs Telnet 1 LUNS LU Table A+B LU Table A Telnet B LUNR LU Table B LUNS LUs VTAM Sess Mgt CICS LUCICS Some LUs are managed by the LUNS, others are managed locally. It depends on the LU group definition. Page 64 32

33 No LUNS, LUNS, or LUNR Three TN3270E Telnet server roles: Classic Telnet Not a member of a Telnet XCF group No shared LU groups permitted XCF Telnet Must Join XCF Group LUNR (LU Name Requestor) capable Ports can use shared LU groups XCF LUNS (LU Name Server) Telnet Must Join XCF Group LUNR capable Coordinates shared LU name assignments One LUNS in control at any given time others are standby Manual or automated takeover by standby LUNS Page 65 DVIPA Details: Fast Response Cache Accelerator (FRCA) Page 66 33

34 Fast Response Cache Accelerator (FRCA) FRCA caches Web pages within the TCP/IP stack. Requests are handled without traversing the full protocol stack up to the Web server. Significant performance improvements when compared to the Web server handling all requests. Web server Sockets layer TCP Layer FRCA IP networking layer Network interfaces The Web server stores static objects in the cache The Web server provides simple exits to parse URIs FRCA Cache Cached static objects are served from the cache without the Web server accepting the TCP connection Page 67 FRCA use by WebSphere Application Server (WAS) Cached responses can be served with high performance using a minimal amount of CPU cycles. Serve static requests from the FRCA cache. Provide equivalent performance on WAS as is possible with the FRCA cache on the web server. Serve dynamic content from the FRCA cache. Serve the same content that the Dynamic Cache serves but serve it from the FRCA cache. Record HTTP Access Log entries for requests served from the FRCA cache. New in WebSphere Application Server V7, WAS will start exploiting the FRCA cache. Amount of CPU time needed to process a request is dramatically reduced using FRCA as compared to Dynamic Caching. Page 68 34