Sysplex Networking Technology Overview

Transcription

1 Summer 2010 Technical Conference Sysplex Networking Technology Overview Thursday, August 5, 2010, 11:00 AM - 12:00 PM Enterprise Networking Solutions Gus Kassimis - kassimis@us.ibm.com Sam Reynolds - samr@us.ibm.com Copyright International Business Machines Corporation All rights reserved. Sysplex Distributor VIPA1 Hot Standby Pagent WLM z/os Sysplex APP Hidden VIPA1 Sysplex Distributor Outbound data path Inbound data path APP Hidden VIPA1

2 IBM Software Group Enterprise Networking Solutions Agenda Pagent Sysplex Distributor WLM z/os Sysplex APP Hidden VIPA1 VIPA1 Hot Standby Inbound data path Sysplex Distributor APP Hidden VIPA1 Outbound data path Sysplex Overview Communication Server enablement for key Sysplex value points Network access - SNA and TCP/IP The Virtual IP Address concept Sysplex-internal or external IP load balancing decision point SNA Availability and load balancing PPRC Subplexing - Isolating network resources CEC-1 Application A OS and middleware infrastructure supporting data sharing OS and middleware infrastructure supporting data sharing CEC-2 Application A Switch Switch

3 1 IBM Software Group Enterprise Networking Solutions Unleashing the benefits of the Parallel Sysplex cluster The promises of the Parallel Sysplex cluster environment are: Application location independence Ability to shift application workload between LPARs Application single system image from the network Application capacity on-demand Component failure does not lead to application failure PPRC Application services to be always available - both during planned and unplanned outages CEC-1 Application A OS and middleware infrastructure supporting data sharing OS and middleware infrastructure supporting data sharing CEC-2 Application A I want to use Application A and its data, but I do not care where it is in this Sysplex! Switch Switch Gaining the benefits, depend on: Carefully designed redundancy of all key hardware and software components in symmetric configurations Supporting functions in z/os and middleware Cooperation by applications Operations procedures 3

4 Sysplex Enables Single System Image Transparent location of applications Multiple images of same application appear as single application to end user Balance Workload within Sysplex Minimize Application failure impact Freedom to move application workload to other images Single System Image Escon NN Coupling Facility NN Network 4

5 Sysplex Enables Horizontal Growth Connectivity Services Discovery of new sysplex members Dynamic connectivity via XCF links Directory Services Dynamic registration for applications Transparent location of resources System Administration Services System Cloning Application Cloning Move Application Network Coupling Facility Add New Image 5

6 Reduced Definitions via System Symbolics Exploit Cloning Support System Cloning System Symbolics in VTAMLST members Allows VTAMLST definitions to be shared among Sysplex members System Symbolics in TCP Config files Allows TCP Config Files to be shared among Sysplex members VTAM Application Cloning Dynamic Definition of VTAM Applications System Symbolics and wildcards in APPL names (e.g. APPL.&sysclone, APPL*) Reduction in VTAM resources used for APPL definitions Network Address allocated at OPEN ACB and released at CLOSE ACB APPL definition created at OPEN ACB and deleted at CLOSE ACB Allows for easy APPL relocation Cloning for TN3270 Server Support includes TN3270 clients represented by VTAM APPL definitions Reduction in VTAM resources used to represent TN3270 clients Simplifies TN3270 Server relocation Single System Image Escon NN Network Coupling Facility NN 6

7 Automatic Recovery Applications discontinue on SYSB SYS A GRP01 Appl1 Appl2 Appl3 XCF Address Space ARM GRP01 Appl1 Appl2 Appl3 SYSB GRP02 Appl4 Appl5 CF Couple Data Set ARM Policy Restart on SYSA Restart on SYSC GRP02 Appl4 Appl5 SYSC Exploit zseries Automatic Restart Manager (ARM) Registered applications automatically restarted on failure ARM policy provides an ordered list for recovery VTAM registers with ARM for restart TCP/IP stack registers with ARM for in-place restart ARM facility is open interface which can be exploited by any application Exploited by CI, IMS, DB2 7

8 Network access to the downstream network from a Sysplex (IP Communications) Application LPAR Application LPAR Application LPAR Application LPAR Application LPAR Application LPAR Network Services LPAR Network Services LPAR OSA QDIO OSA QDIO CEC-1 CEC-2 OSA QDIO VLAN1 VLAN2 VLAN3 VLAN4 OSA QDIO Layer-3 Layer-3 Layer-2 Switch-1 Layer-2 Switch-2

9 Downstream network connectivity to the Sysplex: OSA-Express with QDIO Redundancy at all levels Application LPARs zseries HW CF OSA-E Switch Routers Network Automated failover technologies at all levels OSA QDIO OSA QDIO Application LPAR Application LPAR Application LPAR Network Services LPAR CEC-1 CEC-2 Application LPAR Application LPAR Application LPAR Network Services LPAR OSA QDIO VLAN1 VLAN2 VLAN3 VLAN4 OSA QDIO No single point-of-failure! Layer-3 Layer-2 Switch-1 Layer-3 Layer-2 Switch-2 9

10 Network access to the downstream network from a Sysplex (SNA Communications) Application API Sockets RTP TCP ANR IP DLC RTP ANR ANR x ANR ANR RTP

11 High Performance Routing (HPR) High Performance Routing (HPR) is a high-availability extension to the original SNA architecture API Application Sockets HPR preserves sessions across intermediate node/link failure RTP reroutes sessions in event of a planned or unplanned node or link failure Sessions rerouted to new route determined by Class of Service Sessions may be maintained across link "hit" without any switch occurring Discarded data retransmitted using HPR selective retransmission RTP RTP ANR ANR ANR DLC x TCP IP ANR ANR RTP 11

12 Enterprise Extender (HPR Over UDP) System z Servers Preserves SNA application/device investment No changes required to SNA application Enables single WAN protocol Eliminates native SNA in the WAN End-to-end SNA over IP transport Includes preservation of SNA prioritization Improves datacenter connectivity and access Exploits OSA Express and HiperSockets Simplifies APPN network design Significantly reduces network flows in WAN as compared to base APPN IP Backbone TN3270, or Web client IB M IB M HPR IB M SNA Network Cisco SNASw, Communications Server for Windows, or other IB M Can replace SNI with IP technologies Uses Extended Border Node (EBN) connectivity SNA Clients TCP sessions/ routes SNA routes for SNA sessions EE routes for SNA sessions 12

13 z/os TN3270E Server z/os LPAR TN3270 is a standard protocol for transmitting 3270 data streams over an IP network The TN server has an LU-LU session with the SNA application for each TN3270 client and transforms the datastream back and forth between native SNA and TCP/IP. The TN3270 server can take advantage of many TCP/IP high-availability functions: VIPA Takeover Sysplex Distributor To improve the ability of a set of TN3270 servers in a sysplex to present a single system image to the network, z/os V1R10 added a TN3270 LU Name Management capability to coordinate the assignment of LU names from an LU name pool shared between multiple TN3270 servers. TN3270 server TCP/IP address space TN3270 SNA application VTAM address space SNA TCP/IP 13

14 Virtual IP Addressing My virtual z/os IP host VIPA#2 CI Appl-A VIPA#1 TN3270e Server VIPA#6 Web Services VIPA#3 FTP Services VIPA#4 DB2 subsystem VIPA#5 CI Appl-B OSA OSA OSA IP#10 IP#11 IP#12 Connect to VIPA#1 Name server Use IP address VIPA#2 Resolve CI-Appl-A.xyz.com Connect to CI-Appl-A.xyz.com

15 The network view of a Parallel Sysplex - a single large server with many network interfaces and many services My virtual z/os IP host VIPA#1 TN3270e Server VIPA#6 Web Services OSA VIPA#2 CI Appl-A VIPA#3 FTP Services OSA VIPA#4 DB2 subsystem VIPA#5 CI Appl-B OSA IP#10 IP#11 IP#12 The objective is to make the Sysplex look like one large server that has a number of physical network interfaces for performance and availability - and that provides a number of highly available and scalable services. Connect to VIPA#1 Name server Use IP address VIPA#2 Resolve CI-Appl-A.xyz.com Connect to CI-Appl-A.xyz.com Single-system image (SSI) Scalable Highly available Secure 15

16 Why do I need virtual IP addresses (VIPA)? What does the virtual IP addressing (VIPA) technology promise? My virtual z/os IP host VIPA#2 CI Appl-A VIPA#1 TN3270e Server VIPA#4 DB2 subsystem VIPA#3 FTP Services VIPA#6 VIPA#5 Web Services CI Appl-B OSA OSA OSA IP#10 IP#11 IP#12 Use IP address VIPA#2 Name server Interface resilience: Connect to Resolve CI-Appl-A.xyz.com VIPA#1 Communication with a server host is unaffected by server physical network interface failures. As long as just a single physical network interface is available and operational on a server host, communication with applications on the server host will persist. Connect to CI-Appl-A.xyz.com Application access independent of network topology: Separates network topology from server application topology - a VIPA address can be used to identify a server application instead of a physical network interface. Allows network administrators to renumber physical network topology no impact to end-user accessing server applications by IP address no changes needed in DNS or hosts file configuration no impact to firewall filtering rules Single system image: Allows the Sysplex to be perceived as a single large server node, where VIPA addresses identify applications independently of which images in the Sysplex the server applications execute on. Applications retain their identity when moved between images in a Sysplex. Multiple instances of a server application can be accessed as one server. 16

17 Dynamic VIPA Movement - Stack Managed DVIPAs VIPA VIPA VIPA Network VIPA VIPA COUPLING FACILITY ESCON Cached IP address VIPA Dynamic VIPA Support VIPAs can survive any outage by moving to another stack in Sysplex via VIPA Takeover VIPAs exchanged by TCP/IP stacks in sysplex via XCF messaging Another appl instance can pick up workload or Appl can be restarted on takeover stack Connections broken but Reset sent to client upon takeover Significantly reduces down time Dynamic VIPA Takeback VIPA moves back to recovered primary owner New Connections Handled By Primary Owner again Connections Established To Backup are allowed to continue Data forwarded from primary owner to backup Allows Movement Of Application Server Without Impacting Existing Workload Useful for planned outages as well Operator commands allow you to move Dynamic VIPAs non-disruptively 17

18 Are dynamic routing protocols required on z/os in order to use VIPA? Base IP recovery as well as VIPA address movement were designed and implemented with the use of dynamic routing functions in mind! B Move appl-1 and its associated dynamic VIPA address z/os-b VIPA1 Appl-1 z/os-a VIPA1 Appl-1 A VIPA2 Move a dynamic VIPA address between z/os images z/os-c VIPA2 Connection resilience to OSA network and interface outages OSA OSA R R R C D Load-balance outbound IP traffic (multipath) Always remember that a z/os Sysplex is not a host, it is an IP network in itself and as any IP network it needs the capability to react to topology changes in its own network and in the adjacent networks. The recommended dynamic routing protocol in the Sysplex is OSPF. Connect to VIPA-1 (Appl-1) Dynamic routing is not an absolute requirement, but it is highly recommended when using VIPA addresses (it makes life a whole lot easier)! 18

19 Basic principles for recovery of single-instance IP application in a Sysplex Single-instance applications are applications that only run in one instance in the Sysplex. Either because the application needs exclusive access to certain resources, or because there is no need to start it in more than one instance. Availability from an IP perspective then becomes an issue of being able to restart the application on the same LPAR or on another LPAR with as little impact to end-users as possible. Speed of movement - ARM or automated operations procedures Retain identity from a network perspective (its IP address) - Application Instance DVIPAs cicsappl Application-specific cicsappl Restart application 1 dynamic VIPA addresses come in very handy for this purpose. Connect to cicsappl1.mycom.com : DNS Use Resolve cicsappl1.mycom.com Either 1 Resolve cicsappl1.mycom.com 2 connect to returned address or 3 Connect to cached (or hardcoded!) address 19

20 Single system image (SSI) from an IP perspective in the Sysplex Inbound SSI Sysplex Connect to DRVIPA1 z/os z/os z/os TCP/IP provides several facilities that allow you to specify the IP address to be used on outbound connections from the sysplex SOURCEVIPA, TCPSTACKSOURCEVIPA, and SRCIP statement in TCP/IP profile Can be specified on a system basis or an application basis (even if the application can execute on any system in the sysplex) Connect to DRVIPA1 Connect to? from SRCVIPA1 Connect to? from SRCVIPA2 Connect to? from SRCVIPA3 We have single system image capability for inbound connections where a single distributed VIPA address can represent all images in the Sysplex - and remote users do not need to select a specific image when connecting to their server application. But if we establish outbound connections from the images in the Sysplex, each image has its own source VIPA address - so there is no single system image from an outbound connection perspective - which has implications in firewall filter setup, etc. Sysplex z/os z/os z/os Outbound SSI Connect to? from DRVIPA2 20

21 How to balance your IP workloads so that you can maximize your availability and productivity? Sysplex z/os z/os A1 A1 A1 TCP/IP VTAM TCP/IP VTAM LB

22 Sysplex z/os z/os A1 A1 A1 TCP/IP VTAM IBM Software Group Enterprise Networking Solutions Workload balancing: a question of both performance, availability, and scalability LB TCP/IP VTAM Connection load balancing technologies: Between z/os images: Internal: Sysplex Distributor, Generic Resources External: Cisco M, S, F5 Big IP, etc. Inside single z/os TCP/IP stack: Port sharing Application Characteristics: Multiple instances of the server are able to provide the exact same services to clients (will typically require data sharing) No state preserved at server between two connections (application protocol has to include support for such behavior or store state data in shared storage) Benefits of Intelligent Load Balancing: Performance - improving response time Availability - If one instance goes down, connections with it break, but new connections can be established with remaining instance(s) Scalability - more server instances can be added on demand (horizontal growth) Examples: Web server TN3270 server CI applications FTP server DB2 MQ WAS LDAP RYO... 22

23 Sysplex Distributor: z/os-integrated intra-sysplex workload balancing Independent of network attachment technology. Will work with both direct (including OSA Express) and channel-attached router network connections. All z/os images communicate via XCF. Each TCP/IP stack has full knowledge of IP addresses and server availability in all stacks. A network-connected stack owns a given VIPA address and acts as the distributor of new connection requests to that VIPA address. Sysplex Distributor VIPA1 Hot Standby Pagent Sysplex Distributor WLM Inbound data path z/os Sysplex APP Hidden VIPA1 APP Hidden VIPA1 Outbound data path Distribution of new connection requests is based on real-time information State of target application, system and TCP/IP stack WLM recommendations LPAR CPU capacity or WLM Server specific recommendations (are the target server applications meeting their WLM policy goals?) Additional workload distribution methods Round robin, Weighted Active, Hot/Standby Application Server and TCP/IP health Are the target applications accepting new connections? Do they have network connectivity back to the clients? Network Quality of Service (QoS) metrics (with z/os QoS policy agent) 23

24 z/os Load Balancing Advisor (LBA) for outboard load balancers The SASP (Server/Application State Protocol) control flows will provide relative weights per server instance (based on WLM weight, server availability, and server processing health taking such metrics as dropped connections, size of backlog queue, etc. into consideration) Work requests Server instance z/os LB agent z/os Sysplex Work requests Load Balancer Server instance z/os LB agent SASP control flows z/os workload balancing Support for clustered z/os servers in a z/os Sysplex Not focused on HTTP(S) only, will support all IPbased application workloads into a z/os Sysplex Based on Sysplex-wide WLM policy Scope is a z/os Sysplex z/os LB advisor Server instance z/os LB agent Private protocol control flows 24

25 SNA Sysplex Capabilities for Maximizing Availability VTAM1 VTAM2 VTAM3 VTAM4 EN1 EN2 EN3 EN4 CI1 CI2 CI3 CI4 IMS1 IMS2 IMS3 IMS4 DB2a DB2b DB2c DB2d VTAMA CI5 VTAMB "Directory" NNA CI IMS CI1 - EN1 IMS1 - EN1 CI2 - EN2 IMS2 - EN2 CI3 - EN3 IMS3 - EN3 CI4 - EN4 CI5 - NNB IMS4 - EN4 DB2 DB2a -EN1 DB2b -EN2 DB2c - EN3 DB2d - EN4 "Affinities" LU62A-IMS3 COUPLING FACILITY ESCON Network IMS CI LU62A LU3270A NNB

26 "Directory" CI CI1 - EN1 CI2 - EN2 CI3 - EN3 CI4 - EN4 CI5 - NNB DB2 DB2a -EN1 DB2b -EN2 DB2c - EN3 DB2d - EN4 "Affinities" LU62A-IMS3 IBM Software Group Enterprise Networking Solutions Generic Resources and Multi-Node Persistent Sessions VTAM1 VTAM2 VTAM3 VTAM4 EN1 EN2 EN3 EN4 CI1 CI2 CI3 CI4 IMS1 IMS2 IMS3 IMS4 DB2a DB2b DB2c DB2d VTAMA CI5 VTAMB NNA IMS IMS1 - EN1 IMS2 - EN2 IMS3 - EN3 IMS4 - EN4 IMS LU62A Network COUPLING FACILITY ESCON CI LU3270A NNB SNA Generic Resources Multi- system application seen as single application to end user Balances sessions within the parallel sysplex New logons not affected by application outage Dynamic registration performed by application on activation Application is de-registered when no longer available Exploited by CI, IMS, DB2, APPC/MVS, and TSO/VTAM Available to any SNA application MNPS Support for SNA Sessions Support for both RAPI and APPC sessions Eliminates or reduces outage (VTAM, z/os, or hardware) impact Persistence support also available for planned application workload takeover Requires HPR within a parallel sysplex Exploited by CI, APPC/MVS, IMS, and the IBM Session Manager (ISM) 26

27 MNPS or Generic Resources The usual answer: It depends! The most important aspect to understand is whether session affinities exist for the application workloads involved 3270 workloads typically mean no session affinity exists upon application outage Generic Resources allows the end user to logon to another application instance immediately after the original application outage Recovery is faster than MNPS and requires far less overhead during normal operations LU 6.2 workload typically means that a session affinity does exist upon application outage Generic Resources is not allowed to choose another application instance upon a subsequent logon from the same end user after the original application outage Original application must be recovered Another important aspect is to determine the importance of availability vs overall system performance MNPS provides superior availability but does impact performance during normal operation Storage impact, CF access, etc. The steady-state CPU utilization associated with an application may increase by up to 50% when MNPS-enabled due to the overhead of maintaining the state in the coupling facility Generic Resources does not impact performance of the data path Resolution only done during session establishment Recommendation: Implement Generic Resources for IMS, CI, and DB2 workloads Consider implementing MNPS for select applications only if LU 6.2 workload is critical enough to justify the extra CPU cycles 27

28 How to mesh connect your sysplex images so that you can promote data sharing and grow your business without impact to your customers? Move Application Coupling Facility Add New Image Network

29 XCF Dynamics Enables Horizontal Growth XCF uses Coupling Facility Links for Data Transport Eliminates requirement for ESCON definitions for SNA and IP Provides dynamic discovery and connectivity of other nodes in Sysplex No coordinated definition required to add new images Provides notification of new sysplex members as well as members who have failed Dynamically updates existing members of sysplex Move Application Coupling Facility Add New Image Network 29

30 Is XCF signaling always used for the DYNAMICXCF IP network? TCP/IP Stack-1 IUTSAMEH TCP/IP Stack-2 HiperSockets TCP/IP Stack-3 z/os TCP/IP z/os TCP/IP LPAR-1 LPAR-2 z/os TCP/IP z/os TCP/IP zseries CEC-1 TCP/IP Stack-4 XCF Signaling LPAR-3 CEC-2 From an IP topology perspective, DYNAMICXCF automatically establishes fully meshed IP connectivity to all other z/os TCP/IP stacks in the Sysplex that also have DYNAMICXCF specified. One end-point specification in each stack for fully meshed connectivity to all other stacks in the Sysplex: IPConfig DynamicXCF Automatic connectivity to new stacks as they start up in the Sysplex Only one dynamic XCF network supported per Sysplex Under-the-covers DYNAMICXCF will choose one of three transport technologies depending on availability and location of partner z/os TCP/IP stack: Inside same LPAR: IUTSAMEH (memory-link inside a z/os system) Inside same zseries CEC: HiperSockets (if enabled for that purpose via the IQDCHPID VTAM start option) Outside zseries CEC: XCF signaling 30

31 SNA Intra-Sysplex Connectivity: MPC+, XCF, or EE using QDIO? The usual answer: It depends! All things being equal, MPC+ throughput should exceed XCF throughput, but using multiple XCF links can significantly increase throughput. XCF links can bypass the coupling facility, increasing throughput if the CF is being used for other functions (GR, MNPS) XCF will use more VTAM CPU cycles due to the API to the XCF facility. Many customers want to define both MPC+ and XCF links, but want to prefer MPC+, with XCF available for backup. This can be accomplished by adding COSTBYTE=1 to the XCF TGP (in IBMTGPS) which is automatically associated with XCF TGs (assuming IBMTGPS has been activated). This makes the XCF link have a higher weight (and therefore be less desirable) than the MPC+ link for the IBMsupplied APPN Classes of Service. EE using QDIO is a valid option for intra-sysplex connectivity but consider: Cross CEC traffic must go out onto the data center LAN and via 2 OSA Express cards Cross CEC traffic may not realize a significant performance advantage over well tuned XCF or MPC+ For SNA workloads within the same CEC, EE over HiperSockets (or shared OSA) will provide superior performance unless CPU availability is limited 31

32 Sysplex Network Partitioning Dedicated LPARs with single TCP/ IP stack A1 A2 A1 A2 A1 A2 A1 A2 Intranet IP Intranet IP DMZ IP DMZ IP Intranet DMZ Multi-purpose LPARs with dual TCP/IP stacks A1 A2 A1 A2 A1 A2 A1 A2 Intranet IP DMZ IP Intranet IP DMZ IP Intranet DMZ

33 Dedicated LPARs with single TCP/ IP stack IBM Software Group Enterprise Networking Solutions z/os Sysplex connectivity to multiple security areas has been an issue every since began using Sysplex capabilities How to control level of automatic connectivity XCF signaling (group name) - both IP and SNA IUTSAMEH (same host IP links inside an LPAR) HiperSockets (as enabled via IQDCHPID in VTAM) A1 A2 A1 A2 Intranet IP Intranet IP Intranet A1 A2 DMZ IP DMZ A1 A2 DMZ IP How to control level of IP and SNA resource awareness Dynamic IP address discovery across the Sysplex VTAM generic resource and MNPS resource scope spans the full Sysplex Multi-purpose LPARs with dual TCP/IP stacks A1 A2 A1 A2 A1 A2 A1 A2 How to control scope of IP workload balancing using Sysplex Distributor SD requires Dynamic XCF to be enabled, and Dynamic XCF will establish automatic IP connectivity to all stacks in the Sysplex that also have Dynamic XCF enabled Intranet IP Intranet DMZ IP Intranet IP DMZ DMZ IP To support environments such as these, installations typically end up implementing complex resource controls and disabling many of the dynamic networking functions that are provided by TCP/IP and VTAM. 33

34 Enable use of networking Sysplex functions in a Sysplex that is connected to multiple security areas One SNA subplex per LPAR DMZ SNA Subplex Intranet SNA Subplex VTAM VTAM VTAM VTAM VTAM DMZ IP Subplex Intranet Primary IP Subplex IP-1 IP-1 IP-1 IP-1 IP-1 A TCP subplex cannot span multiple SNA subplexes Research IP Subplex IP-2 IP-2 Development IP Subplex IP-3 IP-3 Different IP stacks in an LPAR may belong to different IP subplexes LPAR1 LPAR2 LPAR3 LPAR4 LPAR5 Standard RACF Networking subplex scope: controls for stack access and VTAM Generic Resources (GR) and Multi-Node Persistent Session (MNPS) application access resources to z/os resources Automatic connectivity - IP connectivity and VTAM connectivity over XCF need to be in place. (including dynamic IUTSAMEH and dynamic HiperSockets based on Dynamic XCF for IP) IP stack IP address (including dynamic VIPA) awareness and visibility Dynamic VIPA movement candidates Sysplex Distributor target candidates 34

35 IBM Software Group Enterprise Networking Solutions Agenda DMZ SNA Subplex Intranet SNA Subplex VTAM VTAM VTAM VTAM VTAM DMZ IP Subplex Intranet Primary IP Subplex IP-1 IP-1 IP-1 IP-1 IP-1 Research IP Subplex IP-2 IP-2 Development IP Subplex IP-3 IP-3 LPAR1 LPAR2 LPAR3 LPAR4 LPAR5 Sysplex Overview Communication Server enablement for key Sysplex value points Network access - SNA and TCP/IP The Virtual IP Address concept Sysplex-internal or external IP load balancing decision point SNA Availability and load balancing PPRC Subplexing - Isolating network resources CEC-1 Application A OS and middleware infrastructure supporting data sharing OS and middleware infrastructure supporting data sharing CEC-2 Application A Switch Switch

36 For More Information... URL IBM System z IBM System z Networking Content IBM Communications Server Twitter Feed IBM Communications Server Facebook Fan Page IBM z/os Communications Server IBM Communications Server for Linux on zseries IBM Communication Controller for Linux on System z IBM Communications Server Library - white papers, product documentation, etc. IBM Redbooks IBM Communications Server Technical Support Technical Support Documentation (techdocs, flashes, presentations, white papers, etc.) Request For Comments (RFCs) IBM Education Assistant

37 Appendix

38 IP Layer-2 based network interface recovery functions Requirement for this feature to function properly: At least two adapters attached to the same network (broadcast media). Adapters must use either L or QDIO The two adapters should be two physical adapters for real availability benefits MAC: M1 z/os TCP/IP Stack PortA QDIO Inbound to Router-1 PortB QDIO MAC: M2 Inbound to Router's initial ARP Cache IP address M M2 IP address M M2 Mac address Router's ARP Cache after movement of to PortB Mac address 10.x.y.0/24 Example: OSA PortA fails or is shut down 1 The z/os TCP/IP stack moves address to the other QDIO adapter (PortB), which is on the same network (same network prefix) as PortA was. 2 The z/os TCP/IP stack issues a gratuitous ARP for IP address with the MAC address of PortB (M2) over the PortB adapter 3 Downstream TCP/IP nodes on the same subnet will update their ARP caches to point to M2 for IP address and will thereafter send inbound packets for both and to MAC address M2 Friendly advice: Make sure you are current on OSA-Express micro code upgrades! 38

39 Load-balancing outbound IP packets over multiple first-hop routers (MULTIPATH) IPCONFIG MultiPath [PerConnection or PerPacket] z/os x.y.0/24 z/os-2 z/os-1's IP Routing Table (extract) VIPA1: VIPA2: Destination Via /24 Direct delivery Default Default Default / PortA / PortB / Port A PortA Static route definitions on z/os: If an adapter fails in such a way that z/os TCP/IP gets informed, it will skip over the corresponding entries from the routing table If one of the first-hop routers loses its connection to the backbone network or if it "dies" - z/os TCP/IP doesn't know anything about it since it doesn't participate in dynamic routing updates - and it will continue to attempt to use the corresponding routing table entries - connections will time out, UDP packets will be lost, etc. If the two routers deploy VRRP or HSRP between them on the interfaces towards the z/os systems, then the fact that one of them turns into a black hole can be hidden for z/os - z/os continues to send packets to both first-hop addresses, they are just both serviced by the one surviving router Dynamic routing updates: z/os TCP/IP will know both if the adapter itself fails or if the first-hop router fails - and dynamically update the routing table entries and recover from the router outage.. PortB PortC PortD QDIO QDIO QDIO QDIO Default / Port B Be careful if using Multipathing without z/os V1R5 raised the Router-1 number of dynamic multipath dynamic routing! routes from 4 to 16. HSRP/VRRP Router-2 39

40 Some QDIO basics with respect to VIPA addresses 10.x.y.0/24 z/os z/os-2 VIPA1: VIPA2: PortA PortB PortC PortD Grat ARP with: QDIO QDIO QDIO QDIO Grat ARP with: (will be ignored by router) Router-1 Router-2 All HOME IP addresses will be registered in the OATs dynamically by the TCP/IP stacks and the OAT content will be changed as the HOME lists change due to movement of IP addresses. When an IP address is registered, the adapter will do a gratuitous ARP if the address belongs to the same network as to which the adapter is attached (in this example the /24 subnet) or if the address is a VIPA address (independent of which subnet the VIPA address is defined on). Gratuitous ARPs are done for two purposes: to enable downstream routers to update their ARP cache if an adapter malfunctions and the TCP/IP stack decides to move an address to another adapter (example: if PortA fails, then will be moved to PortB and PortB will grat ARP ) - Note that downstream routers normally will ignore gratuitous ARPs for IP addresses that do not belong to the subnet on that physical network (in this example the /24 subnet) to check for duplicate IP addresses on the subnet - will continue for up to 5 seconds, but the adapter will accept incoming packets for the new address immediately 40

41 z/os DNS with MVS Workload Manager Client 1 mvsplex1 LAN Client 2 TN3270. mvsplex1 Not Strategic mvsplex1 TCP/IP WLM TN3270 myserve mvsa TCP/IP WLM DNS TN3270 myserve mvsb TCP/IP WLM myserve mvsc Client 3 mvsa. mvsplex1 What is DNS/WLM? Domain Name Service which interfaces with MVS Work Load Manager Targeted for long duration connections DNS resolution for every connection More availability than DNS round-robin methods Provided caching not done at clients or other DNS nodes Work load distribution on user defined goals Clustered host names, server names or Weighted IP Addresses Benefits of DNS/WLM Distributes connections based on current load and capacity Distributes load across adapters on a host Dynamically avoids crashed hosts and servers Client can reconnect to same Server instance if required Dynamically avoids crashed TCP/IP stacks When using sysplex name Highly scalable New servers added without DNS administration Inexpensive to deploy Uses existing technology ** z/os V1R10 was the last release in which DNS/WLM (BIND 4.9.3) was supported ** 41

42 Replacing the dynamic DNS registration part of the DNS/WLM component with a Dynamic DNS (DDNS-based) solution DDNS registration component will use existing z/os load balancing advisor infrastructure and appear to the load balancing advisor as an external load balancer Potentially possible to extend the dynamic registration capabilities to any SASP-server based implementation, such as a global e-wlm manager. Registration/de-registration triggered by the same events that trigger when a server instance is available/not available from an external load balancer perspective. LBA controls to quiesce and resume server instances also apply to SASP-DDNS. Sysplex-wide scope. Server instance z/os LB agent Central Sysplex-wide definitions of which servers, server groups, and stacks to register under which names and in which name servers (DNS domains). Registration/de-registration driven by start/stop of the actual resources as reported by the LBA infrastructure. Server instance z/os LB agent The z/os load balancing advisor may serve both the SASP DDNS registration component and external load balancers at the same time Server instance z/os BIND 9 DNS Server DNS can be on z/os or any other platform that supports a BIND 8 or later name server DDNS update flows z/os SASP DDNS SASP control flows z/os LB advisor z/os LB agent Private protocol control flows Central configuration file with information to identify which servers, server groups, host groups and individual hosts (TCP/IP stacks) to register dynamically. No requirements to have applications register themselves 42

43 Example of Dynamic Application Domain Name Registration Name Server TCP/IP S1 VIPAs TCP/IP S2 VIPAs Agent TN3270 FTPD Agent Advisor ADNR TN3270 FTPD SYSA SYSB z/os Sysplex Selected name server contents: tn3270.mvsplex.mycorp.com sysa.tn3270.mvsplex.mycorp.com sysb.tn3270.mvsplex.mycorp.com ftp.mvsplex.mycorp.com sysa.ftp.mvsplex.mycorp.com sysb.ftp.mvsplex.mycorp.com

44 DNS/WLM - going away or not going away or what? DNS/WLM implemented two distinct functions: Dynamic name registration of servers, server groups, and TCP/IP stacks Workload balancing based on name resolution requests and interaction with WLM WLM-based TCP/IP workload balancing into a z/os Sysplex is today better handled by more modern technologies, such as Sysplex Distributor or external load balancers using the z/os load balancing advisor technology: Less overhead - balancing at connection set up time and not at name resolution time Not sensitive to DNS caching Better load balancing decisions - the new technologies have more metrics available than DNS/WLM had However, the dynamic name registration capabilities of DNS/WLM are still very useful from an availability perspective and are not replaced by any of the currently available alternative load balancing technologies: Dynamic registration of individual application instances when they start up Dynamic registration of groups of application instances when they start up Dynamic registration of TCP/IP stacks when they start up General dynamic registration in modern DNS servers (BIND 8 or later) is supported by a set of DNS protocols that are known as Dynamic DNS (DDNS) z/os V1R8 implemented a new infrastructure that supports DDNS registration of the same type of entries that were supported by DNS/WLM DDNS is a standard protocol Any DDNS capable name server can be the target of the DDNS registrations 44