TPF Communications - TCP/IP Enhancements

Transcription

1 TPF Communications - TCP/IP Enhancements Jamie Farmer

2 TPF TCP/IP Enhancements 7 New TCP/IP Enhancements All native stack enhancements 4 of the enhancements are TPF Users Group Requirements 3 enhancements are other customer requirements

3 Traffic Limiting : PJ28901

4 Traffic Limiting - Background Connection Limiting PUT 17 APAR PJ28493 Limits the number of active connections for a given application Connection Limiting Limitations Only works for TCP sockets Controls connections only, not traffic rates.

5 Traffic Limiting Limits the amount of inbound traffic for a specific TPF server application Traffic Limiting works for TCP and UDP sockets Works as a resource manager Improves intrusion detection services (IDS) One application cannot use all the resources in the system Statistical information about traffic limits is kept and can be displayed online.

6 Traffic Limiting Options Two traffic limiting options Can limit the amount of traffic for the entire application Can limit the amount of traffic for each socket created for a given server (TCP only) Both can be done together for an application No application programming changes needed When application is over the traffic limit, system will slow the rate of data presented to the application. Works for all read type APIs read(), recv(), recvfrom(), activate_on_receipt(), etc.

7 Traffic Limiting Implementation PUT 16 : TPF delivered the network services database (NSD), APAR PJ28195 Original NSD extended the /etc/services file to support other quality of service (QoS) functions. Ability to define traffic limits for servers defined in the NSD using new parameters APPLRATE Msg/sec limit of inbound traffic for all sockets connected to this server application SOCRATE Msg/sec limit of inbound traffic for each socket connected to this TCP server application Defaults to unlimited when parameters are not coded

8 Sample /etc/services file smtp 25/tcp weight-100 socrate-20 applrate-100 #Mail ftp 21/tcp tos-34 weight-100 maxconnin-5 dns 53/udp weight-100 applrate-50 #DNS tftp 69/udp weight-100 #Trivial File Transfer http 80/tcp weight-100 applrate-100 #World Wide Web pop3 110/tcp weight-100 #Post Office Protocol - Version 3 imap 143/tcp weight-100 #Internet Message Access Protocol snmp 161/udp weight-100 #SNMP snmp-trap 162/udp weight-100 #SNMP trap matipa 350/tcp weight-100 #MATIP Type A matipb 351/tcp weight-100 #MATIP Type B https 443/tcp weight-100 #Secure HTTP rip 520/udp weight-100 #RIP mq 1414/tcp weight-100 #MQ ipbridge 9500/tcp weight-100 socrate-50 #IPBRIDGE tpfar 446/tcp weight-100 maxconnout-50 #TPFAR userappl 9999/tcp weight-100 maxconnin-5 maxconnout-10 #APPL

9 Exceeding the Traffic Limit on TPF Transparent to the application that it has reached a traffic limit. If data arrives for a socket and the application is over its traffic limit, TPF will not present the data to the application until the current 1-second interval expires. For TCP Sockets TCP windowing slows the rate so no data is lost. For UDP sockets If read buffer for the socket fills, system will throw away any new messages received for that socket. Nothing new, UDP works like this today.

10 Read Type API issued with Socket Over the Traffic Limit Socket in nonblocking mode Returns to the application with SOCWOULDBLOCK (no change from today) Socket in blocking mode read(), recv(), recvfrom() ECB is suspended until current 1-second interval elapsed activate_on_receipt (AOR) Return to application immediately when AOR is issued (No Change) Specified application program activated in new ECB after current 1-second interval expires Delays can be longer than 1 second if message arrival rate is more than double the APPLRATE limit.

11 Online Display of Network Services Database : PJ28901

12 Original Network Services Database (NSD) Support For PUT 16, TPF delivered the network services database (PJ28195) Database defining the characteristics of server applications Defined in the /etc/services file Gathers information on an application basis Original NSD counted messages by application for use by data collection

13 Online Display of the Network Services Database Displays all information defined in /etc/services Displays message rates for a given application (similar to data collection) Displays connection limiting statistical information Displays traffic limiting statistical information Displays TCP backlog statistical information

14 Online Display of /etc/services File ZIPDB DISPLAY ALL IPDB0003I 10:24:05 NSD DISPLAY APPLICATION PORT PROTOCOL WEIGHT TOS APPLRATE SOCRATE FARES TCP MQ 1414 TCP TPFAR 5005 TCP FREQFLY 6782 UDP END OF DISPLAY

15 Online Display of Individual Applications User:ZIPDB DISPLAY NAME-TEST System:IPDB0003I 10:02:14 NETWORK SERVICES DATABASE DISPLAY NSD Parameters Traffic Limiting Info Connection Limiting Info TCP Backlog Info NAME - TEST PORT PROTOCOL - TCP WEIGHT TOS - 0 SOCRATE - 50 DEFINED APPLRATE LIMIT 300 CURRENT APPL MESSAGE RATE 123 HIGHEST APPL MESSAGE RATE 222 NUMBER TIMES LIMIT EXCEEDED 0 CONNECTIONS INBOUND OUTBOUND DEFINED LIMIT 100 NONE CURRENT VALUE 5 2 HIGHEST VALUE NUMBER REJECTED 0 0 IP - ANY BACKLOG DEFINED 5 CURRENT BACKLOG 1 HIGHEST BACKLOG 3 CONNECTIONS REJECTED 0

16 Online Display of Message Counts by Application ZIPDB MESSAGES ALL IPDB0004I 10:24:05 BEGIN PROCESSING MESSAGE RATES IPDB0005I 10:24:10 MESSAGE RATES FOR A 5-SECOND INTERVAL INPUT INPUT INPUT OUTPUT OUTPUT OUTPUT APP NAME PORT MSG/SEC PKT/SEC BYTES/SEC MSG/SEC PKT/SEC BYTES/SEC FTP-DATA FTP-CTL TFTP RIP PROGA OTHER TOTAL END OF DISPLAY

17 New API to Read a Complete TCP Message : PJ29118

18 Reading TCP Message TCP architecture has no concept of a message Application data usually contains a header in front of the message containing the length of the message At least two reads for each message One or more reads for the length One or more reads for the message For AOR, multiple ECBs may be created for one message This has been a common cause of error in application programming 4-Byte Header: A read for 4 bytes does not guarantee that 4 bytes will be received.

19 New API to Read a Complete TCP Message Two new APIs created tpf_read_tcp_message - Reads an entire TCP message activate_on_receipt_of_tcp_message - Reads in an entire message using AOR. Performance Improvement Less APIs an application must issue (less dispatching of ECB's) Less ECB creation when using AOR Makes application programming easier Makes converting LU 6.2 applications to TCP/IP easier LU 6.2 has structured data, can replace LU 6.2 read API with the new TCP read message API

20 How Does the Read Message API Work? The format of the message being received is passed to the API Where the message length field within the header resides The message length field does not have to be at the start of the header The system will get the size of the message and read the entire message before passing it to the application The entire message, including the header is passed to the application Partial messages are never passed to the application

21 Example of the read message API Sample Application Message Format TYPE LEN ID MESSAGE 4 bytes 2 bytes 2 bytes Two new parameters on the read message API Offset of Length Field: 4 Length of the length field: 2

22 Comparison to read message API Sample Application Message Format TYPE LEN 10,000 bytes ID Data: 9,992 4 bytes 2 bytes 2 bytes Using traditional socket APIs 1. Set the socket low-water mark to 8 to read the header 2. Issue a read type API for a size of 8 3. Parse the header to determine the length of the entire message 4. Set the socket low water mark to 10,000 bytes to read the message 5. Issue a read type API for a size of 10,000 bytes All of this logic can be replaced with a single TCP read message API

23 Separate TCP Keepalive Timer : PJ28996

24 What is Keepalive? TCP is a connectionless protocol In SNA, if remote node fails, TPF would be notified In TCP, if TPF is waiting for data and the remote fails No way to know the remote is no longer available In this case, the socket can use the keepalive option to send keepalive messages to the remote. Heartbeat messages that determine if remote node is still active.

25 The Keepalive Timer The TCP architecture states that the time-frame between keepalive messages should be set at 2 hours For most platforms, this is not a reasonable time to wait before determining the remote is unavailable User-settable option created on most platforms

26 TPF's Original Keepalive Timer Originally, TPF's keepalive processing was kicked off via the socket sweeper. SOCKSWP parameter of SNAKEY Sweeper is kicked off in increments of minutes Each time the sweeper is kicked off, keepalive messages will be sent for all idle TCP sockets with the setsockopt() option SO_KEEPALIVE set. In some cases waiting for minutes to send out keepalive messages is too long also.

27 TPF's New Keepalive Timer New TPF Keepalive Timer TCPALIVE parameter of SNAKEY Value is in increments of seconds Keepalive processing is no longer performed by the socket sweeper If the TCPALIVE parameter is not specified, it defaults to: Value of SOCKSWP converted to seconds TCPALIVE = (Value of the SOCKSWP parameter) * 60

28 Display of Sockets Using Most IPMT Resources : PJ28997

29 Displaying Heavy IPMT Users IP Message Table (IPMT) is a common pool of storage used by all sockets Running low on IPMT blocks causes problems for all sockets. Currently have a display of how much IPMT storage is in use (ZTTCP DISPLAY STATS) Displays the high-water mark Where are most of the IPMT Blocks? Which sockets are using the most?

30 New IPMT Usage Display Display shows the top x number of sockets that are using the most IPMT resources Includes the total number of IPMT blocks used Number of input IPMT blocks Number of output IPMT blocks Whether the socket is blocked from sending any output messages Display can be used to find troubled sockets and free up IPMT resources

31 Example of ZSOCK IPMT ZSOCK IPMT TOP-5 CSMP0097I CPU-B SS-BSS SSU-HPN IS-01 SOCK0033I BEGIN IPMT USAGE DISPLAY SEND RANK FD BLOCKS INPUT OUTPUT BLOCKED C YES 2 00C YES 3 00C NO 4 00C NO 5 00C02DD NO END OF DISPLAY

32 Problem determination of sockets using IPMT Determine if any data is flowing on the socket ZSOCK DATAFLOW SOCK-C00020 CSMP0097I CPU-B SS-BSS SSU-HPN IS-01 SOCK0024I BEGIN PROCESSING SOCKET DATAFLOW STATISTICS CSMP0097I CPU-B SS-BSS SSU-HPN IS-01 SOCK0025I SOCKET DATAFLOW STATISTICS FOR A 5-SECOND INTERVAL SOCKET DESCRIPTOR-C00020 BYTES SENT - 0 BYTES RECEIVED - 0 END OF ZSOCK DATAFLOW SOCKET DISPLAY No data is flowing, deactivate stalled socket ZSOCK INACT SOCK-C00020 CSMP0097I CPU-B SS-BSS SSU-HPN IS-01 SOCK0016I SOCKET 00C00020 IS NOW INACTIVE

33 Congestion Control and Avoidance : PJ29144

34 Congestion Control and Avoidance RFC 2001 followed by RFC 2581 define ideas for doing congestion control for TCP sockets. Congestion control is done on a socket basis to reduce the amount of congestion in the network Original congestion control algorithms have weaknesses, causing many platforms to design variations of these algorithms Short-lived connections are slowed considerably by congestion control

35 TPF's Implementation of Congestion Control and Avoidance Followed RFC 2581, which defines the concepts TPF modified algorithms to make them more efficient. Implemented a Congestion Control algorithm When TPF determines packets are lost, congestion is likely, so reduce the data rate Implemented a Congestion Avoidance algorithm When TPF detects that congestion is about to occur, reduce the data rate before packets are lost Combination of the two algorithms dramatically improves end to end throughput

36 Slow Start Algorithm Congestion control uses the slow start algorithm to slowly ramp up traffic. Slow start processing may impact performance for short-lived connections. TPF implementation gives user's the ability to turn off initial slow start processing for a given socket. Issue the ioctl() API specifying the TPF_NOSLOWSTART parameter If congestion occurs (packets are lost), slow start will be done for the socket. Regardless of TPF_NOSLOWSTART value

37 Congestion Avoidance Source-based congestion avoidance algorithm TPF will monitor round-trip times (RTT) for the socket If the RTT continues to increase for the socket TPF will adjust the amount of data that can be sent to avoid congestion.

38 Congestion Control and Avoidance Test Environment TPF System OSA TPF System OSA GbE Switch GbE Switch Data Flow Router Router Network Cloud

39 Comparison Testing Congestion Control and Avoidance One TPF system sending 10,000 messages to another TPF system 1400-byte messages Each TPF system running with 1 I-stream Without Congestion Control or Avoidance WithCongestion, without Avoidance With Congestion Control and Avoidance Time to Completion 156 sec 53 sec 45 sec Retransmits

40 Summary Traffic Limiting: PJ28901 Online Display of the Network Sevices Database: PJ28901 Online display of message rates by application Connection and traffic limiting statistical information Online Display of TCP backlog information: PJ28901 New API to read a complete TCP message: PJ29118 Separate TCP keepalive timer: PJ28996 Display of sockets using the most IPMT resources: PJ28997 Congestion Control and Avoidance: PJ29144