IWARP Consortium. Network Impairments Test Suite. Technical Document. Revision 0.3

Transcription

1 IWARP Consortium Technical Document Revision 0.3 iwarp Consortium 121 Technology Drive, Suite 2 Durham, NH University of New Hampshire Phone: Fax: Copyright 2007 University of New Hampshire

2 Modification Record University of New Hampshire Date Revision Comments July 26, Initial version August 23, Added a setup step to all tests and minor grammatical edits and other changes. September 20, Changed percentages on some tests to be more realistic. Changed quantities from percentages to ranges on some tests. iwarp Consortium 2 of 36 iwarp

3 Acknowledgments University of New Hampshire The University of New Hampshire would like to acknowledge the efforts of the following individuals in the development of this test suite: Dustin M. Schoenbrun Mikkel Hagen Allen Hubbe Robert Noseworthy University of New Hampshire University of New Hampshire University of New Hampshire University of New Hampshire iwarp Consortium 3 of 36 iwarp

4 Terminology Acronyms LP TS DUT MIM Link Partner Test Station Device Under Test Man in the Middle iwarp Consortium 4 of 36 iwarp

5 Introduction Overview The University of New Hampshire s (IOL) is an institution designed to improve the interoperability of standards based products by providing an environment where a product can be tested against other implementations of a standard. This test suite has been developed to help implementors evaluate the functioning of their iwarp based products in less than ideal network conditions. The tests do not determine if a product conforms to the iwarp standards, nor are they purely interoperability tests. Rather, they provide one method to isolate problems within an iwarp device. Successful completion of all tests contained in this suite does not guarantee that the tested device will operate with other iwarp devices. However, combined with satisfactory operation in the IOL s semi production environment, these tests provide a reasonable level of confidence that the Device Under Test (DUT) will function well in most multi vendor iwarp environments. iwarp Consortium 5 of 36 iwarp

6 Test Organization University of New Hampshire This document organizes tests by group based on related test methodology or goals. Each group begins with a brief set of comments pertaining to all tests within that group. This is followed by a series of description blocks; each block describes a single test. The format of the description block is as follows: Test Label: Purpose: References: Resource Requirements: Discussion: Test Setup: Procedure: Observable Results: The Test Label is a period delimited list composed of the group number and test number within the group. That is, Test Label Test 1.2 refers to the 2 nd test in the 1 st group of the iwarp Interoperability test suite. The Test Label is immediately followed by a brief test name. The Purpose is a short statement describing what the test attempts to achieve. It is usually phrased as a simple assertion of the feature or capability to be tested. The References section lists cross references to specifications and documentation that may aid in the evaluation of the test and any results it generates. The Resource Requirements must be met before the test can be performed. This includes basic equipment and devices to be tested. The Discussion is a general discussion of the test and relevant section of the specification, including any assumptions made in the design or implementation of the test as well as known limitations. The Test Setup section describes the configuration of all devices prior to the start of the test. Different parts of the procedure may involve configuration steps that deviate from what is given in the test setup. If a value is not provided for a protocol parameter, then the protocol's default is used. The Procedure contains the step by step instructions for carrying out the test. These steps might range from enabling interfaces, disconnecting devices from the network, using a test station to capture and/or transmit data, device configuration, and miscellaneous other test related tasks. The Observable Results section enumerates the specific test results that may be evaluated by the tester in order to determine and outcome for the test (e.g. PASS, FAIL). The section also indicates how each result should be interpreted. The determination of a PASS or FAIL for each test is usually based on how the iwarp Consortium 6 of 36 iwarp

7 observed test results compare with the results described in this section. Possible Problems: This section contains a description of known issues with the test procedure which may affect test results in some cases. iwarp Consortium 7 of 36 iwarp

8 References The following documents are referenced in this text: IETF Internet Draft RDDP DDP, Direct Data Placement over Reliable Transports, version 06. IETF Internet Draft RDDP MPA, Marker PDU Aligned Framing for TCP Specification, version 06. IETF Internet Draft RDDP RDMAP, An RDMA Protocol Specification, version 07. IETF Internet Draft RDDP Security, DDP/RDMAP Security, version 10. iwarp Consortium 8 of 36 iwarp

9 Table of Contents Modification Record...2 Acknowledgments...3 Terminology...4 Acronyms...4 Introduction...5 Overview...5 Test Organization...6 References...8 Default Test Setup...10 Group 1: Basic Network Impairments...11 Overview...11 Test 1.1: Data Corruption...12 Test 1.2: Out of Order Packets...14 Test 1.3: Dynamic MTU Changes...16 Test 1.4: Duplicate Packets...18 Test 1.5: TCP Header Errors...20 Test 1.6: Latency...22 Test 1.7: Jitter...24 Test 1.8: Dropped Packets...26 Test 1.9: Dynamic Window Adjustment...28 Group 2: Marker PDU Alignment (MPA)...30 Overview...30 Test 2.1: Markers...31 Test 2.2: MPA Cyclic Redundancy Check (CRC)...33 Test 2.3: Fragmenting of the MPA Frame...35 iwarp Consortium 9 of 36 iwarp

10 Default Test Setup University of New Hampshire iwarp Consortium 10 of 36 iwarp

11 Group 1: Basic Network Impairments Overview The following tests cover several possible impairment scenarios that could occur between a pair of RDMA Capable devices. These tests are designed to verify that the DUT acts correctly in less than ideal network conditions. iwarp Consortium 11 of 36 iwarp

12 Test 1.1: Data Corruption Purpose University of New Hampshire To verify the DUT properly handles data corruption in the packets during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion June 26, 2007 (dschoenbrun): Initial draft. August 16, 2007 (dschoenbrun): Changed name and added a setup step. It is possible for data to be corrupted en route to its destination. There are several different means by which data corruption can be detected and recovered from, such as checksums and CRCs. This test will verify the DUT's ability to handle data corruption within the packets. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to corrupt 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel using RDMA. Once the transfer has completed, determine the integrity of the data. iwarp Consortium 12 of 36 iwarp

13 Repeat the above for 0.1%, 1%, and 5% of the packets being corrupted. Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel using RDMA. Once the transfer has completed, determine the integrity of the data. Repeat the above for 0.1%, 1%, and 5% of the packets being corrupted. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 13 of 36 iwarp

14 Test 1.2: Out of Order Packets Purpose To verify that the DUT correctly handles packets that arrive out of order during an RDMA Transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 26, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added RDMA references and added a setup step. One of the issues with a busy network is that, on occasion, packets can be transmitted out of order. What this test verifies is that the DUT can handle out of order packets while sending a file to the LP and also while receiving a file from the LP. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to reorder 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel using RDMA. iwarp Consortium 14 of 36 iwarp

15 Once the transfer has completed, determine the integrity of the data. Repeat the above for 0.1%, 1%, and 5% packet reordering. Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel. Once the transfer has completed, determine the integrity of the data. Repeat the above for 0.1%, 1%, and 5% packet reordering. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 15 of 36 iwarp

16 Test 1.3: Dynamic MTU Changes Purpose To verify that the DUT correctly handles dynamic changes in MTU (Maximum Transmission Unit) during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general cleanup. The MTU or Maximum Transmission Unit is the size in bytes of the largest packet that a layer of a communication protocol can pass onwards. This test will change the MTU in a certain percentage of all packets transfered and verify how the DUT handles these changes. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to change 0.01%of the MTU Sizes in the packets Part II: DUT as Receiver Transfer 1GB of data from the LP to the DUT over the test channel. After the transfer of the file has completed, verify the integrity of the data. iwarp Consortium 16 of 36 iwarp

17 Repeat the above for 0.1%, 1%, and 5% of packets with a change in MTU Size. Part III: DUT as Sender Transfer 1GB of data from the LP to the DUT over the test channel. After the transfer of the file has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% of packets with a change in MTU Size. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 17 of 36 iwarp

18 Test 1.4: Duplicate Packets Purpose To verify that the DUT can successfully handle a multitude of duplicate packets during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft August 23, 2007 (dschoenbrun): Added a setup step and general clean up. Another issue with busy networks is duplicate packets. This is when one or more packets get retransmitted due to a timeout when they were actually received on the remote side. This test will determine the DUT's ability to handle different quantities of duplicate packets. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to duplicate 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, determine the integrity of the data. iwarp Consortium 18 of 36 iwarp

19 Repeat the above for 0.1%, 1%, and 5% packet duplication. Part III: DUT is Sender Transfer 1GB of data from the DUT to the LP over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% packet duplication. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems Marker, CRC, and Version supported by the DUT and LP may not be compatible. If possible, configure the DUT and LP such that the devices are in a compatible state. iwarp Consortium 19 of 36 iwarp

20 Test 1.5: TCP Header Errors Purpose To verify that the DUT can correctly handle corruption in the TCP Header during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. The TCP header consists of several fields that determine device capabilities as well as making connections and acknowledging packets. This test is going to verify that the DUT can correctly handle bit flips in the TCP Header. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to corrupt the TCP Header in 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% of TCP Headers being corrupted. iwarp Consortium 20 of 36 iwarp

21 Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% of TCP Headers being corrupted. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems Marker, CRC, and Version supported by the DUT and LP may not be compatible. If possible, configure the DUT and LP such that the devices are in a compatible state. iwarp Consortium 21 of 36 iwarp

22 Test 1.6: Latency Purpose University of New Hampshire To verify that the DUT can correctly handle network latency during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. Network Latency occurs when a network is especially busy and cannot process all requests at the same time. As a result the packets get queued up and also could be dropped. This test will verify the DUT's ability to cope with this latency. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to apply a delay of 1 Millisecond to the connection. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 10, 50, and 100 Millisecond delays. iwarp Consortium 22 of 36 iwarp

23 Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 10, 50, and 100 Millisecond delays. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 23 of 36 iwarp

24 Test 1.7: Jitter Purpose To verify that the DUT can correctly handle network jitter during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. Jitter is the variation in packet delay on a network. It is similar to latency, but can vary greatly over time. This test will verify the DUT's ability to adapt to network jitter. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to apply a 1 to 10 Millisecond delay. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 10 to 50, 50 to 100, and 100 to 250 Millisecond delays. iwarp Consortium 24 of 36 iwarp

25 Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 10 to 50, 50 to 100, and 100 to 250 Millisecond delays. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 25 of 36 iwarp

26 Test 1.8: Dropped Packets Purpose To verify that the DUT can correctly handle dropped packets during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. On a particularly busy network, it is possible for routers and switches to drop packets due to the fact that they cannot handle any more incoming packets at that particular moment. This prevents those packets from being delivered to their destinations. This test will verify the DUT's ability to handle dropped packets. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to drop 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, verify the integrity of the data. iwarp Consortium 26 of 36 iwarp

27 Repeat the above with 0.1%, 1%, and 5% of the packets getting dropped. Part III: DUT as Sender Transfer 1GB of data to the LP from the DUT over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 0.1%, 1%, and 5% of the packets getting dropped. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 27 of 36 iwarp

28 Test 1.9: Dynamic Window Adjustment Purpose To verify that the DUT can correctly handle dynamic changes in window size during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 27, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. The TCP acknowledgment window determines how many packets can be outstanding (have not received ACKs for) at a single time. This test will verify the DUT's ability to handle dynamic changes in the acknowledgment window. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to change the window size on 0.01%of the packets. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 0.1%, 1%, and 5% of the packets having adjusted window iwarp Consortium 28 of 36 iwarp

29 sizes. Part III: DUT as Sender Transfer the 1GB data to the LP from the DUT over the test channel. Once the transfer has completed, verify the integrity of the data. Repeat the above with 0.1%, 1%, and 5% of the packets having adjusted window sizes. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 29 of 36 iwarp

30 Group 2: Marker PDU Alignment (MPA) Overview The following tests cover a multitude of possible issues that could occur while using MPA between a pair of RDMA Capable devices. These tests are designed to verify that the DUT acts correctly in less than ideal network conditions. iwarp Consortium 30 of 36 iwarp

31 Test 2.1: Markers Purpose University of New Hampshire To verify that the DUT correctly handles errors with the markers within the MPA frame during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 30, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. MPA Markers are used to identify the start of FPDUs when packets are received out of order. This test verifies the DUT's ability to handle errors with MPA Markers. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to corrupt 0.01%of packets with MPA Markers. Part II: DUT as Receiver Transfer 1GB of data to the DUT from the LP over the test channel using RDMA and MPA Markers. Once the transfer has completed, verify the integrity of the data. iwarp Consortium 31 of 36 iwarp

32 Repeat the above for 0.1%, 1%, and 5% corruption of MPA Markers. Part III: DUT as Sender Transfer the 1GB data to the LP from the DUT over the test channel using RDMA and MPA Markers. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% corruption of MPA Markers. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems The DUT may not support Markers, in which case the test would not be able to be run. iwarp Consortium 32 of 36 iwarp

33 Test 2.2: MPA Cyclic Redundancy Check (CRC) Purpose To verify that the DUT can handle errors in the CRC in the MPA header during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 30, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. MPA provides a CRC check to detect corruption within an MPA Frame. This test will verify the DUT's ability to handle bad MPA CRCs. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to corrupt 0.01%of MPA CRCs. Part II:DUT as Receiver Transfer 1GB of data from the LP to the DUT over the test channel using MPA and RDMA. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% CRC corruption. iwarp Consortium 33 of 36 iwarp

34 Part III: DUT as Sender Transfer 1GB of data from the DUT to the LP over the test channel using MPA and RDMA. Once the transfer has completed, verify the integrity of the data. Repeat the above for 0.1%, 1%, and 5% CRC corruption. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems It is possible that the DUT has CRC checking disabled. If it is possible, try to configure both the LP and DUT so that they both support CRC checking. iwarp Consortium 34 of 36 iwarp

35 Test 2.3: Fragmenting of the MPA Frame Purpose To verify that the DUT can correctly handle fragmenting of the MPA Frame during an RDMA transfer. References Resource Requirements An RDMA Capable DUT An RDMA Capable LP A 10 Gbps Capable Network Emulator A Testing Station Modification Record Discussion July 30, 2007 (dschoenbrun): Initial Draft. August 23, 2007 (dschoenbrun): Added a setup step and general clean up. It is possible that in a busy network, an MPA frame might become fragmented, splitting its contents across multiple packets. This test will verify the DUT's ability to handle fragmentation of the MPA frame. Test Setup Procedure Default Test Setup. All devices are powered ON and in an operational state. Part I: Setup Configure the test network according to the Default Test Setup. Ensure that the test network is free of any impairments left over from a previous test and that the devices can communicate without any impairments present. Configure the Network Emulator to fragment 0.01%of MPA frames. Part II: DUT as Receiver Transfer the 1GB data to the DUT from the LP over the test channel using MPA and RDMA. Once the transfer has completed, verify the integrity of the data. iwarp Consortium 35 of 36 iwarp

36 Repeat the above with 0.1%, 1%, and 5% of MPA frames being fragmented. Part III:DUT as Sender Transfer the 1GB data to the LP from the DUT over the test channel using MPA and RDMA. Once the transfer has completed, verify the integrity of the data. Repeat the above with 0.1%, 1%, and 5% of MPA frames being fragmented. Observable Results Verify the integrity of the data from the LP. Verify the integrity of the data from the DUT. Possible Problems iwarp Consortium 36 of 36 iwarp