Wireless LAN Consortium abgn Infrastructure Interoperability Test Suite v4.4 Report

Transcription

1 Wireless LAN Consortium UNH InterOperability Laboratory 121 Technology Drive, Suite 2 Durham, NH John Vendor Testers, inc. 1 Main St Some City, Some State February 26, 2013 Report Rev. 1.0 Mr. Vendor, Enclosed are the results from the abgn Infrastructure Interoperability testing performed on the: This testing pertains to a set of standard requirements, put forth in the IEEE Std n-2009 Edition. The tests performed are part of the abgn Infrastructure Interoperability Test Suite, which is available on the UNH-IOL s website: ftp://ftp.iol.unh.edu/pub/wireless/testsuites/interop/802.11abgn_infrastructure_interoperability_test_suite_v4.4.pdf Issues Observed While Testing Test #1.1, 1.2, 1.3, 1.4, 1.5 The Ping Loss Percentage was observed to be greater than 5% with AP 3. See these test cases for more details. Test #1.8 The Packet Error Rate was observed to be greater than 5% with AP 3. See test case 1.8 for more details. Test #1.6 The DUT was observed to reject ADDBA requests from the testbed AP. See test case 1.6 for more details. Test #1.9 The DUT was observed to be unable to failover. See test case 1.9 for more details. As always, we welcome any comments regarding this Test Suite. If you have any questions about the test procedures or results, please contact us via at wclab@iol.unh.edu or by phone at Regards, Your Name Mikkel Hagen

2 Digital Signature Information This document was created using an Adobe digital signature. A digital signature helps to ensure the authenticity of the document, but only in this digital format. For information on how to verify this document s integrity proceed to the following site: If the document status still indicates Validity of author NOT confirmed, then please contact the UNH-IOL to confirm the document s authenticity. To further validate the certificate integrity, Adobe 6.0 or later should report the following fingerprint information: MD5 Fingerprint: D 1D 3C A4 1E 95 EE 03 7B 1B 2B 7D SHA-1 Fingerprint: 48 9E 57 F A DA 39 4C B 11 AE 1E 4D 3B 7E Table 1 - Result Key - The following table contains possible results and their meanings. Result PASS FAIL PASS with Warning Informative Refer to Not Applicable Not Available Not Tested Borderline Interpretation The DUT was observed to exhibit conformant behavior. The DUT was observed to exhibit non-compliant behavior. The DUT was observed to exhibit conformant behavior; however, additional explanation of the situation is included. The DUT was observed to exhibit behavior that is not recommended. Results are for informative purposes only and are not judged on a pass or fail basis. From the observations, a valid pass or fail could not be determined. An additional explanation of the situation is included. The DUT does not support the technology required to perform these tests. Due to testing station or time limitations, the tests could not be performed, or were performed in a limited capacity. Not tested due to time constraint of the test period. The observed values of the specified parameter are valid at one extreme, and invalid at the other. Table 2 - The following wireless access points and stations were used during testing. Test System Hardware Driver Version IOL S/N AP Wireless-IOL AP Wireless-IOL AP Wireless-IOL AP C Wireless-IOL AP _ Wireless-IOL AP Wireless-IOL Table 3 - Setup and Configuration Information Product Manufacturer Testers, inc. Model A1234 Firmware MAC Address 01:23:45:67:89 IOL S/N Wireless-IOL Wireless LAN Consortium 1 Report Rev. 1.0

3 GROUP 1: abgn Specific Interoperability Test # and Label 1.1 Initial OOB on Test Procedure Purpose: To test for proper scanning, authentication, association and data exchanges between a wireless station and an access point with Open, and varying other Security mechanisms. This will verify the very basics of authentication, association and data exchanges between a STA and an AP. This test verifies the ability of the AP and STA to communicate by transmitting an ICMP Echo Request from the ETH through the AP to the STA. Various size frames are transmitted including the minimum and maximum ETH frame payload. In all cases, verify that the STA and AP connect properly and the STA responds with ICMP responses and the AP passes ICMP echo requests and responses without error and with a ping loss percentage no greater than 5%. Testbed Access Points AP 1 PASS PASS PASS PASS Not Applicable 2 PASS AP 2 PASS PASS PASS PASS Not Applicable 2 PASS AP Not Not Applicable 2 Applicable 3 AP 4 PASS PASS PASS PASS AP 5 PASS PASS PASS PASS AP 6 PASS PASS PASS PASS Not Applicable 2 Not Applicable 2 Not Applicable 2 PASS PASS PASS 1. The Ping Loss Percentage was observed to be greater than 5%. 2. The DUT was observed to not support server based authentication; therefore this test could not be performed. 3. The testbed AP was observed to not support configuration of Hidden SSID; therefore this test could not be performed. Wireless LAN Consortium 2 Report Rev. 1.0

4 Test # and Label 1.2 Spatial Streams on Test Procedure Purpose: To test for proper data exchanges between a wireless station and an access point while using various MCS Rates within each supported Spatial Stream. This will verify the operations of the various MCS Rates for both STAs and APs. Since most real-world devices will be operated with security, AES CCMP will be enabled for this test. In all cases, verify that the STA and AP connect properly and the STA responds with ICMP responses and the AP passes ICMP echo requests and responses without error and with a ping loss percentage no greater than 5%. The physical layer rates should reflect the utilized number of spatial streams. Testbed Access Points AP 1 PASS PASS Not Applicable 2 Not Applicable 2 AP 2 PASS PASS Not Applicable 2 Not Applicable 2 AP Not Applicable 2 Not Applicable 2 AP 4 PASS PASS Not Applicable 2 Not Applicable 2 AP 5 PASS PASS Not Applicable 2 Not Applicable 2 AP 6 PASS PASS Not Applicable 2 Not Applicable 2 1. The Ping Loss Percentage was observed to be greater than 5%. 2. The DUT was observed to only support two spatial streams; therefore this test could not be performed. Wireless LAN Consortium 3 Report Rev. 1.0

5 1.3 Short Guard Interval on Test Procedure Purpose: To test for proper data exchanges between a wireless station and an access point while using Short Guard Interval. This will verify the operations of the use of Short Guard Interval for STAs and APs. Since most real world devices will be operated with security, AES CCMP will be enabled for this test. In all cases, verify that the STA and AP connect properly and the STA responds with ICMP responses and the AP passes ICMP echo requests and responses without error and with a ping loss percentage no greater than 5%. The physical layer rates should reflect the utilized number of spatial streams. Testbed Access Points Ping Loss Percentages AP 1 PASS - AP 2 PASS - AP 3 FAIL % AP 4 PASS - AP 5 PASS - AP 6 PASS - 1. The Ping Loss Percentage was observed to be greater than 5%. Wireless LAN Consortium 4 Report Rev. 1.0

6 1.4 Channel Width on Test Procedure Purpose: To test for proper data exchanges between a wireless station and an access point while using various channel widths. This will verify the operation of various channel widths between a STA and an AP. 2.4GHz, and 5GHz are defined. All other tests are performed with a single STA and AP. Since most real-world devices will be operated with security, AES CCMP will be enabled for this test. In all cases, verify that the STA and AP connect properly and the STA responds with ICMP responses and the AP passes ICMP echo requests and responses without error and with a ping loss percentage no greater than 5%. The physical layer rate should reflect that a 40MHz channel width is being used in and Testbed Access Points AP 1 PASS Not Applicable 2 PASS Not Applicable 2 AP 2 PASS Not Applicable 2 PASS Not Applicable 2 AP Not Applicable Not Applicable 2 AP 4 PASS Not Applicable 2 PASS Not Applicable 2 AP 5 PASS Not Applicable 2 PASS Not Applicable 2 AP 6 PASS Not Applicable 2 PASS Not Applicable 2 1. The Ping Loss Percentage was observed to be greater than 5%. 2. The DUT was observed to not support 40 MHz operation; therefore this test could not be performed. Wireless LAN Consortium 5 Report Rev. 1.0

7 1.5 Legacy Coexistence on Test Procedure Purpose: To test for proper scanning, authentication, association and data exchanges between a wireless station and an access point with simultaneous 11n and non-11n traffic. This will verify the very basics of authentication, association and data exchanges between 11n and non-11n devices. This test verifies the ability of the AP, STA and LSTA to communicate by transmitting ICMP Echo Requests from the ETH through the AP to the STA and LSTA. Various size frames are transmitted including the minimum and maximum frame payload. In all cases, verify that the STA and AP connect properly and the STA responds with ICMP responses and the AP passes ICMP echo requests and responses without error and with a ping loss percentage no greater than 5%. Testbed Access Points AP 1 PASS PASS AP 2 PASS PASS AP AP 4 PASS PASS AP 5 PASS PASS AP 6 PASS PASS 1. The Ping Loss Percentage was observed to be greater than 5%. Wireless LAN Consortium 6 Report Rev. 1.0

8 1.6 MPDU Aggregation and Block Acknowledgement on Test Procedure Purpose: To test for proper data exchanges between a wireless station and an access point while using MPDU aggregation and Block Acknowledgements. This will verify the operations of A-MPDU and Block Acknowledgment (BA) of 11n devices. Reception of these BA frames is required by 11n devices after all A-MPDUs are transmitted. Since most real-world devices will be operated with security, AES CCMP will be enabled for this test. Verify that the STA and AP connect properly and the STA responds with ICMP Responses and the AP passes ICMP echo requests and responses without error with a ping loss percentage no greater than 5%. Either the STA and AP should setup the BA using an ADDBA Request, ADDBA Response followed by the transmission of the ICMP packets, or the STA and AP should setup the BA using an ADDBA Request, ADDBA Response followed by the transmission of ICMP packets and a BAR and BA. Testbed Access Points AP 1 FAIL 1 AP 2 FAIL 1 AP 3 FAIL 1 AP 4 FAIL 1 AP 5 FAIL 1 AP 6 FAIL 1 1. The DUT was observed to reject ADDBA requests from the testbed AP. Wireless LAN Consortium 7 Report Rev. 1.0

9 1.7 MSDU Aggregation on Test Procedure Purpose: To test for proper data exchanges between a wireless station and an access point while using A-MSDUs. This will verify the operation of A-MSDU for 11n devices. A-MSDU is only required upon reception. Transmission will be tested only on devices that support this feature. ICMP Echo Request frames will be transmitted from the ETH to the STA through the AP and aggregated by either the STA, AP or both. At least one device, that is not the DUT, must be able to transmit A-MSDUs to complete this test. Since most real-world devices will be operated with security, AES CCMP will be enabled for this test. Verify that the STA and AP connect properly and the STA responds with ICMP Responses and the AP passes ICMP echo requests and responses without error with a ping loss percentage no greater than 5%. The STA or AP should aggregate frames into one A-MSDU before transmission. Testbed Access Points AP 1 Not Applicable 1 AP 2 Not Applicable 1 AP 3 Not Applicable 1 AP 4 Not Applicable 1 AP 5 Not Applicable 1 AP 6 Not Applicable 1 1. The testbed AP was observed to not support MSDU aggregation; therefore this test could not be performed. Wireless LAN Consortium 8 Report Rev. 1.0

10 1.8 Packet Error Rate on Test Procedure Purpose: To determine if the DUT can exchange packets with a link partner such that the exchange of packets must produce a packet error rate that is low enough to meet a desired rate. A single vendor s access point will be powered up and the other stations should associate with it. The access point will broadcast the multicast traffic generated by an Ethernet station onto the wireless media. The stations and access point should stay connected and pass traffic with at least 90% efficiency. The PER should be no more than 10% of the total packets sent. This value should be examined with other information gathered during the test process to ensure that the failure is due to bit errors and not resource errors on the testing stations or the distribution system. Testbed Access Points Packet Error Rate AP 1 PASS - AP 2 PASS - AP 3 FAIL % AP 4 PASS - AP 5 PASS - AP 6 PASS - 1. The Packet Error Rate was observed to be greater than 5%. Wireless LAN Consortium 9 Report Rev. 1.0

11 1.9 Failover / Reassociation on Test Procedure Purpose: To observe the behavior of the DUT when an AP within an ESS fails and is forced to Reassociate with another AP within the ESS. Note that this test is only for STA Failover / Reassociation Interoperability testing tests to ensure that a station will scan, assess, and reconnect to other APs if the one it previously was connected to becomes unavailable. If a station loses connection to an AP, it should scan the other available channels until it finds another AP that is an acceptable replacement. After probing and gathering information, a station may associate to this new AP by Authenticating and then using either Reassociation or Association to connect. When the AP within the ESS fails, the STA previously Associated with the AP should Reassociate with the new AP without appreciable packet loss occurring. Testbed Access Points AP 1 FAIL 1 FAIL 1 AP 2 FAIL 1 FAIL 1 AP 3 FAIL 1 FAIL 1 AP 4 FAIL 1 FAIL 1 AP 5 FAIL 1 FAIL 1 AP 6 FAIL 1 FAIL 1 1. The DUT was observed to be unable to failover. Wireless LAN Consortium 10 Report Rev. 1.0

12 1.10 Varying ICMP Payload Ping Loss Threshold on Test Procedure Purpose: To give an informative set of data relating to the ping loss percentage at varying ICMP payload sizes with a constant ICMP Timeout value. This test is informative and as such will not be based on Pass/Fail criteria. This test will show the theoretical limits of interoperability between different AP/STA pairs with a fixed ICMP timeout and increasing payload sizes. Poor results will not necessarily reflect poor operation on the DUT, but could show poor operation between the DUT and a specific device in the testbed. The DUT should respond to ICMP Echo Requests of varying sizes as efficiently as possible. As this test case is informative, there are no strict Pass/Fail criteria Packet Size (Bytes) AP 1 AP 2 AP 3 AP 4 AP 5 AP 6 2, % 0.10% 5.70% 0.05% 1.85% 0.80% 4, % 0.25% 12.10% 0.15% 2.10% 0.55% 6, % 0.40% 13.30% 0.10% 1.50% 0.40% 8, % 0.50% 23.70% 0.10% 1.05% 0.25% 10, % 0.75% 48.50% 0.05% 0.50% 0.50% 12, % 0.75% 99.90% 0.35% 0.50% 0.45% 14, % 1.15% % 0.50% 1.10% 0.60% 16, % 2.30% % 0.35% 0.70% 0.70% 18, % 4.20% % 0.65% 0.55% 1.05% 20, % 4.35% % 0.25% 0.45% 0.75% , % 1.20% 8.80% 0.15% 0.75% 2.70% 4, % 1.60% 9.10% 0.20% 0.55% 1.50% 6, % 1.15% 9.60% 0.05% 1.00% 1.45% 8, % 1.65% 13.10% 0.25% 2.00% 1.70% 10, % 2.35% 55.80% 0.20% 1.05% 2.55% 12, % 0.55% 91.20% 0.15% 1.30% 2.55% 14, % 0.35% 99.40% 0.60% 0.80% 3.05% 16, % 0.35% % 0.40% 0.80% 4.40% 18, % 0.50% % 0.15% 1.55% 4.85% 20, % 0.65% % 0.50% 1.25% 5.55% There were no issues observed during the testing process. Wireless LAN Consortium 11 Report Rev. 1.0

13 1.11 Varying ICMP Timeout Ping Loss Threshold on Test Procedure Purpose: To give an informative set of data relating to the ping loss percentage at varying ICMP Timeout values, with a constant ICMP Payload size. This test is informative and as such will not be based on Pass/Fail criteria. This test will show the theoretical limits of interoperability between different AP/STA pairs with a fixed payload size and decreasing ICMP timeouts. Poor results will not necessarily reflect poor operation on the DUT, but could show poor operation between the DUT and a specific device in the testbed. The DUT should respond to ICMP Echo Requests of varying timeout values as efficiently as possible. As this test case is informative, there are no strict Pass/Fail criteria Timeout (ms) AP 1 AP 2 AP 3 AP 4 AP 5 AP % 4.00% 2.60% 0.90% 0.10% 0.10% % 2.15% 2.80% 0.30% 0.15% 0.80% % 1.10% 4.40% 0.20% 0.10% 0.55% % 1.05% 5.10% 0.10% 0.25% 0.40% % 1.75% 46.20% 0.45% 3.00% 0.25% % 2.90% 59.30% 1.05% 2.25% 0.50% % 8.15% 73.40% 0.85% 1.70% 0.45% % 14.00% 82.70% 0.30% 1.15% 0.60% % 12.40% 96.00% 0.45% 1.85% 0.70% % 7.35% 97.20% 1.55% 4.35% 1.05% % 11.10% % 4.50% 9.60% 0.75% % 17.20% % 6.50% 23.85% 10.55% % 58.10% % 50.70% 55.85% 50.05% % 59.05% % 53.00% 60.70% 60.00% % 0.45% 2.10% 0.55% 0.00% 0.00% % 0.60% 2.10% 0.00% 0.00% 0.00% % 0.40% 2.70% 0.00% 0.00% 0.00% % 1.05% 5.90% 0.30% 0.05% 0.05% % 1.15% 5.10% 0.25% 1.00% 0.75% % 1.50% 66.20% 0.80% 0.90% 1.40% % 1.75% 75.00% 0.95% 1.65% 1.85% % 1.50% 91.90% 0.95% 1.90% 4.30% % 3.30% 99.90% 2.70% 2.00% 6.10% % 7.25% 99.90% 1.50% 5.90% 7.95% % 11.05% % 5.05% 12.70% 12.85% % 21.40% % 8.00% 26.50% 12.05% % 54.60% % 52.00% 52.65% 53.50% % 61.35% % 54.15% 65.65% 67.30% There were no issues observed during the testing process. Wireless LAN Consortium 12 Report Rev. 1.0