Wireless LAN Performance Test Suite using Azimuth W-Series Platform

Transcription

1 Wireless LAN Performance Test Suite using Azimuth W-Series Platform Service Explanation for Applicability and Test Cases ITRI-NCTU Network Benchmarking Laboratory (NBL) 工業技術研究院交通大學網路測試中心 Jun. 15, 2007 Version 1.71 Network Benchmarking Laboratory Service Explanation 1

2 Contact Information ITRI-NCTU Network Benchmarking Laboratory (NBL) 工業技術研究院交通大學網路測試中心 Address: R.604 MIRC, No.1001 Dahsue Rd., Hsinchu City, Taiwan 300, R.O.C. 300 新竹市大學路 1001 號電子資訊研究大樓 604 室 Tel: ext. 201 Fax: Website: Contact Persons Ann Chen, Executive Manager 陳淑宜執行經理 Elia Chen, Chief Technologist 陳世揚技術總監 Abstract This document explains the service applicability and test cases of the Wireless LAN Performance Test Suite using Azimuth W-Series Platform. The application requirement is to provide one to two access points (AP) and/or one station (STA), and the benchmark utilities include Azimuth W-Series Test Platform, Ixia IxChariot, and WildPackets AiroPeek. There are totally 8 test cases (for 4 combinations) can be applied by NCTU NBL, All Rights reserved Some descriptions depend on the Application Notes, 2006 Azimuth Systems Corp. Network Benchmarking Laboratory Service Explanation 2

3 Service Applicability The vendor can provide one to two access points (AP) and/or one station (STA) to apply the performance test suite. Please see the requirements of the test cases on the next page. If not to provide both of AP and STA to NBL, the vendor has to suggest NBL the corresponding wireless device(s) early, and thus some test cases will be applicable by the requirement of the number of devices under test. Access Point(s) under Test (APUT) Must be compatible with Wireless Standards: IEEE , a, b, g, or a/g Distribution Systems (DS): Ethernet Connections by IEEE 802.3, or 802.3u Radio Connectors: RP-SMA, N, RP-N, MMCX, or U.FL/MHF/IPX (Hirose) The Station below to be tested with Station (Client Adapter) under Test (SUT) Must be compatible with Wireless Standards: IEEE , a, b, g, or a/g Host Interfaces: PCMCIA/CardBus, MiniPCI, or USB 1.1/2.0 Radio Connectors: MC Card/SNA (Lucent), U.FL/MHF/IPX (Hirose), or Murata The Access Point(s) above to be tested with Benchmark Utilities Azimuth W-Series Test Platform Including the components of 800W: 8 Slots Chassis 1 MTH-102: 2 Chambers Mini Test Head 1 (for installing APs) RFM-102: RF Port Module 1 (6 ports on the panel for connecting APs) STM-302: 2 Chambers Station Test Module 1 (for installing STA) STM-202: 2 Chambers Station Test Module 1 with Atheros AR5002X cardbus 2 TMM-101: TestMAC Module 1 (can emulate up to 127 clients) Azimuth DIRECTOR v4.2 software with Benchmarking Scripts v.3.4 Ixia IxChariot v6.0 Installed in the Azimuth DIRECTOR Console PC and STM-302 module for generating and analyzing TCP traffic in the rate vs. range test WildPackets AiroPeek Installed in the Azimuth STM-202 module for monitoring frames with 2 Atheros AR5002X in the roaming tests Network Benchmarking Laboratory Service Explanation 3

4 Performance Test Cases Requirements to Apply Test Cases Test Case Combinations Test Case 1 ~ 8 (Full) Test Case 5 ~ 8 Test Case 1 ~ 6 Test Case 1 ~ 4 APUT 1 Application Conditions APUT 2 and SUT 1 or APUT 2 (with other STA 1) or SUT 1 (with other AP 2) Same as above APUT 1 and SUT 1 or APUT 1 (with other STA 1) or SUT 1 (with other AP 1) The test case 1 ~ 4 requires only one APUT, and no SUT is needed. Azimuth TMM is used to emulate stations. 1. AP Association Capacity To measure the maximum associations and concurrent ICMP sessions Requirements AP under test 1 Constants Channel = 6 (802.11/b/g) or 52 (802.11a) Init Client Number = 1 Stop Client Number = 127 Client Number Search Step = 1 Security Type = Open System Variables None Iterations 3 Observations A. Maximum Associations B. Maximum ICMP Sessions 2. AP Association Performance by Client Number To measure the connection time by client number Requirements AP under test 1 Constants Channel = 6 (802.11/b/g) or 52 (802.11a) Security Type = Open System Test Duration = 30 sec Variables Client Number = {1, 3, 7, 15, 31, 63, 127} Iterations 3 Observations A. Maximum/Average Connect Time (sec) vs. client number B. Maximum/Average Authentication Time (sec) vs. client number C. Maximum/Average Association Time (sec) vs. client number Network Benchmarking Laboratory Service Explanation 4

5 3. AP Forwarding Rate by Frame Size To measure the maximum forwarding rate and throughput by frame size Requirements AP under test 1 Constants Channel = 6 (802.11/b/g) or 52 (802.11a) Client Number = 1 Init Itended Load = 100 % Stop Itended Load = 5 % Intended Load Search Step = 5 % Test Duration = 30 sec Variables Frame Size = {64, 128, 256, 512, 1024, 1536, 2048} bytes Traffic Direction = {Downlink, Uplink} Iterations 1 Observations A. Maximum Forwarding Rate (% of Max FPS) vs. frame size B. Maximum Throughput (Mbps) vs. frame size Max FPS is the theoretical maximum frame per second of IEEE series standards 4. AP Forwarding Rate by Client Number To measure the maximum forwarding rate and throughput by client number Requirements AP under test 1 Constants Channel = 6 (802.11/b/g) or 52 (802.11a) Frame Size = 1536 bytes Init Itended Load = 100 % Stop Itended Load = 5 % Intended Load Search Step = 5 % Test Duration = 30 sec Variables Client Number = {1, 3, 7, 15, 31, 63, 127} Traffic Direction = {Downlink, Uplink} Iterations 1 Observations A. Forwarding Rate (% of Max FPS) vs. client number B. Maximum Throughput (Mbps) vs. client number Max FPS is the theoretical maximum frame per second of IEEE series standards Network Benchmarking Laboratory Service Explanation 5

6 The test case 5 ~ 8 requires one or two APUTs and one SUT. The IxChariot traffic generator is used to generate TCP traffic, and the AiroPeek monitor is used to capture the IEEE frames for roaming analysis. 5. Rate vs. Range using Chariot by Channel To measure the TCP throughput vs. path loss (db) by channel Requirements AP under test 1, STA under test 1 Constants Variables Traffic Type = TCP Chariot Script = Throughput.scr Init Path Loss = 48 db Stop Path Loss = 120 db Path Loss Adjust Step = 3 db Duration per Step = 120 sec Channel = {1, 6, 11} (802.11/b/g) or {36, 52, 149} (802.11a) Traffic Direction = {Downlink, Uplink} Iterations 1 Observations A. Throughput vs. path loss and channel B. SUT Data-Link Rate vs. path loss and channel C. SUT RSSI vs. path loss and channel 6. Rate vs. Range using Chariot by Application To measure the TCP throughput vs. path loss (db) by channel Requirements AP under test 1, STA under test 1 Constants Traffic Type = TCP Channel = 6 (802.11/b/g) or 52 (802.11a) Init Path Loss = 48 db Stop Path Loss = 120 db Path Loss Adjust Step = 3 db Duration per Step = 120 sec Variables Chariot Script = { Throughput.scr, Internet\FTPget.scr, Internet\HTTPgif.scr } Traffic Direction = {Downlink, Uplink} Iterations 1 Observations A. Throughput vs. path loss and application B. SUT Data-Link Rate vs. path loss and application C. SUT RSSI vs. path loss and application Network Benchmarking Laboratory Service Explanation 6

7 7. Failover Roaming by Channel Set To measure the roaming time by channel set when the Inter-BSS path loss (db) is suddenly switched Requirements AP under test 2, STA under test 1 Constants Traffic Type = UDP Sending Rate = 100 frame/sec Start Delay = 60 sec Attenuation Delay = 30 sec Test Duration = 120 sec Variables Channel Set = {1+11, 1+6, 6+11} (802.11/b/g) or {36+44, 36+40, 40+44} (802.11a) Iterations 5 Observations A. Average Roaming Time vs. channel set B. Average Transition Time vs. channel set C. Average Authentication Time vs. channel set D. Average Association Time vs. channel set 8. Smooth Roaming by Channel Set Same as the previous test case, but the Inter-BSS path loss (db) is gradually swept Requirements AP under test 2, STA under test 1 Constants Variables Traffic Type = UDP Sending Rate = 100 frame/sec Start Delay = 60 sec Init Path Loss = [Result of Test Case 3] 24 db Stop Path Loss = [Result of Test Case 3] + 12 db Sweep Time = (End Start) Path Loss 2 sec Attenuation Delay = 30 sec Test Duration = 204 sec Channel Set = {1+11, 1+6, 6+11} (802.11/b/g) or {36+44, 36+40, 40+44} (802.11a) Iterations 5 Observations A. Average Roaming Time vs. channel set B. Average Transition Time vs. channel set C. Average Authentication Time vs. channel set D. Average Association Time vs. channel set Network Benchmarking Laboratory Service Explanation 7

8 Appendix For the need to convert the path loss (db) to distance (meters), the free space path loss model can be referred. The conversion is often more effective in the outdoor environment without near obstacles. However, to estimate the indoor distance from the path loss is difficult due to the highly variable radio channel conditions. Free Space Path Loss L FreeSpace 4πfd ( d) = 20log10 c where: f is the carrier frequency, in Hz, d is the distance between the transmitter and receiver antennas, in meters, and c is the speed of light, in meters/second ( ^8 m/s). Solving the Distance L d = 10^ FreeSpace 20log πf c The example values of the path loss and the corresponding distance, while f is 2.44 GHz. Absolute Path Loss (db) Outdoor Distance (m) Indoor Distance (m) (not accurate!) Network Benchmarking Laboratory Service Explanation 8

9 NBL Verification Logo The vendor can explicitly and honestly promote one product to the market if the specifications of the product were verified by the 3rd party testing lab. Thus NBL, a 3rd party testing lab under the non-profit organizations, has developed the Verification Logo Program to aid vendors. NBL has the experience of public benchmarking events and can perform such the verification process to provide the statements on the verification credential for the product under test. For more information, please contact NBL. NBL Wireless LAN Testing The mission of NBL WLAN Testing is to evaluate and review the systems/ components of WLAN devices. In order to improve our testing capability for the WLAN industry in Taiwan, we have to research the features of PHY and MAC innovations and to develop more interesting and valuable methodologies in deeper and further approaches, even to surpass the limitation of test utilities. Now, there is the WLAN test platform which supports stable and configurable RF path/ path-loss, test automation, and multi-client emulation in NBL. Thus, we can provide the WLAN performance testing and achieve more service flexibility. Azimuth W-Series Platform For the need of fair and objective benchmarking for WLAN products, the Azimuth W-Series Test Platform can provide a RF isolated environment which prevents the interference from outer signals and the multi-path effects due to the space. With the Azimuth DIRECTOR console, modules, and chassis, some WLAN tests like capacity, range, and roaming applications can be done more effectively in a limited working place. Especially there is the capability of this platform to flexibly implement methodologies and scenarios, automatically configure WLAN devices, and gracefully perform the testing. For further information, please visit Moreover, NBL is the first testing lab to deploy the test platform in Asia! Network Benchmarking Laboratory Service Explanation 9