White Paper. IxVeriWave Express Evaluation Test Plan

Transcription

1 White Paper IxVeriWave Express Evaluation Test Plan Agoura Road, Calabasas, CA Tel: Fax: Rev A July, 2011

2 The following test plan has been designed to provide a thorough and efficient four step process for use by organizations undertaking vendor selection of Access Points. Three typical network environments are evaluated as part of the express test plan: residential, hotspot/retail, and enterprise. Contents Part 1: Test Bed Verification... 3 Part 2: Benchmarking... 3 Part 3: Network Environments... 3 Part 4: Over The Air Testing... 3 Three Test Categories... 4 Test Plan Outline... 4 Test Bed Verification/RSSI measurement... 5 Lab Performance Testing... 9 OTA/Interoperability The test plan defines three test criteria and set up for each of the network environment tested: Part 1: Test Bed Verification The verification of cabling, system set up and connectivity are defined and confirmed for each test prior to beginning each test segment. Part 2: Benchmarking The benchmark component of the test plan provides the user with a quick assessment of four critical access point features: Client capacity: this test is used to identify the number of clients that can be associated to the AP with a light traffic load on each client. Throughput: this test provides the throughput of an Access Point as packet sizes are increased (82 byte to 1518 byte) and traffic is increased to theoretical limits for each packet size. Latency: this test presents the upstream and down stream latency of various packet sizes as they are transmitted across the Access Point. TCP Goodput: this test is defined as the actual rate at which a TCP stream can traverse the Access Point without loss. Part 3: Network Environments The client make up and traffic content for each of the three environments (residential, hot spot/retail and enterprise) are predefined by the test plan. The focus of this test section is to establish how the Access Point under evaluation performs when a diverse group of client types and traffic flows are present on the network. Part 4: Over The Air Testing The final step of the test plan introduces external clients selected by the organization performing the test plan. The user should select clients that are typical to their network environment such as PC s, handsets and other network clients. The OTA test verifies the performance of the clients to the AP and provides an indication of the Access Points ability to interoperate with specific client devices. The test plan includes a sequence of step by step instructions to ensure that the system has been correctly configured. In addition to the step by step set up instructions a series of sample reports for each test segment has been included to provide the operator examples of what typical test reports from each section should look like. 3

3 Test AP should be set as follows at start of testing: SSID IxVeriWave Allow clients with class B subnet (address range x.x) No security 2.4 GHz channel 6 Three Test Categories: Residential Single AP, 1 to 15 Clients,.11g legacy and.11n, single and dual radio Hot Spot/Retail Single AP, Multi AP, 1 to 50 Clients,.11g legacy and.11n, single and dual radio Enterprise Single AP, Multi AP, 1 to 25 Clients,.11g legacy and.11n, single and dual radio ECO (Residential, Hot Spot/Retail, or Enterprise with Security) Note: IxVeriWave 3x3 MIMO cards should not be used in the open air. IxVeriWave clients require MIMO card running legacy mode or High Power SISO card for 11n and Legacy background traffic. Test Bed Verification/RSSI measurement EET 001 This test is used to test the correct installation of the IxVeriWave hardware/software while also verifying the test bed is connected and configured properly. WaveApps > Benchmark > Packet Loss (all parameters not mentioned left at default) 1. Open WaveApps 2. Double click on Packet Loss test Fig. 1 Test Plan Outline: 1. Test Bed Verification Physical cabling and termination checklist b. Test Bed Integrity Check/RSSI using Packet Loss Measurements 2. Benchmarking (WaveApps) Client Capacity (no security) Throughput (upstream, downstream, bidirectional 82, 1518 bytes with and without security) Latency (upstream, downstream) 82, 1518 bytes TCP Goodput (upstream, downstream 576, 1518 bytes with and without security ) 3. Real World Experience (WaveQoE - allows SLA measurements, % of Load, R-value, MDI, etc.) Residential (1 to 15 Clients, with and without security) Hotspot/Retail (1 to 50 Clients, with and without security) Enterprise (1 to 25 Clients, with and without security) 3. Ports setup Fig. 2 Figure 1 Enter IP address of IxVeriWave chassis and click Connect Put check marks by Cards to be used in the test (one Ethernet & one Wi-Fi) c. Click Wi-Fi card and set Band to 2.4 GHz and Channel to 6 BSSID and SSID of Test AP should appear Uncheck Rx Attenuation 4. OTA (final validation of lab results to real world using real Laptops and IxVeriWave clients as ECO background) WaveInsite Single and 3 Laptops running UDP/TCP WaveAgent (with Security) WaveQoE Single and 3 Laptops running UDP/TCP WaveAgent (with Security with 4 5

4 Figure 2 4. Click Clients button add 1 group (click green +) Fig. 3 Double click inside Interface cell make one & one a/b/g/n Set both groups to Subnet Mask Figure 4 6. Test click Packet Loss Properties tab Fig. 5 Set Frame Size Custom Custom Frame Size List set to 100 bytes only Set Intended Load List Custom Intended Load List set to 5000 Fps only Confirm Ethernet Base IP Address Set Wi-Fi Base IP Address (no security) Set both groups to Gateway Figure 5 Figure 3 5. Click Test button Test Setup Fig. 4 Set Trial Duration to 10 sec Set Loss Tolerance to.1% Set Aging Time to 2 sec 6 7

5 7. Click Mapping button check Bidirectional Traffic Fig. 6 Figure 6 8. Click Green Start Test Button on GUI Fig. 7 Figure 8 Test should achieve correct forwarding rate with no packet loss and have RSSI between -20 to -35dBm as shown in PDF report. Total Test run time = approx. 26 sec. Lab Performance Testing EET-002 Used to determine maximum number of clients an AP can associate with and still forward a small amount of traffic. Used to insure target number of clients can be supported on test AP. WaveApps Benchmark Maximum Client Capacity (all parameters not mentioned left at default) 1. Open WaveApps 2. Double click on Maximum Client Capacity test Fig Confirm test runs to completion Fig. 8 Figure 7 Figure 9 8 9

6 3. Ports setup Fig. 10 Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Click Wi-Fi card and set Band to 2.4 GHz and Channel to 6. BSSID and SSID of Test AP should appear 5. Click Wi-Fi group click Client Options tab select n Fig. 12 set Guard Interval to short set Channel Bandwidth to 40 check Enable AMPDU Aggregation Uncheck Rx Attenuation 6. Click Test button Test Setup Fig. 13 Set Trial Duration to 10 sec Figure 12 Figure 10 Set Aging Time to 2 sec 4. Click Clients button add 1 group (click green +) Fig. 11 Double click inside Interface cell make one & one a/b/g/n Set both groups to Subnet Mask Set Ethernet Base IP Address Set Wi-Fi Base IP Address (no security) Set both groups to Gateway Figure Test click Maximum Client Capacity Properties tab Fig. 14 Set Search Maximum (Number of Clients) to 254 Set Frame Size (Bytes) to 100 Figure

7 1. Open WaveApps 2. Double click on Throughput test Fig. 16 Figure Click Green Start Test Button on GUI Fig. 15 Figure Ports setup Fig. 17 Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Click Wi-Fi card and set Band to 2.4 GHz and Channel to 6. BSSID and SSID of Test AP should appear Uncheck Rx Attenuation Figure Test will stop at Maximum number of clients associated and forwarded 10 pkts/sec with no more than 50% loss. 10. Single Test run time = approx. 1 min. 23 sec. 11. Rerun above test with WPA2-PSK 12. Rerun above test on 5 Gig band with no security and WPA2-PSK EET-003 This test is used to determine the maximum forwarding rate of UDP packets with a fixed amount of loss for specific frame sizes, for single or multiple clients, with upstream/ downstream /bidirectional traffic, with no security and WPA2-PSK. WaveApps Benchmark Throughput (all parameters not mentioned left at default) Figure

8 4. Click Clients button add 1 group (click green +) Fig. 18 Double click inside Interface cell make one & one a/b/g/n Set both groups to Subnet Mask Click Test button Test Setup Fig. 20 Set Loss Tolerance to.1% Set Aging Time to 2 sec Confirm Ethernet Base IP Address Set Wi-Fi Base IP Address (no security) Set both groups to Gateway Figure Test Click Throughput Properties Fig. 21 Figure Click Wi-Fi group Click Client Options tab select n Fig. 19 set Guard Interval to short Set Custom Frame Size List 82, 1518 bytes Set Binary Search Options Starting Point 10% Set Binary Search Options Search Resolution 1% set Channel Bandwidth to 40 check Enable AMPDU Aggregation Figure 21 Figure Click Mapping button Mapping Options confirm Ethernet to Wireless Click Green Start Test Button on GUI Fig

9 Figure Ports setup Fig. 24 Figure The test should run with results for both frame sizes. Results will vary depending on class and capabilities of the AP. Single Test run time = approx. 7 min. Rerun --- Mapping Mapping Options Wireless to Ethernet Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Click Wi-Fi card and set Band to 2.4 GHz and Channel to 6. BSSID and SSID of Test AP should appear Uncheck Rx Attenuation Rerun --- Mapping Mapping Options Bidirectional Traffic checked Rerun above test with WPA2-PSK security Rerun above test on 5 Gig band with no security and with WPA2-PSK Rerun above with 25 Clients, on both 2.4 GHz and 5 GHz, with no security and with WPA2-PSK. Rerun above Upstream test in 2.4 GHz and 5 GHz bands with no security, 1 client, 82 and 1518 byte frames and with A, C, E Channel Models EET-004 This test is used to determine the latency that would be experienced by the users under loaded conditions. Latency under load has a significant impact on the performance of delay-sensitive applications such as voice and video, as well as TCP. WaveApps Benchmark Latency (all parameters not mentioned left at default) 1. Open WaveApps 2. Double click on Latency test Fig. 23 Figure Click Clients button add 1 group (click green +) Fig. 25 Double click inside Interface cell make one & one a/b/g/n Set both groups to Subnet Mask Confirm Ethernet Base IP Address Set Wi-Fi Base IP Address (no security) Set both groups to Gateway

10 Figure Click Wi-Fi group click Client Options tab select n Fig. 26 set Guard Interval to short set Channel Bandwidth to 40 check Enable AMPDU Aggregation Figure Test click Latency Properties tab Fig. 28 Set Frame Size List Click Frame Size cell set to 82 & 1518 bytes Click ILOAD cell set to 95% of 82 byte throughput #, 95% of 1518 byte throughput # Figure Click Test button Test Setup Fig. 27 Set Trial Duration to 1 min Set Loss Tolerance to.1% Figure Mapping Mapping Options confirm Ethernet to Wireless Fig. 29 Set Aging Time to 2 sec 18 19

11 EET-005 This test is used to determine the maximum forwarding rate of TCP packets with a fixed amount of loss for specific frame sizes, for single or multiple clients. WaveApps Benchmark TCP Goodput (all parameters not mentioned left at default) 1. Open WaveApps 2. Double click on TCP Goodput test Fig. 31 Figure Click Green Start Test Button on GUI Fig. 30 Figure Ports Fig. 32 Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Figure The test should run with results for both frame sizes. Results will vary depending on class and capabilities of the AP. Click Wi-Fi card and set Band to 2.4 GHz and Channel to 6. BSSID and SSID of Test AP should appear Uncheck Rx Attenuation 11. Single Test run time = approx. 2 min. 30 sec. 12. Rerun --- Mapping Mapping Options Wireless to Ethernet 13. Rerun above test with WPA2-PSK security 14. Rerun above test on 5 Gig band with no security and with WPA2-PSK 15. Rerun above with 25 Clients, on both 2.4 GHz and 5 GHz, with no security and with WPA2-PSK 20 21

12 Figure Click Clients button add 1 group (click green +) Fig. 33 Double click inside Interface cell make one & one a/b/g/n Set both groups to Subnet Mask Confirm Ethernet Base IP Address Figure Click Test button Test Setup Fig. 35Set Loss Tolerance to.1% Set Aging Time to 2 sec Set Traffic Type to TCP Set Destination Port to 1023 Set Wi-Fi Base IP Address (no security) Set both groups to Gateway Figure Click Wi-Fi group click Client Options select n Fig. 34 set Guard Interval to short set Channel Bandwidth to 40 Figure Test click TCP Goodput Properties tab Fig. 36 Set Maximum Segment Size to 576, 1460 Set Number of Sessions Per Client to 1 check Enable AMPDU Aggregation 22 23

13 Figure Mapping Mapping Options Ethernet to Wireless Fig. 37 Figure The test should run with results for both frame sizes. Results will vary depending on class and capabilities of the AP. 11. Single Test run time = approx. 4 min. 30 sec. 12. Rerun --- Mapping Mapping Options Wireless to Ethernet 13. Rerun above test with WPA2-PSK security 14. Rerun above test on 5 Gig band, no security and WPA2-PSK 15. Rerun above with 25 Clients EET-006 Figure Click Green Start Test Button on GUI Fig. 38 This test is used to determine the traffic forwarding performance with multiple clients in a mixed traffic environment. WaveQoE User Quality of Experience Enterprise Profile (all parameters not mentioned left at default) 1. Open WaveQoE 2. First screen click apply (make sure Wired and Wireless Testing is enabled)--fig

14 Figure Main Screen click Enterprise Profile Fig. 40 Figure Click Clients button click Laptop Client type click Client Options tab select n Fig. 42 set Guard Interval to short set Channel Bandwidth to 40 check Enable AMPDU Aggregation Figure Ports Fig. 41 Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Click Wi-Fi card and set to Channel 6. BSSID and SSID of Test AP should appear Uncheck Rx Attenuation 26 27

15 Server Network Preferences choose fixed Set Gateway IP to Click Apply to All Server Profiles click Y Figure click Network Preferences button Fig. 43 Client Network Preferences select Fixed Check SSID & choose test SSID (IxVeriWave) Set Gateway IP to Click Apply to All Client Profiles click Yes Figure Click Test button Test Setup Test Parameters set Trial Duration to 5 min. Click Green Start Test Button on GUI Fig. 45 Figure Click Servers button click Network Preferences button -Fig. 44 Figure The test should run with all traffic types passing SLA. Results will vary depending on class and capabilities of the AP

16 10. Single Test run time = approx. 5 min. 45 sec 11. Rerun above test with WPA2-PSK security 12. Rerun above test on 5 Gig band, no security and WPA-PSK 13. Rerun above with 25Meg Traffic Load 4. Ports Fig. 48 Enter IP address of IxVeriWave chassis click Connect Put check by Cards to be used in the test (one Ethernet & one Wi-Fi) Click Wi-Fi card and set to Channel 6. BSSID and SSID of Test AP should appear Uncheck Rx Attenuation EET-007 This test is used to determine the stability of the AP/system while multiple clients associate and disassociate in a mixed traffic environment over a long period of time. WaveQoE Association/Disassociation Stability (all parameters not mentioned left at default) 1. Open WaveQoE 2. First screen click apply (make sure Wired and Wireless Testing is enabled) Fig. 46 Figure Click Clients button click Laptop Client type click Client Options tab select n Fig. 49 Figure Main Screen click Enterprise Profile Fig. 47 set Guard Interval to short set Channel Bandwidth to 40 check Enable AMPDU Aggregation Figure

17 Set Gateway IP to Click Apply to All Server Profiles click Yes Figure Click Network Preferences button Fig. 50 Client Network Preferences select Fixed Check SSID & choose test SSID (IxVeriWave) Set Gateway IP to Figure Click Test button Test Setup Test Parameters set Trial Duration to 5 sec--fig. 52 Set Number of Trials to 100 and ensure that Reconnect Clients for each Trial is checked Click Green Start Test Button on GUI Click Apply to All Client Profiles click Yes Figure Click Servers button click Network Preferences button - Fig. 51 Server Network Preferences choose fixed Figure Single Test run time = approx. 16min. 45 sec 10. Rerun above test with WPA2-PSK 32 33

18 11. Rerun above test on 5 Gig band 10. Main Screen click New Profile Fig Rerun above with 25 Mb/s Traffic Load OTA/Interoperability EET-008 This test is used to test performance of External Clients (Laptop, PDAs, Wi-Fi Phones, etc.) against the test AP system in an Open Air environment. This can be done with single or multiple External Clients. IxVeriWave ECO traffic can also be added to insure more thorough testing. WaveQoE WaveAgent (all parameters not mentioned left at default) 1. Choose Wireless devices that will typically be used in test APs deployed environment (Laptop, etc.). This includes multiple operating systems and NICs. 2. Load WaveAgent on External Wireless device. 3. Give External Wireless device static IP address of Figure In Ports Chassis Enter IP address of IxVeriWave chassis click connect Fig Check Ethernet card and confirm Autonegotiation on 4. Reattach antennas to test AP 5. Reconfigure Test AP to normal open air power levels 6. Associate External Wireless device to test AP 7. Activate WaveAgent 8. Open WaveQoE Fig First screen click apply (make sure Wired and Wireless Testing is enabled) Fig. 53 Figure Click Agents button -> Add WaveAgent Manager (click green +) Fig. 56 Enter static IP address of Enter subnet mask of Enter gateway address of Click OK Figure

19 Figure Right Click Manager_1 click add WaveAgent Fig. 57 Enter IP address (IP of External Wireless device) Click OK Figure Click Servers button Add server profile (click green +) Fig. 59 Change IP addressing to fixed Enter IP address of Enter Net Mask of Figure 57 Enter Gateway IP of >press Enter on keyboard 15. Right Click Manager_1 click Connect Agent Manager Fig. 58 Right Click Manager_1 click Scan All WaveAgents Right Click Agent_1 click Reserve WaveAgent (Agent_1 indicator turns blue Figure

20 17. Click Traffic button Add traffic profile (click green +) Fig. 60 With traffic1 highlighted click in traffic type window and choose AgentUDP Fig. 60 Figure Click Test Button click WaveQoE Properties tab click Choose Traffic Profiles button Fig. 61 Click traffic1 click Select click Create New Mix Add Client/Agent Type by double clicking in field In Select Agents choose Agent_1 click apply Figure Double click in % Traffic field type in 100 press Enter on keyboard Fig. 62 Click Green Start Test Button on GUI 38 39

21 Figure Confirm progress and completion of test in Test Log window Fig. 63 Figure

22 White Paper Ixia Worldwide Headquarters Agoura Rd. Calabasas, CA (Toll Free North America) (Outside North America) (Fax) Other Ixia Contacts Info: Investors: Public Relations: Renewals: Sales: Support: Training: Agoura Road, Calabasas, CA Tel: Fax: Rev A July, 2011