TEST METHODOLOGY. Wireless Networking. v1.0 DECEMBER 5, 2016

Transcription

1 TEST METHODOLOGY Wireless Networking DECEMBER 5, 2016 v1.0

2 Table of Contents 1 Introduction The Need for Wireless Networking About This Test Methodology and Report Inclusion Criteria Test Deployment Product Guidance Recommended Neutral Caution Security Effectiveness Wireless Access Policy Enforcement Open Access Policy MAC Address Filtering WEP WPA/WPA2 Personal (PSK) WPA/WPA2 Enterprise (802.1x/RADIUS/EAP) Protected Management Frames (802.11w) (Optional) Virtual LAN (VLAN) and Guest Isolation VLAN Guest Isolation User Isolation Wireless Intrusion Prevention Systems (WIPS) Functionality (Optional) Rogue Access Point Detection Rogue Clients Frame-Based DOS Detection Radio-Based DoS Detection Intrusion Prevention (Optional) Exploit Library False Positive Testing Coverage by Attack Vector Coverage by Impact Type Coverage by Date Coverage by Vendor Coverage by Result Policy enforcement Application Control Performance Signal Wireless Networking Test Methodology v1.0_

3 4.1.1 Measured Distances Signal Degradation Capacity and Throughput Maximum Throughput with users Maximum number of clients Latency Impact of Wireless-Based Latency on VoIP Connections Impact of Wireless-Based Latency on Video Calls Roaming Roaming with VoIP Roaming with Video Call Latency Due to Roaming Latency Due to Secure Roaming Stability and Reliability Persistence of Data Total Cost of Ownership and Value Contact Information Wireless Networking Test Methodology v1.0_

4 1 Introduction 1.1 The Need for Wireless Networking Wireless networking is the fastest growing segment of the enterprise networking infrastructure. Many enterprises are planning networking growth with a wireless first approach, and wireless connections are quickly replacing wired connections. This along with increased adoption of BYOD (bring your own device) and the growth of the Internet of Things will dramatically increase utilization of the wireless infrastructure. To this end, NSS Labs has developed a methodology that tests the security of wireless networks in the enterprise. 1.2 About This Test Methodology and Report NSS Labs test reports are designed to address the challenges faced by enterprise security and IT professionals in selecting and managing security products. The scope of this methodology includes: Security effectiveness Performance Stability and reliability Total cost of ownership (TCO) Wireless devices are deployed as part of the network infrastructure; therefore, the stability and reliability of the wireless infrastructure is imperative particularly in organizations that have a wireless first networking model. Regardless of any new capabilities a wireless device may possess, the main requirement is that it must be stable and reliable. NSS considers the following capabilities essential in any modern wireless infrastructure: Wireless access point (AP): o ac capable o Multi-radio o Capable of being deployed at multiple access points o Highly stable o Able to interoperate with external authentication systems o Possesses backward compatibility with a/b/g/n o Capable of having fully segmented guest network Wireless AP controller o Centralized management for wireless access points o Support for more than one access point via central management o Centralized alert management o Centralized policy management 1.3 Inclusion Criteria To encourage the greatest participation, and to allay any potential concerns of bias, NSS invites all vendors claiming enterprise-grade wireless capabilities, and that meet the above criteria, to submit their products at no cost. Vendors with major market share, as well as challengers with new technology, will be included. Wireless Networking Test Methodology v1.0_

5 Once a system is installed in the test lab, it will be configured for the use case appropriate to the target deployment. 1.4 Test Deployment The system should be configured to provide coverage to 18,000 square feet in a real-world corporate environment. During initial configuration, normal background radio noise will exist, and regardless of whether devices are manually configured or automatically configured, vendors should take environmental factors into account. Wireless Networking Test Methodology v1.0_

6 2 Product Guidance NSS issues summary product guidance based on evaluation criteria that is important to information security professionals. The evaluation criteria are weighted as follows: Security effectiveness The purpose of a wireless deployment is to provide secure wireless access to the corporate network, while also allowing guest access that is isolated from the corporate network. Stability Long-term stability is particularly important for an access device. Performance A wireless system should be correctly sized to ensure network connectivity. Value Customers should seek low total cost of ownership (TCO) and high effectiveness and performance rankings. Products are listed in rank order according to their guidance ratings. 2.1 Recommended A Recommended rating from NSS indicates that a product has performed well and deserves strong consideration. Only the top technical products earn a Recommended rating from NSS, regardless of market share, company size, or brand recognition. 2.2 Neutral A Neutral rating from NSS indicates that a product has performed reasonably well and should continue to be used if it is the incumbent within an organization. Products that earn a Neutral rating from NSS deserve consideration during the purchasing process. 2.3 Caution A Caution rating from NSS indicates that a product has performed poorly. Organizations using one of these products should review their security posture and other threat mitigation factors, including possible alternative configurations and replacement. Products that earn a Caution rating from NSS should not be short-listed or renewed. Wireless Networking Test Methodology v1.0_

7 3 Security Effectiveness This section verifies that the system is capable of effectively enforcing a specified security policy. 3.1 Wireless Access Policy Enforcement Policies are rules that are configured on a wireless system; they are, designed to permit or deny access by a client to a network via a wireless access point. They can be based on authentication, identity, or other mechanisms Open Access Policy No encryption or authentication is required MAC Address Filtering Clients are identified by their MAC addresses, and they are whitelisted or blacklisted based on these identities WEP WEP is an older security protocol, which is flawed and should no longer be utilized. Therefore, it will not be tested by NSS WPA/WPA2 Personal (PSK) Networks can be secured using WPA/WPA2 Personal, a pre-shared key (PSK) authentication method, where a password is required to access a network WPA/WPA2 Enterprise (802.1x/RADIUS/EAP) WPA/2 Enterprise relies on external authentication systems, such as RADIUS, EAP, or 802.1x Protected Management Frames (802.11w) (Optional) The system will be tested to see if it supports w, also known as protected management frames (PMF). This protection also encrypts the management frames, making attacks directed at the AP much more difficult. 3.2 Virtual LAN (VLAN) and Guest Isolation Guest networks are common in the enterprise and are relied upon to provide network access to visitors. These networks must be isolated from an enterprise s corporate network VLAN The system should be able to correctly segment traffic via VLAN Guest Isolation Users connected to the guest wireless network should be isolated from the corporate network User Isolation It should be possible to configure the access point such that wireless users can be isolated from each other. Wireless Networking Test Methodology v1.0_

8 3.3 Wireless Intrusion Prevention Systems (WIPS) Functionality (Optional) The system will be tested for WIPS functionality Rogue Access Point Detection The WIPS will be tested for the following features that assist in remediating the challenges of rogue access points: Detection Notification Isolation Triangulation Rogue Clients The system will be tested for its ability to identify, alert, and triangulate on a rogue client Frame-Based DOS Detection The system will be tested for its ability to identify and alert on frame-based DOS attacks. In such attacks, the attacker creates standard frames that, when used in specific ways, can cause interruption in network access. This test will utilize deauthentication frames to cause a denial of service (DoS) Radio-Based DoS Detection The system will be tested for its ability to identify and alert on radio-based DoS attacks. In such attacks, the attacker creates a large amount of radio noise that overwhelms the true signal and causes loss of connectivity. 3.4 Intrusion Prevention (Optional) In this test, policies consisting of threat protection signatures verify that the system is capable of correctly blocking malicious traffic based on a comparison of packet/session contents against signatures/filters/protocol decoders. The latest signature pack is acquired from the vendor s support site, and the system is deployed with the out-ofthe-box recommended or default security policy. No tuning of the product is allowed by the vendor. NSS considers it unacceptable for a product of this nature to be sold without a recommended or default policy. No custom signatures are permitted during testing. All signatures used must be available to the general public at the time of testing. The intrusion prevention system (IPS) within the WLAN engine is required to block and log exploit attempts and hostile traffic. Wireless Networking Test Methodology v1.0_

9 3.4.1 Exploit Library NSS security effectiveness testing leverages the deep expertise of our engineers who utilize multiple commercial, open-source, and proprietary tools. With thousands of exploits, this is the industry s most comprehensive test to date. Most notably, all of the live exploits and payloads in the NSS exploit test have been validated in our lab such that one or more of the following is true: A reverse shell is returned A bind shell is opened on the target allowing the attacker to execute arbitrary commands Arbitrary code is executed A malicious payload is installed A system is rendered unresponsive Etc. This test goes far beyond replaying packet captures or pressing the button on a test tool. In short, NSS engineers trigger vulnerabilities for the purpose of validating that an exploit is able to pass through the system False Positive Testing The ability of the system to identify and allow legitimate traffic while maintaining protection against threats and exploits is just as important as its ability to protect against malicious content. This test will include a varied sample of legitimate application traffic, which should be identified and allowed, or blocked, based on policy rules Coverage by Attack Vector Threats and exploits can be initiated either by the target or by the attacker targeting either local or remote vulnerabilities; therefore, NSS categorizes threats and exploits into the following matrix: Network Local Attacker RPC Exploit Root Kit Target Browser Exploit Trojan *Example exploits included for reference purposes Attacker Initiated Also referred to as server-side exploits, the threat/exploit is executed remotely by the attacker against a vulnerable application and/or operating system Target Initiated The threat/exploit is initiated by the vulnerable target. The attacker has little or no control as to when the target user or application will execute the threat Network Threats/exploits that are initiated as a result of network communication. Wireless Networking Test Methodology v1.0_

10 Local Local execution that requires existing access to the target. Protective ratings are reported in raw percentages of mitigated attacks and their resulting impact: system, service, fault, reconnaissance. Although a system or service exploit may be partially mitigated by the system, the service could crash as a result of residual communications causing a fault impact on the service or operating system Coverage by Impact Type The NSS threat and attack suite contains thousands of publicly available exploits (including multiple variants of each exploit) from which groups of exploits are carefully selected to test based on appropriate usage. Each exploit has been validated to impact the target vulnerable host(s). Based on the impact of the threat against the target, the following metrics are reported: System Exposure Attacks resulting in remote system compromise and the ability of the attacker to execute arbitrary system-level commands. Most exploits in this class that are weaponized will provide the attacker with a fully interactive remote shell on the target client or server Service Exposure Attacks resulting in an individual service compromise but not arbitrary system-level command execution. Typical attacks in this category include service specific attacks such as SQL injection, which enable the attacker to execute arbitrary SQL commands within the database service. These attacks are somewhat isolated to the service and do not immediately result in full system-level access to the operating system and all services. However, if attackers use additional localized system attacks, it may be possible for them to go from the service level to the system level System or Service Fault Attacks resulting in a system or service-level fault that crashes the targeted service or application and requires administrative action to restart the service or reboot the system. These attacks do not enable the attacker to execute arbitrary commands. However, the resulting impact to the business could be severe given that the attacker could crash the protected system or service Coverage by Date The typical enterprise will run a mix of both old and new applications, and NSS research shows that crimeware kits will frequently include exploits that date back several years. Therefore, NSS security effectiveness testing will include exploits current at the time of the test, as well as exploits targeting vulnerabilities covering multiple years dating backwards from the time of the test. Results will be reported by year for up to 10 years prior to the year of the test. Where applicable, results prior to that time period will be aggregated into the oldest bucket. Wireless Networking Test Methodology v1.0_

11 3.4.6 Coverage by Vendor NSS live exploit test contains many vendors, including but not limited to the following. Within the scorecard, protection capabilities are indicated as percentages. 3Com Adobe Alt-N Apache Apple Atrium Avast BEA BitDefender Borland CA Cisco Citrix ClamAV EMC Facebook GNU Google HP IBM IPSwitch ISC Kaspersky LanDesk lighttpd Linux Macromedia MacroVision Mailenable McAfee Mercury Microsoft MIT Mozilla Mplayer Multiple Vendors MySQL NOD32 Novell Nullsoft OpenLDAP OpenOffice OpenSSH OpenSSL Oracle Other Misc Panda RealNetworks Samba SAP Snort Sophos SpamAssassin Squid Sun Microsystems Symantec Trend Micro Trillian UltraVNC Veritas VideoLan VMWare WinAmp WinFTP Winzip Yahoo Coverage by Result The following results of exploitation are represented in NSS live exploit test. Within the scorecard, protection capabilities are indicated as percentages Arbitrary Code Execution This is a software bug that allows an attacker to execute any commands of an attacker s choice on a target machine or in a target process. Wireless Networking Test Methodology v1.0_

12 Buffer Overflow This is the exploitation of a software bug that occurs due to improperly establishing memory bounds allows an attacker to overwrite adjacent memory and execute a command Code Injection This is the exploitation of a software bug that allows for the processing of invalid data within a program. Code injection can be used by an attacker to introduce code into a computer program to change the course of execution Cross-Site Script This is the exploitation of a web application that enables attackers to insert malicious script into web pages, which can then be executed by other users Directory Traversal This is the exploitation of a lack of security in an application (as opposed to exploiting a bug in the code) that allows user-supplied input with characters representing traverse to parent directory to be passed to the file APIs. The goal of this attack is to order an application to access a file or executable that is not intended to be accessible Privilege Escalation This exploit type allows an attacker to gain access to resources that would not normally have been available Target Type The following web target types are represented in NSS live exploit testing. Within the scorecard, protection capabilities are indicated as percentages. Web server ActiveX Browser plug-ins/add-ons Web browser JavaScript 3.5 Policy enforcement Application Control Where possible, the access point should be configured to block or allow access to applications. Wireless Networking Test Methodology v1.0_

13 4 Performance This section measures the performance of the system using various traffic conditions that provide metrics for realworld performance. Individual implementations will vary based on usage; however, these quantitative metrics provide a gauge as to whether a particular system is appropriate for a given environment. 4.1 Signal This test will utilize a spectrum analyzer to check the level of signal in both the 2.4 Ghz and 5 Ghz bands in order to provide the highest level of network performance and the lowest latency Measured Distances Meter Signal strengths will be recorded at one meter Meters Signal strengths will be recorded at 10 meters Meters Signal strengths will be recorded at 50 meters Signal Degradation At what distance does signal degradation impact performance? 4.2 Capacity and Throughput These tests are used to determine the impact of the user load on the wireless deployment Maximum Throughput with users. This test evaluates the throughput of a single ac access point at one meter with an increasing number of users. Measurement is taken from a single client Single user users users users users users Wireless Networking Test Methodology v1.0_

14 4.2.2 Maximum number of clients How many clients can a single access point support? 4.3 Latency The goal of the latency and user response time tests is to determine the effect the wireless network has on traffic passing through it under various load conditions Impact of Wireless-Based Latency on VoIP Connections This test assesses the impact of wireless-based latency on VoIP connections Impact of Wireless-Based Latency on Video Calls This test assesses the impact of wireless-based latency on streaming video. 4.4 Roaming The use of roaming technologies should for allow seamless movement from one access point to another, without loss of connectivity. This should apply even in high-security networks. These test monitor and log the impact of roaming from one access point to another while performing normal tasks Roaming with VoIP This test will quantify the impact on a VoIP session when a client transitions from one access point to another Roaming with Video Call This test will quantify the impact on a high-bandwidth video call session when a client transitions from one access point to another Latency Due to Roaming This test will measure overall latency when a client transitions from one access point to another Latency Due to Secure Roaming. This test will measure latency when a client is roaming in a secure environment. Wireless Networking Test Methodology v1.0_

15 5 Stability and Reliability Long-term stability is particularly important for an infrastructure device, where failure can produce network outages. Products that are not able to sustain legitimate traffic (or that crash) while under hostile attack will not pass. The device is required to remain operational and stable throughout these tests. 5.1 Persistence of Data The system should retain all configuration data, policy data, and locally logged data once restored to operation following power failure. Wireless Networking Test Methodology v1.0_

16 6 Total Cost of Ownership and Value Implementation of security solutions can be complex, with several factors affecting the overall cost of deployment, maintenance, and upkeep. All of the following should be considered over the course of the useful life of the solution: Product Purchase The cost of acquisition. Product Maintenance The fees paid to the vendor (including software and hardware support, maintenance and other updates). Installation The time required to take the device out of the box, configure it, put it into the network, apply updates and patches, and set up desired logging and reporting. Upkeep The time required to apply periodic updates and patches from vendors, including hardware, software, and other updates. Wireless Networking Test Methodology v1.0_

17 Contact Information NSS Labs, Inc. 206 Wild Basin Road Building A, Suite 200 Austin, TX USA This and other related documents available at: To receive a licensed copy or report misuse, please contact NSS Labs NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, copied/scanned, stored on a retrieval system, ed or otherwise disseminated or transmitted without the express written consent of NSS Labs, Inc. ( us or we ). Please read the disclaimer in this box because it contains important information that binds you. If you do not agree to these conditions, you should not read the rest of this report but should instead return the report immediately to us. You or your means the person who accesses this report and any entity on whose behalf he/she has obtained this report. 1. The information in this report is subject to change by us without notice, and we disclaim any obligation to update it. 2. The information in this report is believed by us to be accurate and reliable at the time of publication, but is not guaranteed. All use of and reliance on this report are at your sole risk. We are not liable or responsible for any damages, losses, or expenses of any nature whatsoever arising from any error or omission in this report. 3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY US. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT, ARE HEREBY DISCLAIMED AND EXCLUDED BY US. IN NO EVENT SHALL WE BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, PUNITIVE, EXEMPLARY, OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software) tested or the hardware and/or software used in testing the products. The testing does not guarantee that there are no errors or defects in the products or that the products will meet your expectations, requirements, needs, or specifications, or that they will operate without interruption. 5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in this report. 6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their respective owners. Wireless Networking Test Methodology v1.0_