1. DUT and Test Equipments Functionality Test... 5

Transcription

1 1. DUT and Test Equipments... 3 DUT Specification... 3 Test Equipments Specification Functionality Test... 5 Management Interface Evaluation... 5 MIB Object Evaluation (under revision)... 7 Cable Diagnostics Test... 8 VLAN Test... 9 Private/802.1Q-in-Q VLAN Test Link Aggregation Test Port Counter Test and Mirror Port Test MAC Address Table Test Address Lookup Test Storm Control Test Rate Limit Test DiffServ Classification Test IGMP Snooping Test Multicast Protocol Snooping Test X Test DHCP/BOOTP Test RADIUS Server Test DNS Server/Client Test SNTP Test Access Control List Test Firmware/Configuration Upgrade Test Performance Test Forwarding Test Congestion Control Test Address Caching Capacity Test Address Learning Rate Test Errored Frame Filtering Test Broadcast Frame Forwarding Test Broadcast Latency Test Forward Pressure Test Quality of Service Test Conformance Test Network Benchmarking Lab 1

2 Spanning Tree Protocol Validation Rapid Spanning Tree Protocol Validation Multiple Spanning Tree Protocol Validation Virtual Local Area Network Validation Remote Monitoring Validation Interoperability Test Spanning Tree Protocol Rapid Spanning Tree Protocol Virtual Local Area Network Network Benchmarking Lab 2

3 1. DUT and Test Equipments DUT Specification Hardware: Layer 2 Switch 24 Gbps ports 1 RS232 port Firmware: Ver Network Benchmarking Lab 3

4 Test Equipments Specification Spirent SmartBits: SmartBits 6000B Chassis TeraMetrics XD 3324A 10/100/1000 Mbps Ethernet Modules AST II 3.1 SmartBits SmartFlow 5.00 SmartWindow 9.0 TeraRouting Tester 5.0 Ixia IxANVL (Automatic Network Validation Library) 6.3: Spanning Tree Protocol Validation Rapid Spanning Tree Protocol Validation Multiple Spanning Tree Protocol Validation Virtual Local Area Network Validation Remote Monitoring Validation Other Tools: Web Browsers: (1) Internet Explorer xpsp_sp2 (2) Firefox (3) Opera 8.5 SecureCRT 3.4 Windows 2003 Server + IAS (RADIUS Server) + DHCP Server Windows XP Professional MIB Browser, MIB Walker Net-SNMP 5.3 TFTP Server Real World SNTP Server Real World DNS Servers Network Benchmarking Lab 4

5 2. Functionality Test Management Interface Evaluation This test evaluates the easiness and correctness of the management interfaces. This test covers the management interface of CLI (Command Line Interface) by Serial Port (RS232) and Telnet, Web, and SNMP (Simple Network Management Protocol), and performs error input checking for CLI and Web interface. SecureCRT Internet Explorer Firefox Opera SNMP Browser SNMP Walker The test configuration of this test is shown in Figure 1. The parameters of the DUT are configured through a management interface. Then, the configured parameters are checked by another management interface to see whether they are operational. After all the parameters are configured and checked, the evaluation of the two management interfaces is done. Additionally, for using Telnet, other 4 sessions are co-existent. The same procedures are repeated until all the management interfaces are evaluated. The result of a configuration should be consistent with all of the management interfaces. The CLI and Web are typed in different error syntax, and test the DUT whether it can determine. Figure 1 Management Interface Evaluation 1. The configuration status and result should be operational and consistent no matter performing these though the CLI or the Web. Network Benchmarking Lab 5

6 2. Sane as above but for different web browsers. 3. Sane as above but using Telnet when other 4 Telnet DUT sessions are opened. 4. The function of SNMP agent should be operational. 5. The problem or somewhere else needed to be improved for management. Network Benchmarking Lab 6

7 MIB Object Evaluation (under revision) This test evaluates the accessibility of the MIB objects of the DUT. This test covers the groups of 802.1X, GMRP, MIB II, Bridge, Etherlike, and IGMP. Net-SNMP Command Tools Perl scripting by NBL The test configuration of this test is shown in Figure 2. The objects in the MIB groups are read according to the MIB definitions. If the access attribute is writable, the object is written and then read to check the correctness. All MIB objects should be accessible. Figure 2 MIB Object Evaluation Network Benchmarking Lab 7

8 Cable Diagnostics Test This test validates the cable diagnostics function of the DUT. The diagnostics includes the connectivity and the pair length. No specific tool The test configuration of this test is shown in Figure 3. Different RJ-45 cable length: 1, 3, 10, 100 meters are used to connect the DUT and others devices, and then DUT shows the connection status and the pair length. Figure 3 Cable Diagnostics Test Configuration The connection status and pair length should be correct. Network Benchmarking Lab 8

9 VLAN Test This test validates the VLAN capacity that DUT declares to support. SmartFlow The configuration of this test is shown in Figure 4. The two ports of the tester (T1 and T2) are connected to the two ports of the DUT (D1 and D2) respectively. VLANs are created on the DUT, and the number of VLAN created is the capacity that the DUT claims to support. The port D1 and D2 are associated to the created VLANs; that is, the two ports are VLAN trunk ports that can accept untagged and tagged frames. The port T1 sends tagged and untagged frames to the port D1. The sent frames contain the VLAN IDs ranging from 1 to the number of the VLAN capacity the DUT claims to support. The DUT forwards the frames to the port D2. The tester checks the frames received from the port T2 and validates the VLAN capacity the DUT claims to support. Figure 4 VLAN Test Configuration The DUT should forward both the untagged frames and tagged frames containing the VLAN IDs ranging from 1 to the number of the VLAN capacity. Network Benchmarking Lab 9

10 Private/802.1Q-in-Q VLAN Test This test validates the function of private and 802.1Q-in-Q VLAN of the DUT. SmartWindow The configuration of this test is shown in Figure 5. Three VLANs are created to perform private and Q-in-Q VLAN test, namely VLAN 2, VLAN 11 and VLAN 12 respectively. Ports D1 to D4 are first associate with VLAN 2. In addition, ports D1 and D3 are associated with VLAN 11, and ports D2 and D4 are associated with VLAN 12. Traffic with VLAN 11 and VLAN 12 tag is sent from tester port D1 and D2 to the DUT to verify the private/802.1q-in-q functionality. Port T5 acts as monitor port to observe any flooding from the DUT. Figure 5 Private/802.1Q-in-Q Test Configuration 1. The DUT should forward the traffic with the VLAN tag 11/12 to the correct port for the private VLAN configuration. 2. Same as above but for the 802.1Q-in-Q VLAN configuration. Network Benchmarking Lab 10

11 Link Aggregation Test This test validates the link aggregation by static and dynamic (LACP) configuration. The ports in the same channel group can share the traffic of the group. SmartFlow The configuration of this test is shown in Figure 6. This test needs two DUTs. The four ports of the DUT1 (D1-D4) are connected to the four ports of the DUT2 (D6-D9). The four links between the DUT1 and DUT2 are configured to form an aggregated link. The two ports of tester (T1 and T2) are connected to the DUT1 (D5) and DUT (D10). The port T1 sends traffic to the DUT1 and receives the forwarded traffic from the port T2. The amounts of the traffic the ports D6-D9 received are checked to see whether the traffic is distributed among the four links. After this, the above procedures are repeated with the scenarios of removing or recovering a link. This is for the purpose of validating the robustness of the link aggregation of the DUT. Figure 6 Link Aggregation Test Configuration 1. The traffic is fairly distributed among the links and transmits correctly in case of the breakdown or recovery of one link for the static configuration. 2. Same as above but for the dynamic (LACP) configuration. Network Benchmarking Lab 11

12 Port Counter Test and Mirror Port Test This test validates the port counter accuracy and the mirror function. SmartWindow The configuration of this test is shown in Figure 7. The two ports of the DUT (D1-D2) are connected to the four ports of the tester (T1-T2). The parameter of mirror is configured on the DUT. The source port of the DUT is D1; destination port of the DUT is D2. The tester sends three different traffics, unicast, multicast, and broadcast from the port (T1) to the port (T2). Figure 7 Port Counter Test and Mirror Port Test Configuration 1. The port counter should be accurate on the web. 2. The mirror port data should be the same as what sent to the DUT port (D1). Network Benchmarking Lab 12

13 MAC Address Table Test This test validates the manually editable MAC address table. The editable MAC entry includes unicast and multicast address for the corresponding traffic. SmartFlow The test configuration of this test is shown in Figure 8. After different MAC entry for the specified unicast or multicast address and the association with port-list/vlan is entered on the DUT, the tester sends the corresponding traffic to verify the entry. Figure 8 MAC Address Table Test Configuration 1. The unicast traffic can be received on the correct port or VLAN. 2. Same as above but for the multicast traffic. Network Benchmarking Lab 13

14 Address Lookup Test This test validates the address lookup (ARL) function when the new MAC address is added into the MAC address table manually or by dynamic learning. SmartWindow The configuration of this test is shown in Figure 9. The two ports of the DUT (D1-D2) are connected to the four ports of the tester (T1-T2). The tester send a frame to the port of the DUT (D1), and then check this MAC address whether it is in the MAC address table by the ARL function of the web and the BCM mode. Figure 9 Address Lookup Test Configuration The MAC address should be shown correctly on the Web and CLI. Network Benchmarking Lab 14

15 Storm Control Test This test validates the storm control function whether it limits the frame rate of the broadcast traffic. SmartWindow The configuration of this test is shown in Figure 10. The two ports of the DUT (D1-D2) are connected to the two ports of the tester (T1-T2). The parameter of storm control is configured on the DUT. The tester sends broadcast traffic from the port (T1) to the port (T2). Figure 10 Storm Control Test Configuration The frame rate of the broadcast traffic should be limited. Network Benchmarking Lab 15

16 Rate Limit Test This test validates the port-based and application-based rate limit function whether it limits the different traffics such as broadcast, unknown unicast and multicast. SmartWindow The configuration of this test is shown in Figure 11. The two ports of the DUT (D1-D2) are connected to the two ports of the tester (T1-T2). The parameter of rate limit is configured on the DUT. The tester sends three different traffics broadcast, multicast, and unknown unicast from the port (T1) to the port (T2). Figure 11 Rate Limit Test Configuration 1. The throughputs of traffic should be limited for port-based configuration. 2. Same as above but for application-based configuration. Network Benchmarking Lab 16

17 DiffServ Classification Test This test validates the DiffServ classifier function of the DUT. SmartWindow SmartFlow The configuration of this test is shown in Figure 12. The nine ports of the DUT (D1-D9) are connected to the nine ports of the tester (T1-T9). The direction which is from any port among port 1 and port 8 to the port 9 is defined as the ingress datapath, and the opposite direction is the egress datapath. Different parameter of the classifier and in-coming and out-going profile is configured on the DUT. The tester sends the untagged traffic into the DUT to verify the DiffServ classifier functionality. The traffic will be inserted with appropriate filed value if the DiffServ functionality is properly implemented in the DUT. In the test, the following features will be verified: Classifier criteria: MAC address, VLAN ID, IP Address, and TCP/UDP Port In-Profile actions: switch action, insert priority, DSCP, and TOS Out-Profile actions: switch action and insert DSCP Figure 12 DiffServ Classification Test Configuration The related field values of traffic should be correctly processed by DiffServ. Network Benchmarking Lab 17

18 IGMP Snooping Test This test validates the IGMP Snooping function when the multicast traffic of a group is only forwarded to ports that have members belonging to that group. SmartWindow SmartFlow The configuration of this test is shown in Figure 13. The two ports of the DUT (D1-D2) are connected to the two ports of the tester (T1-T2). The port 3 of the DUT (D3) is connected to the port 2 of L3 switch (S2). The port 1 of the L3 Switch (S1) is connected to the port 3 of the tester (T3). The parameter of IGMP Snooping and router port are configured on the DUT. The parameter of the IGMP and DVMRP are configured on the L3 switch. The port T1 sends the IGMP Report message to the DUT when subscriber join the IGMP group, the port T3 will send multicast traffic of a group is only forwarded to ports that have members of that group. Meanwhile, the port T2 captures the frame from the port D2 whether it receives any multicast traffic. Figure 13 IGMP Snooping Test Configuration Tester should not receive any multicast traffic or IGMP message on Port D2. Network Benchmarking Lab 18

19 Multicast Protocol Snooping Test This test validates the PIM and DVMRP Snooping function when the multicast traffic of a group is only forwarded to the next hop for the correct multicast route. SmartWindow TeraRouting Tester The configuration of this test is shown in Figure 14. The two ports of the DUT (D1-D2) are connected to the two ports of the tester (T1-T2). The port 3 of the DUT (D3) is connected to the port 2 of L3 switch (S2). The port 1 of the L3 Switch (S1) is connected to the port 3 of the tester (T3). The parameter of PIM-SM/DM or DVMRP snooping is configured on the DUT. The parameters of the PIM-SM/DM/DVMRP are configured on the L3 switch. The port T1 exchanges the routing information with the L3 switch, and the port T3 sends IGMP message to the L3 Switch when subscriber join the IGMP group. Then the port T1 sends multicast traffic for the group which route is only through the L3 Switch. Meanwhile, the port T2 captures the frame from the port D2 whether it receives any multicast traffic. Figure 14 Multicast Protocol Snooping Test Configuration 1. Tester should not receive any multicast traffic on Port D2 for PIM-SM/DM. 2. Same as above but for DVMRP. Network Benchmarking Lab 19

20 802.1X Test This test validates the basic function of IEEE 802.1X for port-based and MAC-based authentication mechanism. RADIUS server - IAS on Windows 2003 Server Supplicant - Windows XP Professional This test uses the PC with Windows XP Professional as supplicant, the DUT as an authenticator and Windows 2003 server as an authentication server. The cable connections between the supplicant, DUT and Authentication Server are shown in Figure 15. The port (S1) of the supplicant is connected to the ports (D1) of the DUT. The port (D2) of the DUT is connection to the port (A1) of the Authentication Server to transmit and receive request and response messages. The parameters of the 802.1X and EAP types are configured on the supplicant and authentication server. Figure 15 IEEE 802.1X Test Configuration. The supplicants should be authenticated by the Authentication Server. Network Benchmarking Lab 20

21 DHCP/BOOTP Test This case validates the DHCP and BOOTP function of the DUT, including 4 subparts: DHCP client, DHCP relay-agent, BOOTP relay-agent and DHCP server. DHCP/BOOTP Server - Windows Server 2003 DHCP/BOOPT Client Windows XP Professional The configuration of the DHCP client test is shown in Figure 16 (a). DHCP client mode is enabled or disable in the DUT in order to validate the correctness DHCP client mode of the DUT. For the DHCP relay-agent test, the configuration is shown in the Figure 16 (b). In this sub-test, two different VLANs are created so that DHCP messages can be passed between DHCP client and DHCP Server. In addition, the relay-agent mode is turned on in the DUT to substantiate that DUC can correctly forwarding the client DHCP request to the DHCP Server as claim by the DUT. The BOOTP test is also configured in the same way by using DUT as a BOOTP relay-agent. The DUT is configured to either lease a static IP address or dynamic IP addresses to the DHCP client in the DHCP Server test as shown in the Figure 16 (c). Figure 16 (a) DHCP Client Test, (b) DHCP/BOOTP Relay-Agent Test (c) DHCP Server Test. Network Benchmarking Lab 21

22 The client should be able to get IP address. Network Benchmarking Lab 22

23 RADIUS Server Test This case validates the function of the RADIUS Server in the DUT. Supplicant - Windows XP Professional This test uses the PC with Windows XP Professional as supplicant and the DUT as an authentication server. The configuration of the supplicant, authenticator and DUT is shown in Figure 17. The port (S1) of the supplicant is connected to the ports (D1) of the DUT through the authenticator. The parameters of the 802.1X and EAP types are configured on the supplicant and authentication server. Figure 17 RADIUS Server Test Configuration The supplicants should be authenticated by the Authentication Server. Network Benchmarking Lab 23

24 DNS Server/Client Test This case validates the function of the DNS Server/Client in the DUT. PC - Windows XP Professional Real World DNS Servers This test uses a PC with Windows XP Professional as a DNS Client, the DUT as a DNS Resolver and two DNS Servers. The cable connection between the PC, DUT and DNS Servers is shown in Figure 18. The port (S1) of the PC is connected to the ports (D1) of the DNS resolver to transmit and receive DNS query and reply messages. The parameters of the DNS Resolver such as domain names for different zones, primary DNS server address and secondary DNS server address are configured on the DUT. Figure 18 DNS Server/Client Test Configuration The DUT should properly resolve the domain name to correct IP address. Network Benchmarking Lab 24

25 SNTP Test This test validates the function of the SNTP in the DUT. Real World SNTP Server The configuration of the SNTP is shown in Figure 19. The DUT time is synchronized the with an SNTP server by using sntp_client command in the DUT. Figure 19 SNTP Test Configuration. The DUT should synchronize with the SNTP Server. Network Benchmarking Lab 25

26 Access Control List Test This case validates the basic function of access control list (ACL). SmartWindow SmartFlow The configuration of the test is shown in the figure 20. The two ports of the DUT are connected to the two ports of the Tester. There are 8 criteria for individual ACL filter and 3 criteria of ACL filter combination considered. Each criterion is verified for both accept (1 entry to accept any specified value and 1 entry to deny all) and deny (just 1 entry to deny) actions and is listed as follow: 1. Source IP accept/deny 2. Destination IP accept/deny 3. Source Port accept/deny 4. Destination Port accept/deny 5. Source IP (in class B) and Destination Port (ex. 21) accept/deny 6. Destination IP and Protocol (ex. UDP) accept/deny 7. Source IP (in class B) deny and Source IP (single) accept The tester sends the traffic with 2 flows, one to match and the other to mismatch the entry, and the traffic that received on the port T2 will be closely verified. Figure 20 Access Control List Test Configuration Two flows should correctly grant different actions for each criterion. Network Benchmarking Lab 26

27 Firmware/Configuration Upgrade Test This test validates the function of firmware/configuration file upgrade and its stability even under the error situations. The firmware upgrade will use TFTP and Xmodem. TFTP Server SecureCRT The configuration of this test is shown in Figure 21. The firmware/configuration file is put on a PC (with TFTP server and Xmodem terminal for the firmware). The DUT is configured to download the firmware/configuration file from the PC. The following results are checked while downloading the firmware/configuration file to the DUT. 1. The DUT downloads an inexistent file. 2. The network connection is broken while the DUT is downloading the file. 3. The DUT is downgraded with an old version of file. 4. The available storage space of the DUT is not enough for the file to be downloaded. 5. The DUT is upgraded with an invalid firmware file/configuration file. 6. The DUT is upgraded with an incomplete firmware file/configuration file. 7. The DUT is upgraded with a redundant firmware file/configuration file. 8. The power is shut down while the DUT is upgrading. Figure 21 Firmware/Configuration Upgrade Test Configuration The DUT should be able to detect the error situations and suggest actions. Network Benchmarking Lab 27

28 3. Performance Test Forwarding Test To test the DUT forwarding rate, frame loss and throughput of layer 2 frames in different distributions traffic. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices SmartFlow 5.00 The configuration of this test is shown in Figure 22. The 8 ports of the DUT (D1-D8) are connected to the 8 ports of the tester (T1-T8). Following Figure 22 shows different types of traffic distributions. During the testing, the binary search algorithm is used to find out the zero-loss maximum forwarding rate. Moreover, the tester sends the maximum offered load traffic to the DUT, and measures the DUT to see whether it can cause frame loss. During the test, the tested frame sizes are 64, 128, 256, 512, 1024, 1280, 1518, 9000 (jumbo) bytes, and the test duration is 30 seconds for each iteration. Network Benchmarking Lab 28

29 Forwarding rate testing cable connections. (a)fully Meshed traffic. (b) Many-to-many, unidirectional traffic. (c) Many-to-many, bidirectional traffic. (d) One-to-many, unidirectional traffic. (e) One-to-many, bidirectional traffic. (f) Many-to-one, unidirectional traffic. (g) Many-to-one, bidirectional traffic. T1 T2 4 (h) Single-pair, intra-asic unidirectional traffic. (i) Single-pair, intra-asic, bidirectional traffic. (j) Single-pair, inter-asic, unidirectional traffic. (k) Single-pair, inter-asic, bidirectional traffic. T1 T2 T2 3 T2 4 (l) Multiple-pair, intra-asic, unidirectional traffic. (m) Multiple-pair, intra-asic, bidirectional traffic. (n) Multiple-pair, inter-asic, unidirectional traffic. Figure 22 Forwarding Test Configuration (o) Multiple-pair, inter-asic, bidirectional traffic. Network Benchmarking Lab 29

30 1. Measure the DUT zero-loss maximum forwarding rate when the DUT doesn t have any frame loss. 2. Measure the DUT to see whether it can cause frame loss during the tester sends the maximum offered load traffic to the DUT. Network Benchmarking Lab 30

31 Congestion Control Test To determine how the DUT handles congestion. Does the device implement congestion control and does congestion on one port affect an uncongested port. This procedure determines if head of line blocking and/or backpressure are presented. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 23(a). The four ports of the DUT (D1-D4) are connected to the four ports of the tester (T1-T4). The parameters of the congestion control function are configured on the four ports of the DUT (D1-D4). T1 and T2 are source ports, T3 is an uncongested port, and T4 is a congested port. The tester provides 50% maximum offered load from the source port (T1) to the destination port (T3), and provides 50% maximum offered load from the source port (T1) to the destination port (T4), as shown in Figure 23(b). So, T4 will cause congestion situation, but T3 will not. T1 50% MOL T3 Uncongested 50% MOL T2 100% MOL T4 Congested (a) Congestion control testing cable (b) Congestion control testing traffic connections. distribution. Figure 23 Congestion Control Test Configuration 1. The number of test frames received on the uncongested port (T3) should be 50% of the test frames transmitted by the first source port (T1). 2. The number of test frames received on the congested port (T4) should be between 100% and 150% of the test frames transmitted by one source port. Network Benchmarking Lab 31

32 3. If there is frame loss at the uncongested port (T3), head of line blocking is presented. 4. If there is no frame loss on the congested port (T4), then backpressure is presented. 5. If the expected percentage frame loss for the congested port (T4) is 33% at 150% overload, there is no congestion control detected. Network Benchmarking Lab 32

33 Address Caching Capacity Test To determine the address caching capacity of a LAN switching device. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 24. The three ports of the DUT (D1-D3) are connected to the three ports of the tester (T1-T3). Port T1 is a learning port, port T2 is a test port, and port T3 is a monitoring port. Firstly, the source port (T2) sends a broadcast frame to the destination port (D2), and the DUT will learn the MAC address from T2. Then T1 sends the learning frames to the first port (D1). Though the source MAC addresses of learning frames are all different, the learning frames only include a fixed destination MAC address, which is the T2 s MAC address. The DUT will learn the source MAC addresses of the learning frames. Finally, the learning port (T2) will send the frames to the DUT s second port (D2), and the destination MAC addresses of the frames were learned from the port D1. Next the DUT will transfer those frames to the test port (T1) and check the learning port (T1) how many frames are received. If the amounts of the frames are more than those sent from the tester, it indicates the unnecessary frames are flood frames. If there are some flood frames on the port T3, the learning port (T1) needs to decrease the amount of learning frames. If there is no flood frame on the port T3, the learning port (T1) needs to increase the amount of learning frames. This test needs to be repeated until finding the maximum capacity of MAC address that the DUT can be learned. Figure 24 Address Caching Capacity Test Configuration Network Benchmarking Lab 33

34 To observe the maximum number of MAC addresses the DUT can learn. Network Benchmarking Lab 34

35 Address Learning Rate Test To test the maximum MAC address learning rate of the DUT. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 25. The three ports of the DUT (D1-D3) are connected to the three ports of the tester (T1-T3). Port T1 is a learning port, port T2 is a test port, and port T3 is a monitoring port. Firstly, the source port (T2) sends a broadcast frame to the destination port (D2), and the DUT will learn the MAC address from T2. Then T1 sends the learning frames to the first port (D1). Though the source MAC addresses of learning frames are all different, the learning frames only include a fixed destination MAC address, which is the T2 s MAC address. The DUT will learn the source MAC address of the learning frames. Finally, the learning port (T2) will send the frames to the DUT s second port (D2), and the destination MAC addresses of the frames were learned from the port D1. Next the DUT will transfer those frames to the test port (T1) and check the learning port (T1) how many frames are received. If the amounts of frames are more than those sent from the tester, it indicates the unnecessary frames are flood frames. If there are some flood frames on the port T3, the learning port (T1) needs to decrease the amount of learning frames. If there is no flood frame on the port T3, the learning port (T1) needs to increase the amount of learning frames. This test needs to be repeated until finding the maximum learning rate. Figure 25 Address Learning Rate Test Configuration Network Benchmarking Lab 35

36 1. To observe the maximum MAC address learning rate of the DUT. 2. Whether address learning rate can be reached wire-speed. Network Benchmarking Lab 36

37 Errored Frame Filtering Test To determine the behavior of the DUT under errored or abnormal frame conditions. The results of the test indicate whether the DUT filters the errors or simply propagates the errored frames along to the destination. The following types of errored frames are tested. Oversize. Undersize. CRC errors. Dribble bit errors. Alignment errors. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 26. The two ports of the DUT (D1-D2) are connected to the three ports of the tester (T1-T2). The tester generates the error frames from T1 to D1, and then the number of frames received at T2 is checked. Errored frames are frames which are over-sized, under-sized, misaligned or with an errored frame check sequence. Switches, unlike IEEE 802.1d compliant bridges, do not necessarily filter all types of illegal frames. Some switches, for example, which do not store frames before forwarding them to their destination interfaces may not filter over-sized frames (jabbers) or verify the validity of the frame check sequence field. Other illegal frames are under-sized frames (runts) and misaligned frames. Oversize - The DUT may filter frames larger than 1518 bytes from being propagated through the DUT. Oversized frames transmitted to the DUT should not be forwarded. DUT supporting tagged Frames may forward frames up to and including 1522 bytes long. Undersize - The DUT must filter frames less than 64 bytes from being propagated through the DUT. Undersized frames (or collision fragments) received by the DUT must not be forwarded. Network Benchmarking Lab 37

38 CRC errors - The DUT must filter frames failing the frame check sequence validation from being propagated through the DUT. Frames with an invalid CRC transmitted to the DUT should not be forwarded. Dribble bit errors - The DUT must correct and forward frames containing dribbling bits. Frames transmitted to the DUT that do not end in an octet boundary but contain a valid frame check sequence must be accepted by the DUT and forwarded to the correct receiving port with the frame ending in an octet boundary. Alignment errors - The DUT must filter frames that fail the frame check sequence validation and do not end in an octet boundary. This is a combination of a CRC error and a dribble bit error. When both errors are occurring in the same frame, the DUT must determine the CRC error takes precedence and prevents the frame from being propagated. Figure 26 Errored Frames Filtering Test Configuration 1. When the oversize frames are transmitted to the DUT. The DUT must drop them and the frames should not be forwarded. So the T2 will not receive any frame. 2. When the undersize frames are transmitted to the DUT. The DUT must drop them and the frames should not be forwarded. So T2 will not receive any frame. 3. When the CRC error frames are transmitted to the DUT. The DUT must drop them and the frames should not be forwarded. So the T2 will not receive any frame. 4. The DUT must correct and forward frames containing dribbling bits. So the T2 will not lose any frame. 5. When alignment errors frame are transmitted to the DUT. The DUT must drop them and the frames should not be forwarded. So the T2 will not receive any frame. Network Benchmarking Lab 38

39 Broadcast Frame Forwarding Test To determine the forwarding rate, frame loss and throughput of the DUT will be determined when the tester transfers broadcast frames to the DUT. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 27(a). The 8 ports of the DUT (D1-D8) are connected to the 8 ports of the tester (T1-T8). The parameters of the broadcast storm are disabled on the 8 ports of the DUT (D1-D8). When the port T1 of the tester transfers a broadcast frame to the 8 ports (D1-D8) of the DUT, the forwarding rate, frame loss and throughput will be measured. During the test, the tested frame sizes are 64, 128, 256, 512, 1024, 1280, 1518, 9000 bytes, and the test duration is 30 seconds for each iteration. (b) Broadcast frame forwarding (a) Broadcast frame forwarding testing cable testing traffic distribution. connections. Figure 27 Broadcast Frame Forwarding Test Configuration 1. To measure the DUT zero-loss maximum forwarding rate when the DUT transfers the broadcast traffic and the DUT doesn t have any frame loss. 2. To measure the DUT whether it can cause frame loss when the tester sends broadcast traffic of maximum offered load traffic to the DUT. Network Benchmarking Lab 39

40 Broadcast Latency Test To determine the latency when the DUT forwards a broadcast frame. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 28(a). The 8 ports of the DUT (D1-D8) are connected to the 8 ports of the tester (T1-T8). The parameters of broadcast storm are disabled on 8 ports (D1-D8) of the DUT. When the port T1 of the tester transfers a broadcast frame to the 8 ports (D1-D8) of the DUT, the latency will be measured. During the test, the traffic frame sizes are 64, 128, 256, 512, 1024, 1280, 1518, 9000 bytes. (a)broadcast frame latency testing cable (b)broadcast frame latency connections. testing traffic distribution. Figure 28 Broadcast Latency Test Configuration To observe the broadcast latency while the DUT transfers a broadcasting frame to all ports. Network Benchmarking Lab 40

41 Forward Pressure Test The forward pressure test overloads a DUT port and measures the output for forward pressure. If the DUT transmits frames with an inter frame gap less than 96 bits, then the forward pressure is detected. Reference: RFC Benchmarking Terminology for LAN Switching Devices RFC Benchmarking Methodology for LAN Switching Devices AST II 3.5 The configuration of this test is shown in Figure 29. The two ports of the DUT (D1-D2) are connected to the two ports of the tester (T1-T2). The tester generates the frames with an interframe gap less than 96 bits from T1 to D1, and observes the T2 whether it has any frame loss. Figure 29 Forward Pressure Test Configuration If the port 2 of DUT (T2) had no frame loss, the forward pressure may exist on the DUT. However, if the DUT had frame loss, it indicates the DUT doesn t have forward pressure mechanism. Network Benchmarking Lab 41

42 Quality of Service Test The following classification mechanisms are tested (1) Port-based (2) Class of Service (3) IP Precedence (4) IP DSCP (DiffServ Code Point) SmartFlow The configuration of this test is shown in Figure 30. The eight ports of the DUT (D1-D8) are connected to the eight ports of the tester (T1-T8). The parameters of QoS and the rule of traffic classifications are configured on the DUT, as shown in Figure 30(b), The tester provide two different offered loads from eight source ports (T1-T8) to the destination port (T9), 12.5%, 100% offered load respectively. The destination port (T8) will cause some congestion situation at destination port (T9) if there is more than 12.5% offered load. So, the DUT will follow the traffic classification and scheduling to provide quality of service. Five classifications are outlined in this test below: 1. No quality of service (This test result will be compared with other results) 2. Classification by port 3. Classification by class of service value of the VLAN tag: 4. Classification by IP precedence (TOS) value: 5. Classification by IP DSCP value: (a) Cable connections. (b) Traffic distribution. Figure 30 Quality of Service Test Configuration 1. The no quality of service scheme: Measure the forwarding rate of the stream sent from the first port of the tester (T1) to the eighth port of the tester (T8), each port should share equal bandwidth. Network Benchmarking Lab 42

43 2. The port-based scheme: Measure the forwarding rate of the stream sent from the first port of the tester (T1) to the eighth port of the tester (T8) whether it conforms to the traffic classification and scheduling to provide quality of service. 3. The class of service scheme: Measure the forwarding rate of the stream sent from the first port of the tester (T1) to the eighth port of the tester (T8) whether it conforms to the traffic classification and scheduling to provide quality of service. 3. The IP precedence scheme: Measure the forwarding rate of the stream sent from the first port of the tester (T1) to the eighth port of the tester (T8) whether it conforms to the traffic classification and scheduling to provide quality of service. 4. The IP DSCP scheme: Measure the forwarding rate of the stream sent from the first port of the tester (T1) to the eighth port of the tester (T8) whether it conforms to the traffic classification and scheduling to provide quality of service. Network Benchmarking Lab 43

44 4. Conformance Test Spanning Tree Protocol Validation To test the conformance degree of DUT to the standard of IEEE 802.1D. Ixia IxANVL - STP Validation Test Suite The configuration of this test is shown in Figure 31. The three ports of the DUT are connected to the three ports of the IxANVL. The parameters of the spanning tree protocol are configured on the DUT. IxANVL has the total of 9 categories that include 53 test cases in the spanning tree protocol test suite, as follows. 1. Quick/Setup Verification 2. Forwarding 3. Port States 4. Elements of Procedure 5. Protocol Operation 6. Spanning Tree Algorithm 7. Parameter Values 8. Negative Tests 9. Stress Tests Figure 31 Spanning Tree Protocol Validation Configuration To observe the DUT whether it can pass each test case. Network Benchmarking Lab 44

45 Rapid Spanning Tree Protocol Validation To test the conformance degree of the DUT to the standard of IEEE 802.1w. Ixia IxANVL - RSTP Validation Test Suite The configuration of this test is shown in Figure 32. The three ports of the DUT are connected to the three ports of the IxANVL. The parameters of the rapid spanning tree protocol are configured on the DUT. Totally, IxANVL has the total of 33 categories that include 172 test cases in the multiple spanning tree protocol test suite, as follows. 1. Encoding of Port Role Values 2. BPDU Formats and Parameters 3. Validation of Received BPDUs 4. Port Role Assignment 5. Spanning Tree Priority Vector Definition 6. Port States 7. Communicating Spanning Tree Information 8. Changing Spanning-Tree Information 9. Updating Learned Station Location Information 10. State Machine Performance Parameters: ForceVersion 11. State Machine Procedures: rcvbpdu() 12. State Machine Procedures: updtrcvdinfowhile() 13. The Port Information State Machine 14. The pport Role Transitions State Machine 15. Port State Transition State Machine 16. Topology Change State Machine 17. Port Protocol Migration State machine 18. Performance: Parameter Values Network Benchmarking Lab 45

46 Figure 32 Rapid Spanning Tree Protocol Validation Configuration To observe the DUT whether it can pass each test case. Network Benchmarking Lab 46

47 Multiple Spanning Tree Protocol Validation To test the conformance degree of the DUT to the standard of IEEE 802.1s. Ixia IxANVL - MSTP Validation Test Suite The configuration of this test is shown in Figure 33. The three ports of the DUT are connected to the three ports of the IxANVL. The parameters of the multiple spanning tree protocol are configured on the DUT. Totally, IxANVL has the total of 33 categories that include 231 test cases in the multiple spanning tree protocol test suite, as follows. 1. ANVL Setup Verfication 2. Relationship of MSTP to RSTP 3. Modelling an MST Region as a single RSTP Bridge 4. Designated Port Selection 5. Force Protocol Version 6. MST Regions 7. Spanning Tree priority vectors 8. CIST Priority Vector Calculations 9. MST Priority Vector Calculations 10. Port Role Assignments 11. Port Role Assignments 12. Changing Spanning Tree Information 13. Updating Learned Station Location Information 14. mstimaster 15. rcvinfocist() 16. rcvlinfomsti() 17. recordproposalcist() 18. recordproposalmsti() 19. updtrcvdinfowhilecist() 20. updtrolescist() 21. updtrolesmsti() 22. Port Receive State Machine 23. Port Protocol Migration state machine 24. Port Information state machine 25. Port Role Selection State machine 26. Port Role Transitions state machine 27. Port State Transition State machine Network Benchmarking Lab 47

48 28. Topology Change state machine 29. Performance 30. Componets 31. STP BPDUs 32. Validation of received BPDUs 33. Encoding and decoding of STP Configuration, RST, and MST BPDUs Figure 33 Multiple Spanning Tree Protocol Validation Configuration To observe the DUT whether it can pass each test case. Network Benchmarking Lab 48

49 Virtual Local Area Network Validation To test the conformance degree of DUT to the standard of IEEE 802.1Q. Reference: IEEE 802.1Q-2003: Virtual Bridged Local Area Networks Ixia IxANVL - VLAN Validation Test Suite The configuration of this test is shown in Figure 34. The three ports of the DUT are connected to the three ports of the IxANVL. The parameters of VLAN are configured on the DUT. Totally, IxANVL has the total of 31 categories that include 147 test cases in the VLAN test suite, as follows. 1. Setup/Verification Tests 2. Spanning Tree 3. Filtering and relaying information 4. Port VLAN identifier 5. Enable Ingress Filtering 6. Frame reception 7. Regenerating user priority 8. The ingress rules 9. Enforcing topology restriction 10. The egress rules 11. The Learning Process 12. The Filtering Database 13. Static Filtering Entries 14. Static VLAN Registration Entres 15. Dynamic Filtering Entries 16. Group Registration Entries 17. Dynamic VLAN Registration Entries 18. Default Group Fitering behavior 19. Allocation of VIDs to FIDs 20. Querying thefiltering Database 21. Determination of themember set and untagged set for a VLAN 22. Bridge Protocol Entities and GARP PROTOCOL Entities 23. Reserved addresses 24. Points of attachment and connectivity for Higher Layer Entities 25. Use of GMRP in VLANs Network Benchmarking Lab 49

50 26. Context idetification in GMRP PDUs 27. Setup/Verfication Tests (of GVRP) 28. Behavior of Bridges 29. Use of the PVID 30. Gvrp Application address 31. Encoding of GVRP Attribute Types Figure 34 Virtual Local Area Network Validation Configuration To observe the DUT whether it can pass each test case. Network Benchmarking Lab 50

51 Remote Monitoring Validation To test the conformance degree of DUT to the standard of RFC Ixia IxANVL - RMON Validation Test Suite The configuration of this test is shown in Figure 35. The three ports of the DUT are connected to the three ports of the IxANVL. The parameters of the spanning tree protocol are configured on the DUT. IxANVL has the total of 50 categories, including 176 test cases, in the remote monitoring test suite for the following MIB groups which are supported by the DUT. 1. Ethernet Statistics Group (22 categories, 86 test cases) 2. History Group (8 categories, 29 test cases) 3. Alarm Group (13 categories, 42 test cases) 4. Events Group (7 categories, 19 test cases) Figure 35 Remote Monitoring Validation Configuration To observe the DUT whether it can pass each test case. Network Benchmarking Lab 51

52 5. Interoperability Test Spanning Tree Protocol This suite of tests evaluates the interoperability of the STP capable product. Reference: IEEE Std D-1998 IEEE Std t-2001 UNH-IOL Spanning Tree Interoperability Test Suite, v1.5 SmartFlow SmartWindow There are 5 different configurations used in the test suite. The test stations TS1-TS4 in the figures are emulated by SmartBits. The test setup (STP root bridge/priority, timer parameters, and the mix-type traffic burst) is same as the IOL document. Totally there are 5 test cases in the test suite, as follows. 1. Link Failure 2. Repeated Network 3. Maximum Hello Time 4. Network Initialization 5. Topology Change Please refer to the IOL document. The DUT should pass each of the test cases defined in the IOL document. Network Benchmarking Lab 52

53 Rapid Spanning Tree Protocol This suite of tests evaluates the interoperability of the RSTP capable product. Reference: IEEE 802.1D-1998 IEEE 802.1w-2001 UNH-IOL Rapid Spanning Tree Interoperability Test Suite, v1.2 SmartFlow SmartWindow There are 6 different configurations used in the test suite. The test stations TS1-TS4 in the figures are emulated by SmartBits. The test setup (RSTP root bridge/priority, timer parameters, and the mix-type traffic burst) is same as the IOL document. Totally there are 9 test cases in the test suite, as follows. 1. Basic RSTP Interoperability 2. Non-default Timers 3. Link Failure and Re-initialization 4. Non-Default VLAN 5. STP Compatibility Mode 6. Legacy Root Bridge 7. Topology Change 8. Root Failure 9. Network Initialization Please refer to the IOL document. The DUT should pass each of the test cases defined in the IOL document. Network Benchmarking Lab 53

54 Virtual Local Area Network This suite of tests evaluates the interoperability of the VLAN capable product. Reference: IEEE Std Q-2003 UNH-IOL Virtual Local Area Network Interoperability Test Suite, v2.2. SmartWindow The configuration of this test is shown in Figure 36. The test station (TS) is emulated by SmartBits. The three ports of the DUT are connected to TS1-TS3; the three ports of the Bridge Partner (BP) are connected to TS4-TS6. The DUT and BP are connected to each other. The test setup (VLAN configuration and traffic distribution) is same as the IOL document. In this test suite, totally there are 3 test groups, including 13 test cases, as follows. 1.1 Blue (Default VID 0x001) Access Link 1.2 Red (VID 0x002) Access Link 1.3 Green (VID 0xFFE or max. VID) Access Link 2.1 Blue Trunk Link 2.2 Red Trunk Link 2.3 Green Trunk Link 2.4 Blue/Red Trunk Link 2.5 Blue/Green Trunk Link 2.6 Red/Green Trunk Link 2.7 Blue/Red/Green Trunk Link 3.1 Blue/Red Trunk, Green Access Link 3.2 Blue/Green Trunk, Red Access Link 3.3 Red/Green Trunk, Blue Access Link By default, the name of VLAN, Blue, Red, and Green, has port members as follows: Blue (default VID 0x001): DUT-to-TS1, BP-to-TS4 untagged Red (VID 0x002): DUT-to-TS2, BP-to-TS5 untagged Green (VID 0xFFE or max. VID): DUT-to-TS3, BP-to-TS6 tagged Each test case will assign additional VLAN configurations for testing. Please refer to the test procedure in the IOL document. Network Benchmarking Lab 54

55 Figure 36 Virtual Local Area Network Interoperability Test Configuration The DUT should pass each of the test cases defined in the IOL document. Network Benchmarking Lab 55

56 Network Benchmarking Lab 56