VulcanManager. User Manual. This is the User Manual for VulcanManager.

Transcription

1 VulcanManager User Manual This is the User Manual for VulcanManager. VulcanManager is the primary software application used to manage and configure VulcanBay, Xena s Layer 4-7 test platform. Last updated:

2 CONTENTS Welcome Installation General Information Xena L4-7 Stateful Testing Architecture End-to-End Testing Subnet and Users Getting Started with VulcanManager Configuring Your First Test Project Configuring Test Case Run Test Case Statistics and Generate Report Resource Reservation Chassis Resources Reservation Mechanism Reservation Hierarchy Chassis Management Connecting to a Chassis Editing Chassis Information Reconnecting to Chassis Disconnecting from Chassis Removing Chassis Troubleshooting Load Test Scenarios None Scenario Raw Scenario Transaction Scenario HTTP GET Scenario Test with TLS Record Size Certificate

3 Cipher Suite Collection TLS Close Notify Subnet Configuration Network Address and Network Mask Bit Limit Address Range Use Gateway Gateway Address Gateway MAC Address NAT Embed IP Address in MAC and Ethernet Address Use ARP Enable VLAN Connection Configuration Subnet Selection User Connection Setup Connection Establishment Profile Payload Configuration Payload in Raw Scenario Payload in Transaction Scenario Test Case Configuration Run Test Case Test Control Realtime Statistics Chart Generate Report Advanced Features Change Port Speed Change Port TX Line Rate Utilization Stateful Layer-4 Payload Replay Application Emulation Connection Update Capture Traffic to PCAP Xena Layer 4-7 Scripting API Statistics

4 Port-Level Statistics Ethernet Rx Bit Rate Ethernet Rx Packet Rate Ethernet Rx Bytes Ethernet Rx Packets Ethernet Tx Bit Rate Ethernet Tx Packet Rate Ethernet Tx Bytes Ethernet Tx Packets Ethernet Events Tx Errors Ethernet Events Rx Errors Ethernet Events Rx Missed Port Rx VLAN Tagged Packets Port Rx Bit Rate Port Tx VLAN Tagged Packets Port Tx Bit Rate Port Events Invalid Ethernet Packets Port Events Unknown Ethernet Packets Port Events VLAN Errors Port Events Packet Rate Limit Hit IPv4 Rx IPv4 Packets IPv4 Tx IPv4 Packets IPv4 Events IPv4 Checksum Errors IPv4 Events Invalid IPv4 Packets IPv4 Events Unknown IPv4 Packets IPv6 Rx IPv6 Packets IPv6 Tx IPv6 Packets IPv6 Events Invalid IPv6 Packets IPv6 Events Unknown IPv6 Packets ARP Rx ARP Requests ARP Rx ARP Replies ARP Tx ARP Requests ARP Tx ARP Replies

5 ARP Events Invalid ARP Packets ARP Events ARP Request Lookup Failures ARP Events ARP Reply Lookup Failures ARP Events ARP Request Retransmits ARP Events ARPs Resolved ARP Events ARPs Failed ARP Events ARP Table Lookup Failures NDP Rx NDP Requests NDP Rx NDP Replies NDP Tx NDP Requests NDP Tx NDP Replies NDP Events Invalid NDP Packets NDP Events NDP Request Lookup Failures NDP Events NDP Reply Lookup Failures NDP Events NDP Request Retransmits NDP Events NDP Resolved NDP Events NDP Failed NDP Events NDP Table Lookup Failures ICMP Rx ICMP Echo Requests ICMP Rx ICMP Echo Replies ICMP Rx ICMP Destination Unreachable ICMP Rx ICMP Time Exceeded ICMP Tx ICMP Echo Requests ICMP Tx ICMP Echo Replies ICMP Tx ICMP Destination Unreachable ICMP Tx ICMP Time Exceeded ICMP Rx ICMPv6 Packets ICMP Tx ICMPv6 Packets ICMP Events Invalid ICMP Packets ICMP Events Unknown ICMP Packets ICMP Events Invalid ICMPv6 Packets ICMP Events Unknown ICMPv6 Packets TCP Rx TCP Packets

6 TCP Tx TCP Packets TCP Events TCP Checksum Errors TCP Events TCP Invalid Packets TCP Events TCP Lookup Failures UDP Rx UDP Packets UDP Tx UDP Packets UDP Events UDP Checksum Errors UDP Events UDP Invalid Packets UDP Events UDP Lookup Failures Scenario-Level Statistics TLS State Inactive TLS State Handshaking TLS State Handshake Done TLS State Handshake Failed TLS State Failed TLS State Internal Failed TLS State Close Notify TLS State Done TLS State Rate Inactive TLS State Rate Handshaking TLS State Rate Handshake Done TLS State Rate Handshake Failed TLS State Rate Failed TLS State Rate Internal Failed TLS State Rate Close Notify TLS State Rate Done TLS State Total Inactive TLS State Total Handshaking TLS State Total Handshake Done TLS State Total Handshake Failed TLS State Total Failed TLS State Total Internal Failed TLS State Total Close Notify

7 TLS State Total Done TLS Rx Payload Bytes TLS Rx Payload Bit Rate TLS Tx Payload Bytes TLS Tx Payload Bit Rate TLS Warning Alerts Close Notify TLS Warning Alerts Unexpected Message TLS Warning Alerts Bad Record Mac TLS Warning Alerts Record Overflow TLS Warning Alerts Decompression Failure TLS Warning Alerts Handshake Failure TLS Warning Alerts Bad Certificate TLS Warning Alerts Unsupported Certificate TLS Warning Alerts Certificate Revoked TLS Warning Alerts Certificate Expired TLS Warning Alerts Certificate Unknown TLS Warning Alerts Illegal Parameter TLS Warning Alerts Unknown CA TLS Warning Alerts Access Denied TLS Warning Alerts Decode Error TLS Warning Alerts Decrypt Error TLS Warning Alerts Protocol Version TLS Warning Alerts Insufficient Security TLS Warning Alerts User Canceled TLS Warning Alerts No Renegotiation TLS Warning Alerts Unsupported Extension TLS Warning Alerts Unknown TLS Fatal Alerts Close Notify TLS Fatal Alerts Unexpected Message TLS Fatal Alerts Bad Record Mac TLS Fatal Alerts Record Overflow TLS Fatal Alerts Decompression Failure TLS Fatal Alerts Handshake Failure

8 TLS Fatal Alerts Bad Certificate TLS Fatal Alerts Unsupported Certificate TLS Fatal Alerts Certificate Revoked TLS Fatal Alerts Certificate Expired TLS Fatal Alerts Certificate Unknown TLS Fatal Alerts Illegal Parameter TLS Fatal Alerts Unknown CA TLS Fatal Alerts Access Denied TLS Fatal Alerts Decode Error TLS Fatal Alerts Decrypt Error TLS Fatal Alerts Protocol Version TLS Fatal Alerts Insufficient Security TLS Fatal Alerts User Canceled TLS Fatal Alerts No Renegotiation TLS Fatal Alerts Unsupported Extension TLS Fatal Alerts Unknown TCP State Closed TCP State Listen TCP State SynSent TCP State SynRcvd TCP State Established TCP State FinWait TCP State FinWait TCP State CloseWait TCP State Closing TCP State LastAck TCP State TimeWait TCP State Rate Closed TCP State Rate Listen TCP State Rate SynSent TCP State Rate SynRcvd TCP State Rate Established TCP State Rate FinWait

9 TCP State Rate FinWait TCP State Rate CloseWait TCP State Rate Closing TCP State Rate LastAck TCP State Rate TimeWait TCP State Total Closed TCP State Total Listen TCP State Total SynSent TCP State Total SynRcvd TCP State Total Established TCP State Total FinWait TCP State Total FinWait TCP State Total CloseWait TCP State Total Closing TCP State Total LastAck TCP State Total TimeWait TCP Retransmit RX Duplicate ACKs TCP Retransmit RX Out-of-Order Segments TCP Retransmit Fast Retransmits TCP Retransmit Fast Retransmit Segments TCP Retransmit Retransmission Timeouts TCP Retransmit SYN Retransmissions TCP Retransmit FIN Retransmissions TCP Rx Payload Total Bytes TCP Rx Payload Total Bit Rate TCP Rx Payload Good Bytes TCP Rx Payload Good Bit Rate TCP Tx Payload Total Bytes TCP Tx Payload Total Bit Rate TCP Tx Payload Good Bytes TCP Tx Payload Good Bit Rate TCP Events TCP RST Received TCP Events TCP RST Transmitted

10 TCP Events Full Window TCP Events Zero Window TCP Events Max SYN RTNS TCP Events Max RTNS TCP Events Reset Local TCP Events Reset Peer UDP State Closed UDP State Open UDP State Active UDP State Rate Closed UDP State Rate Open UDP State Rate Active UDP State Total Closed UDP State Total Open UDP State Total Active UDP Rx Payload Bytes UDP Rx Payload Bit Rate UDP Tx Payload Bytes UDP Tx Payload Bit Rate User State Init User State Active User State Inactive User State Rate Init User State Rate Active User State Rate Inactive User State Total Init User State Total Active User State Total Inactive

11 WELCOME Welcome to VulcanManager User Manual. You can find useful information about configuring, executing, and analyzing Layer 4-7 performance tests ranging from basic operations to advanced features, whether you are new to VulcanManager or an advanced one. INSTALLATION VulcanManager is a standard Windows application which is supported as the table shows: Statistics Type Minimum System Requirements Recommended CPU Intel Core i5 Intel Core i7 RAM 4 GB 8 GB Required Hard Disk Space 175 MB 175 MB Screen Size 1280x x768 Operating System Windows 7 SP1, (Windows Server 2012 SP2 Standard) Updated Windows 10, (Windows Server 2016 Standard) Updated It is installed as part of Xena s Vulcan L4-7 software release package which can be obtained here. After installation you can find a shortcut to the application in the Start -> Programs -> Xena Network menu and also (if you have selected this during setup) on your desktop. 11

12 GENERAL INFORMATION XENA L4-7 STATEFUL TESTING ARCHITECTURE Different from L2-3 stateless packet/stream blasting, VulcanManager establishes stateful TCP connections between two test ports to test the performance of the device under test (DUT). Thus, there is no concept of streams in VulcanManager, which commonly is found in L2-3 testing, but connections/sessions instead. One TCP connection consists a pair of sockets, a client IP, a client port number, a server IP, and a server port number. There are some vital concepts used in VulcanManager. End-to-End Testing Xena L4-7 test platform provides an end-to-end solution for stateful testing. Each test scenario requires two test ports to emulate users and servers. The two ports on Xena L4-7 chassis communicate with each other in an end-to-end fashion, that is, connections between the two ports are end-to-end established. Xena L4-7 Application Application Session device under test (DUT) Session Transport end-to-end TCP communication Transport Application Network Session Network Data Link Network Transport Network Network Data Link Physical Data Link Physical Data Link Physical Data Link Physical Physical Subnet and Users Subnet is a very important concept in VulcanManager. A subnet contains a range of consecutive IP addresses and is presented by a Classless Inter-Domain Routing (CIDR) block, e.g /8 or /16. After being configured, a subnet can be assigned a role as Client Subnet or Server Subnet for a specific test. VulcanManager allows you to define subnets and configure test scenarios according to actual requirements. Client Subnet Role Client Subnet is used only by the client side. All the IP addresses in the Client Subnet will play the roles of clients when establishing TCP connections, which transmit TCP SYN packets. Server Subnet Role Server Subnet is used only by the server side. All the IP addresses in the Server Subnet will play the roles of servers when establishing TCP connections, which wait for TCP SYN packets and respond with TCP SYN-ACK packets. 12

13 User is another important concept in VulcanManager. A User, i.e. emulated user/client, represents one IP address in the Client Subnet. One User can have one or multiple ports assigned. Thus, the number of Users is always equal or less than the number of TCP connections. A typical example is that a User can emulate a PC in reality, where the PC is assigned only one IP address and communicates with servers using multiple ports. DUT client test port server test port One USER represented by a single IP address and one/multiple ports : : : TCP connection TCP connection TCP connection TCP connection Client Subnet /8 Server Subnet /8... One server represented by a single IP address and one/multiple ports : : : Client Subnet /16 Client Subnet /24 Server Subnet /16 Server Subnet /8 Subnets GETTING STARTED WITH VULCANMANAGER This section helps you get started using VulcanManager to create and analyze Layer 4-7 performance. Before proceeding, please ensure you have the latest version of VulcanManager and the latest firmware installed on the chassis. The design philosophy of VulcanManager is to keep it simple for you to configure and run test scenarios, analyze results, and generate reports. To complete a Layer 4-7 test, there are four main steps to follow: Connect to Chassis Set Up Test Case Execute Test View Statistics and Report The following sections will describe these steps in general so that you know the basic test framework of VulcanManager. With the understanding of the concepts adopted in VulcanManager, you will be able to independently set up complex tests to verify the performance of their device under test (DUT). For this initial test, it is recommended to connect the two ports back-to-back on the chassis with a cable. 13

14 Our first step will be to show you how to set up a simple bi-directional raw TCP test scenario. Configuring Your First Test Project A test project is a self-contained collection of ports, subnets, traffic profiles, etc. in a directory commonly found in C:\Users\<username>\Documents\Xena\VulcanManager. 1. Open VulcanManager and choose the Client <-> Server Topology template from Featured quick start templates. This template offers an end-to-end testing where the test platform acts as both client and server. Once opened, notice the three application tabs in the menu; Edit, Run Test and Reporting. These are the three contexts for setting up a test, executing it, observing real time statistics, at last, analyzing the collected statistics, and generating reports. 2. Press Add Chassis button located in the Edit tab. Enter the IP address of the chassis, use the default port number and password for the chassis. Upon arrival, the chassis will be preconfigured with: IP address , port and password xena. If you are using Xena s Live Demo system, the IP address is and the password is xena. 3. After the chassis is added, right click a port to quickly reserve it. If your username appears next to the port, it means the port is yours to use. Do it to another port because you need at least two ports to create Notice that, anywhere in the application, when you see the port icon, you can right-click to get the context menu for port operations. 4. If you only want to see the ports that you use, you can check the checkboxes Used beside those ports and then click the Show Only Used Ports checkbox on top of the Chassis Explorer. Configuring Test Case In the Edit context, you will see two buttons in the Explores group. Use these buttons to switch between Chassis Explorer and Test Explorer. 1. Press Test to switch to test setup. In the Test Explorer, you will see the Test Project tree and with a test case selected, you can see the list view of traffic scenarios and their subnet, port association. 2. Press Add Scenario and choose Raw from the Stateful Loading branch in the Scenario Selector dialog. Select subnets and ports for the client and server part of the traffic scenario. 3. You have successfully created a test scenario of RAW type with default settings. Assign a port you have reserved to Client Port and Server Port, respectively. 14

15 Run Test Case The Run Test tab is a dashboard with a summary of realtime statistics. 1. Press the Run button in the top menu. It turns Stop when the test is started. 2. Click the test case icon in the Realtime Statistics Explorer to see a summary of realtime statistics. The realtime statistics are grouped into three columns. Layer 5-7 client statistics from the client side are on the left. Layer 4 statistics are in the middle. Layer 1-3 client statistics are on the right. 3. See the progress bar of Concurrent Users (user) and Layer 5-7 Throughput (bps) above the statistics table. 4. When the test is finished, the Stop button turns Reset. 5. To see all statistics, go to the Reporting tab. 15

16 Statistics and Generate Report During the test run, counters are sampled and stored. In Statistics Explorer, statistics are grouped into Client and Server. 1. Expand Client and Server to check port related and scenario related statistics. 2. Select statistics to see them in the chart. 3. Click the checkbox next to a statistic to Include Chart in Report. 4. Click Create PDF to generate a test report. 5. Click Save Statistics DB to save all statistics into SQLite database storage file. 16

17 RESOURCE RESERVATION This section explains how chassis resources are reserved, released, and relinquished. CHASSIS RESOURCES Chassis resource can be either the entire chassis, a test module card on the chassis or a test port on a module. Xena test equipment supports multiple simultaneous connections from any mixture of Xena clients, such as the VulcanManager, scripting clients, etc. As soon as a user/client has successfully established a connection to the chassis, any chassis resource can be inspected. However, in order to change the resource configuration, the resource must first be reserved by the user/client. RESERVATION MECHANISM Only one client can reserve a particular resource at a time. The reservation will be active even if the client is disconnected. If the client reconnects later and identifies itself with the same username, any such left-over reservations will automatically be transferred to the new connection. The reservation belongs to a combination of the connection ID in the chassis and the specified username. The username is simply used as a tag for the reserved resource, and the chassis has no notion of actual user accounts. Multiple connections could use the same name, but any resource will only be reserved to one connection at a time. The default username for the VulcanManager is the Windows username for the current user. You can change the username for VulcanManager in the Options menu. The username can contain up to 8 characters. Reserving a Resource To reserve resource, you select the resource in the Chassis Explorer tree view and click the Reserve Resource button in the ribbon menu. Alternatively, you can right-click the resource and select the equivalent menu item. Once you have reserved the resource all configuration options for that resource will now be enabled. Releasing Resource To release any resource reserved by yourself you select the resource in the tree view and click the Release Resource button in the ribbon menu. Relinquish Resource To forcibly take away a resource from another user you can select the Relinquish Resource option instead. You will be prompted to confirm this action before it is executed. RESERVATION HIERARCHY Reservations are hierarchical exclusive, which means that if user Albert has reserved a given test module then user Bertha will be prevented from reserving any port on that module. The same applies to chassis reservations. However, user Albert does not reserve the ports on the test module by reserving the test module itself. 17

18 In general, you do not need to reserve modules and chassis to perform normal traffic generation operations. You should only reserve ports. Reserving modules and chassis are only necessary when performing system maintenance, software upgrades or changing the port types or packet engine allocation on certain ports. module module reserve reserve reserve cannot reserve ports ports 18

19 CHASSIS MANAGEMENT CONNECTING TO A CHASSIS You can add a chassis by pressing the Add button in the main Edit ribbon menu. You will then see the following dialog window: 1. Fill in the IP address of the chassis in the Chassis Address field. 2. Optionally change the Chassis Port Number value if you connect to the chassis through a NAT router that changes the port number. The default port value is If you have changed the port value and want to revert to the default value you can press the Reset to Default button. 3. Enter the assigned password for the chassis in the Chassis Password field. 4. Press the OK button. The next time you open the Add Chassis window it will remember the last values you entered. If you have changed the port number and need to revert to the default Xena port number just press the Reset to Default button. EDITING CHASSIS INFORMATION If you need to modify the address or password details of a chassis, you can select the chassis in the Chassis Explorer and make changes on the panel to the right. Note that the panel will only be enabled for editing if you have reserved the chassis. Remember to the Reboot Chassis after your editing in order for the changes to take effect. RECONNECTING TO CHASSIS If you have lost the connection to a chassis, for instance due to a local network connectivity outage, you can manually reconnect by selecting the chassis in the resource tree view and press the Reconnect button in the ribbon menu. This action is also available in the right-click context menu for the chassis item in the tree view. DISCONNECTING FROM CHASSIS You can forcibly disconnect from a chassis without removing it from your project. This will also prevent VulcanManager from making any attempt to reconnect to the chassis, until you specifically choose to reconnect to that chassis. You can use this option if you have a chassis defined in your configuration that you know will be offline for a longer period of time. 19

20 REMOVING CHASSIS If you no longer need a certain chassis in your project, simply select the chassis and press the Remove Chassis button in the ribbon menu. This action is also available in the right-click context menu for the chassis item in the tree view. TROUBLESHOOTING If the password is lost: The default value of the password is xena, which can be changed from the Chassis panel of VulcanManager. If the password is forgotten, in the first two minutes after the tester has booted (when the test-port LEDs start flashing) you can use the chassis serial number as the password. The serial number is also printed on the label on the back of the chassis. Once logged on, (re)set the password. If the IP address is lost: If you forget the IP address of the chassis you need another way to get in touch with it. This is done by making a point-to-point connection from your PC to the Ext Ethernet port. The Ext port is located to the right side of the Mgmt port on the VulcanBay chassis. The Ext port is pre-configured with the following IP setup: Address: Subnet: Gateway: none You must configure your PC port statically to an IP address in the x range, and then you will be able to ping the chassis again. Now start VulcanManager, and connect to the chassis using the IP address Under the chassis Main Properties, you can see which IP address is configured for the Mgmt port, and you can reserve the chassis and change it if necessary. Changes to the IP address of the Mgmt port take effect after rebooting the chassis. Note that the IP configuration of the Ext port cannot be changed, and that you should not configure the Mgmt port to use this subnet. 20

21 LOAD TEST SCENARIOS Different load test scenarios use different communication patterns. There are three load test scenarios you can create, None, Raw, and Transaction/HTTP GET. Click Add Scenario button, to see the dialog window for scenario selection as shown below. NONE SCENARIO A None type scenario provides the simplest communication pattern: establishes connections, keep connections, and close connections. You can use this scenario to test DUT s capability of connection per second (CPS), concurrent connections, or simply a connection ping (whether a connection can be established or not). client server OPEN CONNECTION CLOSE CONNECTION 21

22 RAW SCENARIO A Raw type scenario establishes connections, generating traffic, and close the connections. The direction of the traffic can be configured as client-to-server (upload), server-to-client (downstream), or bidirectional. client server client server client server OPEN CONNECTION OPEN CONNECTION OPEN CONNECTION data data data data data data data data data CLOSE CONNECTION CLOSE CONNECTION CLOSE CONNECTION upload download bidirectional TRANSACTION SCENARIO Different from Raw scenario, a Transaction scenario uses the request-response commutation model. After a connection is established, the client will send a message to the server to emulate a request. Upon receiving the request message, the server will respond a message to the client. This communication model is so called client-server model. It is the fundamental communication framework for the majority of applications on the internet. Many applications are based on this communication model, e.g. WWW, , network printing. client server OPEN CONNECTION request response request response... CLOSE CONNECTION 22

23 HTTP GET SCENARIO HTTP GET scenario is a special Transaction type in that the content of the request and response are predefined to be HTTP request and response. The content of the request message is predefined to: GET / HTTP/1.1\r\n Host: The content of the response message is predefined to: HTTP/ OK\r\n Content-Length: 1\r\n Content-Type: text/plain\r\n \r\n A\r\n client server OPEN CONNECTION HTTP request HTTP response HTTP request HTTP response... CLOSE CONNECTION 23

24 TEST WITH TLS Xena adds TLS feature in Release 75.0 with native TLS implementation supporting TLS 1.2 standard. Like OSI model, TLS is considered as a layer in the scenario tree. When creating a scenario, you should choose whether or not to include TLS as a layer above TCP. Once you choose TLS, it will appear as shown below: RECORD SIZE You can define the TLS record size (outgoing record size) in the SSL Record Size field. The maximum record size is 16 KB (16384 bytes). CERTIFICATE You can choose server certificates of different key sizes in SSL Certificate from 1K to 8K. To export the certificate to your PC, click Export. CIPHER SUITE COLLECTION Both client and server sides have a list of preferred cipher suite that is used during the TLS handshake phase. To edit the cipher suite list, click Edit. TLS CLOSE NOTIFY You can enable the TLS client to send CLOST NOTIFY message to indicate the end of SSL data transmission. To do so, check the Send Close Notify box. 24

25 SUBNET CONFIGURATION Subnet is a very important concept in VulcanManager. A subnet contains a range of consecutive IP addresses and is presented by a Classless Inter-Domain Routing (CIDR) block, e.g /8 or /16. After being configured, a subnet can be assigned a role as Client Subnet or Server Subnet for a specific test. You can create a subnet for either IPv4 or IPv6 addresses. A subnet defines a range of IP address where the emulated users or servers are located, e.g. an office LAN network connecting a number of PCs. A physical port on the chassis should always be assigned with a subnet. In the Main Properties panel of subnets, you can: Specify IP address range for clients (and servers) Specify the MAC address for clients Specify gateway Enable/disable ARP for MAC address resolution Enable/disable VLAN NETWORK ADDRESS AND NETWORK MASK BIT The Network Address and Network Mask Bit (only for IPv4 subnet) fields together define the Classless Inter-Domain Routing (CIDR) block of the subnet, e.g /8. Users can modify both fields to form the CIDR block for testing purposes. LIMIT ADDRESS RANGE By default, this option is disabled. This will offer a full range of IP addresses available in the CIDR block. For the CIDR block of /8, the available IP addresses will be from to , resulting in 16,777,214 addresses (2 24-2). If this option is enabled, users can limit the number of IP addresses in the subnet by providing the first and the last address. 25

26 USE GATEWAY By default, this option is disabled. This option should be enabled when the port is connected to a remote IP gateway that locates on the DUT. You can enable Use ARP in order to automatically resolve the MAC address of the remote gateway. If the ARP resolution fails, the address defined in Gateway MAC Address will be used as the destination MAC address. GATEWAY ADDRESS When this option is enabled, users can enter the IP address of the gateway function that this subnet is connected to, e.g for /8 CIDR block. GATEWAY MAC ADDRESS When the Use Gateway option is enabled, users can manually enter the MAC address of the port on the DUT that this subnet is connected. NAT By default, this option is disabled. This option should be enabled on a server subnet when the DUT does source NAT, i.e. changing the source IP addresses of the traffic sent from the emulated clients. EMBED IP ADDRESS IN MAC AND ETHERNET ADDRESS By default, this option is enabled. If Embed IP Address is enabled, each emulated client has a unique MAC address. If Embed IP Address is disabled, each emulated client has the same MAC address as defined in Ethernet Address field. USE ARP By default, this option is disabled. If Use ARP is enabled, the test port that is assigned with this subnet will send ARP requests before test traffic begins. Upon receiving the ARP request, the test port will send ARP reply accordingly. ENABLE VLAN By default, this option is disabled. You can enable VLAN and input the VLAN PCP, VLAN DEI, and VLAN Identifier. 26

27 CONNECTION CONFIGURATION You build your load profile in Connection Establishment inside your test scenario. It describes the load profile of your test, i.e. how many users/connections you want to generate, how fast you want these users/connections to be generated, how long these connections should last, and so forth. SUBNET SELECTION You can assign subnets in the Subnet Selection part. As soon as you have selected a subnet, you will see the available IP address from that subnet and how many IP addresses are used in your test scenario. USER CONNECTION SETUP You define the relation between connections and users in this part. Since a TCP connection is defined by a 4-tuple, that is, source IP, source port, destination IP, and destination port, you can change the number of connections per user using Number of Source Ports, Number of Destination IP Addresses, and Number of Destination Ports. CONNECTION ESTABLISHMENT PROFILE You can add multiple load segment as shown below to build the load profile you want. Each row consists of the number of users, offset time, ramp-up duration, steady duration, ramp-down duration, and time scale. 27

28 PAYLOAD CONFIGURATION PAYLOAD IN RAW SCENARIO You can define the application-layer content in the payload configuration panel by expanding the test scenario tree and click either the Download or Upload depending on your traffic direction mode. Payload Pattern defines how the payload content looks, and it provides four modes: Increment, Random, LongRandom, and Custom. Length Type defines the length of the application-layer payload. The Infinite means that the applicationlayer content is large enough so that there will always be payload from the application layer during the test. The Finite means that the application layer has limited content to send to the lower layer, and you can define the length in the Patter Length field. The example below will limit the applicationlayer content to random bytes. As soon as these bytes are transmitted by the lower layer, the application layer will stop generating traffic. PAYLOAD IN TRANSACTION SCENARIO You can define the application-layer content in the payload configuration panel by expanding the test scenario tree and click either the Request or Response. In the Payload Editor, you can freely import or remove the content. The length the of the total applicationlayer content will be automatically calculated. 28

29 TEST CASE CONFIGURATION You can create multiple test scenario in one test case. Each test scenario can be configured differently, i.e. subnet, port, number of users, and line rate utilization. Click the checkbox Active to enable/disable a test scenario. To assign bandwidth among test scenarios, use the two columns Client TX Weight and Sever TX Weight. Bandwidth is evenly distributed among test scenarios that use the same physical test port. In order to prevent the value from changing after you modifying it, click the lock icon to lock the value. 29

30 RUN TEST CASE TEST CONTROL Click Run Test->Test Control->Run to run your test case. Click Run Test->Test Control->Stop to stop your test case. Click Run Test->Test Control->Reset to clear counters and reset your test case so that you are ready for another test run. REALTIME STATISTICS CHART Click Run Test->Tools->Live Charts and select charts from the dialog window Add Realitime Chart. Expand the tree and select the statistics you want to add to a live chart. 30

31 GENERATE REPORT Go to Reporting tab, and you will see the Statistics Explorer. All statistics gathered by VulcanManager are shown in this tree. Click the box beside each statistic to include into the report. Then click Create PDF in the ribbon to generate a PDF report. To save all statistics into a SQLite database file for post data analysis, click Save Statistics DB. 31

32 ADVANCED FEATURES CHANGE PORT SPEED You can change the port speed between 1 Gbit/s and 10 Gbit/s. Go to Chassis Explorer and click a port. Use the Layer-1 Control->Port Speed Selection to change the port speed mode. CHANGE PORT TX LINE RATE UTILIZATION In order to reduce the maximum available TX bandwidth of a port, you can change the port TX line rate utilization. Go to Chassis Explorer and click a port. Use the Layer-1 Control->Port TX Line Rate Utilization to change. As shown below, you will limit the maximum TX bandwidth of a 10G port to 100 Mbit/s (10 Gbit/s x 1.00%) STATEFUL LAYER-4 PAYLOAD REPLAY Traffic replay is an effective way to emulate the real-world network conditions to your device under test (DUT) since all the traffic content, packet sizes, inter-packet gap, packet pattern, and burstiness are from reality. A Replay scenario is different from the four load test scenarios in that the communication content is based on the uploaded PCAP file that is capture from reality instead of modeled traffic, which cannot be found in real-world network. Upon receiving the uploaded PCAP file, the PCAP parser looks for one-to-many communication based on the assumption that the captured traffic is from one client connecting to different servers. If the PCAP contains many-to-one sessions where multiple clients connect to one server, the parser will consider the server as the client and mirror the communicaion to one-to-many. This could in some cases result in incorrect replay. If the pcap contains many-to-many communication, the parser will fail the verification. The PCAP parser will extract the TCP/UDP payload and use Xena TCP/UDP engine to replay statefully. You can scale the traffic load by increasing the number of users. Each emulated client will be assigned a different IP address and communicate to the same group of servers. The basic use flow is as follows: 32

33 1. Create a Replay type scenario 2. Choose your PCAP file to upload to the chassis 3. Chassis parses the PCAP and extract the payload for replay 4. Configure your subnets 5. Configure load profile 6. Configure Layer-4 parameters 7. Run traffic To read more about the benefits of using Xena s stateful Layer-4 payload replay, you can download the white paper here: APPLICATION EMULATION You can perform application emulation using VulcanAppMix by selecting the needed protocols and application traffic in the predefined library. The library has three categories, application-oriented, protocoloriented, and application mixes. Click Add Scenario and select Application Library tab. You can pick applications and protocols from the VulcanAppMix library to create your own mixes in a test case, or choose the predefine mixes from Xena. You can scale the traffic load by increasing the number of users. Each user will be assigned a different IP address and communicate to the same group of servers. To read more about the benefits of using VulcanAppMix for application emulation, you can download the white paper here: CONNECTION UPDATE Use Connection Updates when testing a DUT with user connection repetitions. By default, it is set to No rebirth, which means that as soon as a user completes its TCP connection(s), it will stop generating traffic. The other two options are With same Src IP and With new Src IP. Choose With same Src IP, when user connections should be recreated with the same source IP address. 33

34 Choose With new Src IP, when user connections should be recreated with a different source IP address. CAPTURE TRAFFIC TO PCAP Test ports are able to capture traffic during the test. The capture traffic is saved into PCAP files and stored on the chassis. After the test is finished, you can download the PCAP files to your PC for analysis and debug. To enable the capture, check the box Capture to Wireshark in Run Test->Settings ribbon before running the test. After the test is done, right-click a test port and click Wireshark capture to download the PCAP file to your PC. Your PC will automatically open the PCAP file using Wireshark, if Wireshark is installed. 34

35 XENA LAYER 4-7 SCRIPTING API The Xena layer 4-7 products can be controlled via Xena scripting API. The API uses the same text based protocol as for the Xena layer 2-3 platform. A few commands (mainly chassis commands) are the same as for the layer 2-3 products, but the layer 4-7 products feature several other commands for setting up and controlling TCP connections. For a detailed description of the building blocks of L4-7 scripting please refer to the Scripting Specification (pdf) in the link below. You can implement scripting solutions in any of your favorite scripting languages, including but not limited to BASH, Tcl, Ruby, Python, Perl. We provide a small python (2.x) script collection (see Github link below) which demonstrates how you can build up a set of library functions upon which you can build powerful tests. L47 Scripting Specification (pdf) Python Scripts on Github (for Python version 2.x) The Python script bundle is provided on an as-is basis and are not subject to support or feature requests. Please note that the scripts are based on Python 2.x and that they will not run on Python 3.x. This is due to a change in function syntax for the print command. It is straight forward to correct: Simply substitute all occurrences of print string or text with print( string or text ). 35

36 STATISTICS You can view statistics in both Run Test->Tools->Live Charts, or Reporting->Statistics Explorer. The former is used to view real-time charts, and the latter is used when the test is completed. VulcanManager supports multiple test scenarios using the same pair of test ports. Thus, statistics are grouped into two levels: Port-level statistics show counters per port. Scenario-level statistics show counters per scenario PORT-LEVEL STATISTICS Ethernet Rx Bit Rate Reception rate of Layer-2 traffic in bits/s. Ethernet Rx Packet Rate Reception rate of Layer-2 traffic in packets/s. Ethernet Rx Bytes Total number of bytes of received Layer-2 traffic. Ethernet Rx Packets Total number of packets of received Layer-2 traffic. Ethernet Tx Bit Rate Transmission rate of Layer-2 traffic in bits/s. Ethernet Tx Packet Rate Transmission rate of Layer-2 traffic in packets/s. Ethernet Tx Bytes Total number of Layer-2 bytes transmitted by the port. Ethernet Tx Packets Total number of Layer-2 packets transmitted by the port. Ethernet Events Tx Errors Number of transmission errors reported by the port. Ethernet Events Rx Errors Number of reception errors reported by the port. Ethernet Events Rx Missed Number of packets lost by the Ethernet driver due to RX queue overflow Port Rx VLAN Tagged Packets Number of 802.1! VLAN tagged packets received by the port. 36

37 Port Rx Bit Rate Reception rate of Layer-1 traffic in bits/s. Port Tx VLAN Tagged Packets Number of 802.1! VLAN tagged packets transmitted by the port. Port Tx Bit Rate Transmission rate of Layer-1 traffic in bits/s. Port Events Invalid Ethernet Packets Number of invalid Ethernet packets received by the port. Port Events Unknown Ethernet Packets Number of unknow or unsupported Ethernet packets received by the port. Port Events VLAN Errors Number of packets with mismatching VLAN info received. Port Events Packet Rate Limit Hit Number of times that number of packets transmitted has been limited by the maximum packet rate limit. IPv4 Rx IPv4 Packets Number of IPv4 packets received by the port. IPv4 Tx IPv4 Packets Number of IPv4 packets transmitted by the port. IPv4 Events IPv4 Checksum Errors Number of IPv4 packets with IP header checksum error received by the port. IPv4 Events Invalid IPv4 Packets Number of malformed IPv4 packets received by the port. IPv4 Events Unknown IPv4 Packets Number of IPv4 packets with unknown protocol received by the port. IPv6 Rx IPv6 Packets Number of IPv6 packets received by the port. IPv6 Tx IPv6 Packets Number of IPv6 packets transmitted by the port. IPv6 Events Invalid IPv6 Packets Number of IPv6 packets received by the port, which are malformed. IPv6 Events Unknown IPv6 Packets Number of IPv6 packets received by the port with unknown protocol. 37

38 ARP Rx ARP Requests Number of ARP requests received by the port. ARP Rx ARP Replies Number of ARP replies received by the port. ARP Tx ARP Requests Number of ARP requests transmitted by the port. ARP Tx ARP Replies Number of ARP replies transmitted by the port. ARP Events Invalid ARP Packets Number of invalid ARP packets received by the port. ARP Events ARP Request Lookup Failures Number of ARP requests received that cannot be resolved. ARP Events ARP Reply Lookup Failures Number of ARP replies received that cannot be resolved. ARP Events ARP Request Retransmits Number of ARP requests retransmitted. ARP Events ARPs Resolved Number of IP addresses that are correctly resolved by ARP. ARP Events ARPs Failed Number of IP addresses that are not resolved by ARP. ARP Events ARP Table Lookup Failures Number of destination IP addresses not found in the ARP table. NDP Rx NDP Requests Number of NDP requests received by the port. NDP Rx NDP Replies Number of NDP replies received by the port. NDP Tx NDP Requests Number of NDP requests transmitted by the port. NDP Tx NDP Replies Number of NDP replies transmitted by the port. NDP Events Invalid NDP Packets Number of invalid NDP packets received by the port. NDP Events NDP Request Lookup Failures Number of NDP requests received that cannot be resolved. 38

39 NDP Events NDP Reply Lookup Failures Number of NDP replies received that cannot be resolved. NDP Events NDP Request Retransmits Number of NDP requests retransmitted. NDP Events NDP Resolved Number of IP addresses that are correctly resolved by NDP. NDP Events NDP Failed Number of IP addresses that are not resolved by NDP. NDP Events NDP Table Lookup Failures Number of destination IP addresses not found in the NDP table. ICMP Rx ICMP Echo Requests Number of ICMP Echo (ping) requests received by the port. ICMP Rx ICMP Echo Replies Number of ICMP Echo (ping) replies received by the port. ICMP Rx ICMP Destination Unreachable Number of ICMP Destination Unreachable messages received by the port. ICMP Rx ICMP Time Exceeded Number of ICMP Time Exceeded messages received by the port. ICMP Tx ICMP Echo Requests Number of ICMP Echo (ping) requests transmitted by the port. ICMP Tx ICMP Echo Replies Number of ICMP Echo (ping) replies received by the port. ICMP Tx ICMP Destination Unreachable Number of ICMP Destination Unreachable messages transmitted by the port. ICMP Tx ICMP Time Exceeded Number of ICMP Time Exceeded messages transmitted by the port. ICMP Rx ICMPv6 Packets Number of ICMPv6 packets received by the port. ICMP Tx ICMPv6 Packets Number of ICMPv6 packets transmitted by the port. ICMP Events Invalid ICMP Packets Number of invalid ICMP packets received by the port. ICMP Events Unknown ICMP Packets Number of unknown ICMP packets received by the port. 39

40 ICMP Events Invalid ICMPv6 Packets Number of invalid ICMPv6 packets received by the port. ICMP Events Unknown ICMPv6 Packets Number of unknown ICMPv6 packets received by the port. TCP Rx TCP Packets Number of TCP packets received by the port. TCP Tx TCP Packets Number of TCP packets transmitted by the port. TCP Events TCP Checksum Errors Number of TCP packets with header checksum error received by the port. TCP Events TCP Invalid Packets Number of malformed TCP packets received by the port. TCP Events TCP Lookup Failures Number of TCP packets received that cannot be resolved. UDP Rx UDP Packets Number of UDP packets received by the port. UDP Tx UDP Packets Number of UDP packets transmitted by the port. UDP Events UDP Checksum Errors Number of UDP packets with header checksum error received by the port. UDP Events UDP Invalid Packets Number of malformed UDP packets received by the port. UDP Events UDP Lookup Failures Number of UDP packets received that cannot be resolved. SCENARIO-LEVEL STATISTICS TLS State Inactive Current number of TLS sessions in INACTIVE state. This state represents waiting for TLS handshake to begin after the TCP connection is established. TLS State Handshaking Current number of TLS sessions in HANDSHAKING state. This state represents TLS is in the process of handshaking. TLS State Handshake Done Current number of TLS sessions in HANDSHAKE DONE state. This state represents TLS has successfully completed the handshake and the session is established. 40

41 TLS State Handshake Failed Current number of TLS sessions in HANDSHAKE FAILED state. This state represents TLS has failed handshake process. TLS State Failed Current number of TLS sessions in FAILED state. This state represents TLS session is failed due to external reason. TLS State Internal Failed Current number of TLS sessions in INTERNAL FAILED state. This state represents TLS session is failed due to internal reason. TLS State Close Notify Current number of TLS sessions in CLOSE NOTIFY state. This state represents TLS has sent a CLOSE_NOTIFY message and waiting to close the session. TLS State Done Current number of TLS sessions in DONE state. This state represents TLS session is correctly closed. TLS State Rate Inactive Rate of TLS sessions in INACTIVE state. This state represents waiting for TLS handshake to begin after the TCP connection is established. TLS State Rate Handshaking Rate of TLS sessions in HANDSHAKING state. This state represents TLS is in the process of handshaking. TLS State Rate Handshake Done Rate of TLS sessions in HANDSHAKE DONE state. This state represents TLS has successfully completed the handshake and the session is established. TLS State Rate Handshake Failed Rate of TLS sessions in HANDSHAKE FAILED state. This state represents TLS has failed handshake process. TLS State Rate Failed Rate of TLS sessions in FAILED state. This state represents TLS session is failed due to external reason. TLS State Rate Internal Failed Rate of TLS sessions in INTERNAL FAILED state. This state represents TLS session is failed due to internal reason. TLS State Rate Close Notify Rate of TLS sessions in CLOSE NOTIFY state. This state represents TLS has sent a CLOSE_NOTIFY message and waiting to close the session. TLS State Rate Done Rate of TLS sessions in DONE state. This state represents TLS session is correctly closed. 41

42 TLS State Total Inactive Total number of TLS sessions in INACTIVE state. This state represents waiting for TLS handshake to begin after the TCP connection is established. TLS State Total Handshaking Total number of TLS sessions in HANDSHAKING state. This state represents TLS is in the process of handshaking. TLS State Total Handshake Done Total number of TLS sessions in HANDSHAKE DONE state. This state represents TLS has successfully completed the handshake and the session is established. TLS State Total Handshake Failed Total number of TLS sessions in HANDSHAKE FAILED state. This state represents TLS has failed handshake process. TLS State Total Failed Total number of TLS sessions in FAILED state. This state represents TLS session is failed due to external reason. TLS State Total Internal Failed Total number of TLS sessions in INTERNAL FAILED state. This state represents TLS session is failed due to internal reason. TLS State Total Close Notify Total number of TLS sessions in CLOSE NOTIFY state. This state represents TLS has sent a CLOSE_NOTIFY message and waiting to close the session. TLS State Total Done Total number of TLS sessions in DONE state. This state represents TLS session is correctly closed. TLS Rx Payload Bytes Number of TLS payload bytes received. TLS Rx Payload Bit Rate Number of TLS payload bits/s received. TLS Tx Payload Bytes Number of TLS payload bytes transmitted. TLS Tx Payload Bit Rate Number of TLS payload bits/s transmitted. TLS Warning Alerts Close Notify Number of TLS alert messages with alert level of warning and alert description of close_notify received. See RFC