Introduction Apache JMeter is the industry s most popular open-source performance testing tool, offering load testing through different processing protocols, e.g. HTML, JDBC, WCF. With the right personnel supporting, this can easily be adopted within any Agile team to provide value during the development phase and beyond. Being an open source tool brings obvious benefits in cost, and its implementation in Java allows for it to be used in different development environment, be it Mac or Windows. The purpose of this document is to get you started with JMeter basics and at a stage where you can run simple tests. To get real value from using JMeter, you will first need to understand Performance Testing concepts and then practice (and explore) the tool further to realise its abilities. All examples going forward will be based on Windows systems. Installation and Startup Before you install JMeter itself, you will need to ensure Java is installed on your machine. If you are not sure if you have this, run the following in Command Prompt: java version. If you do not have Java installed, you can get the latest version from: https://java.com/en/download/. JMeter can be downloaded from: http://jmeter.apache.org/. All you have to do is unzip the downloaded file to a location of your choice. To start JMeter, navigate to the /bin directory of your JMeter installation and open the jmeter.bat (Windows Batch File). There is official documentation on that site to help you on your JMeter journey as well. JMeter GUI The JMeter GUI is quite self-explanatory, with a tree system on the left to manage tests and a top bar of icons for major functions, with drop down menus to complement these. On the left window, the TestPlan is what s executed when the Play button is selected, whereas the WorkBench (which can be ignored) is for draft work. Selecting and Right-clicking on the TestPlan node offers a whole host of test preparation options. As a starting point, apart from the above, the main functions I use are Save and Clear All, of which the latter clears previous test results.
A Simple Test Case The main components of a performance test include: a loading mechanism, the tests and some results output. Thread Group In JMeter, load is applied by the concept of having multiple threads (users). To create a Thread Group, right click Test Plan and select Add -> Threads (Users) -> Thread Group and will appear as a child node of Test Plan as below: In the main window, there are a lot of self-explanatory options available, but the most important ones (from a Load perspective) are Thread Properties: - Number of Threads (users). This specifies the number of concurrent users intended. - Ramp-Up Period (in seconds). The number of seconds before all users (above) are active. For example if there are 10 users, and a ramp-up period of 20 seconds, it will take 20 seconds before all 10 users are active, i.e. 2 seconds for each increase in user load. - Loop Count. How many times each user should repeat the tests, or Forever checkbox can be selected, but will require another method to terminate the tests. DO NOT RAMP UP until you have finished reading this document and are comfortable with the implications of the load you plan to apply. Tests Samplers in JMeter (see Section 6) are a list of Request options, which in turn is the method of performing a meaningful task/test, e.g. a HTTP Request. To add such a request to be part of our created Thread Group, right click on the Thread Group and select Add -> Sampler -> HTTP Request. Again, there are a lot of options available, but for simply navigating to a specified webpage; the following are the basic minimum: - Server Name or IP. - Port Number (if applicable) - Path. For example, http://www.bbc.co.uk/news would be: - Proxy Server. This is the case for those who access the internet behind a proxy server, where you will have to enter your own proxy details. Listeners There are many Listeners available in JMeter that are used to display results, the most commonly of which are: View Results in Table and View Results Tree. To add these to our Test Plan, right click Thread Group, select Add -> Listeners - > View Results in Table and repeat for View Results Tree.
To fully understand which Listener will provide the right metrics desired, I suggest a trial and error approach with the other Listeners, all of which can be added in the same fashion. Execution To execute the Test Plan, simply select Play, but before doing so, have a look at the Thread Group and make sure we are applying meaningful load by default it is one user for a single iteration. Set the Number of Threads to 3 and a Loop Count of 3 before executing. (You may also be prompted to Save your work). As the Load is still minimal, you may not notice this, but while the tests are running in the top right corner of JMeter, there are status indicators of the progress of the testing, and a Green light whilst it is still executing. Once execution is complete, select the Listeners nodes from the left tree structure to see the results. A lot of information is available, and again, depending on your requirements, different items might of interest. As an assumption, a Status of Green is good, as the test case ran successfully. The other obvious important piece of information is Latency, as this is also referred to as Time To First Byte (TTFB), and is a common metric for performance comparison purposes. Assertions Assertions is a commonly used method of unit/component testing during development and JMeter offers a host of different assertions to allow for extra checks that a test has ran correctly. Please note that if a test fails to run or receives an error message, it will be flagged regardless of an explicit assertion being present. I also take this opportunity to remind you that the aims of performance testing is not to find functional faults that said, assertions are useful to check the robustness of the test you are creating and depending on the circumstance, a check of functionality may be required. To add an Assertion, right click on the Sampler (Test) that you want to check, and select the appropriate Assertion from the list. The assertions that are available are quite self-explanatory, and again trial and error would be recommended. Following a test run, if any Assertions have failed, these can be viewed under a View Results Tree Listener. Samplers JMeter offers a host of Sampler options, meaning a wide range of tests can be created for a performance test, many of which align closely to our development practises. The list includes (amongst others): - FTP Request - HTTP Request - JDBC Request - JUnit Request - LDAP Request - SMTP Sampler - SOAP/XML-RPC Request From a Agile development/testing point of view, this allows us to target the database (JDBC Request), API (SOAP/XML- RPC Request) and Web (HTTP Request) layers. Database Requests Pre-requisite For JDBC (Database) Interaction, download and extract the Microsoft JDBC Driver: https://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=11774 and (assuming you are on the latest Java version and running at least Windows 7 64 bit), please ensure that the dll, sqljdbc_auth.dll is copied into C:\Program Files\Java\jre1.8.0_121\bin and the sqljdbc42 JAR file into C:\apache-jmeter- 3.1\lib. You will need to restart JMeter after copying these files.
JDBC Connection Configuration To setup a JDBC Connection in JMeter, you will need to add a JDBC Connection Configuration Config Element in your Test Plan, as a child node to Thread Group, the result of which looks like: Please note that the following will need to be entered: Variable Name anything text, e.g. Test, but JDBC Samplers will need to reference this. Database URL jdbc:sqlserver://123.123.123.123:1234;integratedsecurity=true, address of DBServer JDBC Driver class com.microsoft.sqlserver.jdbc.sqlserverdriver, should be correct if pre-requisites followed Username someusername, should be replaced with your own username JDBC Request A JDBC Request can be added to a Thread Group as per any other Sampler in JMeter, and the Database actions can range from simple Selects and Updates, to executing Stored Procedures. After adding a JDBC Request, set the Variable Name Bound to Pool > Variable Name to the same as your Config Element, i.e. Test, and put your SQL inside the Query window. Please see the screenshot below as an example of a Stored Procedure Call: Next add a View Results Tree Listener and you are good to go. Execute and check your results, which show details of request and time it took to run as well as details of the response as well. API Requests As mentioned briefly before, JMeter offers options to make API calls via their application. In fact, you can make a REST call via the same HTTP Request we used previously, as well as through a dedicated SOAP/XML Request. As a side note, I wanted to highlight a Config Element, HTTP Request Defaults, where all proceeding HTTP requests will utilise; we can for example put our Proxy settings in here once instead of in every HTTP Request. REST API HTTP Request example Through a simple Internet search, I found the following web service: http://api.openweathermap.org/data/2.5/weather?q=london,uk&appid=5ad76b332e2fa27ea9859353e5fdd69d Obviously we can use our own internal ones especially when we start applying load. To implement this as a HTTP request, we can add a HTTP Request in JMeter and enter details broken down as follows:
Please note how the server name, path and parameters are broken down into subsidiary parts, and as per normal, have listeners setup to check the results. REST API RPC Request The same REST API request can be achieved using a SOAP/XML-RPC Request, and entering the full path in the URL. SOAP API RPC Request Again, via the internet, I found a SOAP web service available, but again, an internal one can also be used. To create a SOAP call, create a SOAP/XML-RPC Request. The details you will need are highlighted in the screenshot below: Specifically: URL:http://www.webservicex.net/globalweather.asmx Send SOAPAction (Checked): http://www.webservicex.net/getcitiesbycountry
Soap/XML-RPC Data: <soap:envelope xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:body> <GetCitiesByCountry xmlns="http://www.webservicex.net"> <CountryName>Japan</CountryName> </GetCitiesByCountry> </soap:body> </soap:envelope> Use Listeners as per all other tests to see the outputs of the above. Generating HTTP Requests Section 4.2 already demonstrates a method of creating a HTTP Request and indeed, you can create multiple ones of these to create a more meaningful test (in terms of what a user might do). A more efficient method of doing this is to use a recording tool as the following subsections describe. Remember that the aim is to get a working test case, so whichever method works for you is the best method. HTTP(S) Test Script Recorder JMeter provides a built-in Test Script Recorder which can be added via a right-click against the WorkBench. Looking at instructions on the Apache website, this is a very simple procedure, however, this doesn t work behind a proxy server and it seems easier to record via the BlazeMeter plugin (below). Other Recording Tools There are other online sites that provide a test recording environment, e.g. badboy.com.au, which has a window within its application for copying actions that the user makes. Chrome Blazemeter Plugin This plugin allows recording directly from your Google Chrome browser, which creates a.jmx file that can be exported and opened in JMeter directly. I find this to be the most natural way of recording scripts, but beware, once you ve opened your recorded actions, you may need to remove some of the HTTP headers generated and ensure you have set your proxy settings for execution, see 6.2 Config Element HTTP Request Defaults. Command Line JMeter When running tests, the additional Listeners and Assertions impact the resource utilisation of the JMeter application, and as such, once you have a working test case, it is advisable to toggle off any that are not required. The utilisation is also very high when running in GUI mode, and fortunately JMeter has a command line execution option. This also opens up options for Integration with deployment practices, e.g. triggering a performance test via PowerShell. Navigate to the JMeter Bin directory and run jmeter h to see the full list of JMeter execution options. More specifically, the following command will run the test, jmeter n t [location of jmeter test script] l [location of the result file] where -n -> non-gui mode -t -> jmeter test script location and filename -l -> location and filename of output results As an example, C:\apache-jmeter-3.1\bin>jmeter -n -t c:\apache-jmeter-3.1\bin\api.jmx -l c:\apache-jmeter-3.1\bin\api.csv will execute the api.jmx test plan (assuming you have created a test plan with that name), and append the results to the file api.csv. If the file does not exist, it will create it.