Terminal Applications Scalability testing using Rational Performance Tester version 8.1 A practical guide on 5250 Green Screen applications Version: 1.0 Date: 12/05/2009 Author: Benoit Marolleau Product & Solution Support Center IBM Montpellier - FRANCE benoit.marolleau@fr.ibm.com 1
TABLE OF CONTENTS I. Notices...3 II. Introduction...4 III. First Step: Record the scenario...5 a) Recording preparation...5 b) Record the scenario...6 c) Rational Performance Tester specific settings for EBCDIC...8 IV. Variables and datapools...9 V. Variable Substitution...10 a) Java Custom Code for non-ascii substitution...10 b) Retrieve the variables in the client-server dialog...13 c) Insert the custom code...14 d) Substitution from custom code...16 e) Replay the scenario...17 VI. Data Correlation...18 VII. Functional Validation: Verification Points...21 a) Default Screen Validation and Problem determination...21 b) Functional Validation using Custom Code...23 VIII. RPT Infrastructure sizing: example...27 Conclusion...29 Appendices...30 a) Appendix A: EncodeDatapool.java...30 b) Appendix B: DisplayArg.java...31 c) Appendix C: ValidateReceive.java...32 2
I. Notices IBM Corporation 2009. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this publication to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth, savings or other results. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: Rational POWER i5/os POWER6 IBM Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. 3
II. Introduction Many applications still use terminal emulation as graphical interface for end-users: telnet (vt100), 5250 and 3270 emulation, etc. Rational Performance Tester v8.1 allows you to test the quality, performance and scalability of many different types of applications and this document presents how to test your legacy terminal application using RPT and the socket extension. As an example, we decided to test and simulate virtual users on a 5250 application, connecting to an IBM Power Blade JS22 with IBM i V6R1. You can generalize this example to all kinds of terminal emulations. For more information about the Rational Performance Tester product: http://www-01.ibm.com/software/awdtools/tester/performance/index.html 4
III. First Step: Record the scenario a) Recording preparation Create your RPT Project and launch the socket recorder. Then, you specify the program used for terminal emulation. In this example, we used IBM Personal Communications version 5.7 for Windows. Don t forget to unselect the Open console for user input if your emulation software is a window-based application. 5
b) Record the scenario Here is a simple telnet 5250 scenario that we want to simulate. It s a very simple scenario with a login screen, an IBM i menu, a CL command submission, and a logout: Screens Action Variables Login Enter userid and password Main Menu CL command : create a library Main Menu 2 Logout : type 90 on the main menu User profile and password Library name / 6
Login screen: Control language command example: CRTLIB 7
Logout the system: c) Rational Performance Tester specific settings for EBCDIC 8
IV. Variables and datapools Create the datapool, to be used for variable substitution. Don t forget to respect the case: for instance, on IBM i, a user profile (userid field above) has to be uppercase 9
V. Variable Substitution a) Java Custom Code for non-ascii substitution Once the scenario is recorded, you need to replace static strings by some variables. The tricky thing in this version is that you can only substitute from datapools coded in ASCII. If you try to replace a variable from a datapool directly, the system will receive an ASCII value, and an error will occur. To avoid that, we have to use a Java Custom Code which will retrieve a new value from the datapool, encode it into EBCDIC (in this example) and return this new encoded value to the caller. Then, the caller will use this returned value for the substitution Usually, you will need one custom code per datapool, which will retrieve First create a custom code: 10
Then insert the following code in it: 11
package test; import java.io.unsupportedencodingexception; import com.ibm.rational.test.lt.execution.socket.holder.sckconnectionholder; import com.ibm.rational.test.lt.kernel.services.itestexecutionservices; public class EncodeDataPool implements com.ibm.rational.test.lt.kernel.custom.icustomcode2 { public EncodeDataPool() { public String exec(itestexecutionservices tes, String[] args) { try { String encodedstring = SckConnectionHolder.getString(args[0].getBytes("Cp1047")); tes.gettestlogmanager().reportmessage("ascii field "+ args[0] +" translated to EBCDIC: "+ encodedstring ); return encodedstring; catch (UnsupportedEncodingException e) { // TODO: log something return null; This code retrieves the ASCII datapool cell passed in parameter and return the corresponding EBCDIC-encoded value. 12
b) Retrieve the variables in the client-server dialog To replace your static strings contained in a «send» message, first select the message: Once selected, visualize the EBCDIC-encoded data sent: You can see that there are two input fields: the userid (PSSC) and the password (ABC123). 13
On another send message, another command to make dynamic: the name of the library created by each user (TESTLIB): c) Insert the custom code For each variable you have to substitute, copy the custom code in the test, and customize the comment for each custom code related to a given variable. In this example, Username translation, Password translation and Library translation. 14
The custom code has to be inserted so that its scope is accessible where you have to use it in most cases, put it at the beginning of the test, or if your test includes a loop and if you have to get the next value at each iteration, inserting it at the beginning (or at the end) of that loop is a good practice. Then, just add for each custom code (.i.e. each variable in this example), a parameter, which is the name of the field in the datapool: the custom code will retrieve a new value from this field in the pool, and encode it (see the sample code) to the good coding. 15
d) Substitution from custom code Select the piece of data to be replaced by the value in the datapool on the right side (specific coding, here EBCDIC), then switch to the Raw Ascii view: the data is still selected, and you just have to right click on it and choose a substitution. If you see that screen, your substitution from each respective custom code is successful: 16
Fig Username and password will be replaced by their respective custom code. e) Replay the scenario To perform a unit test with several users, insert the test in a schedule; Then control on your system/application that your users are connected, which means that your substitution works properly: 17
Of course, the best way to make sure that everything is correct and validate the substitutions is to check the Test Logs. VI. Data Correlation In particular cases, it is necessary to correlate a server response to a subsequent client request. For example, it may be useful to retrieve an Order ID generated by the application and displayed on the screen, and use it on the next screen as an input. In this example, we simply get the name of the library which has just been created by the previous user validation. We just have to select the correct data on the receive element: Then we create a field reference on it: 18
This data is now usable for another subsequent request. Here is an example of Java Custom Code which uses this value and displays it in the logs: Fig. DisplayArg Custom Code. In parameter, the field reference. 19
This Java class get this EBCDIC value in parameter, encode it into ASCII and display it in the Test Logs. The code is available in Appendix. Result: Of course, in real testing scenarios, the correlated value could be used by another subsequent request and not only displayed as we do here. 20
VII. Functional Validation: Verification Points a) Default Screen Validation and Problem determination You may want to validate that your application replied correctly to the workload. When we use the socket extension, the validation is by default a low level validation: it checks, in each receive element that we receive the recorded number of bytes. This validation is called End Policy. Here is what you can see in the test logs after a screen failure: 21
On the right side, you can see the following statement: Error: The timeout of 10s has elapsed before the end policy has completed Received 569 bytes 0ms after last byte has been sent End of stream not reached Response timeout: 10s Policy: Receives exact number of bytes [expecting 741 bytes, timeout: 10s] And we investigate in the logs to see what happened in the application, we see: User USER4 does not exist In the report, you will see: If you receive an unattended screen during the replay, then you will only obtain receive timeouts (see picture above) because the tool was waiting for another screen,.i.e. another number of bytes, and the default behaviour for each receive is to check the size of the received message. Subsequent receive element can then rise other timouts and/or failures. It s possible to change the End Policy but this validation remains a very low-level validation, not comparable to Verification Points used with other RPT extensions. 22
b) Functional Validation using Custom Code You may want to validate the execution of your scenario using a higher level means, analyzing the content of a response screen, for example. There are several ways to perform this task with Rational Performance Tester: In this example, we used a Java Custom Code which parse a response from the server and try to find a given validation string in parameter. If the response contains the string in parameter, we create a Verdict Pass. If not, we create a Verdict Fail (see picture below). These verdicts are visible in the socket report at the end of the run. NB: See the custom code class ValidateReceive in appendix. This code is generic and may be customized to fit your more specific needs. Fig. Functional validation in the Test Reports Example: To illustrate a functional validation, let s consider the following simple example. We login an IBM i system and we want to check that the virtual user passed the next step in my scenario: we just have to check that the next screen is the IBM i main menu. This menu contains the string System i Main Menu. 23
1) Create a field reference 24
2) Define the string: validation criteria This string could also come from a datapool, a reference, a custom code, etc. In this example, we chose to create a variable visible from the whole test. 3) Insert the Custom Code Insert the verification point Custom Code after the receive element to analyze: The two parameters here are the static string and the previous receive element (field reference) which has to include the string to pass the validation: 25
4) Result in the report Fig. validation failure 26
Fig. The Verdict Pass in the logs (left), means that the custom code found the string in the response data (right). Here the string is System i Main Menu. VIII. RPT Infrastructure sizing: example Unfortunately, there is no universal sizing guide when using a workload simulation tool. In this particular example, we simulate the following scenario: Screens Action Variables Processing / Parsing Login Enter userid and password Main Menu CL command : create a library Main Menu 2 Logout : type 90 on the main menu User profile Parse and replace 2 and password variables Library name Parse and replace 1 variable / Parse the previous response to control library creation CPU intensive: This scenario is really simple: it contains 3 screens (7 send elements, 10 receive elements), 3 variable substitutions, and a functional validation. Each virtual user performs these tasks in an infinite loop, 1 time per minute. 27
Memory usage: We use 2 relatively small datapools (total: 3000 string cells). We decided to simulate on the same injector (PC 1x3 GHz, 1GB RAM) sequentially 400, 600 and 800 virtual users in a performance schedule, and we monitor the system activity on this machine to see how it reacts. Fig. Memory (light blue) and CPU (pink) consumption during 3 steady states: 400 (red), 600 (green) and 800 (dark blue) users: CPU Average: 10% Here is the memory consumption model of the agent (java.exe), deducted from several tests: 28
RPT Agent: Memory Utilization (MB) 200 150 y = 0,2264x + 42,506 100 50 0 0 200 400 600 800 Nb Virtual Users In this case, each virtual user consumes 225 KB of memory. This model is only valid for our dedicated environment, until 900 virtual users, generating 1.3 KB/s and receiving 8KB/s in total. Conclusion Rational Performance Tester helps you to test the quality and the scalability of your applications, using all kinds of protocols. This document describes what is possible to do with the Socket Extension and its capability of using different encodings. The latest 8.1 version is now a great and easy solution for testing the reliability of your legacy Terminal applications: from the 5250 Green Screen to the 3270 Mainframe terminal session, through all types of terminal emulations. Eventually, it s interesting to notice that this new socket extension consumes a very small amount of memory per virtual user on the injector machines, which makes these scalability tests easier and cheaper to realize. 29
Appendices a) Appendix A: EncodeDatapool.java package test; import java.io.unsupportedencodingexception; import com.ibm.rational.test.lt.execution.socket.holder.sckconnectionholder; import com.ibm.rational.test.lt.kernel.services.itestexecutionservices; /** * @author unknown */ public class EncodeDataPool implements com.ibm.rational.test.lt.kernel.custom.icustomcode2 { public EncodeDataPool() { public String exec(itestexecutionservices tes, String[] args) { try { String encodedstring = SckConnectionHolder.getString(args[0].getBytes("Cp1047")); tes.gettestlogmanager().reportmessage("ascii field "+ args[0] +" translated to EBCDIC: "+ encodedstring ); return encodedstring; catch (UnsupportedEncodingException e) { // TODO: log something return null; 30
b) Appendix B: DisplayArg.java package test; import java.io.unsupportedencodingexception; import java.util.regex.matcher; import java.util.regex.pattern; import com.ibm.rational.test.lt.kernel.services.itestexecutionservices; import com.ibm.rational.test.lt.kernel.services.itestlogmanager; /** * @author Benoit Marolleau */ public class DisplayArg implements com.ibm.rational.test.lt.kernel.custom.icustomcode2 { public DisplayArg() { public String exec(itestexecutionservices tes, String[] args) { ITestLogManager itestlogmgr = tes.gettestlogmanager(); try { Pattern pattern = Pattern.compile("[^0-9a-f@]"); Matcher matcher = pattern.matcher(args[0]); String tmp=matcher.replaceall(""); pattern = Pattern.compile("@"); matcher = pattern.matcher(tmp); String tmp2= matcher.replaceall("40"); String asciistring = new String (ValidateReceive.RawHexaStringToBytes(tmp2),"Cp1047"); itestlogmgr.reportmessage(asciistring); catch (UnsupportedEncodingException e) { // TODO Auto-generated catch block e.printstacktrace(); return null; 31
c) Appendix C: ValidateReceive.java package test; import java.util.regex.matcher; import java.util.regex.pattern; import org.eclipse.hyades.test.common.event.verdictevent; import com.ibm.rational.test.lt.kernel.services.ipdlogmanager; import com.ibm.rational.test.lt.kernel.services.itestexecutionservices; import com.ibm.rational.test.lt.kernel.services.itestlogmanager; /** * @author Benoit Marolleau - IBM PSSC Power Benchmark Center */ public class ValidateReceive implements com.ibm.rational.test.lt.kernel.custom.icustomcode2 { public ValidateReceive() { public String exec(itestexecutionservices tes, String[] args) { ITestLogManager itestlogmgr = tes.gettestlogmanager(); IPDLogManager ipdlogmgr= tes.getpdlogmanager(); String asciistring=""; try { itestlogmgr.reportmessage(args[1]); //Replace all the "non-hexadecimal" characters in the received message args[1] //@ has to be kept and replace by the hexa 40, which is a space character in ebcdic: Pattern pattern = Pattern.compile("[^0-9a-f@]"); Matcher matcher = pattern.matcher(args[1]); String tmp=matcher.replaceall(""); pattern = Pattern.compile("@"); matcher = pattern.matcher(tmp); String tmp2= matcher.replaceall("40"); itestlogmgr.reportmessage(tmp2); //encode the hexa ebcdic string to bytes, and re-encode them to ASCII: asciistring = new String (ValidateReceive.RawHexaStringToBytes(tmp2),"Cp1047"); itestlogmgr.reportmessage(asciistring); catch (Exception e) { e.printstacktrace(); //search the string (arg[0]) in the received message we have just encoded in ASCII // if found, create a Verdict PASS, if not Verdict FAIL: int index = asciistring.indexof(args[0]); 32
if ( index == -1) { ipdlogmgr.log(ipdlogmanager.severe,"string \" "+ args[0] + "\" not found. "); itestlogmgr.reportverificationpoint(args[0]+ " FAIL", VerdictEvent.VERDICT_FAIL); return "-1"; else { ipdlogmgr.log(ipdlogmanager.severe,"string \" "+ args[0] + "\" passed. "); itestlogmgr.reportverificationpoint(args[0]+ " PASS", VerdictEvent.VERDICT_PASS); return "0"; //convert a string in raw hexadecimal to bytes: public static byte[] RawHexaStringToBytes(String hex) { // Example of Raw Hexa = "e2a8a2a38594"; byte[] bts = new byte[hex.length() / 2]; for (int i = 0; i < bts.length; i++) { bts[i] = (byte) Integer.parseInt(hex.substring(2*i, 2*i+2), 16); return bts; 33