webmethods Task Engine 9.9 on Red Hat Operating System

Transcription

1 webmethods Task Engine 9.9 on Red Hat Operating System Performance Technical Report Software AG. All rights reserved.

2 Table of Contents INTRODUCTION Benchmark Goals Hardware and Software Information DEPLOYMENT ARCHITECTURE DIAGRAM Test Harness Test Setup and Test Conditions Benchmark RESULTS Conclusion Appendix Terminology Product Tuning OS/HW Tuning Build and Fixes What this Report Does Not Cover Software AG. All rights reserved.

3 INTRODUCTION This technical report is one of a series that defines and measures synthetic benchmarks that represent how Software AG products are used. These benchmarks were designed to be readily applied to various real world deployments. This report focuses on the reliability and scalability of Task Engine running on My webmethods Server 9.9 on a multi-core system. The main objective of this report is to demonstrate how a single instance of My webmethods Server 9.9 with Task Engine performs and scales with different scenarios for scheduling and rule evaluation of millions of task instances from different task types, with the same user load and think time. Rule definitions for the task types are different - some use external data sources, and others do not. The intended audiences are application architects, developers and people involved in capacity planning. This report can be used to weigh design-time choices as well as provide input to the hardware selection process Software AG. All rights reserved.

4 1.0 BENCHMARK GOALS The goal of the benchmarks described in this report was to explore the variations in the response time of My webmethods Server UI when Task Engine is running recalculation and rule evaluation of millions of task instances. This report also demonstrates the effect of different rule definition settings. Task Engine uses date/time events to schedule task actions for a particular time. Event triggering times are defined as fixed dates or time intervals, and the time intervals can be calculated using a business calendar, or the output of an Integration Server service. When a task type with a date/time event is published to My webmethods Server, Task Engine stores precalculated values for the triggering time of the event. Those precalculated values might become irrelevant over time, especially when they are based on a business calendar, or the output of a service. For example, a business calendar change can result in an event being triggered on the wrong date. Consider a task type with a date/time event that assigns the task to a certain user group if the task remains unassigned for more than 5 business days after creation. If the business calendar gets updated after you create a task instance with a recalculated triggering time and one of the working days within the 5-day period is marked as a holiday, the date/time event will trigger on the wrong date. A similar situation can occur when the date/time calculations for an event are based on the output value of a service. To avoid such errors, existing triggering times for date/time events can be recalculated manually, when republishing a task type, or periodically, with automatic recalculation settings that fit the workflow of your organization. Tests were conducted by varying the following scenarios and parameters: Scenarios: Scheduling all tasks with different rule definitions: tasks that use Integration Server Service for external data source tasks that use MWS Business Calendar for external data source tasks that have static values for rule definition and do not use external data source. Stress test in which many tasks are scheduled for evaluation at the same time. Performance test to check how the normal user work is affected when massive recalculation is running on the system. Parameters: Fixed user load and think time The tests measure how responsive the My webmethods Server UI is during Task Engine operations, and how effectively Task Engine utilizes the hardware resources of the system. The results and conclusions in this report focus on how Task Engine operations affect normal user activities. For more information about what the report does not include, see What this Report Does Not Cover. Capacity planning needs: This report by itself cannot facilitate capacity planning; however, the resource utilization statistics can be used to weigh design-time choices and to provide input to the hardware selection process. Note: Benchmarking tests were carried out in a controlled environment. Except the necessary commands and processes to capture resource utilization, no other applications ran during the tests Software AG. All rights reserved.

5 2.0 HARDWARE AND SOFTWARE INFORMATION Three servers and one load generator were used for all tests. One server was running My webmethods Server (MWS) with Task Engine (TE), the second server hosted Integration Server (IS) and Universal Messaging Server (UM), and the third server was used for the DB2 database. HARDWARE RESOURCE DETAILS: Server OS Hardware Task Engine 9.9 Integration Server 9.9 and UM Server 9.9 DB2 Load Generator Red Hat Enterprise Linux Server release 6.4 (Santiago) Red Hat Enterprise Linux Server release 6.4 (Santiago) Red Hat Enterprise Linux Server release 6.4 (Santiago) Windows Server 2008 R2 Enterprise Network speed was 1 Gbps. Type VMware Virtual Platform VMware Virtual Platform DELL PowerEdge M820 VMware Virtual Platform Processor RAM Disk Intel(R) Xeon(R) CPU E L KB Intel(R) Xeon(R) CPU E L KB Intel(R) Xeon(R) CPU E L KB Intel(R) Xeon(R) CPU E L KB 32 GB RAID 0 32 GB RAID 0 54GB 8 GB RAID 0 SOFTWARE INFORMATION: 3.1 Software Information MWS + TE IS UM DB2 Silk Performer 2008 R Software AG. All rights reserved.

6 3.0 DEPLOYMENT ARCHITECTURE DIAGRAM The following diagram shows the test harness deployment architecture: My webmethods Server + Task Load Generators Integration Server + Universal DB2 Database Load OS Metrics Figure 1: Deployment Architecture Diagram Software AG. All rights reserved.

7 4.0 TEST HARNESS Benchmarking includes saturating the tested system by loading it to its maximum stable state and collecting performance data. A load generator was used to generate the load in the form of virtual users that perform desirable actions. The state of the load generator state was observed constantly to ensure that it was never the bottleneck of the system. The tools and utilities used to perform these tests were: Silk Performer 2008 R2 JVisualVM Silk Performer 2008 R2 was used to generate load and to measure the transaction response time of the My webmethods Server UI. The load generator system was used to inject the load into the test system. Since the tests were executed in a controlled environment, the system(s) under test did not run any application(s) apart from MWS/TE. Utilization of the critical system resources was captured for every test scenario. The results are presented in this report in the form of utilization charts. The utility used to capture resources utilization was JVisualVM. The collected server metrics include statistics about: CPU utilization Memory utilization Disk utilization Network utilization The memory allocated to all products was sufficient, hence memory utilization is not presented in this report. Disk and network utilization were also non-critical resources for the tests and are not presented in this report Software AG. All rights reserved.

8 5.0 TEST SETUP AND TEST CONDITIONS The test presented in this report is a standard benchmark test. It is included in the collection of benchmark tests, run by Software AG Performance Lab. The test simulates typical actions, performed by end-users while working with My webmethods. For this report, the test was extended with additional background calculations, performed by Task Engine in order to demonstrate how those calculations affect normal user work. For the test MWS was set up with 1000 My webmethods roles and one user per role. The test setup included a process model with 7 process steps and only 1 task step. A test harness service was used to trigger a process instance that queues tasks for approval or completion. UM was used for publishing documents. The logging level of all My webmethods Server components was set to ERROR. To avoid task collisions, the test harness service assigned a unique task to each role. Task portlet contained 15 text fields and LQOS was used for business data. The autoaccept option of the task application was set to true. All tests were done with the default task audit setting that comes with an out-of-the-box installation of TE. The benchmark scenario includes the following steps: 1. Open the login page of My webmethods Server. 2. Set 20 seconds think time. 3. Login to My webmethods Server. The Welcome page is displayed. 4. Set 3 seconds think time. 5. Open the Process Instances page. 6. Set 20 seconds think time. 7. Search for process instances. 8. Set 3 seconds think time. 9. Open the Process Instance Details page. 10. Set 20 seconds think time. 11. Close the Process Instance Details page. 12. Set think time to 3 seconds. 13. Close the Process Instances page. 14. Set think time to 3 seconds. 15. Logout. The tests were run with think time of 20 seconds. The response time for each action and the resource utilization during every test were captured. For all tests, the number of concurrent users was gradually increased until they reached the defined number. Tests with 100 concurrent users were executed. All tests were executed for 70 minutes, which included 10 minutes of warm up time and 60 minutes of measurement time. The resource utilization on both MWS/TE and the database server were captured. The results of the tests show that My webmethods Server UI has stable response times for user actions, even during Task Engine recalculations Software AG. All rights reserved.

9 6.0 BENCHMARK RESULTS 1. Baseline results benchmark scenario (see Test Setup and Test Conditions) with 100 users and 20 seconds think time: Response Times [sec] 1,6 1,4 1,2 0,8 1 0,6 0,4 0, Users Login 0,46 OpenPIPage 1,462 Search 0,325 OpenPIDetails 0,22 ClosePIDetails 0,203 ClosePITab 0,311 Logout 0,172 Figure 2: Baseline results from Silk Performer Software AG. All rights reserved.

10 2. Scheduled recalculation - benchmark scenario (see Test Setup and Test Conditions) with 100 users and 20 seconds think time, with 7 million tasks with different rule definitions scheduled for recalculation. Rule Data Source Number of Task Instances Total Recalculation Time IS Service seconds ( hours) Business Calendar seconds ( hours) Static Values seconds ( hours) Average MWS CPU Utilization: 8% Higher CPU Usage was observed during garbage collection. Figure 3: CPU & Heap memory diagram Software AG. All rights reserved.

11 3. Rule evaluation - benchmark scenario (see Test Setup and Test Conditions) with 100 users and 20 seconds think time, with rule evaluation started. Response Times During Rule Evaluation Response Times [sec] 1,8 2 1,6 1,4 1,2 0,8 1 0,6 0,4 0, Users Login 0,83 OpenPIPage 1,748 Search 0,615 OpenPIDetails 0,45 ClosePIDetails 0,437 ClosePITab 0,481 Logout 0,399 Figure 4: Response times from benchmark test with 100 concurrent users and 20 seconds think time performed during rule evaluation 4. Mass recalculation - benchmark scenario (see Test Setup and Test Conditions) with 100 users and 20 seconds think time, with mass recalculation running. Response Times during Recalculation Response Times [sec] 1,6 1,4 1,2 0,8 1 0,6 0,4 0, Users Login 0,517 OpenPIPage 1,512 Search 0,358 OpenPIDetails 0,237 ClosePIDetails 0,225 ClosePITab 0,325 Logout 0,2 Figure 5: Response times with 100 concurrent users and 20 seconds think time during recalculation Software AG. All rights reserved.

12 7.0 CONCLUSION The results presented in this benchmark report show that the My webmethods Server UI has stable response times for typical, independent actions performed by a large number of users, even while Task Engine performs rule evaluation and event recalculation in the background. 8.0 WHAT THIS REPORT DOES NOT COVER Testing with more than 100 users Testing with think time higher than 20 seconds TE cluster tests Software AG. All rights reserved.

13 9.0 APPENDIX 9.1 Terminology Think Time Warm up Time Measurement Time LQOS Task Collisions IS MWS TE UM GC Think Time is the time that a user waits between two actions. For example, when a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as think time. The duration, generally at start of the load test for which no measurements are performed. The duration for which measurements are performed during the load test. Low quality of service. If a task is assigned to a role and the role has more than one assigned user, then the same task gets assigned to all the users under that role. If all the users try to complete the same task, then a collision occurs. webmethods Integration Server My webmethods Server Task Engine Universal Messaging Server Garbage Collection 9.2 Product Tuning My webmethods Server: The MaxDBConnections in the mws.db.xml file were set to 300. The following Java options were added to the custom_wrapper.conf file for all the tests: o wrapper.java.initmemory=4000m o 9.3 OS/HW Tuning None. 9.4 Build and Fixes IS Version Build 74 MWS/TE Version Build 195 UM Version Build wrapper.java.maxmemory=4000m Software AG. All rights reserved.

14 ABOUT SOFTWARE AG Software AG offers the world s first Digital Business Platform. Recognized as a leader by the industry s top analyst firms, Software AG helps you combine existing systems on premises and in the cloud into a single platform to optimize your business and delight your customers. With Software AG, you can rapidly build and deploy Digital Business Applications to exploit real-time market opportunities. Get maximum value from big data, make better decisions with streaming analytics, achieve more with the Internet of Things, and respond faster to shifting regulations and threats with intelligent governance, risk and compliance. The world s top brands trust Software AG to help them rapidly innovate, differentiate and win in the digital world. Learn more at Software AG. All rights reserved. Software AG and all Software AG products are either trademarks or registered trademarks of Software AG. Other product and company names mentioned herein may be the trademarks of their respective owners.