Batch Jobs Performance Testing October 20, 2012 Author Rajesh Kurapati
Introduction Batch Job A batch job is a scheduled program that runs without user intervention. Corporations use batch jobs to automate tasks that they need to perform on a regular basis. Batch jobs usually run during off peak hours when systems are not being used for online processing. (For example, systems can run to update files, create printed reports, or purge files.) Batch jobs that need to be processed on a regular basis are incorporated into batch schedules. Batch jobs usually run overnight, during the so called batch window". During this time, online activity is restricted or even completely forbidden. So a good first reason for batch jobs is to be sure that the batch window will end before the agreed online start time. This is becoming more and more important because batch activity is still growing at many sites while the batch window is normally shrinking due to 24X7 online demands. Batch jobs usually run overnight, during the so called batch window". During this time, online activity is restricted or even completely forbidden. The intent of Batch Test is to find errors in job setup or output distribution and correct them before go-live. There is no approach or method to guarantee a schedule completely free of defects 2
Importance of batch job Performance testing Irrespective of the volume of the data, performance of the batch should be good enough to complete the processing within the allowed time frame. Execution of batch job should never affect the performance of other processes. Batch Performance Testing Approach Determine those jobs and jobsets currently used in Production which will need to be setup for the new server/environment and subsequently tested during the appropriate phases Determine any new jobs to be added to the existing schedule as a result of new development/functionality Determine which jobs and jobsets residing on the Production server are obsolete and should be deleted before the testing phase Submit documentation for jobs/jobsets to be copied to the testing environment
Possible ways to predict There can be multiple ways to predict batch performance. Please note that a single option or hybrid of them can be used to predict batch performance. Following are some of the options. 1. Do a proof of concept (PoC) of batches and extrapolate based on to be data volume as per workload model. 2. Get benchmark numbers of the batches from core application vendor (if applicable) and extrapolate. 3. Get performance metrics from other similar projects and extrapolate as per workload model. 4. Profile the batches to get details of SQLs executed, apply base response time and overhead and estimate as per workload model. Environment Setup An environment similar to production environment has to be set up to performance test the batch job. This will include a similar system configuration set up, all the expected concurrent processes running and a similar data setup as in production. In most of the cases since replicating production like environment is expensive a relatively less configured system setup is made for testing and proportionate changes are made in the expected throughput during the test 4
Figure: Batch Performance Testing Process 5
Performance Monitoring All the system resources have to be efficiently and optimally utilized to obtain best performance from batch process Throughput No. of records processed per minute (second) statistics has to be collected. A script can be executed or a DB query lookup can be made to collect to collect throughput metrics of the batch process over the given time period. CPU & Memory Utilization These metrics should be always within the limit of what the batch job is expected to consume. Unlike other applications, high CPU of 90% is acceptable for batch process provided no other concurrent tasks will run and the application is finely tuned. This implies optimal utilization of resources by batch thus resulting in maximum throughput. In case, CPU utilization is higher than expected, it s always advisable to collect server level statistics which eases digging into the issue. DB Query Response Time This is one of the most important monitoring counters to be captured while doing batch job performance testing. Most of the Batch jobs do DB updates. If the Query response time is low, any type of code or configuration changes won t be able to bring any commendable improvement in the batch performance. Application level statistics Need to keep track of no. of concurrent threads or sessions during the batch run. These values have to be increased or decreased as situation demands. 6
Monitoring Tools 1. For applications with Windows OS perfmon is used to collect the System statistics. This includes, total CPU utilization, Memory utilization, process level breakdown of processor utilization, context switches, queue length etc. 2. For UNIX and AIX there will be scripts executing commands in regular intervals to collect all the metrics during the test run. there are commands which can collect CPU, memory, network statistics, DB connections etc. Issues There are few performance issues which are frequently seen to occur during Performance testing batch process 1. Throughput 2. CPU Utilization 3. SQL Queries time out 4. Memory Issues 7
Tuning CPU Utilization 8
If the CPU utilization goes down infrequently during the run, it will be due to slowly responding backend. Whenever backend responds slowly, it makes the processor wait for the Response brings down the CPU utilization. Data or queries taking long time have to be analysed and tuned to respond like other queries. Memory Issues Memory issues in batch jobs are similar to other applications. Whenever the application hits out of memory, either the application will be in need for additional memory or there will be a memory leak in the application. If it s the first case then appropriate memory as required by the process has to be allocated. If it s a memory leak, the leaking objects in the batch process has to be identified and fixed using tools like Heap dump analyser, JProbe Memory debugger etc. 2011 Capgemini. All rights reserved. 9
www.capgemini.com and www.sogeti.com