OPTIMIZED TEST GENERATION IN SEARCH BASED STRUCTURAL TEST GENERATION BASED ON HIGHER SERENDIPITOUS COLLATERAL COVERAGE

Transcription

1 Volume 115 No , ISSN: (printed version); ISSN: (on-line version) url: ijpam.eu OPTIMIZED TEST GENERATION IN SEARCH BASED STRUCTURAL TEST GENERATION BASED ON HIGHER SERENDIPITOUS COLLATERAL COVERAGE T. Vijay Saradhi,T.Guru Vardhan, K. Devi Prasanna, B. Nithusha Computer Science Engineering, K L UniversityGuntur, India Abstract: The optimization process can be done the space of potential inputs. The search space of potential inputs can be very large, even for very small systems under test. A static dependence analysis derived from program slicing that can be used to support search space reduction. Input domain reduction is one of the search base test generation. These results are provide evidence to support the claim that input domain reduction has a significant effect on the performance of local, global and hybrid search, while a purely random search is unaffected. But these processes only consider the effects of input domain reduction on the generation of test data and not its evaluation. we introduce three algorithms which do this without compromising coverage achieved. We present the results of an empirical study of the effectiveness of the three algorithms on five benchmark programs containing non trivial search spaces for branch coverage. The results indicate that it is, indeed, possible to make reductions in the number of test cases produced by search based testing, without loss of coverage. Index Terms: higher serendipitous collateral coverage, benchmark programs, Search-Based Software Testing, Automated test data generation, Input domain reduction. 1. Introduction Software engineering is the study of the application of engineering to design, development, and maintenance of software. In software engineering software testing is the very important aspect in present and currently working expensive activity. Figure 1. Software testing in software engineering process. This has led to an enduring interest in methods to automate both construction of good test input and the determination of output correctness. In this searching of test based generation Search Based Software Testing (SBST) has addressed many different programming paradigms and languages, including conventional 3GL code, Object Oriented systems, Aspect Oriented systems and Model Based systems. The SBST approach has proved to be highly generic, leading to its incorporation in many different testing scenarios including Stress Testing, Exception Testing, Mutation Testing, Functional and Non Functional Testing. Figure 2.The EVOSUITE Process: A Set Of RandomlyGenerated Test Suites Is Evolved To Maximize Coverage. However, the research challenge is to develop ways of achieving this goal without sacrificing the equally important goal of achieving coverage of the System under Test (SUT). In order to do this, we seek test inputs that cover a targeted branch in the SUT, while also maximizing the so-called collateral coverage. One might also hope that the code itself contains pre and post conditions that implement well understood contract driven development approaches. We describe the following contributions: We introduce a new formulation of the search based structural test data generation problem in which the goal is to maximize coverage, while simultaneously minimizing the number of test cases, with a view to taking into account the human oracle cost effort involved in checking the behaviour of the SUT for a given test suite. In these situations, the oracle cost problem is ameliorated by the presence of an automatable oracle to which a testing tool can refer to check outputs, free from the need for costly human intervention. 549

2 2. Related Work Control Dependence: A node i dominates a node j if and only if every path from the entry node to the node j passes through node i. Conversely, a node j postdominates a node i if and only if every path from the node i to the exit node traverses the node j. A node k post-dominates a branch e = (i; j) if and only if every path from the node i to the exit node through e contains the node k. A node j is control dependent on a node i if and only if the node i dominates the node j and the node j post-dominates one and only one of the branches of the node i. A Control Dependence Graph (CDG) is a directed graph that captures control dependence. Search Based Software Testing: Meta-heuristic search techniques are methods which adopt heuristic mechanisms as the principal search strategies. The techniques are generally applied to complex problems when there exists no satisfactory algorithm for the problem or an existing algorithm is not practical with respect to computation time. Meta-heuristic techniques have also been applied to testing problems in a field known as Search Based Software Testing a sub-area of Search Based Software Engineering (SBSE). Most approaches described in the literature aim to generate test suites that achieve as high as possible branch coverage. In principle, any other coverage criterion is amenable to automated test generation. For example, mutation testing is a worthwhile test goal, and has been used in a search-based test generation environment. Our goal is to target difficult faults for which automated oracles are not available (which is a common situation in practice). Because in these cases the outputs of the test cases have to be manually verified, then the generated test suites should be of manageable size. There are two contrasting objectives: the quality of the test suite (e.g., measured in its ability to trigger failures once manual oracles are provided) and its size. 3. Existing Approach I. Traditionally developed test generations technique is Search-based structural test data generation. The search space is formed from the input domain of the function under test. double gradient_calc_radial_factor(double dist, double offset, double x, double y) { double r, rat; (1) if (dist == 0.0) { (2) rat = 0.0; } else { (3) offset = offset / 100.0; (4) r = sqrt (x * x + y * y); (5) rat = r / dist; (6) if (rat < offset) { (7) rat = 0.0; (8) } else if (offset == 1.0) { (9) rat = (rat >= 1.0)? 1.0 : 0.0; //... Figure 3. Code snippet process for test search space test generation. Meta-heuristic optimization techniques require a numerical formulation of the test goal, from which a fitness function' is formed. The purpose of the fitness function is to guide the search into promising, unevaluated areas of a potentially vast input domain, in order to find required test data. For branch coverage, each branch is taken as the focus of a separate test data search. The fitnessfunction is a function fit(t; i)! R, that takes a structural target t and individual input i, and returns a real number that scores how close the input was to executing the required branch. Random Search does not involve a _tness function to guide the optimization process, and so is technically not a `meta-heuristic' search technique. It is possible to cover many structural targets using random search, because there are often several input vectors that can be selected that are good enough to execute most of the structures of a program. 4. Proposed Approach This section produces three different types of algorithm for optimizing general test generation process. 4.1 Memory-Based Test Data Reduction In standard approaches to SBST, each currently uncovered branch is targeted by a distinct search process. The goal of previous work has been largely to cover branches, at any cost. This is clearly sub-optimal from the point of view of reducing the number of test cases required to cover the program under test. In order to 550

3 reduce the number of test cases it makes sense to record all branches hit by a test case that covers some particular branch of interest. Algorithm: Outline of the memory-based approach Input P: target program; B: set of all branches in P Output C: set of test cases Memory-Based-Approach(P, B) (1) U ( B (2) C ( ; (3) while U 6= ; (4) select a target branch, t 2 U (5) search for x s.t. t 2 P(x) (6) if x is found (7) U ( U P(x) (8) C ( C [ fxg (9) else (10) U ( U ftg (11) return C Figure 5. Outline memory search test generation Algorithm: Outline of the CDG-based approach Input P: target program; B: set of all branches in P Output C: set of test cases CDG-BASED APPROACH(P, B) (1) U ( B (2) C ( ; (3) while U 6= ; (4) select t 2 U (5) search for x s.t. t 2 P(x)^ maximizes jp(x) \ Uj (6) if x is found (7) U ( U P(x) (8) C ( C [ fxg (9) else (10) U ( U ftg (11) return C. Figure 6. CDG-based approach This algorithm is similar to tracking of serendipitous coverage in the methods of Wegener et al. and McGraw et al. Here, any input generated in the course of searching for a target that hits a previously uncovered branch may be inserted into the test suite as a separate test case. 4.2 CDG-Based Test Data Reduction The resulting test suite, i.e. the collection of test cases that are generated using this fitness function, would naturally contain some redundant test cases, which in turn results in extra test oracle cost. If we want to reduce the size of the resulting test suite, each search process for test data should not only consider the achievement of a specific structural target but also the amount of extra structural coverage that the candidate test case can achieve. The CDG-based approach actively seeks to maximise the increase in coverage as well as achieving coverage of the target branch, t. The algorithm depends on the CDG representation of the target problem in order to accurately calculate the possible collateral coverage. 5. Performance Evaluation An empirical study was performed that compared the standard search based test data generation algorithm. Table 1: Details Of The Test Subjects Used In The Empirical Study Program Branches Cyclomatic Complexity Domain size check isbn Gdkanji

4 6. Results The average test suite sizes produced by the reduced oracle cost algorithms were significantly smaller than that of the standard approach. For every test subject, average test suite size was smaller than the subject s Cyclomatic complexity number. As can also be seen in the figure, this reduced test suite size did not have a compromising effect on branch coverage of the program. For certain subjects and algorithms, the reduced cost algorithms managed to exceed the average level of coverage obtained by the standard individual branch approach. Figure 7. Average test suite size and branch coverage using the different algorithms If the search is able to keep track of collateral coverage, as with the memory-based approach, the number of test cases in the test suite is always reduced to a size that is less than the program s cyclomatic complexity. However, our results indicate that this situation can be further improved by effectively targeting more deeply nested branches using the CDG-based approach, which results in smaller test suite sizes using fewer distinct searches to do so. The clear winner with respect to test suite size is the greedy set cover algorithm. There was only one program (triangle) for which the greedy set cover method had a larger test suite size than the CDG-based and memorybased approaches, and this was because an additional hard-to-cover branch had been covered that the other algorithms had not. 7. Conclusion II. III. A static dependence analysis derived from program slicing that can be used to support search space reduction. Input domain reduction is one of the search base test generation. These results are provide evidence to support the claim that input domain reduction has a significant effect on the performance of local, global and hybrid search, while a purely random search is unaffected. But these processes only consider the effects of input domain reduction on the generation of test data and not its evaluation. we introduce three algorithms which do this without compromising coverage achieved. We present the results of an empirical study of the effectiveness of the three algorithms on five benchmark programs containing non trivial search spaces for branch coverage. The results indicate that it is, indeed, possible to make reductions in the number of test cases produced by search based testing, without loss of coverage. REFERENCES [1] Phil McMinn, Mark Harman, Kiran Lakhotia, Input Domain Reduction through Irrelevant Variable Removal and its E_ect on Local, Global and Hybrid Search-Based Structural Test Data Generation, IEEE Transactions on Software Engineering, [2] Mark Harman1, Sung Gon Kim1, Kiran Lakhotia,, Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem, In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2004), pages 219{230. ACM, [3] S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar- Walawege. A systematic review of the application and empirical investigation of search-based test-case generation. IEEE Transactions on Software Engineering, To appear. [4] L. O. Andersen. Program Analysis and Specialization for the C Programming Language. PhD thesis, DIKU, University of Copenhagen, May (DIKU report 94/19). [5] A. Arcuri. Theoretical analysis of local search in software testing. In Proceedings of Symposium on Stochastic Algorithms, Foundations and Applications (SAGA 2009), pages 156{168. Lecture Notes in Computer Science, Volume 5792, Springer Verlag, [6] A. Arcuri. It does matter how you normalise the branch distance in search based software testing. In Proceedings of the International Conference on Software Testing, Veri_cation and Validation (ICST 2010), pages 205{214. IEEE,

5 [7] A. Arcuri, M. Z. Iqbal, and L. Briand. Formal analysis of the e_ectiveness and predictability of random testing. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2004), pages 219{230. ACM, [8] J. E. Baker. Reducing bias and ine_ciency in the selection algorithm. In Proceedings of the 2 nd International Conference on Genetic Algorithms and their Application, Hillsdale, New Jersey, USA, Lawrence Erlbaum Associates. [9] Y. Jia and M. Harman, Constructing subtle faults using higher order mutation testing, in SCAM, 2008, pp [10] Y. Zhan and J. A. Clark, The state problem for test generation in simulink, in GECCO, 2006, pp [11] J. Wegener and O. B uhler, Evaluation of different fitness functions for the evolutionary testing of an autonomous parking system, in GECCO, 2004, pp [12] W. Afzal, R. Torkar, and R. Feldt, A systematic review of search-based testing for non-functional system properties, Information and Software Technology, vol. 51, no. 6, pp , Jun [13] M. Harman, A. Mansouri, and Y. Zhang, Search based software engineering: A comprehensive analysis and review of trends techniques and applications, Department of Computer Science, King s College London, Tech. Rep. TR-09-03, April [14] P. Godefroid, N. Klarlund, and K. Sen, DART: directed automated random testing, ACM SIGPLAN Notices, vol. 40, no. 6, pp , Jun [15] Azhagiri. M, Rajesh. A, Interruption Detection System using Unearthing and Probability Clomp Algorithm,International Innovative Research journal Of Engineering And Technology,vol 02,no 04,pp ,

6 554