Heuristic Approaches for Equivalent Mutant Problem Inderjot 1 Dr. Rakesh Kumar 2

Transcription

1 IJSRD - International Journal for Scientific Research & Development Vol. 3, Issue 02, 2015 ISSN (online): Heuristic Approaches for Problem Inderjot 1 Dr. Rakesh Kumar 2 1 Research Scholar 2 Professor 1,2 Department of Computer Science & Application Engineering 1,2 Kurukshetra University, Kurukshetra Abstract Mutation testing works on seeding faults into the program using s. These faulty programs are known as mutants. Test suites are applied on these mutants to recognize if they are killed or not. Test suite that kill mutants are said to be adequate. Sometimes, it is difficult to kill all the mutants because of equivalent mutants. Equivalent mutants are the mutants syntactically different and semantically similar to the original program. In this paper some heuristics have been proposed to avoid equivalent mutant problems. Key words: s, Mutants, Mutation Operators, Mutation Testing, Test Suite I. INTRODUCTION Mutation testing is a fault based testing approach. Mutation testing is a method of white box testing. Mutation testing was invented by Richard Lipton in 1971[1]. Mutation testing refers to seeding slight change into source code [2]. Each faulted version of source code is known as Mutants. Mutants are of two types [3]- 1) First order mutants- mutants that are generated by seeding a single fault in the original program. 2) Higher order mutants- mutants that are generated by seeding more than one fault in the original program. Mutation testing uses two principles- 1) Competent programmer hypothesis- It states that mutants are created by seeding faults in the program. The faults should be like as committed by competent programmers [4]. 2) Coupling effect hypothesis- It states that simple faults (first order mutants) are coupled to form complex faults (higher order mutants). Test suite which can able to detect the simple fault can also detect the complex fault [4]. Mutation testing provides a test adequacy criterion. To evaluate the quality of test-suite, test-suite is executed over the mutants. Output of original program is compared against mutant. If compared outputs are different then mutant is said to be killed by the test suite. If a mutant is not killed by any test suite that type of mutants are said to be equivalent mutants. A test suite must satisfy three conditions (RIP model) [1] to kill mutants- 1) Reachability- mutated statement must be executed by the test suite. 2) State Infection- mutated statement must have different state than original. 3) State Propagation- mutant must have different final state from the original program. Mutation adequacy score is used to measure the effectiveness of test suite in terms of its ability to detect faults. Mutation adequacy score is calculated by [5]- Mutation adequacy score= D/M-E Where, D- no. of killed mutants, M- Total no. of mutants, E- no. of Equivalent mutants Mutation adequacy score results between real values of 0.0 and 1.0. High adequacy score represents adequate test suite. Calculating exact score is difficult due to the presence of equivalent mutants. II. LITERATURE REVIEW Problem is the major obstacle of testing. The problem of determining whether a mutant is equivalent to the original program is theoretically un-decidable. Equivalent mutants are the mutants syntactically different and semantically similar to the original program. A program M is derived from program P by slight change. If P and M both produce same result on every test-suite then M is said to be equivalent mutant of P. For exampleint max(int[]a); int b,i; b=0; n=a.length; for(i=0;i<n;i++) If(a[i]>a[b]) b=i; return a[b]; int max(int[]a); int b,i; b=0; n=a.length; for(i=0;i<n;i++) If(a[i]>=a[b]) b=i; return a[b]; Mutant To overcome equivalent mutant problem there are 17 relevant techniques divided into three categories [6]- A. Detecting Techniques: 1) Compiler optimization techniques [7]. 2) Mathematical constraints to automatically detect equivalent mutants [8],[9]. 3) Program slicing to assist in the detection of equivalent mutants [10]. 4) Semantic differences in terms of running profile to detect non-equivalent mutants [11]. 5) Margrave s change impact analysis [12]. 6) Lesar model-checker for eliminating equivalent mutants [13]. B. Avoiding Techniques: 1) Selective [14]. 2) Avoiding equivalent mutant generation using program dependence analysis [15]. 3) Co-evolutionary search techniques [16]. 4) Equivalency conditions to eliminate equivalent mutants for object-oriented s [17]. 5) Fault hierarchy to improve the efficiency of DNF logic testing [18]. 6) Distinguishing the equivalent mutants by semantic exception hierarchy [19]. 7) Higher testing [20]. All rights reserved by 345

2 Heuristic Approaches for Problem C. Suggesting Techniques: 1) Using Bayesian-learning based guidelines to help to determine equivalent mutants [21]. 2) Examining the impact of equivalent mutants on coverage [22]. 3) Using the impact of dynamic invariants [23]. 4) Examining changes in coverage to distinguish equivalent mutants [24]. Recently researchers focused on avoiding and suggesting techniques. Detecting techniques are hard to implement. Detecting techniques never results erroneous like avoiding and suggesting techniques. III. HEURISTICS FOR AVOIDING EQUIVALENT MUTANTS As discussed, while mutants are generated, there are chances that equivalent mutants may be created. To detect the equivalent mutant is an un-decidable problem. The better option can be, don t create equivalent mutant. In the following text a number of heuristics have been proposed to avoid the creation of equivalent mutants. TableI. [25] shows list of mutant s in Operator-class. Operator Description Operator Description OAAA OABA OAEA OARN OASN OBAN OBBA OBLN OBRN OCNG by by by by shift by by logical by context OAAN OABN OALN OASA OBAA OBBN OBEA OBNG OBSA OEAA by bitwise by logical by shift by by by shift Plain by OEBA Plain OESA Plain OLAN OLLN OLRN ORAN ORLN ORSN OSAN OSBN OSLN OSSN by bitwise by by by by logical by shift by by bitwise by logical OLBN OLNG OLSN ORBN ORRN OSAA OSBA OSEA OSRN OSSA by shift by bitwise by shift by bitwise by by bitwise by by Table 1: List of Operator-Class Operators Using some of the above s, equivalent mutants are generated. Equivalent mutants are produced at certain conditions. Warnings are produced to avoid the generation of equivalent mutants. Some of them are as follows- A. OAAA- Assignment Mutation (A+=B A- =B). %d, if (b==0) if(b==0) a-=b; Be careful if the value of operands is zero. During generation of mutants using OAAA value of operands must greater than zero. All rights reserved by 346

3 Heuristic Approaches for Problem B. OAAN- Operator Mutation (A*B A+B). %d, if(a==2&&b==2) c=a*b; if(a==2&&b==2) c=a+b; Be careful if the values of operands are 0 and 2. During generation of mutants using OAAN, value of operands must greater than 0 and not equal to 2. C. OAEA- Assignment by Plain Assignment (A+=B A=B). %d, a=b; Be careful if the value of operands is 0. During generation of mutants using OAEA, value of operands must greater than zero. D. OBAN- Operator by Operator (A&B A*B). // if input of a and b is 0 or 1 if (a==b) c=a&b; // if input of a and b in 0 or 1 if (a==b) c=a*b; %d, Be careful if the value of operands is same. During generation of mutants using OBAN, value of operands must be different. E. OBBN- Operator Mutation (A&B A B). For Example- %d, if(a==b) c=a&b; %d, if(a==b) c=a b; Be careful on replacing bitwise OR by bitwise AND, operands must have different values. F. OBLN- Operator by Operator (A B A B). char sub; int marks; %c, &marks,&sub,); if((marks>80) (sub= P & &marks>70) (sub= C & & marks>80 )) printf( student will choose medical/nonmedical ); printf( student will choose commerce ); char sub; int marks; %c, &marks,&sub,); if((marks>80) (sub= P & &marks>70) (sub= C && marks>80 )) printf( student will choose medical/non-medical ); printf( student will choose commerce ); Be careful if value of operands uses values as true or false using bitwise OR. If bitwise OR replaced with logical OR using true or false values then they will produce same output. G. OBNG- Negation (~X Op Y ~ (X Op Y)). For Example- c=(~a^b); c=~(a^b); Be careful on using bitwise complement with bitwise XOR. H. OBRN- Operator By Operator (A^B A!=B). int b,c;,%d,&b,&c) if(b^c); printf( b& c not printf( b&c are int b,c;,%d,&b,&c) if(b!=c); printf( b& c not printf( b&c are Be careful if bitwise XOR replaced with!= then equivalent mutant generated. I. OEAA- Plain by (a=b a+=b). For example- &a,&b) if (a==0) a=b; &a,&b) if (a==0) EquivalentMutant Be careful on using OEAA if value of the operand is zero. All rights reserved by 347

4 Heuristic Approaches for Problem J. OLBN- Operator by Operator (A&&B A&B). %d %d,a,b,c); if((a>b)&&(a>c)) printf( a is greater ); if((a<b)&&(c<b)) printf( b is greater ); if((a<c)&&(b<c)) printf( c is greater ); printf( two of them are %d %d,a,b,c); if((a>b)&(a>c)) printf( a is greater ); if((a<b)&&(c<b)) printf( b is greater ); if((a<c)&&(b<c)) printf( c is greater ); printf( two of them are Equivalent mutant Be careful during the replacement of bitwise AND by logical AND if conditions results true or false. K. OLLN- Operator Mutation (A&&B A B). int x,y,z; %d %d, x,y,z); if((x>y)&&(x>z) (!(x==0)) ) printf( %d, x); printf( %d,y); int x,y,z; %d %d, x,y,z); if((x>y) (x>z) (!(x==0)) ) printf( %d, x); printf( %d,y); EquivalentMutant Be careful when single OR in a conditional statement makes the whole result of conditional statement true if any condition in it is true, regardless of AND L. OLRN- Operator by Operator (A&& A<B). &b,&c); a=b-c; if(!a) printf( b &c are &b,&c); a=b-c; printf( b &c are printf( b&c not printf( b&c not Be careful when a! logical is used with which makes condition equal to zero. M. ORRN- Operator Mutation (A<B A<=B). %d, if(a>b) c=a; c=b; if(a>=b) c=a; c=b; Be careful on using ORRN mutant for finding largest of two > should not replace by >=. N. ORBN- Operator by Operator (!= ). int a,b,c,d; d=b-a; if(c!=d) d=b-a; if(c d) printf( a and b are not printf( a and b are not printf( a and b are printf( a and b are Be careful on using ORBN, if!= and bitwise OR used on result of subtraction of two no. produce same result on same input. O. ORLN- Operator by Operator (==!). if(c==0) printf( a and b are if(!c) printf( a and b are printf( a and b not printf( a and b not EquivalentMutant Be careful on replacing with logical, (==) condition of results zero should not replace by!. IV. CONCLUSION In this paper, some of the heuristics are described using manual analysis to various s in C language. Mutation testing is a technique to assess the adequacy of test suite. There are various problems faced by testing- large no. of mutants and equivalent mutants. The problem of equivalent mutants for higher order mutants is not serious due to the coupling effect. Therefore, first order mutants are created using various replacements. On certain conditions these mutants act as equivalent mutants. Avoiding these conditions in creation of mutants, equivalent mutants can be avoided. Detection of equivalent mutants is still np-hard problem. As evident, testing is most effort consuming phase in software development. If by using some heuristics, one can avoid some equivalent mutants, it will result into a lot of effort and cost reduction. REFERENCES [1] Offutt Jefferson A., Roland and Untch H. (2000), Mutation: Uniting the Orthogonal. [2] Demillo A. Richard, Lipton J. Richard and Sayward G. Fred (1978), Hints on test data selection: Help for the practicing programmer, IEEE Computer, 11(4),p All rights reserved by 348

5 Heuristic Approaches for Problem [3] Mathur P. Aditya (2008), Foundation of Software Testing, Pearson Education, 2/e, ISBN: [4] Mathur P. and Wong E.W.(1995), Reducing the cost of Mutation Testing: An Emperical Study, The Journal of Systems and Software,31(3). [5] Hamlet R. (1997), Testing Programs with the Aid of a Compiler, IEEE Trans. Software Eng.,SE-3(4) p [6] Madeyski L., Orzeszyna W., Torkar R., Józala M. (2014), Overcoming the Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation,IEEE Transactions on Software Engineering,40(1). [7] Offutt Jefferson A. and Craft W.M.(1994), Using Compiler Optimization Techniques to Detect s,software Testing, Verification and Reliability, 4(3), p [8] Offutt Jefferson A. and Pan J.(1996), Detecting s and the Feasible Path Problem, Proc. 11th Ann. Conf. Systems Integrity, Computer Assurance, Process Security, Software Safety (COMPASS 96), p [9] Offutt Jefferson A. and Pan J.(1997) Automatically Detecting s and Infeasible Paths,Software Testing, Verification and Reliability,7(3) p [10] Hierons R., Harman M., and Danicic S.(1999), Using Program Slicing to Assist in the Detection of s, Software Testing, Verification and Reliability, 9(4), p [11] Ellims M., Ince D., and Petre M. (2007), The Csaw C Mutation Tool: Initial Results, Proc. Testing: Academic and Industrial Conf. Practice and Research Techniques MUTATION, p [12] Martin E. and Xie T.(2007), A Fault Model and Mutation Testing of Access Control Policies, Proc. 16th Int l Conf. World Wide Web (WWW 07), p [13] Bousquet du L. and Delaunay M. (2008), Towards Mutation Analysis for Lustre Programs, Electronic Notes in Theoretical Computer Science, 203(4), p [14] Mresa S.E. and Bottaci L.(1999), Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study, Software Testing, Verification and Reliability, 9(4), p [15] Harman M., Hierons R., and Danicic S.(2001), The Relationship between Program Dependence and Mutation Analysis,Mutation Testing for the New Century, p [16] Adamopoulos K., Harman M., and Hierons M.R.(2004), How to Overcome the Equivalent Mutant Problem and Achieve Tailored Selective Mutation Using Co-Evolution, Proc. Genetic and Evolutionary Computation Conf. (GECCO 04), p [17] Offutt J., Ma S Y., and Kwon R Y. (2006), The Class-Level Mutants of MuJava, Proc. Int l Workshop Automation of Software Test (AST 06), p [18] Kaminski G. and Ammann P.(2009), Using a Fault Hierarchy to Improve the Efficiency of DNF Logic Mutation Testing, Proc. Int l Conf. Software Testing Verification andvalidation (ICST 09), p [19] Ji C., Chen Z., Xu B., and Wang Z. (2009), A New Mutation Analysis Method for Testing Java Exception Handling, Proc. IEEE 33rd Ann. Int l Computer Software and Applications Conf. (COMPSAC 09), 2, p [20] Jia Y.and Harman M.(2009), Higher Order Mutation Testing, Information and Software Technology, 51, p [21] Vincenzi M.R.A.,Nakagawa Y.E.,Maldonado J.C.,Delamaro M.E., and Romero A.F.R.(2002), Bayesian-Learning Based Guidelines to Determine s, Int l J. Software Eng. and Knowledge Eng., 12(6), p [22] Grun J.M.B., Schuler D., and Zeller A.(2009), The Impact of s, Proc. IEEE Int l Conf. Software Testing, Verification, and Validation Workshops, p [23] Schuler D., Dallmeier V., and Zeller A. (2009), Efficient Mutation Testing by Checking Invariant Violations, Proc. 18th Int l Symp. Software Testing and Analysis (ISSTA 09), p [24] Schuler D. and Zeller A.(2010), Covering and Uncovering s, Software Testing, Verification and Reliability, 23, p [25] Offutt Jefferson A., Lee Ammei, Rothermel Gregg,Untch H. Roland, and Zapf Christian(1996) An experimental determination of sufficient mutant s. Acm Trans. Softw. Eng. Methodol. 5(2),p All rights reserved by 349