An Approach to Filter the Test Data for Killing Multiple Mutants in Different Locations

Transcription

1 Interntionl Journl of Computer Theory nd Engineering, Vol. 5, No. 2, April 2013 An Approh to Filter the Test Dt for Killing Multiple Mutnts in Different Lotions Ngendr Prtp Singh, Rishi Mishr, Silesh Tiwri, nd A. K. Misr presene of n orle for lssifying the output of test exeution s orret or not. The ompetent progrmmer hypothesis ws introdued y DeMillo et l. [4]. It sys tht progrmmers rete orret version of progrm. But it my e possile tht there my e fults in the progrm delivered y ompetent progrmmer, we ssume tht these fults re merely smll simple fults whih n e orreted y smll synttil hnges. Therefore, in Muttion Testing, fults onstruted from severl simple synttil hnges re pplied, whih represent the fults tht re mde y ompetent progrmmers. An exmple of the CPH n e found in Aree et l., s work [6] nd theoretil disussion n e found in Budd et l., s work [7]. The Coupling Effet ws lso introdued y DeMillo et l., [7]. While the CPH onern with progrmmers ehvior, the Coupling Effet is oserved empirilly. DeMillo et l., sttes tht Test dt tht distinguishes ll progrms differing from orret one y only simple errors is so sensitive tht it lso impliitly distinguishes more omplex errors. Offutt [8], [9] extended the Muttion Coupling Effet Hypothesis with preise definition of simple nd omplex fults. Aording to this definition, simple fult is represented y mutnt tht is reted y mking single synttil hnge, while omplex fult is represented s mutnt tht is reted y mking more thn one hnge. Offutt sys the oupling effet hypothesis is omplex fults re oupled to simple fults in suh wy tht test dt set tht detets ll simple fults in progrm will detet high perentge of the omplex fults or sys If test suite kills mutnt, it lso kills mutnts of mutnt [9]. Third ssumption is presene of n orle for lssifying the output of test exeution s orret or not the implementtion of orle is itself omplex prolem nd eyond the sope of our work. In muttion testing, for n originl progrm (OP) set of fulty progrms M lled s mutnts (mutnt progrm) re generted y repling some syntx to the originl progrm OP. In Tle-1 nd 2 shows OP nd its ll mutnt progrms M1, M2, M3, M4, M5 tht re generted y repling the reltionl opertor (>) y the opertors <,<=,>=,==, respetively. A rule tht genertes mutnt from the originl progrm is known s muttion opertor shown in Tle lnd 2, we use ROR opertor. There re mny other opertors tht n e used. Typil muttion opertors re designed to hnge the vriles nd expressions y replement, insertion or deletion opertors. These typil muttion opertors were implemented in the Mothr muttion system [10]. Astrt Muttion testing is fult-sed testing tehnique tht n e used for testing softwre t unit level, integrtion level nd speifition level. In ddition to ssessing the test dt dequy, muttion testing hs lso een used to support other testing tivities suh s test dt genertion, regression testing et. Severl works hs een done on utomti genertion of test dt tht n e effetively kill mutnts. Constrint-sed test dt genertion (CBT) is one of the utomti test dt genertion tehniques using muttion testing, however, existing pprohes of test se genertion generlly generte test dt y killing one mutnt t one time. Thus, more test ses re needed for hieving given muttion sore. In this pper, n pproh is proposed y filtering the test dt ording to neessity ondition nd rehility ondition y killing multiple mutnts, mutted t the different lotion t one time nd filtered test dt lso hieved sme or pproximte sme muttion sore. In proposed pproh, some test dt is filtered out of lrge test dt tht is suffiient to kill multiple mutnts, loted t different lotion. So this pproh redues the testing ost nd time. Index Terms Constrint-sed testing, Muttion testing, muttion opertor. I. INTRODUCTION Softwre testing is tehnique for softwre qulity ssurne. It is time onsuming proess nd ounts for out 50% of the ost of softwre development [1],[2]. An importnt prolem of softwre testing is how to generte the effetive test dt. If the prolem of utomti test dt genertion n e well solved, then the ost of softwre testing n e signifintly redued. Muttion testing is sed on muttion nlysis. Muttion testing is fult-sed testing tehnique whih is originlly introdued y Hmlet [3] nd DeMillo et l. [4] for ssessing the effetiveness of test suites in unit testing. Muttion testing is method of softwre testing, whih involves smll syntti hnge in progrms soure ode [5]. These syntti hnged progrms re lled mutnt progrms whih re reted y repling well-defined muttion opertors. Muttion testing is powerful testing tehnique for hieving orret or loses to orret progrm. Muttion testing is sed on three fundmentl ssumptions, One is known s ompetent progrmmer hypothesis (CPH) or the ompetent progrmmer ssumption nd seond is known s oupling effet [4]. Third ssumption is the Mnusript reeived Septemer 10, 2012; revised Novemer 21, Ngendr Prtp Singh is with the Computer Siene nd Engineering Deprtment t NVPEMI, Knpur, Indi (e-mil: ngendrsngh447@gmil.om). Rishi Mishr is with the I. T. Mnger in United Bnk of Indi, Luknow, Indi (e-mil: rishi.msr@gmil.om). DOI: /IJCTE.2013.V

2 Interntionl Journl of Computer Theory nd Engineering, Vol. 5, No. 2, April 2013 TABLE I OP M1 M2 int ig(, ) if(>) printf( ig= ); else printf( ig= ); int ig(, ) if(<) printf( ig= ); else printf( ig= ); TABLE II M3 M4 M5 int ig(, ) if(>=) printf( ig= ): else printf( ig= ); int ig(, ) if(=) printf( ig= ); else printf( ig= ); int ig(, ) if(<=) printf( ig= ;else printf( ig= ; int ig(, ) if( ) printf( ig= ;else printf( ig= ); After generting the mutnts, ll generted test dt is pplied on originl progrm (OP) nd s well s on reted mutnts tht distinguish mutnts from the originl progrm. Intuitively, test dt tht kill more mutnts prove its effetiveness on others. Speifilly, the rtio of killed mutnts to the nonequivlent mutnts is used s the muttion dequy sore to indite test suite qulity in muttion testing. Although, muttion testing is n effetive mens to find the effetiveness of test suites. Generting the test dt tht n hieve stisftory muttion dequy sore n e very lor-intensive [11]. For this prolem, severl pprohes hve een proposed in the literture. The Constrint-sed testing (CBT) tehnique ws developed y DeMillo nd Offutt [12]. They used symoli evlution, ontrol-flow nlysis, nd informtion of mutnts for generting test dt utomtilly. Test dt generted y CBT n kill more thn 90% of the mutnts for most progrms [13]. There ws prolem in CBT pproh to hndling the nested expressions, rrys nd loops. Another pproh is proposed y Offutt nd Jin for test dt genertion, known s Dynmi Domin Redution (DDR) [14].This pproh pplies some limittions in CBT pproh. So we n sy tht DDR pproh is etter thn CBT pproh ut oth pprohes genertes test dt y solving one mutnts neessity ondition nd reh ility onditions [15], [16]. Aording to the pper [15], [16] the numer of possile mutnts is proportionl to the produt of the numer of dt referenes nd the numer of dt ojets. Whih indites one test dt is generted ording to one mutnt, tht mens if there re N numer of mutnts then N numer of test dt re required to killing the mutnts whih eome ig urden. So nother novel pproh is proposed y Ming-Ho Liu etl. [17] tht generte test dt for killing multiple mutnts tht re pled in sme lotion. This pproh genertes one test dt ording to multiple mutnts tht re mutted t the sme lotion t one time. Thus this pproh n generte smller test suite tht n hieve the sme muttion testing sore. The experimentl results of this pproh show tht it is more ost-effetive. Our proposed pproh is the extension of [17]. In this pproh, generted test dt is filtered y killing multiple mutnts tht re pled in different lotion. This pproh genertes one test dt ording to multiple mutnts tht re mutted t the different lotion t one time. Thus, this pproh n generte smller test suite tht n hieve the sme muttion sore. Thus, the ost of testing is redued. II. BACKGROUND DETAILS Aording to the pper [12], [13], [15], [17] the test dt to kill mutnt must e stisfied three onditions known s rehility ondition, neessity ondition, suffiieny ondition. Lets OP is originl progrm, M is mutent of OP on prtiuler sttement S nd test dt for OP is T then three onditions is defined s follows:- A. Rehility Condition The test dt T must e reh to sttement S euse S is repled y mutnt. Mutnt is syntti hnge to n exeutle sttement, nd the other sttements in the mutted progrm re synttilly equl to the sttements in the originl progrm. If T nnot e rehed to S then it is gurnteed tht T will never kill the M. 1- int,, ; 2- if(>) Rehility Condition 3- if(>) Mutnts re:- (>=), (<), (<=), (==), & ( ) 4- printf( is lrgest ); 5- else 6- printf( is lrgest ); 7- else 8- if(>) 9- printf( is lrgest ); 10- else 11- printf( is lrgest ); Fig. 1. For exmple progrm show in figure-1, if we reple. Sttement (>) of line numer 3 y mutnt opertor ROR s shown in tle 1 nd 2 then the rehility ondition of this S is (>) whih is shown in line numer 2. Tht mens if T not stisfy the rehility ondition then mutnt nnot e exeuted. B. Neessity Condition A generted test se tht kills mutnts needs to hve this hrteristi: it must differentite the mutnts ehvior tht is, why the mutnt is represented y single hnge to t the originl progrm, the exeution stte of the mutnt progrm must differ from tht of the originl progrm fter some exeution of the mutted sttement. This hrteristi is known s the neessry ondition. Given progrm OP nd mutnt progrm M y hnging sttement S in OP, for test dt T to kill M, it is neessry tht the stte of M immeditely following some exeution of S e different from the stte of OP t the sme lotion [12]. Aording to pper [17] neessity ondition for killing the mutnts (sme lotion) of the progrm shown in the Fig. 1. is lulted s:- Neessity ondition for (>) (>) (>=) (>) (<) 254

3 Interntionl Journl of Computer Theory nd Engineering, Vol. 5, No. 2, April 2013 (>) (<=) (>) (==) (>) ( ) Then omined these five we get (>) (>=)&&(>) (<) &&(>) (<=) &&(>) (==) whih is equivlent to C. Suffiieny Condition &&(>) ( ) (<) && ( == ) (1) The neessity ondition never gurnteed tht to e suf-fiient to kill the mutnt. For test se to kill mutnt, it must rete different output,in whih se the finl stte of the mutnt progrm differs from the originl progrm. Although filtering the test ses tht meet this suffiieny ondition is ertinly desirle, it is imprtil in prtie [13]. There re some pprohes [14], [17] tht utilize the rehility onditions nd the neessity ondition to generte test dt tht does not gurntee to kill ll mutnts, ut n kill most of the mutnts [13]. III. OUR APPROACH The proposed pproh in pper [17], Ming-Ho Liu et l. sys one test dt re required to kill multiple mutnts tht re mutted in sme lotion t one time. Thus this pproh generte smll test suite tht hieve sme test dequy sore (muttion sore). But in our pproh generte smll test suite tht hieve sme test dequy sore nd eh test dt of test suite is le to killed multiple mutnts tht re mutted in different lotion t one time. In Fig. 1. the ll mutnts of sttement (>) is known s sme lotion mutnts euse they hve sme rehle ondition. The ll mutnts of the sttement (>) nd (>) is known s different lotion mutnts euse they hve inside the sme onditionl sttement whih is responsile to find out the rehle ondition of oth sttements. Although if two mutted sttements re inside different onditionl sttement tht is lso known s different lotion mutnts. There re some ssumptions in proposed pproh, first the mutnts tht re loted in different lotion hving sme rehility onditions nd the neessity onditions re similr in struture. Seond the omined neessity onditions of different lotions mutnt in one ondition tht is lso omined with shred rehility ondition whih is known s Finl Filtering Condition (FFC). Third ssumption is the onditionl sttements tht led test se to reh the inner lok of ode ontining mutnts re used to provide the rehility ondition. The min im of proposed pproh is to reple the mutnts on different lotion y using one muttion opertor known s Reltion Opertor Replement (ROR). ROR muttion opertor n produe more thn one mutnt on the prtiulr lotion (see Fig.1).Our pproh follows the following four steps:- 1) Find the rehility ondition of mutnts tht re loted in different lotions. 2) Find the neessity ondition of eh different lotion mutnts. 3) Comining neessity ondition of eh different lotion mutnts with shred rehility ondition (if exist) y using onjution (&&) opertor nd generte Finl Filtering Condition (FFC) for eh different lotions. 4) Generte redued test dt y using FFCs. Tle I nd Tle II shows originl progrm OP nd its ll mutnt progrms M1, M2, M3, M4, M5 tht is generted y repling the retionl opertor (>) y the opertors <,<=,>=,==, respetively. Progrm in Fig. 1ontining three reltionl opertors (>) (>) nd (>), eh one of these three reltionl opertors is repled y ny one of other reltionl opertors like <, <=, >=, ==,, so there re totl fifteen mutnts re generted. Proposed pproh genertes FFC ondition tht filter the lredy generted test dt. The generted test dt efore filtering is le to kill the multiple mutnts in sme lotion with some dequy sore ut the filtered test dt is lso le to kill multiple mutnts in sme lotion s well s different lotion with the sme dequy sore s ompred to tht efore filtering. The step-y-step implementtion of proposed pproh (y using the progrm shows in Fig. 1) is s follows:- A. Step-1 Out of three reltionl expressions in progrm (figure-1) the expression (>) is rehility ondition for other two reltionl expressions (>) nd (>) euse oth expression (>) nd (>) depends on the expression (>). Tht mens if ondition (>) is true then expression (>), otherwise expression (>) is exeuted. B. Step-2 The Neessity ondition for lotion-1(line numer-3) is shows in eqution (1) similrly the Neessity ondition for lotion-2 (line numer-8) is C. Step-3 ( < ) && ( == ) (2) In this step the neessity ondition of lotion-1 whih shows in eqution (1) is omined with their rehility ondition (>) y using onjution opertor. (>) &&[ ( < ) && ( == )] [(>) && ( < )] && [(>) && ( == )] [(<< )]&&[(( == ) > )] (3) Similrly the neessity ondition of seond lotion whih shows in eqution (2) is omined with their rehility ondition (<=) y using onjution opertor. (<=) && ( < ) && ( == ) [(<=) && ( < )] && [(<=) && ( == )] 255

4 [((,)<) ((== )>)] && [((<(==)) (====)] [((; ) < ) (( == ) > )] &&[(( < ( == )) ( == == )] (4) Aording to figure-1 line no. 2 (rehlity ondition of line no.3 nd line no.8) is lso repled y five mutnts using ROR muttion opertor. In line no. 2 (>) is not depends on ny rnh predite so there re no ny rehility ondition to kill ll five mutnts. Hene the neessity ondition for (>) is show in eqution 5. ( < ) && ( == ) (5) D. Step-4 Thus the omintion of FFCs (Eqn-3, 4, 5) is used to Generte filtered test dt tht is used to kill mutnts. IV. EXPERIMENTAL RESULT Let us tke n exmple of progrm shown in Fig. 1 tht n find the lrgest integer from the given three integers s inputs.the following test sere generted to test the Progrm shown in Fig. 1. T1:-All three integers re different. T2:-Any two integers re sme. T3:-All integers re sme. Assume tht the vlue of,, re 5,4,6 respetively then ll possile test dt in eh test se is shown in TABLE-III. TABLE III Test Cse Test Dt T1 t 11 =(5,4,6), t 12 = (5,6,4), t 13 = (6,5,4), t 14 = (6,4,5), t 15 = (4,5,6), t 16 = (4,6,5) T2 t 21 =(5,5,4), t 22 =(5,4,5), t 23 =(4,5,5), t 24 =(5,5,6), t 25 =(5,6,5), t 26 =(6,5,5), t 27 =(4,4,6), =(4,6,4), t 29 =(6,4,4) t 210 =(4,4,5), t 211 =(4,5,4), t 212 =(5,4,4) t 213 =(6,6,4), t 214 =(6,4,6), t 215 =(4,6,6) t 216 =(6,6,5), t 217 =(6,5,6), t 218 =(5,6,6) T3 =(5,5,5), t 32 =(6,6,6), t 33 =(4,4,4) Let us selet 15 test dt out of 27 test dt rndomly whih is t 11, t 12, t 13, t 14,t 15, t 16, t 24, t 25, t 26,, t 213, t 214, t 27,, t 29 nd using these test dt find the out put of the tul progrm nd their mutnt progrms whih is shown in TABLE IV. Agin y using our pproh, filter the previesly seleted 15 Test dt y using Eq n -3,4,5. Test dt t 11 nd t 214 stisfy the Eq n -3. Comintion of two test dt, one test dt from t 12, t 16, t 25,, t 213 nd seond test dt stisfied the Eq n -4. Our pproh selet ny one omintion of two test dt out of six possile omintion euse ll six possile omintion is le to kill the sme numer of mutnt, TABLE-V shows the results of two omintions (t12,t31) nd (, ) out of six possile omintions. Test dt tht stisfy the Eq n -5, is lso stisfy the Eq n -4. So our pproh selet only 4 test dt t 11,, t 214,,nd out of 15 test dt fter filtering. The output of the tul progrm nd their mutnt progrm is shown in TABLE VI. TABLE IV T.D. OP M1 M2 M3 M4 M5 M6 M 7 t 11 t 12 t 13 t 14 t 15 t 16 t 24 t 25 t 26 t 213 t 421 t 27 t 29 TABLE V t 12 TABLE VI t 11 t 214 Interntionl Journl of Computer Theory nd Engineering, Vol. 5, No. 2, April

5 Interntionl Journl of Computer Theory nd Engineering, Vol. 5, No. 2, April 2013 In trditionl muttion testing pproh one test dt re require to kill one mutnt. So in this exmple there re 15 test dt re required to kill ll 15 mutnts ut y in ourpproh only four test dt re le to kill ll 15 mutnts, so the testing ost nd time is redued. V. CONCLUSION AND FUTURE WORK This pper presents new pproh for filtering the generted test dt nd filtered test dt n e used for killing multiple mutnts t different lotions effetively. The filtering of test dt is sed on the omintion of neessity ondition with rehility ondition (if exist). The onjuntion opertor (&&) is used in omintion of these ondition whih genertes Finl Filtering Condition (FFC) for eh different lotion. Experimentl results show tht proposed pproh redues the test dt y filtering them. Both neessity nd rehility onditions re stisfied y omintion of Finl Filtering Condition (FFC). The generted test dt efore filtering is le to kill the multiple mutnts in sme lotion with some dequy sore ut the filtered test dt is lso ple to kill multiple mutnts in sme lotion s well s different lotion with the sme dequy sore s ompred to tht efore filtering. Proposed pproh redues the test dt effetively y filtering them so tht less test dt is used for killing the multiple mutnts. Therefore, less exeution time is required to kill the mutnts whih redue the ost of muttion testing tivity nd prove its effiy over other pprohes given in literture. In future, proposed pproh n e implemented s tool to filter the generted test dt or it n lso e integrted with the existing muttion testing tools suh s Mu, JUNIT et. REFERENCES [1] G. Myers, The Art of Softwre Testing, John Wiley nd Sons, New York NY, [2] I. Sommerville, Softwre Engineering, Addison-Wesley Pulishing Compny In., 4th edition, [3] R. G. Hmlet, Testing progrms with the id of ompiler, IEEE Trnstions on Softwre Engineering, vol. 3, no. 4, pp , July [4] R. A. DeMillo, R. J. Lipton, nd F. G. Sywrd, Hints on test dt seletion: Help for the prtiing progrmmer, IEEE Computer, vol. 11, no. 4, pp , April [5] A. Jefferson Offutt, A Prtil System for Muttion Testing: Help for the Common Progrmmer [6] A. T. Aree, T. A. Budd, R. A. DeMillo, R. J. Lipton, nd F. G. S. Wrd, Muttion nlysis, Georgi institute of tehnology, Atlnt, Georgi, Tehnique Report GIT-ICS, vol. 79, no. 8, [7] T. A. Budd, R. A. DeMillo, R. J. Lipton, nd F. G. Sywrd, Theoretil nd empiril studies on using progrm muttion to test the funtionl orretness of progrms, in Proeedings of the 7th ACM SIGPLAN-SIGACT Symposium on Priniples of Progrmming Lnguges(POPL80), Ls Vegs, Nevd, pp , Jnury [8] A. J. Offutt, The oupling effet: Ft or Fition, ACM SIGSOFT Softwre Engineering Notes, vol. 14, no. 8, pp , Deemer [9] A. J. Offutt, Investigtions of the softwre testing oupling effet, ACM Trnstions on Softwre Engineering nd Methodology, vol. 1, no. 1, pp. 520, Jnury [10] K. N. King nd A. J. Offutt, A fortrn lnguge system for muttionsed softwre testing, Softwre: Prtie nd Experiene, vol. 21, no. 7, pp , Otoer [11] A. J. Offutt nd R. H. Unth, Muttion 2000: Uninting the Orthogonl, Muttion 2000: Muttion Testing in the Twentieth nd the Twenty First Centuries, Sn Jose, CA, pp , Otoer [12] R. A. DeMillo nd A. J. Offutt, Constrint-sed utomti test dt genertion, IEEE Trnstions on Softwre Engineering, vol. 17, pp , Septemer [13] R. A. DeMillo nd A. J. Offutt, Experimentl results from n utomti test se genertor, ACM Trnstions on Softwre Engineering Methodology, vol. 2, pp , April [14] A. J. Offutt, Z. Jin, nd J. Pn, The dynmi domin redution pproh to test dt genertion, Softwre Prtie nd Experiene, vol. 29, pp , Jnury [15] A. J. Offutt nd J. Pn, Deteting equivlent mutnts nd the fesile pth prolem, in Proeedings of the 1996 Annul Conferene on Computer Assurne (COMPASS 96), (Githersurg MD), IEEE Computer Soiety Press, pp , June [16] W. E. Wong nd A. P. Mthur, Reduing the ost of muttion testing: n empiril study, Journl of Systems nd Softwre, vol. 31, no. 3, pp , Deemer 1995 [17] M. H. Liu, Y. F. Go, J. H. Shn, J. H. Liu, L. Zhng, nd J. S. Sun, An pproh to test dt genertion for killing multiple mutnts, IEEE Interntionl Conferene on Softwre Mintenne (ICSM 06), /06, 2006 Ngendr Prtp Singh is Assistnt Professor nd Hed of Computer Siene nd Engineering Deprtment t Nrin Vidy Peeth Engineering nd Mngement Institute, Knpur, Indi. He is widely known out work on Muttion Anlysis nd Testing Tehnique. He hs more thn 10 yers of Experiene (Tehing nd Reserh) nd wrded M.Teh. Degree from Moti Ll Nehru Ntionl Institute of Tehnology, Allhd, Indi. His M.Teh. thesis sed on Muttion Testing under the supervision of Prof. A. K. Misr from Moti Ll Nehru Ntionl Institute of Tehnology, Allhd, Indi. He is enrolled in Advne Ph.D. Progrm from Shri G. S. Institute of Tehnology nd Siene, Indore, Indi. 257