Automatic Calculation of Coverage Profiles for Coverage-based Testing

Transcription

1 Auomaic Calculaion of Coverage Profiles for Coverage-based Tesing Raimund Kirner 1 and Waler Haas 1 Vienna Universiy of Technology, Insiue of Compuer Engineering, Vienna, Ausria, raimund@vmars.uwien.ac.a Absrac. Code-coverage-based esing is a widely-used esing sraegy wih he aim of providing a meaningful decision crierion for he adequacy of a es suie. Code-coverage-based esing is also used for he developmen of safey-criical applicaions, as he modified condiion/decision coverage (MCDC) is proposed by he DO178b documen. One criical issue of code-coverage esing is ha hey are ypically applied o source code while he generaed machine code may resul in a differen code srucure due o code opimizaions performed by an compiler. In his work we describe he auomaic calculaion of coverage profiles describing which srucural code-coverage crieria are preserved by which code opimizaion. These coverage profiles allow o easily exend compilers wih he feaure of preserving any given code-coverage crieria by enabling only hose code opimizaions ha preserve i. 1 Inroducion Tesing is an esablished and acceped echnique o increase he confidence in he correcness of a compuer sysem. In conras o formal verificaion, esing is no aimed o cover he full behavior of he sysem. Bu in conras o formal verificaion, esing has he srong advanage ha i operaes on he real operaion, including all low-level sysem deails and physical behavior. Formal verificaion on he oher side allways resides a a cerain absracion level, allowing he full behavioral coverage a his absracion level. Thus, esing and formal verificaion are complemenary approaches, boh are necessary for he developmen of safey-criical sysems. Wihin his paper we focus on he esing par of verificaion, addressing he challenges owards porable es-daa generaion. Derivaion or generaion of es daa is preferably done a he same level where he program is developed, ypically a high-level programming language or any modeling environmen wih auomaic code generaion. Firs, his is he preferred way o do if he es daa are wrien manually. Second, his is also beneficial for auomaic es-daa calculaion, as i allows o reduce complexiy by aking benefi of he absrac program represenaion. Thirs, his is preferred for porabiliy issues, like cross-plaform esing. The research leading o hese resuls has received funding from he Ausrian Science Fund (Fonds zur Förderung der wissenschaflichen Forschung) wihin he research projec Susaining Enire Code-Coverage on Code Opimizaion (SECCO) under conrac P20944-N13.

2 We assume ha es-daa generaion is guided by srucural code coverage, for example, saemen coverage, condiion coverage, or decision coverage. Srucural codecoverage on is own is no a very robus coverage meric for sofware esing, bu i is a useful complemenary meric ha indicaes program locaions of weak coverage by es daa. Using source-code based derivaion of es daa, i is he challenge o ensure ha he es daa fulfill an analogous srucural code-coverage meric a he machine-code level as hey achieve a source-code level. We call i analogous code-coverage meric, because several srucural code-coverage merics make no sense a machine-code level, because, for example, he grouping of several condiions o a decision is a source-level concep ha is no available a machine-code level. If a compiler performs complex code opimizaions ha, for example, inroduce new pahs or change he reachabiliy of some saemens [1], his may disrup he srucural code coverage achieved a he original program. We propose an approach oward he preservaion of srucural code coverage when ransforming he program [2, 3]. For his we use a so-called coverage profiles, i.e., a pre-calculaed able ha specifies for each srucural code-coverage meric which code ransformaions of he compiler guaranee o preserve i. Such a coverage profile can be easily inegraed ino a compiler such ha only hose code ransformaions are enabled ha preserve he chosen srucural code coverage. The concepual inegraion of coverage profiles ino a compiler is shown in Figure 1. In his paper we focus on he absrac specificaion of code ransformaions and on he calculaion of he coverage profiles. Source Code Coverage Selecion Coverage Preserving Compiler Inermediae Code Coverage Preservaion Guard Code Opimizaion X Coverage Profile X Inermediae Code Objec Code Fig. 1. Applicaion of a Coverage Profile Besides he funcional sofware esing, he preservaion of srucural code coverage is also of high ineres for hybrid iming analysis, i.e., an approach o deermine he im-

3 ing behavior of a program based on he combinaion of execuion-ime measuremens and program analysis [4, 5]. 2 Srucural Code Coverage for Sofware Tesing Srucural code-coverage crieria are esing merics o quanify he conrol-flow coverage of he program for a given se of es daa. In his secion we describe a few exemplary srucural code-coverage merics o show he calculaion of compilaion profiles. Formal definiions of some addiional srucural code-coverage merics can be found in [3, 6]. 2.1 Basic Definiions In he following we give a lis of basic definiions ha are used o formally describe properies of srucural code coverage and condiions for preserving srucural code coverage: Program P denoes he program before (P 1 ) and afer (P 2 ) he ransformaions for which we wan o preserve srucural code coverage. Conrol-flow graph (CFG) is used o model he conrol flow of a program [7]. A CFG G N, E, s, consiss of a se of nodes N represening basic blocks (see below), a se of edges E : N N represening he conrol flow (also called conrol-flow edges), a unique enry node s, and a unique end node. Basic block of a program P is a code sequence of maximal lengh wih a single enry poin a he beginning and wih he only allowed occurrence of a conrol-flow saemen a is end. We denoe he se of basic blocks in a program P i as B(P i ). Decision is a Boolean expression composed of condiions ha are combined by Boolean operaors. If a condiion occurs more han once in he decision, each occurrence is a disinc condiion [8]. However, he inpu of a decision is he se of is condiions wihou duplicaes. A decision is composed of one or more basic blocks. We denoe he se of decisions of a program P i as D(P i ). There are programming languages, where decisions are hidden by an implici conrol flow. For example, in ISO C due o he shor-circui evaluaion he following saemen a (b && c); conains he decision (b && c). The shor-circui evaluaion of ISO C saes ha he second argumen of he operaors && and is no evaluaed if he resul of he operaor is already deermined by he firs argumen. The correc idenificaion of hidden conrol flow is imporan, for example, o analyze decision coverage. See [3] for furher deails wih respec o code coverage. Condiion is a Boolean expression. We consider only lowes-level condiions, i.e., condiions ha do no conain operaors wih Boolean argumens [8]. A condiion is composed of one or more basic blocks. We denoe he se of condiions of a decision d as C(d). The se of all condiions wihin a program P i is denoed as C(P i ).

4 Inpu daa ID defines he se of all possible valuaions 1 of he inpu variables of a program. Tes daa TD defines he se of valuaions of he inpu variables ha have been generaed wih srucural code coverage analysis done a source-code level. Since exhausive esing is inracable in pracice, TD is assumed as a rue subse of he program s inpu daa space ID: TD ID. If we would aim for exhausive esing (TD ID) here would be no challenge of srucural code-coverage preservaion. Noe ha a es case consiss, besides he es daa, also of he expeced oupu behavior of he program. Since we are primarily concerned wih he preservaion of srucural code coverage wih consider only he es daa. Reachabiliy valuaion IV R (x) defines he se of valuaions of he inpu variables ha rigger he execuion of expression x, where x can be a condiion, decision, or a basic block. Saisfiabiliy valuaion IV T (x), IV F (x) defines he ses of valuaions of he inpu variables ha rigger he execuion of he condiion/decision x wih a cerain resul of x: IV T (x) is he inpu-daa se, where x evaluaes o TRUE and IV F (x) is he se, where x evaluaes o FALSE. The following properies always hold for IV T (x), IV F (x): IV T (x) IV F (x) IV T (x) IV F (x) IV R (x) Consider he following example of C code o ge an inuiion abou he meaning of he saisfiabiliy valuaions: For his code fragmen we assume From his assumpion i follows ha void f (in a,b) { if (a3 && b2) reurn 1; reurn 0; } IV R (a3) { a, b a, b in} IV R (b2) { 3, b b in} (and no he larger se { a, b a, b in} due o he hidden conrol flow caused by he shor-circui evaluaion of ISO C [3]). I follows ha IV T (b2) { 3, 2 } 1 Valuaion of a variable means he assignmen of concree values o i. The valuaion of an expression means he assignmen of concree values o all variables wihin he expression.

5 Only hose inpu daa ha rigger he execuion of condiion b2 and evaluae i o TRUE are wihin IV T (b2). Wih 3, 2 he condiions a3 and b2 are boh execued and evaluaed o TRUE. Furher, i holds ha IV F (b2) { 3, b b in b 2} The definiions of IV R (x), IV T (x), and IV F (x) depend on wheher he programming language has hidden conrol flow, for example, he shor-circui evaluaion of ISO C [9]. 2.2 Saemen Coverage (SC) Saemen coverage (SC) requires ha every saemen of a program P is execued a leas once. Saemen coverage alone is quie weak for funcional esing [10] and should bes be considered as a minimal requiremen. Using above definiions, we can formally define SC as follows: b B(P ). (TD IV R (b)) (1) Noe ha he boundary recogniion of basic blocks B(P ) can be ricky due o hidden conrol-flow. A saemen in a high-level language like ISO C can consis of more han one basic block. For example, he ISO C saemen f(a3 && b2); consiss of muliple basic blocks due o he shor-circui evaluaion order of ISO C expressions. Remark 1. Source-line coverage is someimes used as an alernaive o SC in lack of adequae esing ools. However, wihou he use of sric coding guidelines, source-line coverage is no a serious esing merics, as i is possible o wrie whole programs of arbirary size wihin one source line. 2.3 Condiion Coverage (CC) Condiion coverage (CC) requires ha each condiion of he program has been esed a leas once wih each possible oucome. I is imporan o menion ha CC does no imply DC. A formal definiion of CC is given in Equaion 2. c C(P ). (IV T (c) TD) (IV F (c) TD) (2) Remark 2. Above definiion of CC requires in case of shor-circui operaors ha each condiion is really execued. This is due he semanics of IV T (),IV F (). However, ofen oher definiions are used ha do no explicily consider shor-circui operaors (as, for example in [11]), hus having in case of shor-circui operaors only a virual coverage since hey do no guaranee ha he shor-circui condiion is really execued for he evaluaion o TRUE as well as for he evaluaion o FALSE. 2.4 Decision Coverage (DC) Decision coverage (DC) requires ha each decision of a program P has been esed a leas once wih each possible oucome. Decision coverage is also known as branch

6 coverage or edge coverage. d D(P ). (IV T (d) TD) (IV F (d) TD) (3) 3 Preservaion of Srucural Code Coverage The challenge of srucural code-coverage preservaion is o ensure for a given srucural code coverage of a program P 1 ha his code coverage is preserved while he program P 1 is ransformed ino anoher program P 2. This scenario is shown in Figure 2. Of course if a program will be ransformed, also he ses of basic blocks B or he se of program decisions D may ge changed. As shown in Figure 2, he ineresing quesion is wheher a concree code ransformaion preserves he srucural code coverage of ineres. ransformaion Program P 1 Program P 2 (PS 1, B 1, D 1 ) (PS 2, B 2, D 2 )? coverage(p 1, TD) coverage(p 2, TD) Fig. 2. Coverage-Preserving Program Transformaion When ransforming a program, we are ineresed in he program properies ha mus be mainained by he code ransformaion such ha a srucural code coverage of he original program by he es-daa se TD is preserved o he ransformed program. Based on hese properies one can adjus a source-o-source ransformer or a compiler o use only hose opimizaions ha preserve he inended srucural code coverage. These coverage-preservaion properies o be mainained have o ensure ha whenever he code coverage is fulfilled a he original program by some es daa TD hen his coverage is also fulfilled a he ransformed program wih he same es daa: TD. coverage(p 1, TD) coverage(p 2, TD) (4) In he following we presen several coverage preservaion crieria aken from [3]. We use hese coverage preservaion crieria ogeher wih absrac descripions of he code ransformaions for he calculaion of he coverage profiles. 3.1 Preserving Saemen Coverage (SC) Equaion 5 of Theorem 31 provides a coverage preservaion crierion for saemen coverage. Equaion 5 essenially says ha for each basic block b of he ransformed program here exiss a basic block b of he original program such ha reaching b wih a given es vecor implies ha also b is reached wih he same es vecor.

7 Theorem 31 (Preservaion of SC) Assuming ha a se of es daa TD achieves saemen coverage on a given program P 1, hen Equaion 5 provides a sufficien - and wihou furher knowledge abou he program and he es daa (here is now knowledge abou he es daa or he program assumed), also necessary - crierion for guaraneeing preservaion of saemen coverage on a ransformed program P 2. (Proof given in [3]) b B(P 2 ) b B(P 1 ). IV R (b ) IV R (b) (5) 3.2 Preserving Condiion Coverage (CC) To define a coverage preservaion crierion for CC (Theorem 32) we use he auxiliary predicae ouches ID(x, ID) given in Equaion 6. The predicae ouches ID(x, ID) is only TRUE if he se of inpu daa ID includes a leas he rue-saisfiabiliy valuaion IV T (x) or he false-saisfiabiliy valuaion IV F (x) of expression x, where x is eiher a condiion or a decision. The predicae ouches ID(x, ID) is used for he coverage preservaion crierion of CC (and also DC) o es wheher he evaluaion of any expression x of he original program o boh, TRUE and FALSE, implies ha he es daa include a leas one elemen of ID, needed for he coverage of an expression in he ransformed program. ouches ID(x, ID) (IV T (x) ID) (IV F (x) ID); (6) Equaion 7 saes ha for each condiion c of he ransformed program here exiss a leas one condiion of he original program whose coverage implies ha c evaluaes o TRUE and here exiss a leas one condiion of he original program whose coverage implies ha c evaluaes o FALSE. Theorem 32 (Preservaion of CC) Assuming ha a se of es daa TD achieves condiion coverage on a given program P 1, hen Equaion 7 provides a sufficien - and wihou furher knowledge abou he program and he es daa, also necessary - crierion for guaraneeing preservaion of condiion coverage on a ransformed program P 2. (Proof given in [3]) c C(P 2 ). c C(P 1 ). ouches ID(c, IV T (c )) c C(P 1 ). ouches ID(c, IV F (c )) (7) 3.3 Preserving Decision Coverage (DC) To define a coverage preservaion crierion for DC (Theorem 33) we use he auxiliary predicae ouches ID(x, ID) given in Equaion 6, which is also used for preserving CC.

8 Equaion 8 of Theorem 33 provides a coverage preservaion crierion for decision coverage. Equaion 8 essenially says ha for each decision d of he ransformed program here exiss a leas one decision of he original program whose coverage implies ha d evaluaes o TRUE and here exiss a leas one decision of he original program whose coverage implies ha d evaluaes o FALSE. Theorem 33 (Preservaion of DC) Assuming ha a se of es daa TD achieves decision coverage on a given program P 1, hen Equaion 8 provides a sufficien - and wihou furher knowledge abou he program and he es daa, also necessary - crierion for guaraneeing preservaion of decision coverage on a ransformed program P 2. (Proof given in [3]) d D(P 2 ). d D(P 1 ). ouches ID(d, IV T (d )) d D(P 1 ). ouches ID(d, IV F (d )) (8) 4 Auomaic Calculaion of Compilaion Profiles This secion discusses he conceps and implemenaion behind auomaic calculaion of coverage profiles. 4.1 Program Model For modeling conrol flow, he sequence of execuion is defined by a se of labeled CFG edges R : E Λ, where E : N N are he CFG edges wih N : B C {s, }, Λ : {T, F } {T, F, X}, and : {δ 1,..., δ R }. The special labels Λ and are used o include informaion abou conrol flow ha depends on condiion/decision evaluaions and influence of inpu valuaions. Condiion/decision labels l Λ are used in case of condiion nodes o deermine he pah a program uses when he conrol flow forks depending on he resul of a condiion evaluaion. For flexibiliy in assigning condiion resuls o differen decision oucomes he condiion/decision labels are wo-pared. The firs par defines he condiion evaluaion resul using he symbols T and F for rue and false. The second par of he label deermines he decision resul correlaed wih he condiion resul accumulaed so far. I can be T, F or X if he decision oucome is no ye deermined. Noe ha X is only allowed for edges originaing and desinaing inside he decision hypernode. All ougoing edges of a decision mus carry a unique decision-label wih T or F. Each edge e i in he graph is assigned a valuaion se δ i. This valuaion se represens all he valuaions of he program inpu ha rigger he execuion of a pah going hrough edge e i. For each node v, excep s and, we have a coninuiy relaion of he form δ i δ j (9) e i IN (v) e j OUT (v) where IN (v) denoes he incoming edges of v and OUT (v) he ougoing edges of v. In oher words, execuion pahs enering a node mus leave he node a leas on one

9 ougoing edge. The only excepions are he enry-node s being he source and he exinode being he sink of each execuion pah. 4.2 Analyzing Code Opimizaions For analyzing he effec of code opimizaions we model he valuaion relaions beween he original and he ransformed code. Based on he coninuiy relaion (Equaion 9) i is easy o obain simple subse relaions ( ) beween he valuaion ses on incoming and ougoing edges inside each program graph. This can be done by walking hrough each node of he CFG and applying he coninuiy relaion in forward and backward direcion. These subse relaions are he basic inpu for coverage preservaion analysis. A code ransformaion adds addiional relaions of valuaion ses beween he original and he ransformed code, characerizing how he ransformaion forms he valuaion ses of he edges in he ransformed code based on he valuaion ses of he edges in he original code. These relaions can be propagaed along he CFG based on he ransiiviy of subse relaions. 4.3 The Mahemaica Implemenaion The implemenaion of he coverage-profile calculaion was done using Mahemaica, a fully inegraed environmen for echnical and mahemaical compuing [12]. In a preparaion-phase he conrol-flow graphs wih he node ses B and C, he decision se D and he edge se R mus be convered o he inernal daa srucures of he program sysem. Each edge e in R is implemened as a uple v, w, l, δ i where v is he sar- and w is he end-poin of he edge. l is he wo-pared condiion/decision label as described above (or empy if he edge is no originaing a a condiion node) and δ is a unique idenifier for he valuaion se. Reachabiliy and saisfyabiliy valuaion are reproduced inernally by collecing he valuaion ses on incoming and ougoing edges. IV R (x) is calculaed as absrac union of all valuaion ses on he incoming edges of node x. To calculae IV T (x) he union of all ougoing edges of x labelled wih T are calculaed and for IV F (x) all edges wih label F are menioned. Dependen weher x is a condiion or a decision, he informaion is exraced from he condiion or decision label. We consruc an auxiliary graph (derived from he CFG) for mainaining he equaliy relaions () or subse relaions ( ) beween valuaion ses. The nodes of he auxiliary graph represen valuaion ses or unions of valuaion ses. A direced edge δ i δ j is included in he suppor graph iff δ i δ j is rue. In case of δ i δ j he auxiliary graph conains edges beween δ i and δ j in boh direcions. Afer consrucing auxiliary graphs for he original code as well as he ransformed code, hese graphs are glued ogeher by adding he addiional relaions caused by he code ransformaion. These subse relaions form he absrac descripion of he code ransformaion ha we use for he calculaion of he coverage profiles. Creaing a graphreconsrucion language [13, 14] ha records he ransformaion relaion while reconsrucing he CFG is a possible exension for he fuure. So far we have implemened preservaion analyses for saemen coverage (SC), condiion coverage (CC), and decision coverage (DC). They ge descripions of he

10 original CFG and he ransformed CFG. Beside documenary informaion hey oupu a verdic rue or false abou he abiliy of he ransformaion o preserve he menioned coverage. The correcness of his verdics relies on providing a correc and precise absrac descripion of he code ransformaion. 5 Examples of Analyzing Coverage-Preservaion This secions shows he coverage preservaion analysis for several code opimizaions. To avoid confusion when relaing he valuaion ses of he original code and he ransformed code, we denoe δ i he valuaion ses of he original code and ϱ i he valuaion ses of he ransformed code. The resuls on coverage preservaion are summarized in Table Condiion Reordering wih Shor-Circui Evaluaion Algebraic simplificaions use algebraic properies of operaors like associaiviy, commuaiviy and disribuiviy o simplify expressions [1]. Alhough hese simplificaions produce logically equivalen expressions, hey may cause unexpeced changes in he flow of conrol. Under cerain circumsances hese changes can disrup srucural code coverage if hey change he order of condiions. This is demonsraed in he following example of a branch wih shor-circui evaluaion. The case sudy demonsraes condiion reordering in an if-saemen wih wo condiions conneced by a logical AND operaor wih shor-circui evaluaion. In a programming language he program code and he opimized code could look similar as in he following C-syled example: if ( A && B ) henblock else elseblock changed o if ( B && A ) henblock else elseblock Addiionally, shor-circui evaluaion of condiions is assumed, a echnique used in several programming languages. In C/C++, e.g., logical expressions inside an if-saemen are evaluaed from lef o righ. If evaluaion of furher erms could no change he resul anymore, evaluaion sops and he branch is execued immediaely. In he example, he second condiion is no evaluaed, if he resul of he firs condiion evaluaes o false. Figure 3 shows he inernal graph models for his use case wih he original program on he lef side and he ransformed program on he righ side. As a convenion he symbols δ are used o noe he valuaion ses of he original program and symbols ϱ are used for he ransformed program. In he original program, he described shorcircui branch is implemened wih he edge from condiion A o he else-block 4. In he ransformed program he shor-circui branch connecs condiion B wih else-block 14. Changing he condiion order by swapping he condiions will no change he valuaion-ses of he decision resul. This is denoed by he equaliy relaions () beween δ 6, ϱ 6 and δ 7, ϱ 7. Therefore, applying he preservaion condiion for DC (Equaion 8) and for SC (Equaion 5) will give a posiive preservaion verdic. Bu he disribuion of he valuaion ses of he condiions inside he decision are changed. Condiion

11 δ 1 s s ϱ 1 A T,X δ 2 T,T δ δ δ 6 B δ 7 δ3 B T,X ϱ 2 T,T ϱ ϱ 6 ϱ 3 A ϱ 5 Fig. 3. Transformaion Relaion for Condiion Reordering (wih shor-circui evaluaion) B in he ransformed code snippe will now decide on a bigger valuaion se han in he unransformed program while condiion A in he ransformed program decides on a subse of he possible valuaions. Applying he preservaion condiion for CC (Equaion 7) herefore resuls in a negaive preservaion verdic. The sample oupu of he implemened analyzing funcion in Figure 4 shows how he funcion makes use of he preservaion crieria o show, ha saemen coverage is preserved. The ool walks hrough each saemen node of he ransformed code. Using he coninuiy relaion ogeher wih he addiional subse relaions on he valuaion ses i deermines hose valuaion ses which are a subse of valuaion se IV R (x) of he currenly invesigaed node x. Finally, i searches for a node in he original code wih a valuaion-se ha is member of he relaed valuaion-ses. In he firs case his happens wih node 3 and i s valuaion se IV R (3) δ 4. The same principle is used o find node 4 as a counerpar for node 14. The las line of he lising gives as he funcion resul he final decision, which is rue in his case. This resul can be used o be included ino a coverage profile. ϱ Loop Peeling The ransformaion called loop peeling replaces he firs k ieraions from he beginning of a loop and insers k copies of he body ogeher wih incremen and es code of he loop index variable immediaely ahead of he loop [1]. A simplified example of his opimizaion is shown in Figure 5, where he compiler has peeled ou he firs ieraion of he loop, placing one copy of he loop body and he loop erminaion es in fron of he loop. From poin of view of code coverage analysis, his lile change in code srucure has severe effecs on preservaion of all coverage crieria. In he original program SC, CC, and DC can be achieved by execuing one ieraion of he loop. Afer applicaion of he ransformaion, he same es daa will no ener he loop, because he firs ieraion has been execued in advance.

12 ** SC-Preservaion ** B(P2): {13, 14} B(P1): {3, 4} 1 IV R ( 13) {ρ 4 } of P2 is relaed wih {{δ 4 }, {δ 6 }, {ρ 4 }, {ρ 6 }} Nodes of P1 saisfying preservaion condiion: {3} Accumulaed scpf: True 2 IV R ( 14) {ρ 3, ρ 5 } of P2 is relaed wih {{δ 3 }, {δ 5 }, {δ 7 }, {ρ 3 }, {ρ 5 }, {ρ 7 }, {δ 3, δ 5 }, {ρ 3, ρ 5 }} Nodes of P1 saisfying preservaion condiion: {4} Accumulaed scpf: True True Fig. 4. Sample Oupu Analyzing Saemen Coverage for a IF-Saemen wih Two Condiions (wih shor-circui evaluaion) Formal analysis canno prove coverage preservaion, because he body of he loop is only riggered by a subse ϱ 4 of he original valuaion subse δ 2. Therefore SC will fail for b B, because no saemen b in he original program could be found such ha IV R (b ) IV R (b). Proofing preservaion of CC and DC fails for similar reasons. 5.3 Loop Inversion Loop Inversion, in source-language erms, ransforms a while loop ino a do-while loop [1]. The loop closing es is moved from he beginning of he loop o he end of he loop. In he simples case his requires, ha i is save o execue he loop body a leas once. Oherwise, a es has o be generaed in fron of he loop o check he exi condiion. This laer case is illusraed in Figure 6. Alhough he relaion of he valuaion ses beween he original and he ransformed code conains many equaliies, only saemen coverage is preserved. This is, because he moved loop closing decision in he ransformed program only decides on a subse of he inpu valuaions compared wih he original program, which is expressed by he union operaion ( ) on he righ side of he equaliy relaion. This relaion is induced by he subse-relaion beween ϱ 1 and ϱ Condiion Reordering wihou Shor-Circui Evaluaion This example goes back o he condiion reordering example presened in Secion 5.1. The example presened in his subsecion is a variaion where all condiions are execued independenly of he oucome of he oher condiions of he decision. Besides SC and DC, also CC is now preserved. The main difference here is, ha each condiion decides on he full valuaion-se δ 2 δ 3 ϱ 2 ϱ 3, alhough he disribuion beween δ 2, δ 3 on one side and ϱ 2, ϱ 3 on he oher side may differ. The CFG in Figure 7 also shows an applicaion for he wo-pared condiion/decision label. Alhough condiion B in he original code on he lef side can

13 δ 4 s A T,T δ 2 B δ 1 δ 3 ϱ 7 s ϱ 1 A T,T ϱ 2 B ϱ 3 ϱ 6 A T,T ϱ 5 ϱ 4 B Fig. 5. Transformaion Relaion for Loop Peeling decide independenly of he resul of condiion A for rue, he decision resul mus be false if he resul of evaluaing condiion A was false. The same is rue in he ransformed program when condiion A decides rue bu he resul of condiion B was false. Code Opimizaion Coverage Preservaion SC CC DC Cond. reordering (wihou shor-circui) Cond. reordering (wih shor-circui) Loop peeling Loop inversion Table 1. Calculaed Coverage Profiles 6 Summary and Conclusion In his paper we addressed he raher novel field of preserving srucural code coverage during program ransformaion. A code ransformer ha ake care of preserving srucural code coverage has many ineresing applicaions. For example, his allows he realizaion of reliable and porable es-daa generaors. Besides funcional sofware esing, his is even ineresing for measuremen-based iming analysis. Our approach is based on he calculaion of so-called coverage profiles, which are ables ha sore he informaion of wha code ransformaions guaranees he preservaion of which srucural code-coverage meric. To calculae hese coverage profiles, we developed a formal coverage preservaion crieria for each srucural coverage meric

14 s s ϱ 1 δ 4 A T,T δ 2 B δ 1 δ 3 ϱ 3 A T,T ϱ 2 B ϱ 4 ϱ 5 T,T A ϱ 6 Fig. 6. Transformaion Relaion for Loop Inversion and infer i wih he absrac descripions of he code ransformaions. We have calculaed such coverage profiles for saemen coverage (SC), condiion coverage (CC), and decision coverage (DC). As fuure work, we are focusing on exending he calculaion of coverage profiles o more complex srucural code-coverage merics like he modified condiion-decision crierion (MCDC) or a scoped pah coverage. References 1. Muchnick, S.S.: Advanced Compiler Design & Implemenaion. Morgan Kaufmann Publishers, Inc. (1997) ISBN Kirner, R.: SCCP/x - a compilaion profile o suppor esing and verificaion of opimized code. In: Proc. ACM In. Conference on Compilers, Archiecure, and Synhesis for Embedded Sysems (CASES 07), Salzburg, Ausria (2007) Kirner, R.: Towards preserving model coverage and srucural code coverage. EURASIP Journal on Embedded Sysems 2009 (2009) doi: /2009/ Wenzel, I., Kirner, R., Rieder, B., Puschner, P.: Measuremen-based iming analysis. In: Proc. 3rd In l Symposium on Leveraging Applicaions of Formal Mehods, Verificaion and Validaion, Poro Sani, Greece (2008) 5. Kirner, R., Puschner, P., Wenzel, I.: Measuremen-based wors-case execuion ime analysis using auomaic es-daa generaion. In: Proc. 4h Inernaional Workshop on Wors-Case Execuion Time Analysis, Caania, Ialy (2004) Vilkomir, S.A., Bowen, J.P.: Formalizaion of sofware esing crieria using he z noaion. In: Proc. 25h Annual Inernaional Compuer Sofware and Applicaions Conference, Honolulu, Hawaii, USA (2001) Aho, A.V., Sehi, R., Ullman, J.D.: Compilers, Principles, Techniques, and Tools. Addison- Wesley (1997) ISBN Chilenski, J.J.: An invesigaion of hree forms of he modified condiion decision coverage (MCDC) crierion. Technical Repor DOT/FAA/AR-01/18, Boeing Commercial Airplane Group (2001)

15 δ 1 s s ϱ 1 δ 3 ϱ 2 T,X A T,X B δ 2 ϱ 3 T,T B T,T A δ 4 δ 5 ϱ 4 ϱ 5 T,F T,F δ 6 δ 7 δ 8 ϱ 8 Fig. 7. Transformaion Relaion for Condiion Reordering (wihou shor-circui evaluaion) 9. ISO: Programming Languages C. 2nd edn. ISO/IEC 9899:1999 (1999) Technical Commie: JTC 1/SC 22/WG Myers, G.J.: The Ar of Sofware Tesing. John Wiley & Sons (1979) 11. Hayhurs, K.J., Veerhusen, D.S., Chilenski, J.J., Rierson, L.K.: A pracical uoral on modified condiion/decision coverage. Technical Repor NASA/TM , Naional Aeronauics and Space Adminisraion, Hampon, Virginia (2001) available in pdf forma. 12. Wolfram, S.: The Mahemaica Book, 4h ed. Cambridge Universiy Press (1999) 13. Lacey, D., Jones, N.D., Wyk, E.V., Frederiksen, C.C.: Proving correcness of compiler opimizaions by emporal logic. SIGPLAN No. 37 (2002) Lerner, S., Millsein, T., Chambers, C.: Auomaically proving he correcness of compiler opimizaions. In: In Proceedings of he ACM SIGPLAN 2003 Conference on Programming Language Design and Implemenaion, ACM Press (2003) ϱ 7 ϱ 6