Abstract Application and System Model /DATE EDAA

Transcription

1 Analysis and Optimization of Fault-Tolerant Embedded Systems wit Hardened Processors Viaceslav Izosimov, Ilia Polian, Paul Pop, Petru Eles, Zebo Peng {viaiz petel Dept. of Computer and Inform. Science Linköping University SE-58 8 Linköping, Sweden Abstract In tis paper we propose an approac to te design optimization of fault-tolerant ard real-time embedded systems, wic combines ardware and software fault tolerance tecniques. We trade-off between selective ardening in ardware and process re-execution in software to provide te required levels of fault tolerance against transient faults wit te lowest-possible system costs. We propose a system failure probability (SFP) analysis tat connects te ardening level wit te maximum number of re-executions in software. We present design optimization euristics, to select te fault-tolerant arcitecture and decide process mapping suc tat te system cost is minimized, deadlines are satisfied, and te reliability requirements are fulfilled.. Introduction Safety-critical embedded systems ave to satisfy cost and performance constraints even in te presence of faults. In tis paper we deal wit transient and intermittent faults (also known as soft errors ), wic are very common in modern electronic systems. Teir number is increasing wit smaller transistor sizes and iger frequencies. Transient faults appear for a sort time, cause miscalculation in logic, corruption of data, and ten disappear witout pysical damage to te circuit. Causes of transient faults can be electromagnetic interference, radiation, temperature variations, software bugs, etc. [8]. Transient faults can be addressed in ardware wit ardening tecniques, i.e., improving te ardware arcitecture to reduce te soft error rate, or in software wit tecniques suc as re-execution, replication, or ceckpointing. In te context of fault-tolerant real-time systems, researcers ave tried to integrate fault tolerance tecniques and task sceduling [,, 4]. A static cyclic sceduling framework for design of fault-tolerant embedded control systems wit masking of fault patterns troug active replication is proposed in [4]. Girault et al. [5] propose a generic approac to address multiple failures wit active replication. Process criticality is used as a metric for selective replication in [0]. Transparent re-execution and constructive mapping and sceduling for fault tolerance ave been proposed in [9]. In [8, 7, 5] we ave proposed sceduling and fault tolerance policy assignment tecniques for distributed real-time systems, suc tat te required level of fault tolerance is acieved and real-time constraints are satisfied wit a limited amount of resources. Te researc mentioned above is focused on software fault tolerance tecniques. However, wit increased error rate due to new tecnologies and/or in te case of particular ars conditions (e.g. ig radiation), pure software tecniques are not sufficient in order to acieve te required level of fault tolerance [, 6]. Researcers ave recently proposed a variety of ardware ardening tecniques. Zang et al. [] propose an approac to selective ardening of flip-flops, resulting in a small area overead and significant reduction in te error rate. Moanram and Touba [] ave studied selective ardening of combinatorial circuits. Zou et al. []. Tis work was partially supported by te Swedis Graduate Scool in Computer Science (CUGS), te ARTES++ Swedis Graduate Scool in Real- Time Systems, and by te DFG project RealTest (BE 76/5-).. We will refer to bot transient and intermittent faults as transient faults /DATE EDAA polian@informatik.uni-freiburg.de Institute for Computer Science Albert-Ludwigs-University of Freiburg D-790 Freiburg im Breisgau, Germany Paul.Pop@imm.dtu.dk Dept. of Informatics and Mat. Modelling Tecnical University of Denmark DK-0 Kongens Lyngby, Denmark ave later proposed a filtering tecnique for ardening of combinatorial circuits. Zou and Moanram [] ave studied te problem of gate resizing as a tecnique to reduce te error rate. Garg et al. [4] ave connected diodes to te duplicated gates to implement an efficient and fast voting mecanism. Finally, a selective ardening approac to be applied in early design stages as been presented in [6], wic is based on te transient fault detection probability analysis. However, ardening comes wit a significant overead in terms of cost and speed [, 9]. Te factors wic affect te cost are te increased silicon area for fault tolerance, additional design effort, lower production quantities, excessive power consumption, and protection mecanisms against radiation suc as sields. Hardened processors are also significantly slower tan te regular ones. Te manufacturers of ardened processors are using tecnologies few generations back [, 9], and ardening enlarges te critical pat on te circuit e.g. because of voting mecanism [4] and increased silicon area. In tis work, we combine selective ardening wit software fault tolerance in order to acieve te lowest-possible system costs wile satisfying ard deadlines and fulfilling te reliability requirements. We use process re-execution to tolerate transient faults in software. To ensure tat te system arcitecture meets te reliability requirements, we propose a system failure probability (SFP) analysis. Tis analysis connects te levels of redundancy (maximum number of re-executions) in software to te levels of redundancy in ardware (ardening levels). We also propose a set of design optimization euristics in order to decide te ardening levels of computation nodes, te mapping of processes on computation nodes, and te number of re-executions on eac computation node. Processes and messages are sceduled using an approac we ave presented in [7, 5]. Experimental results sow an improvement of up 55% on syntetic applications in terms of te number of scedulable and reliable fault-tolerant solutions wit te acceptable cost; and an improvement of 66% for a realistic application in terms of cost. Te next two sections present our application model and fault tolerance tecniques, respectively. In Section 4, we outline our problem formulation. Section 5 illustrates ardening/re-execution trade-offs. Our euristics are discussed in Section 6 and experimental results are presented in Section 7. In Appendix A we present our SFP analysis.. Application and System Model We model an application A as a set of directed, acyclic graps G k (V k, E k ) A. Eac node P i V k represents one process. An edge e ij E k from P i to P j indicates tat te output of P i is te input of P j. A process can be activated after all its inputs, required for te execution, ave arrived. Te process issues its outputs wen it terminates. Processes cannot be preempted during teir execution. We consider tat te application is running on a set of computation nodes N connected to a bus. Processes mapped on different computation nodes communicate wit messages sent over te bus. We consider tat te worst-case size of messages is given, wic implicitly can be translated into te worst-case transmission time on te bus. In tis paper we assume tat communications are fault tolerant (i.e., we use a communication protocol suc as TTP [0]).

2 A : G P m m m 4 m P 4 D = ms N Figure. Application Example = = = t p t p t p P P Cost P Transient faults can affect processes executed on a computation node, wic would lead to a process failure. To reduce te probability of process failure, te designer can coose to use a ardened, i.e., a more reliable, version (-version) of te computation node. Tus, eac node is available in several versions, wit different ardening levels, denoted wit. We denote te -version of node, and wit C j te cost associated wit. A pair {P i, } indicates tat process P i is mapped to te -version of node. Te worst-case execution time (WCET) of P i executed on is denoted t ij. Te probability of failure of a single execution of process P i on is denoted p ij. WCETs (t) are determined wit worst-case analysis tools [], wile process failure probabilities (p) are determined using fault injection tools [, 8]. In Fig. we ave an application A consisting of te process grap G wit four processes,, P,, and P 4. Te deadline of te application grap D = ms. Te execution times (t) and failure probabilities (p) for te processes on different -versions of computation nodes and N are sown in te tables. Te corresponding costs are associated wit tese versions (given at te bottom of te tables).. Fault Tolerance Tecniques As a software fault tolerance mecanism we use process re-execution. We assume tat te error detection and fault tolerance mecanisms are temselves fault tolerant. Te time needed for detection of faults is accounted for as part of te WCET of te processes. Te process re-execution operation requires an additional overead captured as μ. For example, μ is 5 ms for te application A in Fig.. Safety-critical embedded systems ave to be designed suc tat tey meet a certain reliability goal ρ = γ. In tis paper we consider tat γ is te maximum probability of a system failure due to transient faults on any computation node witin a time unit, e.g. one our of functionality. For example, te reliability goal for te application A in Fig. is 0-5 witin one our. Wit sufficiently ardened nodes, te reliability goal can be acieved witout any re-execution at software level, since te probability of te ardware failing is acceptably small. As te level of ardening decreases, te probability of faults being propagated to te software level is increasing. Tus, in order to acieve te reliability goal, a certain number of re-executions ave to be introduced at software level. In Fig., we consider a process executed on a node wit a) ρ = 0-5 = b) = μ = 5 ms 6 N = = = t p t p t p P 4 Cost k = / / k = / / / c) = k =0 t = ms Figure. Re-execution and Hardening t =0ms t = 45ms tree -versions. Te worst-case execution scenario is different for te different -versions. In te first version, two re-executions, k =, ave to be introduced into software in order to meet te reliability goal. Te faults will be tolerated wit re-execution as presented in Fig. a. Te first execution of, denoted /, is affected by a fault and is re-executed as /, after a worst-case recovery overead μ = 5 ms. Te second execution /, in te worst case, also fails and is re-executed. Finally, te tird execution / will complete witout faults. In te second version wit a iger ardening level, only one re-execution, k =, as to be added into software, wic will correspond to te worst-case scenario wit one re-execution in Fig. b. In te most ardened version, k = 0, and process is executed witout re-executions at software level. Note tat te worst-case execution time of process as increased wit te ardening. Neverteless, in te example, an increased level of ardening as resulted in smaller worst-case delays (wic is not necessarily te case in general). In Appendix A we sow ow te maximum number of re-executions k j wic ave to be introduced at software level on node is connected to te reliability goal and te ardening level of te computation nodes. 4. Problem Formulation As an input we get an application A, represented as a set of acyclic directed graps G k A. Application A runs on a bus-based arcitecture as discussed in Section. Te reliability goal ρ, te deadline, and te recovery overead μ are given. Given is also a set of available computation nodes eac wit its available ardened -versions and te corresponding costs. We know te worst-case execution times, and te failure probabilities are obtained wit fault injection experiments [, 8] for eac process on eac -version of computation node. Te maximum transmission time of all messages, if sent over te bus, is given. As an output, te following as to be produced: () a selection of te computation nodes and teir ardening level; () a mapping of te processes to te nodes of te selected arcitecture; () te maximum number of re-executions on eac computation node; and (4) a scedule of te processes and communications. Te selected arcitecture, te mapping and te scedule sould be suc tat te total cost of te nodes is minimized, all deadlines are satisfied, and te reliability goal ρ is acieved. Acieving te reliability goal implies tat ardening levels are selected and te number of re-executions are cosen on eac node suc tat te elaborated scedule, in te worst case, satisfies te deadlines. 5. Motivational Examples Te first example, depicted in Fig., sows ow ardening can improve scedulability if te error rate is ig. In Fig., we consider one process,, and one processor,, wit tree -versions, witout ardening and and progressively more ardened. Te corresponding failure probabilities, te WCET and costs are depicted in te table. We ave to meet a deadline of ms and te reliability goal of 0-5 witin one our. As sown in Appendix A, te ardening levels are connected to te number of re-executions in software, to satisfy te reliability goal. Tus, using, we ave to introduce 6 re-executions to reac te reliability goal, as depicted in Fig. a, wic, in te worst case, will miss te deadline of ms. However, wit te -version, te failure probability is reduced by two orders of magnitude, and only two re-executions are needed for satisfying te reliability goal ρ. Tis solution will already meet te deadline as sown in Fig. b. In case of te most ardened arcitecture depicted in Fig. c, only one re-execution is needed. However, using will cost twice as muc as te previous solution wit less ardening. Moreover, due to performance degradation, te solution wit te maximal ardening

3 a) b) c) / / / Figure. Hardware Recovery vs. Software Recovery () / / /4 /5 /6 /7 / / ρ = 0-5 D = ms μ = 0 ms / = = = t p t p t p Cost P / a) c) N bus N N / m / m P 4 P / P / P 4/ P 4/ d) P P 4 C d = 64 Figure 4. Hardware Recovery vs. Software Recovery () 0 P / 0 C a = 7 C b = b) P / P / P 4/ P 4/ e) P P 4 C e = C c = will complete in te worst-case scenario exactly at te same time as te less ardened one. Tus, te arcitecture wit sould be cosen. In Fig. 4 we consider several arcitecture selection alternatives for te application A, presented in Fig., composed of four processes, wic can be mapped on tree -versions of nodes and N. Te ceapest two-processor solution tat meets te deadline and reliability goal is depicted in Fig. 4a. Te arcitecture consists of te -versions and N and costs 7 units. Based on our SFP calculations, te reliability goal can be acieved wit one re-execution on eac processor. Let us evaluate next some possible monoprocessor arcitectures. Wit te arcitecture composed of only, presented in Fig. 4b, according to te SFP analysis, te reliability goal is acieved wit k = re-executions at software level. As can be seen in te figure, te application is unscedulable. Similarly, te application is also unscedulable wit te arcitecture composed of only N, presented in Fig. 4c. Fig. 4d and Fig. 4e depict te solutions obtained wit te monoprocessor arcitecture composed of te most ardened versions of te nodes. In bot cases, te reliability goal is acieved witout re-executions at software level (k j = 0). It is interesting to observe tat even wit k = 0 wit te arcitecture consisting of, te application is unscedulable. Tis is because of te performance degradation due to te ardening. Tis degradation, owever, is smaller in te case of N and, tus, te solution in Fig. 4e is scedulable. If we compare te two scedulable alternatives in Fig. 4a and 4e, we observe tat te one consisting of less ardened nodes (Fig. 4a) is more cost efficient tan te monoprocessor alternative wit te most ardened node (Fig. 4e). Te decision on ow muc ardening to use is crucial in providing cost-efficient and scedulable fault-tolerant arcitectures. We ave to account for cost, performance degradation, and te number of re-executions in software. Te analysis, wic connects te ardening levels, process failure probabilities, and te maximum number of re-executions k j, is presented in Appendix A. 6. Design Strategy and Algoritms Our design strategy is outlined in Fig. 5. As an input we get te application grap G, te set of computation nodes N, deadline D, and te reliability goal ρ. Te strategy will return te arcitecture AR composed of te selected set of nodes, te ardening levels corresponding to eac node, te number of re-executions to be supported in software, te mapping of te application, and, finally, te static scedule. Te design euristic explores te set of arcitectures, and eventually selects tat arcitecture, wic minimizes cost, wile still meeting te scedulability and reliability requirements of te application. Te euristic starts wit te monoprocessor arcitecture (n = ), composed of only one (fastest) node (lines -). Te mapping, selection of software and ardware redundancy (re-executions and ardening levels) and te scedule are obtained for tis arcitecture (lines 5-9). If te application is unscedulable, te number of computation nodes is directly increased, and te fastest arcitecture wit n = n + nodes is cosen (line 5). If te application is scedulable on tat arcitecture wit n nodes, i.e., SL D, te cost C of tat arcitecture is stored as te best-so-far cost C best. Te next fastest arcitecture wit n nodes (in te case of no ardening) is ten selected (line 8). If on tat arcitecture te application is scedulable (after ardening is introduced) and te cost C < C best, it is stored as te best-so-far. Te procedure continues until te arcitecture wit te maximum number of nodes is reaced and evaluated. If te cost of te next selected arcitecture wit te minimum ardening levels is iger tan te best-so-far cost C best, suc arcitecture will be ignored (line 6). Te evaluation of an arcitecture is done at eac iteration step wit te MappingAlgoritm function. MappingAlgoritm receives as an input te selected arcitecture, produces te mapping, and returns te scedule corresponding to tat mapping. Te cost function used for optimization is also given as a parameter. We use two cost functions: () scedule lengt, wic produces te sortest-possible scedule lengt SL for te selected arcitecture for te best-possible mapping (line 7), and () arcitecture cost, in wic te mapping algoritm takes an already scedulable application as an input and ten optimizes te mapping to improve te cost of te application witout impairing te scedulability (line 9). MappingAlgoritm tries a set of possible mappings (as, for example, in Fig. 4), and for eac mapping it optimizes te levels of redundancy in software and ardware, wic are required to meet te reliability goal ρ. Te levels of redundancy are optimized inside te mapping algoritm wit te RedundancyOpt euristic presented in Sect. 6., wic returns te levels of ardening and te number of re-executions in software. Te function dependencies are sown in Fig. 5. Te re-executions in software are obtained wit ReExecutionOpt euristic, called inside DesignStrategy(G, N, D, ρ) n = AR = SelectArc(N, n) C best = MAX_COST 4 wile n N do 5 SetMinHardening(AR) 6 if C best > GetCost(AR) ten 7 SL = MappingAlgoritm(G,AR,D,ρ, SceduleLengt) 8 if SL D ten 9 C = MappingAlgoritm(G, AR, D,ρ, Cost) 0 if C < C best ten C best = C AR best = AR MappingAlgoritm end if RedundancyOpt 4 else 5 n = n + ReExecutionOpt 6 end if 7 end if Sceduling 8 AR = SelectNextArc(N, n) 9 end wile 0 return AR best Sceduling end DesignStrategy Figure 5. General Design Strategy

4 RedundancyOpt for eac vector of ardening levels. Ten te obtained alternative of redundancy levels is evaluated in terms of scedulability by te off-line sceduling algoritm Sceduling, wic is sortly described in Sect After completion of RedundancyOpt, Sceduling is called again to determine te scedule for eac selected mapping alternative in MappingAlgoritm. 6. Illustrative Example Te basic idea beind our design strategy is tat te cange of te mapping immediately triggers te cange of te ardening levels. Tus, tere is no need to directly cange ardening since it can be guided by te mapping. To illustrate tis, let us consider te application A in Fig. and mapping in Fig. 4a. Processes and P are mapped on, wile processes and P 4 are mapped on N. Bot nodes, and N, ave te second ardening level ( = ), and N. Wit tis arcitecture, according to our SFP calculation, one reexecution is needed on eac node in order to meed te reliability goal. As can be seen in Fig. 4a, te deadlines are satisfied in tis case. If, owever, processes and P are moved (re-mapped) onto node N, resulting in te mapping in Fig. 4e, ten using te tird ardening level ( = ) is te only option to guarantee te timing and reliability requirements, and tis alternative will be cosen by our algoritm for te respective mapping. If, for a certain mapping, te application is not scedulable wit any available ardening level, for example, te mapping in Fig. 4d, tis mapping will be discarded by our algoritm. 6. Mapping Optimization In our design strategy we use te MappingAlgoritm euristic wit two cost functions, scedule lengt and te cost. We ave extended te algoritm from [7, 5] to consider te different ardening and re-execution levels. Te mapping euristic investigates te processes on te critical pat. Tus, at eac iteration, processes on te critical part are selected for te re-mapping. Processes recently re-mapped are marked as tabu (by setting up te tabu counter) and are not touced. Processes, wic ave been waiting for a long time to be re-mapped, are assigned wit te waiting priorities and will be re-mapped first. Te euristic canges te mapping of a process if it leads to () te solution tat is better tan te best-so-far (including tabu processes), or () to te solution tat is worse tan te best-so-far but is better tan te oter possible solutions. At every iteration, te waiting counters are increased and te tabu counters are decreased. Te euristic stops after a certain number of steps witout any improvement. Moreover, in order to evaluate a particular mapping, for tis mapping we ave to obtain te ardening levels in ardware and te maximum number of re-executions in software. Tis is performed in te RedundancyOpt function, presented in te next section. 6. Hardening/Re-execution Trade-off Every time we evaluate a mapping move by te MappingAlgoritm, we run RedundancyOpt to obtain ardening levels in ardware and te number of re-executions in software (te latter obtained wit ReExecutionOpt). Te euristic takes as an input te arcitecture AR wit te minimum ardening levels and te given mapping M. At first, te euristic increases te scedulability of te application by increasing te ardening levels in a greedy fasion, obtaining te number of re-executions for eac vector of ardening. Te scedulability is evaluated wit te Sceduling euristic. Once a scedulable solution is reaced, we iteratively reduce ardening by one level for eac node, again, at te same time obtaining te corresponding numbers of re-executions. For example, in Fig. 4a, we can reduce from to, and from N to N. If te application becomes unscedulable, for example, in te case we reduce from to, suc a solution is not accepted. Among te scedulable ardened alternatives, we coose te one wit te lowest cost and continue. Te euristic iterates wile improvement is possible, i.e., tere is at least one scedulable alternative. In Fig. 4a, te euristic will stop once -versions N to N ave been reaced, since te solutions wit less ardening are not scedulable. Te ReExecutionOpt euristic is called in every iteration of RedundancyOpt to obtain te number of re-executions in software. Te euristic takes as an input te arcitecture AR, mapping M, and te ardening levels H. It starts witout any re-executions in software and increases te number of re-executions in a greedy fasion. Te euristic uses te SFP analysis and gradually increases te number of reexecutions until te reliability goal ρ is reaced. Te exploration of te number of re-executions is guided towards te largest increase in te system reliability. For example, if increasing te number of reexecutions by one on node will increase te system reliability from 0 - to 0-4 and, at te same time, increasing re-executions by one on node N will increase te system reliability from 0 - to 5 0-5, te euristic will coose to introduce one more re-execution on node N. 6.4 Sceduling In tis paper we adapt an off-line sceduling strategy, wic we ave proposed in [7, 5], tat uses recovery slack in order to accommodate te time needed for re-executions in case of faults. After eac process P i we assign a slack equal to ( t ij + µ) k j, were k j is te number of re-executions on te computation node wit ardening. Te slack is sared between processes in order to reduce te time allocated for recovering from faults. Te Sceduling euristic is used by te RedundancyOpt and mapping optimization euristics to determine te scedulability of te evaluated solution, and produces te best possible scedule for te final arcitecture. 7. Experimental Results For te experiments, we ave generated 50 syntetic applications wit 0 and processes. Te worst-case execution times (WCETs) of processes, considered on te fastest node witout any ardening, ave been varied between and 0 ms. Te recovery overead μ as been randomly generated between and 0% of process WCET. Regarding te arcitecture, we consider nodes wit five different levels of ardening. Te failure probabilities of processes running on different -versions of computation nodes ave been obtained using fault injection experiments. We ave considered tree fabrication tecnologies wit te average transient (soft) error rates (SER) per clock cycle at te minimum ardening level of 0-0, 0 -, and 0 -, respectively, were 0-0 corresponds to te tecnology wit te igest level of integration and te smallest transistor sizes. Te ardening performance degradation (HPD) from te minimum to te maximum ardening level as been varied from 5% to 00%, increasing linearly wit te ardening level. For a HPD of 5%, te WCET of processes increases wit eac ardening level wit,,, 4, and 5%, respectively; for HPD = 00%, te increases will be, 5, 50, 75, and 00% for eac level, respectively. Initial processor costs (witout ardening) ave been generated between and 6 cost units. We ave assumed tat te ardware cost increases linearly wit te ardening level. Te system reliability requirements ave been varied between ρ = and witin one our. Te deadlines ave been assigned to all te applications independent of te transient error rates and ardening performance degradation of te computation nodes. Te experiments ave been run on a Pentium 4.8 GHz processor wit Gb memory.

5 % accepted arcitectures 00 % accepted arcitectures HPD = 5% HPD = 50% HPD = 00% HPD = 5% 5% 50% 00% (a) % accepted arcitectures as a function of ardening performance degradation (HPD) for SER = and ArC = 0 (b) % accepted arcitectures wit different HPD and ArC for SER = 0 - MAX 00 MIN 00 OPT MAX MIN OPT 0 MAX MIN OPT (c) % accepted arcitectures as a function of soft error rate (SER) (HPD = 5%, ArC = 0) Figure 6. Experimental Results (d) % accepted arcitectures as a function of soft error rate (SER) (HPD = 00%, ArC = 0) % accepted arcitectures MAX MIN OPT MaxCost HPD = 5% 0 In our experimental evaluation, we compare our design optimization strategy from Section 6, denoted OPT, to te two strategies, in wic te ardening optimization step as been removed from te mapping algoritms. In te first strategy, denoted MIN, we use only computation nodes wit te minimum ardening levels. In te second strategy, denoted MAX, only te computation nodes wit te maximum ardening levels are used. Te experimental results are presented in Fig. 6, wic demonstrates te efficiency of our design approaces in terms of te applications (in percentage) accepted out of all considered applications. By te acceptable application we mean an application tat meets its reliability goal, is scedulable, and does not exceed te maximum arcitectural cost (ArC) provided. In Fig. 6a, for SER = 0 - and ArC = 0 units, we sow ow our strategies perform wit an increasing performance degradation due to ardening. Te MIN strategy always provides te same result because it uses te nodes wit te minimum ardening levels and applies only software fault tolerance tecniques. Te efficiency of te MAX strategy is lower tan for MIN and is furter reduced wit te increase of performance degradation. Te OPT gives 8% improvement on top of MIN, if HPD = 5%, 0% improvement if HPD = 5%, and 8% improvement for 50% and 00%. More detailed results for ArC = 5 and ArC = 5 cost units are sown in te table in Fig. 6b, wic demonstrate similar trends. In Fig. 6c and Fig. 6d, we illustrate te performance of our design strategies as a function of te error rate. Te experiments in Fig. 6c ave been performed for HPD = 5%, wile te ones in Fig. 6d correspond to HPD = 00%. Te maximum arcitectural cost is 0 units. In te case of a small error rate SER = 0 -, te MIN strategy is as good as our OPT due to te fact tat te reliability requirements can be acieved exclusively wit only software fault tolerance tecniques. However, as SER is increased to 0 -, our OPT strategy already outperforms MIN. For SER = 0-0, OPT is significantly better tan bot oter strategies since in tis case finding a proper trade-off between te levels of redundancy in ardware and te levels of software re-execution becomes more important. Te execution time of our OPT strategy for te examples tat ave been considered is between minutes and minutes. We ave also run our experiments on a real-life example, a veicle cruise controller (CC) composed of processes [8]. Te CC considers an arcitecture consisting of tree nodes: Electronic Trottle Module (ETM), Anti-lock Braking System (ABS) and Transmission Control Module (TCM). We ave set te system reliability requirements to ρ =. 0-5 witin one our and considered μ between and 0% of process average-case execution times. Te SER for te least ardened versions of modules as been set to 0 - ; five -versions ave been considered wit HPD = 5% and linear cost functions. We ave considered a deadline of 00 ms. We ave found tat CC is not scedulable if te MIN strategy wit te minimum ardening levels as been used. However, CC is scedulable wit te MAX and OPT approaces. Moreover, our OPT strategy wit te trading-off between ardware and software redundancy levels as produced results 66% better tan te MAX in terms of cost. 8. Conclusions In tis paper we ave considered ard real-time applications mapped on distributed embedded arcitectures. We were interested to derive te least costly implementation tat meets imposed timing and reliability constraints. We ave considered two options for increasing te reliability: ardware redundancy and software re-execution. We ave proposed a design optimization strategy for minimizing of te overall system cost by trading-off between processor ardening and software re-execution. Our experimental results ave sown tat, by selecting te appropriate level of ardening in ardware and re-executions in software, we can satisfy te reliability and time constraints of te applications wile minimizing te cost of te arcitecture. Te optimization relies on a system failure probability analysis, wic connects te level of ardening in ardware wit te number of re-executions in software. 9. Appendix A A. System Failure Probability (SFP) Analysis In tis appendix we present an analysis tat determines te system failure probability, based on te number of re-executions in software and te process failure probabilities on te computation nodes wit different ardening levels. Te process failure probability p ij of process P i, executed on computation node wit ardening level, is obtained wit simulation using fault injection tools suc as [, 8]. Mapping of a process P i on te -version of computation node will be denoted as M(P i ) =. In te analysis, first, we calculate te probability Pr(0; ) of no faults occurring (no faulty processes) during one iteration of te application on te -version of node, wic is te probability tat all processes mapped on N j will be executed correctly: Pr 0; N = j ( p ) ij () P M( P ) = N i i j To account for faulty processes and re-executions, we will first refer to f-fault scenarios as to combinations wit repetitions of f faults on te number Π( ) of processes mapped on te computation node. Under a combination wit repetitions of n on m, we will understand te process of selecting n elements from a set of m elements, were eac element can be selected more tan once and te order of selection does not matter [7]. For example, an application A is composed of processes, P, and, wic are mapped on node. k = transient faults may occur, e.g. f =. Let

6 us consider one possible fault scenario. Process fails and is re-executed, its re-execution fails but ten it is re-executed again witout faults. Process P fails once and is re-executed witout faults. Tus, in tis fault scenario, from a set of processes, P and, processes and P are selected; moreover, process is selected twice, wic corresponds to one repetition. Te probability of recovering from a particular combination of f faults consists of two probabilities, te probability tat tis combination of f faults as appened and tat all te processes, mapped on, will be eventually (re-)executed witout faults. Te latter probability is, in fact, te no fault probability Pr(0; N j ). Tus, te probability of successful recovering from f faults in a particular fault scenario S* is Pr S fn ( ; j ) = Pr( 0; ) p s j () s ( S m, ) were ( S m, ) = f, ( S m, ) ( Sm, ), S N, S = Π( ), sup( m()a a S) = f. Te combination wit repetitions is expressed ere wit a finite submultiset (S*, m*) of a multiset (S, m) [7]. Informally, a multiset is simply a set wit repetitions. Formally, in te present context, we define eac our finite multiset as a function m: S N on set S, wic includes indices of all processes mapped on N j, to te set N of (positive) natural numbers. For eac process P a wit index a in S te number of repetitions is te number m(a), wic is less or equal to f faults (expressed as a supremum of function m(a)). Te number of elements in S* is f, e.g. te number of faulty processes. Tus, if a is repeated f times, m(a) = f, i.e., P a fails f times, S* will contain only repetitions of a and noting else. From (), te probability tat te system recovers from all possible f faults is a sum of probabilities of all f-fault recovery scenarios : Pr( f; ) = Pr( 0; ) p s j () (, ) ( S, m) (, ) Suppose tat we consider a situation wit maximum k j re-executions on te -version of te node. Te node fails if more tan k j faults are occurring. From () and (), we will derive te failure probability of te -version of node wit k j re-executions as S m Pr( f > k j ; ) = Pr( 0; ) Pr( f; ) (4) f = were we subtract from te initial failure probability wit only ardware redundancy, Pr( 0; ), te probabilities of all te possible successful recovery scenarios provided wit k j re-executions. Finally, te probability tat te system composed of n computation nodes wit k j re-executions on eac node will not recover, in te case more tan k j faults ave appened on any computation node, can be obtained as follows: n. Te combinations of faults in te re-executions are mutually exclusive.. Symbols and indicate tat numbers are rounded up wit 0 - accuracy; and indicate tat numbers are rounded down wit 0 - accuracy. It is needed for pessimism of fault-tolerant design. n s k j S m Pr ( f > k j ; ) = ( Pr( f > k j ; )) (5) j = j = According to te problem formulation, te system non-failure probability in te time unit τ (i.e., one our) of functionality as to be above te reliability goal ρ = γ, were γ is te maximum probability of a system failure due to transient faults witin te time unit τ. Considering tat te calculations above ave been performed for one iteration of te application (i.e., witin a period T), we obtain te following condition for our system to satisfy te reliability goal n Pr ( f > k j ; ) τ T -- ρ (6) j = A. Computation Example To illustrate ow te formulae ()-(6) can be used in obtaining te number of re-execution be introduced at software level, we will consider te arcitecture in Fig. 4a. At first, we compute te probability of no faulty processes for bot nodes and N : Pr(0;N ) = (. 0-5 ) (. 0-5 ) = Pr(0;N ) = (. 0-5 ) (. 0-5 ) = According to formulae (4) and (5), Pr(f > 0; ) = = Pr(f > 0; N ) = = Pr((f > 0; ) (f > 0; N )) = ( ) ( ) = Te system period T is ms, ence system reliability is ( ) 0000 = , wic means tat te system does not satisfy te reliability goal ρ = 0-5. Let us now consider k = and k = : Pr(; )= ( ) = Pr(;N )= ( ) = According to formulae (4) and (5), Pr(f >; )= = Pr(f >;N )= = Pr((f > ; ) (f > ; N )) = Hence, te system reliability is ( ) 0000 = and te system meets its reliability goal ρ = References [] J. Aidemark, J. Vinter, P. Folkesson, and J. Karlsson, GOOFI: Generic Object-Oriented Fault Injection Tool, Intl. Conf. on Dependable Systems and Networks (DSN), , 00. [] C. Ferdinand, R. Heckmann, M. Langenbac, F. Martin, M. Scmidt, H. Teiling, S. Tesing, and R. Wilelm, Reliable and Precise WCET Determination for a Reallife Processor, EMSOFT 00, Worksop on Embedded Software,, Lecture Notes in Computer Science, , Springer-Verlag, 00. [] C.C. Han, K.G. Sin, and J. Wu, A Fault-Tolerant Sceduling Algoritm for Real-Time Periodic Tasks wit Possible Software Faults, IEEE Trans. on Computers, 5(), 6 7, 00. [4] R. Garg, N. Jayakumar, S.P. Katri, and G. Coi, A Design Approac for Radiation- Hard Digital Electronics, Design Automation Conf. (DAC), , 006. [5] A. Girault, H. Kalla, M. Sigireanu, and Y. Sorel, An Algoritm for Automatically Obtaining Distributed and Fault-Tolerant Static Scedules, Intl. Conf. on Dependable Systems and Networks (DSN), 59-68, 00. [6] J.P. Hayes, I. Polian, B. Becker, An Analysis Framework for Transient-Error Tolerance, IEEE VLSI Test Symp., 49-55, 007. [7] V. Izosimov, P. Pop, P. Eles, and Z. Peng, Design Optimization of Time- and Cost-Constrained Fault-Tolerant Distributed Embedded Systems, DATE Conf., , 005. [8] V. Izosimov, Sceduling and Optimization of Fault-Tolerant Embedded Systems, Licentiate Tesis No. 77, Dept. of Computer and Information Science, Linköping University, 006. [9] N. Kandasamy, J.P. Hayes, and B.T. Murray, Transparent Recovery from Intermittent Faults in Time-Triggered Distributed Systems, IEEE Trans. on Computers, 5(), 5, 00. [0]H. Kopetz, G. Bauer, Te Time-Triggered Arcitecture, Proc. of te IEEE, 9(), 6, 00. [] F. Liberato, R. Melem, and D. Mosse, Tolerance to Multiple Transient Faults for Aperiodic Tasks in Hard Real-Time Systems, IEEE Trans. on Computers, 49(9), , 000. [] K. Moanram and N.A. Touba, Cost-Effective Approac for Reducing Soft Error Failure Rate in Logic Circuits, Intl. Test Conf. (ITC), 89-90, 00. [] P. Patel-Predd, Update: Transistors in Space, IEEE Spectrum, 45(8), 7-7, 008. [4] C. Pinello, L.P. Carloni, and A.L. Sangiovanni-Vincentelli, Fault-Tolerant Distributed Deployment of Embedded Control Software, IEEE Trans. on CAD, 7(5), , 008. [5] P. Pop, V. Izosimov, P. Eles, and Z. Peng, Design Optimization of Time- and Cost- Constrained Fault-Tolerant Embedded Systems wit Ceckpointing and Replication, IEEE Trans. on VLSI (In Print), 009. [6] P. Sivakumar, M. Kistler, S.W. Keckler, D. Burger, and L. Alvisi, Modeling te Effect of Tecnology Trends on te Soft Error Rate of Combinational Logic, Intl. Conf. on Dependable Systems and Networks (DSN), 89-98, 00. [7] R.P. Stanley, Enumerative Combinatorics, Vol. I, Cambridge Studies in Advanced Matematics 49, pp. -7, Cambridge University Press, 997. [8]L. Sterpone, M. Violante, R.H. Sorensen, D. Merodio, F. Sturesson, R. Weigand, and S. Mattsson, Experimental Validation of a Tool for Predicting te Effects of Soft Errors in SRAM-Based FPGAs, IEEE Trans. on Nuclear Science, 54(6), Part, , 007. [9] I.A. Troxel, E. Grobelny, G. Cieslewski, J. Curreri, M. Fiscer, and A. George, Reliable Management Services for COTS-based Space Systems and Applications, Intl. Conf. on Embedded Systems and Applications (ESA), 69-75, 006. [0] Y. Xie, L. Li, M. Kandemir, N. Vijaykrisnan, and M.J. Irwin, Reliability- Aware Co-syntesis for Embedded Systems, Proc. 5 t IEEE Intl. Conf. on Appl.-Spec. Syst., Arc. and Proc., 4 50, 004. [] M. Zang, S. Mitra, T.M. Mak, N. Seifert, N.J. Wang, Q. Si, K.S. Kim, N.R. Sanbag, and S.J. Patel, Sequential Element Design Wit Built-In Soft Error Resilience, IEEE Trans. on VLSI, 4(), 68-78, 006. [] Q. Zou and K. Moanram, Gate Sizing to Radiation Harden Combinational Logic, IEEE Trans. on CAD, 5(), 55-66, 006. [] Q. Zou, M.R. Coudury, and K. Moanram, Tunable Transient Filters for Soft Error Rate Reduction in Combinational Circuits, European Test, 79-84, 008. [4] D. Zu and H. Aydin, Reliability-Aware Energy Management for Periodic Real-Time Tasks, Real-Time and Embedded Tecnology and Applications Symp., 5-5, 007.