Methods for Multi-Dimensional Robustness Optimization in Complex Embedded Systems

Transcription

1 Methods for Multi-Dimensional Robustness Optimization in Complex Embedded Systems Arne Hamann, Razvan Rau, Rolf Ernst Institute of Computer and Communiation Network Engineering Tehnial University of Braunshweig, Germany {hamann rau ABSTRACT Design spae exploration of embedded systems typially fouses on lassial design goals suh as ost, timing, buffer sizes, and power onsumption. Robustness riteria, i.e. sensitivity of the system to variations of properties like exeution and transmission delays, input data rates, CPU lok rates, et., has found less attention despite its pratial relevane. In this paper we introdue multi-dimensional robustness metris, expressing the stati and dynami design robustness of a given system, the former assuming a fixed parameter onfiguration, and the latter inluding parameter adaptations as response to property variations. Additionally, we propose a metri measuring the robustness gain that an be ahieved through system reonfigurability. Sine determining multi-dimensional robustness is omputationally expensive we introdue effiient exploration methods based on a stohasti sensitivity analysis tehnique apable of deriving upper and lower robustness bounds for a given system with low omputational effort. We demonstrate the robustness optimization methods by means of a small but realisti ase study. Categories and Subjet Desriptors C.3 [Speial-Purpose and appliation-based systems]: Realtime and embedded systems; C.4 [Performane of systems]: Modeling tehniques; Performane attributes; Reliability, availability, and servieability General Terms Algorithms, Design, Performane, Reliability, Verifiation 1. INTRODUCTION In the embedded systems design flow, design robustness to property variations, suh as exeution and transmission delays, input data rates, CPU lok rates, et., is playing an inreasingly important role. Generally, system robustness is desirable to aount for estimation errors in early design phases, minor hanges of speifiations, bug fixes or later extensions and updates of the design to name just a few of the many situations where tolerane of hardware and run time system to modifiations is expeted. For instane, it is known that small task ore exeution time modifiations in systems with Permission to make digital or hard opies of all or part of this work for personal or lassroom use is granted without fee provided that opies are not made or distributed for profit or ommerial advantage and that opies bear this notie and the full itation on the first page. To opy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speifi permission and/or a fee. EMSOFT 07, September 30 Otober 3, 2007, Salzburg, Austria. Copyright 2007 ACM /07/ $5.00. omplex performane dependenies an have drasti non-intuitive effets on the overall system behavior, and might lead to severe performane degradation effets [10]. In the urrent state of pratie, designers reserve some slak for ritial system parameters to ensure system robustness. A prominent example is the bus load model, where designers limit the average bus utilization to ensure system funtionality and extensibility [11]. While suh design guidelines used to work reasonably well in pratie to ensure design robustness, they are gradually running out of steam. The main reason for this is the growing size and omplex networked nature of modern embedded systems, making it extremely diffiult to predit the effets of modifiations on load and timing. The omplexity of the problem is additionally inreased by the large number of independently developed appliations that are integrated on the same system leading to unknown oupling effets or limitations. Examples are ars or airrafts. Results are an inreasing design risk and non-extendable systems. In this paper we address the appliation of formal models to robustness optimization. We first formulate the problem we address in this paper (Setion 2) and give a brief survey of related work (Setion 3). Afterwards, we introdue two formal robustness metris for different design senarios (Setion 5), whih are based on sensitivity analysis (Setion 4), and propose effiient exploration methods onsidering them during design spae exploration (Setion 6). The stati design robustness expresses the robustness of a given system with respet to property variation of a set of ritial system properties for a fixed parameter onfiguration. The dynami design robustness additionally inludes ounter ations (i.e. system parameter adaptations) as response to property variations. Based on the stati and dynami design robustness metris we introdue a metri measuring the robustness gain that an be ahieved through system reonfigurability. Finally, we demonstrate the proposed metris and tehniques by means of a small but realisti ase study (Setion 7). 2. PROBLEM STATEMENT There are many notions of robustness in the ontext of embedded system design. Frequently, robustness is assoiated with fault tolerane, i.e. tehniques that ensure that the system funtions orretly in the presene of faults. Examples are task re-exeution or repliation mehanisms [8]. However, in this paper we define robustness differently. We all systems robust, if they an sustain property hanges (e.g. WCETs, periods, CPU lok rates, et.) without severe degradation of system funtioning and performane. Consequently, the tehniques presented in this paper are meant to support the designer in oneiving systems that are robust with respet to property variations. Basially, we distinguish two different kinds of system property variations: variations influening the system load, and variations influening the system servie apaity. 104

2 Reasons for system load variations are mainly hanges of software exeution path lengths, ommuniation volumes, and input data rates. Senarios under whih suh load variations an our during design time or even in the field inlude late feature requests, produt variants, software updates, and bug-fixes. System servie apaity variations are aused by modifiations of the exeution platform, e.g. proessor or ommuniation link performane hanges. Suh variations rarely our in the field. However, they they are of partiular interest during early design spae exploration, where load requirements are still subjet to hanges, and where different alternative system omponents and arhitetures need to be evaluated. Small but pratial industrial ase studies [14] have proven that the influenes of unpredited property variations on the system performane is an important issue during the design of omplex embedded systems. In the automotive industry, for instane, the global system is deomposed into several omponents, whih are independently developed by multiple suppliers based on requirement definitions. One design is finished, the OEM integrates all omponents, whih is mainly a network integration problem, i.e. the OEM must deide about bus topology, throughput, number of nodes, as well as assignment of priorities and time slots. This massively parallel design style leads to a variety of severe problems. For instane, many system parameters, suh as CAN message IDs (priorities), are fixed quite early in the design flow, where exeution delays, periods, and transmitted data sizes are only rough estimations and still subjet to hanges. In order to aount for suh unertainties, it is essential that robustness riteria are taken into aount as early as possible in the design flow, when system parameters an still be hanged to optimize the design. By this means ritial bottleneks an be prevented, whih redues the overall design risk. Therefore, we present in this paper methods helping the designer finding system parameter onfigurations, inluding the assignment of free system parameters like sheduling (priorities, time slots, et.), leading to systems with high robustness (low sensitivity) to property variations. In other words, the system shall meet its timing onstraints even under onsiderable hanges of system properties, inluding worst-ase exeution and ommuniation times, CPU lok rates, input data rates, et. Note that we fous in this paper on hard real-time onstraints. 3. RELATED WORK One approah to ahieve design robustness is parameter adaptation at run-time, suh as in adaptive sheduling strategies [9]. Adaptation is very effiient but omes with run-time overhead and makes it more diffiult to determine the resulting design robustness. Consequently, adaptation potentially improves the system but does not irumvent the need to determine system robustness. If we want to determine robustness in a systemati way we need sensitivity analysis. Sensitivity analysis determines the boundaries between working and non-working systems for arbitrary properties. The one-dimensional sensitivity analyses approahes presented in [13, 15], for instane, an apture the effets of single system property variations taking into aount omplex global timing effets. They are apable of alulating the maximum (or minimum) permissible values for large variety of system properties, inluding worst-ase exeution times, ommuniation volumes, input data rates, proessor, ommuniation link performane, et. One we have a sensitivity analysis available, we an take the next step to inlude robustness as a regular design goal. For that purpose we need formal robustness metris that allow to quantify robustness that are then used in design spae exploration. The approah presented in [5] defines robustness metris using onedimensional sensitivity analysis tehniques. However, sine the underlying sensitivity analysis is one-dimensional these robustness metris ignore dependenies between the onsidered system properties. For instane a small variation of one system property might have drasti effets on the robustness potential of another system property. There are approahes that onsider multi-dimensional robustness optimization problems [4, 1]. However, these approahes are based on very simple appliation models, whih on the one hand failitates the determination of the needed multi-dimensional robustness data, but on the other hand also relativizes the expressiveness of the obtained results. In order to be signifiant, robustness optimization results must be based on state-of-the-art performane models [7, 2] that are able to tightly determine system performane properties suh as timing, buffer sizes, and power onsumption. Unfortunately, exat multidimensional sensitivity analysis approahes [12], whih are appliable to suh performane models, annot be adapted in a straightforward manner to robustness optimization. The main reason for this is the high omputational omplexity of these methods, partiularly for sensitivity analyses with dimensions greater than two, prohibiting the evaluation of a large number of system onfigurations for optimization. In this paper we formally define expressive multi-dimensional robustness metris for different design senarios. We show how stohasti sensitivity analysis methods based on state-of-the-art performane models an be utilized to effiiently bound the multidimensional sensitivity properties of a given system with relatively little omputational effort ompared to exat approahes. 4. SENSITIVITY ANALYSIS The robustness optimization methods presented in this paper are based on sensitivity analysis tehniques. Sensitivity analyses [13, 15] an apture the global effets of system property variations. They are apable of alulating the boundaries representing the transition between working and non-working systems with respet to a large variety of system properties, inluding worst-ase exeution times, ommuniation volumes, input data rates, proessor and ommuniation link performane, et. Note that the tehniques desribed in the remainder of this paper are appliable to system properties subjet to maximization (e.g. worst-ase exeution times, proessor and ommuniation link performane, et.). However, all definitions and algorithms an be easily adapted to also over system properties subjet to minimization (e.g. input data rates, et.). Modern sensitivity analysis approahes [13, 12] are independent of the underlying appliation and arhitetural model. They an thus be applied to arbitrary formal analysis engines. For the remainder of this paper we assume that we dispose of a system evaluation funtion (Definition 1) apable of heking whether or not a given system with speifi system property values is working, i.e. the system fulfills all onstraints (maximum end-to-end latenies, maximum jitters, maximum buffer sizes, et.) and that no resoure is overloaded. Definition 1 (System Evaluation Funtion) Let S denote the system S with parameter onfiguration and P = {p 1,..., p n } a set of system properties. The system evaluation funtion f S heks whether or not S is working if the property values v = (v 1,...,v n ) are applied to the properties P. true, if S is working with values v for the f S (P, v) = properties P false, otherwise. Given this system evaluation funtion we an, for instane, use the approah in [13] to alulate the extreme values for arbitrary 105

3 system properties, whih still lead to a working system. Calulating the extreme value for a single system property is alled onedimensional sensitivity analysis, and an be formalized as follows: Definition 2 (Original and extreme property values) Let S denote the system S with parameter onfiguration. For a given system property p the original property value is denoted as v(p). The extreme property value for p is denoted as v + S (p) and defined as follows: v + S (p) = max{x R + f S (p,x)} If we want to extend sensitivity analysis to the multi-dimensional ase, i.e. taking into aount dependenies between multiple system properties, we an use the results of one-dimensional sensitivity analysis to bound the spae ontaining the sought-after front between working and non-working system property value ombinations. We all this multi-dimensional bound the bounding hyperube. Definition 3 (Bounding Hyperube) Let S denote the system S with parameter onfiguration. For a given set of system properties P = {p 1,..., p n } the n-dimensional bounding hyperube H S (P ) R n + is defined as follows: [ ( ) ( )] H S (P ) = min v(p 1 ),v + S (p 1 ),max v(p 1 ),v + S (p 1 ) [ ( min v(p n ),v + S (p n )... ),max ( v(p n ),v + S (p n ) The exat boundary between working and non-working system property ombination is alled sensitivity front. It an be formally haraterized using the notion of Pareto-optimality (Definition 4). Definition 4 (Pareto-optimality) Given a set V of n-dimensional vetors in R n, the vetor v = (v 1,...,v n ) V dominates the vetor w = (w 1,...,w n ) V iff for all elements 1 i n we have 1. minimization problem: v i w i and for at least one element l we have v l < w l. 2. maximization problem: v i w i and for at least one element l we have v l > w l. A vetor is alled Pareto-optimal iff it is not dominated by any other vetor in V. Notations: v + w ( v w) denotes v Pareto-dominates w in the sense of a maximization (minimization) problem. Ω + (V ) (Ω (V )) denotes the set of vetors in V whih are Pareto-optimal in the sense of a maximization (minimization) problem. The formal haraterization of the sensitivity front is given in Definition 5, whih generalizes Definition 2. Definition 5 (Sensitivity Front) Let S denote the system S with parameter onfiguration. For a given set of system properties subjet to maximization P = {p 1,..., p n } the sensitivity front is defined as the set of all working property value ombinations whih are not Pareto-dominated by any other working property value ombination: FS sens (P ) ={ x H S (P ) f S (P, x) y H S (P ) : ( f S (P, y) y + x)} )] The sensitivity front separates the spae of working and nonworking system property ombinations. The robustness metris presented in Setion 5 are based on the sensitivity information it represents. 5. MULTI-DIMENSIONAL ROBUSTNESS METRICS In this setion we introdue multi-dimensional robustness metris. We therefore first disuss the notion of hypervolume (Setion 5.1), on whih the proposed metris are based. Afterwards, we define stati and dynami design robustness metris, the former assuming a stati system parameter onfiguration, and the latter inluding dynami parameter adaptations as response to property variations (Setions 5.2 and 5.3). We explain several appliation senarios ranging from platform optimization to ritial omponent identifiation. 5.1 Hypervolume alulation In this setion we shortly disuss hypervolume alulation for an arbitrary number of dimensions. We first introdue the notion of absolute hypervolume (Setion 5.1.1), whih is for instane used as a metri in evolutionary optimization. Based on the absolute hypervolume we then define the weighted perentage hypervolume (Setion 5.1.2), whih we use in this paper to define expressive multi-dimensional robustness metris Absolute hypervolume Aording to Definition 6 we distinguish two different types of hypervolumes: the inner and the outer hypervolume. Definition 6 (Absolute Inner and Outer Hypervolumes) We onsider a n-dimensional hyperube H = [ b 1,b 1 ]... [ bn,b n ] R n + and a set of n-dimensional vetors V = { v 1,..., v m } with i v i H. 1. The inner hypervolume of V in H is denoted as λ H (V ) and is defined as the volume of the spae in H ontaining all vetors w whih are Pareto-dominated by at least one vetor v V : λ H (V ) = vol{ w H v V : v + w } 2. The outer hypervolume of V in H is denoted as λ + H (V ) and is defined as the differene between the volume of the hyperube H and the volume of the spae in H ontaining all vetors w Pareto-dominating at least one vetor v V : λ + H (V ) = vol(h ) vol{ w H v V : w + v } Figures 1a and 1b visualize the differene between the inner and the outer hypervolumes in the two dimensional ase: the inner hypervolume orresponds to the spae overed by the lower step funtion, whereas the upper hypervolume orresponds to the spae overed by the upper step funtion. The inner hypervolume is usually used as a measure to ompare effiieny and to ensure diversity in evolutionary multi-objetive algorithms [18]. Espeially for the seond point it is required that the hypervolume an be alulated effiiently. Therefore, several effiient algorithms for hypervolume alulation were proposed in the last years [3, 16, 17]. Given the algorithm for alulating the inner hypervolume λ (V ), the outer hypervolume λ + (V ) an be alulated aording to Algorithm 1. First, the hyperube bounding V is alulated (lines 1 6). Afterwards, the origin of the vetors in V is translated to the extreme point of the bounding hyperube (lines 7 9). Note that the inner 106

4 (a) Absolute inner hypervolume (b) Absolute outer hypervolume 1: Absolute inner and outer hypervolumes Algorithm 1 λ + (V ) Require: Set of n dimensional Pareto-optimal vetors V Ensure: Outer hypervolume λ + of V 1: for all i suh that 1 i n do 2: min[i] = 3: max[i] = 4: for all v V do 5: min[i] = min(min[i], v[i]) 6: max[i] = max(max[i], v[i]) 7: for all v V do 8: for all i suh that 1 i n do 9: v[i] = max[i] v[i] 10: λ + tmp = 1 11: for all i suh that 1 i n do 12: λ + tmp = λ + tmp (max[i] min[i]) 13: λ + (V ) = λ + tmp λ (V ) hypervolume of the translated vetor set orresponds to the spae ontaining all vetors dominating at least one vetor of the initial set V. Finally, λ + (V ) is alulated by subtrating the inner hypervolume of the translated vetor set from the hypervolume of the bounding hyperube (lines 10 13) Weighted perentage hypervolume Based on the absolute hypervolume we define the so-alled weighted perentage hypervolume. For the perentage hypervolume the oordinates of the onsidered vetors are translated to the perentage inrease with respet to the orresponding minimum values of the bounding hyperube. This is neessary for obtaining expressive and omparable results in ase that the oordinate values in some dimensions exhibit signifiant differenes in terms of absolute values (i.e. different orders of magnitude). Additionally, the weighted perentage hypervolume allows to attah importane levels to eah dimension, i.e. the spae overed in one dimension might be onsidered more important than that overed in other dimensions. Definition 7 (Weighted Perentage Hypervolumes) We onsider a n-dimensional hyperube H = [ b 1,b 1 ]... [ bn,b n ] R n + and a set of n-dimensional vetors V = { v 1,..., v m } with i v i = (v i1,...,v in ) H. Given a set of weights W = {w 1,...,w n } with i w i 1 the inner and outer weighted perentage hypervolumes of V are defined as follows: λ H (V,W ) = λ H W (ṼW ) and λ + H (V,W ) = λ+ H W (ṼW ) Thereby: H W = [ 0, f 1 (b 1 ) ]... [ 0, f n (b n ) ], and ṼW = { v 1,..., v m}, with v i = ( f 1 (v i1 ),..., f n (v in )), ( ) where f i (x) = x b i x bi b i b w i 1 i Figure 2a visualizes the absolute inner hypervolumes for two example vetors sets. The orresponding inner perentage hypervolume without weighting is shown in Figure 2b. As we an observe, both vetor sets over exatly the same amount of spae. However, the vetor set represented by the squares overs more spae in x-dimension, whereas the vetor set represented by the diamonds overs more spae in y-dimension. This is refleted by the metri if we apply weighting. Figure 2 visualizes the perentage hypervolumes with weight 3 for the x-dimension. As expeted we observe that the vetor set represented by the squares has a higher weighted perentage hypervolume ompared to the vetor set represented by the diamonds. The other way around, if we assign the weight 3 to the y-dimension we observe, as shown in Figure 2d, that the vetor set represented by the diamonds results in a higher weighted perentage hypervolume. 5.2 Multi-dimensional stati design robustness The stati design robustness (SDR) metri expresses the robustness of a fixed parameter onfiguration for a given onstrained system with respet to a set of ritial system properties. The SDR metri is relevant for the design senario where parameters are defined and fixed early at design time and annot be modified later to reah ompatibility for variants, bug-fixes, and updates. Definition 8 (Multi-dim. stati design robustness) Let S denote the system S with parameter onfiguration and P = {p 1,..., p n } a set of system properties. Given a set of weights W = {w 1,...,w n } with i w i 1, the multi-dimensional stati design robustness of S with respet to P is defined as follows: SDR S (P,W ) = λ sens H S (P (F ) S (P ),W ) Sine it annot be known beforehand to what extent the inluded system properties are subjet to hange, it is desirable that a large number of different possibilities an be sustained by the system. Exatly this is expressed by the SDR metri. The more the system an sustain property value variations, the higher the hypervolume overed by the sensitivity front, and thus the higher the value of the SDR metri. Note that the SDR metri allows to weight the influene of the inluded system properties by attahing weights to eah dimension. This enables the designer to aount for different levels of relevane linked to eah of the onsidered system properties. Criteria for her to assign these weights inlude the estimated probability of future hanges and the impat on the overall system performane. 107

5 (a) Absolute inner hypervolumes for two vetor (b) Perentage inner hypervolumes without sets weighting () Perentage inner hypervolumes with weight (d) Perentage inner hypervolumes with no 3 for x-dimension and no weighting for y- weighting for x-dimension and weight 3 for y- dimension dimension 2: Examples for weighted perentage hypervolume 5.3 Multi-dimensional dynami design robustness While stati design robustness assumes a stati system with fixed parameter onfiguration, dynami design robustness (DDR) inludes potential designer or system ounterations in reation to system property variations. In other words, DDR desribes the robustness potential of a system with respet to the variation of given properties, whih an be ahieved by system reonfiguration, i.e. for instane sheduling parameter adaptation. Consequently, the DDR metri is relevant for a design senario where parameters an be modified during produt life time or in the field. Definition 9 (Multi-dim. dynami design robustness) We onsider a system S and a set of system properties P = {p 1,..., p n }. Given a set of possible parameter onfigurations C for S and a set of weights W = {w 1,...,w n } with i w i 1, the multi-dimensional dynami design robustness of S with respet to P is defined as follows: DDR S,C (P,W ) = λ H ( [ H = [ C C FS sens (P ),W ), where H S (P ) The dynami design robustness depends on the set of possible onfigurations C. As an example, it an be possible to reat to property hanges by adaptation of sheduling parameters or by remapping parts of the appliation. DDR an obviously be used to evaluate dynami systems, but it an more generally be used for the evaluation of the design risk onneted to speifi omponents in a given system. More preisely, already early in the design flow the DDR metri allows the designer to determine boundaries for properties of speifi omponents, allowing their integration into the system. This information effetively failitates feasibility and requirements analysis and greatly assists the designer in pointing out ritial system omponents requiring speial fous during speifiation and implementation. Another usage senario for the DDR metri onerns reonfigurable systems. In suh a senario the designer an use the DDR metri to determine the theoretial robustness head room of ruial system omponents with respet to future hanges of their properties. By early hoosing a system arhiteture offering high DDR values for these ruial omponents the designer an signifiantly inrease system stability and maintainability. 5.4 Robustness gain through reonfiguration Given the formal definition of the stati design robustness (Definition 8) and the dynami design robustness (Definition 9), we an formally derive a metri (Definition 10) expressing the robustness inrease, whih an be ahieved through dynami system reonfiguration ompared to the stati ase, where all parameters remain fixed during the systems lifetime. Definition 10 (Dynami Robustness Gain) We onsider a system S and a set of system properties P = {p 1,..., p n }. Given a set of possible parameter onfigurations C for S and a set of weights W = {w 1,...,w n } with i w i 1, the dynami robustness gain whih an be ahieved through dynami system reonfigurability ompared to the stati ase is defined as follows: G S,C (P,W ) = DDR S,C (P,W ) max C {SDR S (P,W )} Given this metri the benefit in terms of system robustness of designing reonfigurable system omponents is expliitely measurable. Consequently, it an help system arhitets to deide whether or not investing engineering effort into reating reonfiguration mehanisms is worthwhile. More generally, by adjusting the reonfiguration spae C the metri an be used to identify ritial omponents for whih reonfigurability is partiularly advantageous, i.e. leading to a signifiant robustness gain. By this means, engineering effort an be effiiently foused to oneive robust systems. 108

6 6. EXPLORING MULTI-DIMENSIONAL ROBUSTNESS The robustness metris introdued in the previous setion are defined on the sensitivity fronts of one or several system onfigurations. However, the exat alulation of a multi-dimensional sensitivity front is omputationally expensive. In this setion we therefore introdue exploration tehniques for effiiently deriving upper and lower robustness bounds for the stati (Setion 6.2) and the dynami ase (Setion 6.3). The proposed methods are based on stohasti multi-dimensional sensitivity analysis (Setion 6.1). 6.1 Stohasti sensitivity analysis Performing an exat multi-dimensional sensitivity analysis for dimensions greater than 2 is omputationally expensive. The authors of [6] therefore proposed a stohasti sensitivity analysis approah. This stohasti approah formulates multidimensional sensitivity analysis as optimization problem, and is apable of bounding the sought-after sensitivity front from two sides, i.e. oming from the spae of non-working system property ombination and oming from the spae of working system property ombinations. These two bounding fronts are alled bounding working Paretofront and bounding non-working Pareto-front and are defined as follows: Definition 11 (Bounding Pareto-fronts) Let S denote the system S with parameter onfiguration. For a given set of system properties P = {p 1,..., p n } the bounding working and non-working Pareto-fronts are defined as follows: 1. The bounding working Pareto-front FS w (P ) is defined as a set of vetors f 1 w,..., f n w with the following properties: (I.) i f i w H S (P ), (II.) Ω + (FS w (P )) = FS w (P ), (III.) i x FS sens (P ) : x + f i w 2. The bounding non-working Pareto-front FS nw (P ) is defined as a set of vetors f 1 nw,..., f n nw with the following properties: (I.) i f nw i (III.) i x F sens S H S (P ), (II.) Ω (FS nw (P )) = FS nw (P ), (P ) : x f nw During stohasti sensitivity analysis the bounding Paretofronts are onstantly refined using evolutionary exploration tehniques [19]. Thereby, it is always assured that the exat sensitivity front FS sens (P ) lies between the bounding working and the bounding non-working Pareto-fronts FS w (P ) and FS nw (P ). If the stohasti sensitivity analysis runs long enough the bounding Pareto-fronts ultimately onverge to the sought-after sensitivity front. Details about the multi-dimensional sensitivity analysis and the searh spae bounding strategy an be found in [6]. The main reason for applying the stohasti multi-dimensional sensitivity analysis to system robustness optimization is its apability to quikly derive upper and lower system sensitivity bounds with relatively little omputational effort. This property will be used in in the following setions to effiiently approximate the stati and dynami multi-dimensional robustness metris proposed in Setion Stati ase In the stati ase the optimization task onsists in finding the system parameter onfiguration for a given system S that maximizes the multi-dimensional stati design robustness with respet to a set of system properties P weighted aording to W. In order to optimize a system for stati design robustness we propose a nested design spae exploration approah. The outer i exploration loop variates free system parameters (e.g. sheduling parameters), whereas the inner exploration loop evaluates the robustness of eah andidate system parameter onfiguration using the stohasti multi-dimension sensitivity analysis presented in Setion 6.1. Based on the alulated bounding working Pareto-front FS w (P ) and bounding non-working Pareto-front FS nw (P ) we an derive upper and lower bounds for the sought-after stati robustness. Definition 12 (Stati Robustness Bounds) Let S denote the system S with parameter onfiguration and P = {p 1,..., p n } a set of system properties. Given the bounding working and non-working Pareto-fronts F w (P ) as S (P ) and FS nw well as the set of weights W = {w 1,...,w n }, the stati design robustness SDR S (P,W ) is bounded by the minimum guaranteed robustness R S and the maximum possible robustness R + S : R S (P,W ) SDR S (P,W ) < R + S (P,W ), where R S (P,W ) = λ H S (P ) (F w S (P ),W ) R S + (P,W ) = λ nw H S (P (F ) S (P ),W ) It is always guaranteed that the real robustness of the analyzed system onfiguration is ontained in the interval defined by R and R +. Consequently, the preision of the approah an be safely saled. This represents a huge advantage of the presented approah sine even for low approximation qualities, i.e. little invested omputational effort for stohasti sensitivity analysis, the minimum guaranteed and the maximum possible robustness metris are very good indiators for identifying interesting system onfigurations worth to be analyzed in detail. In order to find all system onfigurations with high robustness properties, despite variations in the evaluation results of the stohasti multi-dimensional sensitivity analysis, we propose to perform a two-dimensional Pareto-optimization of the minimum guaranteed robustness R and the maximum possible robustness R + in the outer exploration loop. In so doing, the multidimensional robustness optimization yields on the one hand system onfigurations with large guaranteed robustness and on the other hand system onfigurations with possibly large robustness potential, whih needs to be onfirmed or disapproved by further analysis. It is obvious that using this approah it is suffiient to invest omparatively little omputational effort in the robustness evaluation during exploration to identify interesting system onfigurations, and to postpone their detailed robustness analysis after the exploration proess. 6.3 Dynami ase In order to determine the dynami robustness of a given system we have to integrate the robustness potential of several system onfigurations. In order to do so we propose an iterative exploration approah. During exploration we maintain and update a dynami bounding working Pareto-front and a dynami bounding non-working Paretofront. The dynami Pareto-fronts integrate the information ontained in the bounding Pareto-fronts of eah evaluated system parameter onfiguration. They therefore dynamially bound the spae of working and non-working system property ombinations over all evaluated system onfigurations at any time during exploration. Note that eah suessive exploration iteration refines the approximation of the dynami bounding Pareto-fronts. 109

7 For a given set of system properties P and a set of system parameter onfigurations C the multi-dimensional dynami design robustness of the system S an be approximated using the following iterative design spae exploration approah: Initialization. The initialization phase onsists of the following operations: 1. Initialization of the dynami bounding hyperube H S (P ) = 0 2. Initialization of the dynami bounding working and nonworking Pareto-fronts F S,C w nw = /0 and F S,C = /0 3. Initialization of the sets ontaining the bounding working and non-working Pareto-fronts of all so far evaluated system parameter onfigurations Γ w = /0 and Γ nw = /0 The dynami bounding hyperube an simply be determined by alulating the union set of the bounding hyperubes orresponding to all so far evaluated system onfigurations: H S (P ) = [ H S (P ) evaluated The update of the dynami bounding working Pareto-front is straightforward. It an be determined by simply alulating the Pareto-optimal vetors of the union set of the bounding working Pareto-fronts stored in Γ w : F w S,C = Ω+ ( [ Γ w ) = Ω + ({ x F Γ w : x F }) The alulation of the dynami bounding non-working Paretofront is more involved. Figure 3 shows how the bounding nonworking Pareto-fronts of three onsidered system onfigurations are merged in order to obtain the integrated dynami bounding nonworking Pareto-front. Exploration iteration. During an exploration iteration eah onsidered onfiguration is evaluated using the stohasti multidimensional sensitivity analysis (see Setion 6.1). The resulting bounding Pareto-fronts FS w (P ) and FS nw (P ) are utilized to alulate the fitness value of, and are stored in the sets Γ w and Γ nw, respetively, to update the dynami bounding Pareto-fronts at the end of the urrent exploration iteration. One possibility to assign fitness values to the evaluated system onfigurations is to simply take their robustness potential, i.e. the hypervolumes overed by their bounding Pareto-fronts. However, the robustness of a system onfiguration does not represent a very good metri to ontrol the exploration in the dynami ase. The reason for this is that a high individual design robustness does not automatially mean that the onfiguration ontributes to the dynami design robustness, sine another onfiguration might already over the same property spae. Therefore, we propose to use the improvement of the dynami design robustness, whih is ahieved by a onfiguration, as its fitness value. More preisely, the fitness value of an evaluated system onfiguration is defined as the weighted perentage hypervolume of the spae overed by its bounding working Pareto-front that is not already overed by the dynami bounding working Pareto-front that was alulated after the previous exploration iteration: fitness() = λ H (Ω+ ( F w S,C F w S (P )),W ) λ H S (P ) ( F w S,C,W ), where H = H S (P ) H S (P ) Note that during our iterative exploration approah the dynami bounding working Pareto-front F S,C w, whih represents the baseline of the fitness assignment strategy, is updated after eah iteration. By this means the exploration is adaptively ontrolled and re-foused after eah iteration. This onept allows the exploration to onentrate in eah iteration on areas of the searh spae with the most approximation improvement potential. After eah exploration iteration. After eah exploration iterations the dynami bounding hyperube H S (P ) as well as dynami bounding working and non-working Pareto-fronts F S,C w and F nw S,C are updated. 3: Dynami bounding non-working Pareto-front update In order to orretly integrate the information of the involved bounding non-working Pareto-fronts into the dynami bounding non-working Pareto-front, the latter must over all vetors whih potentially represent a working system property ombination. This is true for all vetors whih are overed by at least one of the onsidered bounding non-working Pareto-fronts. Therefore, we annot simply proeed in the same way as for the dynami bounding working Pareto-front, i.e. building the union set and then eliminating all Pareto-dominated vetors. This fat is visualized in Figure 3. On the left side we see three bounding non-working Paretofronts, whih we want to integrate into the dynami bounding nonworking Pareto-front. On the right side two different methods for deriving an integrated Pareto-front are visualized. The red dotted line onneting all Pareto-optimal vetors (marked with a square) with an upper step funtion represents the Pareto-front we obtain if we pursue the same integration strategy as for the dynami bounding working Pareto-front. However, this strategy is not valid if we ompare the obtained Pareto-front with the real bounding nonworking Pareto-front visualized by the solid red line. In fat, the red dotted line inorretly does not over several regions olored in light gray, whih orresponds to an underestimation of the real dynami bounding non-working Pareto-front. Additionally, it overs the region hathed in light gray, whih orresponds to an overestimation of the real dynami bounding non-working Pareto-front. In order to determine the orret dynami bounding non-working Pareto-front, we need to alulate help vetors. The set of help vetors is equivalent to the set of all Pareto-optimal vetors that Pareto-dominate (in the sense of a maximization problem) no vetor in at least one of the onsidered bounding Pareto-fronts. For the example given in Figure 3 the help vetors are marked with a ross. As we an see, they orretly define the limits of the integrated dynami bounding non-working Pareto-front, and their inner hypervolume orresponds to the volume of the spae it overs. Aordingly, the dynami bounding non-working Pareto-front an be 110

8 alulated as follows: F nw S,C = Ω+ { f H S (P ) F Γ nw : ( x F : f + x)} Dynami robustness bounds. Like in the stati ase the dynami bounding Pareto-fronts F S,C w and F nw S,C an be used to derive bounds for the dynami design robustness of the analyzed system. Definition 13 formally haraterizes the dynami minimum guaranteed robustness R S, representing a lower robustness bound, and the dynami maximum possible robustness R S +, representing a upper robustness bound. Definition 13 (Dynami Robustness Bounds) We onsider a system S and a set of system properties P = {p 1,..., p n }. Given a set of evaluated parameter onfigurations C for S, the dynami bounding working and non-working Pareto-fronts F S,C w and F nw S,C, as well as the set of weights W = {w 1,...,w n }, the dynami design robustness DDR S,C (P,W ) is bounded by the dynami minimum guaranteed robustness R S and the dynami maximum possible robustness R S + : R S (P,W ) DDR S,C (P,W ) < R S + (P,W ), where 7.1 Example system We onsider the example setup shown in Figure 4. It ontains four different omputational units onneted by a bus. Three appliations are mapped on the arhiteture. A video appliation (solid hain) gathers data from a amera ontrolled by the miro ontroller uc, performs preproessing on the DSP, and post-proessing on the PPC ore. The seond appliation (dashed hain) reads data from a sensor, whih is first proessed on the ARM ore and then forwarded to the PPC ore, whih, in turn, ontrols an ator. The third appliation (dotted line) is a streaming appliation that runs on the ARM proessor ore and uses the DSP for data proessing. All three appliations have onstrained end-to-end latenies whih need to be satisfied for the system to funtion orretly (Table 1). The omputational resoures (PPC, uc, DSP, and ARM) are all sheduled aording to the stati priority preemptive poliy, and the interonneting bus is arbitrated by the CAN protool. Core exeution and ore ommuniation times as well as priorities of all tasks and exhanged messages are given in Tables 1a and 1b, respetively. The periods of inoming data at the system inputs (Cam, Sens, and S in ) are speified in Table 1d. R S (P,W ) = λ H S (P ) ( F w S,C,W ) R S + (P,W ) = λ H nw ( F S (P ) S,C,W ) Note that R S represents a real lower dynami robustness bound for the analyzed system, i.e. its real dynami robustness potential is larger or equal to R S. However, R S + only represents an upper dynami robustness bound for the set of system parameter onfigurations C evaluated during robustness approximation. Note that this is not a limitation of our approah, sine generally a hard global upper bound annot be determined without evaluating all possible system parameter onfigurations. It is obvious that the approximation quality of the dynami robustness bounds depends very muh on the number of performed exploration iterations and the preision of the underlying multidimensional stohasti sensitivity analysis. Illustrative example. Figure 5 gives an example for the multidimensional dynami robustness approximation. Figure 5a shows the initial approximations of the dynami bounding working Pareto-fronts. Two new system parameter onfigurations are evaluated in the pending exploration iteration. The approximations of the bounding Pareto-fronts for these two onfigurations, obtained through stohasti multi-dimensional sensitivity analysis, are visualized in Figures 5b and 5d, respetively. The fitness values assigned to the newly evaluated onfigurations, i.e. the weighted perentage hypervolume of the spae overed by their bounding working Pareto-fronts that is not already overed by the dynami bounding working Pareto-front, are visualized in Figures 5 and 5e. Figure 5f shows the updated dynami bounding Pareto-fronts at the end of the performed exploration iteration. 7. CASE STUDY In this setion we apply the robustness optimization tehniques presented in this paper to a small but realisti example system. The system is desribed in Setion 7.1 and in Setion 7.2 we optimize its robustness properties. Channel CCT Priority C0 [10.4, 12.4] 5 C1 [12, 14.4] 3 C2 [20, 26.4] 2 C3 [15.2, 20] 6 C4 [10.4, 14.4] 1 C5 [15.2, 26.4] 4 4: Example system Task CET Priority T5 [30.9, 43.1] 2 T2 [15.8, 27.4] 1 T3 [36.9, 46.3] 2 T0 [20.1, 48.5] 1 T4 [13, 86] 2 T7 [40.3, 44.5] 1 T1 [27.1, 160] 2 T6 [14.2, 63.6] 1 T8 [11.5, 291.2] 3 (a) Core Communiation (b) Core Exeution Times Times Path Deadline System Input Period P Sens At 850 Sens 500 Cam V out 1100 Cam 100 S in S out 1000 S in 1000 () End-to-End Constraints (d) Input Event Models 1: System Parameters 7.2 Robustness optimization We optimize the two-dimensional stati and dynami robustness of the worst-ase ommuniation time of ommuniation hannel C3 and the worst-ase exeution time of task T1. Thereby, we attah a higher importane to the robustness of the task T1 (weight 2) than to the ommuniation hannel C3 (weight 1). Table 2 shows the four relevant system onfigurations that we 111

9 (a) Dynami bounding Pareto-fronts before urrent exploration iteration ated (b) Bounding Pareto-fronts of first evalu- () Fitness of first evaluated onfiguration onfiguration (d) Bounding Pareto-fronts of seond evaluated onfiguration (e) Fitness of seond evaluated onfiguration urrent exploration (f) Dynami bounding Pareto-fronts after iteration 5: Multi-dimensional dynami design robustness approximation example Config. # CAN DSP PPC ARM uc Robustness (C3 weight=1, T1 weight=2) org C4 > C2 > C1 > C5 > C0 > C3 T 7 > T 4 T 2 > T 5 T 6 > T 1 > T 8 T 0 > T C4 > C2 > C5 > C0 > C1 > C3 T 7 > T 4 T 2 > T 5 T 6 > T 1 > T 8 T 3 > T C4 > C5 > C1 > C0 > C2 > C3 T 7 > T 4 T 2 > T 5 T 6 > T 1 > T 8 T 3 > T C0 > C5 > C4 > C2 > C1 > C3 T 7 > T 4 T 2 > T 5 T 6 > T 1 > T 8 T 3 > T C4 > C5 > C0 > C2 > C1 > C3 T 7 > T 4 T 2 > T 5 T 6 > T 1 > T 8 T 0 > T : System onfigurations obtained during robustness optimization (a) Stati design robustness - original and optimized onfigurations (b) Dynami design robustness - inluding () Dynami design robustness - inluding sheduling on CAN bus sheduling on CAN bus and uc 6: Robustness optimization results: stati and dynami ase 112

10 found during robustness optimization annotated with their individual robustness properties. The highest stati design robustness (0.6452) is ahieved by onfiguration 1. Compared to the original onfiguration, whih has a stati robustness of , this represents an inrease of approximately 125% (Figure 6a). If we add dynami parameter reonfiguration we an further inrease system robustness. If we assume that the CAN message Ids (priorities) on the bus an be adapted as a reation to property variations we an inrease system robustness by more than 13% to (Figure 6b). Adding the priorities on uc to the reonfiguration spae leads to a further dynami design robustness inrease to , whih orresponds to a total robustness inrease of approximately 19% ompared to the stati ase (Figure 6). If we apply the robustness gain metri G we obtain the following values for our two different reonfiguration spaes: G S,{CAN} ({C3,T 1},{1,2}) = G S,{CAN,uC} ({C3,T 1},{1,2}) = From these numbers we an onlude that bus reonfigurability is important to ahieve high robustness for C3 and T 1. Also, more surprisingly, a further robustness inrease an be ahieved if uc is reonfigurable. The robustness gain is smaller but still relevant. 8. CONCLUSION In this paper we addressed the problem of system property variations during the design and the maintainane of omplex embedded systems. We presented expressive multi-dimensional robustness metris for different design senarios. The stati design robustness is adapted to the senario where parameters are defined and fixed early at design time and annot be modified later as a reation to system property variations, whereas the dynami design robustness inludes potential ounterations in reation to system property variations. Sine robustness optimization is a omputationally intensive problem, we proposed effiient exploration methods allowing to derive upper and lower robustness bounds with little omputational effort. The onsideration of the proposed multi-dimensional robustness riteria during early phases of the embedded system design flow an signifiantly redue design risk and inrease system stability and maintainability. 9. REFERENCES [1] S. Ali, A. Maiejewski, H. Siegel, and J. Kim. Measuring the robustness of a resoure alloation. IEEE Transations on Parallel and Distributed Systems, 15(7): , July [2] S. Chakraborty, S. Künzli, and L. Thiele. A general framework for analysing system properties in platform-based embedded system designs. In Pro. of the IEEE/ACM Design, Automation and Test in Europe Conferene (DATE), Munih, Germany, [3] M. Fleisher. The measure of pareto optima: Appliations to multi-objetive metaheuristis. Leture Notes in Computer Siene, 2632: , [4] D. Gu, F. Drews, and L. Welh. Robust task alloation for dynami distributed real-time systems subjet to multiple environmental parameters. In Pro. of the 25th IEEE International Conferene on Distributed Computing Systems (ICDCS), Columbus, Ohio, USA, June [5] A. Hamann, R. Rau, and R. Ernst. A formal approah to robustness maximization of omplex heterogeneous embedded systems. In Pro. of the IEEE/ACM/IFIP International Conferene on HW/SW Codesign and System Synthesis (CODES-ISSS), Seoul, South Korea, Otober [6] A. Hamann, R. Rau, and R. Ernst. Multi-dimensional robustness optimization in heterogeneous distributed embedded systems. In Pro. of the 13th IEEE Real-Time and Embedded Tehnology and Appliations Symposium (RTAS), Bellevue, WA, USA, April [7] R. Henia, A. Hamann, M. Jersak, R. Rau, K. Rihter, and R. Ernst. System level performane analysis - the SymTA/S approah. IEE Proeedings Computers and Digital Tehniques, 152(2): , Marh [8] V. Izosimov, P. Pop, P. Eles, and Z. Peng. Design optimization of time- and ost-onstrained fault-tolerant distributed embedded systems. In Pro. of the Design Automation and Test in Europe Conferene (DATE), Munih, Germany, Marh [9] C. Lu, J. Stankovi, S. Son, and G. Tao. Feedbak ontrol real-time sheduling: framework, modeling, and algorithms. Real-Time Systems Journal, 23(1-2):85 126, [10] R. Rau and R. Ernst. Sheduling anomaly detetion and optimization for distributed systems with preemptive task-sets. In 12th IEEE Real-Time and Embedded Tehnology and Appliations Symposium (RTAS), San Jose, USA, April [11] R. Rau, R. Ernst, M. Jersak, and K. Rihter. A virtual platform for arhiteture integration and optimization in automotive ommuniation networks. In Pro. of the SAE World Congress, Detroit, USA, April [12] R. Rau, A. Hamann, and R. Ernst. A formal approah to multi-dimensional sensitivity analysis of embedded real-time systems. In Pro. of the Euromiro Conferene on Real-Time Systems (ECRTS), Dresden, Germany, July [13] R. Rau, M. Jersak, and R. Ernst. Applying sensitivity analysis in real-time distributed systems. In Pro. of the 11th IEEE Real-Time and Embedded Tehnology and Appliations Symposium (RTAS), San Franiso, California, Marh [14] M. Verhoef, E. Wandeler, L. Thiele, and P. Lieverse. System arhiteture evaluation using modular performane analysis - a ase study. In Pro. of the 1st IEEE/ACM International Symposium on Leveraging Appliations of Formal Methods (ISOLA), Pafos, Cyprus, Ot [15] S. Vestal. Fixed-priority sensitivity analysis for linear ompute time models. IEEE Transations on Software Engineering, 20(4), april [16] L. While, P. Hingston, L. Barone, and S. Huband. A faster algorithm for alulating hypervolume. IEEE Transations on Evolutionary Computation, 10(1):29 38, February [17] E. Zitzler. Hypervolume metri alulation:. ftp://ftp.tik.ee.ethz.h/pub/people/zitzler/hypervol., [18] E. Zitzler and S. Künzli. Indiator-based seletion in multiobjetive searh. In Pro. 8th International Conferene on Parallel Problem Solving from Nature (PPSN VIII), volume 3242 of Leture Notes in Computer Siene, Heidelberg, Germany, September Springer. [19] E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the Strength Pareto Evolutionary Algorithm for multiobjetive optimization. In Pro. Evolutionary Methods for Design, Optimisation, and Control, pages , Barelona, Spain,