Recent Researces in Circuits, Systems, Communications and Computers Using data fow anaysis for te reiabiity assessment of safety-critica software systems BÖRCSÖK J., SCHAEFER S. Department of Computer Arcitecture and System Programming University Kasse, Wiemsöer Aee 71, D-34121 Kasse GERMANY j.boercsoek@uni-kasse.de scaefer@uni-kasse.de www.rs.eecs.uni-kasse.de Abstract: Reiabiity anaysis for safety-critica software systems often needs additiona expert knowedge, because of te sma data-sets avaiabe. A Bayesian approac is used to deveop a reiabiity mode based on expert knowedge and sma data-sets. Te expert knowedge is obtained wit data fow anaysis. Certain variabes in te program code are examined to cacuate teir probabiity of causing a faiure. Tese additiona information are incorporated into a suitabe distribution function to be abe cacuate reiabiity caracteristics wit greater precision. Key-Words: software reiabiity, bayesian reiabiity, static anaysis, vaue anaysis, abstract interpretation 1 Introduction 1.1 Motivation Safety-critica software as to be deveoped according to standards and as to be certified. Terefore it is necessary to ave exact cacuations of te required reiabiity parameters. Te usua software reiabiity modes use faiure data-sets to estimate te parameters of te underying mode. Tis metod is often not suitabe for safety-critica software, because of te sma data-sets tat are avaiabe. Safety-critica software as a ig degree of maturity, wen data coection begins, so tat ony a few faiures if any can be recorded. 1.2 Requiring additiona information Terefore it is necessary to deveop a reiabiity mode tat accounts for tis ack of data. A reiabiity mode for safety-critica systems requires terefore additiona information. Te additiona information can be derived from different areas of te software deveopment process. Experience from past projects can be integrated into te mode, data from software projects tat beave simiar or are deveoped under simiar conditions is existent, expert knowedge is avaiabe, tat was generated from code reviews and inspections or te software can be subjected to static anaysis. Because of te ig compexity of modern safety-critica software systems e.g. te source code of modern infusion pumps can be comprised of up to 170,000 ines of code [6], compete code reviews are not feasibe. On te oter and it is not possibe to use data from former or simiar projects, because even sma canges e.g. in te operationa profie makes it impossibe to extrapoate to te project tat is examined. 1.3 Reated Work A compreensive study of software reiabiity modes and te underying matematica structures is given in [7] and [8], atoug te specia conditions of safety-critica software is not taken into considerations in tese works. [9] examines different reiabiity modes on te basis of scarce faiure data from safety-critica space sutte software, but does not incorporate any extra information into te cacuations. Static anaysis is in [1] appied to safety-critica software, but is mosty used to examine certain caracteristics of te software rater tan use it to cacuate its reiabiity. 2 Static anaysis Static anaysis as none of te aforementioned restrictions tat are making it ard to use knowedge from past experience. It can be automaticay appied to a arge code basis and te resuts can processed automaticay. Static anaysis anayses te source code of te system, witout executing it. For ISBN: 978-1-61804-056-5 34
Recent Researces in Circuits, Systems, Communications and Computers our purpose its aim is to verify specific data dependent caracteristics tat can ead to fauts in a running system and subsequenty to faiures. Typica data dependent fauts are: Division by zero, Pointer anaysis, Buffer overruns or Vaue anaysis. Because every non-trivia probem in a compex system is undecidabe it is impossibe to proof te aforementioned caracteristics. Tis often resuts in a tradeoff between fase positives and fase negatives tat as to be baanced. In spite of tis static anaysis yieds usefu resuts. It gives probabiities for occurrence of certain kinds of fauts tat can be incorporated into te reiabiity mode. Wit te use of abstract interpretation it is possibe too derive sound resuts Vaue Anaysis Wit te use of abstract interpretation [2] it is possibe to perform vaue anaysis. Abstract interpretation aows approximating te concrete semantics of a system. Te concrete semantics of a program describes a possibe executions of tis program. It is not possibe to infer directy te required information from te program because a non-trivia caracteristics are undecidabe for te concrete semantics. Te abstract interpretation constructs conservative approximations wit a superset, te so-caed abstract semantics from te concrete semantics, e.g. te abstract information of a certain integer vaue coud be: odd, even, negative or te restriction to an interva. Te advantage of abstract interpretation is due to its soundness, i.e. a property tat is proven in te abstract semantic ods in te concrete semantics, tus avoiding fase positives. In addition abstract interpretation is precise enoug to avoid too many fase negatives Abstract interpretation consists of two functions α and γ. Te function α is caed te abstraction function. It maps te concrete state of a date to an abstract vaue. Te function γ is te concretization function, wic maps te abstract vaue into concrete vaues. Te functions α and γ form a gaois connection: S γ (a) iff α(s) a (1) were S is a concrete state and a is a abstract property, e.g. negative/non-negative. Te gaois connection aows te abstract interpretation to be exact, because if α and γ are monotone (1) yieds: S γ α (S) and α γ (a) a (2) wit te restriction α γ(a)=a for te rigt term of (2) tis gives proof tat an abstract interpretation is exact. Te vaue anaysis tries to compute a possibe vaues for a program variabe for every eement of te concrete semantics wit abstract interpretation. Tere are two basic approaces: 1 Constant propagation: Te exact vaue of te variabe is known or tere is no information at a for tis variabe. 2 Interva anaysis: Te abstract property a is defined as an interva. Te vaue of te variabe is witin te range of tis interva or it is not. [4]) describes a genera framework for interva anaysis. In [10] an impementation for te genera framework is given. Tis impementation is used as an exampe in tis paper to demonstrate te feasibiity of tis approac. An own impementation wit possibe improvements is intended for future work. Te impementation in [10] uses forward and backward propagation to ave more precise resuts, wic means tat te source code is anayzed twice. Te first iteration expores te code from start to end and te second iteration from end to start. An exampe transfer function is given to iustrate te vaue anaysis. An interva for an variabe x is given as <x,x >. Possibe operations on sets are defined, ike unions and intersections, e.g. x, x > < y y >=< max( x, y ),min( x, y ) > (3) <, An addition in te concrete semantics is ten computabe in te abstract domain: x = y+ z S y = y < x z, x z > α( S) z = z < x y, x y > α( S) (4) x = x < y + z, x + z > α( S) Te direction of te arrow describes if forward or backward propagation is computed. Te transfer functions for oter operations in te concrete domain are obtained in a simiar fasion. For every variabe for wic tis is requested a vaue anaysis can be done. Te interva tat is examined is ere te range tat is representabe on te given system. On a 32 bit system te resuting interva for an unsigned integer is <0,2 32 >. If te variabe does not exceed or underruns tis interva in te abstract domain no underfow or underfow is possibe. Tus tis variabe can not cause a faut in any program state. After te static anaysis information is avaiabe wic variabes are safe, i.e. can never cause fauts and wic variabes are unsafe, i.e. tey can cause fauts but do not necessariy cause probems. ISBN: 978-1-61804-056-5 35
Recent Researces in Circuits, Systems, Communications and Computers Usuay it is too expensive to execute vaue anaysis for every program variabe. Te software or reiabiity engineer as to fag te variabes tat are safety critica and ave to be examined. Te resuting information is ten incorporated into te reiabiity mode as prior information, wic is independent from te registered faiure-data. 3 Bayesian reiabiity 3.1 Principes Te common approac to add extra information into a reiabiity mode is troug bayesian reiabiity. Te difference to te cassic frequentist approac is tat te Bayesian approac yieds a confidence interva for te estimated parameters of te reiabiity mode. Tis gives additiona certainty in te cacuated vaues for faiure rate or faiure intensity. Te Bayesian approac is based upon four density functions wit te foowing reation: f ( x λ) λ) g ( λ x) = (5) 0 f ( x λ) λ) dλ were λ x) is te posteriori distribution. Tis is te fundamenta distribution for te software reiabiity mode. Te parameters are x as te coected dat in tis case te recorded faiure times and λ as te estimated parameters of te mode. Te posteriori distribution gives te most ikey parameters given te recorded faiure times. Te prior distribution is given by λ). Tis te initia estimate of te parameters before any faiure times are coected. Te form of te prior distribution is cosen because of te underying constraints or because of matematica convenience. Te prior distribution contains te information from te static anaysis. Te function f(x λ) is caed te ikeiood function. It describes te probabiity of te occurrence of te coected data given te parameter of te mode. Te denominator of equation (5) is caed te margina distribution and it represents a normaizing factor for te posteriori distribution. A compete teoretica background of Bayesian reiabiity is given in [3]. 3.2 Discrete Case In te discrete case te software tat is to be anayzed as ony discrete runs. It is a piece of code tat wi be executed on demand and performs a specific function, e.g. safety measures. It wi ten be suspended unti te next demand of te software arises. Te interesting parameter is te probabiity tat tis software modue wi fai on demand. Te parameters λ and x in equation (5) can ten be interpreted as parameters of bernoui trias p Te parameter p denotes te probabiity tat an event occurs. Here an event is te faiure of te software. Te number of trias tat ave successfu outcomes is represented by k. Te software is tested wit a certain amount of runs n. Te trias were an event, i.e. a faiure, as occurred are recorded and can be used to directy estimate p or to use tese parameters as input for te ikeiood function. For te atter case te ikeiood function is regarded as a binomia distribution n k n k f ( p k) = Pr( K = k) = p (1 (6) k Figure 1 sows two binomia distributions wit different probabiities. Te direct consideration of te number of runs of te software is often not possibe, because for safety-critica software tere is not enoug data avaiabe. Due to te fact tat for safety-critica software ony a imited number of faiure times or even no faiure times at a can be measured it is not possibe to infer reiabe mode parameters from a direct maximum ikeiood procedure. Furtermore a iger precision of te parameters is expected wit te use of prior information. Fig. 1: Two exampes of a binomia distribution 3.3 Continuous case Te second approac considers te run of te software as continuous, i.e. tere are no competed runs as in te previous section. Tat aows for a time dependent anaysis so tat time dependent ISBN: 978-1-61804-056-5 36
Recent Researces in Circuits, Systems, Communications and Computers reiabiity caracteristics ike MTTF, faiure rate or faiure intensity can be cacuated. A distribution tat is often used because of its fexibiity and it s matematicay tractabiity is te gamma distribution Ga( as a prior distribution: a b a 1 bλ λ) = λ; b= ) = λ e (7) a) Te parameters of equation (7) are a te so caed sape parameter and b te rate parameter, and te function Г(a) denotes te gamma function. Te fexibiity stems from te fact tat, wit te gamma distribution it is possibe to mode increasing, decreasing and constant faiure rates. For a=1 te gamma distribution becomes te exponentia distribution wic is used to mode constant faiure rates. If te sape parameter is a<1 te resuting faiure rates are decreasing and for a>1 te faiures are increasing. Fig. 2: Four different densities of te gamma distribution Te parameters a and b ave to be estimated based on te vaue anaysis. In order to do tat an assumption, regarding te software ife cyce, to determine parameter a as to be made. If every error tat caused an faiure is found and fixed witout introducing new errors or if te error-introducing rate is smaer tan te fixing rate te faiure rate of te software over its ife time is decreasing and terefore te sape parameter as to be smaer tan 1. Accordingy if te software underies software entropy te code becomes ess reiabe and tis resuts in increasing faiure rates and a>1. If te faiures are ony recorded, but no error-fixing are performed te code basis is uncanged and a=1 wit a constant faiure rate. [5] Te parameter b represents te rate of te distribution. As an initia estimate te ratio of unsafe to safe variabes can be used. Te practicaity of tis ratio can be improved by using te number of executions of unsafe and safe variabes as underying ratio. Since not every execution of an unsafe variabe eads to an erroneous program state, it is necessary to use a proportionaity factor tat refects te probabiity tat an execution of an unsafe variabe resuts in an error. Tis proportionaity factor can be found troug empirica data or wit expert knowedge. 4 Appication 4.1 Appication of te discrete case Te vaue anaysis gives information about a fraction of variabes in te code tat can cause errors and subsequenty faiures, i.e. every time a variabe tat was fagged unsafe is executed tere is a sma probabiity of a faiure. Tis beavior can be used to mode te prior distribution of equation (5). For te form of te prior distribution usuay a distribution is cosen tat beongs to te same famiy as te posterior distribution. Te prior distribution is ten caed a conjugate prior in respect to te ikeiood. Te beta distribution can be used as a conjugate prior for te binomia distributed ikeiood function in equation (6): a 1 b 1 g ( = Be( = p (1 (8) a) Te parameters a and b of tis distribution are cosen as te ratio of unsafe v u to overa variabes v a and vice versa. Te rationae beind tis is as foows. Te ratio a=v u /v a describes te initia trust into te system on te basis of te static anaysis. Even wit no actua run of te software te probabiity of faiure is sma wen tere are ony a sma number of unsafe variabes, wic coud cause teses faiures, in comparison to te overa number of variabes. Te second parameter is used in a simiar fasion. Te ratio b=v a /v u describes te statistica spread of te distribution. Tis ratio is supposed to be arge, wic in turn makes te spread smaer. Te fact tat bot parameters are dependent of eac oter can be negected, because tis specific prior distribution is cosen due to it s matematicay convenience and ony one parameter is needed. Equations (6) and (8) are used to derive te posterior distribution. Ten equation (5) becomes: ISBN: 978-1-61804-056-5 37
Recent Researces in Circuits, Systems, Communications and Computers p b, k) = 1 0 n p k n k p k k (1 (1 n k n k Be( Be( dp (9) After integration te denominator in (9) gives te beta-binomia distribution and wit transformation and simpifying tis resuts in: k+ a 1 b k 1 p x') = p (1 k+ a) b k)) (10) p x') = Be(( k+ a),( b k)) wit x as te set of parameters (b,k). 4.1.1 Exampe On-demand software is tested 100 times. Two faiures are recorded. Te resuting MLE for te corresponding binomia distribution yieds p=k/n=0.02. Te mean and te standard deviation of te cassica approac to estimate p are µ=2 and σ 2 =1.4. Te software consists of 1000 variabes tat are anayzed, for wic 10 of tese variabes are fagged as potentiay unsafe. Te resuting initia prior Beta distribution = Be(0.01,100). For te posteriori distribution foows p x )=Be(2.01,198). Te mean and te standard deviation for te bayesian approac wit static anaysis are µ=0.01 and σ 2 =0.07. 4.1.2 Specia Case If te on-demand software is tested 100 times and no faiure is recorded it is not possibe to cacuate a meaningfu probabiity for te binomia distribution, because p =0 and terefore µ=0 and σ 2 =0, wic impies error-free software wic is not reaistic assumption. If te bayesian metod is used wit te same prior distribution as in te above section te resuting distribution as meaningfu parameters µ=0.00005 and σ 2 =0.005, wic are usabe to determine reiabiity caracteristics. 4.2 Appication of te continuous case In te continuous case te prior distribution is modeed by te gamma distribution. Te gamma distribution can act as te conjugate prior for different ikeiood functions. A widey used distribution in reiabiity engineering is te exponentia function f Ex(λ) = λe (-λt). Te use of te gamma distribution Ga( as prior and of te exponentia Ex(λ) as ikeiood yieds te foowing gamma distributed posteriori distribution: n ( b+ T ) λ b, n, T) = λ n) λ b, n, T) = Ga( n, b+ T) e r 1 ( b+ T ) λ (11) were n and T consists of te information tat comes from testing te software, n is te number of faiures tat are recorded and T is tota test time or te sum of faiure times t i. Te parameter a of te prior distribution is estabised troug te assumption on te software ife-cyce. An increasing faiure rate is assumed wit a=1.5. Parameter b takes te vaue of te ratio of te safe and unsafe variabes tat are anayzed in te vaue anaysis step. 4.2.1 Exampe Te exampe of te section 4.1.1 is used again, but te software is tested for 100 ours, instead of 100 trias. Two faiures are recorded at 30 ours and 60 ours. If te exponentia mode is used directy wit n= 2 and T= 30+60, te maximum ikeiood estimate for λ is n/t=0.022. Tis can be interpreted as te faiure rate of te system. Te mean, wic is te mean time to faiure (MTTF) and te standard deviation are µ=mttf=45 and σ 2 =45. For te Bayesian approac te number of anayzed variabes tat are safe is 1000 and 10 unsafe variabes are found. Te resuting posterior distribution is Ga(3.5,190), wit MTTF= 665 and σ 2 =0.0098. 4.2.2 Specia Case Anaog to te discrete case tere is no possibiity to cacuate meaningfu vaues for te MTTF for te exponentia distribution if tere are no faiures recorded. Te prior distribution can be used to estimate parameter, wic can ten be refined wit te ikeiood function even wen no furter faiures occur. 5 Concusion Te exampe cacuations for te discrete and te continuous case sow promising resuts. In bot cases te estimated reiabiity is arger witin te bayesian approac. Te trust in tese resuts is furtermore increased as te standard deviations are smaer, wen using te bayesian metod wit static anaysis. Te cassic metods fai to produce usefu reiabiity measurements wen no faiure data is recorded, wic is often te case, wen testing software for safety critica systems. Te vaue anaysis aows making reasonabe estimates for ISBN: 978-1-61804-056-5 38
Recent Researces in Circuits, Systems, Communications and Computers prior distributions so tat usefu reiabiity caracteristics can be cacuated. 6 Future Work Te teoretica considerations ave to be verified wit rea systems. It is necessary to investigate faiure data tat stems from safety-critica software and to compare tese wit te syntetic resuts. For tis purpose an impementation of te vaue anaysis is necessary tat can anayze te software in an efficient and precise way. Wit te acieved resuts te parameter of te prior distributions can be refined to gain more precise reiabiity caracteristics. References [1] Bancet B. et a. 2003. A static anayzer for arge safety-critica software, Proceedings of te ACM SIGPLAN 2003 conference on Programming anguage design and impementation, June 09-11, 2003, San Diego: ACM [2] Cousot, P. 1996. Abstract interpretation. ACM Computing Surveys Voume 28 Issue 2: 324-328, New York: ACM [3] Hamad M.S. et. A. 2008. Bayesian Reiabiity, New York: Springer [4] Harrison, W.H. 1977. Compier Anaysis of te Vaue Ranges for Variabes, IEEE Transactions on Software Engineering arcive Voume 3 Issue 3: 243-250, May 1977 Piscataway: IEEE Press [5] Jacobson, I. et. A., 1993. Object-oriented software engineering : a use case driven approac Wokingam Addison-Wesey [6] Jones, P. et a. 2010. A Forma Metodsbased Verification Approac to Medica Device Software Anaysis. Embedded.com, February 2010. [7] Lyu, M.R. 1996. Handbook of Software Reiabiity Engineering. New York: McGraw-Hi. [8] Pam, H. 2006. System Software Reiabiity. London: Springer [9] Scneidewind, N., 2008 Comparison of Reiabiity and Testing Modes. Reiabiity, IEEE Transactions on Reiabiity Voume 57 Issue 4: 607 615 Dec. 2008 [10] Stepenson, M. 2000. Bitwise: Optimizing Bitwidts Using Data-Range Propagation, Master Tesis, Massacusetts Institute of Tecnoogy ISBN: 978-1-61804-056-5 39