Software Reliability Assessment Using High-Order Markov Chains

Transcription

1 Internatonal Journal of Engneerng Scence Inventon ISSN (Onlne): , ISSN (Prnt): Volume 3 Issue 7ǁ July 2014 ǁ PP Software Relablty Assessment Usng Hgh-Order Markov Chans Vtaly Yakovyna, Dmytro Fedasyuk, Oksana Nytrebych, Iur Parfenyuk, Volodymyr Matselyukh Software department, Lvv Polytechnc Natonal Unversty, Ukrane ABSTRACT : The assumpton of ndependent executon of the component n the software relablty predcton models usng component-based approach s a smplfcaton of the real work of the software. In ths paper the method of software relablty predcton that takes nto account dependences between software components s descrbed. It uses hgher-order Markov processes and conssts of the followng steps: defnton of each component s falure rate, probablty of transtons between components, the optmal order of Markov process and computaton of the whole software system relablty tself. KEYWORDS : hgher-order Markov chan, software usage model, transton probablty matrx, falure ntensty, software relablty. I. INTRODUCTION Nowadays the mpact of computer system falures on socety has dfferent consequences from mnor nconvenences (malfunctonng applances) to the loss of human lfe (a refusal of flght control system or a falure n the medcal software). Therefore, t s mportant to determne the software relablty that not only provdes customers more confdence n the product, but also can reduce the cost of testng. In ths paper, accordng to [1], relablty means the probablty of software executon durng specfed perod of tme under partcular crcumstances wthout any falures. Many models for the software relablty analyss exst nowadays. They are usually splt nto "whte box" and "black box" models [2] dependng on the use of nformaton about the archtecture of software. It s clear that the "whte box" model can descrbe the software relablty more adequately as they evaluate the nternal structure of software and nteracton of ts components. Therefore ths type of model s also called dfferently models whch are based on an archtectural approach. These models are dvded nto addtve models, path-based and component-based models [3]. Component-based approach for software relablty assessment and predcton usng Control Flow graph to descrbe the software archtecture [2, 3, 4] s the most common. Analyss of the relablty of ths approach s done usng software Markov usage model [5, 6], whch allows to avod general assumptons about falure ntensty dstrbutons and ncorporate both the operatonal profle and test coverage explctly and automatcally nto relablty. Dsadvantage of models whch use component-based approach s usage of the frst-order Markov chan for software relablty modelng [1-4]. Such models smply don t take nto account the nterdependence of software components executon, whch s a usual case n real software systems. To count the mutual dependency of components executon n software usage model t s proposed to use hgher-order Markov chans (HOMC). Ths paper descrbes a method of relablty analyss of software based on hgher order Markov chans n order to mprove the adequacy of software relablty predcton. II. MODEL OF SOFTWARE RELIABILITY EVALUATION BASED ON HIGHER ORDER MARKOV CHAINS As mentoned, the usage of hgher order Markov process wll allow to assess the software relablty more accurately. Based on the Gochale s evaluaton model of software relablty [7], the relablty of the whole system s calculated as: R n l 1 R l. (1) l R relablty of each component, n number of software components. Use of hgher-order Markov process (let N - the order of the model) n ths model wll result n next defnton of each component s relablty: 1 Page

2 Software Relablty Assessment Usng Hgh V t j kl j kl l ( t ) d t 0 R e (2) l, To fnd V - the expected number of vsts to component l dependng on executon of prevous N j.. kl components, a system of lnear equatons must be solved: j... kl n 1 V q V p, (3) j.. kl j.. kl j... k j... kl 1 p - probablty of transton to component l dependng on executon of prevous N components; q - ntal j... k probablty vector; t - executon tme of component l whch depends on executon of prevous N j... kl components. Gven the numercal values of all model parameters, the relablty of each component usng formula (2) and the value of relablty of the whole software system (1) can be calculated. Ths model s herarchcal, because ntally archtectural model parameters are calculated and then the behavor of falures of each component s taken nto account to assess the software relablty. Obvously the usage of HOMC ntroduces other mportant subtasks to be solved: Defnton of falure rate (t ) of each software component; Defnton of the transton probablty matrx; Determnaton of the optmal order of Markov chan. Approaches to resolve these ssues are presented n the next sectons. III. DETERMINATION OF THE FAILURE INTENSITY OF SOFTWARE COMPONENTS Famous software relablty models can be used to calculate the falure rate of each component. Most of these models are based on a nonhomogeneous Posson process. Many exstng models of software relablty can be descrbed wthn the nhomogeneous Posson process [8-9]. Numerous emprcal researches confrm the valdty of the models of ths type [10]. Some models of the nhomogeneous Posson process descrbe the exponental ncrease of relablty, whle others show the S-shaped growth, dependng on the nature of the phenomenon of relablty growth durng testng [11]. The appearance of S-curve s explaned wth the varous factors and lterature contans suffcent amount of models based on ths form. In lterature references S-lke growth s assocated wth dfferent reasons: the nterdependence of software defects [11], less effcent testng on ntal stages n comparson wth followng [12] and so on. To ncrease the degree of adequacy of exstng software relablty models based on a nonhomogeneous Posson process and to take nto account the mpact of model complexty on the behavor of software falure flow parameter a model of software relablty wth complexty ndex [13] s constructed and analyzed. In ths model, flow falures parameter functon s as below: 1 ( ) s s t t e x p ( t ), (4) coeffcent whch shows general amount of falures n the -th software component; coeffcent whch shows general duraton of testng process n the -th software component, s- software complexty ndex, whch generalzes S-shaped models. Obvously, for calculaton (t ) of each component usng ths model t s necessary to make a test of software system to obtan statstcs of falures and ther dstrbuton n tme. For hgher data accuracy testng should be performed wth the maxmum code coverage, whch wll allow dentfyng more falures. To ensure ths condton, the method of automated test scrpts constructon usng ether "whte" or "black box" strateges [14] based on a new varable state-based software usage model based on ts varables [15], has been developed. Varable state-based software usage model, whch s an nput parameter for the automated tests constructon, s shown as a drected graph G = {S, P}, where S = { S, S,..., S components S, P set of transtons between components. Each node of graph 0 1 n } set of software S s the set { V, used V, V change error }, where V set of varables and the correspondng equvalence classes used n the used 2 Page

3 Software Relablty Assessment Usng Hgh component S, lst of varables that change n the component S ; V change lst of varables that change n the component S (ths set s the unon of the set of varables that can be changed by the user and tested wth "black box" method, and set of varables that can be changed only by the nternal logc of the program and tested only wth "whte" box testng); V error the lst of varables and correspondng ncorrect equvalence classes, whch can cause falures n the software system. The mportance of consderaton of the usage of the software varables durng usage model constructon, and constructon of test cases based on ths model, follows from the fact that many software relablty analyss models use software metrcs, whch n turn take nto account the use of software varables. It should be noted that the proposed test generaton method [14] s teratve: frstly test sute wthout nformaton about the falure s constructed, and then after ths set run on the software and gettng the nformaton on detected falure, the next set of tests s bult untl falure wll not be fxed. Ths method of automated constructon software test cases provdes unform code coverage and mproves the effcency of the testng process, reducng the tme, fnancal and human resources. IV. DETERMINATION OF TRANSITION PROBABILITIES BETWEEN COMPONENTS Determnaton of transton probabltes between components s a dffcult task today for mplementaton. One way of solvng ths problem s approach, based on results of software executon montorng [16], snce durng software executon control s passed between components, and havng the results of the transmsson management the transton rates between the components can be determned. To determne the transton probabltes to the component C j from C t s necessary to determne the rato of the number of transtons from C to C j component to the total number of transtons from component C. To test the proposed approach, the authors developed logger on Java. Desgned logger lets to explore the software wthout makng any changes to the code. Ths approach makes t possble to defne and refne the matrx of transton probabltes between software components even at the stage of mplementaton and actual usage, wth consderaton of real software usage scenaros by dfferent classes of users. In addton to software relablty predcton on the early stages of the software lfe cycle, whch allows avodng costly fxng of falures on the later stages, determnaton of transton probabltes matrx can be done by analyzng the UML dagrams [17]. Transton probabltes matrx, more precsely ts structure, can be created from the class dagram, but t should be noted that t s mpossble to obtan transton probabltes and tme spent n the components. Usng use-case dagrams based on all use cases executon probabltes (or relatve frequences) the probablty of executon of software components and transtons between them can be estmated. Besdes, usng UML Sequence dagram, we can get the number of transtons between the components and then easly determne the transton probabltes matrx between them and the tme spent n the component. After analyzng UML Sequence Dagrams developed durng the software desgn t s requred to calculate the total number of transtons between components and form a transton probabltes matrx accordng to the approach descrbed above [16]. But t must be remembered that at ths stage of the lfe cycle (desgn phase) software components desgn can only be approxmated, and mentoned probabltes are pror and based on expert estmates. Another way to obtan a matrx of probabltes of transtons between the components s the use of hdden Markov chans. As a result of ther capabltes, hdden Markov models (HMMs) become more and more frequently used n modelng. Examples nclude, eg, the detecton of ntruson nto software systems, fault dagnoss, network traffc modelng, estmaton and control, speech recognton, part-of-speech taggng and genetc sequence analyss applcatons. There are three fundamental questons we mght ask of an HMM. What s the probablty of an observed sequence? What s the most lkely seres of states to generate the observatons. And how can we learn values for the HMM's parameters P and B gven some data? Where P s transtons probabltes matrx and matrx B encodes the probablty of hdden states generatng output gven that the state at the correspondng tme was s j [18]. Last and the most needed for us queston s reached by usng Baum- Welch algorthm [19]. Notes on the Baum-Welch algorthm. Intal pont for the Baum-Welch algorthm s a completely ntalzed HMM. Ths means that the number of states, number of observaton symbols, transton probabltes, ntal probabltes and observaton probabltes need to be defned. The algorthm then teratvely mproves the model s parameters λ = (P, B, q) untl a (local) maxmum n sequence lkelhood s reached. In each M-step, the expectaton values of the prevous E-step are used and vce versa. Several propertes of the algorthm can be derved from that: The number of states and the sze of the alphabet are not changed by the algorthm. k 3 Page

4 Software Relablty Assessment Usng Hgh The model structure s not altered durng the tranng process: f the res no transton from state s to s j ( p j 0 ), the Baum-Welch algorthm wll never change ths. Intalzaton should explot as much a-pror knowledge as possble. If ths s not possble, random ntalzaton can be used [20]. Therefore, to get transton probabltes matrx we need to swtch to the hdden Markov model where executon of one or other system component can be a hdden state, and possble observatons - stop respondng, or the ssuance program falures, system crashes at a certan stage of ts mplementaton. To run the Baum- Welsh algorthm, we need to ntalze the transton matrx, the matrx of observatons and ntal state vector of hdden Markov models and t need to have a sequence of system observatons. On the output of the algorthm the ntal probablty vector q and the transton probabltes matrx P, whch can then be used n the dstrct (3) to calculate the expected number of vsts to component dependng on vsts to prevous N components, are receved. V. DETERMINATION OF MARKOV CHAIN ORDER Another problem to be solved n order to apply the hgher-order Markov chans n the analyss of software relablty s to determne the optmal order of Markov process, whch depends on the software system under research, n other words on the modeled object. To determne the optmal order of Markov processes n the software relablty assessment tasks t s proposed to use nformaton crtera, namely crtera wth penalty for complexty, because they are not tests of hypothess and do not use the sgnfcance level [21]. In [22] cases of applcaton of these crtera based on the number of observed data samples are shown. Thus, when the sze of emprcal data set s small, BIC crteron should be used for the software relablty behavor assessment, snce t uses stronger penaltes even when the sample sze n> 8 (ln (n) k> 2k), whch should allow to avod unwarranted ncrease n the optmal model order due to nsuffcent emprcal data and ncreases the probablty of selectng "exact" model (assumng the exstence of such model). When the set of emprcal data s large, crtera selecton queston remans open and requres expermental researches as AIC on the one hand does not provde for "exact" model, and estmates the degree of proxmty of the set of test patterns to the "true" model, on the other hand the crteron BIC ncreases the probablty of selectng "exact" model (assumng the exstence of the such model) wth set sze growth. VI. ESTIMATION OF SOFTWARE RELIABILITY BASED ON HIGHER ORDER MARKOV CHAINS Thus, the process of software relablty analyss usng HOMC that allows to count the mutual dependences between component executons and respectvely model the process of software relablty analyss more adequately, conssts of the followng two steps: 1. Defnton of the nput parameters of the model for evaluatng the software relablty, that n turn has a number of sub-tasks; 1.1. Determnaton of the falure rate of each component usng "black box" models (proposed to use software relablty model wth a dynamc ndex of project complexty); 1.2. Calculaton the transton probabltes matrx and components executon tme (usng the logger, UML dagrams, Hdden Markov Process) Obtanng of optmal order Markov process (dependng on the sze of the data sample used crtera for AIC or BIC-famly). 2. Evaluaton of software relablty usng MCHO. Generalzed scheme of ths method s presented n Fgure Page

5 Software Relablty Assessment Usng Hgh Start Falure ntensty evaluaton of software components Logger Probablty transton matrx calculaton UML dagram Hdden Markov process Determnaton of optmal Markov process order Large sze of emprcal data sample + AIC crtera Software relablty assessment usng mproved Gochale model - BIC crtera End Fg. 1. Schema of method for software relablty analyss based on HOMC. VII. CONCLUSION The artcle descrbes the usage of hgher order Markov chans for the problem of software relablty modelng that allows consderaton of the mutual dependences n software components executon, as due to the complexty of modern software archtecture and the set of t s usage scenaros, the assumpton of ndependence of the software component executon s not always true. As a result - the method for the analyss of software relablty based on HOMC has been proposed and all of ts subtasks wth possble solutons have been descrbed. Namely, the software relablty model wth a dynamc ndex of project complexty s consdered to obtan falure rate of each component; an example of the logger, UML dagrams, Hdden Markov Process to calculate the transton probabltes matrx have been provded; dependng on the sze of the data set crtera of AIC or BIC-famly are beng used to determne the optmal order Markov process; software relablty estmaton usng HOMC. REFERENCES [1]. W. Burkhart, Z. Fatha, Testng Software and Systems (23rd Ifp Wg 6.1 Internatonal Conference, 2011, 236). [2]. K. Goseva-Popstojanova, A.P. Mathur, K.S. Trved, Comparson of archtecture based software relablty models,12th Internatonal Symposum on Software Relablty Engneerng,2001, [3]. K. Goševa-Popstojanova, S. Trved, Archtecture-based approach to relablty assessment of software systems, Performance Evaluaton,4, 2001, [4]. Thanh-Trung Pham, Xaver Defago, Relablty Predcton for Component-based Systems: Incorporatng Error Propagaton Analyss and Dfferent Executon Models,12th Internatonal Conference on Qualty Software, 2012, [5]. Heko Kozolek, Operatonal Profles for Software Relablty, Dependablty Engneerng,2, 2006, [6]. S. Jungmayr, J. Stumpe, Another motvaton for usage models: generaton of user documentaton, CONQUEST'98, Nüremberg, Germany, [7]. S. Gokhale, W. Wong, J. Horgan, S. Kshor, An analytcal approach to archtecture-based software performance relablty predcton, Performance Evaluaton, 58(4), 2004, [8]. A.L. Goel, K. Okumoto, Tme-Dependent Error-Detecton Rate Model for Software and other Performance Measures, IEEE Transactons on Relablty, 28(3), 1979, [9]. J.D. Musa, A theory of software relablty and ts applcaton, IEEE Transactons on Software Engneerng, 1(3), 1975, [10]. A. Wood, Predctng software relablty, Computer, 29 (11), 1996, [11]. S. Yamada, M. Ohba, S. Osak, S-shaped relablty growth modelng for software error detecton, IEEE Transactons on Relablty, 32(5), 1983, [12]. C. Rahman, A. Azadmanesh, Explotaton of Quanttatve Approaches to Software relablty (Unversty of Nebraska at Omaha, 2008, 32). [13]. Y. Chabanyuk, V. Yakovyna, D. Fedasyuk, М. Senv, U. Hmka, Constructon and research of software relablty model wth project sze ndex, Software Engneerng, 1, 2010, (n Ukranan) [14]. D. Fedasyuk, V. Yakovyna, P. Serdyuk, О. Nytrebych, The method of software test case constructon based on the analyss of t s varables, Informaton Technology and Computer Engneerng, 2014 (n Ukranan, to be publshed). [15]. D. Fedasyuk, V. Yakovyna, P. Serdyuk, О. Nytrebych, Varable state-based software usage-model based on ts varables, Econtechmod : an nternatonal quarterly journal on economcs n technology, new technologes and modellng processes, 2014 (to be publshed). 5 Page

6 Software Relablty Assessment Usng Hgh [16]. V. Yakovyna, I.Parfenuk, Determnaton of transton probabltes between software components, wrtten n java, based on montorng of ts executon, Proceedngs of the XIIth Internatonal Conference "The Experence of Desgnng and Applcaton of CAD Systems n Mcroelectroncs", 2013, p [17]. V. Yakovyna, I. Parfenuk, Evaluaton matrx transton probabltes between software components based on UML Use Case Dagram, Proceedngs of the IXth Internatonal Conference PERSPECTIVE TECHNOLOGIES AND METHODS IN MEMS DESIGN, 2013, p [18]. D. Ramage, Hdden Markov Models Fundamentals, Stanford Unversty CS229 Secton Notes, 2007, [19]. S. Tu, Dervaton of Baum-Welch Algorthm for Hdden Markov Models, [20]. Felx Salfner. Event-based Falure Predcton: An Extended Hdden Markov Model Approach. dssertaton.de - Verlag m Internet GmbH, Berln, Germany, (Avalable as PDF). [21]. P.Burnham, D. Anderson, Model Selecton and Mult model Inference: A Practcal Informaton-Theoretc Approach (Sprnger, 2002, 488). [22]. V. Yakovyna, D. Fedasyuk, O.Nytrebych, The analyss of nformaton crtera usage n software relablty assessment models, Proceedngs of the Natonal Techncal Unversty "KPI", 2014, (n Ukranan) 6 Page