development speeds up. The shorter development time results in reduced costs. The extensibility and resolvability

Transcription

1 Analyss of Software Relablty Growth Models for Quanttatve Evaluaton of Software Relablty and Goodness of Ftness Metrcs Harmnder Pal Sngh Dham 1, Vabhav Bansal 2 1, 2 Department Computer Scence & Engneerng, OPJS Unversty, Churu, Rajasthan, Inda Abstract: Tll now there have been many Software Relablty models developed for assessng the relablty of software product. Most of these are based upon past falure data gathered durng the testng phase. These models have been utlzed to evaluate the qualty of the software and for future predcaton of relablty. They have been used n many crtcal management decson mang problems that occur durng the testng phase. But none of these models can clam to be the best and hence there s a need for further research. In ths paper dfferent modelng approaches are brefly studed and we present procedures to estmate the parameters of SRGMs and a crtcal analyss of Goodness of Ft usng some exstng Software Relablty Growth Models. On the bass of our observatons we recommend the methodologes for estmatng parameters of SGRM and the varous metrcs used for comparson of Goodness of Ft and predctve valdty. Keywords: SRGM, SRE, NHPP, Mean Value Functon, Parameter Estmaton, Falure Intensty Rate, MSE, AIC, R 2, MLE, LSE, RPE 1. Introducton development speeds up. The shorter development tme results n reduced costs. The extensblty and resolvablty Software Relablty Engneerng (SRE) s a conventonal of software systems s mproved, because components can area of software engneerng research and s concerned wth flexbly be substtuted by another component that satsfes the mprovement and measurement of relablty. SRE wors the requrements. Software components enhance the by quanttatvely characterzng and applyng two thngs relablty of the software, as they are mproved, tested and about the product, the projected relatve use of ts functons debugged over years [2]. and ts requred major qualty characterstcs. The major qualty characterstcs are relablty, avalablty, delvery Software Engneerng Methodologes consttute the date and lfe-cycle cost. IEEE has suggested the followng framewor that gudes software developers n optmally standard defnton of a fault (may be referred as a defect) developng the software systems. These framewors defne [29]. A fault occurs when a user maes a mstae, called an the dfferent phases of software development. The selecton error, whle performng some software actvty. Many of whch methodology to apply n a specfc development desgners do not dstngush between faults, errors or bugs, process s closely related to the sze, complexty, relablty as ther effects are the same, the dreaded falures. A falure s and mantanablty of the software, and to the envronment a departure from the system's requred behavor and t can t s supposed to functon. Now a days the fuson of all these occur any tme n a system delver, durng testng, or methodologes s ncorporated. All the developers loo at the operaton or mantenance. Few faults may never turn nto low cost & rs, hgh qualty and small cycle of tme, so that falures, f faulty code s never executed or a partcular state the productvty and qualty of the product can be optmzed. never occurs. Nevertheless software cannot be made fault There should be a tradeoff between the development tme free and these faults certanly lead to falures. Ths demands and the qualty of the product [1]. applcaton of Software Engneerng tools, technques and procedures, n an effectve way. The ablty and qualty of software depends greatly on ts relablty. The relablty of any software reles on testng The basc am of software engneerng s to produce hgh phase. Whle executon, f a system fals, t mples that there qualty effcent software at low cost. Wth growth n sze are faults nherent n the system. Ths reduces the relablty and complexty of software, management ssues began of the software system. Software Relablty mproves as domnatng. An optmal desgn strategy wthout any faults are detected and corrected [4]. Therefore, the testng compromses e.g. cost & tme, for the system does not phase emphaszes on detectng and correctng the faults n develop an optmal desgn. The reason for ths s the changes the software. Pham [5] revewed and compared varous n requrements that may occur n later development cycles. NHPP based SRGMs on ther ft and predctve power. The Such changes may cause desgn decsons taen earler to be comparson of SRGMs on defect nflow data s studed by less optmal. Desgn eroson s nevtable wth the current Wood [6] and he found t correlated wth past released way of developng software. Refned methods only defects. Even though a number of studes have compared contrbute by delayng the moment that a system needs to be and evaluated SRGM wthn dfferent context. We are stll wthdrawn or retred. These approaches do not address the not capable to mae a consensus on how to choose SRGMs fundamental problems that cause desgn eroson and maes for specfed purpose and whch models are best for gven system unrelable [3]. Component based desgn s expected process characterstcs. to have a strong mpact on the qualty of software development: Due to the smplcty, the software 1540

2 2. Software Relablty model ncorporatng mperfect debuggng and fault ntroducton wth upper bound on the number of faults. Software Relablty s one of the man features of software qualty. As the defnton of relablty s user orented, t has The Marov model s analysed n order to measure the become the fundamental qualty attrbute of any product, be probablty of beng n a gven state at a gven pont of tme, t software or hardware. Software Relablty models can be the amount of tme a system s expected to spend n a gven classfed n a number of ways. For the analyss normally state, as well as the expected number of transtons between stochastc software relablty models are used. They model states: for nstance representng the number of falures and the falure process of the software and use other software repars. falure data as a bass for parameter estmaton. The models are capable (a) to estmate the current relablty and (b) to Marov models provde great flexblty n modelng the predct future falure behavor. tmng of events. They can be appled when smple parametrc tme-based models, such as exponental or Software error detecton method can be consdered as a Webull tme-to-falure models, are not suffcent to descrbe software relablty growth process. A software relablty the dynamc aspects of a system's relablty or avalablty growth model has been studed by many researchers, as a behavor; as may be the case for systems ncorporatng mathematcal model for the relablty growth process. The standby redundancy. software relablty growth model descrbes the relatonshp between the cumulatve number of detected errors n B. Fault Countng Model software and the tme of software testng. Xe[33] The majorty of these falure count models are based upon categorzed the Software Relablty Models nto followng the NHPP. Schnedewnd [31] proposed a fault detecton groups accordng to ther probablstc assumptons. model based on NHPP. Dfferent NHPP models are dstngushed by ther unque mean value functons. Yamada 1. Software Relablty Models whch descrbe the et al. [35] proposed the delayed S-shaped model. Ths dynamc aspects of the falure occurrence process. category ncludes the models, whch descrbe the occurrence Marovan Model, Falure Count Models, and Model of falure method by stochastc processes le Homogeneous Based on Bayesan Analyss. Posson Process (HPP), Non Homogeneous Posson Process 2. Software Relablty Models whch do not mae any (NHPP), and Compound Process (CPP) etc. Ohba [28] dynamc assumptons of the falure process. proposed the nfecton S-shaped model. Musa et al. [26],[27] The Input Doman Models, Fault Seedng Models, have proposed the basc executon tme model and Log- Software Metrcs Models, and some Software Posson model. Goel [17] modfed Musa's orgnals model Relablty Growth Models Based on NHPP Goel- by ntroducng the test qualty parameter. Yamada et al. [36] Oumoto Model, Delayed S-Shaped SRGM and also proposed a dscrete tme model. The effect of testng Inflecton S-Shaped SRGM. effort on falure process was taen nto consderaton by Yamada et al. [37]. In recent tmes efforts have been 3. Models wth Dynamc Aspects of Falure drected towards development of general SRGMs [22],[30]. General SRGMs are flexble models and many of the above Occurrence Process models can be derved from them. Kapur and Garg [20] modfed Goel-Oumoto [15] model by ntroducng the A. Marovan Models concept of mperfect debuggng. Kareer et al. [23] proposed The memory less property of the Marovan process mples a model wth two types of faults where each fault type s that the tme between consecutve falures follows an module as by an S-shaped curve model. Xe and Zhao [34] exponental dstrbuton. When a Marov process represents have llustrate how the Schnedewnd model can be modfed the falure process, the resultant model s called Marovan to result n many of the above SRGMs. Zeephongseul et al. model. The software can attan several states at any [39] proposed a model descrbng the case when a major partcular tme wth respect to number of faults remanng or fault ntroduces secondary faults. Attempts as lsted above number of faults already removed. The transton between for new models were made wth the prmary ntenton of states depends on the current state of the software and the gettng flexble models that could descrbe a range of falure transton probablty. Numerous attempts have been made to count curves or relablty growth curves le exponental develop Marovan models; especally n earler days due to curves and hghly S-shaped curves. Models wth such smlar theory n hardware relablty was already well property are termed as flexble SRGMs [13],[22],[28]. developed. One of the popular relablty models developed by Jelns and Moranda [19] s a Marov process model. Recently relablty modelng for dstrbuted development Lttlewood [25] proposed a model, based on semenvronment has caught the attentons of many researchers Marovan process to descrbe the falure phenomenon of [38]. Large software systems have modular desgn. A system software wth module structure. Cheung [14] has also s sad to be modular when each actvty of the system s proposed a Marovan model to descrbe module-structured carry out by exactly one component, and when the nputs software. Kremer [24] proposed brth-death model, whch and outputs of each component are well defned [29]. Often ncorporates the probabltes of fault removal and such components of software are developed separately by ntroducton. Goel [17] modfed Jelns-Moranda model by dfferent development teams wth avalablty of ntroducng the concept of mperfect debuggng. Xe and communcaton networs at cheaper rates. Some software Bergman [32] proposed a model relatng the fault detecton components are developed at separate geographc locaton probablty to the fault sze. Kapur et al.[21] proposed a also. Software developed under ths dstrbuted development 1541

3 envronment has proved to be economcal. Many tmes, 2) Falure rate of the software s equally nfluenced by components from other software projects are also reused. faults left behnd n the software. Therefore SRGMs for software developed under dstrbuted 3) The number of faults detected at any nstant of tme s envronment needs to have dfferent approach. But very few proportonal to the remanng number of faults n the attempts have been made n ths regard [38]. software. 4) Repar effort starts and fault causng the falure s C. Models Based on Bayesan Analyss removed wth certanty, on a falure. In the earler two categores, the unnown metrcs or 5) From falure detecton pont of vew, all faults are parameters of the models are estmated ether by the mutually free. Maxmum Lelhood Method or by the Least Squares 6) The proportonalty of falure detecton/ fault removal/ Method (wll be dscussed later). But n ths, the Bayesan fault solaton s constant. analyss technque s used to estmate the unnown 7) The fault detecton/removal phenomenon s modeled by parameters of the models. Lttlewood and Verral, NHPP. recommended the frst software relablty model based on Bayesan Analyss [25]. Sngpurwalla [40] have proposed a Notatons number of Bayesan software relablty models for varous m(t): Expected number of faults dentfed n (0,t), mean testng envronments. Ths technque asssts the use of value functon of NHPP gathered nformaton by developng smlar software project. a, b : Constants, representng ntal fault content and Based on ths nformaton the parameter of gven model are rate of fault removal per remanng faults for a software. assumed to pursue some dstrbuton, nown as pror dstrbuton. Gven the software test data and based on a p, q: Proportonalty constants pror dstrbuton, a posteror or subsequent dstrbuton can be acqured, whch n turn descrbes the falure phenomenon. Non Homogeneous Posson Process (NHPP) Let (N(t); t > 0) be a countng process denotng the 4. Models Wthout any Dynamc Assumptons cumulatve number of falures (or faults solated as the case of Falure Process may be) by tme t, N(t) s a random varable and (N(t); t > 0) s a Non Homogeneous Posson Process (NHPP) f N(0) = 0, The followng four categores of Software Relablty (N(t); t > 0) has ndependent ncrements models are defned brefly here. P (two or more falures n (t, t + t)) = 0( t) A. The Input Doman Models P (exactly one falure n (t, t + t)) = (t) + 0( t) The basc approach n ths category s to generate a set of Where (t) s ntensty functon of N(t). If we let m (t) = tests from a dstrbuton. The dstrbuton should be chosen so that t s representatve of the operaton of the software t (x) dx represent the mean of number of faults and the relablty s estmated from the outcome of the test 0 cases. removed n (0, t). It can be shown that [m(t)] n e m(t) B. Fault Seedng Models P[N(t) = n] =, n = 0,1,2 (1) In ths class a nown number of faults are ntroduced nto n! the software. Durng the software testng, these seeded faults.e. N (t) has a Posson dstrbuton wth expected value as well as orgnal faults are vsble. The proporton of the E[N(t)]= m(t) for t > 0 and m(t) s called the mean value seeded faults found compared to the unseeded faults found functon of NHPP. can gven an estmate of the total fault content n the software. 1) Goel-Oumoto Model : The dfferental equaton results from assumpton-3 are as C. Software Metrcs Models follows The models n ths class relate the fault content n the d m(t) = b[a-m(t)] (2) software to some features of the software program such as program length, complexty, volume etc. These models are The frst order lnear dfferental equaton, as above, when emprcally bult and the result obtaned by a model s solved wth the ntal condton as m(0) = 0 gves the dependent on the software development process followng mean value functon for NHPP (2) envronment, whch may not be the same n the other m(t) = a (1-e -bt ) (3) projects. The mean value functon s exponental n nature and does not offer a good-ft to the S-shaped growth curves that D. Some SRGMs Based On NHPP normally occur n Software Relablty. But the model s In ths segment few SRGMs are presented brefly. These popular due to ts straghtforwardness. SRGMs and ther hypothess have been presented durng the development of some of the models and theores of research. 2) S-Shaped SRGMs Some of the basc assumpton or postulates, apart from some Few S-shaped SRGMs whch wll be dscussed are Delayed specal ones for specfc models dscussed, for the models S-Shaped SRGM and Inflecton S-Shaped SRGM are as follows: 1) Software system s subject to falure durng executon, caused by faults left behnd n the systems. 1542

4 n 1 2. Delayed S-Shaped SRGM Fault detecton n the model s assumed to be a two-phase process comprsng of falure detecton and ts eventual removal by solaton. It consders the tme taen to solate and remove a fault and, so t s mportant that the data to be used at ths tme should be that of fault solaton. In addton, t s assumed that the number of faults solated at any tme nstant s proportonal to the remanng number of faults n the software. The falure rate and solaton rate per fault are assumed to be smlar and equal to b. Thus d mf(t) = b[a-mf(t)] (4) d m(t) =b [mf (t) m(t)] (5) mf (t) s the expected number of a falures n (0,t). Solvng (4) and (5), we get the mean value functon as m(t) =a {1-(1+bt)e bt } (6) Alternately, the model can also be devsed as one stage process drectly as follows: d b 2 t m(t)= (a-m(t)) (7) 1 bt b 2 t It s observed that b as b. Ths model was 1 bt specfcally developed to account for delay n the falure observaton and later ts removal. Ths nd of dervaton s unusual to software relablty only. Xe and Zhao have also proposed alternatve ways of dervng the above model.. Inflecton S-Shaped SRGM The model features S-shaped ness to the mutual dependency between software faults. Other than assumpton-3 t s also assumed that the software contans two types of faults, namely mutually ndependent and mutually dependent. The mutually ndependent faults are those, whch are located on dfferent executon paths of the software, thus they are equally lely to be detected and removed. On the other hand, the mutually dependent faults are those faults, whch are located on the same executon path. Accordng to the order of the software executon, some faults n the executon path wll not be removed untl ther earler faults are removed. Accordng to the assumptons of the model, the fault removal ntensty per unt tme can be wrtten as d m(t) = b(t) [a-m(t)] (8) b s the fault removal rate n the steady state. Solvng (8) under the ntal condton M(0) = 0 we get bt 1 e m(t) = a (9) 1 r bt 1 e r If r = 1, the model reduces to the Goel-Oumoto model. For dfferent values of r, dfferent growth curves can be attaned and n that case the model s flexble. 5. Parameter Estmaton The parameters of the SRGMs are estmated based upon these (nput falure) data. Therefore, efforts should be made to mae the data gatherng more precse and scentfc. Usually data s collected n one of the followng two ways. In the frst case, Tme-Doman data the tme between successve falures are recorded. The accomplshment of mathematcal modelng approach to relablty estmaton depends greatly upon qualty of falure data gathered. Though ths type of data gatherng s more preferable, t may not be smple. Problems can come n measurng the testng effort for each fault and t may be very nconvenent to note the tme at each falure report. The other convenent and commonly collected data type s nown as the grouped data or Interval-Doman data. Here testng ntervals are specfed and number of falures observed durng each such nterval s noted. Some exstng software relablty models can handle both type of data but Tme- Doman data provdes better accuracy of parameters estmaton wth current exstng Software Relablty Models [5]. For both these data type method of Maxmum Lelhood and Method of Least Squares have been recommended and wdely used for estmaton of parameters of SRGMs. A. Parameter Estmaton Methods Parameter Estmaton s of major mportance n Software Relablty predcaton. Once the mean value functon m(t) of analytcal model s nown, the parameter n the soluton s requred to be determned. Durng the testng and ntal operatonal phases of Software Development Lfe Cycle (SDLC), falure events are encountered. They are recorded and underlyng faults that caused them are removed, whch results n process called Relablty Growth. The basc dea behnd the SRGM s smple; f the hstory of fault detecton and removal follows a certan recognzable pattern, t s possble to descrbe the mathematcal form of the pattern. The functon that represent ths pattern s called mean value functon m(t), whch s cumulatve number of faults descrbe n a gven tme t. If we are able to ft ths functon to the exstng hstorcal fault detectve data, we can predct the future falure behavour of software. The mean value functon s often transferred to falure ntensty (rate) functon λ (t) by formula λ (t)= d m(t). The parameters of SRGM are estmated by one of the followng methods. 1) Least Square Estmaton (LSE) 2) Maxmum Lelhood Estmaton (MLE) 1) Method of Least Squares In ths method the square of the dfference between observed response and value predcted by the model s mnmzed. If the expected value of the response varable s gven by m(t) (can be a mean value functon of an SRGM), then the least square estmators of the parameters of the model may be obtaned from n pars of sample values (t1, y1), (t2, y2), (tn, yn) by mnmzng J gven by J = [ y m(t)] (10) y and t dependent varables and observed values of explanatory respectvely. For small and average sze samples Least Square Estmaton s preferred [27]. For estmaton of 1543

5 1 1 t ( x) dx 1 ^ 2 the parameters of the analytcal models, Method of Least Square (Non Lnear Regresson) has been used. Non Lnear Regresson s a technque to fnd a nonlnear model of the relatonshp between the dependent varable and a set of ndependent varables. Unle conventonal lnear regresson, whch s restrcted to estmate lnear models, nonlnear regresson can estmate models wth arbtrary relatonshps between ndependent and dependent varables. 2) Maxmum Lelhood Estmaton Maxmum Lelhood Estmaton(MLE) method has been broadly adopted for estmaton of parameters of SRGMs based upon NHPP. We brefly dscuss below the MLE procedure for two types of software falure data as dscussed above. For the frst type of data, suppose that estmaton s to be performed at a specfed tme t, not essentally correspondng to a falure and wth total of m falures beng experenced at tme t1, t2, tm. Then the lelhood functon for the NHPP s: 0 L = ( t ) e (11) 6. Comparson Crtera for SRGMS The performance of SRGMs are evaluated by ther ablty to ft the hstory software fault data (goodness of ft) and to forecast satsfactorly the future performance of the software fault removal process (predctve valdty)[11],[12]. Musa et al. [8] have suggested the followng attrbutes for choosng an SRGM. a) Capablty: The model should possess the ablty to estmate wth satsfactory accuracy metrcs needed by the software managers. b) Qualty of assumptons: The assumptons should be plausble and must depct the testng envronment. c) Applcablty: A model can be adjudged as the better one f t can be appled across software products of dfferent szes, structures, platforms and functonaltes. d) Smplcty: The data requred for an deal SRGM should be smple and nexpensve to collect. The parameter of estmaton should not be too complex and easy to understand and apply even for persons wthout extensve mathematcal bacground. The MLE of the Parameters can be obtaned by maxmzng Lelhood functon or ts Log lelhood functon (log L). Other than the above qualtatve aspect the followng ndces If the software falure data s grouped nto ponts (t, y); help n comparng SRGMs. =1, 2,, where y s the cumulatve number of falure reports at tme t. Then the Lelhood functon L s gven as A. Goodness of Ft Metrcs follows: The common used metrcs for model comparson of y y goodness of ft and the predctve power are dscussed as [m(t1) m(t 1)] {m(t ) m(t 1)} L = e (12) below 1 (y y 1) 1) The Mean Square Fttng Error (MSE): The model Tang natural logarthm of (12) we get the log lelhood under comparson s used to create the fault data, the functon dfference between the expected values, m(t) and the true or observed data y s measured by MSE as, LogL = (y y 1) ln[m(t ) m(t 1)] m(t ) ln[(m(y y 1)! ] MSE = m( t ) y ) 1 1 (14) where s the number of observatons. (13) The lesser MSE ndcates, lesser the fttng error, thus mproved goodness of ft. The MLE of the parameters of SRGM can be obtaned by maxmzng (13) wth respect to the model parameters. 2) The Aae Informaton Crteron (AIC): It s defned as Estmaton of parameters usng MLE requres solvng set of AIC = -2 (The value of the max. log lelhood functon) + 2 (The smultaneous equatons to maxmze the lelhood defect no. of the parameters used n the model) (15) data comng from gven functon to fnd parameters. Ths ndex taes nto account both the number of parameters that are estmated and the statstcal goodness of ft. Wood [6] appled both MLE and LSE and found LSE to be Lesser values of AIC ndcate the preferred model,.e. the more stable and better correlated to feld data although MLE one wth the mnmum number of parameters that stll offers results were more reasonable. It can be safely assumed that a good ftness to the data. statstcally MLE s much better parameter predctons method than LSE as LSE s much easer and provde 3) Coeffcent of Multple Determnaton (R 2 ) They are consstent results n wder data sets than preferred methods calculated usng the parameters estmated usng LSE [9],[10],[5]. estmators. R 2 = 1 (resdual SS / corrected SS) (16) Both the estmaton methods can also be used to other If value of R 2 s close to 1 then the model provdes better stochastc processes. Maxmum Lelhood Estmators goodness of ft. possess many desrable propertes such as effcency, asymptotc normalty, consstency, and the nvarance B. Predctve Valdty Crteron property. Hence, t s the most preferred estmaton The number of faults removed by tme t can be antcpated procedure for reasonably large sample sze. So we conclude by the SRGM and compared to the reported fault removal, MLE s more sutable for the large sample of data and the.e. y. The dfference between the antcpated/predcted LSE for small to medum sze sample. 1544

6 r value m^ (t) and the reported value measures the fault n 7. Parameters Estmatons Numercal antcpated/predcton. The rato [(m^(t) y)/y] s called the Implementaton Relatve Predcton Error (RPE). If the RPE s postve (negatve) the SRGM s sad to overestmate (underestmate) In ths secton we wll llustrate the procedure for estmaton the fault removal process. Segments of the falure data are of unnown SRGM model parameters wth practcal serally chosen to calculate the RPE. Values nearer to zero consderatons. for RPE ndcate more accurate predcton, thus more confdence n the model and better predctve valdty [8] [12]. The varous metrcs such as MSE, AIC, R 2 A. Models Used Wth Data Set, SSE and The data set used for ths study s Tme-Doman data for a PRR can be used for predctng the valdaton of SRGM. real tme control system provded by Ohba [16]. In data, ffteen (15) faults have been reported wth ther tme between the falures. In ths study we use the followng two wdely used SRGMs as gven n Table I. Model Model-1 Model-2 Table 1: SRGM Models Used Name of Mean Value Functon m(t) SRGM Exponental Goel- Oumoto(G-O) Inflecton S-Shaped Model [16] m(t) = a(1 exp[ bt]), a > 0, b > 0 m(t) = a. 1 exp bt 1 + β(r) exp[ bt], β(r)= 1 r,, a > 0, b > 0, r > 0 Parameters to be Estmated Two, (a, b) Three, a, b, β B. Parameter Estmaton and Analyss In ths study the parameters of software relablty growth model mentoned n the Table I are estmated by usng Least Square Estmaton (Non- Lnear Regresson). The statstcal pacage IBM SPSS s used for estmaton of parameters for goodness of ft and predcton of the models. The values of parameters estmated by usng Least Square Estmaton (LSE) of Non Lnear Regresson (NLR) and parameters values usng Maxmum Lelhood Estmaton (MLE) obtaned wth same data and models by Pham-Zhang [18] are summarzed n Table II for comparson. The presentaton of the models gven n Table I usng LSE and MLE estmators to the observed data s also presented n Fgure 1 and Fgure 2. It s observed from these fgures the curves of the presented models usng LSE and MLE Estmators are almost consstent whch show Goodness of Ft. It s also observed that for both curve and growth rate our estmates usng LSE are much closer to the parameters estmates obtaned usng MLE. Table 2: Estmated Parameters Estmated values of Model Mean Value Functon m(t) Parameters LSE MLE Model- m(t) = a(1 exp[ bt]), a= a= a > 0, b > 0 b= b= Model- 1 m(t) = a. exp bt a= a= β ( r ) exp [ bt ] b= b= , β(r)= 1 β= β= r, r, a > 0, b > 0, r > 0 C. Parameter Valdty and Accuracy Table 3: Comparson of Goodness of Ft Metrcs To predct the goodness of ft and valdty of Model usng Metrcs MSE AIC RMSPE LSE and MLE estmators, the varous metrcs are evaluated Model LSE MLE LSE MLE LSE MLE such as Sum of Squared Errors (SSE), Mean Squared Error Model (MSE), (AIC) and Root Mean Square Predctve Error Model (RMPSE) summarzed n Table III. We now compare the predctve accuracy of the value of a metrcs obtan usng The coeffcents of Multple Determnaton (R 2 ) are MLE and NLR estmators as shown n Table III. It s calculated usng the parameters estmated usng LSE observed that the values of SSE, MSE, PRR and estmators. The Table IV summed up the values of R 2 usng RMSPE obtaned wth MLE estmator are less than the LSE and growth rate of the models wth LSE and MLE values obtaned wth LSE estmator n Model-1 and Model-2 estmators to predct the valdty of the models. The values whch ndcates that MLE s better estmator. of R 2 obtaned for varous models le close between 0 and 1. The values are very close to 1 or equal to 1, whch ndcate the valdty of the models. It s also observed that growth 1545

7 rates n both the cases of varous models are almost equal Growth Rate wth MLE whch ndcates the Goodness of Ft of these models. D. Models Asymptote for Goodness of Ft Table 4: R 2 and Growth Rate to Predct Valdty and Asymptotc vew of goodness of ft for the Model-1.e. Accuracy of the Model Goel-Oumoto (G-O) and Model-2.e. Inflecton S-Shaped Model Model-1 Model-2 Model, s shown below n Fgure-1 and Fgure-2 respectvely. Value of R 2 usng LSE Growth Rate wth LSE Fgure 1: Model 1 - Goel-Oumoto (G-O) Fgure 2: Model 2 - Inflecton S-Shaped Model 8. Concluson In ths paper we have dscussed, when a software system s desgned, the major concern s the software qualty. The qualty of software depends on varous factors such as software relablty, effcency, date of delvery and development cost etc. Ths paper analyzed varous exstng software relablty models wth ther falure ntensty functon and the mean value functon. Therefore, we recommend that most mportant process, Least Square Estmaton and Maxmum Lelhood Estmaton for estmatng parameters of Software Relablty Growth Models and the varous metrcs used for comparson of Goodness of Ft and predctve valdty. Usng the falure data set from the Lterature and exstng SRGMs for mplementaton of LSE and MLE practcally for comparson of Models, t was noted that MLE gves the better results as compared to LSE for Goodness of Ft and predctve valdty of Models. It was observed that MLE s dffcult to apply whch lmts ts use n ndustry, especally due to lac of tools support whereas LSE s easy to use due to avalablty of compatble tools. It was concluded from the results presented here and the propertes of LSE and MLE estmators suggested that Maxmum Lelhood Estmator s better for predcton of Relablty Growth Models whereas Least Square Method for Non Lnear Regresson s a good estmator for fttng the data to observed falure data. In future the above recommendatons can be mplemented and results of the model can be compared wth the exstng model results. References [1] A. Kaur, H. P. S. Dham and J. Kaur (2009): Component Based Software Engneerng, IEEE-SOFT- 3140, IEEE Internatonal Advance Computng Conference IACC [2] H. P. S. Dham (Sept.2016): Comparatve Study and Analyss of Software Process Models on Varous Merts, Internatonal Journal of Advanced Research n Computer Scence and Software Engneerng, Vol 6(9), pp

8 [3] H. P. S. Dham and Dr Anuj Kumar (July 2013): performance Evaluaton. New Yor: Academc Press: Analyss of Software Desgn Eroson Issues, pp Internatonal Journal of Advanced Research n [20] Kapur PK, Garg RB.(1990c). Optmal release polces Computer Scence and Software Engneerng 3 (7), pp. for software systems wth testng effort. Int. Journal 1-7. System Scence. 22(9), pp [4] Quadr, S.M.K. and Ahmad, N.(2010), Software [21] Kapur PK, Sharma KD, Garg RB. (1992). Transent Relablty Growth Modellng wth new modfed solutons of software relablty model wth mperfect Webull testng- effort and optmal release polcy, debuggng and error generaton. Mcroelectroncs and Internatonal Journal of Computer Applcatons, Vol. 6, Relablty. 32: pp No. 12. [22] Kapur PK, Garg RB, Kumar S (1999). Contrbutons to [5] Pham, H. (2003), Software relablty and cost hardware and software relablty. Sngapore: World models: Perspectves, comparson and practce, Scentfc Publshng Co. Ltd. European Journal of Operaton Research, Vol. 149, [23] Kareen N, Grover PS, Kapur PK (1990). An S-shaped No. 3, Relablty growth model wth two types of errors. [6] Wood, A.(1996), Predctng Software relablty, Mcroelectroncs and Relablty. 30(6):pp Computer, Vol. 29, No.11, pp [24] Kremer W (1983). Brth death bug countng. R-32: [7] Harmnder Pal Sngh Dham (Dec.2016): Improvng pp Software Relablty, Productvty and Qualty usng [25] Lttlewood B, Verrall JL (1997). A Bayesan relablty software metrcs, Internatonal Journal of Computer growth model for computer software. Appled Statstcs. Scence & Technology (IJCST) Vol.7 Issue: 4(1), ISSN 22: pp (onlne), ISSN (Prnt). [26] Musa JD (1979). Valdty of Executon Tme Theory of [8] Musa, J.D., Iannno(1987), A. and Komodo, K., Software Relablty. IEEE Transactons on Relablty. Software Relablty: Measurement, Predcton and R-28: pp Applcaton, McGraw-Hll. [27] Musa JD, Iannno A, Oumoto K (1987). Software [9] Ullah, N., Morso, M. and Vetro, A.(2012), A relablty: Measurement, predcton, Applcatons. New Comparatve Analyss of Software Relablty Growth Yor: MC Graw Hll. Models Usng Defects Data of Closed & Open Source [28] Ohba M (1984). Software Relablty Analyss Models. Software, n Software Engneerng Worshop (SEW), IBM Journal of Research and Development. 28: pp , 35 th Annual IEEE, pp [10] Rana, R., Staron, M., Berger, C., Hanson, J., Nlsson, [29] Pfleeger SL(1998). Software Engneerng- Theory and M. and Torner, F.(2013), Evaluatng the long-term Practce. New Jersey: prentce Hall. predctve power of standard relablty growth models [30] Pham H(2000). Software Relablty. Sprnger-Verlag on automotve systems, Pasadena, CA, USA. Pte. Sngapore [11] Musa JD (1999), Software Relablty Engneerng, [31] Schnedewnd NF(1975). Analyss of Error Process n McGraw- Hll. Computer Software. Sgplan Notces. 10(6): pp.337- [12] Kapur PK, Garg RB and Kumar S (1999), 346. Contrbutons to Hardware and Software Relablty, [32] Xe M, Bergman B (1980). On Modellng Relablty World Scentfc, Sngapore. Growth For Software. Proceedngs 8 th IFAC symposum [13] Bttant S, Bolzern P, Pedrott E, Scattoln R (1988). A on Identfcaton and Parameter estmaton, Bejng; flexble modellng approach for software relablty 1980; Chna: pp growth. Software Relablty Modellng and [33] Xe M (1991). Software Relablty Modelng. World Identfcaton (Ed.) G. Goos and J. Harmans. Sprnger Scentfc. Verlag. Berln. pp [34] Xe M, Zhao M (1992). The Schnedewnd Software [14] Cheung RC (1980). A User Orented Software Relablty Growth Model Revsted. IEEE proceedngs. Relablty Growth Model. IEEE Transactons on pp Software Engneerng. SE-6: pp [35] Yamada S, Ohba M, Osa S(1983). S-Shaped Software [15] Goel AL, Oumoto K (1979). Tme dependent error Relablty Growth Modellng for Software Error detecton rate model for software relablty and other Detecton. IEEE Trans. On relablty. R-32(5): pp.475- performance measures. IEEE Transactons on 484. Relablty. R-28(3): pp [36] Yamada S, Osa S, Narhsa H (1986). Dscrete Models [16] Ohba (July 1984), Software relablty-analyss models, for Software Relablty Evaluaton. In: Basu AP IBMJ. Res. Develop.,vol. 28, pp (Edtor) Relablty and Qualty Control. North Holland: [17] Goel AL (1985), Software relablty models: Elsever scence Publshers. assumptons, lmtatons and applcablty. IEEE [37] Yamada S, Hshtan J, Osa S (1993). Software- Transactons on Software Engneerng. SE-11, pp Relablty Growth Model wth a Webull Test Effort: A Model And Applcaton. IEEE Transactons on [18] Pham, H., Zhang X.(1997), An NHPP Software Relablty 42(1): pp relablty model and ts comparson, Internatonal [38] Yamada S, Tamura Y, Kmura M (2003). A Software Journal of Relablty, Qualty and Safety Engneerng, Relablty Growth Model for A Dstrbuted Vol. 4, No.3, pp Development Envronment. Electroncs & [19] Jelns Z. Moranda PB(1972). Software Relablty Communcatons n Japan. part 3,Vol. 83, pp Research. In: Freberger W. (Ed.) Statstcal Computer [39] Zeephongseul p, Xa G, Kumar S (1992). A Software Relablty Growth Model Wth Prmary Errors 1547

9 Generatng Secondary Errors. Research report Number- 2, Department of Mathematcs; Royal Melbourne Insttute of Technology, Australa. [40] Sngpurwalla ND, Wlson SP (1999). Statstcal Methods n Software Engneerng-Relablty and Rs. New Yor: Sprnger-Verlag. 1548