Empirical Validate C&K Suite for Predict Fault-Proeess of Object-Orieted Classes Developed Usig Fuzzy Logic. Mohammad Amro 1, Moataz Ahmed 1, Kaaa Faisal 2 1 Iformatio ad Computer Sciece Departmet, Kig Fahd Uiversity of Petroleum ad Mierals, Dhahra, Saudi Arabia Abstract Empirical validatio of software metrics suites to predict fault proeess i object-orieted (OO) compoets is essetial to esure their accuracy i practical idustrial. I this paper, we empirically validate the Chidamber ad Kemerer (CK) metrics suite metrics for their ability to predict software quality i terms of fault-proeess: we explore the ability of these metrics suites to predict fault-proe classes usig defect data for six versios of Rhio, a ope-source implemetatio of JavaScript writte i Java. We coclude that the C&K suite cotai similar compoets ad produce statistical models that are effective i detectig error-proe classes. Aalyzig Fuzzy Logic models across six Rhio versios idicates these models may be useful i assessig quality i OO classes produced usig moder highly iterative or agile software developmet processes. Keywords- fault-proe; fuzzy logic; software quality; predictio model 1 Itroductio Several Object-Orieted metrics have bee developed by researchers to help evaluate software desig quality [1-3]. While a measure may be correct from a theoretical perspective, it may ot be of practical use i software idustrial[4, 5]. Metrics may be difficult to collect or may ot really measure the iteded quality properties of software. Empirical validatio is ecessary to determie the usefuless of a metric i assessig ope source software quality. Ope source tools are becomig ever more importat for the user these days. May compaies are usig this kid of software i their ow work. Therefore, may of these projects are beig developed rapidly ad are quickly becomig very large. However, because ope source software is usually produced by voluteers, ad the developmet approach employed is quite differet from the usual methods applied i commercial software developmet especially for level of testig, the quality ad reliability of the code eeds to be ivestigated. Various kids of code measuremets ca be quite helpful i obtaiig iformatio about the quality ad fault-proeess of the code. I this paper, we describe how we calculated ad validated the object-orieted metrics suite give by Chidamber ad Kemerer [3] for fault-proeess detectio from the source code of the ope source Mozilla Rhio JavaScript writte i Java[6]. 2 Chidamber ad Kemerer s (CK) Metrics Chidamber ad Kemerer origially defied the CK metrics suite i 1991. I 1994, they published aother paper cotaiig revised defiitios of some of the metrics [3]. I this research, all CK metrics are selected to be validated its ability to predict the fault, i total CK suite cotiue six metrics which describe i Table 1.
Classes TABLE 1: CK SUITE METRICES [3, 7] Metric DIT WMC RFC CBO LCOM NOCL Descriptio Depth of Iheritace Tree (DIT) it measure the geeral classes, which are expected to be reused by other classes, are usually at a high level i the iheritace hierarchy. Weighted Methods per Class Number of Methods per Class is a measure of software size, ad hece a idicator of complexity Respose for Class is a measure of couplig. It couts the umber of methods that are immediately available to ad potetially used by a class. Couplig Betwee Objects (CBO) is a measure of couplig, coutig the umber of other classes to which a class is coupled. A class A is said to be coupled to aother class B, if class A accesses methods or variables defied by class B. large CBO value ofte idicates a high degree of depedecy o other classes Lack of Cohesio of Methods Number of Childre is measure the complexity of a iheritace hierarchy.it couts the umber of immediate subclasses derived from the curret class. 3 Experimetal Evaluatios 3.1 Datasets: We chose the Mozilla Rhio project to examie i this study because it was a real ope source project ad because of the availability of fault data for several versios of the project, Rhio is a ope source implemetatio of JavaScript. The developmet team of Rhio cosists of three programmers. All i separate locatios deliverig the java implemetatio with a varyig cycle time from two to 16 moths. I this study, we aalyzed 14R3, 15R1, 15R2, 15R3, 15R4, ad 15R5. Error data exists for Rhio i the olie Bugzilla website[8]. We Collect the Rhio fault data form a published work doe by Hector M et al[5]. Figure 1 shows the statistic for selected Rhio versios that had bee ivestigate durig the study. 250 200 Defects reported Ehaceme ts Made Class cout 150 100 50 0 rhio14r3 rhio1_5r 5 rhio15r4 rhio15r3 rhio15r2 rhio15r1 Defects reported 21 61 153 41 10 29 Ehacemets Made 1 37 76 0 0 3 Class cout 95 201 198 178 179 126 Rhio versio Figure 1: Defects reported ad ehacemets made per Rhio versio.
Table 2 shows the descriptive CK metrics statistics for the Rhio datasets which extracted by usig commercial tool amed METAMATA. Table 2: THE DESCRIPTIVE STATSTICS FOR THE DATASETS versio Statistics DIT WMC RFC CBO LCOM NOCL 14R3 6 464 165 59 2681 2 Mea 2.506494 109.4805 26.66234 10.22078 115.3377 1.012987 StdDev 1.154207 123.0208 33.43517 11.28182 420.9545 0.113961 15R1 6 688 202 65 3305 3 Mea 2.578431 122.6765 28.87255 10.89216 112.8627 1.019608 StdDev 1.120787 140.9371 37.09919 11.98424 460.8304 0.19803 15R2 7 732 203 69 4126 3 Mea 2.779817 139.7064 29.23853 10.21101 141.2477 1.027523 Std Dev 1.480477 167.6788 38.70709 12.13128 546.7883 0.213382 15R3 7 730 206 76 4524 3 Mea 2.841121 144.1402 29.96262 10.4486 152.1308 1.065421 Std Dev 1.486731 169.6414 40.12408 12.59697 594.096 0.315362 15R4 7 764 205 77 4951 3 Mea 2.756757 147.5225 30.32432 10.25225 158.1982 1.117117 Std Dev 1.472266 173.6812 41.31831 12.5274 615.8118 0.398605 15R5 6 922 214 67 5172 6 Mea 2.825688 156.1193 31.66055 10.25688 166.6422 1.155963 Std Dev 1.470979 181.4662 41.41484 11.65746 665.6276 0.626192 Table 3:Correlatios betwee: DIT, WMC, RFC, CBO, LCOM, NOCL, ad umber of Defects reported DIT WMC RFC CBO LCOM NOCL WMC 0.188 RFC 0.349 0.941 CBO 0.829 0.535 0.671 LCOM 0.460 0.904 0.838 0.757 NOCL -0.267 0.859 0.667 0.093 0.692 # of Defects 0.325 0.371 0.328 0.626 0.600 0.160 I order to get most relevat idepedet variables to the depedet variable, we used Pearso s Correlatio Coefficiets (PCC), idicates the stregth ad directio of a liear relatioship betwee two variables. Table 3
shows the PCC betwee umber of Defects ad each of the CK metrics. From the table, there is a sigificat correlatio betwee umber of Defects ad the CK metrics. Table 3 shows that, there highly correlatios betwee CBO, LCOM ad WMC metrics ad umber of Defects. 3.2 Predictio Accuracy Measures The term predictio accuracy i this paper meas how well a predictive model costructed usig kow data ca predict the outcomes of ukow data. This paper evaluates ad compares the Rhio software Fault- Proeess predictio models quatitatively, usig the described below predictio accuracy measures. For all the used measures the lower the error measure, the better is the performace. Root-mea-square error (RMSE) shows differeces betwee values predicted by a model ad the values actually observed from the thig beig modeled. RMSE i 1 ( f ( x ) y ) i i 2 (1) Normalized root-mea-square error (NRMSE): to ormalize the RMSE to the rage of the observed data. NRMSE RMSE f ( x) f ( x (2) max ) mi MRE is a ormalized measure of the discrepacy betwee actual values ad predicted values. MRE y f ( x) y (3) Mea magitude of relative error (MMRE) : 1 MMRE MRE i (4) i 1 4 Result ad Discussio This sectio describes the experimets coducted i our study. I the coducted experimets, we traiig the model usig oe time all CK metrics ad other with oly high correlated metrics CBO, LCOM ad WMC. We repeated the experimet more tha oe time to produce reliable results. Figure 2 ad 3 show the result for two error measures (NRMSE, MMRE) for fuzzy Mamdai model. 1.4000 1.2000 1.0000 0.8000 0.6000 0.4000 0.2000 0.0000 Mea (WMC,CB O,LCOM) (WMC,CB O,LCOM) Mea NRMSE 1.2042 0.2982 0.8925 0.05190945 Figure 2: NRMSE error measures usig Mamdai model 0.0160 0.0140 0.0120 0.0100 0.0080 0.0060 0.0040 0.0020 0.0000 NRMSE (Rhio dataset) (Rhio dataset) Mea (WMC,C BO,LCO M) (WMC,C BO,LCO M) Mea MMRE 0.0035 0.0112 0.0037 0.0137 Figure 3: MMRE error measures usig Mamdai model
5 Coclusio ad Future Work I this paper, we coducted the experimets to evaluate the performace of the fuzzy iferece systems models to predict Fault-Proeess of Object-Orieted Classes Developed Usig CK metrics. As show i table 3, there is sigificat correlatio betwee the measure provided by three CK metrics (LOC,CBO,WMC) ad the umber of defects i a class. We use to two Accuracy Measures (NRMSE,MMRE ) to validate the used model. As a future work, we pla to coduct the experimet with larger dataset, which will ehace the performace of fuzzy iferece models. [7] Yu, P., T. Systa, ad H. Muller. Predictig faultproeess usig OO metrics. A idustrial case study. i Software Maiteace ad Reegieerig, 2002. Proceedigs. Sixth Europea Coferece o. 2002. IEEE. [8] Database, B. Mozilla Foudatio. July 2004; Available from: https://bugzilla.mozilla.org/. ACKNOWLEDGMENT The authors ackowledge the support of Kig Fahd Uiversity of Petroleum ad Mierals. Referece [1] Basiya, J. ad C.G. Davis, A hierarchical model for object-orieted desig quality assessmet. Software Egieerig, IEEE Trasactios o, 2002. 28(1): p. 4-17. [2] Brito e Abreu, F. ad W. Melo. Evaluatig the impact of object-orieted desig o software quality. i Software Metrics Symposium, 1996., Proceedigs of the 3rd Iteratioal. 1996. IEEE. [3] Chidamber, S.R. ad C.F. Kemerer, A metrics suite for object orieted desig. Software Egieerig, IEEE Trasactios o, 1994. 20(6): p. 476-493. [4] Basili, V.R., L.C. Briad, ad W.L. Melo, A validatio of object-orieted desig metrics as quality idicators. Software Egieerig, IEEE Trasactios o, 1996. 22(10): p. 751-761. [5] Olague, H.M., et al., Empirical validatio of three software metrics suites to predict fault-proeess of object-orieted classes developed usig highly iterative or agile software developmet processes. Software Egieerig, IEEE Trasactios o, 2007. 33(6): p. 402-419. [6] N. Boyd. Rhio Home Page. July 2006; ]. Available from: http://www.mozilla.org/rhio/.