A Novel Method for Early Software Quality Prediction Based on Support Vector Machine

Transcription

1 A Nove Method for Eary Software Quaity Prediction Based on Support Vector Machine Fei Xing 1,PingGuo 1;2, and Michae R. Lyu 2 1 Department of Computer Science Beijing Norma University, Beijing, 1875, China 2 Department of Computer Science & Engineering The Chinese University of Hong Kong, Shatin, NT, Hong Kong, SAR of China xsoar@163.com, pguo@bnu.edu.cn, yu@cse.cuhk.edu.hk Abstract The software deveopment process imposes major impacts on the quaity of software at every deveopment stage; therefore, a common goa of each software deveopment phase concerns how to improve software quaity. Software quaity prediction thus aims to evauate software quaity eve periodicay and to indicate software quaity probems eary. In this paper, we propose a nove technique to predict software quaity by adopting Support Vector Machine (SVM) in the cassification of software modues based on compexity metrics. Because ony imited information of software compexity metrics is avaiabe in eary software ife cyce, ordinary software quaity modes cannot make good predictions generay. It is we known that SVM generaizes we even in high dimensiona spaces under sma training sampe conditions. We consequenty propose a SVM-based software cassification mode, whose characteristic is appropriate for eary software quaity predictions when ony a sma number of sampe data are avaiabe. Experimenta resuts with a Medica Imaging System software metrics data show that our SVM prediction mode achieves better software quaity prediction than some commony used software quaity prediction modes. 1. Introduction Modern society is fast becoming dependent on software products and systems. High reiabiity is one of the most important probem facing the software industry. A software quaity mode is a too for focusing software enhancement efforts. Such modes yied timey predictions on a modueby-modue basis, enabing one to target high-risk modues. Software metrics represent a quantitative description of program attributes and they pay a critica roe in predicting the quaity of the resuting software [1]. Software compexity metrics have been shown to be cosey reated to the distribution of fauts in program modues. That is, there is a direct reationship between some compexity metrics and the number of fauts ater found during test, vaidation and operation [2, 3]. Consequenty, investigating the reationship between the number of fauts in a program and its software compexity metrics attracts attentions from many researchers. Software compexity metrics can be used as input variabes of quaity prediction mode to predict the faut number, but predicting the exact number of fauts in each modue is often not necessary. Severa different techniques have been proposed to deveop predictive software metrics for the cassification of software program modues into faut-prone and non faut-prone categories. These techniques incude discriminant anaysis [4, 5], factor anaysis [6], booean discriminant functions [7], cassification trees [8, 9], pattern recognition (Optima Set Reduction, OSR) [4, 1], EM agorithm [11], feedforward neura networks [12], random forests [13], and many other methods [14]. With these predictive modes, deveopers and managers can focus resources on the most faut-prone modues eary and prevent probems of poor quaity ater in the software ife cyce. To buid a predictive mode, the number of changes (fauts) is usuay required. However, to obtain the dependent criterion variabes, we need to take a ong time for coecting the feedback of test and vaidation resuts. For exampe, for the Medica Imaging System (MIS) software presented ater in this paper, the actua number of changes (fauts) in that program was coected during a three-year observation period. As software compexity metrics can be obtained reativey eary in the software ife-cyce, it is worthy to expore new techniques for eary prediction of software quaity based on software compexity metrics. On the other hand, the reationships between software metrics and the cassification of program modues are often compi-

2 cated and noninear, imiting the accuracy of conventiona approaches. So it is difficut to mode with the traditiona methods, and an appropriate noninear mode needs to be deveoped to sove the probem. Support Vector Machine (SVM) [15] is a new technique for data cassification, which has been used successfuy in many object recognition appications [16, 17, 18, 19]. SVM is known to generaize we even in high dimensiona spaces under sma training sampe conditions and it is adaptive to mode noninear functiona reationships that are difficut to mode with other techniques. A these characteristics make SVM appropriate for software quaity modeing as such conditions are typicay encountered. 2. Support Vector Machine Here we briefy review the basics of SVM first. SVM was introduced by Vapnik in the ate 196s on the foundation of statistica earning theory [2]. In theory, the SVM cassification can be traced back to the cassica structura risk minimization (SRM) approach, which determines the cassification decision function by minimizing the empirica risk The Optima Separating Hyperpane SVM empoys a inear mode to impement noninear cass boundaries through some noninear mapping of the input vectors x into the high-dimensiona feature space F via a noninear mapping φ. The optima separating hyperpane is determined by giving the argest margin of separation between different casses. For the two-cass case, this optima hyperpane bisects the shortest ine between the convex hus of the two casses. The data are separated by a hyperpane defined by a number of support vectors. The SVM attempts to pace a inear boundary between the two different casses, and orient the boundary in such a way that the margin is maximized. The boundary can be expressed as foows: (w x)+b = ; w 2 R N ; b 2 R; (1) where the vector w defines the boundary, x is the input vector of dimension N and b is a scaar threshod. The optima hyperpane is required to satisfy the foowing constrained minimization as minf 1 2 kwk2 g (2) y i (w x i + b) 1; i = 1;2;:::;; (3) where (x i ;y i ) is the training set, and is the number of training sets. Using standard Lagrangian duaity techniques, one arrives at the foowing dua Quadratic Programming (QP) probem: maxf i α 1 2 α i α j y i y j (x i x j )g (4) i; j=1 y i α i = ; (5) α i ; i = 1;2;:::;; (6) where the α i are the Lagrangian mutipiers and are nonzero ony for the support vectors. Thus, the hyperpane parameters (w;b) and the cassifier function f (x;w;b) can be computed by an optimization process. The decision function is obtained as foows: f (x) =signf y i α i (x x i )+bg: (7) 2.2. The Generaized Optima Separating Hyperpane For the ineary non-separabe case, the minimization probem needs to be modified to aow miscassified data points. This modification resuts in a soft margin cassifier that aows but penaizes errors by introducing positive sack variabes ξ i (i = 1;2;:::;) as the measurement of vioation of the constraints: minf 1 2 kwk2 +C( ξ i )g (8) y i (w x i + b) 1 ξ i ; i = 1;2;:::;; (9) where C is used to weight the penaizing variabes ξ i,anda arger C corresponds to assigning a higher penaty to errors. The soution to this minimization probem is identica to the separabe case except for a modification of the bounds of the Lagrange mutipiers. Equation (6) is thus changed to:» α i» C; i = 1;2;:::;: (1) 2.3. Generaization in High Dimensiona Feature Space In the cases where the inear boundary in input spaces is not enough to separate two casses propery, it is possibe to create a hyperpane that aows the inear separation in a higher dimension space F. The method consists of projecting the data in a higher dimension space F via a noninear mapping φ, where they are considered to become

3 ineary separabe. The transformation into a higher dimensiona feature space is reativey computation-intensive. A kerne can be used to perform this transformation, and the dot product in a singe step providing the transformation can be repaced by an equivaent kerne function. This heps in reducing the computationa oad and retaining the effect of higher-dimensiona transformation at the same time. The kerne function K(x i ;x j ) is defined as foows [15]: K(x i ;x j )=φ(x i ) φ (x j ): (11) There are some commony used kerne functions: 1. Poynomia: K(x i ;x j )=[(x i x j )+1] q 2. Radia basis: K(x i ;x j )=exp( kx i x j k 2 =2σ 2 ) cass1 cass Figure 1. Optima Separating Hyperpane, C 1 = C 2 = 2 3. Sigmoid: K(x i ;x j )=tanh(ν(x i x j )+c) 2.4. SVM with Risk Feature cass1 cass2 Generay speaking, the oss incurred for making an error is greater than the oss incurred for being correct, and minimizing risk means that we try to find decision procedure that minimizes serious errors. We therefore introduce risk contro into SVM, which takes into account the cost of different types of errors by adjusting the error penaty parameter C to contro the risk. We suppose the first k training sets beong to cass 1 and the remaining k + 1 training sets beong to cass 2. Equation (8) is then converted to: minf 1 2 kwk2 +C 1 ( k ξ i )+C 2 ( i=k+1 ξ i )g (12) y i (w x i + b) 1 ξ i ; i = 1;2;:::;; (13) where C 1 is the error penaty parameter of cass 1 and C 2 is the error penaty parameter of cass 2. The soution to this minimization probem is aso simiar to that of the separabe case, but the bounds of the Lagrange mutipiers is changed to:» α i» C 1 ; i = 1;2;:::;k;» α i» C 2 ; i = k + 1;k + 2;:::;: (14) Various effects of adjusting the error penaty parameter C are shown in Figs. 1, 2 and 3, respectivey. It is noted that when the cass 1 and 2 have the same vaue of parameter C, the optima separating hyperpane is achieved Transductive SVM Traditiona SVM ony empoys abeed training sampes to buid a cassifier. As a kind of semi-supervised earning Figure 2. Separating Hyperpane with Risk Feature, C 1 = 1;C 2 = 2 methods, Transductive Support Vector Machines (TSVM) [21] take into account a particuar test set as we as training set, and try to minimize miscassifications of ony those particuar exampes. Besides of abeed training sampes (x i ;y i ), we consider another m unabeed sampes x Λ i. To find a abeing yλ i of the test sampe, the hyperpane < w;b > shoud separate both training and test data with the maximum margin: Minimizeover(y Λ i ;w;b) : 1 2 kwk2 (15) y i (w x i + b) 1; i = 1;2;:::;k; y Λ i (w xλ i + b) 1; i = 1;2;:::;m: (16) To be abe to hande non-separabe data cases, we can introduce sack variabes ξ Λ i (i = 1;2;:::;m) simiar to the way we hande the traditiona SVM. That is, the earning process of TSVM can be formuated as the foowing opti-

4 cass1 cass2 where α j is the prior probabiity, m j is the mean vector, and Σ j is the covariance matrix of the jth cass. Equation (2) is often caed the discriminant function for the jth cass in the iterature [23]. Furthermore, if the prior probabiity α j is the same for a casses, the term 2nα j can be omitted and the discriminant function reduces to a simper form [24]. The parameters in Eq. (19) and Eq. (2) can be estimated with the traditiona maximum ikeihood estimator: Figure 3. Separating Hyperpane with Risk Feature, C 1 = 2;C 2 = 1 mization probem: Minimizeover (y Λ i ;w;b;ξ i ;ξ Λ i ) : 1 2 kwk2 +C ξ i +C Λ m ξ Λ i (17) y i (w x i + b) 1 ξ i ; i = 1;2;:::;k; y Λ i (w xλ i + b) 1 ξ Λ i ; i = 1;2;:::;m; ξ i ξ Λ i (18) where C and C Λ are user-specified parameters. C Λ is caed the effect factor of the unabeed exampes and C Λ ξ Λ i is caed the effect term of the ith unabeed exampe in the above function. To sove this optimization equation, agorithms can be referenced from [22]. 3. Other Modes 3.1. Discriminant Anaysis Severa researchers have appied discriminant techniques to study software compexity metrics. One of severa discriminant techniques may be appropriate for a given anaysis. Among them quadratic discriminant anaysis (QDA) is the most widey used in the fied of pattern recognition when sufficient training sampes coud be suppied. Given the data points D = fx i g N, we can appy Bayesian decision rue to cassify the data x into jth cass: with j Λ = argmin j d j (x); j = 1;2; ;k (19) d j (x) =(x m j ) T Σ 1 j (x m j )+njσ j j 2nα j (2) α j = n j N ; (21) m j = 1 n j n j x i; (22) Σ j = 1 n j n j (x i m j )(x i m j )T : (23) Now x i is a sampe from cass j with probabiity one, and n j is the training sampe number of cass j: When an unbiased estimation is used, then Σ 1 j = n j 1 n j (x i m j )(x i m j )T ; (24) which is caed the sampe covariance matrix in the iterature [25]. Using the cassification rue in Eq. (19) and Eq. (2) with the above covariance estimation is known as quadratic discriminant anaysis Cassification Tree A cassification tree (CT) can be used to predict the cass membership of objects on the basis of one or more predictor variabes. It is widey used in pattern recognition fied, whereas Khoshgoftaar introduced the cassification and regression trees (CART) agorithm to software quaity prediction [26]. The CT as we as constructing CT agorithms are described in the iterature as the foowing [27]. The tree consists of a set of decision rues, which are appied in a sequentia manner, unti each object has been assigned to a specific cass. The first decision rue, appied at the root node of the tree to the vaues of a objects aong one or more predictor variabes, has two possibe outcomes: Objects are either sent to a termina node (eaf), upon which a cass is assigned, or to an intermediate node, upon which another decision rue appies. Utimatey, a objects are sent to a termina node and assigned a cass abe. The simpest type of CT is the binary tree, in which the spits are binary (that is, each parent node is attached to two daughter nodes) and the decision rues are univariate. CTs can be constructed based on continuous or discrete predictor variabes, or on a mixture of both (when univariate spits

5 are appied), and the trees are generay constructed by recursive partitioning (i.e., a given predictor variabe can be engaged in more than one decision rue). To construct CTs, there are two commony-used agorithms. One agorithm is cassification and regression trees (CART), which was deveoped by Breiman et a [28], and the other is the quick, unbiased, efficient statistica trees (QUEST), which was deveoped by Loh and Shih [29]. The CART agorithm finds the optima univariate spits by carrying out an exhaustive search of a possibe spits, whereas by appying a modified form of discriminant anaysis QUEST agorithm finds the optima univariate or mutivariate spits. These agorithms have some different features. It is reported that the CART agorithm is biased toward seecting predictor variabes having more eves, whereas the QUEST agorithm avoids this bias, and is therefore more appropriate when some predictor variabes have few eves whie other predictor variabes have many eves [29]. Conversey, an advantage of CART is that it is a non-parametric cassifier, i.e., no assumptions are made about the distributions of the variabes. Thus, CART anaysis can be used when the assumptions of inear discriminant anaysis (LDA) and binary ogistic regression (BLR) have not been satisfied. The optima CT is one that minimizes costs. When the prior probabiities of objects beonging to different casses are set proportiona to the cass size, and if miscassification costs are set to be the same for every cass, then minimizing costs is equivaent to minimizing the overa proportion of miscassified objects. However, if the prior probabiities are set according to previous knowedge, or if different miscassification costs are used, then minimizing costs does not correspond exacty to minimizing the miscassification rate. Unequa miscassification costs are used when one kind of miscassification is considered worse than another. For exampe, incorrecty predicting a faut-prone modue to be non faut-prone (Type II error) might be considered worse than incorrecty predicting a non faut-prone modue to be faut-prone (Type I error). In such a case, when assessing accuracy of the cassification, miscassifications of the former type coud be penaized more than miscassifications of the atter. When constructing a CT, if it is grown unti a termina nodes are homogeneous, the resuting tree is ikey to overfit the data, and wi therefore suffer a ower accuracy of cassification when appied to new objects. To avoid this case, one shoud therefore appy a stopping rue so that spitting stops at nodes that are either competey homogenous, or have no more than a specified number of objects in the case of nodes containing more than one cass. Other strategies incude specifying the size of the tree to be grown (i.e., the number of termina nodes increases), or defining the minimum heterogeneity that a node must have in order to be spit. Even when a stopping rue is appied, the resuting tree may not be the best tree, in the sense of maximizing accuracy of cassification whie at the same time minimizing compexity. Thus, in order to obtain the optima tree, we shoud derive procedures for pruning trees, i.e., for successivey prune the east important spits unti the best tree is produced. In minima cost-compexity pruning, a nested sequence of optimay pruned subtrees is generated when the tree of the maximum size is pruned to the root node. The maximum size is determined by the stopping criterion. The sequence is optimay pruned since there is no other tree of the same size with ower cost for every size of tree in the sequence. At first, the earning sampe cost decreases as the size of the tree increases. However, the cost generay decreases sowy as the first termina nodes are removed, unti a point is reached when the cost rises rapidy upon remova of additiona nodes. This turning point can be used to define the best-sized tree. Aternativey, the CV costs can be used to identify the best tree in the sequence if cross-vaidation (CV) is performed at each step of the pruning process. This is caed as minima cost-compexity CV pruning. Generay, the CV cost fas sowy to a minimum vaue as termina nodes are removed, and then rises rapidy as the ast few nodes are removed. Thus, the best tree can be defined as the tree cosest to the minimum, i.e., the tree with the minimum CV cost. In addition, Breiman et a suggested that the best-sized tree can be identified as the smaest tree whose CV cost does not exceed the cost of the minimum CV cost tree pus one standard error of this tree s CV cost [28]. 4. Experiments 4.1. Data Description In this section, we present a rea project to which we appy SVM for quaity prediction. The metrics data used for the appication represents the resuts of an investigation of software for a Medica Imaging System (MIS). It was from the data and too CD of the book [3] and widey disseminated. The tota system consists of approximatey 45 routines amounting to about 4 ines of code written in Pasca, FORTRAN, assemby, and PL/M. From the set of programs written in Pasca and FORTRAN, a random sampe of 39 routines was seected for anaysis. These routines incude approximatey 4 ines of code. The software was deveoped over a period of five years, and was in commercia use at severa hundred sites for a period of three years [6]. In MIS data set which was coected by Randy Lind [32], the number of changes made to a modue, docu-

6 LOC TComm Hastead s program ength McCabe s cycomatic compexity CL MChar Hastead s estimated program ength Beady s bandwidth metric TChar x DChar Jensen s estimator of program ength Figure 4. The reationship between the metrics the number of mented as Change Reports (), was used as an indicator of the number of fauts introduced during deveopment [31]. The changes made to the routines were anayzed, and ony those that affected the executabe code were counted as fauts (aesthetic changes such as comments were not counted). Fig. 4 shows the reationships between the number of and the software compexity metrics. We can find that the ow software compexity metric corresponds to the sma number of. It is a obvious case in rea project. In addition to the change data, the foowing 11 software compexity metrics were measured for each of the modues:

7 ff Tota ines of code incuding comments (LOC) ff Tota code ines (CL) ff Tota character count (TChar) ff Tota comments (TComm) ff Number of comment characters (MChar) 1 5 non faut prone faut prone ff Number of code characters (DChar) ff Hastead s program ength (N), where N = N 1 + N 2 and N 1 represents a tota operator count, and N 2 represents a tota operand count [33] x 1 4 ff Hastead s estimated program ength ( ˆN), where ˆN = η 1 og 2 η 1 + η 2 og 2 η 2,andη 1 and η 2 represent the unique operator and operand count, respectivey [33]. ff Jensen s estimator of program ength (N F ), where N F = og 2 η 1! + og 2 η 2![34]. ff McCabe s cycomatic compexity (v(g)), where v(g) =e n + 2, and e is the number of edges in a contro fowgraph representation of a program with n nodes [35]. ff Beady s bandwidth metric (BW), where BW 1 = n il i i and L i represents the number of nodes at eve i in a nested contro fowgraph of n nodes [34]. This metric is indicative of the average eve of nesting or width of the contro fowgraph representation of the program. In cassifying a modue to be faut-prone or non fautprone, there are two types of errors that can be made in the partition. A Type I error is the case where we concude that a program modue is faut-prone when in fact it is not. A Type II error is the case where we beieve that a program modue is non faut-prone when in fact it is faut-prone. We denote the types of errors as T1ERR and T2ERR, respectivey. Of the two types of errors, Type II error has more serious impications, since a product woud be seem better than it actuay is, and testing effort woud not be directed where it woud be needed the most. The nature of the impacts of these error types suggests that the Type II error rate is more important than the Type I error rate in considering the quaity of a cassification mode. In the experiment, we consider those modues with or 1 to be non faut-prone, and those with from 1 to 98 to be faut-prone. For the MIS data used in the experiment, there are 114 non faut-prone modues and 89 fautprone modues. The distribution of these software compexity metrics of MIS software in the first three principa Figure 5. Distribution of MIS data set in first three principa components space components (PCs) space is shown in Fig. 5. The bootstrap technique [36] is appied in the experiment. The tota 23 sampes are divided into two parts, where haf independent random sampes drawn from each cass are used to train the SVM cassifier, and the remaining haf sampes are used as test sampes to cacuate correct cassification rate (CCR). The experiment is repeated 25 times with different random partitions and the mean and standard deviation of the cassification accuracy are reported The Comparison of Severa Methods Quadratic discriminant anaysis (QDA) is a widey used cassification technique when sufficient training sampes coud be suppied. In the experiment, firsty QDA is used to cassify the origina data directy. Tabe 1 shows that for cassification directy using QDA, the CCR coud achieve 85.49%, and the Type I error and Type II error are 7.37% and 7.14%, respectivey. Principa components anaysis (PCA) [37] can aso be appied. Munson and Khoshgoftaar found that software compexity metrics are actuay inear combinations of a sma number of underying orthogona metric domains [38]. To reduce the interreated effect, we adopt PCA to transform the origina compexity metrics space into an orthogona vector space. The principe of PCA is simpe. Let us assume the data set has a covariance matrix Σ,whichisa rea symmetric matrix and can be decomposed as foows: Σ = UΛU T ; (25) where Λ is a diagona matrix with the eigenvaues λ 1 ;λ 2 ; ;λ m on its diagona, U is an orthogona matrix where coumn j is the eigenvector associated with λ j,and U T is the transpose of U. The m eigenvectors in U give the coefficients that define m uncorreated inear combinations

8 Eigenvaue Eigenvaue index Tabe 1. The comparison of severa methods appied to software quaity prediction Methods CCR Std T1ERR T2ERR QDA 85.49% % 7.14% PCA+QDA 86.53% % 7.52% PCA+CART 83.2% % 6.41% SVM 89.% % 8.67% PCA+SVM 89.7% % 8.87% TSVM 9.3% % 7.86% Figure 6. Eigenvaue in decreasing order of the origina compexity metrics. These orthogona inear combinations are the principa components of Σ. An eement u ij of U gives the coefficient of the ith compexity metric in the jth principe component. λ j gives the amount of compexity metric variance that is expained by the jth principe component. In principa components anaysis, the eigenvaues aong the diagona of Λ form a decreasing series that expains a of the compexity data variance; that is, λ 1 > λ 2 > > λ m,and m j=1 λ j gives the tota variance in the compexity metric data. The first few principa components typicay expain a arge proportion of the tota observed variance. Thus, restricting our attention to the first few principa components can achieve a reduction in dimensionaity with an insignificant oss of variance. From Fig. 6, we can find that the first two PCs have ony.23% reconstruction error, so we choose them to define a 2-dimensiona subspace and form 2-dimensiona vectors in this subspace. Then QDA is used to compete the cassification. The experimenta resuts are shown in Tabe 1, where we can find that the composite cassifier PCA+QDA achieves higher CCR 86.53% than QDA. Cassification and regression trees (CART) anaysis, a powerfu nonparametric approach in cassification or prediction, is aso empoyed. We use PCA de-correated data asinputofcart,and obtain83.2%ccr, which is ower than that of PCA+QDA. Consequenty Type II error aso decines compared with PCA+QDA, as shown in Tabe 1. Furthermore, we discuss the approach in buiding software quaity prediction mode using SVM. Firsty, the origina data are directy empoyed as the input to SVM. Considering the adaptive capacity of SVM, Radia Basis Function (RBF) is seected as the kerne function. From Tabe 1 we can see that direct cassification by SVM coud achieve 89.% CCR. According to the appied hypothesis tests (ttest), the SVM method achieves a mean of the cassification accuracy in the vaidation set, which is significanty (α=.5) arger than that of the QDA method (p-vaue is Tabe 2. The cassification accuracy comparison of the three kernes of SVM Kerne function CCR Std T1ERR T2ERR Poynomia 7.74%.28.45% 28.81% Radia basis 88.68% % 8.67% Sigmoid 88.75% % 8.96% equa to 9: ). But Type II error using SVM is higher than that of QDA. As a comparison, we aso engage the dimension reduced data with PCA as the input of SVM. However, according to the hypothesis tests (t-test) appied, the cassification resut is not significanty improved (pvaue is equa to.96). The reason is that SVM can simuate a non-inear projection which can map ineary inseparabe data into a higher dimension space where the casses are ineary separabe. So data space transform has itte infuence on SVM. TSVM is a new method derived from SVM. When appying it to buid the cassifier, we find that the best CCR of 9.3% is achieved. But it is more time consuming for TSVM training than other methods The Comparison of the Three Kernes of SVM For three kinds of commony-used kerne functions that are mentioned in Section 2.3, we compare their performance in the experiment aso. Tabe 2 shows the experimenta resut, where we can ceary see that Radia Basis kerne function and Sigmoid kerne function a achieve exceent performance, which are significanty better than Poynomia kerne function Experiments with the Minimum Risk In practica case, Type II error often causes more serious consequence than Type I error does, so it is necessary to reduce Type II error. In Section 2.4, we introduce the

9 Tabe 3. SVM with the risk feature C 1 C 2 CCR Std T1ERR T2ERR % % 5.1% % % 7.52% % % 8.67% % % 8.87% % % 8.87% Tabe 4. The Bayesian decision with the minimum risk Risk ratio CCR Std T1ERR T2ERR 1: % % 6.47% 1: % % 5.14% 1: % % 3.37% 1: % % 1.57% 1: % %.27% SVM theory combined with error risk contro. In this experiment, we appy the theory to adjust Type II error. Tabe 3 shows the experimenta resut. When C 1 and C 2 are equa to 2, 89.7% CCR is achieved, which is the optima soution of SVM. But in this case Type II error is much arger than the Type I error. However, we can adjust C 1 to reduce Type II error. From Tabe 3, we find that when C 1 is reduced graduay, CCR aso reduces, but this eads to the increase of Type I error. When C 1 is reduced to 5, CCR is changed to 86.53%, where ow Type II error 5.1% is achieved. As a comparison, we aso empoy the Bayesian decision with the minimum risk to buid a quaity prediction mode. The resuts are shown in Tabe 4, where the first coumn is the risk ratio of Type I error versus Type II error. We can observe that SVM with risk is superior to the Bayesian decision based on the minimum risk in the cassification performance. Nevertheess, the Bayesian decision can adjust type II error to a very ow vaue at the cost of ower CCR and higher Type I error. 5. Concusions A nove technique for software quaity prediction is proposed in this paper. This methodoogy is based on support vector machine. The vaidity and robustness of SVM make it a meaningfu too for rea-word appications in software quaity prediction. Firsty, SVM is adaptive to modeing noninear functiona reationships which are difficut to mode with other techniques. Secondy, SVM generaizes we even in high dimensiona spaces under sma training sampe conditions. Consequenty, software quaity prediction modes can be buit much earier with SVM than other conventiona techniques. Furthermore, as a new approach, Transductive SVM (TSVM) takes into account particuar test sampes as we as training sampes in buiding cassifiers. Thirdy, the SVM-based software quaity prediction mode achieves a reativey good performance. According to the hypothesis tests (t-test) appied, the cassification resuts of the SVM are better than those of either QDA or a cassification tree. Finay, we can contro Type II error by adjusting the error penaty parameter C of SVM, which can achieve better cassification performancethan using the minimum-risk-based Bayesian decision when considering both CCR and Type II error. To summarize, a the above characteristics show that the new approach which has never been expored in the software engineering fieds offers a very promising technique in software quaity prediction. We beieve that our method can be extensivey appied in many software engineering fieds. Acknowedgements This research work described in this paper was fuy supported by a grant from the Nationa Natura Science Foundation of China (Project No ) and a grant from Hong Kong Research Grants Counci (Project No. CUHK425/4E). References [1] N. E. Fenton and S. L. Pfeeger, Software metrics: a rigorous and practica approach, 2nd ed. Boston, MA: PWS Pubishing Co., [2] V. Y. Shen, T. J. Yu, S. M. Thebaut, and L. R. Pausen, Identifying error-prone software an empirica study, IEEE Transactions on Software Engineering, vo. SE-11, no. 4, pp , apr [3] T. M. Khoshgoftaar and E. B. Aen, Cassification of faut-prone software modues: Prior probabiities, costs, and mode evauation, Empirica Software Engineering, vo. 3, no. 3, pp , September [4] L. C. Briand, V. R. Basii, and C. Hetmanski, Deveoping interpretabe modes for optimized set reduction for identifying high-risk software components, IEEE Transactions on Software Engineering, vo. SE-19, no. 11, pp , [5] J. Munson and T. Khoshgoftaar, The detection of fautprone programs, IEEE Transactions on Software Engineering, vo. SE-18, no. 5, pp , [6] T. Khoshgoftaar and J. Munson, Predicting software deveopment error using software compexity metrics, IEEE Transactions on Software Engineering, vo. 8, no. 2, pp , 199. [7] T. M. Khoshgoftaar, Improving usefuness of software

10 quaity cassification modes based on booean discriminant functions, in Internationa Symposium on Software Reiabiity Engineering, (ISSRE 2), 22, pp [8] T. M. Khoshgoftaar, R. Shan, and E. B. Aen, Improving tree-based modes of software quaity with principa components anaysis, in Internationa Symposium on Software Reiabiity Engineering, (ISSRE ), 2, pp [9] T. M. Khoshgoftaar, V. Thaker, and E. B. Aen, Modeing faut-prone modues of subsystems, in Internationa Symposium on Software Reiabiity Engineering, (ISSRE ), 2, pp [1] L. C. Briand, V. R. Basii, and W. M. Thomas, A pattern recognition approach for software engineering data anaysis, IEEE Transactions on Software Engineering, vo.se- 18, no. 11, pp , [11] P. Guo and M. R. Lyu, Software quaity prediction using mixture mode with em agorithm, in Proceedings of the First Asia-Pacific Conference on Quaity Software,(APAQS 2), Hong Kong, 2, pp [12] T. M. Khoshgoftaar, D. L. Lanning, and A. S. Pandya, A comparative study of pattern recognition techniques for quaity evauation of teecommunications software, IEEE Journa on Seected Areas in Communications, vo. 12, no. 2, pp , [13] L. Guo, Y. Ma, B. Cukic, and H. Singh, Robust prediction of faut-proneness by random forests, in Internationa Symposium on Software Reiabiity Engineering, (ISSRE 4), 24, pp [14] I. Myrtveit, E. Stensrud, and M. Shepperd, Reiabiity and vaidity in comparative studies of software prediction modes, IEEE Transactions on Software Engineering, vo. 31, no. 5, pp , 25. [15] C. Cortes and V. Vapnik, Support-vector network, Machine Learning, vo. 2, pp , [16] S. Dumais, Using svms for text categorization, IEEE Inteigent Systems, vo. 13, no. 4, pp , [17] M. Ponti and A. Verri, Support vector machines for 3d object recognition, IEEE Transactions on Pattern Anaysis & Machine Inteigence, vo. 2, no. 6, pp , [18] H. Drucker, D. Wu, and V. N. Vapnik, Support vector machines for spam categorization, IEEE Transactions on Neura Networks, vo. 1, no. 5, pp , [19] F. Xing and P. Guo, Cassification of stear spectra data using svm, in Internationa Symposium on Neura Networks (ISNN 24), ser. Lecture Notes in Computer Science, vo Berin, Heideberger: Springer-Verag, 24, pp [2] V. N. Vapnik, The Nature of Statistica Learning Theory. New York: Springer-Verag, [21] T. Joachims, Transductive inference for text cassification using support vector machines, in Proceedings of the 16th Internationa Conference on Machine Learning, (ICML 99), Bed, Sovenia, 1999, pp [22] T. Joachims, Ed., Learning to Cassify Text using Support Vector Machines: Methods, Theory, and Agorithms. Kuwer, may 22. [23] S. Aeberhard, D. Coomans, and V. O. de, Comparative anaysis of statistica pattern recognition methods in high dimensiona settings, Pattern Recognition, vo. 27, no. 8, pp , [24] X. Wang, F. Xing, and P. Guo, Comparison of discriminant anaysis methods appied to stear data cassification, in Third Internationa Symposium on Mutispectra Image Processing and Pattern Recognition (MIPPR 3), ser. Proceedings of SPIE, vo Beingham, WA: The Internationa Society for Optica Engineering, 23, pp [25] K. Fukunaga, Introduction to Statistica Pattern Recognition, 2nd ed. Boston: Academic Press, 199. [26] T. M. Khoshgoftaar and E. B. Aen, Predicting faut-prone software modues in embedded systems with cassification trees, in Proceedings of Fourth Internationa Symposium on High-Assurance Systems Engineering, Washington, DC USA, 1999, pp [27] A. P. Worth and M. T. D. Cronin, The use of discriminant anaysis, ogistic regression and cassification tree anaysis in the deveopment of cassification modes for human heath effects, Journa of Moecuar Structure(Theochem), vo.622, pp , 23. [28] L. Breiman, J. H. Friedman, R. A. Oshen, and C. J. Stone, Cassification and Regression Trees. Caifornia: Wadsworth, Inc., [29] L. W. Y. and S. Y. S., Spit seection methods for cassification trees, Statistica Sinica, vo. 7, pp , [3] M. R. Lyu, Ed., Handbook of Software Reiabiity Engineering. IEEE Computer Society Press and McGraw-Hi Book Company, [31] V. R. Basii and D. H. Hutchens, An empirica study of a syntactic compexity famiy, IEEE Transactions on Software Engineering, vo. SE-9, no. 6, pp , nov [32] R. K. Lind, An experimenta study of software metrics and their reationship to software error, Master s thesis, University of Wisconsin-Miwaukee, Miwaukee, Dec [33] M. H. Hastead, Eements of Software Science. NewYork: Esevier, [34] H. Jensen and K. Vairavan, An experimenta study of software metrics for rea-time software, IEEE Transactions on Software Engineering, vo. SE-11, no. 2, pp , feb [35] T. J. McCabe, A compexity measure, IEEE Transactions on Software Engineering, vo. SE-2, no. 4, pp , [36] B. Efron and R. J. Tibshirani, An introduction to the Bootstrap, Chapman and Ha, [37] I. Joiffe, Principa Component Anaysis. New York: Springer-Verag, [38] J. C. Munson and T. M. Khoshgoftaar, The dimensionaity of program compexity, in Proc. 11th Internationa Conference on Software Engineering, Pittsburgh, 1989, pp