Index Terms-Software effort estimation, principle component analysis, datasets, neural networks, and radial basis functions.

Transcription

1 ISO 9001:2008 Certfed Internatonal Journal of Engneerng and Innovatve Technology (IJEIT The Effect of Dmensonalty Reducton on the Performance of Software Cost Estmaton Models Ryadh A.K. Mehd College of Informaton Technology Abstract- Estmatng software efforts s very mortant to a roject manager as the cost and schedule for comletng the software can be determned. Buldng these models requres develong a sound comutatonal model as well as dentfyng the arorate set of roject features that accurately redct the software effort. Many data sets have been exermented wth n buldng these models. These data sets dffer n the features set used. One roblem whch has been cted wth these data sets s that they contan sgnfcantly correlated or statstcally not sgnfcant redctor varables. Choosng the rght set of features can have a strong mact on the overall redctve ower of the model. Ths aer nvestgates the effectveness of usng rncle comonent analyss as a mechansm to elmnate correlated or not sgnfcant arameters on the accuracy of estmatng software effort. The set of exerments conducted n ths work on software cost estmaton do not rovde conclusve evdence that dmensonalty reducton of the ublcally avalable data sets. The results obtaned also ndcate that the accuracy of the models are drectly related to the attrbutes that consttute the data set.another fndng of ths work s that the Chna dataset rovded the most accurate redctons usng a radal bass functon neural network based software effort estmaton model. Index Terms-Software effort estmaton, rncle comonent analyss, datasets, neural networks, and radal bass functons. I. INTRODUCTION Estmatng software develoment efforts s one of the crtcal tasks n managng software develoment rojects. Predctng software develoment effort wth hgh accuracy s of aramount mortance for roject managers. However, estmatng software develoment effort s stll a challengng roblem and one that attracts consderable research. There are two requrements for buldng an accurate software effort estmaton model. Frst, a robust comutatonal model must be develoed, and secondly a relevant set of software roject features set must be dentfed n addton to the qualty of data beng collected[1]. Wth regard to the frst requrement numerous software effort estmaton models have been develoed [2, 3, 4, 5]. The conventonal models use a mathematcal formula to redct roject cost based on the estmates of arameters such as roject sze measured n lnes of source code or functon onts, number of software engneers, and other rocess and roduct attrbutes [6]. Among the software cost estmaton technques, COCOMO (Constructve Cost Model s the most commonly used algorthmc cost modelng technque because of ts smlcty for estmatng the effort n erson-months for a Ajman Unversty of Scence & Technology roject at dfferent stages. COCOMO uses a mathematcal formula to redct roject cost estmaton [7].n algorthmc models of cost estmaton such as artfcal neural networks (ANN have also been used. Artfcal neural networks are good at modelng comlex nonlnear relatonshs. They are massve arallel dstrbuted rocessor made u of smle rocessng unts, whch are caable of storng exermental knowledge and makng t avalable for use[6]. An ANN resembles the bran n two resects [6]: 1 Knowledge s acqured from ts envronment through a learnng rocess,2 Interneuron connecton weghts are used to store the knowledge. As far as the second requrement of buldng a sound software effort estmaton model, many datasets have been used for ths urose. These data sets dffer n the arameters and attrbutes they use n descrbng the software roject features that may be used n buldng a cost estmaton model. One roblem whch has been cted wth these data sets s that they contan sgnfcantly correlated or statstcally not sgnfcant redctor varables [8]. Best estmaton models are those n whch the redctor varables each correlate hghly wth the deendent varable but correlate at most only mnmally wth each other, otherwse the redundancy wll result n overfttng of the redcton model [9]. In statstcs and machne learnng, overfttng generally occurs when a model s excessvely comlex, such as havng too many arameters relatve to the number of observatons. A model that has been overft begns to "memorze" tranng data rather than "learnng" to generalze from trend [10].Thus, choosng the rght set of features can have a strong mact on the overall redctve ower of a software effort estmaton model. Ths aer nvestgates the effectveness of usng rncle comonent analyss to elmnate correlated or not sgnfcantarameters on the accuracy of estmatng software effort usng a radal bass functon neural network. II. RESEARCH METHODOLOGY Neural networks have been shown to outerform other aroached n redctng software effort [1, 2, and 12]. The most commonly used neural network archtecture s the back roagaton traned multlayered feed forward networks wth sgmodal actvaton functon [11]. Each neuron n a MLP takes the weghted some of ts nut values. That s, each nut value s multled by a coeffcent, and the results are all summed together. A sngle MLP neuron s a smle lnear classfer, but 68

2 ISO 9001:2008 Certfed Internatonal Journal of Engneerng and Innovatve Technology (IJEIT comlex non-lnear classfers can be bult by combnng these neurons nto a network. Fg 1. Radal Bass Functon Neural Network Another tye of ANN that has been used n the lterature s the radal bass functons neural network (RBFNN [12]. A RBFNN can aroxmate any functon whch makes t sutable to model the relatonsh between nuts (the varous cost drvers and outut (effort requred. RBFNN erforms classfcaton by measurng the nut s smlarty to examles from the tranng set. Each RBFNN neuron stores a rototye, whch s just one of the examles from the tranng set. To classfy a new nut, each neuron comutes the Eucldean dstance between the nut and ts rototye. In other words, f the nut more closely resembles class A rototyes than the class B rototyes, t s classfed as class A.It has been show that RBFNN erform better than other tyes of neural networks based models [13]. In ths aer we wll use RBFNN to examne the effect of rerocessng datasets wth on the accuracy of software effort estmaton models. RBFNN Neural Network Imlementaton The followng generc descrton of a RBF neural network s based on a tutoral gven n [12].Fgure 1 descrbes a tycal archtecture of an RBFNN. It conssts of an nut vector, a layer of RBF neurons, and an outut layer wth one node er category or class of data. The nut vector s the m-dmensonal vector to be classfed. The hdden layer conssts of neurons where each one stores a rototye vector whch s just one of the vectors from the tranng set. Each RBFNN neuron comares the nut vector to ts rototye, and oututs a value between 0 and 1 whch s a measure of smlarty. If the nut s equal to the rototye, then the outut of that RBFNN neuron wll be 1. As the dstance between the nut and rototye grows, the resonse falls off exonentally towards zero. The neuron s resonse value s also called ts actvaton value. The rototye vector s also often called the neuron s center, snce t s the value at the center of the bell curve. The outut of the network conssts of a set of nodes, one er category to be classfed or a value to be comuted. Each outut node comutes a sort of score for the assocated category. Tycally, a classfcaton decson s made by assgnng the nut to the category wth the hghest score. The score s comuted by takng a weghted sum of the actvaton values from every RBF neuron. Because each outut node s comutng the score for a dfferent category, every outut node has ts own set of weghts. The outut node wll tycally gve a ostve weght to the RBF neurons that belong to ts category, and a negatve weght to the others. Each RBFNN neuron comutes a measure of the smlarty between the nut and ts rototye vector (taken from the tranng set. Inut vectors whch are more smlar to the rototye return a result closer to 1. There are dfferent ossble choces of smlarty functons, but the most oular s based on the Gaussan functon. Equaton 1 descrbe the formula for a Gaussan wth a one-dmensonal nut. Where x s the nut, µ s the mean, and s the standard devaton. Ths roduces the famlar bell curve shown below n Fgure 2, whch s centered at the mean,. In the Gaussan dstrbuton, refers to the mean of the dstrbuton. Here, t s the rototye vector whch s at the center of the bell curve. Fg 2. A Gaussan functon wth µ=5, and =1. For the actvaton functon, (, the standard devaton,, s not mortant and the followng two smlfyng modfcatons can be made.the frst modfcaton s to remove the outer coeffcent, 1 / ( * sqrt (2 * π. Ths term normally controls the heght of the Gaussan curve. Here, though, t s redundant wth the weghts aled by the outut nodes. Durng tranng, the outut nodes wll learn the correct coeffcent or weght to aly to the neuron s resonse.the second change s to relace the nner coeffcent, 1 / (2 * 2, wth a sngle arameter β whch controls the wdth of the bell curve. Agan, n ths context, the value of s not n tself mortant and what s 69

3 ISO 9001:2008 Certfed Internatonal Journal of Engneerng and Innovatve Technology (IJEIT needed s some coeffcent that controls the wdth of the bell curve. Thus, after makng these two modfcatons, the RBFNN neuron actvaton functon can be wrtten as n equaton 2 [12]: There s also a slght change n notaton here when we aly the equaton to n-dmensonal vectors. The double bar notaton n the actvaton equaton ndcates that we are takng the Eucldean dstance between x and mu, and squarng the result. For the 1-dmensonal Gaussan, ths smlfes to just (x µ 2. Maxwell (27 attrbute, 62 roject, effort n functon onts Albrecht (8 attrbute, 24 roject, effort n ersonhours Attrbutes that reresent dentfcaton numbers had been removed from the datasets n ths work. For more detals on these datasets, see [15]. The evaluaton crteron used to assess the redcton accuracy obtaned from the dfferent datasets s the mean magntude error (MME and the mean magntude relatve error (MMRE comuted from the formulae 3 and 4: 1 MME n n 1 1 MMRE n n 1 redctedeffort( redctedeffort( ( (4 Fg 3. Actvaton Functon for dfferent values of beta. It s mortant to note that the underlyng metrc here for evaluatng the smlarty between an nut vector and a rototye s the Eucldean dstance between the two vectors. Also, each RBF neuron wll roduce ts largest resonse when the nut s equal to the rototye vector. Ths allows to take t as a measure of smlarty, and sum the results from all of the RBFNN neurons. As we move out from the rototye vector, the resonse falls off exonentally. Every RBF neuron n the network wll have some nfluence over the classfcaton decson. The exonental fall off of the actvaton functon, however, means that the neurons whose rototyes are far from the nut vector wll actually contrbute very lttle to the result [12]. Prncle Comonent Analyss Prncal comonents analyss ( s normally used for two objectves: reducng the number of features used n a dataset whle retanng the varablty n the data, and dentfyng hdden atterns n the data, and classfyng them accordng to how much of the nformaton, stored n the data, they account for [14]. In ths work has been used to elmnate correlated attrbutes and examne the effect on estmaton model accuracy. Software Effort Estmaton Datasets The followng datasets are used n ths research: Desharnas (11 attrbute, 77 roject, effort n erson-hours Cocomo81 (18 attrbute, 61 roject, effort n erson-month Chna (19 attrbute, 499 rojects, effort n ersonhours The measure redcton level PRED( s also used as comlementary crteron to count the ercentage of rojects that have MMRE of or less.a value 0.25 for s commonly used to comare software cost estmaton models[16]. III. RESULTS AND TABLES MATLAB neural network toolbox was used to mlement the RBFNN usng newrb functon: Net=newrb(P, T, goal, sread, MN,DF, where P s am n matrx of nut vector, m s the number of cost drvers for each roject, andn s the number of rojects used n the tranng hase. T s 1 n vector of actual efforts for each roject used n the tranng hase. Goal: mean squared errors, s used. Sread: A value between 1.0 and 3.0 s used. MN: maxmum number of neurons (default n DF: number of neurons to add between dslays, 2 s used. Fg 4. Actual, estmated, and estmated efforts for the Albrecht data set. The MATLAB functon rocessca( has been used to rerocess datasets usng. RBFNN was aled to the 70

4 ISO 9001:2008 Certfed Internatonal Journal of Engneerng and Innovatve Technology (IJEIT datasets lsted above wth and wthout alyng rncle comonent analyss as a mechansm to reduce the dmensonalty of data and reduce correlaton. Fgures 4 to 8 shows the redctons obtaned wth and wthout alyng and the actual effort for the Albrecht, Desharnas, Cocomo81, Maxwell, and Chna datasets resectvely. may also be attrbuted to the large number of rojects (499 roject contaned n the data set. Fg 7. Actual, estmated, and estmated efforts for the Maxwell data set. Fg 5. Actual, estmated, and estmated efforts for the Desharnas data set. Lookng nto the results, t s clear that there s no conclusve evdence that alyng has a ostve mact on the redcton accuracy of neural networks based software efforts estmaton models. Fgures 4 to 8 also ndcate that redcton accuracy s deendent to a large extent on the dataset beng used and consequently the set of features used to descrbe software rojects. Fg 8. Actual, estmated, and estmated efforts for the Chna data set. Data Set MME MMRE PRED(0.25 Albrecht % 41.9% 42.8% 42.8% Desharnas % 101% 30.4% 39.1% Cocomo % 466% 0 0 Maxwell % 86% 47.4% 36.8% Chna % 18.5% 80% 83% Table 1. Performance statstcs of datasets used. Fg 6. Actual, estmated, and estmated efforts for the Cocomo81 data set. The comuted mean absolute error, mean relatve absolute error, and PRED (0.25 are shown n Table 1. It s evdent from the redcton results and the statstcs of Table 1 that the Chna dataset rovded the most accurate redcatons comared to the other data sets. Ths accuracy IV. CONCLUSIONS The conclusons to be drawn from ths the set of exerments conducted n ths work on software cost estmaton do not rovde a conclusve evdence that dmensonalty reducton of the ublcally avalable data set used n buldng software cost estmaton models can hel wth the accuracy of these models. The results obtaned also ndcate that the accuracy of the models are 71

5 ISO 9001:2008 Certfed Internatonal Journal of Engneerng and Innovatve Technology (IJEIT drectly related to the attrbutes chosen by the data set. Accordng to the work conducted n ths research, the Chna data set rovded the most accurate results from among the data sets used n ths work. Future work wll nvestgate the mortance of the varous attrbutes n exlanng the sze of efforts requred so that standardzed set of attrbutes can be used n collectng data sets. REFERENCES [1]. B. Ktchenham, and E. Mendes, Why comaratve effort redcton studes may be nvald, 5 th Internatonal conference on software redctors PROMISE 09, Vancouver, Canada, ACM Internatonal Conference Proceedngs, Seres archve, Artcle number 4. [2]. S. A. Abbas, A. R. Lao, A. Azam, Cost Estmaton: A Survey of Well-Known Hstorc Cost Estmaton Technques, Journal of Emergng Trends n Comutng and Informaton Scences, vol. 4, no. 1, , [3]. R. Malhotra, A. Jan, Software Effort Predcton usng Statstcal and Machne Learnng Methods, Internatonal Journal of Advanced Comuter Scence and Alcatons, vol. 2, no. 1, [4]. Abbas Heat, Comarson of Artfcal Neural Network and Regresson Models for Estmatng Software Develoment Effort, Informaton and Software Technology, vol. 44, , [5]. R. Sharma, Survey: n Algorthmc Models for Estmatng Effort, Euroean Internatonal Journal of Scence and Technology, vol. 2, no. 3, [6]. Kaushk, A. Chauhan, D. Mttal, and S. Guta, COCOMO Estmates Usng Neural Networks, Internatonal Journal of Intellgent Systems and Alcatons, vol. 9, , [7]. Reddy, and K. Raju, An Otmal Neural Network Model for Software Effort Estmaton, Internatonal Journal of Software Engneerng, vol.12 no.1, [8]. P. Reddy, K. R. Sudha, and C. Ramesh, Software Effort Estmaton usng Radal Bass and Generalzed Regresson Neural Networks, Journal of Comutng, vol. 2, no. 5, [9]. B. S. Evertt, Cambrdge Dctonary of Statstcs, 2 nd ed., Cambrdge Unversty Press, [10]. V. Tetko, D. J. Lvngstone, A. I. Luk, "Neural network studes. 1. Comarson of Over fttng and Overtranng, Journal of Chemcal Informaton and Comuter Scences, vol 35, no. 5, , [11]. C. Reddy, and K. Raju, A Concse Neural Network Model for Estmatng Software Effort, Internatonal Journal of Recent Trends n Engneerng, vol. 1, no. 1, , [12]. Lu, Radal Bass Functon (RBF Neural Network Control for Mechancal Systems. Srnger, [13]. E. Praynln, and P. Latha, Performance Analyss of Software Effort Estmaton Models Usng Radal Bass Functon Network, Internatonal Journal of Comuter, Informaton, Systems and Control Engneerng, vol. 8, no. 1, [14]. T. W. Anderson, Prncal Comonents: An Introducton to Multvarate Statstcal Analyss, 3 rd Edton, , Wley, [15]. S. J. Shrabad, and T. J. Menzes, The PROMISE Reostory of Software Engneerng Databases, School of Informaton Technology and Engneerng, Unversty of Ottawa, Canada. Avalable: htt://romse.ste.uottawa.ca/sereostory, [16]. S. Conte, H. Dunsmore, and V. Shen, Software Engneerng, Metrcs and Models. Benjamn/Cummngs,