Avalable onlne at www.jpe-onlne.com vol. 13, no. 7, November 2017, pp. 1101-1110 DOI: 10.23940/jpe.17.07.p13.11011110 Software Trustworthness Statc Measurement Model and the Tool Yan L, Zhqang Wu, Yxang Chen* Shangha Key Laboratory of Trustworthy Computng, East Chna Normal Unversty, Shangha, 20062, Chna Abstract Software trustworthness has become one of the promnent studes n software qualty assurance, n whch the trustworthness measurement s the prmary topc. Compared wth the method to evaluate the software development process, we measure to what extent the entty of software better fts users requrement. In ths paper, we propose a bottom-up method of software trustworthness measurement based on the source code. Frst, for the trustworthness measurement of attrbutes, a comprehensve model s proposed. Second, the valdty and stablty of the model are verfed by Monte Carlo smulaton. Fnally, the proposed method s developed based on the open source statc detecton tool for Cppcheck, whch forms the software trustworthness statc measurement tool for TSMT. Keywords: software trustworthness; trustworthness measurement; cppcheck (Submtted on July 25, 2017; Revsed on August 30, 2017; Accepted on September 15, 2017) (Ths paper was presented at the Thrd Internatonal Symposum on System and Software Relablty.) 2017 Totem Publsher, Inc. All rghts reserved. 1. Introducton Due to the gradual expanson of software, software defects are growng and software qualty s dffcult to predct and control. The avalablty and trustworthness of key doman applcaton are especally dffcult to guarantee for software. These are the mportant problems n the feld of embedded system and put forward hgher requrements for the software trustworthness [4]. How to ensure software trustworthness s the core and dffcult scentfc problem n software engneerng feld [11]. To solve ths problem, we frst need to solve the relevant work about how to measure the software trustworthness. Amng at the problems of hgh complexty, large scale and uncertan demand for the feld of embedded system, Me [7] puts forward a method to compute the trustworthness of the attrbute based on the agng problem of trusted evdence, and constructs the trusted evaluaton system of the CPS (Cyber-Physcal Systems) n combnaton wth the weght dstrbuton to construct method for software trustworthness. Y Huang [2] puts forward a trustworthness method based on evdence reasonng for embedded software. Snce trustworthness metrcs s supported by related evdence, the theory of trustworthness evaluaton based on evdence s unversally recognzed n the process of software development. C-language programs commonly used n the feld of embedded system, ths paper puts forward a statc measurement model of software trustworthness based on untrusted evdence. We put forward the defnton of the untrusted evdence. Accordng to the dfferent data types for the untrusted evdence, we come up wth a dfferent model of the trustworthness rank metrc, whch corresponds to the trusted attrbute, and the number of the untrusted evdence of the attrbute n the CWE (Common Weakness Enumeraton) flaws storehouse. The attrbute s classfed by a trusted level, and then a metrc model s presented based on the untrusted evdence. The valdty and stablty of the model are verfed by Monte Carlo smulaton. In ths paper, the software metrcs tool TSMT based on Cppcheck s desgned and mplemented, whch can call Cppcheck to nspect the program. The extended nterface based on Cppcheck provdes the custom rules on the bass of the * Correspondng author. E-mal address: yxchen@se.ecnu.edu.cn
1102 Yan L, Zhqang Wu, and Yxang Chen orgnal nspecton rules, so that the tool can detect more untrusted evdence. The tool mplements the model proposed n ths paper, and perfects the software trustworthness metrc system based on the untrusted evdence. The rest of the paper s organzed as follows. Secton 2 manly ntroduces the untrusted evdence and ts trusted analyss method, whch lays the foundaton for the attrbute trustworthness measurement. Secton 3 ntroduces a knd of metrc model for trustworthness attrbute based on trusted and untrusted evdence, and weghts the exstng metrc model for software trustworthness based on attrbute, and verfes the valdty of the model by smulaton. Secton 4 ntroduces metrc tool for the software trustworthness called TSMT based on Cppcheck. The last secton presents the conclusons and looks forward to the next research work. 2. Untrusted Evdence 2.1. Defnton of untrusted Evdence In ths paper, we use the defnton of software trustworthness n lterature [3]. Accordng to the defnton, we get the defnton of untrustworthness from the reverse thnkng. Software untrustworthness refers to devatng from the user s expectatons for the dynamc behavor and result of the software system, or cannot provde contnuous servce when sufferng from nterference [5]. Untrusted evdence refers to a program element or program unt hdden n the source code that leads to software untrustworthness, as shown n the sx-tuple: Evdence = < Descrpton, Type, Property, T-Value,T-Level,Attrbutes > In the above sx-tuple, Descrpton s the descrpton of the evdence; Type s the data of the evdence; Property ndcates the trustworthness of the evdence to be satsfed; T-value ndcates the measure value of the untrusted evdence; T-level represents the level of trustworthness for evdence; Attrbutes ndcates a trusted attrbute that represents the effect of evdence. The program elements and the program unts n the defnton refer to the relevant defntons n [9]. It s beleved that program elements are varables, constants, data types, algorthmc expressons, logcal expressons, empty statements, assgnment statements, sequental statements, condtonal statements, crcular statements, and declaratons of varables. The effect of untrusted evdence on software trustworthness can be decomposed nto one or more trusted attrbutes. Accordng to the lst of the program elements, program unts and attrbutes correspondng to the work of Tao et al. [10]. We construct the correspondng relatonal table of attrbutes related to each pece of untrusted evdence n Table 1. Table 1. Untrusted evdence wth attrbutes for correspondng lst Untrusted evdence Trustworthness attrbute Improper UI Functonalty, Mantanablty Improper break Functonalty, Relablty Data overflow Functonalty, Survvablty Dvson by zero Functonalty, Relablty Dmenson unty Functonalty, Mantanablty Logcal error Functonalty, Mantanablty Improper process schedulng Functonalty, Relablty Hgh complexty Relablty, Mantanablty Lack annotaton Mantanablty A set of untrusted evdence may affect a number of software propertes at the same tme, and we only gve the trustworthy evdence to affect the man attrbutes. Users can expand on the bass of actual needs. 2.2. Classfcaton Our goal s based on the mpact of untrusted evdence for software. Accordng to the extent to whch the software s affected by untrusted evdence, we dvde the trustworthy level of the untrusted evdence from hgh to low. When the trustworthness level become much hgher, t s more dffculty to mprove. Therefore, the hgher the trustworthness level, the much strcter the quantty and nfluence degree of the untrusted evdence s requred. So we propose a classfcaton model of untrusted evdence. In the gradng model of untrusted evdence, classfcaton range s not equal. It approxmates the rato of gold dvson to decrease when we ncrease the value nterval of a trusted level from the lowest. The dvson contents are shown n Table 2. The method of calculatng the rato of gold dvson s as follows.
Software Trustworthness Statc Measurement Model and the Tool 1103 2.3. Trusted Analyss Model 0. 45 5 1) / 2 0. 25, 0. 25 ( 5 1) / 2 0. 15, 0. 15 ( 5 1) / 2 0. 10, 0. 10 ( 5 1) / 2 0. 05. Table 2. The classfcaton of trustworthness of untrusted evdence Trustworthness Level Defnton Trustworthness Degree V Totally trustworthy 1.0 IV Trustworthy 0.95 III Partally trustworthy 0.85 II None trustworthy 0.70 I Software cannot run or run ncorrectly 0.45 There are many types of untrusted evdence n the procedure, and a sngle trustworthness analyss method can hardly cover all the untrusted evdence. Therefore, the data types of the untrusted evdence are dvded nto two knds: the Boolean type (represented by "B"), the numerc type (expressed n "D"). The measure dstrbuton of Boolean untrusted evdence s a dscrete dstrbuton, and t's only 2 measures to "1" or "0". Generally, the "0" ndcates that the untrusted evdence does not satsfy the trustworthness, and the "1" ndcates that the untrusted evdence satsfes the trustworthy nature. When the untrusted evdence satsfes the trustworthy nature, the procedure statement represented by the evdence s trustworthness. So the measure value of the untrusted evdence s 1, and the level of the untrusted evdence s the maxmum level whch can be obtaned by the type of the change. The procedural statement represented by the evdence s not trustworthness when the untrusted evdence doesn t satsfy the trustworthy nature. So the measure value of the untrusted evdence s 0, and the untrusted evdence s Grade I. The maxmum level here s not necessarly the maxmum level n a herarchy, but the maxmum degree of trustworthness that the evdence of untrustworthness can acheve. An example of untrusted evdence show " f-else statements do not match " the form of Table 3. Table 3. Untrusted evdence " f-else statements do not match " sample Descrpton Type Property T-Value T-Level Attrbute If-else statement msmatch B Whether the condtonal Combnaton statement matches 0 3 functonal The model s used to denote the level of Boolean untrusted evdence. 1, f Evdence. Performance Evdence. Property T Value 0, f Evdence. Performance Evdence. Property hghest _ level, f T value 1 T Level I, f T value 0 The value of numercal model of untrusted evdence s number, whch can be contnuous or dscrete. We dvde nto dfferent grades n the metrcal value range correspondng dfferent nterval. For example, what's the complexty about loop n the untrusted evdence. Accordng to MACCABA, [6] propose the relatonshp between the cyclomatc complexty and error; we dvde the value range of the cyclc complexty n Table 4. We gve an example for untrusted evdence n Table 5 f the complexty of the loop s 20. In ths example, although the metrc s dvded nto four ntervals, f the trustworthness of the complexty s level IV and also satsfes the V-level defnton, the trustworthness of untrusted evdence s stll level IV. Table 4. Relatonshp between cyclc complexty and error rate Complexty of loop Error rate Trusted level 1-10 5% IV 20-30 20% III >50 40% II >100 60% I
1104 Yan L, Zhqang Wu, and Yxang Chen Table 5. Untrusted evdence " complexty of loop " sample Descrpton Type Property T-Value T-Level Attrbute Complexty of Loop s 20 D Whether complexty of Loop s hgh 20 3 Relablty Mantanablty Ths s the model to represent the grade judgment of the numercal type of untrusted evdence. T-Value correspondng nterval level, f T value fallswthnnterval T Level T Valueloweradjacent nterval level, f T value notnany nterval hghest_level, f T-value better than all nterval optmal values Note: The hghest_value here s not the hghest level of untrustworthness, but the hghest level of evdence of untrustworthness. 2.4. The statstc of the evdence of CWE CWE (Common weakness enumeraton) s a software communty project desgned to create a software flaw enumeraton class to better understand software bugs and create automated tools to dentfy, fx, and prevent such defects. For common software flaws, the CWE organzed a lst of more than 1500 dfferent vulnerablty samples n the real world, and form an artcle for PLOVER [1] to publc use. From ths artcle, we sorted out 25 knds of software flaws and ncluded 258 flaws entry. The defect refers to the software code n the error or vulnerable to vulnerable weaknesses, and belongs to the category of untrusted evdence. Accordng to the detaled descrpton of PLOVER, each type of software flaws s mapped to software attrbutes, and the correspondng untrusted evdence s gven. Accordng to the data of untrusted evdence, we respectvely gve model of untrusted evdence classfcaton. Fnally, we apply the model of untrusted evdence to flaws lbrary of the CWE. We gve a relatonal table of trustworthness attrbute for each CWE flaw n Table 6. Table 6. Relatonal table of trusted attrbute for each CWE flaw Sort Descrpton Trustworthness attrbute BUFF Buffer overflow Functonalty, Relablty, Survvablty SVM Structure problems Functonalty, Mantanablty, Survvablty SPEC Specal element ssues Functonalty, Mantanablty SPECM Operatonal problems of Functonalty, Relablty, Mantanablty specal elements PATH Traversal problems of path Functonalty, Mantanablty CCC Normatve problems of code Functonalty, Relablty, Mantanablty INFO Informaton management Functonalty, Mantanablty, Survvablty RACE Resource competton Functonalty, Relablty, Survvablty PPA Authorty problems Survvablty HAND Error handler Functonalty, Relablty, Survvablty UI Desgn problems Relablty, Survvablty INT Error nteractons Functonalty, Relablty, Survvablty INIT Intalzaton errors Functonalty, Relablty RES Resource management Functonalty, Relablty, Survvablty problems NUM Numerc problems Functonalty, Relablty, Survvablty AUTHENT Authorzaton problems Relablty, Mantanablty, Survvablty CRYPTO Encrypton errors Relablty, Survvablty RAND Random and predctve Relablty, Survvablty problems ERS Excepton handng problems Functonalty, Relablty VER Data verfcaton problems Relablty, Survvablty ATTMIT Attack defense problems Relablty, Survvablty MAID Change errors of mmutable Functonalty, Relablty, Survvablty data MAL Inserton of vcous codes Functonalty, Relablty, Survvablty CONT Senstve data problems Relablty, Survvablty MISC Other problems Functonalty, Relablty, Survvablty Mantanablty Fgure 1 gves the number of untrusted evdence for each software flaw category, the horzontal axs n Englsh abbrevaton for each category of software flaws, the number of weaknesses contaned n the vertcal shaft for each flaw category, or the number of untrusted evdence.
Software Trustworthness Statc Measurement Model and the Tool 1105 3. Comprehensve Trustworthness Metrc Model 3.1. Model Fgure 1. The dstrbute of the untrusted for the correspondng flaws In the prevous Natonal Fund project, we make a deep research on the software trustworthness metrc model. A trustworthy metrc model based on program slcng complexty can be wdely appled to software entty wth clear code. However, ths model s only for the trustworthness of the evaluaton of the program, not consderng the mpact of untrusted evdence on the program. Therefore, we propose a comprehensve and trustworthy metrc model about attrbute based on the orgnal model. The man dea of the model s that trustworthness of the software entty s determned by the trustworthy nature of the mplementaton and the evdence of the untrustworthy n the procedure. Frst of all, the dfferent degree about trustworthness mpacts overall trustworthness for software trustworthness. So, trustworthy measure of software trustworthness s classfed as dfferent level n Table 7. Accordng to the actual stuaton, the mnmum attrbute level s 0.45, so t s necessary to ensure that the trustworthness of each attrbute s kept at a good level to ensure that the software s trustworthy. Table 7. Software trustworthy attrbute dvson Degree Trustworthness range No mpact >0.95 Lttle mpact 0.85-0.95 Moderate mpact 0.70-0.85 Serous mpact 0.45-0.70 No run <0.45 Consderng the nfluence of the untrusted evdence on the trustworthness attrbute, our comprehensve model meets the followng crtera and trustworthy attrbute of satsfy: 0 1. T y The trustworthness of attrbutes ncreases when the trustworthness of untrusted evdence ncreases. T y T y a represents the trustworthness of attrbute of y, and 0 T y a represents the trustworthness of every untrusted evdence. The attrbute trustworthness decreases when the number of untrusted evdence ncreases. (1) T y n 0 (2) n represents the number of untrusted evdence of n the y. Accordng to the second rule, t represents the greater the number of untrusted evdence and the lower the trustworthness attrbute. By the classfcaton of trustworthness attrbute, trustworthness of attrbute s reduced to a lower
1106 Yan L, Zhqang Wu, and Yxang Chen level, whch should contan more untrusted evdence by the classfcaton of trustworthness attrbute. The dfference between levels ncreases gradually, and the dfference between levels of untrusted evdence ncreases gradually. Therefore, we requre the number of as the ndependent varable, and the U y represents dependent varable. The curve of the U y wth the n n s roughly as shown n Fgure 2. the horzontal axs represents the number of untrusted evdence; the vertcal axs represents the trustworthness attrbute. The relatonshp functon of trustworthness attrbute and the number of untrusted evdence can be obtaned as follow. U y n e (3) Among them, the s the nfluence factor for the attrbute of th, and the value range s (0; 1]; n represents the number of untrusted evdence that affect the attrbute of th n the program. Fgure 2. The relatonshp shows between trustworthness attrbute and the number of untrusted evdence We take 4 values from the range of get the change curve as shown n Fgure 3. to smulate the relatonshp between the dfferent trustworthness attrbutes. We Fgure 3. The dfferent show the relaton curve of the correspondng attrbute and the number of the untrusted evdence From the above Fgure 3, the greater the nfluence of the untrusted evdence on the trustworthness attrbute, the faster the trustworthness of the attrbute s reduced wth the ncreasng of the number of the untrusted evdence. It s assumed that each untrusted evdence has the same effect on the program, ncludng the dfferent attrbutes of the same number of untrusted evdence, and the smaller the weght of the property whose trustworthness decreases faster. Therefore, the nfluence factor for λ of the untrusted evdence to trustworthness attrbute s nversely related to the attrbute weght for α, and the relaton s expressed as follow. k (4) Among them, α represents the weght of th, and the k ndcates that the nfluence factor parameter of the untrusted evdence to the attrbute about th.
Software Trustworthness Statc Measurement Model and the Tool 1107 The above analyss obtans the relatonshp between λ and α from the trustworthy relatonshp for curves of all attrbutes, and then s determned the range of k by studyng the trustworthness of the attrbute. Frstly, we convert the trustworthy attrbutes for calculaton model as follow: U y 1 k n e (5) Assumng weghts of attrbute about the α are known, and when the k takes dfferent values, the trustworthy degree of the same property changes as shown n Fgure 4. Fgure 4. The dfferent k show the curve of trustworthy degree of the attrbute From the above Fgure 4, wth the decrease of value of the k, the trend of trustworthy degree of attrbute becomes slow down and gets closer to the lnear relaton. For the same untrusted evdence, wth a greater value of the k, the trustworthy degree becomes lower. If we want to acheve the same trustworthy level, when value of the k ncreases, the untrusted evdence wll be reduced. 3.2. Smulaton and Analyss For the model proposed n ths paper, a large number of examples need to be valdated, n whch the acquston of untrusted evdence s a dffcult problem n the process of verfcaton n software, so we use a large number of data smulatons to smulate the number of untrusted evdence of each attrbute, the results can be used to observe the stablty of the model. Monte Carlo smulaton s a method to study ts dstrbuton characterstcs by settng up a stochastc process, generatng tme seres teratvely, and calculatng parameter estmates and statstcs [8]. Mathematc s a scentfc computng software that combnes numercal and symbolc computng engnes, graphcs systems, programmng languages, and text systems that are good for advanced connectons wth other applcatons. The smulaton parameters and results are descrbed n detal below. The number of y s 4, and the evaluaton value of each attrbute s 1, namely: y (0, 1]; The weght of the attrbute s α : α 1 = 0.294, α 2 = 0.382, α 3 = 0.169, α 4 = 0.155; The weghts of the trustworthy elements and the untrusted elements n the attrbute are set to 0.5; We use 0.001 as the basc unt, and randomly generate 100,000 sets of data to smulate the model n the [0, 1] nterval of the cumulatve number of occurrences (n the graph by pont), and the smulaton statement s as follows: The smulaton results show the Fgure 5. The horzontal axs represents the software s trustworthness, and the number of tmes the vertcal shaft represents the trustworthness attrbute.
1108 Yan L, Zhqang Wu, and Yxang Chen Fgure 5. Model smulatons The results of the smulaton under the condton of dfferent untrusted evdence show that wth the decrease of the untrusted evdence n the attrbute, the software trustworthness s ncreasng and the number of the untrusted evdence n each attrbute need to be strctly controlled. In addton, the shape of the smulaton dagram s smlar to the normal dstrbuton, whch shows the stablty of the model. 4. Software Trustworthness Tool Based on the models and methods mentoned above, we developed a statc predctve software relablty tools based on Cppcheck TSMT. The tool realzes the evaluaton of untrusted evdence, trustworthness attrbute and software trustworthness n code, and provdes the mprovement method whle detectng the untrusted evdence. Ths tool embodes the avalablty of source code orented software trustworthy metrcs. Cppcheck s developed to use the C++ language, provdng a varety of types of error to check for C++, and ts checkponts nvolve ponters, arrays, memory, and problems n functons. Selectng Cppcheck as the bass for ths tool s based on the fact that Cppcheck s an open-source tool, opens the user extenson nterface, and makes t easy for developers to embed custom rules nto Cppcheck. Fnally, we can customze the personalzed nspecton tools. We ntroduce the TSMT tool that expands the nspecton rules n Cppcheck to enable t to detect more untrusted evdence. So, t supplements measurement system of software trustworthness based on untrusted evdence. It s shown n Fgure 6 as follow. Fgure 6. Usecase of TSMT The tool body can be dvded nto three parts: user, metrc tool, and Cppcheck. In addton to provdng a user-frendly GUI n Fgure 7, TSMT provdes executable fle for Cppcheck.exe that can be executed at the command-lne prompt, and users can get the results of the check drectly usng the command lne. The trusted metrcs tool nvokes cmd.exe background executon Cppcheck to get executon results. The specfc mplementaton enters the Cppcheck command n the specfed text box through the button call cmd.exe to execute the command n the text box, reads the result from the command lne through the StreamReader class, and drectly dsplays t n another text box. Cppcheck can be dversfed by adjustng the parameters, ntroducng the feature nto the trusted Metrcs tool, and gettng a varety of results by enterng related commands n the specfed text box.
Software Trustworthness Statc Measurement Model and the Tool 1109 Fgure 7. Man Vew The tool mplements nternal call Cppcheck to check the program. The extended nterface provded by Cppcheck add custom rule to check on the bass of the orgnal nspecton rules, and allow the tool to detect more untrusted evdence. Webbench s a pressure test tool for webste and uses C language development, t ncludes 600 lnes of C. We make trustworthy measurement for webbench.c about the man fle for ths tool. When adjustng the rules, we clck on rght-hand functon for check and open the nterface shown n Fgure 8. It enters the Cppcheck executon statement n the text box and dsplays the results n the text box below. If you do not understand the Cppcheck command and you can clck the button for help. The man vew shows untrusted evdence n the dsplayed error lsts n Fgure 9. Fgure 8. check of Cppcheck Fgure 9. Lsts of untrusted evdence It analyses ten untrusted evdence by detectng and showng that these untrusted evdence belong to procedural normatve problem. These problems don t mpact the program short term for runnng software. But there are secure rsks, such as f-else msmatch problem, whch may be attacked and affect the mantanablty of the program. The untrusted evdence s detected n source code, although t wll affect trustworthness of the program. It belongs to lttle mpact and the program s relatvely trustworthness. 5. Conclusons In ths paper, the software trustworthness evaluaton s through the detecton of untrusted evdence n software source code. Because the characterstcs of software source code are easly accessble and the generalty of embedded software s commonly used n C language development, ths work can be appled to the research of the trustworthness of embedded software. In the future, we need to perfect the trustworthy metrc model and verfcaton by the formal method to enhance the persuason. Second, to mprove the trustworthness of the measurement tool and acheve a hgher automatc degree of software trustworthness metrc, t s necessary to mprove the tool.
1110 Yan L, Zhqang Wu, and Yxang Chen References 1. S. Chrstey, Prelmnary Lst of Vulnerablty Examples for Researchers, 2006.http://cwe.mtre.org/documents/PLOVER.pdf. 2. Y. Huang, X. He, J. Wang, and Z. Le, An Evaluaton Method Orented to the Comprehensve Credblty of Smulaton Data Source Based on Evdence Theory, vol. 5, no. 2, 2016. 3. R. Jang, A Trustworthness Evaluaton Method for Software Archtectures Based on the Prncple of Maxmum Entropy (Pome) and the Grey Decson-makng Method (Gdmm), Entropy, vol. 16, no. 9, pp. 4818 4838, 2014. 4. K. Lu, Overvew on Major Research Plan of Trustworthy Software, Bulletn of Natonal Natural Scence Foundaton of Chna, 2008. 5. Y. L and Y. Chen, A Measurement Model for Trustworthy Software Based on Trusted Evdences, n Internatonal Symposum on System and Software Relablty, 2017, pp. 20 24. 6. T. J. Mccabe, A Complexty Measure, IEEE Transactons on Software Engneerng, vol. SE-2, no. 4, pp. 308 320, 2006. 7. M. Rong, A Model for CPS Software System Trustworthness Evaluaton Based on Attrbutes Classfyng, 2013. 8. A. F. Sela, Smulaton and the Monte Carlo Method, Technometrcs, vol. 24, no. 2, pp. 167 168, 2012. 9. K. Shbata, K. Rnsaka, and T. Doh, Metrcs-based Software Relablty Models Usng Non-homogeneous Posson Processes, n Internatonal Symposum on Software Relablty Engneerng, 2006, pp. 52 61. 10. H. Tao and Y. Chen, A New Metrc Model for Trustworthness of Softwares, Kluwer Academc Publshers, 2012. 11. H. Tao and Y. Chen, A Metrc Model for Trustworthness of Softwares, n Ieee/wc/acm Internatonal Jont Conference on Web Intellgence and Intellgent Agent Technology, 2009, pp. 69 72.