Computing Software Metrics from Design Documents

Transcription

1 Computing Software Metrics from Design Documents Cara Stein Computer Science Department University of Alabama in Huntsville Huntsville, AL USA Letha Etzkorn Computer Science Department University of Alabama in Huntsville Huntsville, AL USA Dawn Utley ISEEM Department University of Alabama in Huntsville Huntsville, AL USA ABSTRACT Software metrics can provide an automated way for software practitioners to assess the quality of their software. The earlier in the software development lifecycle this information is available, the more valuable it is, since changes are much more expensive to make later in the lifecycle. Semantic metrics, introduced by Etzkorn and Delugach, assess software according to the meaning of the software s functionality in its domain. This is in contrast to traditional metrics, which use syntax measures to assess code. Because semantic metrics do not rely on the syntax or structure of code, they can be computed from requirements or design specifications before the system has been implemented. This paper focuses on using semantic metrics to assess systems that have not yet been implemented. Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics complexity measures, process measures, product measures; K.6.3 [Management of Computing and Information Systems]: Software Management software development, software process. General Terms Measurement, Documentation Keywords Design, design document, object-oriented software, semantic metrics 1. INTRODUCTION Estimating the size, effort required, and cost of software to be developed is a difficult problem. Methods such as COCOMO [6] and Function-Point Analysis [1] have been proposed to help with making such estimates, but more information is still needed in order to make accurate estimates. One possible source for such information is software metrics. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACMSE 04, April 2 3, 2004, Huntsville, Alabama, USA. Copyright 2004 ACM /04/04 $5.00. Traditional metrics are calculated based on the syntax of code. Examples include McCabe s Cyclomatic Complexity [25] and Chidamber and Kemerer s suite of metrics [8][9]. In contrast, semantic metrics, introduced by Etzkorn and Delugach [14] and expanded by Etzkorn, Gholston, and Hughes [15], are not based on the structure of the code. Semantic metrics are calculated based on the meaning of the code s functionality in the problem domain. Thus, instead of calculating the complexity of an algorithm or the number of paths through a piece of code, semantic metrics calculate the psychological complexity or cohesion of a module. Since semantic metrics do not depend on the structure of the code, they can be calculated from design specifications before the code has been written. This allows project managers and developers an earlier insight into the quality of the design and the potential quality of the eventual software. For instance, if the values of semantic metrics show that a module is very complex or lacks cohesion, the project manager could order a code inspection or redesign of the module. Furthermore, many syntactic metrics have been shown to correlate with aspects of software quality such as fault-proneness and changeability. If semantic metrics also correlate with aspects of software quality, project managers could use values of semantic metrics computed from design specifications in making their estimates of size, effort, and cost for software projects. 2. BACKGROUND 2.1 Semantic Metrics Etzkorn and Delugach [14] introduced a suite of semantic metrics that are calculated based on concepts in a knowledge base that are associated with a class or method. These metrics include: LORM (logical relatedness of methods): the number of relations in a class divided by the number of pairs of methods in the class. Here a relation is defined as a pair of methods x and y in which there is a conceptual relation between a concept that is in method x but not in method y and a concept that is in method y but not in x. CDC (class domain complexity): the sum of the concepts associated with a class, each multiplied by a weighting factor, plus their associated conceptual relations. RCDC (relative class domain complexity): the class s CDC value divided by the maximum CDC value for any class in the system 146

2 KCI (key class identity): 1 if the class s RCDC value is at least.75; 0 otherwise. COa (class overlap): the sum of the concepts in common between two classes, divided by the total number of unique concepts in either class, computed for all classes in the system and divided by the number of classes in the system [14]. 2.2 Design Metrics Although many object-oriented metrics claim to be design metrics, most syntactic metrics can be calculated only on source code. However, Bieman and Kang [5] proposed a new way to assess the cohesion of a module (here, a procedure or function) from the design alone. They defined six types of relationships that could exist between any pair of outputs of a module. These relationships are: Coincidental Relationship (R1): the two outputs do not depend on each other, and they don t depend on any common input Conditional Relationship (R2): the two outputs depend on the same input, and that input is a condition in a branch control structure Iterative Relationship (R3): the two outputs depend on the same input, and that input is a condition in a repetition control structure Communicational Relationship (R4): two outputs depend on the same input, and that input is not a condition in any branch or repetition control structure Sequential Relationship (R5): one output depends on the other Functional Relationship (R6): the module has only one output [5] The strength of the weakest relationship (that is, the one with the lowest number) in the module is the cohesion rating for that module [5]. Although some of these relationships might still be difficult to identify during the design phase, particularly the ones that depend on whether the input is a condition in a control structure, this metric is a step in the right direction for true design metrics. Bieman and Kang also defined three other design-level cohesion metrics. They created input/output dependency graphs (IODGs) of the modules to be analyzed. They defined isolated components to be those that affect only one output of the module; essential components are those that affect or depend on all outputs of the module. In this context, a component is an input or output of a module [5]. From these definitions, their metrics are defined as follows: LC (loose cohesiveness): the number of isolated components divided by the number of components in the module TC (tight cohesiveness): the number of essential components divided by the number of components in the module MC (module cohesiveness): the sum over all components of the connectedness of each component, divided by the number of components in the module [5] Another study of design-level metrics was performed by Bansiya and Davis [3]. This study defines a model called QMOOD, containing four levels to be analyzed in object-oriented design: Components: objects, classes, and relationships Metrics (several new ones) Properties: abstraction, encapsulation, coupling, cohesion, complexity, and size Quality attributes: functionality, effectiveness, understandability, extensibility, reusability, and flexibility [3] Then they created mappings between the levels and used the model to predict the quality of software systems [3]. These approaches are more abstract than the usual method of counting units of code, such as lines, semicolons, or function calls, allowing at least some of them to be performed before implementation. However, metrics that provide the same level of abstraction in the design phase as in the implementation phase would be valuable. 2.3 Metrics Used to Predict Aspects of Software Quality A considerable amount of research has been done into the use of metrics to gauge or predict software quality. Li and Henry [24] assessed their suite of metrics, together with those defined by Chidamber and Kemerer [9], and discovered a strong relationship between the metrics and changeability of code [24]. In another study, Ohlsson and Alberg [26] found that one could predict the fault-proneness of software before it is implemented. They found that using a model of the number of signals plus the number of branches, they identified a set of 20% of the modules; they predicted this set to be the most fault-prone. The modules selected by their model had 47% of the faults in the system. The actual most fault-prone 20% of the modules in the system contained 60% of the faults [26]. This study shows that even simple measures can be used effectively to identify fault-prone modules. More extensive studies were done by Basili, Briand, and Melo [4] and Briand et al. [7]. These studies analyzed existing metrics for their use as predictors of probability of fault. Probability of fault is the likelihood that a fault will be detected in a module during an inspection. Basili, Briand, and Melo analyzed the Chidamber and Kemerer suite [8][9]; Briand et al. analyzed those and others too numerous to mention. Both studies had good results. Basili, Briand, and Melo found that fault probability had significant positive correlation to DIT (depth of inheritance tree), RFC (response for class), and CBO (coupling between objects). They also found a significant negative correlation to NOC (number of children), which they attributed to more design and testing effort being expended on classes on which other classes will be based [4]. Overall, Basili, Briand, and Melo found that the best model for predicting fault probability contained DIT, RFC, and NOC. They compared this model with a set of non-object-oriented code-level metrics and found that the model containing DIT, RFC, and NOC performed better [4]. 147

3 Briand et al. also had promising results. They began with a set of 49 metrics compiled from 12 different sources. Using logistic regression, they found that the best model contained 11 of the original metrics, including Chidamber and Kemerer s RFC (two versions) and NOC [8][9] and a version of Li and Henry s DAC [24]. This model found 95% of the faults in the system, and 85% of the modules it flagged as probably having faults actually had faults [7]. 2.4 Analyzing Design Documents There has not been a concentrated effort regarding automation of design document understanding. However, a few studies have addressed different perspectives on processing design specifications. Lague et al. [18] did a study comparing design documents descriptions of layered architecture systems with the way the source code was organized into files. Their goal was to answer questions such as Does a layer ever use services from a layer above it?, Can a layer be broken down into subsystems?, and Are there any extraneous or missing files? [18]. Using a file dependency graph, they were able to answer these questions. A different study, conducted by Li and Horgan [22], involved analyzing a design specification to check its correctness before using a tool to automatically generate code from it. They developed a tool called XSuds to go through the design specification, generate a flow diagram, and analyze coverage features of the flow diagram. Then the tool would run a simulation of the design specification, collect the flow data from that, and compare the two sets of flow data. The goal of this study was to facilitate round-trip engineering [22]. Lakshminarayana et al. [19] took a different approach to analyzing design specifications. Their goal was to generate visual representations of the metric values for each class in a system, to aid developers in quickly pinpointing areas for improvement. They used Rational Rose s extensibility interface and Rose scripting language to get class information from UML diagrams. Then they computed metrics created by Chidamber and Kemerer [8] and Li [23] using that information. They developed a visual representation for each class based on its value for each metric. They represented each class as a cube and each metric as a feature of that cube as follows: DIT (depth of inheritance tree) [8]: number of layers in the cube represents level in inheritance hierarchy NDC (number of descendent classes) [23]: arrows pointing down represent descendent classes NOC (number of child classes) [8]: red arrows pointing down represent direct child classes NAC (number of ancestor classes) [23]: arrows pointing up represent ancestor classes CTA (coupling through abstract data type) [23]: hooks represent instances of aggregation NLM (number of local methods) [23]: box border is a warmer color for relatively higher numbers of local methods in a class DCTM (design coupling through message passing) [19]: envelopes sticking out of the top of the cube represent messages passed [19] In the resulting model, the visual representation makes it immediately clear which classes have complicated interactions with other classes. This allows developers to analyze a large system much more quickly than they ever could with a standard printout of metric values [19]. This work was expanded in Lakshminarayana [20]. This study is most relevant to the present research because both involve processing design specifications to calculate metrics on classes before they have been implemented. However, whereas Lakshminarayana et al. processed UML diagrams to compute syntactic metrics, we address performing natural language understanding on text in design documents to compute semantic metrics. Overall, the paucity of research on design specification understanding is probably because design documents are written for humans to understand already, so understanding them in the sense of program understanding may not seem important or useful at first glance. However, if design documents were processed to retrieve semantic concepts and keywords at the system, class, and function levels, then semantic metrics could be calculated from design documents. This would allow software quality analysis to be done in the design phase, before the code is written, allowing errors to be corrected earlier in the development. 3. COMPUTING METRICS ON DESIGN SPECIFICATIONS Although many tools exist to compute metrics, most of them calculate only syntactic metrics, and only on code. We have created a tool called semmet to compute semantic metrics on software systems. SemMet is based on parts of Etzkorn s PATRicia (Program Analysis Tool for Reuse) system [11]. In its current form, semmet consists of two parts: the source code interface and the main processing module. The source code interface performs the following steps: Generate abstract syntax tree information from code. Process the abstract syntax tree to retrieve the inheritance hierarchy and each class s attributes and behaviors and their accessibility (public, private, or protected). Process the code itself to retrieve all comments at both class and function levels. Use natural language processing to try to determine the part of speech for each identifier. Perform sentencelevel natural language processing on comments to determine the part of speech of each word. The main processing module performs the following steps: Process all words from comments and identifiers through a knowledge base of concepts and keywords of the domain of the system. Count concepts and keywords related to each class and each method of each class. Use class- and method-level concept and keyword information to calculate metrics and generate a report. These two parts together allow the semmet system to calculate semantic metrics on code. Currently, semmet calculates the semantic metrics proposed by Etzkorn and Delugach [14] and 148

4 Stein et al. [28]. The next step is to modify semmet to calculate semantic metrics from design specifications. Semantic metrics are calculated for code by associating concepts and keywords with classes or functions; the same processing can be done for design documents. Therefore, the same main processing module can be used for both code and design specifications. However, an interface must be created to extract the needed information from design specifications. Although there are many similarities between calculating metrics from a paragraph in a design document and calculating metrics from natural language-based program understanding on comments and identifiers in code, there are also some significant differences. For one thing, comments have been shown to have their own sublanguage of the English language [13]. Paragraphs in design documents probably do not belong to the same sublanguage as comments. Design documents may have their own sublanguage of English, but if they do, it is broader than that of comments, since design documents are written in a much more narrative style. The writing style of design documents is analyzed and criticized, whereas the writing style in comments is usually not. Comments are meant to convey information in the most efficient way possible, since most programmers hate writing comments, whereas design documents have the luxury to be less concise. Furthermore, design documents have a different audience than comments: design documents are meant for the customer and managers as well as developers, whereas comments are meant only for other programmers. Design documents may even be written by a different set of people than comments in code, depending on the size and structure of the organization in which the software is written. All of these factors point to design documents not fitting within the sublanguage of comments. Another difference is that, unlike many comments, design documents are already made up of complete sentences. Thus, for design document processing, it should not be necessary to perform any heuristics to try to make sentences as is done for comments [12]. Also, words in design documents should all be English words, unlike identifiers, which may be abbreviations or several words concatenated together [2][11]. Thus, with the exception of identifiers themselves appearing in design documents, there should be no need to perform any transformations to make the words in design documents into real English words. The other major difference between processing code and processing design documents is that design documents contain larger textual constructs than comments or identifiers. Whereas identifiers and comments are made up of words, or generally no more than a few sentences together, design documents are made up of paragraphs. This means that a much broader context is available within a design document for analysis of any one word or sentence. On one hand, harnessing that additional information has the potential to be very helpful in disambiguation, the process of choosing the correct meaning or structure from among many possibilities for a word or sentence. On the other hand, harnessing that information is a very complex task. When analyzing only one word at a time, one must consider the word s definitions, but when analyzing a word in its context, there are all of the word s definitions, with the words around it and their various definitions taken into consideration, plus the previous and next sentence, plus the whole paragraph and even the whole document. This quickly adds up to an unwieldy number of combinations. Ideally, each level of added context will serve to limit the number of definitions under consideration for a word, but the words and sentences in the context also have multiple meanings each. The process quickly becomes very complicated. To get an idea of the problem, consider the following example. The word throw has different meanings in these phrases: throw up (a ball? one s lunch?), throw away (a ball? trash?), throw out (an idea? trash?), throw (a ball? a pot? a match? a party? one s weight around?) [27]. Beyond the problems of understanding individual phrases, multiple sentences and the context as a whole also present problems. One sentence can impact the meaning of another [29]. To calculate semantic metrics from design specifications, we are creating a new interface to process design specifications compliant with the IEEE standard for design specifications [16]. This interface will extract a class hierarchy and class members from the decomposition description section of the design document. Then it will retrieve prose descriptions of functionality from the detailed description section. For these prose descriptions, paragraph-level natural language processing must be performed. All information gathered by the design specification interface will be passed on to the main processing module for inference through the knowledge base and metrics calculation. Both the code interface and the design specification interface will use the same main processing module to compute metrics. class Account { private: float balance; int type; // 0=savings, 1=checking, //2=money market float interestrate; Entity owner; Bool isactive; public: int makedeposit(float amount); int makewithdrawal(float amount); float getbalance(); void closeaccount(); }; Figure 1: Bank Account Code Example For example, given the code sample in Figure 1, the code interface of the semmet system would pick out concepts and keywords account, balance, interest, rate, owner, active, deposit, amount, withdrawal, balance, and close from the code and checking, savings, money, and money market from the comments. Then it would try to associate parts of speech with these concepts, send them through the knowledge base, and calculate metrics from the results. To process the text description in Figure 2, the design specification interface will associate the Account section with the appropriate class already recognized from the decomposition description section. Then it will perform paragraph-level natural language processing on each paragraph of the text description. The current natural language parser used by the semmet system performs only sentence-level natural language processing. We will incorporate a new, more powerful natural language parser such as 149

5 LT POS [21] or Connexor Machinese Syntax [10] to perform the additional analysis needed. Both of these natural language parsers perform paragraph-level processing. 6.1 Module Detailed Design Class Account The account class represents a bank account. It can be a checking, savings, or money market account. Each account will have a balance and an interest rate, which may be 0%, particularly if the account is a checking account. Each account is owned by an entity, which could be a person, multiple people, or a company. The owner of the account can make deposits, check the balance, and make withdrawals. Figure 2: Bank Account Design Specification Example The design specification interface will associate each section of the detailed description with the proper class or member, perform natural language processing on that section, and send the information to the main processing module. At this point, the information will be in the form of a list of words associated with each class or function. Each word will have a part of speech associated with it if the natural language processor could determine one. The main processing module will then assert the words with their locations into the knowledge base. As inference occurs through the knowledge base, the words taken directly from the design document may trigger other concepts to be recognized and associated with a class or method. Finally, the main processing module will tally the concepts and keywords associated with each class and method, and calculate semantic metrics just as it does for code. 4. CONCLUSION Semantic metrics computed for code or design can indicate relative cohesion, complexity, and coupling of the system s modules. Computing semantic metrics in the design phase, before the code is ever written, can give the development team valuable insight. Managers can use this information in the area of project planning. They can also take preventative measures in modules that are complex or lacking in cohesion, by redesigning or including activities such as code inspections. Cost savings, better delivery date estimation, and code with better design and fewer errors are the potential results. 5. FUTURE WORK Once semantic metrics have been collected for several systems, it is important to validate these metrics for use as intended and to check for relationships with aspects of software quality such as effort, changeability, or fault-proneness. Also, this work could easily be extended to compute semantic metrics on requirements documents. One approach would be to compute semantic metrics for each major function and system requirement from a requirements specification compliant with the IEEE standard for requirements specifications [17]. This information could be used for project planning or compared with semantic metric values computed for the implemented software to assess the degree to which the software matches the requirement specification. 6. ADDITIONAL AUTHORS Additional authors: Phillip Farrington (ISEEM Department, University of Alabama in Huntsville), Glenn Cox (Computer Science Department, University of Alabama in Huntsville), Julie Fortune (ISEEM Department, University of Alabama in Huntsville), and Sampson Gholston (ISEEM Department, University of Alabama in Huntsville). 7. ACKNOWLEDGEMENTS The research in this paper was partially supported by NASA grants NAG and NCC REFERENCES [1] A. Albrecht, J. Gaffney. Software Function, Source lines of code, and development effort Prediction: A software science validation. IEEE Transactions on Software Engineering, vol. SE-9, no. 6, Nov. 1983, pp [2] G. Antoniol, G. Canfora, G. Casazza, A. De Lucia, E. Merlo. Recovering Traceability Links Between Code and Documentation. IEEE Transactions on Software Engineering, vol. 28, no. 10, Oct. 2002, pp [3] J. Bansiya, C. Davis. A Hierarchical Model for Object- Oriented Design Quality Assessment. IEEE Transactions on Software Engineering, vol. 28, no. 1, Jan. 2002, pp [4] V. Basili, L. Briand, W. Melo. A Validation of Object- Oriented Design Metrics as Quality Indicators. IEEE Transactions on Software Engineering, vol. 22, no. 10, Oct. 1996, pp [5] J. Bieman, B.-K. Kang. Measuring Design-Level Cohesion. IEEE Transactions on Software Engineering, vol. 24, no. 2, Feb. 1998, pp [6] B. Boehm. Software Engineering Economics. Englewood Cliffs, NJ: Prentice-Hall, [7] L. Briand, J. Daly, V. Porter, J. Wust. Predicting Fault-Prone Classes with Design Measures in Object-Oriented Systems. Proceedings of the 9 th International Symposium on Software Reliability Engineering, 1998, pp [8] S. Chidamber, C. Kemerer. A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering, vol. 20, no. 6, June 1994, pp [9] S. Chidamber, C. Kemerer. Towards a Metrics Suite for Object Oriented Design. Proceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, 1991, pp [10] Connexor. Products > Machinese Syntax. Accessed 9/24/03. [11] L. Etzkorn. A Metrics-Based Approach to the Automated Identification of Object-Oriented Reusable Software Components: a Dissertation. Huntsville, AL: University of Alabama in Huntsville, 1997, pp [12] L. Etzkorn, L. Bowen, C. Davis. An Approach to Program Understanding by Natural Language Understanding. Natural Language Engineering, vol. 5, no. 1, 1999, pp

6 [13] L. Etzkorn, C. Davis, L. Bowen. The Language of Comments in Computer Software: A Sublanguage of English. Journal of Pragmatics, vol. 3, no. 11, 2001, pp [14] L. Etzkorn, H. Delugach. Towards a Semantic Metrics Suite for Object-Oriented Design. Proceedings of the34th International Conference on Technology of Object-Oriented Languages and Systems, 2000, pp [15] L. Etzkorn, S. Gholston, W. Hughes. A Semantic Entropy Metric. Journal of Software Maintenance and Evolution, vol. 14, no. 4, July/August 2002, pp [16] Institute of Electrical and Electronics Engineers. IEEE Recommended Practice for Software Design Descriptions. IEEE Standard New York: IEEE, [17] Institute of Electrical and Electronics Engineers. IEEE Recommended Practice for Software Requirements Specifications. IEEE Standard New York: IEEE, [18] B. Lague, C. Leduc, A. Le Bon, E. Merlo, M. Dagenais. An Analysis Framework for Understanding Layered Software Architectures. Proceedings of the 6 th International Workshop on Program Comprehension, 1998, pp [19] A. Lakshminarayana, T. Newman, W. Li, J. Talburt. Automatic Extraction and Visualization of Object-Oriented Software Design Metrics. Proceedings of SPIE The International Society for Optical Engineering, vol. 3960, 2000, pp [20] A. Lakshminarayana. A Dimensional Reduction Analysis and Glyph-Based Visualization of Object-Oriented Software Metrics: a Thesis. Huntsville, AL: University of Alabama in Huntsville, [21] Language Theory Group. LT POS. Accessed 9/24/03. [22] J. J. Li, J. R. Horgan. A tool suite for Diagnosis and Testing of Software Design Specifications. Proceedings of the International Conference on Dependable Systems and Networking, 2000, pp [23] W. Li. Another metric suite for object-oriented programming. The Journal of Systems and Software, vol. 44, no. 2, 1998, pp [24] W. Li, S. Henry. Object-Oriented Metrics that Predict Maintainability. Journal of Systems Software, vol. 23, no. 2, 1993, pp [25] T. McCabe. A Software Complexity Measure. IEEE Transactions on Software Engineering, vol. SE-2, no. 4, Dec. 1976, pp [26] N. Ohlsson, H. Alberg. Predicting Fault-Prone Software Modules in Telephone Switches. IEEE Transactions on Software Engineering, vol. 22, no. 12, Dec. 1996, pp [27] S. Small, C. Reiger. Parsing and Comprehending with Word Experts (A Theory and its Realization). In Strategies for Natural Language Processing. Edited by Wendy G. Lehnert and Martin H. Ringle. Hillsdale, NJ: Lawrence Erblaum Associates, 1982, pp [28] C. Stein, L. Etzkorn, G. Cox, P. Farrington, S. Gholston, D. Utley, and J. Fortune. A New Suite of Semantic Metrics for Object-Oriented Software. Accepted to 1st International Workshop on Software Audit and Metrics, April 13-14, [29] R. Wilensky. Points: A theory of the Structure of Stories in Memory. In Strategies for Natural Language Processing. Edited by Wendy G. Lehnert and Martin H. Ringle. Hillsdale, NJ: Lawrence Erblaum Associates, 1982, pp