C-QM: A PRACTICAL QUALITY MODEL FOR EVALUATING COTS COMPONENTS

Transcription

1 C-QM: A PRACTICAL QUALITY MODEL FOR EVALUATING COTS COMPONENTS Soo Dong Kim, Ji Hwan Park Department of Computer Science Soongsil University 1-1 Sangdo-5-Dong, Dongjak-Ku, Seoul South Korea, want to evaluate COTS components thoroughly before they purchase, because these components are developed by third party producer and most consumers have notinvented-here syndrome [2]. Hence, evaluating the quality of COTS components becomes an important prerequisite to a successful component-based application development. ABSTRACT A commercial-off-the-shelf (COTS) component is an implementation of common functionality among family members, where an in-house component implements an organization-specific functionality. Typically, a COTS component has a producer and a set of potential consumers. Consumers evaluate COTS components thoroughly before they purchase, because these components are developed by third party producers and most consumers have not-invented-here syndrome. Hence, evaluating the quality of COTS components becomes an important prerequisite to a successful component-based application development. In this paper, we identify the characteristics of COTS components, and derive a practical quality model for components, C-QM, which consists of quality factors, criteria and metrics. The top design goal of C-QM is set to provide a practically applicable comprehensive quality model which can be effectively applied in assessing the various quality aspects of COTS components. In this paper, we identify the characteristics of COTS components, and derive a quality model for components from the characteristics. The proposed model, C-QM, consists of quality factors, criteria and metrics. The proposed framework can be practically used in assessing the various quality aspects of COTS components. 2. Foundation 2.1. Software Quality Model A software quality model is a specification of software quality attributes and their relationship. ISO 9126 is a representative quality model for generic software [3]. In that, the quality model has two layers; characteristics layer and sub-characteristic layer. A characteristics is further refined into multiple sub-characteristics, and each subcharacteristic has a set of associated metrics, where a metric has a formula used to compute the metric value. However, this quality model was proposed for assessing the quality of finished software products rather than reusable modules such as COTS components. Key Words: Component-Based Development, Quality Model 1. Motivation Component-Based Development (CBD) is gaining popularity in both academia and industry for its promise on large-scaled software reuse. A COTS component is a reusable software module that provides common services among family members of a business sector [1]. COTS components are differentiated from in-house components in the scopes of functionality provided. A COTS component is an implementation of common functionality among family members, where an in-house component implements an organization-specific functionality. Hence COTS components are targeted for inter-organizational reuse, where in-house components are mainly for intraorganizational reuse. Another representative quality model was proposed by McCall [4]. This model also has two layers but uses different names; factor for ISO s characteristic, and criteria for ISO s sub-characteristic. McCall s quality model has 11 factors and 25 criteria. For the same reason as ISO 9126, this quality model is not applicable to COTS components. Boehm s quality model [5] and Dromey s quality model [6] also provide quality frameworks for completed software applications. In C-QM framework, we use a 3-layer hierarchy for specifying quality model; factor, criteria, and metric. A COTS component has a producer and a set of potential consumers. By using the components, consumers hope to reduce overall software cost and shorten time-tomarket. In searching for the right components, consumers

2 2.2. External Metrics A metric is a unit for measuring a specific quality attribute of the target software. Internal metrics are used to measure quality attributes of intermediate software artifacts such as specification, design model, and source code. External metrics are used to measure quality attributes of finished software products. In component-based software development projects, consumers are provided with information on the external features of COTS components such as functional category, interface specification, deployment specification, constraints, and customization instruction. Information on intermediate artifacts is not provided for reasons. Hence, they can only use external metrics to evaluate the quality of components. For this reason, metrics proposed in C- QM are all external Component Reference Model A component reference model is a specification of component architecture, static organization of a component, composition model, standard services and interfaces. There are only a few component reference models available, and they have similarities and differences. In order to precisely define a quality model, a well-defined reference model of component must be chosen. From our technical survey of component reference models, we decide to use the CORBA Component Model (CCM) by OMG [7] as the basis for defining C-QM framework. CCM provides a comprehensive set of features of CBD components, and it can be considered as the superset of other representative reference models. Figure 1 shows static organization of CCM components. A CCM component is a meta-type that extends and specializes object meta-type. Each component has a reference and a set of interfaces. Equivalent Interface Facets Attributes A Component Implementations of Facet Figure 1. CORBA Component Model (CCM) Receptacles Event sinks Event source A port in a CCM component is a communication channel for component clients and different types of the port are defined. Facet is the provide interface of an object inside a component, and it provides services to clients directly at object level. Equivalent interface is a provide interface at the level of component as a whole, and often it combines several facet interfaces of the component. Equivalent interface is especially useful in constructing black-box components where member objects in a component are hidden and only a componentlevel API is provided. Receptacle is a require interface of the component, which is used to acquire external references or resources needed by the component. Hence, component consumers must provide necessary references or resources during component adaptation or customization process. Receptacle is one of unique features of CBD components, since functions and methods in conventional computing paradigms and programming languages only provide service, and they never require external fulfillment such as plug-in object [8]. Event source and event sink provide a mechanism for trapping and consuming events. 3. Characteristics of COTS Components In order to derive a quality model, we need to understand the unique characteristics and constraints of the target software. Hence, in this section, we identify main characteristics of COTS components that are based on CCM reference model. Providing Common Functionality: A COTS component should capture a common functionality among family members in order to increase inter-organizational reusability. This makes the component-based reuse distinguishable from object-oriented reuse. If a standard domain model exists, COTS components must conform to the standard. Recently, organizations such as OMG and Microsoft have been developing standard domain models in major business sectors such as finance, health care, and telecommunication. Providing Require Interface for Customization: Commonality and variability (C&V) analysis is a key activity in extracting components from a model [9]. For a common functionality among family members, a minor variability on logic, attributes or workflow may exist. For example, the loan approval function, approveloan (Customer, Amount, Period, RepaymentMethod, Mortgage), is common for the loan management system in finance domain. However, the logic to evaluate the loan application and approve a loan varies among financial organizations. Some organizations give a higher weight on customer credit rate while others give a higher weight on the value of mortgage. In this case, the loan management component should implement this variability and provide a mechanism to customize the logic. CBD is the first development paradigm that considers the variability seriously in software design. Larger Granularity Reusability through Component Interface: A Component typically consists of multiple objects and classes. Hence, a component is a largergrained unit than the object of OOP [10]. By providing a 992

3 component-level interface, i.e. Equivalent Interface, in addition to facet interfaces, component consumers need to know a smaller number of APIs, and it can lower the degree of coupling between COTS components and the application integrating the components. This becomes the reason why blackbox components are preferred [10]. Binary-level Composition and Replaceability: Commercial components produced by CBD approaches are in binary or byte-code form. COM/DCOM components are in binary form, and Enterprise JavaBeans (EJB) components are in byte-code form. Hence in CBD, the composition between components and the integration between components and the target application should be done at binary or byte-code level. The composition mechanism is typically defined by its component reference model, and that mechanism is implemented by component platforms. Components integrated in an application should be replaceable with newer version of components efficiently without modifying other parts of the application. Blackbox components have a higher level of replaceability. Component Reference Model Dependency: A component implementation environment, i.e. component platform, implements a specific component reference model. Components developed on an EJB platform are not interoperable with components developed on.net platform unless a specialized translator or gateway is used. Hence, component consumers should consider the component reference model of the target components, and how well they conform to the reference model. Functionality: The factor functionality measures how faithfully the software conforms to the stated and implied functional requirement [3]. In case of COTS components, the functional requirement is defined as the set of functions that are common to family members, called family requirement specification (FRS). In some cases, there may already be a pre-defined domain model, which defines the standard business logic in that domain. With these two observations in mind, the functionality of COTS components should be measured differently from conventional measurement of software functionality. Reusability: Since COTS components are targeted for inter-organizational reuse, they must be highly reusable to be valuable. Promise of reusability is convincing and it is the main driving force for the component market. The require interface of a component which realizes the variability from C&V analysis will yield a higher reusability since a broader range of companies may be able to reuse the component through the require interface and customization. The larger-grained nature of component will also yield a higher reusability than objectoriented reuse. Maintainability: The facts that components are largergrained and in binary form contribute the need for the maintainability factor. Software maintenance involves correction, enhancement, adding new functionality and preventive maintenance. Maintainability factor will measure how effectively the maintenance activities can be carried out on COTS components. Blackbox components in binary form will yield a higher level of maintainability than whitebox components in source-code form. 4. FRAMEWORK OF C-QM : Components are produced on a Based on the characteristics of COTS components, we component platform which in turn implements a now define its quality model which consists of factors, component reference model. Hence, a component is criteria, and metrics. C-QM has four factors as high level dependent on a component reference model and partially quality attributes, and each factor has a set of quality on the component platform. The factor will criteria, i.e. sub-factors. measure how well the components conform to the selected reference model. Components that do not conform to a 4.1. Factors of C-QM reference model will not easily be interoperable with others and so they are not highly reusable. Each characteristic of COTS components has a set of associated quality factors. Figure 2 shows how the characteristics of the components affect the selection of quality factors. For example, Reusability is an important factor of Q-CM related to the three characteristics of COTS components; common functionality, Interface for Customization, and Larger Granularity Reuse. Common Functionality Functionality Interface for Customization Reusability Larger Granularity Reusability Binary-level Composition Maintainability Reference Model Dependency Figure 2. Mapping Characteristics to Factors Given the description of four factors in C-QM, it is evident that names of the four factors may appear in conventional quality models but their definition and semantics are distinguishable from conventional quality factors. In the next section, we show how each factor can be indirectly measured by using its associated quality criteria Criteria of C-QM A criterion is a lower-level quality attribute of a quality factor. Each criterion measures a specific and detailed aspect of the given factor, and often more than one criterion are defined for a factor. Figure 3 shows the hierarchy of C-QM, and each criterion is defined in this section. 993

4 Functionality Commonality Suitability Completeness Reusability Commonality Modularity Customizability Comprehensiveness Maintainability Modularity Interface Abstractness Changeability Figure 3. Hierarchy of C-QM Standard Reference Model The functionality factor is measured by commonality, suitability and completeness criteria. The reusability factor is measured by measured by modularity, customizability, and comprehensiveness. The maintainability factor is measured by modularity, interface abstractness, and changeability. The conformance factor is measured by standard conformance and reference model conformance. Commonality for both functionality and reusability factors measures how well the functionality of components matches to the functionality of FRS and its standard domain model. If a COTS component provides a superior functionality but does not meet the commonness of the domain, its reusability among organizations will be limited. Suitability for functionality measures how many of the functions required in FRS and the domain model have been realized in the component. If a component implements only a small subset of functions required, it will partially fulfill the functional requirement. Completeness for functionality measures how completely each function in FRS and the domain model has been implemented by the component. This criterion is needed in addition to the suitability since a function may be realized in a component, satisfying the suitability, but a partial behavior of the function might have been implemented in that component. Modularity for both reusability and maintainability measures how each component is independent from others. A component with high modularity has a high cohesion and lower couplings with other components. Sine a component is larger-grained than an object, a single component should provide a relatively coarse-grained part of the whole domain functionality, and it should not require much interaction with others. Customizability for reusability measures how much of the variability on logic, attributes, and workflow has been implemented and how effectively the component can be customized by component consumers. If a component implements largely the fixed part and less the variable part, its wide range of applicability will be limited to a smaller number of family members. Comprehensiveness for reusability measures how easy the target component is for component consumers to understand its functionality, interface, and constraints. A specification and manuals of a component will be the primary source for comprehension, but other information such as demonstration program and examples of using them may be provided. Since the consumer of a COTS component is often not the producer, the comprehensiveness becomes essential for successful distribution of components. Interface Abstractness for maintenance measures how complete the component interface is and considers the delivery format of components. A component with only an equivalent interface will be easier to modify than a component with only object level interfaces, i.e. facets. This is because changes to objects and classes inside a blackbox component do not propagate beyond the component unless the equivalent interface itself is modified. In contrast, modifying a whitebox component may require changes of its client components since client components can directly access the object and classes inside the component. Also, a binary component yields higher maintainability since application programs cannot refer to the internal information of the component such as source code and data structures. Changeability for maintenance measures the quality of internal organization and engineering of a component. A component with well-defined program structure, disciplined use of data elements, and description of internal elements will have a higher changeability. Standard for the conformance factor measures how faithfully the component adopts any standards existing in the domain. The standard may be governmental regulation, de factor standard, standard policies in the domain. A component that lacks the conformance to relevant standards will not be appreciated by consumers. Reference Model for the conformance factor measures how strictly a component conforms to the pre-determined reference model. Current state-of-the-art CBD technologies do not yet support the interoperability among heterogeneous components developed on different reference models. 5. METRICS OF C-QM In this section, we provide a set of representative metrics for the criteria defined in section 4.2. Like all quality models, the set of proposed metrics here may be extended or tailored for each organization and project. Hence, the proposed set of attributes should serve as the foundation and a starting point for further tailoring and enhancement. 994

5 5.1. Metrics for Functionality Metric for Commonality: This metric measures the degree of functional commonness provided by components, as follows. X = A / B, where A is the number of family members that the functionality of a component is applicable to, and B is the total number of family members in the domain. the functionality of such a component can be applied to all family members in the domain. The higher the metric value, the wider range of applicability the component has. Metric for Suitability: The metric measures the scope of functionality provided by a component or a set of components. We define the metric for the case of single component as follows: X = A / B, where A is the number of functions implemented in a component, and B is the number of functions required to be implemented in the component, which can be obtained from FRS. such a component implements and provides all required services. This metric can also be applied to a set of components if the identification of the functional requirement per component is not feasible. Metric for Completeness: This metric measures how complete the functions implemented in a component are. X = A / B, where A is the number of functions that are completely implemented in a component where a completeness is applied to the individual function, and B is the number of functions implemented in the component. The range of X is 0..1, and the value 1 is the best. The completeness of a function has been defined elsewhere and it can be custom defined for organizations Metrics for Reusability Metric for Commonality: This metric is same as the metric for commonality defined for Functionality factor. It is shown that the commonality has a strong influence on the overall quality of component. Metric for Modularity: This metric measures the degree of independence of the component, which is the degree of functional cohesion. X = A / B, where A is the scope of functionality provided solely by the component itself without invoking functions of other components, and B is the total scope of the functionality provided by the component regardless of the need to invoke other components. such a component is self-contained. However, components with the value 1 will be atypical since most components will have some degree of interaction with other components in forms of invocation and delegation. Metric for Customizability: This metric measures the degree of variability implemented and effectiveness of customization mechanisms provided. We define a metric for variability as follows. X = A / B, where A is the number of variable features implemented a component, and B is the number of variable features modeled during C&V analysis and pertained to the component. A variable feature may vary on logic, data attribute, workflow, and persistence. such a component implements all variable features needed. A metric for measuring the effectiveness is on our list of future research topics. The state-of-the-art CBD technology does not yet include a set of widely accepted and practically effective customization techniques yet. Metric for Comprehensiveness: This metric measures the ease of understanding a component. X = A / B, where A is the number of descriptive items that are sufficiently described in the component specification, manual and any other relevant documents provided, and B is the number of descriptive items needed to be specified for the component. The descriptive item can be an interface specification, functionality description, constraints, customization description and deployment instruction. The value of X is 0..1, and the value 1 is the best, although practically components will have a value that is lower than 1 due to the informal nature of description Metrics for Maintainability Metric for Modularity: This metric is same as the metric for modularity defined for Reusability factor. Hence, it can be concluded that the modularity along with commonality has a strong influence on the overall quality of components. Metric for Interface Abstractness: This metric measures how abstract the component interface is. A component with component-level interface, i.e. equivalent interface, has a higher abstraction level than a component with only object-level interfaces, i.e. facets. X = A / B, where A is the number of function signatures in the equivalent interface of a component, and B is the sum of A and C, where C is the number of function signatures in the facet interfaces of the component. Hence, the range of X is 0..1, and the value 1 is the best since such a component provides an equivalent interface only while hiding object-level interfaces. Metric for Changeability: This metric measures the efficiency of modification. There can be several sub- 995

6 criteria for determining this efficiency, and we use variables for those possible sub-criteria. X = (A 1 + A 2 +. A n ) / n, where A i is a measure of a sub-criterion for determining the efficiency of modification, such as overall program structure, disciplined usage of data structures, and description of program elements. The value n and types of A i should be tailored based on the nature of CBD projects and organizations. The range of A i is 0..1, and so the range of X will be Metrics for Metric for Standard : This metric measures the degree of conformance to relevant standards. X = A / B, where A is the number of standard elements strictly followed in the implementation of a component, and B is the total number of standard elements specified in the relevant set of standards, where a standard element can be governmental regulation, de factor standard, standard policies in the domain, and others. Hence, the range of X is 0..1, and the value 1 is the best since such a component conforms to all relevant standard elements. Metric for Reference Model : This metric measures the degree of conformance to the adopted component reference model. X = A / B, where A is the number of syntactic and semantic elements strictly obeyed in the implementation of a component, and B is the number of syntactic and semantic elements specified in the component reference model, where an element can be names of interface and classes, rules, constraints, policies or requirements. such a component conforms to all syntactic and semantic elements of the component reference model. APPLICATION OF C-QM IN PRACTICE Using the proposed a set of factors and criteria, one can compute the overall quality of a component as follows. Let F i be the ith factor defined in C-QM, and therefore the range of i is Let C i,j be the jth criterion of the F i, and F i has 2 to 4 criteria as shown in figure 3. For example, C 2,3 will be the customizability criterion for the reusability factor. Then, the composite value, MF i for the factor F i, is defined as: MFi = n j= Cij 1 where n is the number of criteria in F i n Depending on the domain and purpose, often different weighs are assigned to the factors. Let W i be the relative weight on F i and let (W 1 + W 2 + W 3 + W 4 ) be equal to 1. Then, the overall quality of a component can be computed as follows, and the range of overall quality will be 0..1, which is same as a single percentage representation. Overall Quality = 4 i= 1 ( Wi MFi ) In order to apply the proposed C-QM effectively, a quality evaluation process, instruction and a set of templates should also be defined. 7. CONCLUDING REMARK A COTS component is an implementation of common functionality among family organizations, and consumers evaluate COTS components thoroughly before they decide to purchase. Hence, evaluating the quality of COTS components becomes an important prerequisite to a successful component-based application development. In this paper, we identified main characteristics of COTS components, and derive a practical quality model for evaluating components, C-QM. We defined four quality factors, criteria, and metrics that can be effectively used to derive the overall quality of components. C-QM has been applied to a large number of components produced by a nationwide project CIP [2], and is currently being evaluated for its adoption as a national standard. We believe that C-QM can serve as the basic framework for developing domain-specific, organization-specific or project-specific quality model. REFERENCES [1] Crnkovic, I. and Larsson, M., Building Reliable Component- Based Software Systems, Artech House, Inc., [2] Kim, Soo, Lessons Learned from a Nation-wide CBD Promotion Project, Communications of The ACM, 45(10), October [3] ISO/IEC, FCD Information Technology Software product quality Part 1:Quality model, [4] McCall, J. A., Software Quality Management, A Petrocelli Book, [5] Boehm, B. W., Brown, J. R., Lipow, H., MacLeod, G. J. and Merrit, M. J., Characteristics of Software Quality, Elsevier North Holland, [6] Dromey, R. G., A Model for Software Product Quality, IEEE Transactions on Software Engineering, 21(2), February [7] OMG, CORBA Components, Version 3.0, Object Management Group, Inc., June [8] Heineman, G. T. and Councill, W. T., Component Based Software Engineering, Addison-Wesley, [9] Atkinson, C., Bayer, J., Bunse, C., Kamsties, E., Laitenberger, O., Laqua, R., Muthig, D., Paech, B., Wust, J., and Zettel, J., Component-based Product Line Engineering with UML, Pearson Education Ltd, [10] D Souza, D. F. and Wills, A. C., Objects, Components, and Frameworks with UML, Addison Wesley Longman, Inc.,