An investigation of quality profiles for different types of software T. Musson," E. Dodman* * Department of Computer Studies, Napier University, 219 Colinton Road, Edinburgh, EH 14 1DJ, UK Email: tim@dcs.napier.ac.uk School of Computing, Leeds Metropolitan University, The Grange, Beckett Park, Leeds, LS6 3QS, UK Email: e.dodman@lmu.ac.uk ABSTRACT This paper describes some of the trends which may be seen in the development of software quality models and their interaction with the software development process. Based on these trends two hypotheses are presented: 1. it is possible to define the quality requirements for any software by selecting one from a standard set of models (a model includes a profile of desired quality attributes) 2. based on readily identifiable characteristics, software types can be identified for which a particular model will be appropriate Some of the implications of these hypotheses are explored and a brief description is given of ongoing work which is aimed at determining their validity and usefulness. INTRODUCTION Several major methodological decisions need to be made, more or less consciously, in the development of a software system. These include the lifecycle model to be used, design methods to be used and the quality model to be used (defining non-functional or quality attributes of importance and the way in which they will be measured). This paper considers a possible approach to making these decisions.
120 Software Quality Management BACKGROUND The first serious software quality models were developed in the late 70s, starting with the work of McCall et al [1] and of Boehm et al [2]. These were "general" models which assumed that quality is an attribute which can be hierarchically decomposed into a wide range of measurable components. The top-level "quality factors" (such as maintainability, reliability, usability), which are mainly external attributes, are decomposed through a series of levels to give "quality metrics" which may be measured directly and are commonly internal code attributes. These models assumed that the same set of factors and decomposition was appropriate for all software products. The problems associated with the early general quality models have been widely documented (e.g. by Kitchenham [3] and by Musson [4]). The main problem of these general models, their lack offlexibility,has been addressed more recently by work such as that of Gilb [5] and that of Kitchenham [3]: both of these approaches advocate, and give guidelines for, the development of a local quality model for each project. The models proposed are still general (based on a range of attributes) and still use hierarchical decomposition. Kitchenham's COQUAMO includes attributes which are: 1. common to most applications, but have a general definition (e.g. maintainability) 2. common to most applications, but need specific definition (e.g. usability) 3. specific to certain types of system (e.g. integrity) 4. closely related to development process (e.g. correctness) Criticising Kitchenham's approach, Roche and Jackson [6] state that "A predictive model should provide interpretation of metric values and recommend action to be taken within context of the development process". This suggests that, for any particular software project, it may be appropriate both to define a local quality model and to use it to determine certain aspects of the development process (quality assurance procedures, development methods, testing criteria, etc). The trend towards the development of local quality models has been continued by Delen and Rijsenbrij [7]. Their work specifies attributes for a particular type of system (information systems), but emphasises the need for local specification of the requirements for each attribute. Another interesting development in this direction is quality function deployment (Erikkson and McFadden, [8]), which involves mapping customer requirements onto software characteristics and, in turn, software characteristics onto production features. Some work has been carried out specifically on the selection of an appropriate software process model for a particular project (Alexander and Davis [9]). This work considered both aspects of the product itself (e.g. product size and complexity, "ility" requirements, such as usability reliability
Software Quality Management 121 maintainability) and of the project's environment (e.g. funds available, staff profile, user's experience). The choice of the 20 criteria used was based on a literature search and informal observation, with most being subjectively evaluated as, effectively, low, medium or high. The above discussion shows a trend towards local models (i.e. adapting the quality model to the software being produced) and towards adapting the process to attributes of the software. This trend is explicitly supported by Fuggetta and Ghezzi [10]: "In general, much work is still needed to better understand what is a realistic and effective characterization of different application domains and problems with respect to the variety of available process technologies" (p55). If we accept this as being an appropriate direction then there still remain the issues of how, for any particular project, the local model is determined and how it is to be used in determining the development process. The following sections of this paper hypothesise an approach, related to that of Alexander and Davis [9], to resolving these issues and explore its consequences. THE HYPOTHESES It is proposed that: 1. it is possible to define the quality requirements for any software product by selecting one from a standard set of models (a model includes a profile of desired quality attributes) 2. based on readily identifiable characteristics, software types can be identified for which a particular model will be appropriate A quality model is regarded as a quality profile, together with any definitions needed in order to measure quality attributes. A software quality profile is a ranking of quality attributes in order of importance. In defining a profile it is considered appropriate to use high-level attributes, such as: maintainability usability reusability portability reliability efficiency The definitions which accompany the profile will normally include hierarchical decomposition of the attributes and low level metrics to allow measurement. An initial questionnaire-based investigation of the hypotheses has not given the information needed to confirm them (Musson and Dodman, [11]). However, it did confirm that there is considerable confusion with respect to quality metrics in the software industry: many quality attributes which selected by software developers as most or very important were often measured either very crudely or not at all.
122 Software Quality Management The failure to reach any conclusions concerning the hypotheses is considered to be a result of: 1. the small number (23) of questionnaires returned, 2. the difficulties referred to at the end of the previous section concerning the realistic and effective characterisation of application domains Both of these issues are currently (December 1994) being addressed in a followup survey. The questionnaire being used asks software quality managers to describe, in terms of certain characteristics, a software product which has been developed recently by their company. They are also asked to indicate the important quality characteristics together with their assessment procedures and to summarise important features of the development process used. It is hoped that this survey will allow the identification of the standard set of quality models, together with appropriate software types related to them. In the next section some different ways in which software can be characterised are presented, and in each case suggestions are made for quality attributes which are likely to be considered important and possible implications for the development process. Software characteristics which are considered to be important include both the application domain and the application environment. SOFTWARE TYPES The following categories of software are not intended to form a taxonomy: they are certainly not mutually exclusive. They are candidate characteristics (individually or in tuples) for identifying the types hypothesised above. Batch vs Real-time Reliability is likely to be the most important attribute for both these categories of software (the authors' previous survey [11] has found that reliability is perceived as being the most important quality attribute for software in general), particularly for process control systems. For batch software maintainability is also expected to be very important, as, typically, it is run many times over a long time period in a continually evolving environment. Such software is usually developed using the standard waterfall life-cycle model, or some variation of it, and a method such as SSADM or Yourdon. Apart from reliability, attributes which are considered particularly important for real-time systems are usability and efficiency. Typically, process control systems incorporate user involvement in the control feedback loop and hence require good usability. Similarly file interrogation and transaction processing systems will require high levels of usability. For most of these systems, where user interaction is involved, time efficiency is important to avoid
Software Quality Management 123 user dissatisfaction - this becomes particularly important for process control, where process instability may be dangerous. For such systems, particularly those which are safety-critical, formal or semi-formal methods are likely to be appropriate for development, together with prototyping of the user interfaces Interactive vs non-interactive Usability is likely to be the most important attribute for interactive systems. Reliability is also likely to be important to an interactive user. This suggests that prototyping is likely to be important in the development process, allowing incremental development of the interface. A fairly formal approach to development (including the interface design) may be appropriate for the required reliability. Much interactive software is developed for use in a Windows environment. It is becoming normal to develop this making considerable use of standard libraries and object-oriented development. This facilitates development by encouraging considerable reuse of code and also makes it easier to conform to a standard interface style hence giving a greater level of usability. Goal-oriented vs Output-oriented The distinction between goal-oriented and output-oriented software has been made by Bull and Musson [12]. The essential difference is that the output of output-oriented software is seen as intrinsically useful (perhaps to be taken away as hard copy) whereas that from goal-oriented software is only considered useful in terms of supporting the user in achieving a goal. Traditional data processing systems tend to be very output-oriented whereas decision support systems, computer based learning environments and other knowledge-handling systems tend to be goal-oriented. Correctness will be an important attribute for most output-oriented systems whereas usability will be important for goal-oriented systems. A much more difficult aspect of the evaluation of goal-oriented systems concerns the assessment of the extent to which the user is supported in achieving the goal. This goes considerably beyond usability and requires an analysis of user behaviour and a comparison of the success in performing relevant tasks both using the system and either unsupported or using other systems. The development of output-oriented software is likely to be carried out using a conventional waterfall model life-cycle (or similar). However, the development of goal-oriented software should involve a high degree of prototyping, checking that the goal is being met at each stage of development: this can only be achieved by continually involving members of the target user group to obtain feedback into the development process.
124 Software Quality Management Long-lived vs Short-lived The expected lifetime of a software system can be expected to influence the required level of most quality attributes, particularly maintainability. Short lifetime systems will often be developed as cheaply as possible with a very low level of quality assurance: it is usually not considered worthwhile spending a great deal of money on achieving high quality levels for such systems. Such systems may well be produced with very little formal design and testing and a small amount of documentation. Appropriate development methods may include heavy reliance on reuse or modification of code from previous projects. Preliminary results from the survey currently being undertaken suggest that the importance attached to maintainability is directly related to the expected lifetime: companies producing software products for which a lifetime of around ten years is expected normally class maintainability as one of the two most important quality attributes, whereas for shorter lifetime products it does not always appear as a required attribute at all. DISCUSSION The previous section has given some potentially useful ways of characterising software products and has considered their implications in terms of expected desired quality attributes and development issues. There are many other ways in which software may be characterised: these include data processing, systems software, networked, decision support, expert system, computer based learning environment, market sector, safety-critical. The usefulness of these categories in the context of our hypotheses is an open research issue, however it seems fairly sure that some of them will be relevant The characteristic "safety-critical" implies a need for high reliability and this suggests that formal methods will be an appropriate technique. It is also anticipated that market sector will have a strong influence on the required quality attributes: this characteristic itself may be seen as having several components, including commercial sector, vertical vs horizontal and inhouse vs external user. The hypotheses themselves have arisen from a consideration of trends in the development of software quality models, in particular the movement from general, fixed models towards specific, locally-developed models. They have the potential for giving considerable guidance to software developers in terms of both appropriate development methods and quality models.
CONCLUSION Software Quality Management 125 It has become clear from recent work in the area that quality models and development methods should be defined locally in the context of a particular project. Suitable processes for defining these are still not clear. This paper has discussed one possible approach to this problem. Its validity is not yet established, however, survey work currently being undertaken is aimed at demonstrating this. Results from the survey are expected to be available by April 1995. ACKNOWLEDGEMENT The authors would like to express their gratitude to Professor Inger Eriksson for her constructive and valuable comments on an earlier draft of this paper. REFERENCES 1. McCall, J.A., Richards, P.K., Walters, G.F. 'Factors in Software Quality' Vols I, II, III. US Rome Air Development Center Reports NTIS AD/A-049 014,015,055, 1977 2. Boehm, B.W., Brown, J.R., Kaspar, L.R., Lipow, M., MacCleod, G.J., Merrit, M.J. Characteristics of Software Quality. North Holland, 1978. 3. Kitchenham, B. Towards a Constructive Quality Model. Part 1: Software Quality Modelling, Measurement and Prediction* Software Engineering Journal, Vol.2,4,ppl05-113, 1987. 4. Musson, T. The Evaluation of Instructional Software Quality' Proceedings of SQM'94, Edinburgh, UK; Computational Mechanics Publications, 1994. 5. Gilb, T. Principles of Software Engineering Management. Addison Wesley 1988. 6. Roche, J. and Jackson, M. 'Software Measurement Methods: Recipes for success?' Information and Software Technology, Vol. 36, 3, pp!73-189, 1994. 7. Delen, G.P.AJ. and Rijsenbrij, D.B.B. The Specification, Engineering, and Measurement of Information Systems Quality' Journal of Systems and Software, Vol 17, 3,pp205-217, 1992.
126 Software Quality Management 8. Erikkson, I. and McFadden, F. 'Quality Function Deployment: a Tool to Improve Software Quality' Information and software Technology, Vol. 35, 9, pp491-498, 1993. 9. Alexander, L.C. and Davis, A.M. 'Criteria for Selecting Software Process Models' Proceedings of IEEE COMPSAC91, Washington DC, pp521-528, IEEE Computer Press, 1991. 10. Fuggetta, A. and Ghezzi, C. 'State of the Art and Open Issues in Process- Centered Software Engineering Environments' Journal of Systems and Software, Vol. 26, 1, pp53-60, 1994. 11. Musson, T. and Dodman, E. 'An Investigation of Desired Quality Attributes for Different Types of Software' Proceedings of ICSQP'94, Hong Kong: Software Quality and Productivity: Theory, Practice, Education and Training, (Lee, M. et al, eds), Chapman and Hall, 1995. 12. Bull, S. and Musson, T. 'Goal-Oriented Software: When the Development Process is Different...' Proceedings of SQM'94, Edinburgh, UK; Computational Mechanics Publications, 1994.