Quality Assurance & Standards PSD3 Nov 2013 1
Overview Quality Assurance (QA)» checking what we are producing QA standards» checking the checking 2
Further Reading Sommerville» 9 th Edition, Chap. 24» 7 th & 8 th Editions, Chap. 27 3
Capability Maturity Model framework for improving software engineering processes Recommended set of best practices Organizations are appraised 4
5
CMM and Quality Key feature of CMM is centrality of process and product quality" How is this achieved?" Answer: Quality Assurance or QA" 6
What is Quality Assurance? Quality Assurance is the process of defining how software quality can be achieved and how the development organisation knows that the software has the required level of quality. [Sommerville]" 7
What Kinds of Problem defect failure to meet requirements fault condition that lead to a system failure bug error, mistake or omission in the software which are faults (lead to failure or unexpected behaviour) Note: quality is not simply the absence of faults 8
Sommerville on Quality Assurance for Sommerville:"» Quality Assurance framework of procedures and standards"» Quality Plan selection and adaptation of procedures and standards for a project!» Quality Control carrying out processes that ensure procedures and standards are followed" the term Quality Assurance sometimes refers to all of the above" 9
What s involved in software quality assurance? At the level of an institution, company or standards body"» Develop procedures and standards"» Perform certification" to prove that the QA mechanism used is acceptable and effective 10
What s involved in software quality assurance? At the level of a particular project"» Prepare a quality plan" specifying processes, deliverables, measures of quality (metrics, standards)"» Carry out quality control" Collect data" Metrics" Compared to standards" Conducting reviews" checking reality against plan and against standards" 11
What s involved in software quality assurance? At all levels"» Change attitudes" convince staff that quality is important" develop a quality culture " 12
Techniques for Assessing Quality progress review» examination of progress with respect to plans quality review» examination of project artefacts with respect to attributes of quality» more or less structured Inspection» a (technical) review that tries to identify faults and lack of conformity to standards» not led by author» often applies to code code walkthrough» less formal than inspection» author leads pair programming deskcheck or passaround ad hoc review 13
Review Dos and Don ts DO make it peer group review, applicable to all stages of software development"» method of finding faults cheaply"» method of training and learning"» method of control"» method of encouraging egoless teamwork " IT IS NOT"» a problem-solving session faults should be identified, but solutions should not be patched together in committee "» a managerial appraisal of personnel" 14
Code Review 15
Standards for Assessment Lots of sources of standards"» Standards Bodies"» Professional Associations"» Technology and Service Providers"» Company Standards (e.g., House Style )" Can be based on"» Objective Software Metrics"» Conventions and Agreements"» Informal Professional Judgements" 16
reviewing the inspection process statistics collected on"» details of items inspected"» list of faults found & classification"» resources required for re-working"» number of people involved & time" analysis provides"» fault checklists"» management reports on effectiveness of inspections" It s the process being assessed, not the authors or inspectors" 17
What is a Standard? "A standard is a document approved by a recognized body, that provides, for common and repeated use, rules, guidelines, or characteristics for products, processes or services with which compliance is not mandatory. " "A Guide to Project Management Body of Knowledge, 1996" 18
Standards Organisations International Organization for Standardization (ISO)"» non-governmental"» develops standards for various technical fields (more than 11000)"» 120 national members, which are themselves standards organisations" 19
Standards Organisations BSI (UK)" ANSI (USA)" DIN (Germany)" IEC (International)" Other standard setting bodies include"» IET"» BCS"» EU"» W3C"» OMG" 20
Software Engineering/IT Standards ISO/IEEE/IET/W3C/EU have specific standards that can be used for measuring product and process quality E.g.,» ISO/IEC TR 14471:1999 : Information Technology Software Engineering Guidelines for the adoption of CASE tools» ISO 9241: Ergonomics of Human System Interaction Includes requirements for keyboard and non-keyboard input devices, workstation layout, displayed colours, menu/command/ direct manipulation and form-filling dialogues» W3C Standard = recommendation currently includes: XML 1.1 (plus XQuery, XPath), XSLT, HTML, XHTML, SOAP, SSML, VoiceXML, OWL, RDF, P3P, SMIL, 21
ISO 9000 international standard for quality management and quality assurance"» states what must be in a quality management system"» first established in 1987"» derived from BS 5750, a British standard" ISO 9001"» applies to products involving design"» Latest version is ISO 9001:2008"» 90003 interprets 9001 for software development" 22
ISO 9000 (cont d) certification is not carried out by ISO"» carried out by independent certification bodies"» organisation is awarded a Certificate of Conformity" 23
What s in ISO 9000? covers 20 topics, including"» Quality System"» Design Control"» Process Control"» Inspection & Testing"» Contract Review"» Quality Records"» Internal Quality Audits"» Training" 24
What s in ISO 9000? (cont d) standard is abstract; it sets ends not means" for example," " The supplier shall establish and maintain a documented quality system as a means of ensuring that product confirms to specified requirements. [from Section 4.2 of ISO 9001]" 25
TickItPlus developed by UK and Sweden" provides a nationally accredited software quality certification scheme" interpretation of ISO 9000 and related standards" 26
www.tickitplus.org 27
TickItPlus (cont d) gives concrete guidelines on how software development should conform to the standard"» based on developing a scheme of internal audits"» audits related to standards compliance"» auditors require training"» audits involve document reviews and staff interviews" 28
QA Standards: A Good Thing? pro"» makes quality assurance assessable"» as of the end of 2009, 1,064,785 organisations in 161 countries held ISO 9001:2000 certificates"» US survey" 89% reported greater operational efficiency" 48% reported increased profitability" 29
QA Standards: A Good Thing? con"» fosters command & control style of management" emphasises inflexible compliance with a set of rigid written rules"» standards rely heavily on assessors judgements" standards are not completely objective" 30
QA Standards: A Good Thing?» staff will pay attention to controls, not the things affected by the controls" attention to quality inspection and monitoring can deflect from attention to quality itself" like the problem of exams distorting education" 31
Tutorial this afternoon Please sit in teams Each team will split in half the inspectors and the inspected Inspect outcomes from other team s prototyping sprint last week Inspection checklist will be available 32