Quality in Use: Achieving Stakeholder Needs for Quality

Transcription

1 1 Quality in Use: Achieving Stakeholder Needs for Quality Nigel Bevan Professional Usability Services Nigel Bevan. Reproduction permitted provided the source is acknowledged. 1 Two views of usability Product measure ( Little usability ): Nielsen (1993) System acceptability Social acceptability Practical acceptability Cost Compatibility Reliability Usefulness Utility Usability Outcome of interaction ( Big usability ): ISO (1998) The extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use 2

2 2 Usability as quality in use (ISO ) Big usability Context of use Goals Task Inputs Product (little usability) User Environment Task Outputs % Effectiveness Efficiency (Task Time) Satisfaction 3 3 ISO standards for usability ISO (1998) Guidance on usability Defines usability as an outcome of interaction ISO/IEC (2001) Software engineering -- Product quality -- Part 1: Quality model Defines usability as a product measure (little usability) Defines quality in use as an outcome of interaction (big usability) ISO/IEC CD (2010): Quality models for software product quality and system quality in use Extends the concept of quality in use (big usability) 4

3 3 ISO/IEC Software Product Quality Model q u a l i t y i n u s e functionality reliability accuracy suitability interoperability security maturity fault tolerance recoverability availability maintainability analysability changeability stability testability usability understandability learnability operability attractiveness efficiency time behaviour resource utilisation portability adaptability installability co-existence replaceability 5 Including negative consequences (ISO/IEC CD 25010) Safety The degree to which a product or system does not, under specified conditions, lead to a state in which human life, health, property, or the environment is endangered Economic damage (commercial property, operations or reputation) Health and safety (people) Environmental harm (property or the environment) Managers may be more concerned with risk-avoidance than increased business benefit! 6

4 4 UK Passport fiasco cost taxpayers 12m UK Passport Office 1999 New software for issuing passports took operators twice as long Caused delays of up to 3 months in obtaining a passport Huge cost of additional clerical staff UK National Audit Office recommendations: Organisations should pay special attention to the interaction between the new system and those expected to use it, and take into account users views on the practicability and usability of the new system. 7 Quality in use (usability + safety) Context of use Goals Avoid Adverse Consequences Task Inputs System User Environment Task Outputs % Effectiveness Efficiency (Task Time) Satisfaction 8 8

5 5 The value of setting quality in use goals The goal is to achieve effectiveness, efficiency, safety and satisfaction Specifying requirements for effectiveness, efficiency, safety and satisfaction in each intended context of use leads to design requirements necessary to achieve the desired levels of effectiveness, efficiency, safety and satisfaction, eg: Accessibility Preventing errors that would lead to commercial risk It is important to specify these requirements, even if they will not all be measured For web sites, effectiveness, efficiency, safety and satisfaction can now be easily be measured using remote usability testing 9 Requirements and Quality stakeholders' unstated needs actual effect on stakeholders Business Requirements Quality in use needs Test quality in use Acceptance Tests Detailed requirements (including little usability) Software Specifications described solution Factory Gate Tests Test product characteristics Ian Alexander 2009 Software Product implemented solution

6 6 Additional requirements for usability/quality in use Learnability Accessibility Flexibility Context conformity 11 Learnability (ISO CD 25010) The extent to which a product can be used by specified users to achieve specified learning goals with effectiveness, efficiency, safety and satisfaction in a specified context of use Establish goals for the effectiveness, efficiency, safety and satisfaction of learning Design product features to support these goals 12

7 7 Accessibility (ISO/IEC CD 25010) The extent to which a product can be used by users with specified disabilities to achieve specified goals with effectiveness, efficiency, safety and satisfaction in a specified context of use Establish goals for the effectiveness, efficiency, safety and satisfaction for users with particular types of disabilities Design product features to support these goals W3C-WAI Web Content Accessibility Guidelines 13 Context conformity (ISO/IEC CD 25010) The degree to which usability and safety requirements are met in all the intended contexts of use It is important to specify usability and safety requirements for all intended contexts of use All potential types of users All potential types of goals and tasks All potential social and organizational environments All physical environments 14

8 8 Flexibility (ISO/IEC CD 25010) The usability and safety in contexts beyond those initially explicitly identified Enables the product to be used for purposes that cannot be specified in advance Requirements are never complete Design product features to support these goals Personalization/customization features (eg parameters or macros) Avoid unnecessary constraints (eg Excel) 15 Additional requirements for usability/quality in use Quality in use Effectiveness Efficiency Safety Satisfaction Learnability (quality in use for learning) Accessibility (quality in use for people with disabilities) Context conformity (quality in use for specified contexts of use) Flexibility (quality in use for additional contexts of use) 16

9 9 Broadening satisfaction ISO Satisfaction Freedom from discomfort, and positive attitudes towards the use of the product ISO/IEC Satisfaction Likeability (pragmatic user goals) Pleasure (hedonic goals) Comfort (physical satisfaction) Trust (satisfaction with security) 17 Quality in use for a wider range of stakeholders Stakeholder: End User User experience Usage Organisation Cost-effectiveness Technical support Maintenance Characteristics Goals: Personal pragmatic and hedonic goals Task goals Support goals System effectiveness Perceived effectiveness Task effectiveness Support effectiveness System resources Perceived time Productivity Cost efficiency (money) Support cost Adverse consequences Perceived risks (e.g. of making a mistake) All potential risks System failure or corruption Stakeholder satisfaction User satisfaction with personal goals User satisfaction with task goals Support satisfaction 18

10 10 What type of goals matter to an organisation? Performance targets have been criticised for distorting organisational objectives Quality in use goals can be set at a high level to match organisational goals related to the productivity, effectiveness in meeting user needs, satisfaction and risk associated with organisational outputs 19