Evaluating and Improving Software Usability

Transcription

1 Evaluating and Improving Software Usability 902 : Thursday, 9:30am - 10:45am Philip Lew

2 Understand, Evaluate and Improve 2

3 Agenda Introduc7on Importance of usability What is usability? What is UX? Usability Modeling and Measurements Case studies on measuring and improving Summary

4 What is Quality? Pertaining to soeware J Requirements 4

5 Importance of Usability

6 Why is Usability Important? Usability is important especially for web applica7ons And now especially mobile applica7ons Saas, mobile are in alignment with and soon becoming equivalent to cloud Most prevalent implementa7on method in the next 5 years Without good usability: Users will leave the applica7ons For mobile, if they can t learn in 30 seconds, they won t come back

7 Web and Mobile have Changed the Business models have changed Instead of paying upfront and owning the soeware Pay as you go, pay by subscrip7on Behavior and expecta7ons have changed Terrain

8 What is Usability In rela7on to quality In rela7on to user experience

9 When you think Usability Naviga7on Efficiency Responsiveness- performance Learnability AZrac7veness Understandability Anything else come to mind?

10 Current Research Usability User Experience Quality Satisfaction Usability ISO ISO Usability Design Experts Product Owners ISO 25010

11 Source: ISO Usability - Key Characteris7c of Product Quality

12 Quality in Use ISO 25010

13 Notes on Sa7sfac7on and Usability don t have what I want Highly usable software My password doesn t work Nice weather today low usability software Finished my work today I m unsatisfied I m satisfied!! Sa7sfac7on is a subjec7ve feeling dependent on many things other than usability: A user can be highly sa7sfied but the applica7on with low usability. An applica7on can be highly usable (high usability) but the user is not sa7sfied!

14 Usability- Major Component of Quality Quality and Usability in the so1ware development lifecycle Usability Usability Can be measured from the design point of view or of the product Can be measured in- use with real users

15 Evolu7on of SoEware quality ISO 9000 ISO 9126 ISO 9241 ISO Type of quality Process quality Software Quality (internal) Software Quality (external) Software Quality (in use) What is measured Software processes Code Product? How measured? CMMI assessment model white box testing black box testing? Who measures? CMMI Assessment Company Programmer Tester End User

16 Source: ISO What is Usability- Effect of the SoEware Product Degree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

17 Usability in Actual Usage User role Objec7ve Task Environment Domain specified users specified goals specified context of use What else can you think of?

18 Usability and User Experience UX, Usability and Quality in Use Difficult to understand their relationships Lack of consensus in meaning in literature Not totally clear how related to quality Standard definition for user experience is still not available

19 Some UX Defini7ons UX DESCRIPTION D1 a continuous process of user engagement with the product [10] D2 entire set of affects that results in user-product interaction [11] D3 the evolution of usability [12] D4 elaboration of the satisfaction component of usability [13] D5 a categorization of do-goals (pragmatics) and be-goals (hedonics) [3], [14] D6 infinite small experiences relating to people, products and contexts [6] D7 consequence of user s and product s characteristics when interacted in a specific environment [1] D8 degree to which specified users can achieve actual usability, safety, and satisfaction in use in a specified context of use [15] D9 A person's perceptions and responses that result from the use or anticipated use of a product, system or service [16] D10 the degree of positive or negative emotions that can be experienced by a specific user in a specific context during and after product use and that motivates for further usage [17]

20 User Experience (UX) Pragmatic (do-goals) : refers to the product's perceived ability to support the task achievement and focuses on the product s utility and usability in completing tasks, the do-goals of the user. Hedonic (be-goals) : product's ability to support the user s achievement of be-goals, such as being satisfied, pleasure. Real goal of the end user is fulfilling be-goals such as being autonomous, competent, related to others, stimulated, etc. Be-goals are the driver of UX Hassenzahl

21 Rela7ng Usability and User Experience Quality characteris7c 508 compliancy Func7onality User Interface Usability Learnability Product azributes Text/Sound azributes Appropriate Func7ons Easy to use Easy to learn UX Pragma7c Do Goals Usability Measurement Effec7ve and Efficient Effec7veness in use Efficiency in use Learnability in use UX Hedonic Be Goals Sa7sfac7on (pleasure, sense of community ) Sa7sfac7on Measurement Pleasure Comfort Trust Sense of Community Context USER EXPERIENCE 21

22 Now that we understand it ALL We Need to Define Usability For Our Own Purposes 22

23 Let s Define Usability/UX From the Product Viewpoint Usability Characteris7c Naviga7on 1 Understandable Characteris7c 2 Characteris7c n Subcharacteris7c Control Stability 1 Subcharacteris7c Previous- Next 2 Subcharacteris7c n Home AZribute Loca7on 1 Save AZribute Loca7on 2 Ease of finding AZribute n

24 Defining Usability from an Effect- Real usage Point of View Usability Quality Effec7veness Efficiency Sa7sfac7on Characteris7c n Accuracy Completeness Subcharacteris7c n Errors AZribute 2 AZribute 3 AZribute 1

25 Some Poten7al AZributes Measurements for in use Usability Effec9veness Comple7on rates Error rate Help usage Efficiency Task 7me Backtracking Learnability Learning rate Task 7me devia7on Can you think of any others, par9cular to your applica9on?

26 Adding in UX Sa9sfac9on in use Degree to which users are sa7sfied in a specified context of use. Sa7sfac7on is the user s response to interac7on with the product, including amtudes towards use of the product. Subdivided into sub-characteris7cs: Likability (cogni7ve sa7sfac7on) Pleasure (emo7onal sa7sfac7on) Comfort (physical sa7sfac7on) Trust UX Component 26

27 Example of Product Quality Model External Quality Requirements (for Shopping Cart Entity) 1 Usability with AZributes (Operability in ISO 25010) 1.1 Understandability Icon/label ease to be recognized Information grouping cohesiveness 1.2 Learnability Ease of Use Control permanence Helpfulness

28 Learnability Degree to which the soeware product enables users to learn its applica7on - 7 principles (ISO ) Suitability for the task - should be suitable for the user s task and skill level Self- descrip7veness- should be clear what the user should do next Controllability - the user should be able to control the pace and sequence of the interac7on Conformity with user expecta7ons - should be consistent Error tolerant - forgiving Suitability for individualiza7on - should be able to be customized to suit the user Suitability for learning - should support learning 28

29 Ease of use Degree to which the soeware product makes it easy for users to operate and control it. controllability error tolerance (by operator) conformity with user expecta7ons 29

30 Ease of Use - Helpfulness Degree to which the soeware product provides help when users need assistance including help that is: easy to find comprehensive effec7ve 30

31 Example: Usability Measurement ACribute Scale How Calcula9on Goal Current Help completeness Percent of Menu items with help Compare menus and help items % 90% 40% Ease of access Keystrokes to find/use a feature/ func7on/ informa7on Sample 50 items average Consistency Number loca7ons for same buzon Examine menus and doc. integer 1 5 What types of usability measures are these? 31

32 Using a measurement model Usability Evalua7on composed of Indicator composed of Evalua7on Evalua7on Characteris7c Metrics Measurement Func7on Evalua7on Evalua7on Sub- characteris7cs Measurements 32

33 Usability- Review usability usability in use UX Can be measured from the design point of view or of the product Can be measured in- use with real users

34 Measuring Usability Methods and models Now that we know what it is, and what to measure, then HOW? 34

35 Measurable AZributes Usability characteris7c Descrip7on and purpose How to measure What is measured Measurement/Calcula7on Range (min, max) Metric Once you have a model (what you are going to measure), then you start doing IT! Objec7ve (goal) versus Current (indicator)

36 UX/Usability Measurement Methods Ques9on naires Focus groups Heuris9c Evalua9on Observa9on Labs Logging Let s get started

37 Measurement methods Expert Evalua9on (Heuris9c evalua9on) Define a set of rules or criteria with measurements and evaluate against them Web- based logs Collect user ac:vity data Mistakes and errors How long it takes to complete tasks Comple:on rates Ques9onnaires Quan7ta7ve subjec7ve measurement of UX characteris7cs 37

38 Heuris7c Evalua7on Using a checklist But a lizle more complicated than yes/no 38

39 Heuristic Evaluation of a Pharmacy application Each usability attribute has a quantifying metric. For example, for Predic9ve textual acribute, users should be able to understand a bucon s results prior to pressing it. Direct metrics need to be designed, i.e.: (0) No support at all (1) Par9al (2) Complete Need a mapping from 0, 1, 2 to something more understandable, i.e. 2 = 100, and 1 = 60 Need an indicator to interpret the level of sa9sfac9on met with decision criteria with acceptability ranges in a percentage scale: 0-40 (unsa9sfactory red) means changes must take place with high priority; (marginal yellow) indicates a need for improvement ac9ons; indicates a sa9sfactory level green- for the analyzed acribute. 39

40 2.2.2 Error Recovery Support In the current state, users filling a new prescription are supported well in error recovery (2.2.2) in automatic cursor positioning 40

41 Heuristic Evaluation Pharmacy Software The purpose of the evaluation Understand the external quality level of the learnability sub-characteristic for filling a new prescription Table 2 - excerpt of the whole current evaluation 41

42 Heuris7c Evalua7on of Usability- Shopping Cart External Quality Requirements Measure EI value P/GI value Global Quality Indicator 61.97% 1 Usability 60.88% 1.1 Understandability 83% Icon/label ease to be recognized 100% Information grouping cohesiveness 66% 1.2 Learnability 51.97% Operability 49.50% Control permanence 100% Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% Basic Information Coverage 50% Line item information completeness 2 50% Product description appropriateness 50% Coverage of other Contextual Information 76.89% Return policy information completeness 33%

43 JIRA Heuristic Usability Evaluation

44 Usability Logging Collec7ng user ac7vity and behavior Tradi7onally used for other things like 44

45 Usability Logging Iden7fy users by using session ID to iden7fy a unique user. Itera7vely insert code into the applica7on Collect data Analyze the data for each azribute in different dimensions and aggrega7ons Determine the need for further calcula7ons and what azributes to measure further Revise the data we are collec7ng, adding or decreasing granularity

46 Some easy and some hard 46

47 Jira Evaluation Summary 47

48 Using Both Product Perspec7ve and the User Perspec7ve If we find some azribute with low performance from the user perspec7ve Then we can go back to the product perspec7ve, figure out why, and improve 48

49 49 JIRA Evaluation by Attribute Excerpt

50 For example:login BuZon Visibility (product usability azribute) 50

51 Using Ques7onnaires The process: 1. Use your model of what you want to measure and improve 2. Design the ques7onnaire according to the model 3. Execute the ques7onnaire 4. Analysis

52 One UX Model

53 Ques7onnaires Used SUMI (So1ware Usability Measurement Inventory ) QUIS (Ques:onnaire for User Interface Sa:sfac:on ) PSSUQ (Post- Study System Usability Ques:onnaire ) ASQ (A1er- Scenario Ques:onnaire ) PUTQ (Purdue Usability Tes:ng Ques:onnaire ) PHUE (Prac:cal Heuris:cs for Usability Evalua:on ) SUS (System Usability Scale ) IUI (Isometrics Usability Inventory)

54 54

55 55

56 What do you do with all this? 56

57 Itera7ve Improvement Measure external quality usability with heuris7cs Measure in- use usability with logging or ques7onnaires

58 Goal is to Understand, Evaluate and Improve 58

59 Next Steps Produce an ac7on plan What usability azributes are important to your organiza7on? Develop a model What data can you collect/which technique can you use Maybe some elements of the model drop out- can t be measured that easily Start collec7ng and developing benchmarks

60 Conclusion Usability and UX are both abstract concepts Cri7cal component of quality Defining is different for each organiza7on Need a model for your organiza7on The model is the founda7on of what to measure Once you can measure, then you can evaluate and improve

61 Resources These are all journal publications that can be accessed via these links 61

62 Thanks Ques7ons and Answers Please fill out an evalua:on form and drop it in the collec:on basket located at the back of the room. Philip phone: