INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Usability Inspection Methods SMD157 Human-Computer Interaction Fall 2003 Nov-20-03 SMD157, Usability Inspection Methods 1 L Overview Usability Measures Formative Evaluation Details of Heuristic Evaluation Example Nov-20-03 SMD157, Usability Inspection Methods 2 L INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Usability Measures Nov-20-03 SMD157, Usability Inspection Methods 3 L 1
Kinds of Evaluation Summative evaluation - Goal is to assess the impact, usability, and effectiveness - Occurs after one has a system or completed subsystem - More structured testing, goal is to measure Formative evaluation - Goal is to refine and formulate the design - Occurs during the design-evaluate-redesign cycle. - Can be performed on system fragments - Less structured testing, goal is to learn Nov-20-03 SMD157, Usability Inspection Methods 4 L Usability Measures Ease of learning - Important if there are many novices - Usually defined as time to reach a specified level of performance - Tests both system and training materials Efficiency of use - Important for experience, frequent use - For a new system, one must wait until users are experienced - Measuring on the job performance is best Nov-20-03 SMD157, Usability Inspection Methods 5 L Usability Measures Error rates - Another efficiency measure - Complete elimination is impossible, recovery and tolerance are important - Often must be traded of against efficiency and satisfaction Ease of remembrance - Important for infrequent users - Involves re-testing after a period of non-use - Note, memorable isn t always easy to learn Nov-20-03 SMD157, Usability Inspection Methods 6 L 2
A Road Sign Found Near Many Washington, DC Metro Stations Kiss and Ride Nov-20-03 SMD157, Usability Inspection Methods 7 L Usability Measures Flexibility - Ability to adapt to variations in work routines - Subjective and hard to measure. Subjective satisfaction - Measured by survey + Questions should be both negative and positive to avoid the give it all sevens syndrome. + Questions should ask about specific things, not just, How did you like it? + Users remember the worst incident and base their evaluation on that. - Important for system acceptance Nov-20-03 SMD157, Usability Inspection Methods 8 L Example Satisfaction Questions 3.1 Overall reactions to the system: terrible wonderful 1 2 3 4 5 6 7 8 9 NA frustrating satisfying 1 2 3 4 5 6 7 8 9 NA 5.4.1 Instructions for commands or functions confusing clear 1 2 3 4 5 6 7 8 9 NA 5.4.2 Instructions for correcting errors confusing clear 1 2 3 4 5 6 7 8 9 NA From the Questionnaire in User Satisfaction (QUIS) Nov-20-03 SMD157, Usability Inspection Methods 9 L 3
INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Formative Evaluation Nov-20-03 SMD157, Usability Inspection Methods 10 L Formative Evaluation Evaluation can be with or without users. Reasons for user involvement - Experts/designers cannot put themselves in the user s place, background is different - The design team is too close to the system, has a good mental model - True feedback Reasons against user involvement - Time consuming - Requires a somewhat complete system - May not be easily accessible Non-user methods are called inspection methods. Nov-20-03 SMD157, Usability Inspection Methods 11 L Inspection Methods Can gain valuable information. Can be focused on aspects that may take many users to measure. Can be focused on aspects that are not measurable with users. Recommended procedure: - Use both user and inspection methods - Proceed any evaluation with users by an inspection and fix the problems found Nov-20-03 SMD157, Usability Inspection Methods 12 L 4
Inspection Methods Guideline review - single/group, inspect UI with respect to guidelines Pluralistic usability walkthrough - mixed group, inspect UI with respect to heuristics Consistency inspection - like guideline review, focuses on internal & external consistency Standards inspection - as above, but focuses on conformance to standards Nov-20-03 SMD157, Usability Inspection Methods 13 L Inspection Methods Cognitive walkthrough - expert with tasks, focuses on problems performing the task. Formal usability inspection - team, like a software inspection Heuristic Evaluation - group inspects individually using heuristics Nov-20-03 SMD157, Usability Inspection Methods 14 L INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Heuristic Evaluation Nov-20-03 SMD157, Usability Inspection Methods 15 L 5
Heuristic Evaluation Uses non-experts as well as experts. Assumes that individual evaluators will miss things, but that a group of 5-10 individuals will find most of the important problems. Requires the evaluations to be independent. Can be performed before there is an operational prototype. Nov-20-03 SMD157, Usability Inspection Methods 16 L Ten Usability Heuristics Visibility of system status - Appropriate feedback - Timely feedback Match between system and the real world - Speak the users' language - User, not system-oriented terms. - Information appears in a natural and logical order. User control and freedom - Clearly marked "emergency exits" - Support undo and redo. 1 2 3 4 Nov-20-03 SMD157, Usability Inspection Methods 17 L Ten Usability Heuristics Consistency and standards - Same words, situations, and actions throughout. - Follow platform conventions. Error prevention Recognition rather than recall - Make objects, actions, and options visible - The user should not have to remember information - Instructions for use of the system should be visible or easily retrievable 1 2 3 4 Nov-20-03 SMD157, Usability Inspection Methods 18 L 6
Ten Usability Heuristics Flexibility and efficiency of use - Accelerators - Allow users to tailor frequent actions. Aesthetic and minimalist design - Dialogues should not contain information which is: + Irrelevant + Rarely needed 1 2 3 4 Nov-20-03 SMD157, Usability Inspection Methods 19 L Ten Usability Heuristics Help users recognize, diagnose, and recover from errors - Error messages should be expressed in plain language - Precisely indicate the problem - Suggest a solution. Help and documentation - Easy to search - Focused on the user's task - List concrete steps to be carried out - Not too large. 1 2 3 4 Nov-20-03 SMD157, Usability Inspection Methods 20 L Steps for a Heuristic Evaluation Give each individual a copy of the prototype or system - Scenarios/task optional, can be helpful. Individually inspect the system using the heuristics - May be screen-by-screen with respect to all heuristics, or - Heuristic-by heuristic with respect to all screens - For audio based systems, replace screen with interaction point Nov-20-03 SMD157, Usability Inspection Methods 21 L 7
Steps for a Heuristic Evaluation Note problems and record: - Place (screen/interaction point) - Description - Heuristics violated Merge individual records into a group problem list Individually evaluate the severity of the problems Average the severity ratings Nov-20-03 SMD157, Usability Inspection Methods 22 L Severity Ratings Severity should combine - Frequency - Impact - Persistence Scale - 0 = don t agree that it is a problem - 1 = Cosmetic - 2 = Minor, fix if time/easy - 3 = Major, important to fix - 4 = Catastrophe, fix before release Nov-20-03 SMD157, Usability Inspection Methods 23 L Variations in the Method One may use a different set of heuristics. - For example, Shneiderman s 8 golden rules One may add new heuristics to the set - For example, some of the speech guidelines One may delete some heuristics - For example, the help and documentation Nov-20-03 SMD157, Usability Inspection Methods 24 L 8
Pros and Cons Pros - Cheap. - Finds many problems. - Can be used very early. Cons - Does not use real users. - Evaluators are usually on the design team, unfortunately they know how the system works. - Non-expert evaluators may waste time on false problems. - Tends to find static problems and miss dialog problems Nov-20-03 SMD157, Usability Inspection Methods 25 L INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Example Nov-20-03 SMD157, Usability Inspection Methods 26 L Alternate All-in-One Interface Drop-down list Originals: 1-sidedÍ Displays settings for detectí the selected configuration Copies: 1-sidedÍ sortedí staple upper leftí plain, A4Í exact sizeí Account researchí Lightness Contrast Fill in field # of copies 1 Start Pressing drops down a list of configurations. New and delete are on the list. See the Configuration Management Page. Pressing here causes the document loaded into the copier to be duplicated. Go to the "status page". H1 H2 H3 H4 Nov-20-03 SMD157, Usability Inspection Methods 27 L 9
Configuration Management Page Originals: 1-sidedÍ 1-sided to 2 detectí book to 1-sided, A4 Copies: 1-sidedÍ *New Configuration sortedí *Edit Configuration staple upper leftí *Delete Configuration plain, A4Í exact sizeí Account researchí Lightness Contrast # of copies 1 Start Selecting an item: 1. Selects configuration, 2. Pops-up field for New, 3. Pops-up a list for Replace and Delete. (Replace saves changes made to setup.) H1 H2 H3 H4 Nov-20-03 SMD157, Usability Inspection Methods 28 L Selecting Replace Originals: 1-sidedÍ 1-sided to 2 detectí book to 1-sided, A4 Copies: 1-sidedÍ *New Configuration sortedí *Edit Configuration staple upper leftí *Delete Regular Configuration Duplicate plain, A4Í 1-sided to 2 exact sizeí book to 1-sided, A4 Account researchí Lightness Contrast # of copies 1 Start H1 H2 H3 H4 Nov-20-03 SMD157, Usability Inspection Methods 29 L Editing Italic Font Displays settings for the selected configuration Originals: 1-sidedÍ detectí Copies: 1-sidedÍ sortedí staple upper leftí plain, A4Í exact sizeí Account researchí Lightness Contrast Restore Save E Test An E appears. H1 H2 H3 H4 Nov-20-03 SMD157, Usability Inspection Methods 30 L 10
Questions? Nov-20-03 SMD157, Usability Inspection Methods 31 L 11