Methods: Deciding What To Design

Transcription

1 Methods: Deciding What To Design Usability 3 Evaluation Licínio Roque lir@dei.uc.pt

2 Founded 1297 AC Licínio Roque PhD in Information Systems Development Methodology Research in Interaction Design, Digital Games, Learning Games, Participatory Media, Context Engineering Engineer, Professor, Researcher, Entrepreneur Licínio Roque, 2

3 Usablity 3 Contents Usability - why? Usability Factors Usability & Usability Evaluation Quick&Dirty, s,, Field Evaluations, Tests Gathering Data Results Licínio Roque, lir@dei.uc.pt 3

4 Usability Why evaluate artifacts? Usability evaluations main goals: To best understand the user goals and context To guide the design towards usage goals To promote convergence of the design process To compare/benchmark designs or solutions We also evaluate usability to reduce risks: Avoid shipping unwanted products Correct disfunctional products Resolve usage problems as early as possible Licínio Roque, 4

5 Usability Adoption is influenced by social and pratical factors Social factors work though the relationships software systems promote or inhibit Practical concerns include: fittness, cost, compatibility, trustworthiness, etc... Fittness has two faces: Effectiveness utility or fittness for purpose Efficiency user performance with the artifact, how well, how fast... In Usability we assume utility or effectiveness, and work on quality attributes that promote efficiency Licínio Roque, 5

6 Usability Factors Desirable interaction qualities that contribute to Usability: Interaction efficiency - task speed Error Rates volume, frequency, importance Learnability - learning speed and length Memorability - retention between uses Subjective Satisfaction Licínio Roque, lir@dei.uc.pt 6

7 Usability & Factors User segmentation impacts results: with computation in the application domain with this or similar applications User segmentation may also reveal that not all tasks have the same frequency and importance for each user profile Licínio Roque, 7

8 Usability Evaluation Evaluation Timing Formative: during development in time to make a difference Summative: post-mortem serves the purpose of refection on practice Licínio Roque, lir@dei.uc.pt 8

9 Usability Some Evaluation Quick & Dirty Evaluation Design (or Pluralistic) Test Field Evaluations Licínio Roque, 9

10 Usability Quick and Dirty Ad-Hoc practice, the idea is to take any chance you can get to run your design/prototype though available users or experts Minimal intervention by management or designers Let the user explore Observe and document behaviour and comments Remember to characterize user/evaluator Results may vary Danger to make generalizations based on this data Quick and timely oportunistic Licínio Roque, 10

11 Usability Evaluation Define set of heuristics/criteria the system must pass s can be made to reflect valued factors in specific contexts and for user profiles Prepare a post-experimentation questionaire Questionaire should enable the evaluation and computation of heuristic indexes Conduct experimentation sessions in a guided manner to make certain that the user gets exposed to relevant functionality or performs a relevant task set User-Evaluator/User-Designer answers questionaire Compute heuristic indexes based on data gathered Relatively quick and easy evaluation Licínio Roque, lir@dei.uc.pt 11

12 Usability s: prêt-a-porter Simple and natural dialog Speak the users language Minimize the user s memory load Consistency Feedback Clearly marked exits Shortcuts Prevent errors Help and documentation Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 12

13 Usability Design Simulate the user problem resolution process Needs a prototype (paper, maquette or functional) Follow specified action sequences for each task goal Select a relevant task set for validation (critical, freq?) Go though each task, step-by-step, with designers, experts or target users, discussing and validating possible task execution and interpretation Checks the design proposal within the context of a use scenario May use think-aloud protocol : asking the user to say what is thinking (a trade-of, influences behavior) Easy, convenient, cheap, widely used Licínio Roque, lir@dei.uc.pt 13

14 Usability Design Inputs Description of users Sequences Correct action sequence for each sequence User interface Analyzing involves simulating four simple steps of an underlying model of exploratory learning, at each stage of interaction, by asking the questions: Will the users try to achieve the right effect? Will the users notice that the correct action is available? Will the user associate the correct action with the effect the user wants to achieve? Will the user interpret the system s response to the chosen action correctly? Success story or failure story Recommendations -- problems and possible fixes Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 14

15 Usability Design Watch out for: In open-ended discussions designers become defensive... Experts can go into elaborate explanations of user behavior You can gain or loose a lot of time! Pluralistic Cognitive Design Walkthrough variant For each scenario, screen usage sequences are run before a panel of evaluators Panel members document actions they would take, individualy The panel later discusses issues and sugestions for each scenario Representative users speak first so as not to be influenced, then usability experts and last, the designers explain their options Licínio Roque, lir@dei.uc.pt 15

16 Usability Field Evaluations Field Evaluation propose the experimentation of artifacts by final users AND in their use environment Can be used at the beginning of development process, with existing artifacts, as a contribution for requirements engineering (as in Contextual Inquiry) Can be used at the end, when usable prototypes are available to validate lab test results, in real context Performance involves observation of users while they go about their daily tasks with the artifacts, interviewing and user satisfaction questionaires Can be very time consuming Licínio Roque, lir@dei.uc.pt 16

17 Usability Lab Test For formal lab tests you will need to develop a test script and an experimentation protocol Preparing the test includes design the experience (what and how) select participants based on user profile develop or prepare tasks to be performed prepare materials (prototypes, forms ) conduct the experiment collect data (record video is possible) analyze and report results Licínio Roque, lir@dei.uc.pt 17

18 Usability Data Gathering Data gathering is a difficult task. Prepare. Separate roles: conductor, scribe, user Record time and occurence of events Prepare forms for registering relevant info: Example data gathering table: Event desc User, task importance frequence Proposed corr Process results statisticaly, if significant Don t mix tests from diff. user profiles Licínio Roque, lir@dei.uc.pt 18

19 Usability Results Task duration Task A Task B Task C Task D Task E Licínio Roque, lir@dei.uc.pt 19

20 Usability Results Learning time Task A Task B Task C Task D Task E Licínio Roque, lir@dei.uc.pt 20

21 Usability interpreting results Error Rates Question Frequency & Criticality Learnability When is it especially relevant? With whom and when can you measure learning speed? Retention When is it especially relevant? Time frame: Day? Week? Month? Year? Licínio Roque, 21

22 Usability how many tests? How many tests do you need? Pareto, again... 6 tests ~> 80% issues N(1 (1-(1 (1-L) n ) N: total number of usability problems L: proportion found testing 1 user n: number of users tested From Jakob Nielsen's Alertbox, March 19, 2000, Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 22

23 Subjective Satisfaction Focus on qualities that the users have trouble voicing Use special purpose questionaires to enable users to express sensations they find dificult to rationalize User-artifact relationships like: confort, elegance, helpfulness, trustworthyness, aesthetics, beauty, fun, creativity... Licínio Roque, 23

24 Usability Likert Scales Agree Disagree Statement A Can be very useful for collecting data for calculating Evaluation indexes Option: Use cross-checking questions Lookout for lenght (time) indiference effect Avoid complex or multiple question formulations Simpler questions make faster questionaires Licínio Roque, lir@dei.uc.pt 24

25 Usability Semantic Differential Scales e.g. Confortable Unconfortable Best option for subjective satisfaction index Scales may be tunned in precision, usualy at the cost of user taking more time to answer Even vs Odd scales: odd scales force decision Licínio Roque, 25

26 From Usability to A glimpse: think about how you would evaluate a videogame What qualities would you be looking for? How would you test/measure them? Now think about evaluating a multiplayer online game... Licínio Roque, lir@dei.uc.pt 26

27 Georgia Tech Lab Setup Adapted from Usability 3, by Jim Herbsleb Licínio Roque, 27

28 Usability Lab at Sun Adapted from Usability 3, by Jim Herbsleb Licínio Roque, 28

29 Lab Setup at the UEgroup Adapted from Usability 3, by Jim Herbsleb Licínio Roque, 29

30 Microsoft Lab Layout Adapted from Usability 3, by Jim Herbsleb Licínio Roque, 30

31 Catching Problems Early Evaluations (E) evaluation Cognitive walkthrough Prototyping (P) User testing (UT) P E UT Earlier Later Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 31

32 Formative versus Summative Evaluations (E) evaluation Cognitive walkthrough Prototyping (P) User testing (UT) P E UT Formative Summative Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 32

33 Cost Evaluations (E) evaluation Cognitive walkthrough Prototyping (P) User testing (UT) E P UT Low Cost High Cost Adapted from Usability 3, by Jim Herbsleb Licínio Roque, lir@dei.uc.pt 33

34 Usability 3 Testing? Licínio Roque, With slides adapted from Jim Herbsleb Licínio Roque, lir@dei.uc.pt 34