Usability Testing. November 9, 2016

Transcription

1 Usability Testing November 9, 2016

2 Announcements Milestone III Average: 77/90 (86%) Most marks lost for not following instructions (e.g. missing sections) 2

3 Questions? 3

4 Today More analytical evaluation Usability testing What is it Considerations Case study Data collection and analysis (maybe) 4

5 Clarification Fitts s Law? Hick-Hyman Law Search item 5

6 GOMS Goals, Operators, Methods, Selection Rules (Card et al. 1983) Higher-level than KLM Input: detailed description of UI and task(s) Output: various qualitative and quantitative measures 6

7 GOMS Hierarchy Goal(s) Select a Menu Item Methods Find the Desired Menu Open the Menu Select the Desired Item Primitive Operators Point Click Type Visual Search Double Click 7

8 Example Goal: delete text (n chars long) Select: method 1 if n > 10 method 2 if n < 10 Method 1: Goal: highlight text & delete Goal: highlight text Point Click Point Shift Click Verify Click Method 2: Goal: delete n chars 8 Adapted from Rob Miller

9 Performance Modeling: Advantages Can evaluate components of interface prior to building it Good for comparing different interface possibilities Evaluations with human subjects are expensive and experts can be hard to find Can get the kinks out of interface prior to full user testing/experimentation 9

10 Performance Modelling: Problems Difficult to model complex tasks E.g., consider designing a KLM for your complete project Most models consider only expert behaviour For really accurate predictions, coefficients need to be determined empirically 10

11 Summary: Analytical Evaluations Analytical evaluations are a set of structured techniques for evaluating interfaces without (necessarily) involving end users Often used to address major usability issues and inefficiencies prior to involving end users Should always complement rather than replace evaluations with end users 11

12 Usability test A usability test is a formal method for evaluating whether a design is learnable, efficient, memorable, can reduce errors, meets users expectations, etc. users are not being evaluated the design is being evaluated 12

13 Usability test Rough Outline Bring in real users Have them complete tasks with your design, while you watch (ideally with your entire team) Measure and record things task completion, task time, error rates satisfaction, problem points, etc. use a think-aloud protocol, so you can hear what they are thinking 13

14 Usability test Rough outline Use the data to identify problems (major ones minor ones) provide design suggestions to design/engineering team iterate on the design, repeat 14

15 Important Considerations Usually takes place in a usability lab or other controlled space Major emphasis is on selecting representative users developing representative tasks 5-10 users typically selected Tasks usually last no more than 30 minutes The test conditions should be the same for every participant Informed consent form explains ethical issues 15

16 Testing Environments 16

17 Testing Environments 17

18 Testing Environments Best environment depends on pragmatic considerations, as well as what you re looking for Do you want your whole team to be able to view? Do you want to be able to review a test? How important are interruptions? What are you resources? 18

19 Pilot studies Practice runs Especially important for usability testing Make sure your plan is viable All the corners are checked (your script, questionnaires, tasks, etc., all work) It is worth doing several to iron out problems before doing the main study Ask colleagues if you can t spare real users Fall 2016 COMP

20 Case Study: Testing Goals and questions Identify range of usability problems Particularly testing whether topic categorization on homepage worked Also testing whether navigation within site was effective MEDLINEplus 20

21 Case Study: Testing MEDLINEplus Selection of participants 9 participants from health care practices in DC area Asked to fill out factual questionnaire Screened based on frequency of web usage (> two times per month) 7 Female, 2 Male 21

22 Case Study: Testing MEDLINEplus Five tasks were developed Wanted to check categorization and navigation support Task 1: Find information about whether a dark bump on your shoulder might be skin cancer Task 2: Find information about whether its safe to use Prozac during pregnancy Task 3: Find information about whether there is a vaccine for hepatitis C Task 4: Find recommendations about the treatment of breast cancer Task 5: Find information about the dangers associated with drinking alcohol during pregnancy 22

23 Case Study: Testing Test procedure MEDLINEplus 5 scripts prepared in advance Done in lab setting Goals of the study and test procedure were explained 23

24 Case Study: Testing MEDLINEplus 24

25 Case Study: Testing MEDLINEplus 25

26 Case Study: Testing MEDLINEplus 26

27 Case Study: Testing MEDLINEplus 27

28 Case Study: Testing MEDLINEplus 28

29 Data collection Case Study: Testing MEDLINEplus Completion times automatically recorded and calculated from the video and log data Search moves of participants were collected Data collection contained: Start and completion times of tasks Page count (i.e. pages accessed during the search task) Medical publications accessed during the search task Users search path Negative comments, mannerisms User satisfaction questionnaire data 29

30 Case Study: Testing MEDLINEplus Data analysis categorized according to: Website organization such as arrangement of topics, menu depth, organization of links, etc. Browsing efficiency such as navigation menu location, text density Search features such as search interface consistency, feedback, etc. Conclusions and reporting the findings Reaching external sites was often difficult Analysis of search moves revealed that participants experienced difficulty finding health topics devoted to different types of cancer Post-test questionnaire showed opinion was fairly neutral Rated well for ease of learning but poorly for ease of use 30

31 Case Study: Testing MEDLINEplus 31