Cognitive Walkthrough. Design and User Evaluation of Augmented-Reality Interfaces Jorma Nieminen

Transcription

1 Cognitive Walkthrough Design and User Evaluation of Augmented-Reality Interfaces Jorma Nieminen

2 Cognitive Walkthrough Outline Modified from the Handbook of Human-Computer Interactions [1] : Define asumed users background (knowledge, skills etc.) Choose a sample task for the test Specify correct action sequence for the task (e.g. print screens) Determine Interface states along the sequence (e.g. feedback messages) Analysis For each correct action: a) Construct a success story (why a user would choose that action)...or b) Construct a failure story (why a user would not choose that action) Record problems, reasons and assumptions Consdider and record design alternatives Modify the interface design to remove the problems

3 Define user an choose a sample task Heikki is a regular third year computer science student from Helsinki. He has just had a lunch at the Unicafe in Chemicum building in Kumpula and now wants to go to the central library of the university to study for tomorrow s classes. Heikki knows that library is near the railway station.

4 Specify correct action sequence

5 Specify correct action sequence

6 Specify correct action sequence

7 Specify correct action sequence

8 Analysis: Step 1 Information related to the analysis questions are from the book Usability Inspection Methods [2]. Will the user associate the correct action with the desired effect?

9 Analysis: Step 1 Success Story 1: Defining the users current location is part of the original task. Will the user associate the correct action with the desired effect?

10 Analysis: Step 1 Failure Story 1: User s goal is to get to the library. He may assume that the system knows he s location automatically (likelihood 0,5%). Will the user associate the correct action with the desired effect?

11 Analysis: Step 1 Success Story 1: Defining the users current location is part of the original task. Success Story 1: The input field is clearly visible (e.g. Not behind some menu). Will the user associate the correct action with the desired effect?

12 Analysis: Step 1 Success Story 1: Defining the users current location is part of the original task. Success Story 1: The input field is clearly visible (e.g. Not behind some menu). Will the user associate the correct action with the desired effect? Success Story 1: Input field is near the label From (You a can also base the story on e.g. user s past experience or that there are no competing actions).

13 Analysis: Step 1 Success Story 1: Defining the users current location is part of the original task. Success Story 1: The input field is clearly visible (e.g. Not behind some menu). Will the user associate the correct action with the desired effect? Failure Story 1: User may try the timetable search on the right column (likelihood only 10% because the reading direction is from left to right and top to bottom).

14 Analysis: Step 1 Success Story 1: Defining the users current location is part of the original task. Success Story 1: The input field is clearly visible (e.g. Not behind some menu). Will the user associate the correct action with the desired effect? Failure Story 1: User may try the timetable search on the right column (likelihood only 10% because the reading direction is from left to right and top to bottom) Success Story 1: User sees the written text appear in the input field.

15 Analysis: Step 2 Success Story 1: Entering the destionation is part of the user s original goal to get to the library Success Story 1: The input field is clearly visible. Will the user associate the correct action with the desired effect? Success Story 1: Input field is near the label To and there are no competing alternatives visible. Success Story 1: User sees the written text appear in the input field.

16 Analysis: Step 3 Success Story 1: User wants to see the alternative journey options because this is a part of his original goal. Success Story 1: Search button is clearly visible on the screen. Will the user associate the correct action with the desired effect? Success Story 1: Button is labeled Search and based on the users past knowledge on websites he knows that he can click the button.

17 Analysis: Step 3 Success Story 1: User wants to see the alternative journey options because this is a part of his original goal. Success Story 1: Search button is clearly visible on the screen. Will the user associate the correct action with the desired effect? Failure Story 1: User clicks the bicycle button because it s on different colour (likelihood 2%). Failure Story 2: User clicks the search button related to timetable search on the right column (likelihood 0,0001%).

18 Analysis: Step 3 Success Story 1: User wants to see the alternative journey options because this is a part of his original goal. Success Story 1: Search button is clearly visible on the screen. Will the user associate the correct action with the desired effect? Failure Story 1: User clicks the bicycle button because it s on different colour (likelihood 2%). Failure Story 2: User clicks the search button related to timetable search on the right column (likelihood 0,0001%). Success Story 1: The available routes appear on the screen after clicking the button.

19 Analysis: Step 4 Will the user associate the correct action with the desired effect?

20 Record problems, reasons and assumptions Failure Story 1: User s goal is to get to the library. He may assume that the system know he s location automatically (likelihood 0,5%). Will the user associate the correct action with the desired effect? Failure Story 1: User may try the timetable search on the right column (likelihood only 10% because the reading direction is from left to right and top to bottom). Will the user associate the correct action with the desired effect? Failure Story 1: User clicks the bicycle button because it s on different colour (likelihood 2%). Failure Story 2: User clicks the search button related to timetable search on the right column (likelihood 0,0001%).

21 Cognitive Walkthrough Limitations Depends heavily on the skills of the evaluator Found problems depend on the chosen sequence Not possible to diverge from the chosen path Problems are mainly related to learnability When evaluation was done on patient monitoring systems used in hospitals, CW did not find all critical problems that were found with usability tests [Liljegren 2004].

22 Cognitive Walkthrough References [1] Lewis, C. and Wharton, C. Cognitive Walkthrougs. In Helander, M., Landauer, T. K. and Prabhu, P. Handbook of Human-Computer Interactions, Elsevier, 1997, [2] Wharton, C., Rieman, J., Lewis, C., Polson, P., The Cognitive Walkthrough Method: A Practitioner s Guide. In Nielsen, J. Usability Inspection Methods, Wiley & Sons, 1994, [3] E. Liljegren and A.-L. Osvalder, "Cognitive engineering methods as usability evaluation tools for medical equipment,"international Journal of Industrial Ergonomics, no. 34, pp , 2004.