3 Prototyping and Iterative Evaluations

Transcription

1 3 Prototyping and Iterative Evaluations Viktoria Pammer-Schindler March 15, 2016 Prototyping and Iterative Evaluations 1

2 Days and Topics March 1 March 8 March 15 April 12 April 19/21 April 26 (10-13) April 28 (9-14) May 3 May 10 Administrative Stuff, Intro to Designing Interactive Systems Understanding Context of Use Assignment 1 Handed Out Prototyping and Iterative Evaluations Assignment 1 Deadline (before/in lecture) Assignment 2 Handed Out Participatory Design (Theory and Workshop) Assignment 2 Interviews with Tutor (throughout the day) (maybe guest lecture on April 21, lecture on April 19 will be cancelled) Evaluations Workshop (Cognitive Walkthrough, Observation and post-hoc discussion of prototypes Assignment 3) Assignment 4 Handed Out Android Sensing / Context-Aware Interactive Systems Tutorial Day (different room) Ubiquitous Computing, particularly Ubiquitous User Interfaces Questions on Programming to Tutor May 24 (10-13) Presentations 1 (Assignment 4) May 31 (10-13) Presentations 2 (Assignment 4) Prototyping and Iterative Evaluations 2

3 Recap 1. What do you need to plan for in an observation / semi-structured interview? a. Objective and entities of interest b. Cooperation with participants c. Data collection d. Interview guideline with key questions (for the interview) Prototyping and Iterative Evaluations 3

4 Recap 2. What are the five perspectives of an activity that contextual design is interested in for work modelling? a. Sequence and intents (sequence model) b. Interaction and coordination (flow model) c. Rules and policies governing behaviour (cultural model) d. Physical space of activity (physical model) e. Artefacts of relevancde (artefact model) Prototyping and Iterative Evaluations 4

5 Forward So, you observed and interviewed key users and have now some solution ideas which are so cool that you immediately want to implement them and try them out. If you do this, you might spend months fine-tuning your functionality then trying to sell this to your target users only to find out that for a long list of reasons users don t like your solution. Prototyping and Iterative Evaluations 5

6 Today Prototyping Key Concepts and a few tricks Iterative Evaluations we focus on early, informal evaluations that help you mature your solution ideas Prototyping and Iterative Evaluations 6

7 Learning Goals After today s lecture you should 1. Know about different kinds of prototypes 2. Know about different kinds of evaluations that are useful in UCD 3. Know and understand in principle the methods cognitive walkthrough and heuristic evaluation (will be practiced in Assignment 3 after Easter) After today s assignment you should 1. Be able to represent the key problem (pain point) that your system will solve in a storyboard 2. Be able to represent the measure of success for your system in a storyboard 3. Be able to develop prototypes Prototyping and Iterative Evaluations 7

8 Analyse Evaluate Design/ Implement DESIGN Prototyping and Iterative Evaluations 8

9 Prototyping Easier (for users + designers) to critique existing designs than to immediately think up a good one. Early prototypes - low fidelity Many! Different! Throw away! Evolutionary prototypes - from low to high fidelity Slowly develop into a final product Don t start too early Prototyping and Iterative Evaluations 9

10 Prototyping Horizontal prototypes: Capture all of system/product, but not in depth Vertical prototypes: Go in depth w.r.t. a specific functionality / aspect, but don t cover the whole system/product Prototyping and Iterative Evaluations 10

11 Focus Low-Fidelity Prototypes Sequence of actions (both of user and system) Content Insert into earlier, visionary storyboards Implementation: Paper: Hand-drawn Interactivity: Create multiple layers of content with sticky notes, plastic overlays Wireframes more detailed issues like layout come into focus Prototyping and Iterative Evaluations 11

12 Prototyping and Iterative Evaluations 12

13 Prototyping and Iterative Evaluations 13

14 Prototyping and Iterative Evaluations 14

15 Tutorials / Manuals Write a tutorial before implementing the system Explain and illustrate key steps Can also walkthrough a tutorial with users to check understanding / questions they would have before buying target system Prototyping and Iterative Evaluations 15

16 Prototyping with a Computer Simulate or animate parts of the system Depending on used technology may be the start of evolutionary prototyping Take longer to implement than no-coding low-fidelity prototypes Stakeholders may already think this is for real Fundamental flaws may be hard to detect BUT useful for testing more complex functionality (e.g., Wizard of Oz evaluations) Prototyping and Iterative Evaluations 16

17 High-fidelity prototypes For stakeholders, high-fidelity prototypes may be undistinguishable from the real thing Prototyping and Iterative Evaluations 17

18 So, what are high-fidelity prototypes missing? Full integration with other information systems Convenience functionalities (e.g., import from different filetypes ) Working on multiple platforms Assuming for all these that they are not key functionalities integral to user oriented evaluations! Prototyping and Iterative Evaluations 18

19 Create prototypes Evaluate prototypes How to use Prototypes a lot with users in the beginning of the process, to make sure that you got the problem, the task, the language etc. right (informal, interactive mixed design/evaluate processes) A lot with experts as your design matures, to capture usability problems use experts to save users time, use experts to systematically capture design problems based on experts experience A lot with users again throughout the rest of the process to make sure that your development efforts have the right prioritisation Prototyping and Iterative Evaluations 19

20 Analyse Evaluate Design/ Implement EVALUATE Prototyping and Iterative Evaluations 20

21 Evaluation Goals Does system match user needs? Which needs to what extent (different kinds of requirement on functionality or qualities of system) Which users/stakeholders Which effects does system have on stakeholders? On user s cognitive or work processes? On organisational processes? On key peformance indicators Prototyping and Iterative Evaluations 21

22 Evaluation Goals Formative evaluations Identify directions for ongoing/future development Summative evaluation Make a stmt about extent of match Prototyping and Iterative Evaluations 22

23 Types of Evaluations Over time, evaluations get more formal and extensive. Expert Evaluations (no users) Save on evaluations by reviewing literature! Cognitive Walkthrough Heuristic Evaluations Model-based Evaluations Evaluating with Users Observations (Thinking aloud, protocol analysis, post-task walkthrough) Experiments (most often: control group vs. intervention group) Query-based (Interview, questionnaire) This can be very powerful. Just grab a study colleague that does not go to this course, show him/her your prototype, give him/her a task, and observe/lislten to your friend trying to figure out what to do! (Read up in Nielsen s Thinking Aloud Link on Last Slide) Prototyping and Iterative Evaluations 23

24 Cognitive Walkthrough Goal: Figure out learnability what prior knowledge do users need to have, over what can be explored within a system You need A prototype system (whatever the quality/maturity) A task that the user shall perform E.g.: Specify that you need to see your boss asap. A detailed list of actions the user needs to carry out to achieve the task with the prototype system An indication of what prior, relevant, experience the user has Prototyping and Iterative Evaluations 24

25 Cognitive Walkthrough At each action, ask 1. Does the effect of the user s interaction with the system match the goal of the action? 2. Does the user see that an action is available? (Recognition, Match betw. System and real world, consistency) 3. Can the user recognize that the action is the right one? (Recognition) 4. After action, can the user understand the system s feedback? (Visibility, Help in recovering from errors) Prototyping and Iterative Evaluations 25

26 Example Cognitive Walkthrough Prototyping and Iterative Evaluations 26

27 Heuristic Evaluation Goal: Systematically figure out usability problems based on rules-of-thumb You need A prototype system (whatever the quality/maturity) Option: A task that the user shall perform E.g.: Specify that you need to see your boss asap. List of heuristics Prototyping and Iterative Evaluations 27

28 1. Visibility of system status Heuristics 2. Match between system and real world 3. User control and freedom 4. Consistency and standards 5. Error prevention 6. Recognition rather than recall 7. Flexibility and efficiency of use 8. Aesthetic and minimalistic design 9. Help users recognize, diagnose and recover from errors 10. Help and documentation Prototyping and Iterative Evaluations 28

29 Example Heuristic Evaluation Prototyping and Iterative Evaluations 29

30 Expert-based Evaluations Use multiple experts Once you understand the methods Vary the questions in cognitive walkthrough For instance, simplify to: Will users see what they need to do? If the user does the right thing, will he know it? Try to insert user actions into the walkthrough that correspond to common errors, like pressing a button multiple times, the network connection failing, etc. Prototyping and Iterative Evaluations 30

31 Readings Recommended Reading Nielsen 1995, Ten Usability Heuristics for Design: Further Readings Nielsen 2012, Thinking Aloud: usability-tool/ Rohrer 2014, When to Use Which User Evaluation: Prototyping and Iterative Evaluations 31