UX Research. test scenarios. UX Research test scenarios. KP Ludwig John

Transcription

1 UX Research

2 Usability Test Preconditions Already existing Personas Definition of target groups Context of use where, when, how long, equipment! Test goals What do we want to learn?

3 Usability-Test Predefinitions Test goal Example (IA6_2013; Handy-Game Obacht: Konkret genug nur die letzten 3 Punkte, weil einzelne Funktionalitäten im Fokus

4 Usability-Test Vorab definieren: 1. Testziel Beispiel (IA6_2013; Lumenaer LED-Installation

5 Usability-Test 1. Testziel Beispiel (IA6_2013; Lumenaer LED-Installation Klare Testziele

6 Usability-Test Comparison of 3 print Layouts IMS.UX Test goal Which Layout is best? Source: Team UXperts, IMS.UX 2017

7 Usability-Test Aspects to consider Comparison of 3 print Layouts IMS.UX Extracting Information How easy is it to get the information? What information can you get? Memorizing Information Can you memorize the information after reading? Is it easy to remember Clarity Is the structure obvious to the user? (Naming, Positioning, Design of elements) Comprehensibility Does the user understand the content? Readability - Is the text easy to follow? Look & Feel - How is the first impression? Do you have a good feeling reading the paper? Source: Team UXperts, IMS.UX 2017

8 Usability-Test Comparison of 3 print Layouts IMS.UX Aspects to consider Source: Team Group Title, IMS.UX 2017 How fast can information be perceived and processed? Measuring unit: Time to complete tasks. How much attention does relevant information get? Measuring unit: Time a test subject s focus rests on relevant information. How well is information communicated? Measuring unit: Amount of misunderstandings and missed information

9 Usability-Test Predefinitions Test goal What do you want to learn by the test? Which elements of the IA design have been discussed controversial? Which elements seam to be critical for the general success of the application? Which elements of your application have to be covered Main functions / use cases Crucial functions of the application Use Cases with many Alternate Flows

10 Usability-Test Predefinitions Test object What kind of Prototype is needed in order serve your test goals Which functionality has your prototype to provide in order serve your test goals? paper based software based

11 Usability-Test Predefinitions Test method Which test method do you want to apply? thinking aloud Video observation Eyetracking Screen recording Interview diary study (etc.)

12 Usability-Test Predefinitions Test equipment What kind of equipment serves best the test goals? Consider testing under of real context of use conditions (Screen size, Space, time, mobile, distraction, sound)

13 Usability-Test Predefinitions Test scenario Create a test scenario which defines each single step of the test in detail Including Introduction, Tasks and Expression of Thanks

14 Usability-Test Predefinitions Test space Test location Where do you want to test? Availability, Influence of Weather or Light? Test equipment Where do you get it? How to handle it?

15 Usability-Test Preparation Test team Testleader Technical manager Observer Analyst

16 Usability-Test Preparation Test persons Find test persons make appointments and a test schedule How many Test persons are needed? (about 7-8 per Test version (!) ) Where can the test persons be recruited from? (do they represent the target group, are they available (time)?) Do they get some expense allowance (Aufwandsentschädigung)? (which?) recruit test persons

17 Usability Test persons UX Research research methods overview

18 Usability Test Test persons Who do the test with? It s best to test with representatives of your main target group. But don t be obsessed about it! If its difficult to recruit them, test with people you can get hold of. Take it in account when evaluating the test results: Would our users have that problem? Or was it only a problem because the participant wasn t familiar with the field? How many test persons? Rather few but frequently start early in the development process and repeat and repeat and repeat UX Research research methods overview

19 Usability Test How many test persons? Example: Test persons Test with 8 users found problems: 5 major ones Quelle: Steve Krug, Don t make me think! Redline Heidelberg, 2006 UX Research research methods overview

20 Usability Test Test persons How many test persons? Example: 2 Tests with 3 users each = 6 Test persons found problems: 5 major + 4 minor = 9 problems Quelle: Steve Krug, Don t make me think! Redline Heidelberg, 2006 UX Research research methods overview

21 Usability Test Test persons Wen testen? Datenquelle: Faulkner (2003): Beyond the five-user assumption: Benefits of increased sample sizes in usability testing; in Sarodnik/Braun: Methoden der Usability Evaluation (Huber Verlag 2011) UX Research research methods overview

22 Usability Test Test persons source: PDF, page 19 UX Research research methods overview

23 Usability Test How many test persons? Test persons Most significant problems will be discovered by small number of test users already. Finding three participants is easy to handle: test and debrief (Nachbesprechen) the same day! Design small tests with focussed tasks. Learn, move on and test again. Can you use the same participants again? Not for the same task or application. They know to much already. Reference: Steve Krug: Rocket Surgery Made Easy, New Riders 2010, chapter 5 UX Research research methods overview

24 Test scenarios in detail

25 Usability-Test Two Steps 1. Test person works on given tasks often under direct observation (screen recording software, video, human observer) 2. Interview with the user to better understand critical situations during the test. Possibility to discuss improvements together

26 Test scenario Content and style Identify at least 3 essential activities to be covered by the test Create 1 or more tasks per activity to be followed by the test person Example: Become a new member of this online community and create your individual account! Which of the dishes offered at Mensa next Monday you prefer? Note it on paper.

27 Test scenario Content and style Tasks have to be - complex enough (not to simple) - not too complex either (shouldn t take too long to work on each single task) - cover all important issues of your test

28 Test scenario Content and style Important: - create realistic tasks Users should try to do the same things in Lab as they would do at home or at work Tasks determine looks! Nielsen/Pernice: Eyetracking Web Usability, New Riders, Nielsen Norman Group 2010

29 Test scenario Same content Tasks determine looks! Website of Usability Week Conference Task 1: Decide which of two conferences rather to attend Task 2: Decide which seminar to choose from that conference Nielsen/Pernice: Eyetracking Web Usability, New Riders, Nielsen Norman Group 2010

30 Test scenario Tasks determine looks! Same content - Different Tasks People look differently at the same information, depending on what they are trying to do. In both cases: Heatmap visualisation of Eyetracking Recordings (objective measurement) identical content But: Different users interest (determined by the task) Nielsen/Pernice: Eyetracking Web Usability, New Riders, Nielsen Norman Group 2010

31 Test scenario Don t ask - Observe! Tasks determine looks! Screenshots from us census bureau (statistics) Task: Find the US population and that of the state Texas Looks: Menus, content headings, population clock at the right source: PDF page 145

32 Test scenario Don t ask - Observe! Tasks determine looks! Screenshots from us census bureau (statistics) Task: What do you think of this page? Comment on it! Looks: All over This will not happen in real user situations! source: PDF page 146

33 Test scenario Tasks determine looks! If testing just one page Don t open the page and ask the user to Please look at the page and tell us what you think. This encourages the user to explore a page in a way they probably never would. Don t open the page, let the user examine it, then give him a task. Doing these steps in this order would encourage a user to examine the page when he might have never done this on his own.! Do ask the user to do a task, then launch the page source: PDF page 147

34 Test scenario Content and style First steps Determine: What do you want to find out? Which elements of the application are crucial? Formulate the tasks straight forward, short and precise! Each task should have a defined goal (that defines, when the task is done) Start with simple, easy to handle tasks! More difficult (or more complex) tasks are possible, but not good at the very beginning of the test)

35 Test scenario Content and style Let the test person act rather independently and on its own Test leader although positioned next to the proband, remains silent / interferes only in critical situations. Don t get into conversations with the test person during the test itself. That can be done after the test. Set up test conditions close to the real context of use Try out the tasks with team members or random users before starting the official test

36 Test scenario Structure 1. Welcoming and short introduction by test leader My name is Martin and I am guiding you through this test First, we would like to thank you for participating in our test, we really appreciate your help and the time you took for this. Very important before we get started: We are not going to test you in person, we try to test different layouts for a magazine, so don t worry, you can t do or say anything wrong. Source: Team Group Title, IMS.UX 2017

37 Test scenario Structure 2. Introduction of the team members and their functions If there are any questions or misunderstandings, just ask. Vivi, Kate, Daniel and Chris are here to observe the whole testing sequence. Ruben is our technical manager. You will have to do some tasks here in the UX-Lab, which I am going to explain later in detail. Bastian is a member of Academica. He will guide you to another room where you are going to get your last task, which is answering some questions in a questionnaire. Source: Team Group Title, IMS.UX 2017

38 Test scenario Structure 3. Declaration of consent For the following test here in the UX-Lab, my team members will be observing the test. We would like to do audio recording during the test. We are not going to provide this data to 3rd parties, but if you don t want your voice being recorded, feel free to tell us. Source: Team Group Title, IMS.UX 2017

39 Test scenario Structure 4. Definition of tasks There will be 3 different PDFs which contain several information. You have to do some tasks, they will be displayed on the screen. Pressing space on the keyboard will start the first question, We would like to encourage you to tell us what you are thinking while doing the tasks. It helps us analyzing your final answers, but don t feel forced to comment if you don t feel like it. If there are no more questions from your side about the test procedure itself, you can start by pressing the SPACE key After a short calibration of the eye-tracking software the actual test is going to start immediately. Source: Team Group Title, IMS.UX 2017

40 Testszenarien entwickeln Aufbau Begrüßungstext oder: Beispiel: IA A Serious Game (Kinect)

41 Test scenario Content and style Tasks Beispiel: FS 2007, Test Hochschulseiten

42 Test scenario Content and style Beispiel: IA6 2013; Handy-Game Obacht? Konkrete Handlungsaufforderungen mit erkennbarem Abschluss ggf. Fertig. sagen lassen >> 1,2,5: gut Anweisungen 3 und 4 ohne erkennbares Ergebnis

43 Test scenario Content and style Beispiel: IA6 2013; Wetterstation Konkrete Handlungsaufforderungen mit erkennbarem Abschluss ggf. Fertig. sagen lassen >> alle gut bis auf (1)

44 Test scenario Content and style Expression of Thanks oder: or probably! post test Interview to collect impressions and to discuss critical situation observed during the test

45 Test scenario Duration depends on test goal and circumstances Rule of Thumb: Between minutes per person is OK less than 10 minutes seems rather short as well to the test person as for the team longer than 30 minutes requires a lot of effort from both sides

46 Practice Prepare your test Formulate your test scenario in detail! Plan and organize your test: Appoint Functions and responsibilities in the test team Find and appoint test persons Schedule the test (incl. equipment and location) UX Research research methods overview

47 Practice Prepare your test Summarize your test design into one document, including Test goals Criteria of Success and Measurements for that criteria Target group Prototype technical equipment scenario post test interview (if applicable) Be prepared to discuss it in detail next week hs-augsburg.de/homes/john UX Research research methods overview