Software Design User testing exercise

Transcription

1 Software Design User testing exercise Joan Serrat December 12, Introduction Goal The purpose of this exercise is to learn what is and how to do a user or formative testing session by doing it. During an user testing session an user tests a prototype of our application with the goal of finding usability problems to be fixed on the next iteration. Only users can do this because we are not the users, we must not design for ourselves but for them. Hence, only from real users can we get the feedback to improve the interface design. What s a prototype? Anything fast to create and on which representative users can perform realistic application tasks by interacting with it. If it s a paper prototype it means it s the interface we have designed drawn on papers, containing realistic data, that is manipulated by a person playing computer who doesn t explain how the interface works, because a computer has no intelligence and does not talk. Why a prototype? Simply because the sooner we test, the less effort in correcting design/usability errors. To start testing soon we avoid programming, which is a long task and takes a lot of effort. A prototype can be anything from low fidelity like a paper prototype to higher prototypes created by tools like Just in Mind. Which kind of prototype? In our case we recommend to test a paper prototype. Those generated by PopApp, Just in Mind or Wireframe Sketcher have the advantage of being runnable programs (no need of a computer person) and also can be shared and tested at a distance. But take much longer to create (though much, much less than programming the interface). In addition, on a paper prototype executed by a computer person, the user can do whatever kind of input, gesture and get a response which may be simply swap pieces of paper, write or erase something directly on the paper prototype etc. Not to mention that you are free to draw what you want. Won t my sketchboard already do the job? Paper prototypes are different from the sketchboards that you have already made: sketchboards are blueprints of the interface whereas prototypes are meant to be executed (really or simulated by a computer person). What to draw? Therefore, you need to draw again the different interface screens of the sketchboard but now 1

2 including the actual data for simulating the execution of a certain task. You need to take into account also user actions which are wrong in the sense of not following the right sequence of interaction (user clicks the wrong button, or does not select the right item in a menu, in order to perform the task you have asked him to complete). Prepare the feedback for any kind of input (correct or wrong) the user can do on your interface if it is supposed to give some, because you don t know what he/she can do! However, do not take it to the extreme. If the user has to write something on the keyboard, just overlap on the (paper) screen an small piece of paper with a (printed) keyboard and at the end of the writing show the result overlapping another small piece of paper on the corresponding edit text box. Draw the prototype on mobile templates we provided here teach/a21291/temes/ui_process/problemes/ui_process_1.pdf. Any trick that works is fine to simulate the interface dynamics, like when need to scroll a list or show how the time and duration of projects, tasks and intervals change. 2 Materials We are following as much as possible the process described in the Steve Krug s book Rocket surgery made easy: the do-it-yourself guide to finding and fixing usability problems, New Riders (2010), Ideally you should read some chapters of this nice book, and view again the example video shown there or here watch?feature=player_embedded&v=qckizhc99xc. What s definitely necessary to do the user testing session is the set of forms, checklists and scripts provided as additional material in this book, that you ll find at the course web page (direct link here: materials_adicionals/forms_rocket_surgery.zip) You need to read all of them carefully, all of you. Obviously, you also need to bring a prototype to simulate an execution, as described above. Other simple but very useful stuff in case you need to amend the prototype or simulate some output is shown: pencil, eraser, post-it adhesive notes, clean sheets of paper, scissor. And finally, secure an user for about 1 hour, possibly more. With regard to the user, CAUTION : can be anyone, the only constraint is that he/she can not be a present or former student of this course 2

3 do not tell him/her anything about the time tracker application: you ll explain its purpose during the session never explain to him/her how the interface works! No need either to get a zero errors result of the test. That would actually be bad, a sign that it failed or was fake. Design errors are normal at this stage, and what I m going to assess is how well do you have prepared and performed the session, so finding errors is normal and desirable. However, the errors should be on the interface design concept, not because the prototype is incomplete, badly drawn or somehow unusable. 3 Todo short list before the session The user testing session is a group activity, but is leaded by the person with the facilitator role. So you have to 1. Decide which member of the group does what: who s the facilitator, who s the computer and who is/are the observers, at most 2. The facilitator, in addition, is the responsible of the whole thing. He has to make sure that the group collaborates to: 2. Download the set of forms and print them 3. Read them, specially Instructions for Usability Test Observers, Usability test script, Things a therapist would say, they are short documents anyway. 4. Adapt and translate the Usability test script document so that you can read it aloud to your user. Instead of a web site we are testing a mobile application so you have to change them slightly. For instance ask which mobile phone model he/she owns, the Android/IOS version installed in it, which apps he/she uses the most and the like. Try to characterize the user profile (student, studies, age... ) 5. As said, make the prototype. 6. Print the user task descriptions to give them later to the user, so he/she doesn t forget them while struggling with your interface. 7. important: do the pilot test, i.e. make sure the user can complete the tasks with your prototype. More items of this list (like preparing some kind of non-cash reward for the user) are in the Usability Testing Checklist. Mark there the tasks you have completed. 4 User tasks We have selected three tasks intended to test the main features of the application: Task 1 At the very beginning of the application, when there s no user projects or tasks: will the user be able to add a task? 3

4 Si us plau, fes una tasca anomenada Desplaçaments per tal de poder després comptar el temps total que dediques a venir i tornar a la UAB o a altres llocs, en qualsevol medi de transport. Task 2 Navigation through the hierarchy of projects, tasks and intervals: will the user understand the meaning of these three entities? Si us plau, digues si el divendres passat 11 de desembre l usuari d aquesta aplicació va estar fent la pràctica de la assignatura de Disseny de software entre 3 i 8 de la tarda, i si és així quant de temps hi va dedicar. You need to insert a project Disseny de software at the top level, inside it the tasks Estudiar, Practica, Problemes, each of them with some intervals. Inside Practica include an interval from 15:35 to 16:20 plus 4 other intervals before and after that. At top level add some other projects, Requisits de software, Gestio de projectes without any tasks or projects. Task 3 Counting time: will the user know how to start and stop a task? Imagina t que ets un estudiant de la assignatura Disseny de software i que ara mateix comences a estudiar per l examen, i que ho faras nomes durant 10 segons (nomes vols veure si hi ha gaire temari i has de marxar corrents a fer una altra cosa). Com que vols comptar tot el temps que hi dediques, fes per que la aplicació ho enregistri. Task 4 Make a report: is the form well done? La aplicació no només enregistra el temps dedicat a les tasques que defineix l usuari sino que en fa informes, que son resums d aquests temps en forma de taules. Si us plau, demana un informe el mes detallat possible del temps dedicat a totes les activitats durant la setmana passada (del passat dilluns 12 al diumenge 18 de desembre), i demana l per tal que es pugui després visualitzar en un navegador web. 5 Todo list during the session Each testing team consists of the 2 or 3 persons of the project team, playing the roles of facilitator, computer and observer. It must be so because the session tests the prototype they have built according to their app scketchboard, a compulsory step before coding, and the errors found are to improve it. All of them will sit together, including the observers (they won t stay in a separated room watching through a TV monitor as supposed in the forms). Just to make sure the session goes as it should, remind that observers and computer don t talk and don t make approval or any other kind of gestures, specially they don t give cues or instructions to the user on how to do a task or explain or justify anything observers just take notes, but can help the computer person the facilitator will also be the greeter 4

5 the facilitator follows as close as possible the instructions in the Usability test script form, and speaks as suggested in the document Things a therapist would say facilitator asks the user to think aloud, explain what he/she thinks and if the user gets stuck may switch to another task in sum, read Instructions for Usability Test Observers, Usability test script 6 Deliverables 1. A short explanation of how you have prepared the session and who did what 2. The prototype itself, please put it inside some envelope with the name, NIU and role of each participant written on it and a who did what of the checklist 3. The Usability test script, translated and adapted to our application 4. The answers of the user to the questions in the Usability test script form, starting at First, what s your occupation? and the others being adapted as explained above (one of the observers has to write down them) 5. From each observer, a list of the problems he thinks the user had plus an explanation for each one of what the user did, on a blank sheet 6. The top three usability problems form with the selection of the 3 problems that the group members consider the most important (but not how to solve them) as agreed during a short discussion to be held when you are done with the user 7. If you record the session or part of it, a link to the movie file 7 Assessment Maximum score is 2 points to be added to the mark of exam 2, with an upper bound of 10 points. The score will be the one of the following situations which is the closest to your case: 0 points The prototype is poor, or does not comply with the present Android style, or is unfinished or incomplete, or is a sketchboard not a prototype. Not enough evidence that the session has been well prepared, for instance no translation of the script, or questions in it do not apply. The user already knows the application (for example, he/she is a former student of this course). There is some missing role. The observer or computer persons have not done enough in the preparation, for instance the observer has not worked on the prototype and/or does not know its logic. Some member of the team does not perform well enough the role it should play. 2 points Well done and complete prototype, covering the four tasks. It has been pilot tested before and in principle considers all possible/reasonable user inputs. Members have translated the user testing script and adapted it to our application, with questions to the user relevant with respect to it. All members play well their initial role and change it from one task to another everyone plays every role (say so in the deliverables: when has each member played who). 5

6 1 point Something in between 0 and 2 points. For example, fixed roles throughout all tasks, prototype covering just 3 tasks, prototype not implementing well navigation and new task/project or report forms, etc. Using prototyping tools like PopApp, Sketch or Justinmind will not necessarily make the session better than a paper prototype (could be even a problem if there s some mistake in the prototype), but if well done can be a plus; if it s the case, provide a link to it. 6

7 Llista de tasques per a l usuari Tasca 1 Si us plau, fes una tasca anomenada Desplaçaments per tal de poder després comptar el temps total que dediques a venir i tornar a la UAB o a altres llocs, en qualsevol medi de transport. Tasca 2 Si us plau, digues si el divendres passat 11 de desembre l usuari d aquesta aplicació va estar fent la pràctica de la assignatura de Disseny de software entre 3 i 8 de la tarda, i si és així quant de temps hi va dedicar. Tasca 3 Imagina t que ets un estudiant de la assignatura Disseny de software i que ara mateix comences a estudiar per l examen, i que ho faras nomes durant 10 segons (nomes vols veure si hi ha gaire temari i has de marxar corrents a fer una altra cosa). Com que vols comptar tot el temps que hi dediques, fes per que la aplicació ho enregistri. Tasca 4 La aplicació no només enregistra el temps dedicat a les tasques que defineix l usuari sino que en fa informes, que son resums d aquests temps en forma de taules. Si us plau, demana un informe el mes detallat possible del temps dedicat a totes les activitats durant la setmana passada (del passat dilluns 12 al diumenge 18 de desembre), i demana l per tal que es pugui després visualitzar en un navegador web. 7