Diseño y Evaluación de Sistemas Interactivos COM Usability Tests 28 de Octubre de 2010

Transcription

1 Diseño y Evaluación de Sistemas Interactivos COM Usability Tests 28 de Octubre de 2010 Dr. Víctor M. González y González victor.gonzalez@itam.mx Agenda 1. Comments on Partial Exam / next week. 2. Usability Tests 1

2 Usability Tests Chapter 10 Observing the User Experience by Mike Kuniavsky Usability Tests / What to Test? Usability testing should be used to examine the functionality of individual features. It is better to highlight potential misunderstandings or errors inherent in the way features are implemented rather than to evaluate the entire user experience. Usability testing is never one-time event: every round of testing can focus on a small subset of features 2

3 Usability Tests / When to Test? A good time to start usability testing is when the development cycle is somewhat underway, but not so late that testing prevents the implementation of extensive changes. Ocassionally, usability testing reveals problems that require a lot of work to correct. Tests scheduled throughout the development process. Usability Test Schedule Timing t 2 weeks t 2 weeks t 1 week t 3 days t 2 days t 1 day t t + 1 day t + 2 days t + 3 days t + 1 week t + 1 week Activity Determine the test audience; start recruiting immediately Determine feature set to be tested Write first version of the script; construct test tasks; discuss with the development team; check on recruiting Write second version of script; review tasks; recruiting should be completed Complete script; schedule practice test, setup and check all equipment Do practice test in the morning; adjust script and tasks as appropriate Test (usually 1-2 days, depending on schedule) Discuss with observers; collect copies of all notes Relax; take a day off and do something else Watch all tapes; take notes Combine notes; write analysis Present to development team; discuss and note directions for further research 3

4 Recruiting Time consuming, Time constrained: You need to find the right people and to match their schedules to yours The best people to invite are those who are going to need the service your are providing in the near future or who have used a competing service in the recent past. High level of interest and knowledge. Example of Recruiting Criteria Web site for Harley-Davison USA Men or women, preferably men 25 years old or older, preferably Have internet access at home or work Use the Web five or more hours a week Have bought something online in the last three months Are interested in Harley Davison Products 4

5 Recruiting For a focused task-based usability testing you should have a least five participants for each round of testing. Jakob Nielsen has shown that the cost-benefit cutoff for usability testing is about five users per target audience. Time slots management: Double book; leave at least half an hour between participants. For 60-min test you can do four or five in a single day. Choosing Features Features -> Tasks -> Order of Test Five features can be tested in a given minute test. A test duration in average is one to two hours. Together with the development team: Making a list of the top five features to test. 5

6 Choosing Features Look for features that are: Used Often New Highly publicized Considered Troublesome, based on feedback from earlier versions Potentially dangerous or have bad side effects if used incorrectly Considered important by users Choosing Features Competitive Testing Although comparing two interfaces is more time consuming that testing a single-interface, it can reveal strengths and weakness between products. Performing the same tasks with an existing interface and a new prototype can reveal whether the new design is more functional. 6

7 Creating Tasks Tasks need to be representative of typical user activities and sufficiently isolated to focus attention on a single feature Good task should be Reasonable: typical of the kinds of things that people will do. Described in terms of end-goals: phrase your tasks as something related to evaluator s life. Specific: explicitly articulated end goal. Doable: avoid frustration, ask what it s possible Domain neutral: everybody is able to do it, but not necessarily know how to do it. A reasonable length: 90 min test for tasks, 12 min average. Metrics Primarily a qualitative exercise Performance metrics: investigate the relative efficiency of different designs The speed with which someone completes a task How many errors they make How often they recover from their errors How many people complete the task succesfully 7

8 Scripts / Protocol / Guide A script for the moderator to follow so that the interviews are consistent and everything gets done. The script is divided into three parts: The introduction and preliminary interview The tasks The wrap-up Tecnología Anoto TM Las tecnologías digitales El manipular, almacenar y recuperar grandes cantidades información de manera sencilla. Replicar información. Compartir información. El papel Facilidad de uso. Transporte. Bajo costo en su elaboración. Flexibilidad de navegación a través de documentos. 8

9 Área de aplicación: Ambiente medico Se maneja gran cantidad de información Uso intensivo del papel Hoja de Enfermería 18 9

10 Prototipo de diseño 19 Sección de Signos Vitales 20 10

11 Sección de Medicamentos 21 Sección de Envió de mensajes 22 11

12 Evaluación 22 estudiantes de enfermería 5to. Semestre de licenciatura Sin experiencia en el uso de plumas digitales 5 computadoras 2 plumas digitales (Logitech io 1 ) 3 plumas tradicionales (verde, rojo y azul) 1 audífonos 1 impresora (HP Láser 600) 4 tabla con clip tamaño carta 23 Satisfaction - SUS The System Usability Scale (SUS) was originally developed by John Brooke while working at Digital Equipment Corporation (1986) Ten statements to which participants rate their level of agreement. A 5-point scale of agreement. No attempt is made to assess different attributes of the system (e.g., usability, effectiveness, etc.). The intent is that one look at the combined rating. 12

13 Design the Test Where: Determine the Location of the Tests Design the Test Where: Typical equipment and facilities 13

14 Costs Where: Costs of facilities: From Hire of our usability lab, observation room and sofa-filled break out area costs a flat fee of $ TAX for a full day (9am-5.30pm). Alternatively, the rooms can be hired out for $200 + TAX per hour. The labs can also be hired outside of working hours. For these prices, you'll receive: Full use of the usability lab and observation room Exclusive access to our break out area, complete with comfy sofas Technical support staff on hand in case of any problems Light refreshments Full session recordings on DVD Mexico City Costs Usability Lab: $ 510 daily Expert: $ 650 daily Assistant: $ 300 daily User recruitment: $ 100 each (high - mid) $ 60 each (low end) Report: $ 1000 Expert Analysis: $ daily Video equipment: $ daily Video Analysis: $ 400 daily Translator: $ 600 daily 14