Improving user interfaces through a methodological heuristics evaluation framework and retrospective think aloud with eye tracking Progress Report Supervisors: Dr. Tom Gedeon Mr. Christopher Chow Principal Researcher: Manik Mahajan
Overview Discount usability evaluations: Heuristics analysis: Heuristics Analysis Framework: Human Centered Computing (HCC) Workshop: Low-cost, low-time, and lowresource intensive techniques of conducting usability evaluations. A heuristics analysis is a discount usability technique, which typically sets a select number of evaluation criterion (heuristics) against which the usability of an interface is evaluated. Typically conducted by expert usability evaluators. This is to say, individuals with some background or familiarity with usability evaluation theory or techniques. However, typical heuristics evaluations are drafted uniquely for the interfaces of which the usability is to be evaluated. This research project posits the development and then utilization of a framework that abstracts and encapsulates complex concepts and practices into a standardised platform that expert usability evaluators can use to conduct stand-alone and comparative heuristics analysis, using mixedmethods. The primary goal of this research project is to help improve the usability of the HCC Workshop. The secondary goals of this project is to develop a Heuristics Analysis Framework. This framework would be open to use and improvement beyond the scope of this research project. 2
Research Processes Background Research Environment Scan Gap Analysis The cumulative methodological repertoire of HCI research is substantially broad, as a result a systematic review of literature has been deemed to be best practice om literature and background analysis in HCI research. Insights generated through consultation with usability practitioners, and subject matter experts to evaluate the difference between best practices and practices employed in the real world. (Design Thinking) Instead of creating something for the sake of it, substantial gap analysis of existing literature and practices was undertaken to find scope of surgical solutions in the problem space. As a consequences of this the Heuristics Analysis Framework was created. GOMS (Goals, Operators, Methods, and Selection Rules) Source:https://en.wikipedia.org/wiki/File:Extraction_machine.gif Source:http://designthinkingforlibraries.com Source:http://technologyadvice.com/blog/sales/finding-filling-gaps-sales-pipeline/ 3
The Design Squiggle, Damien Newman 4
Current Project State Heuristics Analysis Framework Pilot Studies Insights Generated Usability Heuristics: Visibility of System Match Between System and Real World User Control and Freedom Consistency and Standards Error Prevention Recognition rather than recall Flexibility and efficiency of use Evaluation Criteria: Effectiveness Efficiency Satisfaction Usability Score, determined as a weighted average of 33 usability heuristics evaluated against 4 evaluation criteria. In order to evaluate the desirability, feasibility and viability of the Heuristics Analysis Framework (HAF) a pilot studies were undertaken with 4 experts : Chris Chow; Manik Mahajan; and two professional usability testing practitioners. These enabled generation of early feedback which would help improve HAF to reduce the cognitive workload and other user experience aspects experienced while using HAF. The pilot studies were undertaken with the experts participating in a Think Aloud conducted by the Researcher, and then using HAF as a structured methodology to evaluate HCC Workshop s usability. The HCC Workshop has been rated a usability score of 3.85 on average in the studies so far. The feedback and experience of users of during Think Aloud was generally good. On average it took the users 40 minutes to use the HAF to evaluate the usability of the HCC Workshop. Simplification of some aspects of HAF is required to further drop the time it takes for it's users to fully implement it. This is to achieve a higher degree of discount -ness in its construct. HAF accomplishes its intention of enabling users to perform a structured heuristics analysis. 5
Future Work Eye Tracking Studies HCC Workshop The Heuristics Analysis Framework helped develop key insights into usability testing and user experience design. Eye Tracking will help further HCI research skills and knowledge. Retrospective Think Alouds with Eye Gaze Tracking allow rapid generation of user testing driven insights that can be coupled with empirical data through eye gaze tracking. Additionally this empirical data could be used to verify and validate the effectiveness of HAF. (if time permits) 10-20 participants will be recruited to participate in a Retrospective Think Aloud coupled with Eye Gaze Tracking. This project will compare the effectiveness of Retrospective Think Alouds and Retrospective Think Alouds with Eye Tracking, to compare and analyse the participants ability to identify usability issues in both cases. These studies will be conducted using the HCC Workshop as the test object, and use Eye Tribe and Empatica E4 to gather the empirical data. The HCC Workshop is the test object for this experiment. The experiment will test usability of four tasks: register/login; using eye tracking activity; using electrocardiography activity; using neural network activity. These activities were chosen as observation for workshops conducted in the past demonstrated particular interest in the three above mentioned activities. As a result of this experiment, useful feedback for the HCC Workshop will be achieved. 6
Summary The Heuristics Analysis Framework is an attempt to marry multiple theories and practices in discount usability testing, ergonomics efficiency and standardisation, and human-computer interaction principles, that enables a cheap, easy, and standardised way to test interfaces. Retrospective Think Aloud with Eye Gaze Tracking is a relatively discount usability method that can be run as a cheap and easy method by an expert. Eye tracking allows V&V of verbalisations (that are part of Think Aloud), in addition to allowing users to more deeply identify usability issues. This research project will generate useful feedback for the HCC Workshop, to help the improvement of user experience and the interface design. This adds direct value to the HCC Research Group, and RSCS. Additionally, the Heuristics Analysis Framework and the comparative analysis of Retrospective Think Alouds help add value to HCI as a discipline by: one, creating new benchmarked usability testing framework; two, corroborating the goodness of multiple usability testing methods. Core Research Considerations The World View Model 7
Questions? 8