Stream Features Application Usability Test Report Erin Norton and Katelyn Waara HU 4628: Usability and Instruction Writing Michigan Technological University April 24, 2013
Table of Contents Executive Summary 3 Description of Usability Test Project Description 4 Test Description 4 Methodology 4 Test Environment and Equipment 5 Evaluation and Measures 5 Test Results and Recommendations 6 Conclusion 7 Appendices Appendix A: Post-test Questionnaire 8 Stream Features Usability Test Report Page 2
Executive Summary The main focus of this report is to analyze the usability of the Stream Features Application. To properly address this problem, we recruited three participants and asked them to engage in a usability test. From their comments, we gathered helpful information to improve the usability of the Stream Features Application. In this report, you will find a project description, which details the background information and purpose for this assignment. You will also find a test description and the methodology of our testing, which details the logistics of the test and a schedule of the events that took place. Also included is a description of the environment in which we conducted our testing, and the evaluation measures we used to quantify the results. Finally, we have provided recommendations to improve the user s experience with the Stream Features Application. Our recommendations are based on the comments we received from our users during the testing sessions, which have revealed various problems and improvement opportunities for the application s design and also from the post-test questionnaire, which we had each participant fill out at the completion of each usability test. Much of the information gathered relates to clarifying terminology and also improving the navigability of the interface by adding navigation or comfort buttons. These recommendations we will discuss further in the results and recommendation section of our report. Stream Features Usability Test Report Page 3
Description of the Usability Test Project Description The purpose of this project is to test the usability of the Steam Features Application. The Stream Features Application gives every day citizen scientists the opportunity to engage in research by visiting locations of streams and other water crossing sites, collecting and recording the data they find, and eventually submitting it to be used for analyzing the condition of the site. The application s primary audience is scientists, researchers, landowners, and land managers who are more familiar with the data collection and specifications that are needed. Test Description For the usability testing, we used a task-oriented approach. The test was conducted in Walker 120C at Michigan Technological University on Tuesday, April 16, 2013 between the hours of 4:00 p.m. and 5:30 p.m. The participants were given 30 minutes to complete the tasks described in the scenario. The sessions were done using paper prototypes with sticky notes serving as pop-up help documentation. One member of the team tracked the user s progress through the application while taking notes and keeping time. The other team member moderated the session, prompting the user when necessary. The information we hoped to find from conducting the usability tests was as follows: 1 How does the user feel about the general design of the interface? 2 Is the procedure for creating a site intuitive? 3 Is the language used to describe functions appropriately helpful? 4 Is the app easy to navigate? 5 How does the user perceive the reliability of the app? 6 Does the user leave feeling satisfied with their experience? Methodology The usability test consisted of a short introduction, the description of tasks to be completed, the testing, followed by a post-test questionnaire. We instructed our participants to think-aloud and share their thoughts, however, we did not audio record their comments. Both members of the team had the opportunity to play the role of moderator and note taker. After the participant finished with the post-test questionnaire, we concluded with open-ended comments and cookies. The participants of our usability testing consisted of two female students and one male student from Michigan Tech. Their areas of study varied between Civil Engineering, Biological Sciences, and Scientific and Technical Communication. All participants were between the ages of 19 and 22. Two of the participants have experience using smartphone technology on a regular basis while the remaining participant has not had much experience using a smartphone. The participants have different contextual backgrounds and therefore were able to give us a wider variety of perception concerning the application and its usability. Stream Features Usability Test Report Page 4
Introduction: The introduction for the testing included the participant signing the consent form, an introduction to the project and the concept of citizen science, followed by a description of the scenario and related tasks they were asked to complete. Tasks: The participants were guided through the tasks using a formulated scenario, in which they were asked to imagine they were collecting data at a stream. This helped them work through simulated application functionality using the paper prototypes. The participants were asked to complete the tasks using a paper prototype in place of a functioning application on a smartphone. They were asked to create a site, add stream related parameters, and then save the site. Post Test Questionnaire (Appendix A): Participants were asked to complete a two-question questionnaire after the testing. This questionnaire allowed us to gather data about how the user felt about the application after they experienced using the prototypes. In addition to our observations, we used this data in our results. Test Environment and Equipment The usability tests were conducted in Walker 120C on the campus of Michigan Technological University. We used a reserved conference room with table and chairs that provided more than enough space to conduct the testing. The participant and the moderator sat and worked at one corner of the table while the note-taker sat at the opposite corner to observe while the participant worked through the tasks. Morae or other usability software was not used in order to complete the testing. Instead, and as stated above, a member of the usability team took notes while the other moderated the test. Paper prototypes were used for the participant to complete the task. The participants were also asked to handwrite their post-test questionnaires. Evaluation Measures To evaluate the results of our usability tests, we have used a top-down organizational format which is illustrated by the inverted pyramid. We looked at what types of issues occurred and grouped our results accordingly. We then looked for any outlying issues, or issues that occurred singularly. We then took the results and used them to formulate our recommendations for the improvement of the Stream Features Application. Stream Features Usability Test Report Page 5
Test Results and Recommendations Positive reinforcement we received during the usability testing about the Stream Features Application concerned the color choices utilized in the design. One of the participants remarked, It goes along with the theme of the app. The participants also did not experience any trouble understanding which buttons and interactive features preformed what action. Based on information from our usability testing participants, both through the usability testing and the post-test questionnaires, we were able to observe the following problems and have decided upon the following recommendations. A global problem the participants experienced was confusion with terminology used in the application, specifically the terms crossing and low-point which the participants ran across in their tasks. Another problem that was encountered was the lack of proper navigational buttons, such as the back or home buttons, so that the user can orient themselves within the application. One of our participants expressed his feelings in this way: My comfort buttons, the home and back buttons, were nowhere to be found. This is a travesty. These are the two major problems we discovered users faced when using the application. There were no outlier problems that the participants commented on either during the testing or on the post-test questionnaire. Two of the three participants said they would not use the application unless it was applicable to their work or necessary for their major, while one said they were mildly interested in the subject of citizen science. Our recommendations to improve upon the issues we have discussed are as follows: 1 To improve the application, terms could be defined in the pop-up help to give users a better idea of what data they need to collect. In the sticky note pop-up help we developed for our usability test, we attempted to define the function of the buttons but were unsure of the terminology ourselves. In the final implementation of the application, terms should be defined for novice users. 2 The application should also have the appropriate navigation buttons on each page so that the user can always orient themselves while using the application. Buttons such as a back button, a home button, and even a cancel button would help the user to accomplish the task they set out to do. If the user becomes lost or frustrated while using the application, they can use these buttons to find their way back to the home screen and/or cancel the action that they are performing. 3 Our final recommendation would be to create a functioning prototype and conduct usability tests with it so that users can interact with a prototype in a more realistic manner, i.e. closer to the nature of what the final product will be like. In using paper prototypes we discovered that it was difficult for the user to examine the actual purpose and functionality of the application. They were only able to do what was asked of them and there was little room for deviation or error, which we feel gave us a somewhat unrealistic feel for how the users will interact with the final application. Stream Features Usability Test Report Page 6
Conclusion After completing the usability testing, we have come to understand that the application has a great foundation. There is always room for improvement and we ve come to believe that this usability test is only one step on the road to final development. Through the process the application will need to be vetted at different stages to make sure that it is meeting the requirements for its intended purpose. It is our hope that our recommendations will be useful and implemented for the development of the application. Stream Features Usability Test Report Page 7
Appendix A Post-test Questionnaire 1. How did you feel while using the Stream Features application? 2. Did using the Stream Features application make you more interested in Citizen Science? Stream Features Usability Test Report Page 8