USABILITY REPORT A REPORT OF USABILITY FINDINGS FOR OFF THE BEATEN PATH WEBSITE Prepared by: Cori Vandygriff, Joseph Kmetz, Cammy Herman, and Kate DeBusk
To: Off the Beaten Path Team From: Cammy Herman Cori Vandygriff Kate DeBusk Joseph Kmetz Date: 4 December 2013 Re: Usability Test Report The attached documents present our findings after conducting usability testing on your current website, offthebeatenpathstuff.com. We have spent the past several weeks compiling data and creating plans of action for you to re-design your website. In the following report, you will find our information delivered through the use of tables and figures, an overall review of our test, screener and user profiles, and our test findings. Along with our findings, we have also included our specific recommendations to improve your site. Our group thanks you for the opportunity to test your website, and we hope you find our recommendations helpful in creating a more usable site for your customers. 2
Executive Summary The purpose of this study was to discover usability issues within the Off the Beaten Path website, and to recommend solutions to these problems. Tests were conducted in the Texas Tech Usability Lab in November of 2013. The five participants engaged in four phases of evaluation: a briefing, a preliminary screener, a task-based performance test, and a post-test questionnaire to rate their overall experience using the site. Throughout testing, users typically experienced difficulty completing tasks that involved; navigating through the site, finding further information on the shop s wares, as well as searching for specific items. It is our recommendation that Off the Beaten Path work to remedy these usability issues through a redesign of the site in which they work to achieve the following goals. 1. Enhance the searchability of wares by incorporating a search bar and categorize items being sold to limit excessive scrolling. 2. Improve interaction with the site by including hyperlinks that navigate users to further information on wares sold in the shop such as prices, materials used, and size. 3. Clarify the purpose of navigation tabs by giving them more specific titles. 4. Eliminate unnecessary information and pages that are not conducive to easing the users searching. 5. Incorporate minimalist design techniques and change the font to enhance readability. 3
TABLE OF CONTENTS EXECUTIVE SUMMARY...3 USABILITY OVERVIEW...6 PURPOSE...6 PROBLEM STATEMENT AND TEST OBJECTIVES...6 AUDIENCE PROFILE...7 TEST ENVIRONMENT AND EQUIPMENT...7 Testing Computer Hardware and Software Configuration...7 Usability Lab Equipment...7 METHODOLOGY AND TASK LIST...8 Usability Task Methodology Overview...8 Briefing...8 Performance Test...8 Post-Survey...8 Scenario Script and Task List...8 FINDINGS AND RECOMMENDATIONS...9 GENERAL OBSERVATIONS...9 PERFORMANCE TEST FINDINGS...9 Task 1: Locating Off the Beaten Path...10 Recommendations for Improving Website Search...10 Task 2: Locating Obscure Products...11 Recommendations for Improving Website Satisfaction...11 Task 3: Locating Major Product...12 Recommendations for Improving Website Organization...12 Task 4: Locating Store Information...13 Recommendations for Improving Website Effectiveness...13 CONCLUSION...13 APPENDICES...14 APPENDIX A: SCREENING QUESTIONNAIRE...14 APPENDIX B: ORIENTATION SCRIPT...15 APPENDIX C: POST-TEST QUESTIONNAIRE...16 4
LIST OF FIGURES Figure 1: Scenario Script...Page 6 Figure 2: Locating Website Graph...Page 10 Figure 3: Locating Calligraphy Graph...Page 11 Figure 4: Locating Earrings Graph...Page 12 Figure 5: Locating Contact Information...Page 13 5
USABILITY TEST OVERVIEW The following usability report summarizes the methods used to test the Off the Beaten Path website and offers suggestions based on the findings. PURPOSE The purpose of this report is to present the testing procedures, findings, and suggestions of the usability test designed to accumulate information of the usability of the Off the Beaten Path website. This report will explain in-depth on the effectiveness of the website in particular instances. In effort to provide the best possible recommendations for the website, we have analyzed where the user had problems, indicated verbally or nonverbally. This report is organized into three parts: the Usability Test Overview, the Findings and Recommendations, and the Appendices. The Usability Test Overview contains the following: Problem statement and test objectives Audience profile Test environment and equipment Methodology and task list The Findings and Recommendations section is composed of a list of concerns and usability problems with specific tasks and suggests solutions for fixing these usability issues. The Appendices contains documents used throughout testing for reference. PROBLEM STATEMENT AND TEST OBJECTIVES The Off the Beaten Path website is encouraged to do the following in favor of improving site usability: Be easily accessible when searched for on the internet. Present information in an organized, intuitive manner. Present information clearly using design principles tailored for internet use. Maintain inventory and update stock and prices accordingly. Provide an easily accessible search bar. Given the purpose of the Off the Beaten Path website (promotion of the store,) the primary issues we identified via testing are as follows: Can the user locate the website? Does the website address confuse users? What are the user s first impressions? Is the splash page effective? Can the user navigate the website with efficiency? Does the website satisfy the user s goals? 6
AUDIENCE PROFILE To identify prospective test participants who might hold similarities to the site s actual users, we identified a primary user group and created a description of those users. Because the website appeals to a specific interest - artwork that is inspired by/created in New Mexico, we have only created one user group, but we will test two user profiles we have identified from within this one user group. The characteristics of this user group, along with other questions about prior experience and interaction with internet-enabled computers, among other criteria, will make-up the screening questionnaire (Appendix A) for identifying suitable test participants. The user characteristics we identified for our user group include: Age between 18 and 30 Has lived in the Southern/Southwest area of the United States of America for at least three years o These areas include any of the following: Texas, Oklahoma, Colorado, New Mexico, and Arizona. Must have some experience with computer use and the Internet Speaks English proficiently Interest in art Need a mix of females and males TEST ENVIRONMENT AND EQUIPMENT TESTING COMPUTER HARDWARE AND SOFTWARE CONFIGURATION All participants used a computer with the following configuration: Operating System: Windows 7 Internet browser: Google Chrome, Internet Explorer (users were given the option) USABILITY LAB EQUIPMENT We used both audio and video recording for qualitative and quantitative data collection. Video Sources: Web-camera located on top of the computer captured a front-view of the participant. A video channel captured the image on the computer monitor via a scan converter connected to the video output of the testing computer. A microphone captured the testing conversation. Audio and video information was recorded using Morae software. This information was labeled with information including participant name, date, and time of recording. 7
METHODOLOGY AND TASK LIST The usability test for the Off the Beaten Path was conducted in November of 2013. Five participants were isolated for testing. Kate Debusk acted as test facilitator in the interest of time. The remaining group members acted as observers and data loggers. All tests were conducted in the Texas Tech English Department Usability Lab. Each test was under 20 minutes long, including a brief post-test questionnaire. USABILITY TEST METHODOLOGY OVERVIEW The usability test consisted of three segments: the Briefing, the Performance Test, and a Post-Survey. These are described as following: BRIEFING Kate Debusk, the test facilitator, oriented each user using the orientation section from the script (see Appendix B). Each participant signed a consent form, guaranteeing anonymity and enabling us to use their comments. PERFORMANCE TEST Participants were asked to complete tasks involving searching for the Off the Beaten Path website, the first impression of the website, finding information, finding products, and site location. POST-TEST SURVEY The test facilitator issued a post-survey questionnaire to the participant, asking about the user s overall experience and ease or difficulty experienced during use. (See Appendix C). SCENARIO SCRIPT AND TASK LIST Users were asked to complete tasks as they would normally in their natural environment and utilize think-aloud protocol. We used the following script to communicate with participants during the testing: (See figure 1). 8
GENERAL OBSERVATIONS USABILITY TESTING FINDINGS Overall, the test participants uncovered several major usability issues: Figure 1. Read aloud scenario: 1. You want to begin researching the shop and its wares. a. Open a browser and search to find Off the Beaten Path in Cloudcroft, New Mexico. b. If found, go ahead and click on the website. 2. You re interested in what the shop offers its customers. Your friend claims Off the Beaten Path sells calligraphy, or decorative handwriting. a. Find out if Off the Beaten path sells calligraphy. b. Question: If you found calligraphy, how did you find it and how long do you feel it took? 3. You think that Off the Beaten Path may have something to give your friend for her birthday, so you want to find affordable earrings. a. Browse and find a jewelry set for your friend. 4. Intrigued by what you find, you decide that you do want to visit the store while on your trip. a. Find the store location and any other relevant information you will need to visit the business. Terminology- Users were confused by some of the labels on the navigation tabs. Locating information- Users found it difficult to search for specific products and information about wares. Lack of information- Users found some areas of the website that were vague. They also found several instances in which they wanted to find out more information on an item for sale when none was given. They wanted to click pictures of wares and be directed to the prices of works, materials used in making the product, and other relevant information. Design- Some users found the background of the site distracting, and several users also found the font difficult to read. NOTE: The following graphs below display the time (in seconds) that users required to complete each task; the accompanying descriptions explain each test and provide recommendations for fixing the issues uncovered. 9
Task 1: Finding Website locate the site. Overall, users did not have trouble finding the site when completing the search as asked. Some became confused when reaching the splash page, and didn t know how to proceed to the homepage. They also acknowledged that the page did not tell them enough about the site or what it offers users. Five of the six participants took under 35 seconds to Recommendation: To expedite the process of users accessing search information, we recommend that the splash page be removed from the site. This will ensure that users are able to enter the site easily without confusion. Task 2: Finding Calligraphy Overall, when searching for the more unusual item, calligraphy, users found the product difficult to find. Not able to search for the item, as a search feature was not included on every page and was difficult to locate, they struggled to find the ware on the long list of products sold by Off the Beaten Path on the other cool stuff page. 10
Many became frustrated scrolling through all the items sold, and said they would have discontinued their search before completing the task. Half of the participants tested took over a minute and 20 seconds to complete the task. Recommendation: To remedy the difficult searchability of unusual products on the site, we recommend that a search bar is added and made accessible to the user on every page. We also recommend that the location of the search feature be located above the navigation tabs on the right side of each page. Task 3: Finding Earrings All five users agreed that the displayed result was not what they expected to find. Each user was expressed frustration that clicking on the earrings did not display relevant prices or bigger pictures. Users also wanted to see more variety and different types of earrings than were available on the website. On average, completion time was 51 seconds for this task; participant C completed it substantially faster because he started the task from the earring webpage, by coincidence. Recommendation: We recommend you make the following changes to improve the user browsing experience: 1) Ensure that pictures are available live links. 2) Set displays prices and larger picture. 3) Group earrings and add more available jewelry currently for sale. 11
Task 4: Contact Information While most users did not struggle to find the contact information and address of the shop, none of the participants noticed the contact information located at the bottom of each page, opting instead to click the where we are navigation tab to complete the task. Overall, four out of six users completed the task in under 40 seconds. Recommendation: Though every participant was able to complete the task successfully, time to perform the task could be shortened if the contact information currently located at the bottom of each page was moved to the top of every page in a universal location. This would eliminate the need for users to use the where we are page to find this information. OTHER ADDITIONAL RECCOMENDATIONS: In addition to the aforementioned recommendations, we have compiled the following list of other improvements that could be made to the site. These recommendations are based on other concerns and comments made by participants throughout testing. Some users found the background and font used on the site to be distracting. To ease the readability of the site, the font should be altered to sans-serif and the background should be made simpler. This will help incorporate minimalist design techniques making searchability and skimming information easier for the user. Every participant noted that the pictures of cats used on the about otbp page served more to confuse the user than to aid them with their searches. It is therefore recommended that these pictures be removed from the site. Two of the six users said they would have prefered the where are we page to include a map. We recommend that an interactive map be included on this page to aid users in locating the shop. 12
CONCLUSION From our tests, we confirmed that typical users struggle with site navigation and are frustrated by a lack of comprehensive content on the website. After conducting our five tests and analyzing the results, our team recommends the following changes to improve site effectiveness and users browsing experience: 1. Enhance Searchability Relocate search bar Categorize items being sold to limit excessive scrolling. 2. Improve Site Interaction Include hyperlinks to simplify navigation Group items by prices, materials used, and size. 3. Strategically Name Navigation Tabs Clarify the purpose of navigation tabs by giving them more specific titles. Ensure new tab names are relevant to items contained in that section o For Example, users are often confused by which items might be contained in the stuff and our stuff categories 4. Eliminate Unnecessary Information Remove pages and information that are not conducive to a positive browsing experience o Areas that could be deleted include the splash page before a user can land on the website and the cat pictures throughout the site. 5. Increase Readability Incorporate minimalist design techniques. o For example, change website background to a more neutral design Change the font to improve readability. 13
APPENDIX A SCREENER Name: Recruited By: All respondents must be able to communicate clearly in English. If you have doubts, terminate.) Candidate speaks in clear, understandable voice? o Yes (continue) o No (terminate) If selected, will you allow us to videotape you? o Yes (continue) o No (terminate) Would you be willing to sign a consent form? o Yes (continue) o No (terminate) Because this study involves internet usage, we re looking for someone who is familiar with computers and uses the internet regularly. Does this sound like you? o Yes (continue) o No (terminate) How long have you lived in South/Southwest America, including Texas, Oklahoma, Colorado, New Mexico, and Arizona? 14
APPENDIX B ORIENTATION SCRIPT We have asked you to come in today to help us evaluate a website. Throughout the process we will be asking you to perform a series of tasks. We will be using a think out loud procedure to help us understand what you are thinking about as you perform tasks. Therefore, please talk aloud as you perform the tasks to let us know what you expect to happen when you interact with the website or whether or not it meets your expectations. We want to know what is surprising, confusing, or frustrating and why. When you share with us what you re thinking as you go along, we get a better understanding of how the process works for you. We would like to emphasize that this is not a test of your ability to use the site but rather to test the effectiveness of this website. Please note that I will be referring to my test script often throughout the test. Do you have any questions so far? Also, in order to prevent our data from becoming distorted, we are not allowed to answer questions directly. Please do not be alarmed if we do not answer your questions. At the end of the test we will ask you to fill out a short questionnaire about your experience with the website. We will be recording you both digitally and audibly. Consent Form: I d like to give you a consent form. We use this to comply with Texas Tech s Ethics Review Board. Any other questions? 15
APPENDIX C POST-TEST QUESTIONARE 1. Overall, please rate how easy or difficult it was to use this website. 1. very difficult 2. somewhat difficult 3. neither difficult nor easy 4. somewhat easy 5. very easy 2. How easy or difficult was it to find or navigate where you needed to go in the website? 1. very difficult 2. somewhat difficult 3. neither difficult nor easy 4. somewhat easy 5. very easy 3. Please rate your overall satisfaction with this website. 1. very dissatisfied 2. somewhat dissatisfied 3. neither dissatisfied nor satisfied 4. somewhat satisfied 5. very satisfied 4. Were there any parts of the website that you had trouble with or found confusing? Please explain. 16