Physical Plant Usability Report. Appalachian State University s physical plant website usability report

Transcription

1 Physical Plant Usability Report Appalachian State University s physical plant website usability report

2 Physical Plant Usability Report Appalachian State University s physical plant website usability report For Mr. Drew Basinger Web Designer Appalachian State University Boone, North Carolina By English 3700 Students Devin Williams Evalynne Lee Hayley Hepburn Jacquilyn Lavalle Kelsey Bohon Kristi Brown 13 December 2012

3 Appalachian State University Boone NC December 2012 Drew Basinger Web Designer Appalachian State University Boone, NC Mr. Basinger: Attached you will find our report entitled Physical Plant Usability Report. The purpose of this report is to communicate any necessary background information, coupled with our usability study methods, materials, findings, and recommendations for your website. Thank you for your help in conducting this study. You kept us up to date, and provided links to the necessary webpages we needed to test. We d also like to acknowledge Dr. Brewer. She greatly contributed to our efforts by acquiring willing participants, and helping us organize our testing. While conducting our usability tests, we noticed a few strong usability trends. The most prevalent usability problem occurred when participants used the AiM system. The majority of participants were either unable to complete a work order request, or became extremely frustrated while teaching themselves the system. We recommend that changes be made to the AiM system, and we provide our suggested changes in the analysis section of this report. However, on a more positive note: participants generally had no trouble finding the website and found the information, descriptions, organization, and appearance of the website professional and extremely usable. We recommend that these elements on the physical plant website see no change they re efficient and usable just as they are. We hope you find our report helpful during your web designing process. If you have any questions or comments about the report, please feel free to contact us via our student s. Dr. Brewer will have a list of student s should you not already have this information. Sincerely, Devin Williams Evalynne Lee Haley Hepburn Kelsey Bohon Kristi Brown Jacquilyn Lavalle LETTER OF TRANSMITTAL i

4 CONTENTS Letter of Transmittal i Executive Summary iii Introduction Methods Defining Usability Objectives, Procedures, and Goals Screening Participants Acquiring and Scheduling Participants Usability Testing Process Data Compilations Materials Screening Questionnaire Pre-Test Questionnaire Usability Scenarios Post-Test Questionnaire Testing Locations Testing Equipment Results Pre-Test data Post-Test data Transcriber data Data in Context Data Limitations Analysis Areas of Satisfaction Areas of Improvement Our Recommendations Appendix Pre-Test Questionnaire Post-Test Questionnaire Usability Scenario Script Transcriber Form Consent of Release CONTENTS ii

5 EXECUTIVE SUMMARY In today s new technological era, more and more people are turning to the internet as a source of information and as a way to complete various tasks. Online usability concerns are becoming more of a concern, as businesses seek to attract and maintain potential users, clients, and potential audiences. We were charged with the task of testing the Appalachian State University Physical Plant Website. We used discount usability testing to test how the target audience both viewed and used the website. We specifically examined the accessibility of the sustainability, motor pool, and maintenance information and online submission processes on the website. We screened participants to determine if they met our participant and user profile. Specifically, we searched for ASU administrative staff members who had little to no experience with the website. Participants were asked to complete a series of questionnaires and tasks, which examined the aspects of the website being tested (sustainability, motor pool, and maintenance). Participants were both recorded and timed while being led through the tasks. We found that accessibility was not a particular issue for the physical plant website; all participants were able to locate it quite easily. However, finding sustainability information proved to be more of a challenge. The different locations of links, along with the labeling, may have been the primary cause of confusion. In addition, once on the ASU Recycles site, participants seemed confused when navigating the page as it did not have a clear direction or movement. Both the maintenance and motor pool pages frustrated participants; a few even gave up on completing the related tasks. The AiM system was very difficult to navigate unless the user was recently trained or extremely experienced. The motor pool page was difficult to navigate, and participants expressed frustration that certain information was not where they would expect it to be. We recommend step-by-step instructions for the AIM system; this was frequently recommended by participants as well. In addition, we recommend more specific labels for links, along with more consistent navigation techniques (don t be repetitive with links, utilize one bar: top, bottom, or side). The motor pool page also could benefit from a simpler, more intuitive navigation and perhaps the highlighting of important information, such as how to reserve a vehicle. The sustainability page could also use a redesign with a clearer, more direct navigation. EXECUTIVE SUMMARY iii

6 INTRODUCTION In today s increasingly digital world, people are relying on the web for more and more information. There is also an increased demand for online forms, applications, resources, and more. Traditional methods for acquiring information such as calling, ing, or visiting an office are becoming outdated. As this development occurs, it is important to invest in and maintain an accessible and usable online experience for potential customers, employees, and interested audiences. Our goal in this report is to empirically compile data that shows how usable the Appalachian State University Physical Plant website is, and suggest possible changes to increase the usability of the website. This usability report, in turn, should help influence any web design decisions, and keep the Physical Plant website both user-friendly and professional. Our team, under the direction of Dr. Pam Brewer and Mr. Drew Basinger, tested a range of participants, in order to evaluate accessibility and ease of use of the website. We considered three potential audiences for the website: (1) faculty, (2) staff, and (3) students. We determined that staff members were the most likely to interact with the various aspects of the site (especially AiM and motorpool), and due to time and resource constraints, we chose to focus solely on these specific users. This report reflects the data we gathered from the usability testing involving four administrative assistants from various departments, along with one residence life staff member. As suggested by Mr. Basinger, we specifically tested the website s usability in three areas: (1) sustainability information, (2) the motor pool request system, and (3) the maintenance request system. The testing was performed using discounted usability testing, a simplified and interactive usability process that required each participant to interact with different questions and scenarios while performing tasks on the website. We videotaped each participant, as well as took observational notes throughout each testing. Our participants used think aloud protocol, where they narrated their actions to us while performing their tasks. Our observational notes, videotapes, and questionnaires all helped us to gather and analyze usability data about the Physical Plant website. This report will explain our methods and materials used during our usability testing process, as well as report our observations and present our recommendations. In addition, we have prepared a presentation and included some of the significant video footage taken during the testing. INTRODUCTION 1

7 METHODS Defining Usability Objectives, Procedures, and Goals Before we began testing, we needed to understand our goals, procedures, and objectives necessary for our study. These elements were determined in part by our client, Mr. Drew Basinger, as well as by the general rules and objectives of standard usability studies and processes. Usability Objectives Mr. Drew Basinger described the Physical Plant website to us, as well his expectations and goals for the website. Because the website was still under construction, our usability study would greatly benefit the designing process. He defined three areas of focus for the website s usability testing: (1) sustainability information, (2) motor pool system, and (3) maintenance system. Our usability study s objectives were to test these three areas. Usability Procedures Like all usability studies, not only did we need to meet the needs of our client, but we also had to collect accurate, verifiable, and objective data. To do this, we decided we needed to script our testing process, use the same equipment, and test in the same environment. We also needed to run a pilot test, in order to detect and correct any overlooked mistakes. Usability Goals The purpose of all usability testing is to ultimately compile the gathered data, present the data, and recommend solutions. In order to meet this ultimate purpose, our goal was to create a system of documents that accurately measured the quantitative and qualitative data from the study. We needed a consent of release form, a transcriber form to record observations, a series of questionnaires, and carefully defined scenarios. Each of our testing documents needed to have relevant questions that helped us explore our study s objectives. METHODS 2

8 Screening Participants After defining our usability objectives, procedures, and goals, we needed to define our participants. Again, we combined our client s needs with the needs of accurate usability testing. Our Client s Focus Mr. Drew Basinger narrowed down the Physical Plant users into three categories: (1) faculty, (2) staff, and (3) students. Due to limited resources and time constraints, we decided to focus on staff, since they were most likely to interact with the website on a frequent basis. Usability Testing Needs We further narrowed down our participants to a group of five individuals. Five is the minimum number of participants needed for an accurate usability study, since it has been proven that the results of five participants will capture 85% of the results gathered in larger usability testings. Qualifying Participants Not only did our participants need to match a certain demographic and number, but they also needed to be screened for their level of website experience and occupation. We did not want to test users that were experts with the website or its systems, nor did we want to test individuals who were not staff members employed at the university. We created a set of oral screening questions to ask our potential participants. Selecting Participants We carefully selected and screened five participants, with four being administrative staff members, and one being a housing and residence life student staff member at the university. All participants had little to no experience with the Physical Plant website. Acquiring and Scheduling Participants We acquired our participants by speaking with university staff members that individuals on our team knew. However, Dr. Pam Brewer helped us find the majority of our participants. We screened each participant and then scheduled them for one of our three testing days: November 11, 13, and 19. No participant was allowed to be tested in the presence of another participant, so we scheduled separate rooms and times for each usability test. METHODS 3

9 Usability Testing Process Before we tested any participants, we did a single pilot run. This allowed us to get a feel for the length of the testing process, calculate our ideal times per usability task, and make any changes that were necessary to our process or testing documentation. We were able to add more relevant questions to our test, better specify our task instructions, and take out unnecessary information in order to keep our testing time down. During this pilot session, we realized the importance of having a uniform transcriber sheets to ensure that our data collection was consistent. We also learned how to ask the participant questions about what they were doing and how they were feeling, rather than have them navigate the website silently. Our testing process consisted of five sequential parts: 1. Setting up the testing location and equipment When we got to the testing room, we set up the video camera on the tripod, facing the computer. We minimized the new motor pool webpage (since the new page was not linked to the physical plant website yet). We also minimized the AiM testing webpage since the testing page was different from the actual linked page. 2. Prepping our participant for testing Each participant had a microphone hooked up to his or her shirt. They also were directed to fill out the consent of release form (Appendix E). 3. Asking participant to fill out pre-test Each participant was given a one page pre-test form to fill out prior to testing. 4. Conducting usability test During the usability test, transcribers were instructed to record all observations and times. A facilitator read from a script, directing the participant to perform various scenarios. Participants were directed to narrate what they are doing as they navigated through the website. 5. Asking participant to fill out post-test Each participant was given a one page post-test form to fill out after testing. We thanked our participants and offered them prizes for their time and participation. Data Compilations Each team member was given a role and a certain aspect of usability data to collect. There were three roles, with two team members interchanging one role: (1) video recorder, (2) facilitator, and (3) transcriber. The recorder was in charge of positioning the camera and setting up the audio. The facilitator read through the script and directed the participant, while the transcribers wrote down all observations and times throughout the testing. At the end of our usability testing, we went over the data collected by each individual and compiled that data into several different graphs and figures for this report. METHODS 4

10 MATERIALS Screening Questionnaire The first step of our usability testing process was to screen potential participants for the study. To do this we created screening questions that were designed to help us select the most qualified participants, as well as to get an idea of each participant s competency with the programs we were to test. Our screening questions were asked orally, either over the phone or in person, and never varied. The questions asked were: Are you familiar with the physical plant website? Do you submit requests for the motor pool? If so, how often? Do you submit requests for work orders? If so, how often? Would you like to participate in a usability test for educational purposes? Pre-Test Questionnaire Prior to testing, each participant was asked to fill out a set of pre-test questions (Appendix A). Presenting the pre-test questions to the participants enabled them to get a better understanding of what we were trying to test. Our pre-test questionnaire consisted of questions about the demographics of the user, and their familiarity with the website, its information, and its systems. These questions were very specific and easy to answer because they utilized a Likert scale, as well as multiple choice format. Usability Scenarios To ensure consistency between usability tests, we created a script for the assigned facilitator to read (Appendix C). This script took each participant through the entire testing process, starting with the signing of the release form, through the testing itself, and ending with the post-test and a sincere thank you. Usability scenarios were included inside this script and were designed to lead the participant through a set of three specified usability-oriented tasks. Our client, Drew Basinger, requested we test the usability of the motor pool, work orders, and the sustainability website. To meet these requests, we created three scenarios designed to test the usability of each of these categories. These scenarios not only defined the tasks that each participant must complete, but also gave some background information as to why the task was being performed and described what result the participant was seeking. What follows is a summary of each usability scenario: Scenario One: tested the ease of locating the website and its sustainability information, by instructing the participant to launch the website and locate information about ASU Recycles. Scenario Two: tested the usability of the motor pool site by instructing the participant to find and compare prices of vehicle rentals, as well as locate information on how to reserve a vehicle. Scenario Three: tested the maintenance request system by requiring the participant to locate the system, log in, and submit a work order. MATERIALS 5

11 Post-Test Questionnaire In order to record the testing results, we provided the participant with a post-test questionnaire (Appendix B). The post-test questionnaire was designed to record what the participants thought about the webpages they visited during the testing. These questions were specific and direct; they inquired about the level of difficulty the user had in completing the scenarios, as well as asked for additional comments or suggestions about the website. Again, the majority of our questions featured multiple choice format, utilizing a Likert scale. Testing Locations The majority of our testing was conducted in Appalachian s library s study rooms. Our testing took place over the course of three different days: November 11, 13, and 19. Each room had a whiteboard, a table, multiple chairs, and a single computer. However, two of our tests took place in Dr. Brewer s technical writing classroom, which featured multiple computers. Testing Equipment In order to maintain consistency between each usability test, we used the same equipment throughout our testing process. The following is a list of our testing equipment used during each usability test: An Appalachian State University computer with Windows operating system Google.com search engine homepage (all participants started out on this webpage) Firefox browser (all participants used this browser to navigate internet) The newly designed motor pool webpage (minimized at bottom of screen) The AiM testing webpage (minimized at bottom of screen) Usability Scenario Script, release form, pre-test questionnaire, and post-test questionnaire Transcriber form (designed to record times and direct observational notes; see Appendix D) Optional prizes (provided to thank the participant for their time) Sony microphone (captured audio of participant) Video camera and tripod MATERIALS 6

12 RESULTS Pre-Test Data From the pre-test data (table 1.1), we were able to assess the demographics of our participants, as well as their level of experience with the Physical Plant website and its maintenance and motor pool request systems. Clearly there are limitations to our data, considering that eighty percent of our participants were both female, over the age of 44, and with an administrative position at the university. In terms of website experience, we found several clear trends in our data: 80% of participants had some prior experience or knowledge with the website 80% of participants had worked for the university for at least two years 80% of participants perceived the website as at least somewhat useful 80% of participants occasionally use the maintenance system 60% of participants had never used the motor pool portion of the website ABOUT OUR PARTICIPANTS Detail Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Age Sex F M F F F University position Program Assistant Office Manager Associate Resident Assistant Program Assistant Business Officer Years holding position 7 years 2 months 2 years 3 years 6.5 years Familiarity with website None Some Some Some Some Perceived usability of website Somewhat useful Useful and easy to navigate Somewhat useful and difficult to navigate Somewhat useful Motor pool request Occasionally Never Never Often Never frequency of use Maintenance request Never Occasionally Occasionally Occasionally Occasionally frequency of use Usability of motor pool Difficult Usability of maintenance Very easy Easy Use of website to access motor or maintenance No: paper documents Yes No Yes No Table 1.1 The following table displays the pre-test results that qualify our participants and show their level of experience with the website and systems tested. RESULTS 7

13 Post-Test Data Our post-test data (table 1.2) helped us to calculate the usability of the website, based on the participants subjective evaluations of the website. Our questionnaire turned up these distinct trends: 100% of participants found the website s information at least somewhat useful 80% of participants found it easy to locate the website 80% of participants found the descriptions and instructions useful 80% of participants found the website professional and visually appealing 60% of participants found the website clearly organized 60% of participants found the motor pool request system easy to use 60% of participants found the maintenance request system easy to use 60% of participants suggested usability changes for both the motor pool and maintenance At the bottom of table 1.2, we ve compiled brief summaries of the usability suggestions our participants left on their questionnaires. While forty percent had no suggested changes, a large majority of participants did. Their suggestions were: Participant 1: Make the maintenance system more user-friendly Participant 2: Make instructions for requesting rental vehicles Participant 3: Make instructions for completing maintenance requests Participant 4: Clarify how to use the vehicle request form on motor pool, and inform community about the website and its valuable resources for staff Participant 4: Make instructions for completing maintenance requests Participant 5: Differentiate the availability and reservation process for motor pool WEBSITE EXPERIENCES Detail Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Easy to find website Very difficult Easy Very easy Easy Easy Information usefulness Extremely Useful Useful Somewhat useful Useful useful Descriptions and Useful Somewhat Useful Not useful Useful instructions usefulness useful Website visually Agree Agree Strongly agree Agree Disagree appealing/ professional Website clearly organized Agree Agree Strongly agree Disagree Disagree Easy to find and use Very easy Difficult Easy Difficult Very easy motor pool Easy to find and use maintenance Very difficult Easy Very easy Easy Difficult Suggested changes for motor pool Suggested changes for maintenance Make more user-friendly Instructions for rental request Instructions for maintenance requests Clarify request form for motor pool; inform community about Instructions for maintenance requests Table 1.2 The following table details the post-test results. This table shows the experiences of the participants, and allows conclusions to be drawn about the usability of the website. Differentiate availability and reservation process RESULTS 8

14 Transcriber Data During every usability test, two assigned transcribers documented all their observations about the participant, along with the time it took for the participant to complete their tasks. This data was then used to further study the usability of the website. Table 1.4 (on following page) documents the data trends we found in our observational data: 80% of participants used the A-Z index to find the website 60% of participants had trouble locating the sustainability information 60% of participants had trouble locating the motor pool information 80% of participants had trouble filling out a maintenance request 40% of participants failed to complete a maintenance request 100% of participants were confused by the browser blocking the AiM pop-ups The time it took for each participant to complete each task was very important to our study. We assigned an ideal time to each task within the scenarios. We determined these ideal times by timing one of our team members as they used the website. Graph 1.3 shows a comparison between our ideal times and our participants actual times per task. Our data trends are as follows: 60% of participants located the website in under 40 seconds 60% of participants took over 1 minute to find the sustainability information 80% of participants took over 1.5 minutes to find the motor pool information 80% of participants took over 3 minutes to complete a maintenance request TIME PER USABILITY TASK 5.00 Recorded Minutes Locating Website Locating recycling PDF Locating and comparing motor pool Locating and submitting maintenance Graph 1.3 This graph compares the participants times to our predicted ideal times. The differences between these times can be used to determine the usability of the website. RESULTS 9

15 OBSERVATIONS Participant 1 Task Time Ideal Time Observations Locating website 0:22 0:40 Used A-Z Index to find website Locating recycling PDF 0:12 1:00 Found PDF quickly and easily Locating motor pool and comparing motor pool data 0:35 1:20 Found information quickly and easily Locating maintenance and submitting work order 3:32* 2:40 Very frustrated when filling out work order. Clicks on different links, but unable to fill out form. Gives up after 3 minutes of attempting.* Participant 2 Locating website 1:20* 0:40 Used A-Z Index to find website Locating recycling PDF 3:19* 1:00 Has trouble finding information. Does not immediately scroll to find information. Unsure which section to click. Locating motor pool and comparing motor pool data Locating maintenance and submitting work order 2:19* 1:20 First time using motor pool system. Does not immediately scroll down to find information. Confused about rental process and links. Could not find rental procedure under procedures. 5:09* 2:40 Knows how to operate AiM (taught by coworker). Only used few times before. Confused by browser blocking pop-up options. Has to experiment and guess at form options and procedure. Participant 3 Locating website 0:20 0:40 Used ASU web search to find website Locating recycling PDF 1:05* 1:00 Unsure where to click for information. Locating motor pool and comparing motor pool data 2:27* 1:20 Had not used motor pool before. Located information quickly and easily. Locating maintenance and submitting work order 1:45 2:40 Has used AiM before. Confused by browser blocking pop-up options. Completed quickly. Participant 4 Locating website 1:00* 0:40 Used A-Z Index to find website Locating recycling PDF 0:35 1:00 Scrolled very little when looking for information. Locating motor pool and comparing motor pool data 1:25* 1:20 Confused about where to find bus/van reservation procedure and information. Locating maintenance and submitting work order 4:19* 2:40 Confused about where to find new work order button. Went back to main menu several times to experiment (took 3 minutes). Had trouble filling out location and problem code. Confused by browser blocking pop-up options. Participant 5 Locating website 0:30 0:40 Used A-Z Index to find website Locating recycling PDF 1:02* 1:00 Navigated site easily, despite never using before. Locating motor pool and comparing motor pool data Locating maintenance and submitting work order 4:01* 1:20 Frustrated about how to find vehicle reserve information. Goes to FAQ but finds it too hard to locate information. 5:20* 2:40 Was trained with AiM but cannot remember how to use program. Always thought program needed more usability. Gives up after 5 minutes.* Table 1.4 The following table displays the data gathered from the transcribers recorded observations. RESULTS 10

16 Data in Context There were many positives and negatives that our participants encountered during their use of the Appalachian State Physical Plant website. Feedback on the site s description and usefulness was generally positive. In fact, many of our participants stated that there was no need for the motor pool or maintenance offline options (filling out paperwork), since it would be much simpler and efficient to keep everything online (see videotape). The website was also considered visually appealing and professional by the majority of our participants. However, when it came to the motor pool and maintenance systems, many of the participants had difficulty finding information about the motor pool and submitting a simple work order. Participants were confused about the request process verses the searching for the availability of a vehicle on the motor pool website. When it came to the work order process, we observed a few participants give up before they completed their work orders. Even familiar users faced difficulty when it came to finding the new request icon. The pop-ups also caused issues. The sustainability section of the website also caused difficulty for participants. See table 1.6 (on following page) for a complete summary of our findings. At least one participant in every scenario was able to meet our ideal time (table 1.4). However, when averaging our participants times, we saw that our ideal times were never met (graph 1.5). Furthermore, the level of difficulty, or lack of usability, was reflected in the gap between the average and ideal times: we saw that the participants average time to locate the website was very close to our set ideal time; however, the maintenance request task took, by far, the longest to complete, and has the largest gap between the average and ideal times. Recorded Minutes AVERAGE V. IDEAL TEST TIMES Locating Website Locating recycling PDF Locating and comparing motor pool Locating and submitting maintenance Average Time Ideal Time Graph 1.5 This graph displays the our ideal times compared with the average of our participants actual times. It also compares the average time per task. RESULTS 11

17 USABILITY SUMMARY Website Usability Detail P. 1 P. 2 P. 3 P. 4 P. 5 Usability Easy to find X X X X 80% Information useful X X X X X 100% Descriptions and X X X X 80% instructions useful Visually appealing and X X X X 80% professional Clearly organized X X X 60% Use for access to motor X X 40% or maintenance system Sustainability Usability Easily completed X X X 60% sustainability scenario Motor Pool Usability Used motor pool before X X 40% Motor pool easy to use X X X 60% Easily completed motor X X 40% pool scenario Suggested changes for X X X 60% motor pool Maintenance Usability Used maintenance before X X X X 80% Maintenance easy to use X X X 60% Easily completed X 20% maintenance scenario Suggested changes for maintenance X X X 60% Table 1.6 The following table analyses the percentage of each participant s experiences and opinions about the website. Data Limitations There are a few limitations to our collected usability data. (1) the majority of the participants were female administrative employees, age 44 and over, so there was not much diversity in the participants studied. (2) Each usability test varied slightly from the last: forms were updated, different facilitators and transcribers were used, and technical issues arose during some of the usability tests. (3) Our scenarios and pre-test questions may not have been specific enough. Some questions didn t differentiate between the online and offline experiences, while some scenarios were vague enough for our participants to constantly ask for clarification. (4) No member of our team had ever conducted a usability study before; therefore, we ran into roadblocks that more experienced testers might have avoided. We were also without a usability lab, which would have made things like obtaining video footage and monitoring the computer screen easier. RESULTS 12

18 ANALYSIS Areas of Satisfaction Some key areas of your website are extremely usable, and we recommend that these areas remain unchanged in the future. Evidence of satisfactory usability became evident after closely looking through our data gathered mainly from the post-tests, observational notes, and video recordings. Visual and Professional Appeal Eighty percent of the participants found the website to be both visually appealing and professional (table 1.6). In fact, when asked for additional comments about the website, many participants mentioned altering the motor pool or maintenance, but never suggested changes for the overall design of the website (table 1.2). The current design elements of the website are very important to the website s usability and credibility as a university website. Location and Access The participants had almost no trouble locating the website. Participants typically used the A-Z index on the Appalachian State University website, and looked up the Physical Plant website with extreme ease (table 1.4). Eighty percent of the participants were able to find the website in one minute or less (graph 1.5). Information and Descriptions On the pre-test questionnaire alone, eighty percent of participants agreed that the information on the Physical Plant website was useful (table 1.1). After participating in the usability test, one hundred percent of the participants indicated on the post-test questionnaire that the information they viewed on the site was useful (table 1.2). In fact, Participant 4 wished that she had known about the site beforehand; she recommended spreading the word about the website to other administrative employees (table 1.4). ANALYSIS 13

19 Areas of Improvement The data that we gathered from the usability tests suggests that there are some aspects of the website that are not user-friendly. These aspects should be taken into consideration for improvement. Sustainability Information Only sixty percent of participants easily located the sustainability information (table 1.6). Many participants had trouble finding where to click for the ASU Recycles page (table 1.4). In fact, after our pilot, we had to change the term sustainability in Scenario One to recycling information in order to ensure participants wouldn t become confused, since there is no general sustainability information on the website, only a link to ASU Recycles. Also, recycling is only a small part of the green effort that Appalachian State is so famous for, so expanding the sustainability information to include other sustainability efforts may be extremely beneficial. Motor Pool Information We observed that sixty percent of participants had difficulty specifically with finding the information about vehicle reservation (table 1.4); we noticed that only sixty percent felt that the motor pool webpages were easy to use; and sixty percent had suggested changes for the motor pool webpages (table 1.6). This consistent sixty percent shows that the motor pool information is not user-friendly. In fact, Participant 5 became extremely frustrated, and when she went to the FAQ section, she refused to look through the information-laden page for the information, claiming it was not user-friendly (table 1.4). Maintenance System (AiM) When looking at graphs 1.3 and 1.5, it is clear that the maintenance system takes more than twice as long to complete in comparison to the other tested sections of the website. Furthermore, we observed only twenty percent of participants completing the maintenance scenario without difficulty (table 1.6), with two of our participants being completely unable to fill out a maintenance request. This number is particularly concerning since eighty percent of participants had used AiM before, whether through training, work related tasks, or when watching another employee submit a request. (table 1.1). Many of our participants became noticeably frustrated with the AiM system. And sixty percent of participants suggested changes for the maintenance system (table 1.4). Our data overwhelmingly concludes that the maintenance request system is not user-friendly. ANALYSIS 14

20 Our Recommendations As a team, we ve come up with a list of usability changes for each tested section of the website. We recommend making changes to the sustainability information, motor pool, and maintenance areas on the website, based on our usability findings. We ve included suggestions for the website s general navigation and organization, but these areas are more user-friendly and not as necessary to change. Website Appearance, Information, Navigation, and Organization o Create more consistency with links: either open an new tab or remain in the same window; don t use an inconsistent mix. o Label and clearly differentiate the links on the top, bottom, and side of the website. Consider getting rid of repetitive links. o Actively advertise and inform faculty and staff about the Physical Plant s website and its online resources. Sustainability o Include more sustainability information on the website; ASU Recycles is only one sustainable activity that Appalachian participates in. o Create a sustainability section so that the ASU Recycles link isn t perceived as random and disconnected from the rest of the website s purpose. o Highlight important sustainability information, rather than taking them to another in depth website like ASU Recycles. Motor Pool o Highlight important information about the rental process, rather than having a mass of archived information under the FAQ section. o Reorganize the FAQ section in order to make it less daunting and more inviting. You can do this by dividing the FAQs into sections, with each section on a different page. o Differentiate how to check for vehicle availability verses how to reserve a vehicle. o Provide instructions for how to request a vehicle. o Consider making the rental procedure both an online procedure with an optional offline procedure. Don t make people go to offices or call when they could just fill out a form online. Maintenance o Consider renaming buttons that are convoluted in description: i.e. label the new page icon so that people know why they should click on that particular symbol. Change the customer request button to create a work order. o Provide step-by-step instructions on the AiM website. This will remind people what they need to do, without them needing to go back and look up the information again, or be forced to rely on their memory. o Rename the video that describes how to submit a work order. Not one participant tried to use that resource, probably because it wasn t properly described. Pair the video with step-by-step written instructions as well. o Create a disclaimer about browsers potentially blocking AiM s pop-up options. This issue caused many participants to become confused and frustrated. Consider using drop down menus rather than pop-ups for AiM s selections. ANALYSIS 15

21 APPENDIX A APPENDIX 16

22 APPENDIX B APPENDIX 17

23 APPENDIX C APPENDIX 18

24 APPENDIX D APPENDIX 19

25 APPENDIX E APPENDIX 20