Usability Testing Report of College of Liberal Arts & Sciences (CLAS) Website Submitted to: Ceily Hamilton, Director of Information Technology Alex Chapin, Executive Director of Academic Technology Compiled by: Andrea Patawaran-Hickman, Jess Cope, and Kailan Smith February 2015
February 26, 2015 Ceily Hamilton, CLAS Director of IT Alex Chapin, Executive Director of Academic Technologies College of Liberal Arts & Sciences UNC Charlotte Dear Ms. Hamilton & Mr. Chapin, The attached report presents results of a usability test focusing on faculty and staff s exchanges with the UNC Charlotte College of Liberal Arts & Sciences (CLAS) website for Dr. Wickliff s ENGL 5181 Writing User Documents course. The goal of the usability test was to observe and analyze how well the website functioned for the UNC Charlotte community. This report includes specifics from the observation of four faculty members from departments within the College of Liberal Arts and Sciences (two professors of Philosophy, one associate professor of Geography & Earth Sciences, and one professor of English) who agreed to participate in a 30 minute usability test. These tasks included navigating and locating UNC Charlotte s CLAS website, History department s CLAS page, an Ecology research topic, the CLAS Connections log-in, the help page for editing and updating profiles, Exchange Online to submit news or events, and a blog, which in this case was a blog titled Class Dismissed. In completion, the results, observations, and responses by the Participant show that UNC Charlotte s CLAS website is functioning, but this report discusses suggestions and/or recommendations that will further make the CLAS website a success. Effectively, the CLAS website was easily found and all participant were generally satisfied with the lay-out of the website. Most participants were particularly frustrated with locating the help page, research topics, and finding/logging onto CLAS Connections. In fact, the usability test indicated that the participants did not know the exact purpose of CLAS Connections. These observations indicate that the UNC Charlotte CLAS website is usable, but is not entirely efficient in accessibility of resources in reaching its audience. Recommendations include more obvious methods to present resources such as live links within paragraphs with larger font size and bold face, the three main attributes of CLAS website (CLAS Connections, Thinking Matters, and Exchange Online) relocated to left side of screen so that users can recognize them as links instead of photos, and conducting more workshops about CLAS website for faculty and staff. Sincerely, Andrea Patawaran-Hickman, Jess Cope, Kailan Smith UNC Charlotte Graduate Students
Contents Executive Summary... 4 Introduction... 5 Methodology... 5 Pre-Interview & Participants... 5 Recommendations... 11 Conclusion... 12 Appendices... 14 Appendix A: Participant Task Times... 14
Executive Summary To analyze the functionality of the UNC Charlotte College of Liberal Arts & Sciences (CLAS) website for target audience of faculty, staff, alumni, donors, and students, a sequence of usability tests were conducted to exam if the UNC Charlotte CLAS website is a useful resource to the UNC Charlotte community. This usability test targeted faculty and staff. During the usability test, we observed that participants had most difficulty with: Locating research topics Locating and logging into CLAS Connections Locating help page for updating profiles Submitting news/events to Exchange Online The participants expected for these tasks to be effortless, but were often deterred because the information needed was concealed beneath other parts of information or not easily accessible. To improve the UNC Charlotte s CLAS website and fulfill faculty and staff s needs, primary recommendation includes clear, concise, and readily available resources to appear on homepage of CLAS website and within compartments: CLAS Connections, Thinking Matters, and Exchange Online. This report includes specifics from the observation of four faculty members from departments within the College of Liberal Arts and Sciences (two professors of Philosophy, one associate professor of Geography & Earth Sciences, and one professor of English) who agreed to participate in the usability test. These participants were given a pre-questionnaire to confirm that they fit within the criteria of the targeted audience. The usability testing was conducted at convenient locations, suggested by the four participants, where they would feel most comfortable. For each test, the participants were asked to complete seven tasks while navigating the UNC Charlotte CLAS website. Participant activity was audio and video recorded with an online screen recorder, Screencast-O-Matic, and notes were taken by two observers. After the test, each participant was asked a series of follow-up questions to acknowledge their preferences on functionality and presentation of the CLAS website. The participants were able to complete the test within a 30 minute timeframe, however, this does not mean that each task was completed. The CLAS website was positively and effortlessly found and all participants were generally satisfied with the lay-out of the website. Yet, most participants were particularly frustrated with locating the help page, research topics, and finding/logging onto CLAS Connections. In fact, the usability test indicated that the participants did not know the exact purpose of CLAS Connections. All four participants had to move on from at least one specific task because it was either too difficult or too ineffective. This indicates that the UNC Charlotte CLAS website works well, but is not entirely efficient in presentation and accessibility of resources. 4
Introduction In order to test the usability of the UNC Charlotte s CLAS website, a series of tests were conducted that specifically focused on the CLAS website s CLAS Connections, Thinking Matters, and Exchange Online webpages. The usability test included pre-questionnaire screening, consent forms, tasks, and follow-up questions for each participant. The objective of the usability test was to observe how well the CLAS website connects faculty members to other faculty, staff, and community. Other aspects test how easily faculty members could navigate between sections of CLAS website and contribute to exchanges. The follow-up questions focused specifically on participants layout and design preferences. This report consists of findings and recommendations from results that analyze four faculty participants (two professors of Philosophy, one associate professor of Geography & Earth Sciences, and one professor of English). Although the faculty participants identified themselves as frequent internet users, they admitted to having limited experience with the CLAS website. The participants efforts in this study have allowed us to gain a better understanding of user needs, locate problematic areas, and offer suggestions for improvement. Methodology Pre-Interview & Participants Targeted audience of UNC Charlotte s CLAS website was primarily faculty and staff. After examining the names of faculty and staff, a list was compiled of possible participants. To contact these faculty members, an email blast was sent stating objectives and goals for testing. Possible participants were given a pre-questionnaire to confirm that they fit within the criteria of the targeted audience (see Figure 1). The following information requested included: Name Gender Years at UNCC Average internet use per week Blogging experience Experience with the CLAS website Preferred meeting day and time 5
Participant Professional Title Gender Years at UNCC Average Internet Usage per week Active blogger Experience with CLAS website Participant A Professor F 7 months Daily No Limited Department of Philosophy & Department Chair Participant B Associate Professor Department of Geography & Earth Sciences F 10 years 10-20 hrs. weekly No Limited Participant C Lecturer of Philosophy Department of Philosophy M 9 years 30 hrs. weekly No Limited Participant D Associate Professor Department of English M 25 years Daily No Limited Figure 1: Pre-questionnaire results. The participants selected for usability testing included faculty members who were targeted as having limited experience with the CLAS website. Usability Test All four participants were asked to perform a 15-30 minute usability test while navigating and completing tasks on UNC Charlotte s CLAS website. These tasks included navigating and locating: UNC Charlotte s CLAS website History department s CLAS webpage An Ecology research topic CLAS Connections log-in Help page for editing and updating profiles Exchange Online to submit news or events A blog titled Class Dismissed. Participants were informed that the test was for Dr. Wickliff s ENGL 5181Writing User Documents course and directors of the website, and will only be seen by Dr. Wickliff, fellow ENGL 5181 students, and directors of the CLAS website for further research and improvements. The participants were then asked to sign a consent form, stating that they agree to the terms of the usability test. The participants performance was audio and video recorded through an online screen recorder, Screencast-O-Matic. Lastly, the participants were reminded that the CLAS website was being tested, not the participants, and were encouraged to verbalize their thought process and actions performed while the observers silently analyzed and took notes. 6
The participants were all frequent users of the internet with an average internet usage per week ranging from daily to 30 hours a week, however, all four participants had difficulty performing tasks given for usability test (see Figure 1). Although these participants use internet regularly, they did state that their experience with CLAS website was limited. This is evident by results from usability test of each participant. Analyses were compiled of each participant s task times by minutes and the results show that logging into Class Connections, locating departmental CLAS pages and specific research topics acquired the longest time (see Appendix A). Observations concluded that participants are likely to give-up on task completion after three unsuccessful attempts or after three minutes of time. Participants identify their next step after failing to complete a task as seeking help through email or telephone communications. Post-Test: Follow-Up Questions The participants were vocal during the usability test, stating their opinions on what was difficult or easy to use. Therefore, follow-up questions focused on key components in regards to the design of UNC Charlotte CLAS website and asked for preferential choices about the functionality and presentation. There was a total of eighteen questions programmed for quick, reflexive answers about website design by having participant choose one of two options for each question. These set of questions provided valuable findings and were analyzed for recommendations of further improvement (see Appendix B and C). Findings This section addresses the main areas of strength and recommended revisions that were identified in both the usability tests and the follow-up questions for the CLAS website. This report has prioritized and condensed information from follow-up questions and provide recommendations when appropriate. Landing Page The CLAS website s rolling graphics are preferred to competitive universities nonrolling graphics and selectable graphics. Participants prefer UNC Charlotte s current method of only embedding text and pictures with no videos. Participants indicated not realizing that CLAS Connections, Thinking Matters, and Exchange Online graphics are links. While these links are colorful and visually appealing, they resemble a picture and not readily identifiable as links. Their right column position also makes them less noticeable to users. We recommend changing the link images and moving them to the left-side navigation column of CLAS website. 7
Layout and Design A majority of participants prefer the current CLAS website three column layout to paragraph or tab format. The link hover color is more visible and clear than competitive universities. CLAS website s current method of grouping the Liberal Arts and Sciences departments under the Academics link is preferred to separating them. Participants did not find links in the right-side column of the CLAS Connections page: Department & Programs, Research, and Connection Update (see Figure 2). The right column gives minimal attention to these important links, which is contributing to lack of noticeability. The font size and color also diminishes their noticeability. Moving these links to the left-side column and giving them a darker and larger font would make website layout more user-friendly. Figure 2: CLAS website right-side navigation. Moving the research topics to the left-side of the page may help users realize that these are navigation links. Figure 3 demonstrates a mock design of how research topics on left-side could appear. 8
Figure 3: Mock design of CLAS website research topics on left-side column of CLAS website. Participants preferred having only sanserif in the CLAS website. Most participants described serif font as old or outdated while sanserif is considered more modern and cleaner looking. Participants did not realize that the Exchange Online link would change We Want Your News! after clicking on it, nor did they immediately notice the change (see Figure 4). In addition to changing the Exchange Online link, adding We Want Your News! with its own link on the landing page might be beneficial to users since left-side of website is preferred search area for all participants. This change would reduce the amount of clicks to reach the news submission and increase visibility (see Figure 5). Figure 4: Change from Exchange Online to We Want You News! 9
Figure 5: We Want Your News! on left-side column of CLAS landing page. Content and Language The current CLAS website s Academics page can be navigated more quickly than those of competitive universities; the list form is preferred to block form or only a jump-to bar. However, adding a jump-to bar to our current list form would be preferred by two of the participants (see Figure 6). Figure 6: UNC Charlotte CLAS Academics list versus jump-to menu from competitive university website. Participants were not aware that a department page is different than a CLAS Connections department page. All participants were instructed to go to the History department s CLAS page during the usability test and went straight to the department page and never visited the department CLAS Connections page. In follow-up discussion those participants said they were not aware that CLAS Connections page 10
existed. Highlighting during future instructions to faculty and staff that CLAS Connections consists of departments pages on the landing screen might be beneficial to user and website utility. The current Faculty and Staff page has no links to departments, people, or research topics. Participants are not unified in a preferred method of linking to these things, but some method is desired. One participant preferred having faculty finder search or a jump-to bar, while two participants preferred having links. Further investigation into methods of linking to faculty, staff, and research will accommodate these issues. This could be an opportunity to link to the CLAS Connections department pages and research links (see Figure 7). Figure 7: UNC Charlotte s CLAS Faculty and Staff page versus Faculty and Staff links from competitive university website. Three out of four participants were confused on whether they were searching the entire UNC Charlotte website or only the CLAS website and preferred to be able to manually switch between searching the UNCC website and the CLAS website. Adding a button that would allow users to select their preference would be effective. UNC Charlotte s search function did not yield results in many cases, which led participants to leave the CLAS page and search through Google. Recommendations This section provides a roadmap for the areas of improvement that were identified in the usability tests to advance the function, efficiency, and desirability of the UNC Charlotte CLAS website. Recommendations are prioritized on the basis of how much participants were affected by these issues. Finding a Department s CLAS Connection Page Make the CLAS Connections link look less like a graphic and move to the left-side column of the CLAS landing page. 11
Have a link to department s CLAS Connections page on each department page. Move department links on the CLAS Connections page to the left-side and give them darker color. Finding a Research Topic Provide a link to the CLAS Connections research topics under the research navigation link. Move research links on the CLAS Connections page to the left-side column and give them a darker color (see Figure 3). Logging in and Updating a Profile Make the login link on the CLAS Connections page bigger and either move it to the left-side column or the upper-right portion of the screen where it is located on faculties profiles. Submitting News or Events Make the Thinking Matters and Exchange Online links look less like graphics and move them to the left-side column of the CLAS landing page. Have a We Want Your News! link placed permanently next to the Thinking Matters and Exchange Online links (see Figure 5). Conclusion The results and recommendations of the usability assessments referred to in this report examined whether the UNC Charlotte CLAS website s principles in layout and design proved usable for faculty members, and whether faculty members could perform basic and valuable tasks that would enable them to actively participate in the content of the website. Major links are not currently being seen or recognized by faculty members which has a negative impact on their ability to add or search new knowledge. Admittedly, other factors beyond the scope of this assessment may be impacting on the overall effectiveness of the website and hindering faculty engagement. In summary, the results of this usability test demonstrate that the CLAS website functions well in its overall layout and design. Some of the best practices that were identified include: Column style layout Side navigation Distinct color changes for links when hovered over Rolling graphics 12
However, there remain areas of improvement if the website desires to have more active faculty participation. The greatest needs for consideration include: Create links for department s CLAS Connections page on each department academic page. Move research links on the CLAS Connections page to the left-side column and darken the color. Make the login link on the CLAS Connections page a larger font size and move the link to the left-side column or the upper right-side of CLAS website. Create a We Want Your News! link next to Thinking Matters and Exchange Online. Increase the font size and bold the links that exist within the paragraphs and consider moving the link to the top of the submission paragraph. These recommendations may help faculty members perform basic tasks in a timely manner which will increase the CLAS website s efficiency. By implementing the recommendations, faculty members may quickly learn how to login, find help, and contribute more effectively to the platform. In addition, the exact purpose of the website should be clearly defined to users. 13
Appendices Appendix A: Participant Task Times Task Times by minutes 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Submit news Help page Research Topic Department CLAS page Login CLAS website Blog Participant A Participant B Participant C Participant D 14
Appendix B: Follow-Up Questions and Graph Key: 4 out of 4 3 out of 4 2 out of 4 2 out of 3 Not applicable Preference Participant A Participant B Participant C Participant D Results Column or Paragraph Paragraph Column Column No preference 2 out of 3: Column Column or Tab Column Column Column Tab 3 out of 4: Column Paragraph or Tab Paragraph Tab Paragraph Both Sanserif with serif or sanserif only Hover color or hover shading Faculty info or faculty finder and jump to No preference Sanserif only Sanserif only Sanserif with serif 2 out of 4: Sanserif only Hover color Hover color Hover color Hover color 4 out of 4: Hover color Faculty Both Faculty finder, Faculty info finder and jump to, and jump to keyword finder for research Faculty info Department Department Faculty info 2 out of 4: each links links Faculty info or department links Department links or Faculty Department Department Department 3 out of 4: faculty finder and jump finder and links links links Department to jump to Links Academics grouped or Academics Academics No preference Academics 2 out of 3: academics separated separated grouped grouped Grouped List for academics or List for Both List for List for picture blocks of academics academics academics academics List for academics or List for List for Jump to Both jump to for academics academics academics Block pictures or jump to Block Block pictures Jump to Jump to 2 out of 4: each pictures Rolling graphics or nonrolling Rolling Rolling-if Rolling-if not Rolling-If 4 out of 4: graphics updated too quickly updated Rolling Rolling graphics or Rolling Rolling Selectable Rolling 3 out of 4: selectable graphics Rolling Search with no option or Search Search Search options No options 3 out 4: Options search with options options options Only embedded text and Embedded Embedded No preference Embedded 3 out of 4: pictures or embedded text and text and text and Embedded videos pictures pictures pictures text and pictures Side navigation or top navigation Side navigation Both Side navigation Side navigation 15