The Website Teaching Thoughts Usability Report By Jon Morris Original November 13 th, 2009 Modified on November 21 st 2009
Table of Contents 1. Introduction... 3 2. Executive Summary...3-4 3. Methodology...5-6 Sessions...5-9 Participants...9-11 Evaluation Tasks/Scenarios...11-12 4 Results...12-14 Task Completion Success Rate...12-13 Task Ratings...13-14 Overall Metrics...14-16 4.6.1 Overall Ratings...14-17 4.6.2 Likes, Dislikes, Participant Recommendations...17-18 5 Recommendations... 18 6 Conclusion... 18 2
1. Introduction http://cosmo.marlboro.edu/amorris This is a website created for the class Web Design 1 for the Marlboro College Graduate Program. A survey is embedded within the website as a usability test. The purpose of this usability test/website is to show that the student (me) has mastered the basic skills of website design such as embedding pictures, video, audio, slide shows presentations, uploading documents, as well as website design itself. The test was conducted by Jon Morris using Google Documents as the Survey form. The overall responses were positive. The website was easy to navigate, but needed to be a little more professional. The videos were entertaining, but need a little more work. The session data and graphs are included below. 2. Executive Summary The usability test was created in Google Documents as a survey form. This survey form allows participants to easily express their thoughts about the usability of the website. These surveys were conducted in three ways: Online, in person, and over the phone. The online survey was used in all three instances. The users surveyed were two IT professionals and one IT student (a former MAT graduate). The average length of time to complete the survey was approximately 20 minutes. Overall, I thought the responses were valid and several minor things were pointed out. The big ones were editing the videos for sound and transitions. Get rid of the um s and ah s. Explain the Change Management Plan better so people will understand the purpose of the plan. It was also indicated the site should be more professional. It really helped to have Reggie point out some of the difficulties as he is a former MAT student and understands what the purpose of the course is and what Caleb is looking for in a website. 1. Make site more professional 2. Improve Introduction video 3. Remove Audio 4. Explain purpose of the Change Management Plan. 5. None of the three participants liked the blog format as the homepage. Please view the graphs for demographics information. 3
4
3 Methodology Sessions Individuals were recruited from family and friends. The first attempts were done online and over the phone. The first telephone session was with my mother. The results are not included here. It was a beta session to make sure that everything was working. It took twenty minutes to get all the questions answered. The results were fine, if a bit personal. The second survey was done on the phone with a friend that is an IT Professional. The test took 15 minutes. His big thing was making the site more professional. The third test was done completely online. The participant 5
was a graduate of the MAT program. He had a better idea of the purpose of the survey. He offered valuable information to improve the website. The fourth test was from the Directory of Technology of my school. This survey was done in person. The test took 15 minutes. His biggest complaint was of the format of the site itself. He did not like the blogging aspect of the site. None of the participants like the blog. To fix this, I ll move the blog away from the home page. Please view graphs below to view website usability. 6
In addition, please view the survey s written responses. What did you like best about the website? Reggie Martell: The visuals are really engaging. There's something about comics They invite reading much more than text, and do a nice job drawing in the reader. It's an idea I definitely plan on ripping off. The mix of media also makes for fun browsing. The best part is that everything is interesting and useful. It invites browsing, and rewards visitors with useful information. Only 1 in a billion websites are actually useful, so this is like A++++ territory. John Kendzie: The videos were the most entertaining. Overall, good job. Rick Mallon: The site is visually appealing. 7
What did you like least about the website? Reggie Martell: I kinda hate the blog format, in general, but that's more a product of spending a trillion hours in the mid 1990s learning how to build static web pages, than any real issue with the design. Website creation should require a person to walk uphill through snow, both ways. Blogs are too easy. I could play the role of wheel smasher, or fire douser in that Luddite cartoon. John Kendzie: Dump the audio introduction. It is redundant. You already have video with audio, so what is the point of having more audio... unless it is a course requirement. Rick Mallon: There is a lack of navigation links. The front page bog does not allow user to avoid the linear view. How can this website be improved? Reggie Martell: There are a few "ahs" and "ums" in the audio. It's not enough to be truly distracting, but if you want to obsess about audio, muting out those bits in a program like Audacity can make for super-slick audio. Once you're familiar with Audacity, you could edit a 3 minute clip, in as few as 5 minutes. I'm sure you can do all this in Aviary too. If not, the people over at aviary should be shot. Maybe add a brief introduction to the capstone page. Visitors might not understand that a capstone is your thesis project. John Kendzie: I didn't like your website theme. But this is more a personal preference. It should be more professional if you are looking to make this an educational website. Rick Mallon: See above comment. If this site had more pages to it, it would become increasingly difficult to locate desired information. Can you find my resume? Only one pasted in the link. Reggie Martell: Found it quick-smart. No problem. John Kendzie: Yes, I found the resume quickly. Rick Mallon: http://cosmo.marlboro.edu/amorris/about/ Very easy to complete. Please comment on the Halloween Two Video. Was it easy to find? Did you like the video? Reggie Martell: The video was easy enough to find now, but it will be interesting to try and find in in a year, after a year's worth of content has accumulated. I liked the video. Nit-picky feedback would include asking for a little more light, and a little clearer sound, but both were good enough to get the job done. John Kendzie: Too many transitions. Recut the movie. Nice seeing the kids. Rick Mallon: It was interesting to watch. Harder to find because I had to remember to go back into the blog. The video had interesting effects. I liked it. Please note that I am not a video person. Please comment about my Introduction Video. Was it easy to find? 8
7B Reggie Martell: The intro video was really cool. Of course, Youtube necessitates tiny video so if there is a way to draw attention to whichever sample you're discussing, at a given time, that might be helpful, but that's just nit-picky too. John Kendzie: Easy to find. You need to fix some of the sound. Rick Mallon: Yes easy to locate. It was easier to watch one video with representational sections than to watch 4-5 separate videos. Good job. See Google Documents survey Teaching Thoughts for complete questionnaire at this address: HH Uhttp://spreadsheets.google.com/viewform?hl=en&formkey=dFJ6SEFoe U5qNE1BamVfM0NldVgyclE6MAUH Participants Reggie Martell: 11/13/2009 10:21:51 John Kendzie: 11/13/2009 1:21:51 Rick Mallon: 11/16/2009 12:08:48 Four participants took the survey; one was removed due to family bias. 9
10
Role Evaluation Tasks/Scenarios The tasks were created by Jon Morris. The point of the tasks was to find out how easy it was to navigate the site. 1. Find Resume 2. Find the Introduction Video 3. Find the Halloween Two Video 11
Can you find my resume? Only one participant pasted in the requested link. Reggie Martell: Found it quick-smart. No problem. John Kendzie: Yes, I found the resume quickly. Rick Mallon: http://cosmo.marlboro.edu/amorris/about/ Very easy to complete. Please comment on the Halloween Two Video. Was it easy to find? Did you like the video? Reggie Martell: The video was easy enough to find now, but it will be interesting to try and find in in a year, after a year's worth of content has accumulated. I liked the video. Nit-picky feedback would include asking for a little more light, and a little clearer sound, but both were good enough to get the job done. John Kendzie: Too many transitions. Recut the movie. Nice seeing the kids. Rick Mallon: It was interesting to watch. Harder to find because I had to remember to go back into the blog. The video had interesting effects. I liked it. Please note that I am not a video person. Please comment about my Introduction Video. Was it easy to find? Reggie Martell: The intro video was really cool. Of course, www.youtube.com necessitates tiny video so if there is a way to draw attention to whichever sample you're discussing, at a given time, that might be helpful, but that's just nit-picky too. John Kendzie: Easy to find. You need to fix some of the sound. Rick Mallon: Yes easy to locate. It was easier to watch one video with representational sections than to watch 4-5 separate videos. Good job. 4 Results Task Completion Success Rate Everyone was easily able to find the videos, and resume information. This was done without prompting. As the survey was a good guide to help focus the participants. Task Completion Rates Participant Task 1 Task 2 Task 3 1 2 3 12
Participant Task 1 Task 2 Task 3 Success 3 3 3 - - - - Completion Rates 100% 100% 100% Task Ratings After the completion of each task, participants rated the ease or difficult of completing the task for three factors: It was easy to find my way to this information from the homepage. One person indicated that the Halloween Two video was not in a good spot. As I was searching for this information, I was able to keep track of where I was in the website. All participants agreed that the website was easy to use, but one person wanted more specific webpage titles. I was able to accurately predict which section of the website contained this information. All participants agreed, they knew where they were on the website. One person indicated that the Halloween Two Video was not in a good spot. Ease in Finding Information Everyone was able to navigate the website only one participant was not able to easily find the information. This was due to the blog factor on the front page. None of the participants like the blogging format on the front page. Keeping Track of Location in Site Everyone was able to navigate the website only one participant was not able to easily find the information. This was due to the blog factor on the front page. None of the participants like the blogging format on the front page. Predicting Information Section All participants were able to predict the section in which the information would be with the exception of the Halloween Two Video. It was in the blogging area. Test 1 Mean Task Ratings & Percent Agree Task Ease Finding Info Location in Site Predict Section 1 Find items within 100% 100% 66% the website. *Percent Agree (%) = Agree & Strongly Agree Responses combined Overall 13
Overall Metrics 4.6.1 Overall Ratings After task session completion, participants rated the site for eight overall measures. These measures include: Ease of use All participants felt the site was fairly easy to use, but more than one didn t like the blogging format. Frequency of use Only one of the participants would use the website frequently. The other two were more not into the frequent use of the website. Difficulty of keeping track of where they were in the site Two of the participants were able to keep track of where they were easily. 14
How quickly most people would learn to use the site Two of the participants felt most people would be able to navigate the site easily. Getting information quickly Two of the participants felt they could get information quickly. Homepage s content facilities exploration This was not a specific question that was asked. From the interviews, I get the feeling that overall that the homepage was easy to navigate, but participants did not like the blog on the homepage. They would have preferred the bio s and resume on the homepage. 15
Relevancy of site content This question goes back to focusing my website. I spent a little time getting the website more focused prior to sending out my survey. I had mostly general thoughts on the website, but I ve now focused on teaching thoughts. Site organization Two of the participants felt the site was well organized. Post-Task Overall Questionnaire Strongly Disagree Disagree Neutral Agree Strongly Agree Percent Agree Thought Website was easy to use 1 2 100% Would use website frequently 1 1 1 66% Found it difficult to keep track of where they were in 0 0 0 0 0% website Thought most people would learn to use website 0 1 2 100% quickly Can get information quickly 0 0 0 0 0% Homepage s content makes me want to explore 0 0 1 2 100% site Site s content would keep me 1 1 1 66% coming back Website is well organized 0 1 2 100% *Percent Agree (%) = Agree & Strongly Agree Responses combined 16
4.6.2 Likes, Dislikes, Participant Recommendations Upon completion of the tasks, participants provided feedback for what they liked most and least about the website, and recommendations for improving the website. Liked Most The following comments capture what the participants liked most: Reggie Martell: The visuals are really engaging. There's something about comics They invite reading much more than text, and do a nice job drawing in the reader. It's an idea I definitely plan on ripping off. The mix of media also makes for fun browsing. The best part is that everything is interesting and useful. It invites browsing, and rewards visitors with useful information. Only 1 in a billion websites are actually useful, so this is like A++++ territory. John Kendzie: The videos were the most entertaining. Overall, good job. Rick Mallon: The site is visually appealing. Liked Least The following comments capture what the participants liked the least: Reggie Martell: I kinda hate the blog format, in general, but that's more a product of spending a trillion hours in the mid 1990s learning how to build static web pages, than any real issue with the design. Website creation should require a person to walk uphill through snow, both ways. Blogs are too easy. I could play the role of wheel smasher, or fire douser in that Luddite cartoon. John Kendzie: Dump the audio introduction. It is redundant. You already have video with audio, so what is the point of having more audio... unless it is a course requirement. Rick Mallon: There is a lack of navigation links. The front page bog does not allow user to avoid the linear view. Recommendations for Improvement Reggie Martell: There are a few "ahs" and "ums" in the audio. It's not enough to be truly distracting, but if you want to obsess about audio, muting out those bits in a program like Audacity can make for super-slick audio. Once you're familiar with Audacity, you could edit a 3 minute clip, in as few as 5 minutes. I'm sure you can do all this in Aviary too. If not, the people over at aviary should be shot. Maybe add a brief introduction to the capstone page. Visitors might not understand that a capstone is your thesis project. John Kendzie: I didn't like your website theme. But this is more a personal preference. It should be more professional if you are looking to make this an educational website. Rick Mallon: See above comment. If this site had more pages to it, it would become increasingly difficult to locate desired information. 17
5 Recommendations Change Justification Severity Make site more professional Improve introduction video Remove audio Explain purpose of the Change Management Plan None of the three participants liked the blog format Changing the header graphics of the website and focusing the information of the site. Change the introductions video by modifying the sound, too many ums and ahs. Audio assignment is part of the course requirements and will not be removed. Explain the purpose of the change management plan and why it s on the website. Change the homepage from a blog to the resume and bio page. High High Not Removed High High 6 Conclusion The participants felt the website needed the following improvements: Make the website more professional and focused on teaching. Change the homepage to bios and resume because the blogging style was too long and confusing and the page header graphics were not professional. Explain the Change Management Plan, and modify the video and audio page. They like the survey to improve the website and felt it was a great tool to use. These changes will be implemented within days (with the exception of the audio removal, as it is a required assignment). 18