Calisphere UI Testing Findings and Recommendations

Calisphere UI Testing Findings and Recommendations Chico High School, 14-15 September 2005 Prepared for: Calisphere Redesign Team Author: Jane Lee Contributor: Felicia Poe Last modified: 25 October 2005 http://www.cdlib.org/inside/assess/evaluation_activities/docs/2005/calispherechicoreport_sept2005.pdf

Table of Contents CALISPHERE UI TESTING FINDINGS AND RECOMMENDATIONS...1 TABLE OF CONTENTS...2 FIGURES AND TABLES...2 INTRODUCTION...3 PURPOSE OF ASSESSMENT...3 TARGET AUDIENCE...3 USABILITY ASSESSMENT DESIGN...3 DATA COLLECTION METHODOLOGY...3 UI FINDINGS AND RECOMMENDATIONS...4 GENERAL FINDINGS...14 AMAZING. SO WHEN IS THIS GOING TO BE UP?...14 DESCRIPTIVE CONTENT...14 IMAGE SELECTION...14 DESIRED INFORMATION ABOUT IMAGES...14 OUTPUT ACTIONS...15 TOPICS A-Z...15 VOCABULARY...15 BASKET...15 APPENDIX: TEST OBJECTIVES AND TASKS...16 Figures and Tables FIGURE 1: CALISPHERE HOMEPAGE...4 FIGURE 2: FEATURED CLUSTERS PAGE...6 FIGURE 3: WWII CLUSTER PAGE...7 FIGURE 4: THEMED COLLECTION VERSION 1...8 FIGURE 5: THEMED COLLECTION VERSION 2...10 FIGURE 6: VERSION 2 IMAGE PREVIEW PAGE...11 TABLE 1: USER-REPORTED REASONS FOR PREFERENCES...13 Page 2 of 16

Introduction Purpose of Assessment This assessment consisted of a first-round usability test of the user interface for Calisphere s themed collections. The test utilized an HTML prototype, and participants began on the Calisphere homepage. The purpose of this assessment was to identify weaknesses of the user interface design early in the design process. Specifically, the assessment addressed questions surrounding navigation, object viewing, printing, labels and terminology, and presentation of metadata for decision-making. This assessment took place on 14-15 September 2005 in Chico, CA. Target Audience For this round of assessment, Peter Milbury, Calisphere Teacher Advisory Board member and Chico High School Librarian, assisted the assessment team by recruiting and scheduling participants, who were experienced teachers and student teachers from Chico High School in Chico, California. Usability Assessment Design For this assessment, the assessment team conducted seven, task-based user interface tests onsite at Chico High School. Each 45-minute session consisted of 1-2 participants, a facilitator (Felicia Poe), and two observers who took notes (Jane Lee and Rosalie Lack). Task-based User Testing Each participant was greeted by the facilitator and made to feel as comfortable as possible. The facilitator explained the purpose of the test. Participants were assured that the system was being tested, not them. The facilitator summarized test procedures and instructed participants on the thinking aloud protocol. At the end of the introduction, the facilitator told the participants about their right to stop testing at any time and asked them to sign consent forms. For the first part of the test participants were asked to complete a series of tasks to the best of their ability. Then they discussed their experiences with the facilitator, and the facilitator debriefed them. Participants received a $40 gift certificate for their efforts, and the facilitator thanked them at the end of the session. Observation As the participant completed the tasks during the task-based user testing, the observers took notes on the steps taken by the participant and any signs of frustration or satisfaction from the participant. Post-test Interview After the task-based portion of the test, the facilitator asked participants to share their overall comments about their experience and the system. The facilitator asked about any specific problems encountered during the test. Data Collection Methodology During task-based user tests, data was collected using the thinking aloud protocol. Data collected include the following: 1. The steps taken by the participant for each task 2. The indications of frustration or satisfaction from the participant 3. The opinions of usability and aesthetics of the system expressed by the participants Page 3 of 16

UI Findings and Recommendations Figure 1: Calisphere Homepage 1. Calisphere s UC pedigree is important to teachers. Teachers reported that it gives Calisphere credibility. They will trust the information on the site because it s UC. As one teacher stated, It's not some bizarre collection from some guy who spends too much time at his computer." They teach their students to be wary of information they get on the Internet. They look at the site s URL. If it s a.edu domain, they are more likely to trust the site. Some users did not realize that Calisphere was affiliated with the University of California because of the size and location of the logo. The logo did not appear on one user s screen because she did not scroll all the way to the top of the page. Recommendation: Increase the font size of University of California and place it in a prominent position near the Calisphere logo. 2. Some users reported initially not being able to tell what the site is about. One stated that the tagline, a world of digital resources, did not help much. Recommendation: Homepage needs to convey the aboutness of the site more clearly and readily. Page 4 of 16

3. The What s Here section gives users the impression that Calisphere contains a variety of primary sources. It does not indicate that Calisphere offers primarily images. "It says primary sources, but I haven't seen any documents or diaries." Users are assuming that Calisphere is choosing the best materials for teaching, and they appreciate not having to go fishing for information in the big, bad world. Recommendation: Improve description of what Calisphere is and what it contains. We need to make it clear that the entire site may contain information that is useful to teachers, but that there is a special section especially for teachers. Along those lines, the relationship between the Selections from Calisphere area and the materials for teachers description needs to be explicit. 4. Users appreciated the direct statement that Calisphere offers "excellent materials especially selected for teachers." 5. Some users did not realize that the cluster headings are hyperlinks. One noted that too much color might impede recognition of what s a link. Recommendation: Make link treatment identifiable and consistent. 6. There is a label mismatch in the Selections from Calisphere section. Furthermore, the current see all topics link could be confused with the see all topics link in the Topics A-Z section. Recommendation: Change see all topics to see all selections or just see all. 7. Users had difficulty seeing the more link. Recommendation: Make links consistent and ensure that they contrast sufficiently with background color. 8. Users commented on the attractiveness of the homepage. Their eyes were drawn immediately to the triptych. Users noted that the Japanese Prints image indicated that the site had prints in addition to documents and that it was multicultural. Some, however, reported that the three images gave the impression that Calisphere was limited to those 3 topics. One user thought that each image would lead offsite. Another was not sure why those three images were chosen. Recommendation: Give this section a label and some kind of visual treatment in order to give users a sense that these images/collections are only a sample of what s in Calisphere. Page 5 of 16

Figure 2: Featured Clusters Page General comments Some users liked having all topics listed on a single page and noted that the images are visually pleasing. One user thought this page was unnecessary because of the dropdowns available on the homepage. Page 6 of 16

Figure 3: WWII Cluster Page 9. The options for all of the words and for exact phrase were confusing to some. One user did not see the need for both options. Another did not see the need for for exact phrase, since quotation marks serve the same purpose. Some users were not sure what this box searched. Note: The search options differed between this page and the homepage. Also the Search label looks like it is part of the top level navigation because of its placement. Recommendation: Reconsider the need for having search box options. If the decision is made to offer options, make sure they are consistent throughout the site. Separate search area from top navigation links. Move Go button to the right of the text box. 10. One user reported being unsure if these topics were the same as the topics listed in the homepage dropdown menus because the labels are different. On the same note, he was not sure if the embedded text links were the same as the topic links because they use different phrases. This user thought that having both embedded links and topics links was confusing and redundant. Recommendation: Remove embedded text links. Make topic link labels and dropdown menu labels consistent. Page 7 of 16

Figure 4: Themed Collection Version 1 11. All users used or reported using breadcrumbs to navigate websites. Those who did not use breadcrumbs during the test did so because they did not see them. When the facilitator pointed out the breadcrumbs, these users remarked that the font was too small. One stated, "These are key features for maneuvering around the site." If they're not big enough or visually appealing enough, students are going to miss it. Recommendation: Increase font size of breadcrumbs. Page 8 of 16

12. One user did not realize that clicking on a thumbnail changed the slideshow preview image. Others commented that the relationship between the preview image and its thumbnail should be made more explicit so that it s clear that one may control the preview image with the thumbnails. Recommendation: Mark the thumbnail of the current slideshow preview image with a highlighting border that distinguishes it from the rest of the thumbnails. 13. Some users were confused about this dropdown menu s function. One thought its options led to items within that page. In general, this dropdown did not seem to add value to the interface. Some users did not use or notice it at all. Recommendation: Remove this dropdown menu. 14. When asked how they would expect to get a larger version of an image, half of the participants said that they would click the slideshow preview image. Some tried clicking the preview image without prompting. One user said that she would click the print view of this image link to view a larger image. One said that she expected to see a hyperlink to a larger version. Recommendation: Users seemed to desire the ability to view larger versions of images. Provide a link to a larger version or hyperlink a larger version to the slideshow preview image. 15. Some users believed that the Also see links led to more themed collections. Recommendation: Users will be confused and disoriented if they are taken to a search results page when they were expecting another themed collection. Re-examine strategies for helping users discover additional related resources. 16. "We are so standards focused. This is fabulous." Because they are trying to align their curriculum with content standards, users appreciated the listing of applicable content standards. They liked that they were written out and not just reference numbers. The listing of content standards gave users the sense that not only was Calisphere relevant to them, it was created for them. Somebody has done work here that I don't have to do." New teachers especially appreciated this section, since they were less familiar with content standards. When asked about the placement of this section, users responded that its current placement at the bottom of the descriptive text section was fine. General comments Some users commented that the font size is too small. Several users commented that the browser s back button is not useful when browsing through a lot of images. Some used the browser history menu because they don t want to hit back five times. One noted that if you use the back button, "oftentimes you lose where you were." Recommendation: Provide links for controlling font size. For navigation, provide alternatives to the back button. Page 9 of 16

Figure 5: Themed Collection Version 2 General comments Five of eight participants preferred version 2 to version 1. Clicking on a thumbnail on this page leads to a slideshow preview page, e.g. Figure 6. Page 10 of 16

Figure 6: Version 2 Image Preview Page 17. Several users tried to click on Slideshow, expecting it to produce an automated slideshow. Recommendation: Remove the text: Slideshow. 18. Users commented that it was difficult to keep track of where they were in the slideshow. One noticed that after clicking next, the cursor doesn t always end up in the same place on the page, i.e. hovering over next, which is annoying if you want to click through pictures. Two additional comments about the navigation links were that the font was too small and that there should be icons as well as or instead of text links. One user said that he usually does not use slideshow navigation ( back and next ) because he doesn t look at things in order. Recommendation: Provide [current #] of [total #] text near navigation links. Increase font size of navigation links. Consider adding navigation icons. Consider changing back to previous. Make sure that navigation elements are in same absolute position on every slideshow page. Page 11 of 16

19. Users had difficulty finding button options, e.g. for printing, because they appear below the fold. Half of the participants believed that clicking print view of this image would immediately send the image to the printer. One user initially thought that print view with this description would print the image with a short, descriptive paragraph about the image. Users were not sure what clicking details about this image would give them. More than half believed that details meant information about the content of the image, i.e. what s happening in the picture, not technical details. Recommendation: Many websites provide printable version links in the upper right corner of the content section. Consider moving printable version links. Change wording of links to printable version. To users, print is a verb and print view is a command. Reconsider the use of the terms description and details, or provide explanations or examples of what these are. 20. User noted that the size of thumbnails is good, especially since they're bigger than others she's seen. She further commented that American Memory does not have thumbnails. Users would prefer thumbnails on the left and slideshow preview on the right. Recommendation: Place thumbnails on the left and slideshow preview on the right. Page 12 of 16

Table 1: User-reported Reasons for Preferences Version 1 Version 2 Pros Three of eight users preferred version 1 s layout because everything is on a single page Slideshow preview is available immediately Thumbnails in upper left corner make site seem image dominant Cons Thumbnails are too small and there is no identifying text. Should not sacrifice larger thumbnail in order to put everything on one page. Narrower text column means page will get longer and users will have to scroll more It s a lot of stuff. Too messy. Slideshow preview image size may be too small for group shots and landscapes Five of eight users preferred version 2. More prominent position of text encourages reading Text on left, thumbnails on right promote a better flow Larger thumbnails and wider text column Easier to bounce between text and thumbnails Second page enhances image viewing experience. Text is not necessary if focus is on images. Layout is more academic. Using thumbnail to go to larger image and possibly to description is more logical. Since text does not refer directly to image, it does not need to appear with larger slideshow preview If on slideshow preview page, might forget that there is descriptive content available on the first page Might be harder to combine excerpts from text with images Forced to bring up a second page in order to see larger slideshow preview images Note: Images are not drawn to scale. Page 13 of 16

General Findings Amazing. So when is this going to be up? Teachers were excited about the themed collections and asked when they would be available. They appreciated that someone had taken the time and energy to create something especially for them. Teachers felt that themed collections would meet their needs and provide valuable content for use in lessons that were based on California content standards. Descriptive Content Most users found the descriptive text useful. Some said that they would copy excerpts to use as notes for themselves. They would take the information and weave it into their lesson plans. Some users reported that if they are looking for images, they don t pay attention to text. If there is a lot of scrolling required, they are less likely to read it. Some users would consider showing the text to their students. Users recognized that text was a general overview of the theme. Some expressed a desire to know why those particular photos were chosen for each theme. Users stated that three to five paragraphs were sufficient for an overview. Because the text is an overview and does not contain specific details, one user believed that it was suited more for teachers than for students. The Why these images? opening made the site seem very image centric. The relationship between the text and the thumbnails was not clear to one user. Image Selection Users were drawn more toward images of people, especially those with larger subjects in the foreground. Most users felt that 10-15 images were ideal. There were comments that the number of photographs in American Memory was overwhelming and that many of them were repetitious. Users viewed thumbnails to get a sense of the images, and they used the slideshow preview images to confirm their choices. Desired Information About Images For use in handouts: o Date o Source o Caption/Title o Photographer o Excerpts from descriptive content Size of image file for knowing how long it will take to download How photo was used. The context for the photo. o What was the occasion? o Was this used for a poster? o What's the context for the photo? Page 14 of 16

Output Actions Email materials to self, so user can work on things at home When asked to print an image, one user went directly to the browser s print function. Saving images o When preparing for lesson, create a folder on the computer and right-click and save to that folder. Give files and folders descriptive names. o Use captions to name files. o One user would never Download all images. Instead, right-click and save each image directly to the appropriate folder on computer. o o One user likes ability to Save all images. One user expressed the desire to know the size (dimensions) of the image he is saving. Reusing images o Assemble text and image and print on an overhead transparency o Would not print because of cost. Would only print if he wanted to create a transparency. o Display on a TV or computer in classroom o Copy into a Word document for easy modifications, e.g. addition of text, and printing o Insert into presentation software, such as PowerPoint or Keynote o Likes to include caption, but sometimes doesn t for photo analysis exercises. Uses worksheets from national archives. Topics A-Z More of a browsing kind of feature Some users expressed concern that there would be a lot of scrolling. Use for known item lookup Likes topic a-z, but would rather use search Users expect that a-z items will be similar to themed collections. Users don t understand difference between topics a-z and themed collections. Would Great Depression themed collection be included in topics a-z? Vocabulary "Topics" sounds bigger than "selections. " Selections sounds like it's been assembled for me." Topics a-z more specific, maybe events or people Cluster same as selection. Both imply someone has put together something for me. Collection is bigger than selection. Selections are big topics. Prefer collections from calisphere to selections because collection gives sense that things have been gathered. Selections implies a sampling from website, not entire collection Basket For shopping gather first, decide what s most interesting later A way to save things to your computer Unsure about function. Maybe save for later? User thinks it works across sessions. For saving things you ve collected within a session Basket makes you feel like you re shopping. Do you have to pay for images at the end? What s the point if you can already save and email? Might give mistaken impression that images are being saved to computer. Page 15 of 16

Appendix: Test Objectives and Tasks Version II 1. Area: Navigation Objective: Determine whether or not users are able to navigate from the Calisphere homepage to a specific topic. Task: You are looking for material to use for a lesson on the San Francisco General Strike. Browse the Great Depression cluster for an interesting image. 2. Area: Navigation across clusters Objective: Determine whether or not users are able to navigate from a topic in one cluster to a topic in a different cluster. Task: You would like to show your class a Rosie the Riveter type image for your next lesson about World War II. Find an appropriate image using Calisphere. 3. Area: Navigation within a cluster Objective: Determine whether or not users are able to navigate from one topic to another topic within the same cluster. Task: Your next lesson examines Mexican immigration programs during World War II. Find two images to use for class. 4. Area: Object viewing Objective: Determine whether or not users are able to easily compare objects in Version I. Determine whether users will notice and use the printable version links. Task: Time constraints force you to choose only one of your images. Reexamine both and select one to print. 5Q. Area: Printing Objective: Determine whether or not users understand what each print option does. Question: What do you think each of these print links does? Version I 6. Area: Navigation Objective: Determine whether or not users are able to navigate to a topic from the Calisphere homepage. Task: You would like to begin your next lesson about women in World War II with a few photographs. Find appropriate images using Calisphere. 7. Area: Object viewing Objective: Determine whether or not users are able to easily compare objects in Version I. Determine whether users will notice and use the printable version links. Task: Time constraints force you to choose only one of your images. Re-examine the ones you chose and select one to print. Discussion 8Q. Area: Object Viewing Objective: Determine users preferences for object viewing. Review the two different ways to view images. Question: Do you prefer Version I or Version II for looking at the actual images? Why? 9Q. Area: Printing (Optional) Objective: Determine whether or not users are aware of the various printing options that are available. Determine which they prefer. Question: Of all the available information about each image, which pieces would you want to give to your students or use yourself? Page 16 of 16