Fall 2013 Harvard Library User Survey Summary December 18, 2013 The Discovery Platform Investigation group placed links to a User Survey on the four major Harvard Library web sites (HOLLIS, HOLLIS Classic, E-Research@Harvard Library, and the Harvard Library Portal). The survey included 3 questions and ran from October 25 November 15, 2013. Four hundred thirty-four undergraduates, graduate students, faculty, staff, and other non-harvard researchers responded to the survey (Information about the affiliation and status of respondents is in Appendix A). The User Survey Team (Corinna Baksik, Kathleen Donovan, Renata Kalnins and Emily Singley) offers the following summary. Question 1 provides insight on how users prioritize features that are included in web scale discovery systems. Single search was ranked the highest by the most respondents, and was the also ranked highest by each group when analyzed by respondent status (undergrad, graduate student, faculty, staff or non-harvard). All Respondents 235 62 142 121 41 60 64 25 41 116 100 50 53 89 59 77 93 57 70 119 74 98 94 77 72 Single search Relevancy Save/email/export /tag Mobile Customize for my discipline Very Important Somewhat Important Neutral Less Important Not important Question 2 asks if users prefer a single (blended) or Bento Box style presentation of search results. Two thirds prefer a single, blended list. For the 4% that preferred Grouped another way, there was no clear preferred alternative in the responses. Page 1 of 5
Question 3 asks: What ONE change to library search tools (HOLLIS, HOLLIS Classic, E-Research@Harvard, and the Harvard Library web site) would have the biggest impact on your research? The 340 responses are summarized below. This summary groups responses by theme. It highlights the most commonly occurring themes and begins with those most frequently mentioned. Comments that were unclear or were a single mention of one narrowly-focused concern were excluded from this summary. The summary includes excerpts from respondents comments to illustrate how needs and concerns were articulated. For the quoted material, respondent status is indicated in parenthesis by this code: (UG) Undergrad (G) Graduate student (F) Faculty (S) Staff (O) Non-Harvard Single Search [46] Forty-six respondents requested a more unified search (F), asking us to create one, searchable database for all library resources. (UG) Many specifically asked for the ability to search articles and books simultaneously (S), asking to be able to search for an article by typing directly into the Hollis search bar, rather than by clicking on e-research and then choosing JSTOR, EBSCO, etcetera. (UG) Others asked for better access to articles across disciplines, i.e. sciences and arts. (O) Within the requests for a single search, a few voiced the continued need for direct access to specific resources or formats: But I wouldn't want a full integration all the time because sometimes all that I wish to look for are books, or articles, or images. So I'd want the option not to do an integrated search, too, but to search in separate categories more or less as I can now. (F) A common search theme across most of the single search comments was the desire for a simpler, more intuitive search: One form, one search, one syntax. (F) Search Precision [31] and Keep HOLLIS Classic [29] Thirty one asked for types of search precision currently available only in HOLLIS Classic, including searching or browsing by call number and searching known titles or exact phrases. Nine of these indicated the need to search in non-latin characters or non-english languages: It's grossly unacceptable, and frankly somewhat offensive, that Chinese, Japanese, and Korean characters cannot be used with the "modern" HOLLIS interface. Until this is rectified, not a minute should be spent on any other initiative. It's a disgrace to the institution. (G) Twenty-two respondents urged, pleaded, or admonished us not to get rid of HOLLIS Classic. Another seven just wrote, HOLLIS Classic, presumably indicating its importance to them. The capabilities of Hollis Classic MUST be preserved if the catalogue is to be able to sustain real research. New Hollis makes searching by subject SO much more difficult. It is also not well suited Page 2 of 5
to foreign languague searching by subject. ALL my students, including knowledgable undergraduates, prefer OLD Hollis, and for good reason. (F) Enhanced Description and Content [46] Twenty-five respondents commented about the quality, quantity or usability of metadata. Faculty complained about too many mistakes in names/titles and incorrect, missing or misleading keywords, or the lack of full subject cataloguing for books sent to the depository! Another 13 asked for enhanced content, including more detailed descriptions, more tables of content, and recommendations: Maybe a thing like Amazon's Frequently Bought Together and Customers Who Bought This Item Also Bought, except with library patrons and borrowed." (G) An additional eight asked for more digital content: Have everything (books especially) available online in a digital form. (G) Usability and Navigation [40] Forty users commented on issues related to overly complicated user interfaces or poor navigation: It's difficult to completely specify this but your search engines take much more time than I believe is reasonable or necessary. The navigation is complex and generally requires training. I believe it should be much more intuitive, with perhaps the possibility of mini-tutorials embedded. I would say, overall, that given the richness of the universe of resources available through the University, the searches are unnecessarily tedious and complex. (F) Many commented on poor navigation: My biggest sadness about the current state of our search is that it's just CLUNKY. (UG) Among these, eight complained about the Back button not working in HOLLIS in certain browsers. Several asked for fewer clicks to specific resources, including HOLLIS Classic and the databases and e-journals lists: The drill down into databases can be confusing. (G) No change! [18] Eighteen respondents reported general satisfaction with the status quo, and asked us to keep things as they are now. Please do not make any radical changes. It works fine the way it is. (F) While most of these comments may reflect a predictable aversion to change, this comment is based on experience: I like it the way it is! / Learn from the Boston University library experience: Not a good switch to single-search. (G) Faceting [16] Sixteen respondents asked for ways to limit results sets by format type, online full text, availability, or historical or publication date. Fulfillment [15] Fifteen respondents requested easier access to materials they have discovered. The library s fulfillment services were described as difficult to access (Borrow Direct or WorldCat search) or inconsistently Page 3 of 5
available (Scan & Deliver). Five of these respondents asked to be able to get books delivered to any library. I know that's not what you're asking, but that really would make the biggest difference. (F) Linking to Full Text [12] Twelve respondents asked us to make access to articles online through Harvard clearer, please. (UG) They asked for fewer choices, fewer clicks, and links that go directly to the article. Relevancy [8] Eight respondents commented on poor relevance ranking. One noted, Relevancy ranking always feels arbitrary and opaque to me. (S) Another stated, It would save me a LOT of time if it just came up with the book I need at the top! (G) Miscellaneous (4 6 comments) [45] Six respondents wanted to be able to save searches and article lists. Six made general comments about wanting better/more access to library resources and facilities. Five asked for typo/spelling correction and Did you mean... suggestions for failed searches. Five commented about citation management tools and issues. Five wanted additional online help. Five listed HOLLIS by name, without further comment. Five listed the Library website, without further comment. Four requested discipline-based research options. Four commented on features for mobile device users. Appendix A: Information about Respondents How often do you use library resources? Affiliation: Page 4 of 5
Status: School/Unit: Page 5 of 5