Getting Them There: A Small-Scale Usability Test of a University Library Website

Transcription

1 City University of New York (CUNY) CUNY Academic Works CUNY Library Assessment Conference Conference Proceedings Getting Them There: A Small-Scale Usability Test of a University Library Website Jason C. Simon Fitchburg State University Follow this and additional works at: Part of the Library and Information Science Commons Recommended Citation Simon, Jason C., "Getting Them There: A Small-Scale Usability Test of a University Library Website" (2014). CUNY Academic Works. This is brought to you for free and open access by the Conference Proceedings at CUNY Academic Works. It has been accepted for inclusion in CUNY Library Assessment Conference by an authorized administrator of CUNY Academic Works. For more information, please contact AcademicWorks@cuny.edu.

2 Getting Them There: A Small-Scale Usability Test of a University Library Website Jason C. Simon Technology & Serials Librarian Fitchburg State University jsimon6@fitchburgstate.edu

3 Prior Situation 2011: Old text-heavy site needed a redesign University introduces new CMS requiring all university sites to migrate in a short time-frame. Quickly discovered that all advanced code/includes would be stripped Some discussion amongst librarians over layout of content, but time was limited.

4 Old old site

5 Migration to CMS

6 Need to Redesign Consensus existed in the library that there were problems Needed to do some testing before making changes Identified areas of focus: What the library provides: Services Resources What Students want/need: How do I...?

7 Testing Models Some standardized question areas were identified based on prior research (Letnikova, 2008) Format Type: Form for user to fill out Pros: no user anxiety Cons: problem with accuracy of data (users may treat it like test, and think they have to get it right ) Observation Pros: Can observe actions Can provide clear instructions Cons: user might be nervous with observer IRB Approval needed

8 Test Model: Observation Test subject seated at computer Monitor shows mouse movement of the test subject Two observers enables one to catch things that another might miss.

9 Sample Who to use Student Workers Rationale: point of study is to identify initial problems of layout. These students are theoretically knowledgeable. If they have trouble clear indication of problems Sample size Wanted 10. Used 7. While larger studies may be more granular (VandeCreek, 2005) Small numbers can work for this type of test. Testing one user is better than none.

10 Nielsen model where N is the total number of usability problems in the design and L is the proportion of usability problems discovered while testing a single user. The typical value of L is 31%, averaged across a large number of projects we studied. Plotting the curve for L =31% gives the following result: (Nielsen, 2000).

11 Instrument Test required no name, a number was given to each test subject. Each observer had the associated number on their result form Instructions Read aloud to subject Demographics User fills out Content: Action statements, not questions. Results observed and recorded by observers User-told to indicate when done with each action (or told to move on if it was clear that a solution was not being found)

12 Sample Questions Find if the library has a copy of Huckleberry Finn by Mark Twain. Find whether it is checked out or not. Find instructions on how to locate it on the shelves How can you find scholarly articles on humor, child development and education? Assume you have been unable to find anything satisfactory. Find a guide or a person to help you in your research Find out if the library has the item referenced in the citation below: Dozois, D. A., Martin, R. A., & Faulkner, B. (2013). Early maladaptive schemas, styles of humor and aggression. Humor: International Journal Of Humor Research, 26(1), doi: /humor

13 Analysis Since the sample was small, mostly qualitative Able to group some answers together Could tie together some results based on various demographic characteristics Not looking for statistical data Looking for any cases of confusion

14 Results Some Expected Some not No student spent more than 20 minutes No subject wanted to spend more than 3 minutes, most less Minor problems with Testing Instrument Instructional wording, briefer, more explicit Convert to Action Statements Sampling issues

15 Results (cont) No trouble finding catalog (note: sample were student workers who should have been familiar with this) Keyword search seemed okay Trouble with title or author searching Need for less library language e.g. Interlibrary Loan Research Help Concept of Research Guides seemed alien Needed clearer access to citation help

16 Results (cont) Need for content which did not exist: Tutorials; guides Students did not understand what Research Guides were More explanatory terminology New navigation systems Overall navigation troubles Artifacts of CMS (e.g. left navigation) Users unwilling to go more than one click deep Need to have better system of content promotion

17 Current Site

18 Current site (2)

19 Current site (2a)

20 Current site (3)

21 Conclusions/Discussion Potential problems with redesign Evidence students still have trouble understanding some concepts No significant change in usage stats Problems may be due to Usage may be tied to instruction Trouble with overall site (outside of library control) but Tool can be re-used Larger sample, different groups: Undergrads Grad Students Faculty

22 References Letnikova, G. (2008). Developing a standardized list of questions for the usability testing of an academic library web site. Journal of Web Librarianship, 2(2), doi: / Nielsen, J Why you only need to test with 5 users. Nielsen-Norman Group. (Last Accessed May 19, 2014) VandeCreek, L. M. (2005). Usability analysis of northern illinois university libraries' website: A case study. OCLC Systems & Services, 21(3), doi: /