Gauging the User: TESTING THE UX IN AN INSTITUTIONAL REPOSITORY AFTER AN ACADEMIC LIBRARY AND A PUBLISHER COLLABORATE

Gauging the User: TESTING THE UX IN AN INSTITUTIONAL REPOSITORY AFTER AN ACADEMIC LIBRARY AND A PUBLISHER COLLABORATE

Laura Spears, PhD Assessment Librarian laura.spears@ufl.edu Chelsea Dinsmore, MLIS Chair, Digital Production Services chelseaz@ufl.edu

Acknowledgement Robert Philips, PhD, an IT specialist in our Information Technology department and an integral member of our project team, provided a great deal of support for the UX testing as well as in the development of this presentation.

Overview The Project Goals Implementation Assessment Goals Methods Findings

Project Goals The IR@UF Elsevier Collection project addresses shared library and publisher needs for integrated systems to meet common goals including: improving compliance; sharing scholarly journal articles via institutional repository; and, increasing awareness of and showcasing UF scholarship. 5

Pilot Project Phase 1 Increase comprehensiveness of coverage for UF authored content published in Elsevier journals through the IR@UF Provide subscribers with access to the best available (published) version through the IR@UF Aggregate published articles with other IR@UF content Allow users to return to their IR search results when reading final articles from ScienceDirect

Pilot Implementation 30,000 articles by UF authors from 1949 forward Metadata, abstracts, and (in Phase II) full text for indexing in the IR@UF Links to ScienceDirect for access Includes both open access and subscription articles Full access for users with subscriptions; alternative access for other users (Phase II) 95% of ScienceDirect usage is by users with subscriptions* 9

Pilot Project Phase 2 Usability testing

Assessment Goals Determine what the user s experience is like when navigating the interface of the IR@UF- Elsevier Collection, and is intended to inform practical decision- making. What labels and text are most helpful in navigating to successful article access within the IR@UF- Elsevier Collection; and, What labels and text are most helpful in navigating through the inter- library loan process once a user determines that the article must be obtained through another source? How satisfied are users with the Sobek CM interface navigation of the UF authored Elsevier article search results as presented in the IR@UF?

Assessment Method Usability on the Fly task- scenario protocol (Daly, 2016) Effective navigation Efficient response Engaging design Error tolerant operation Ease of learning

Assessment Method Designed a 17- step scenario based instrument Tested on library staff familiar with the interface Filed for IRB approval Tested 15 individuals (Sauro, 2016) Interviewed each subject with 10 post- scenario questions

The Interface http://ufdc.ufl.edu/

Institutional Repository http://ufdc.ufl.edu/ufirg

Brief View

View of Elsevier articles in IR@UF Search results can either be directly accessible, or require log in, or are not available to user 36

View of Elsevier article in IR@UF 37

Subscriber Access

Visitor Access ILL and CHORUS options to be added here in the future 6

Findings: Task Completion 1.2 Task Completion Rate by Role (n=12) 1 0.8 0.6 0.4 0.2 0 TASK 1 TASK 2 TASK 3 TASK 4 TASK 6 TASK 7 TASK 8 Librarians 1 1 1 0.6 0.8 0.4 0.4 Faculty 1 1 1 1 0.67 0.67 0 Students 1 1 1 0.67 0.33 0.33 0.33 Admin 1 1 1 1 1 1 1 Total Avg 1 1 1 0.75 0.67 0.50 0.33 Librarians Faculty Students Admin Total Avg

Findings: Preferences of Labels and Text You Have Access (n=12) Check Access (n=12) 5- Other 1 Other 4 4- View full text 6 3- Get full text at journal 1 Explore Access 0 2- Access final article 0 Verify Access 0 1- You have access 3 Check Access 8 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9

Findings: Preferences of Labels and Text Label preference of users with access to the final article? Label preference of users who must check if they have access to the final article. Text preference that most clearly indicates to users they have access to the full text final version? Publisher Version- You Journal Have Access Title Science Direct Other 40% 30% 10% 20% The label with the journal title Other Science Publisher Version Direct Version 50% 30% 10% 10% View Full Text You Have Access Other 70% 20% 10%

Findings: Preferences for finding alternate access Label preference of user who needs to check the library catalog or request a document from inter- library loan ILL and CHORUS options to be added here in the future Find it @UF- 80% 80% Check Access 10% Find Full Text Elsewhere 10% 6

Sentiment Analysis Sentiment Analysis by Task (n=12) Comments by Sentiment (n=11) 10 9 8 7 6 5 4 3 2 1 0 TASK 1 TASK 2 TASK 3 TASK 4 TASK 5 TASK 6 TASK 7 TASK 8 9 8 7 6 5 4 3 2 1 0 Negative Positive Neutral Confused Negative Positive Neutral Confused Librarians Faculty Students

Pilot Project Phase 2 (other progress) Offer full text searching through the IR@UF, with links to the published article on ScienceDirect (tested and implemented) Provide an access option for users without a subscription: viewing of post- embargo Author Accepted Manuscripts (2013 forward, in final testing) Research on open access publishing by UF authors and use of Elsevier metadata for other University purposes, including meeting compliance needs

References Daly, E. (2016, 30 March). On- The- Fly usability testing: From once a semester to once a month. Presentation to Association of Southeast Research Libraries webinar. Duke University Libraries. Available at http://www.aserl.org/wp- content/uploads/2016/03/daly- ASERL_Webinar_website_Usability.pdf Sauro, J. (2016). The challenges and opportunities of measuring the user experience. Journal of Usability Studies, 12(1), 1-7. Russell, J., Wise, A., Dinsmore, C.S., Spears, L.I., Phillips, R.V. & Taylor, L.N.(2016). Academic library and publisher collaboration: Utilizing an institutional repository to maximize the visibility and impact of articles by university authors. Collaborative Librarianship, 8(2), Avaialbe at http://digitalcommons.du.edu/collaborativelibrarianship/vol8/iss2/4

Questions? Chelsea Dinsmore, MLIS chelseaz@ufl.edu Laura Spears, PhD laura.spears@ufl.edu