Gauging the User: TESTING THE UX IN AN INSTITUTIONAL REPOSITORY AFTER AN ACADEMIC LIBRARY AND A PUBLISHER COLLABORATE

Transcription

1 Gauging the User: TESTING THE UX IN AN INSTITUTIONAL REPOSITORY AFTER AN ACADEMIC LIBRARY AND A PUBLISHER COLLABORATE

2 Laura Spears, PhD Assessment Librarian Chelsea Dinsmore, MLIS Chair, Digital Production Services

3 Acknowledgement Robert Philips, PhD, an IT specialist in our Information Technology department and an integral member of our project team, provided a great deal of support for the UX testing as well as in the development of this presentation.

4 Overview The Project Goals Implementation Assessment Goals Methods Findings

5 Project Goals The Elsevier Collection project addresses shared library and publisher needs for integrated systems to meet common goals including: improving compliance; sharing scholarly journal articles via institutional repository; and, increasing awareness of and showcasing UF scholarship. 5

6 Pilot Project Phase 1 Increase comprehensiveness of coverage for UF authored content published in Elsevier journals through the IR@UF Provide subscribers with access to the best available (published) version through the IR@UF Aggregate published articles with other IR@UF content Allow users to return to their IR search results when reading final articles from ScienceDirect

7 Pilot Implementation 30,000 articles by UF authors from 1949 forward Metadata, abstracts, and (in Phase II) full text for indexing in the Links to ScienceDirect for access Includes both open access and subscription articles Full access for users with subscriptions; alternative access for other users (Phase II) 95% of ScienceDirect usage is by users with subscriptions* 9

8 Pilot Project Phase 2 Usability testing

9 Assessment Goals Determine what the user s experience is like when navigating the interface of the IR@UF- Elsevier Collection, and is intended to inform practical decision- making. What labels and text are most helpful in navigating to successful article access within the IR@UF- Elsevier Collection; and, What labels and text are most helpful in navigating through the inter- library loan process once a user determines that the article must be obtained through another source? How satisfied are users with the Sobek CM interface navigation of the UF authored Elsevier article search results as presented in the IR@UF?

10 Assessment Method Usability on the Fly task- scenario protocol (Daly, 2016) Effective navigation Efficient response Engaging design Error tolerant operation Ease of learning

11 Assessment Method Designed a 17- step scenario based instrument Tested on library staff familiar with the interface Filed for IRB approval Tested 15 individuals (Sauro, 2016) Interviewed each subject with 10 post- scenario questions

12 The Interface

13 Institutional Repository

14 Brief View

15 View of Elsevier articles in Search results can either be directly accessible, or require log in, or are not available to user 36

16 View of Elsevier article in 37

17 Subscriber Access

18 Visitor Access ILL and CHORUS options to be added here in the future 6

19 Findings: Task Completion 1.2 Task Completion Rate by Role (n=12) TASK 1 TASK 2 TASK 3 TASK 4 TASK 6 TASK 7 TASK 8 Librarians Faculty Students Admin Total Avg Librarians Faculty Students Admin Total Avg

20 Findings: Preferences of Labels and Text You Have Access (n=12) Check Access (n=12) 5- Other 1 Other 4 4- View full text 6 3- Get full text at journal 1 Explore Access 0 2- Access final article 0 Verify Access 0 1- You have access 3 Check Access

21 Findings: Preferences of Labels and Text Label preference of users with access to the final article? Label preference of users who must check if they have access to the final article. Text preference that most clearly indicates to users they have access to the full text final version? Publisher Version- You Journal Have Access Title Science Direct Other 40% 30% 10% 20% The label with the journal title Other Science Publisher Version Direct Version 50% 30% 10% 10% View Full Text You Have Access Other 70% 20% 10%

22 Findings: Preferences for finding alternate access Label preference of user who needs to check the library catalog or request a document from inter- library loan ILL and CHORUS options to be added here in the future Find 80% 80% Check Access 10% Find Full Text Elsewhere 10% 6

23 Sentiment Analysis Sentiment Analysis by Task (n=12) Comments by Sentiment (n=11) TASK 1 TASK 2 TASK 3 TASK 4 TASK 5 TASK 6 TASK 7 TASK Negative Positive Neutral Confused Negative Positive Neutral Confused Librarians Faculty Students

24 Pilot Project Phase 2 (other progress) Offer full text searching through the IR@UF, with links to the published article on ScienceDirect (tested and implemented) Provide an access option for users without a subscription: viewing of post- embargo Author Accepted Manuscripts (2013 forward, in final testing) Research on open access publishing by UF authors and use of Elsevier metadata for other University purposes, including meeting compliance needs

25 References Daly, E. (2016, 30 March). On- The- Fly usability testing: From once a semester to once a month. Presentation to Association of Southeast Research Libraries webinar. Duke University Libraries. Available at content/uploads/2016/03/daly- ASERL_Webinar_website_Usability.pdf Sauro, J. (2016). The challenges and opportunities of measuring the user experience. Journal of Usability Studies, 12(1), 1-7. Russell, J., Wise, A., Dinsmore, C.S., Spears, L.I., Phillips, R.V. & Taylor, L.N.(2016). Academic library and publisher collaboration: Utilizing an institutional repository to maximize the visibility and impact of articles by university authors. Collaborative Librarianship, 8(2), Avaialbe at

26 Questions? Chelsea Dinsmore, MLIS Laura Spears, PhD