Improving the Reported Use and Impact of Institutional Repositories

Transcription

1 Improving the Reported Use and Impact of Institutional Repositories Kenning Arlitsch, Dean Patrick OBrien, Semantic Web Research Director Montana State University Library Coalition for Networked Information Washington, D.C. December 13, 2016

2 Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and Library Services (IMLS) Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital Repositories

3 Agenda A New Reporting Model Accuracy of Analytics Reporting Tools Page Tagging Log file RAMP New Prototype Web Service for IR reporting

4 A New Reporting Model Page Type Definition Examples Citable Content Downloads Item Summary Ancillary Non-HTML scholarly content that may be formally cited in the research process HTML pages to help user decide to download the full publication HTML pages that provide general information or navigation Publication (.pdf) Presentation (.ppt) Data Sets (.csv) Title & Abstract Item Metadata Search Results Browse by Author Statistics

5

6

7

8 Two Classes of Web Analytics 1 Analytics Service (SaaS) Page Tagging {JavaScript} HTML 2 Log Files

9 Google Analytics used by most academic libraries Tested 279 academic library websites ARL DLF OCLC-RLP 90% US libraries contained Google tracking code

10 Page Tagging does not track non- HTML Citable Content Downloads Non-HTML Page Tagging HTML

11 Show Google Scholar direct link graphic

12 Web Analytics Accuracy Risks Risks Analytics Method Area Types Page Tagging Log Files Visits Low High OverCount Downloads Low High Page Views Low Low Visits Medium Medium UnderCount Downloads High Low Page Views Low Low

13 Typical Page Tagging Data Sources Page Type Pages Analytics Events Citable Content Downloads - Item Summary - Ancillary -

14 Four IR During 2016 Spring Semester January 5 to May 17 (n = 134 days) Study Participant IR Pla(orm URL Montana State University ScholarWorks DSpace scholarworks.montana.edu McMaster University MacSphere DSpace macsphere.mcmaster.ca University of New Mexico LoboVault DSpace repository.unm.edu University of Utah USpace CONTENTdm uspace.utah.edu

15 Page Tagging does not track non- HTML Citable Content Downloads Search Analytics Does! Non-HTML

16 Montana Method Includes Google Search Console Page Type Analytics Search Console Pages Events Search Analytics Citable Content Downloads - Item Summary - - Ancillary - -

17 All four IR using Google Analytics CCD Tracking Improvement Search Analytics Console Page Type Pages Events Search Analytics 2,000% - 26, ,933 Ancillary 201, Citable Content Downloads Item Summary 284,

18 Most IR Activity Unreported by Google Most IR Activity is Unreported by Google Analytics Analytics. 01/05/16-05/17/16 (days=134) 42.5% 57.5% 47.9% 52.1% 46.2% 53.8% 54.3% 45.7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Reported Invisible

19 Most IR activity consists of Citable Content ~51% to 63% of Citable IR Activity is Unreported by Google Analytics 01/05/16-05/17/16 (days=134) Downloads 63% 20% 17% 52% 20% 28% 54% 27% 19% 51% 35% 14% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Citable Content Downloads Item Sumary PV Ancillary PV

20 Montana Method Challenges Missing non-google direct link CCD events Yahoo Bing Search Analytics limits time and access Moving 90 day window Granular data = programing skills to access API

21 RAMP - Repository Analytics and Metrics Portal Cloud Web service Currently accessible and free During grant period (December 2017) No training or configuration required Consistent method and terminology Benchmarking across time and organization Request mixterj@oclc.org

22 Two New Publications Published: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. Undercounting File Downloads from Institutional Repositories, Journal of Library Administration, vol. 56, no. 7, 2016 Forthcoming: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. RAMP: Repository Analytics and Metrics Portal: A Prototype Web Service that Accurately Counts Item Downloads from Institutional Repositories, Submitted to Library Hi Tech, expected publication early 2017

23 Undercounting Research Team Montana State University Kenning Arlitsch, Patrick OBrien, Semantic Web Research Leila Sterman, Scholarly Communication Susan Borda, Digital Technologies OCLC Research Jeff Mixter, Software University of New Mexico Jonathan Wheeler, Data Curation Librarian