Reference rot analysis of PhD theses in Spectrum, Concordia s Institutional Repository

Transcription

1 Reference rot analysis of PhD theses in Spectrum, Concordia s Institutional Repository 14 th Annual Research Forum April 29, 2016 Kathleen Botter, Systems Librarian, Concordia University Library Mia Massicotte, Systems Librarian, Concordia University Library

2 Long-term access to the scholarly record One of our major challenges is preservation text must be stable citations and references must be retrievable a digital library must remain a repository of fact in addition to being a dissemination point of information -- Dimitrova and Bugeja. Consider the source: predictors of online citation permanence in communication journals. Portal: Libraries and the Academy. 6(3), p.270

3 Why do we care? research builds upon previous work of others; verifiable evidence referencing is a manifestation of academic scholarship quality and thoroughness substantiated via attribution of sources current as well as future access to born digital is in jeopardy unless preservation is taking place

4 Why electronic theses and dissertations (ETDs)? electronic deposit is mandated by parent institution should remain stable over time library maintains stewardship of thesis collection scholarly content which is born-digital

5 Hard to pin down digital different kinds of digital content websites come and go page content is in flux digital scholarly content is susceptible to loss collections are in jeopardy unless preservation is taking place

6 Where is preservation taking place? In his 2015 piece on Preserving the born-digital record, James Neal bluntly states: WE ARE IN TROUBLE. * Preserving the born-digital record: many more questions than answers. American Libraries, May 28,

7 Peter Burnhill, Standing on the digits of giants: research data, preservation and innovation. ALPSP/DPC, London, UK, 8 March

8

9 Reference Rot and Law Harvard team studied link permanence in Harvard law and policy-related publications ( ) 70% of links suffer reference rot database of 555 links U.S. Supreme Court opinions 50% of links suffer reference rot* *Zittrain J., Albert K., & Lessig L. (2014). Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations. Legal Information Management 14(2), doi: /S

10 Law library community takes action Harvard study prompted the creation of Perma.cc (2013) shared digital archiving service maintained and used by network of law libraries Hiberlink Project ( ) two-year funded project to study reference rot

11 What is Reference Rot? Term coined in 2013 by Hiberlink Team Two characteristics: Link Rot + Content Drift Link Rot = 404 Page Not Found Content Drift = change in page content over time

12 Reference Rot in STM Landmark study (2014) determined 1 in 5 STM articles suffers from reference rot science, technology and medical articles, million articles collected 3 sources: arxiv, Elsevier, Pubmed Central 1.8 million articles with open web references analyzed: of those, 7/10 suffer reference rot* *Klein M., Van de Sompel H., Sanderson R., Shankar H., Balakireva L., Zhou K., et al. (2014). Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot. PLoS ONE 9(12), e doi: /journal.pone

13 Research Questions What percentage of links in Concordia s ETD collection are affected by reference rot? Which disciplines exhibit more reference rot than others? What can be done to mitigate the problem?

14 Spectrum Research Repository Concordia University s institutional repository scope limited to PhD dissertations started with first mandatory deposit year, Spring years, Spring 2011-Fall 2015, 719 total dissertations

15 What to look for Using the Hiberlink typology, documents (dissertations) can be immune, healthy or infected: Immune: dissertation contains no links Healthy: contains links, all of which are active Infected: one or more links cannot be resolved

16 Methodology

17 Sequence for Extracting and Testing Links 1. download and convert pdfs to xml 2. use regular expression on each converted file to find links 3. manually verify (and fix) links 4. use curl to get http status code for each link output: original URL, final URL, http status code

18 Selecting a Regular Expression How to identify a link? Search for known top level domains (TLD) e.g..com,.edu)? Search for only well-formed web links e.g. start with Gruber v2 aka Daring Fireball intended to match any URLs, including mailto:foo@example.com, x-whatever://foo, etc. (

19 Gruber v2 Regular Expression (?i)\b((?:[a-z][\w-]+:(?:/{1,3} [a-z0-9%]) www\d{0,3}[.] [a-z0-9.\-]+[.][az]{2,4}/)(?:[^\s()<>]+ \(([^\s()<>] + (\([^\s()<>]+\)))*\)) +(?:\(([^\s()<>]+ (\([^\s()<>]+\)))*\) [^\s`!()\[\]{};:'".,<>? ]))

20 Challenges not all pdfs could be converted to xml inconsistent formatting of links in dissertations ellipses ( ) in long links typos, and less obvious mistakes missing.edu, :, //

21 Preliminary Results 719 PhD dissertations 11,438 links

22 719 Total PhDs ( ) 5% 35 3% 20 Embargoed 46% % 160 Not Converted Immune 24% 172 Healthy Infected

23 Link Distribution by HTTP status code (11,438 links) 1% % 1,915 1% 76 0% 1 4% 507 0% 3 0s - Empty response 100s - Informational 200s - Successful 77% 8, s - Redirected 400s - Client Error 500s - Server Error 900s - Request denied

24 Healthy Links / By Year 8,834 (77%) of 11,438 total links healthy % 1,249 72% 1, % 2,017 76% 2, % 1, Healthy Unhealthy

25 Total PhDs (719) by Discipline 28 4% % % JMSB (Business) Science Fine Arts Eng Comp Sci Arts 86 12% 45 6%

26 Total links (11,438) by Discipline 3,260 28% 5,946 52% JMSB (Business) Science Fine Arts Eng Comp Sci Arts 1,728 15% 294 3% 210 2%

27 Total Link Health / by Discipline JMSB (210) 77% 2,528 79% 74% 75% 1, Science (294) Fine Arts (1,728) Eng Comp Sci (3,260) 76% 4,556 1,390 Arts (5,946) Healthy Unhealthy

28 Surprise! 3,143 DOIs found 68% (2,146) DOIs appeared in last two years examined 42% (1,322) DOIs appeared in last year alone (2015)

29 Next steps: Content Drift in progress: sampling healthy links assessing Content Drift in healthy links visit link and compare to mementos

30 Early evidence of Content Drift evidence of custom 404 in link rot results original URL: /cp_gfk_ march de_la_bd_-_39eme dition_de_la_fibd.pdf final URL: http status code: 200

31 Mitigating Reference Rot Mementos digital snapshot in time of a webpage examples include: Internet Archive s Wayback Machine Archive-It archive.today incidental archiving

32 Mitigation at the source browser plugins (e.g. Zotero) that can create mementos referencing mementos in the citation proposed addition of 2 HTML elements: <a href= data-versionurl= data-versiondate= > better editorial review of link formatting avoid shortened URLs (bit.ly)

33 Institution / Library s Role Should mementos be part of document deposit process? Who creates the memento? Who owns / guarantees the mementos existence? copyright issues public vs. institutional memento archives national initiatives

34 References. Dimitrova,D. and M. Bugeja. Consider the source: predictors of online citation permanence in communication journals. Portal: Libraries and the Academy. 6(3), p.270. Neal,J. Preserving the born-digital record: many more questions than answers. American Libraries, May 28, Burnhill, P. Standing on the digits of giants: research data, preservation and innovation. Presentation at ALPSP/DPC, London, UK, 8 March /95/ensuring-the-integrity-continuity-of-our-record-of-scholarship jpg?cb= Zittrain J., Albert K., & Lessig L. (2014). Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations. Legal Information Management 14(2), doi: /S Perma:cc Klein M., Van de Sompel H., Sanderson R., Shankar H., Balakireva L., Zhou K., et al. (2014). Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot. PLoS ONE 9(12), e doi: /journal.pone Gruber, J. Liberal, Accurate Regex Pattern for Matching Web URLs. Chrome Memento Time Travel plugin for Zotero

35 Thank you Questions?

36 Kathleen Botter, Systems Librarian Concordia University Library Mia Massicotte, Systems Librarian Concordia University Library