Automating Authority Work

Size: px

Start display at page:

Download "Automating Authority Work"

Suzanna Roberts
5 years ago
Views:

1 Mike Monaco Coordinator, Cataloging Services May 14, 2018 Automating Authority Work

2 Automating authority work, or, Be your own authority control vendor Ohio Valley Group of Technical Services Librarians 2018 Conference, May 13-15, 2018 Hesburgh Libraries The University of Notre Dame South Bend, Indiana Mike Monaco Coordinator, Cataloging Services The University of Akron

3 Who are you? John Carroll University ( ) Part-time AV cataloger Akron-Summit County Public Library ( ) Substitute librarian Cleveland Public Library ( ) Catalog librarian The University of Akron (2016- ) Coordinator, Cataloging Services

4 The University of Akron Libraries University Libraries (Separate Units) Bierce Library Wayne College Library Science & Technology Library Akron Law Library Archival Services Center for the History of Psychology

5 Authority control at UA 1995 migration and vendor (BNA) supplied one-time authority processing Local authority work put on hold in expectation contracting with a vendor which never happened Authority work resumed early 2000s Full authority control for tangible items only Shift to batches of e-resources over time made authority work for batches overwhelming 2013: Budget 80:20 electronic:tangible 2018: ratio is about 95:5

6 What this is NOT about Automated authority control within the ILS Working with an authority control vendor

7 What this IS Grabbing the low-hanging fruit for batches of records When traditional authority work is not practical (the item is not in-hand or headings reports are too vast to address individually)

8 Wouldn t it be nice if... The Headings used for the first time report could export a list of the headings, and we could batch search OCLC for records?

9 The tool box MarcEdit OCLC Connexion Client Excel (or other program for sorting textual lists) pgadmin (or similar for a SQL query, III/Sierra only) A rudimentary grasp of Regular Expressions EditPad (or similar RegEx-compatible text editor: Google Sheets, EmEditor)

10 The process Before loading, correct variant headings (with MarcEdit) After loading, extract headings from report (with SQL query or ILS s output) Separate names and subjects (in a spreadsheet or text editor) Remove extraneous data (with RegEx-capable editor) Batch search for authority records (in Connexion Client) Load authority records

11 Validate Headings MarcEdit can check name and subject fields against LC authorities in the Linked Data Service, and automatically correct headings that match a variant ( Use for ) heading*. *NB: The process is imperfect!

12 Because this is an extra step, we ve been comparing record sets from various vendors to determine which ones really benefit.

13 Selected Vendor Loads (March 2017-March 2018) Record Source Alexander Street Press Records per load Invalid headings per load Variants changed per load Invalid heading: Record ratio Variant: Record ratio EBSCO Films on Demand Kanopy Proquest EBC World Share

14 III Sierra

15 SQL query of Headings used for the first time report

16 Headings used for the first time Hundreds or even thousands of entries after batch loads...

19 SQL query*

20 Results...

21 In Excel... Be sure to import as Unicode (UTF-8) if your ILS is encoding characters as Unicode rather than MARC8!

22 Sort the terms Sorting A-Z arranges the headings by field group tag and MARC tag (a=names, b=other names, d=subject) So a100-b730 : names used as names d600-d630 : names used as subjects d650- : subjects

23 Notice You can t feed this raw data into a batch search in Connexion Client

24 In EditPad (or other RegEx-enabled editor) Strip out MARC tags, delimiters, punctuation, etc.

25 Find/replace using RegEx (.*\ a) (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.) (\ e.* \ 4.* \ 0.*) (\ x.* \ v.* \ z.*) (\. \ $) (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near )

26 Names (.*\ a) Everything before a

$(\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\.$

27 (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.) AACR2 abbreviations b. d. fl. ca.

28 (\ e.* \ 4.* \ 0.*) Relator terms, URIs

29 (\. \ $) Any remaining delimiters and subfield codes

$(; :  \? and or &c\. & in an, the for on so with to by be that \.$

30 (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near ) Punctuation, operators, and stopwords that foil OCLC searches

31 Names as subjects (\ x.* \ v.* \ z.*) Subdivisions

32 Converting SQL output to batch searchable text file with RegEx

33 Name headings

34 (.*\ a)

35 (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.)

36 (\ e.* \ 4.* \ 0.*)

37 (\. \ $)

38 (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near )

39 Names (can be skipped) (.*\ a) (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.) (\ e.* \ 4.* \ 0.*) (\ x.* \ v.* \ z.*) (\. \ $) (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near )

40 Names as subjects (.*\ a) (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.) (\ e.* \ 4.* \ 0.*) (\ x.* \ v.* \ z.*) (\. \ $) (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near )

41 Topical subjects (can be skipped) (.*\ a) (\ db\. ca\. \ db\. \ d\. ca\. \ dd\. \ dca\. -ca\. \ dfl\. ca\. \ dfl\.) (\ e.* \ 4.* \ 0.*) (\ x.* \ v.* \ z.*) (\. \ $) (; :  \? and or &c\. & in an, the for on so with to by be that \.{3} near )

42 Sirsi/Dynix Symphony

43 List unauthorized tags report

46 Slightly different procedure to clean these up Open.txt file in editor Delete header of report Find/Replace to delete page headers ( Tags With UNAUTHORIZED Headings / Produced on Sat Jul 1 17:00: ) Separate name and topical headings RegEx to remove other data

47 So far so good... (.*\ a)

48 Uh oh... (\ e.* \ 4.* \ 0.*) Misses?UNAUTHORIZED by itself. Only captures it if preceded by e 4 0

49 (\ e.* \ 4.* \ 0.* \ \?.*) \ \?.* captures? followed by anything

50 Problems

51 Spaces

52 Scraps Portions of?unauthorized that wrapped to new line

53 Asterisks Output changes any diacritics to them

54 Line breaks Name/Title headings are especially likely to get broken up. Here, the delimiter was even separated from the subfield code t

$FIND: \* REPLACE:[nothing] to delete asterisks Use EditPad s Extras to delete blank lines, duplicate$

55 Workaround FIND: \* REPLACE:[nothing] to delete asterisks Use EditPad s Extras to delete blank lines, duplicate lines, etc. Depending on the number of items, you might close up split lines by hand.

56 Searching in batches

57 Searching a batch of terms in Connexion Client

58 Batch searching Use default index settings nw: for names/titles su: for topics/geographic terms Maximum number of matches to download: 1 (Tools>Options>Batch)

59 Batch searching NOTE: Your local save file has a maximum capacity of 10,000 records, so don t search more than that many strings!

60 Successful name searches (of 1941 entries)

61 Names, names as subjects, and subjects III requires name headings that are to be used as subjects to be loaded separately from name headings to be used as names! SirsiDynix does not have this issue.

62 A four month test Total headings extracted Hits in batch search Success rate (ARs found for heading) Names 36,244 21, % Names as Subjects 3, % Subjects 29,147 1, % I m very pleased with hit rate on names!

63 A four month test Total headings extracted Hits in batch search Success rate (ARs found for heading) Names 36,244 21, % Names as Subjects 3, % Subjects 29,147 1, % Main issue: Name/Title headings often not established

64 A four month test Total headings extracted Unique hits in batch search Success rate (ARs found for heading) Names 36,244 21, % Names as Subjects 3, % Subjects 29,147 1, % Main issues: Sierra treats subdivided subject headings as single headings, inflating report (3.4 should correct this issue!) Music headings (instruments, Arranged) often valid but not established in an AR

65 Is it worth it? Typically this process (excluding Validate Headings in MarcEdit) takes less than an hour and resolved 36% of unauthorized headings -- averaging over a thousand ARs a week. Several known issues in Sierra made me place this project on hold until they are fixed however. So consider the number of new headings you normally see in a report, the quality of your incoming bib records (can the headings be authorized?) and the capability of your ILS to use authority records effectively

66 Questions? If you decide to test this process at your institution or discover any refinements, please let me know!

67 Useful tools Help with regular expressions: Regular Expressions RegEx enabled text editors: Editpad Lite EmEditor pgadmin free software: This presentation:

OhioLINK e-book refresh project

OhioLINK e-book refresh project At The University of Akron Mike Monaco Coordinator, Cataloging Services The University of Akron OhioLINK refresh : Why? 1. Are there records for every title? 2. Record quality