Adventures in Minimal MARC and Bulkimport; Or,

Size: px

Start display at page:

Download "Adventures in Minimal MARC and Bulkimport; Or,"

Chastity Agatha Marsh
5 years ago
Views:

1 Adventures in Minimal MARC and Bulkimport; Or, How to Get OCLC Numbers or URLs or Other Bits of Information into Lots of Bib Records with Minimal Fuss Maggie Rioux Systems Librarian MBLWHOI Library Woods Hole, Mass.

2 What we ll cover What in the world am I talking about? Why would anyone want to do this? Basic principles and overview of process Serious look at how to do it Parting advice from one who s been there

3 You have: Basic Orientation 1. Plenty of perfectly good bib records in your database 2. A long list of record-specific data that needs to be added, each piece to its proper record You don t want to either: 1. Destroy your database or, 2. Individually edit half-a-bazillion records

4 Basic Orientation You do want to: 1. Take your list of data and, using batch procedures, insert the proper new field in each specific record 2. Do this with as little blood, sweat, and (especially) tears as possible

5 Why specifically would I use this? OCLC record reload project Need to load each returned OCLC number into an 035 field in the correct bib record Bulk scanning project Need to load an 856 link to each scanned item (Google books or Internet Archive?) into the correct bib record Or any other data set that needs to be inserted into matching bib records

6 Basic terms Minimal MARC record Bulkimport MarcEdit OCLC Number = 035 field URL = 856 field Marc Leader Encoding_Level = Elvl = Leader/17 Bidirectional merge

7 Step 1: Prepare your incoming data a. Add the (OCoLC) to the front of all the OCLC numbers using this formula in a blank column. (Column A needs to be formatted as text in order to preserve the leading zeros.)

8 Step 1: Prepare your incoming data b. Propagate the formula down the list, then copy and paste special to get the values into a column. c. Lastly, tidy up and export the worksheet as tab-delimited text.

9 Step 2: Make minimal Marc records a. Use MarcEdit Delimited Text Translation Wizard to create the records. You will want the BibID mapped to 887$a (or another non-standard tag below 900) and the OCLC number mapped to 035$a. Save your template to use on later batches of records b. The actual Marc records are then created in an mrk file.

10 Step 2: Make minimal Marc records c. Open the MarcEditor and then open the.mrk file d. In all records, change LDR/17 (Elvl) to X or another unused value so they will fail the quality check in bi-directional merge

11 Step 2: Make minimal Marc records e. Choose Edit subfield data on the tools menu or press F9 f. Click Replace Text to complete the edit g. Compile the file into Marc, saving it as a.mrc file

12 Step 3: Use Bulk import with bidirectional merge to load the data a. Define a Bulk Import Rule and a Duplicate Detection Profile which will cause records to be matched on the incoming 887$a mapped against the existing 001 and also to fail the quality check (more on this later) b. FTP the.mrc file to your ILS server (or use webadmin) c. Use bulk import to load the records, starting with a small test set

13 The Elephant in the room: Let s talk about bi-directional merge How it works: Voyager compares the incoming record with what s already there If the incoming record is better, it overlays the current record, but some parts of the current record are saved If the current record is better, it stays and some parts of the incoming record get added

14 The Elephant in the room: Let s talk about bi-directional merge We want the incoming record to fail, so that the current one stays intact, but our new 035 field gets added This is the reason for the exercise with LDR/17 and X We need a Duplicate Detection Profile and a Bulk Import Rule These can be a little hairy to define, but once you get it right you don t need to worry about it anymore

15 Duplicate Detection Profile Full details are in the handout, but here s a quick look

16 Duplicate Detection Profile Field Definition tab: BIBID code matches against the 887$a

17 Duplicate Detection Profile Quality Hierarchy tab: Needs a line for every possible value of LDR/17 except X (so that your incoming records will fail)

18 Duplicate Detection Profile Merge fields tab: Here s where you tell it to merge the 035$a into the current record

19 Bulk Import Rule The Bulk Import Rule is a parameter in the Bulk Import command line and it calls the Duplicate Detection Profile.

20 Bulk Import Rule The Rules tab is the most important one to get right.

21 Maggie s Guidelines for a Successful Project Read everything through several times These documents and also the Voyager Technical User Guide Work with a small batch at first and don t do anything you can t undo when it doesn t work right the first (or second or third) time Read the documents and manuals while lying down more blood flow to the brain, no mental effort wasted on sitting up

22 Most important: Don t let any cats help you in any way!

23 And if you want to get really carried away with using MarcEdit: Optimized Metadata Repurposing in a Library Using MarcEdit - Presentation at ALA Midwinter 2009 by Sai Deng, Wichita State University

24 Getting Your Returned OCLC numbers into Your Voyager Records Once you have sent your Marc records to OCLC for upload, you will get back a spreadsheet containing two columns: OCLC number and BIBid (which will be labeled submitted 001 field ). Now you need to get the OCLC number into the proper bib record in your catalog as an 035$a. This document will guide you through the process. There are two main steps: first, creating minimal Marc records from the returned spreadsheet and second, loading those records into the online catalog so that they merge with the records already there, rather than replacing them. Part I: Creating Minimal Marc Records from the Spreadsheet 1. The XREF spreadsheet returned from OCLC contains 2 columns: OCLC control # and Submitted 001 Field. Relabel the 2 nd one as BibID for clarity. Add another blank column labeled OCLC. The incoming OCLC column should be formatted as text to preserve any leading zeros. 2. In the OCLC column for the first row, enter the formula = (OCoLC) &$A2 where A2 is the cell in that row containing the OCLC control #. Use of the $ in the formula locks the cell. 3. Once this shows the right value, drag the lower right corner of the cell down the column to copy the formula to all rows. 1

You should now have 2 columns labeled BibID and OCLC. 6.

25 4. Copy and paste special the column values to another column and delete the one with the formulas and the original OCLC control # column. 5. You should now have 2 columns labeled BibID and OCLC. 6. Export the worksheet as tab-delimited text. In order to test the process and also to break up the batch load into manageable pieces, add a new worksheet to the 2

spreadsheet, then copy (or move) a chunk of the data to the new worksheet. Do about 10 rows for the first test chunk and then larger numbers (up to a couple thousand) for succeeding chunks.

26 spreadsheet, then copy (or move) a chunk of the data to the new worksheet. Do about 10 rows for the first test chunk and then larger numbers (up to a couple thousand) for succeeding chunks. Export the new worksheet and use the resulting text file for input in the next step. Try to exclude the column labels from the export. If you include them it will make setup easier, but you will get an extra minimal Marc record which you will need to delete. 7. Use the MarcEdit Delimited Text Translator to convert the text file into minimal Marc records containing only an 035 and an 887 field (or use any other Marc tag that you don t usually find in your records and which is below 900). The OCLC field (field 0) should map to the 035 and the BibID field (field 1) to the other tagged field. Leave indicators and terminal punctuation as is. If you save the specifications as a template, you can use them again without re-definition. Your specification screen should look like this, but with different values in the Field 0 and Field 1 cells: Now use MarcEdit s main program to change all LDR/17 (Elvl) to X (so that they will fail the quality check in bulkimport), 1 save the.mrk file and then compile the result into Marc (.mrc file). To do this choose Edit subfield data on the Tools menu or press F9. Click Replace Text to complete the edit. 1 You need to change the value of this byte to something which is NOT already in your database. To see what values already exist use the following SQL in an Access query: SELECT BIB_TEXT.ENCODING_LEVEL, Count(BIB_TEXT.BIB_ID) AS CountOfBIB_ID FROM BIB_TEXT GROUP BY BIB_TEXT.ENCODING_LEVEL ORDER BY BIB_TEXT.ENCODING_LEVEL; I have used X since it is not a legal value and should not be used in your database. 3

27 8. Use FTP to transfer the.mrc file to your ILS server (or use webadmin for the bulk import). Use bulk import, utilizing the bi-directional merge feature to import/add the data in the minimal Marc records to your database You will need to have both a bulk import rule and a duplicate detection profile defined which cause records to be matched on the incoming 887a mapped against the existing 001 (BIBID) and also to fail the quality check. See Appendix A for parameters on setting this up. 10. I recommend testing the process using the following procedure: a. Copy about 4-10 records to a separate worksheet, export and process them as outlined above. b. Before running the bulk import step, go into cataloging, bring up each of the test records in turn and save them to disk as a backup. c. Run the bulk import then check what happened both in the import log (in /m1/voyager/xxxdb/rpt) and also by looking at the records again in cataloging. d. If something went wrong, you can import your locally-saved records and re-overlay them onto the mistakes. If all is well, delete your local save file. 11. The final result will have your 035$a with the OCLC number. There may possibly be an extra 035$a containing the Voyager BibID (I ve found it only occasionally and am not sure what causes it). If so, this extra field should not present any problems, but as catalogers work with records for other purposes, they can be instructed to delete the extraneous field. 2 To do this from the command line, log into your server as user voyager and cd to /m1/voyager/xxxxdb/sbin. Enter the following command:./pbulkimport f<full path to your input file> -o<operator name> -i<bulk import rule code> An example is:./pbulkimport f/export/home/voyager/incoming_data.mrc omrioux -iaddoclc 4

28 Appendix A: Bulk Import Rule and Duplicate Detection Profile for Bi- Directional Merge In the Voyager System Administration module, you need to define a Bibliographic Duplicate Detection Profile for Bi-Directional Merge which will cause the incoming minimal Marc records to fail the quality check, thus preserving the record already in the database, but adding the incoming fields. The record matching should occur on BIBID in the existing record matched to 887$a in the incoming minimal record. You also need a Bluk Import Rule, which is invoked in the bulk import command line and calls your Duplicate Detection Profile. Screenshots follow: Part I: Duplicate Detection Profile Basic Profile Screen Field Definition Tab Select BIBID from the Available Indexes and fill in the Field/Subfield Override to match to the 887$a in the incoming record. 1

29 Quality Hierarchy Tab Enter a line for every value of Leader/17 in your bibliographic database Make sure you enter a space character to represent blank values see line one of the grid below don t just delete the *. Also, make sure to add a line for the _ character as this can look like a blank character in displays and queries. Having done this, changing Leader/17 to X (which is not a valid value for this byte and which should not appear in your database) in your incoming minimal Marc records will cause them to fail quality check and not overlay your existing records. 2

30 Merge Fields Tab These are the fields in the incoming (minimal) records which will be added to your existing records. You want to enter only 035 so that only the OCLC number will be added. Part II: Bulk Import Rule Only the first two tabs (Rule Name and Rules) are relevant. Basic Rule Name Screen 3

31 Rules Tab This is the most important screen to get right. Auth Dup Profile is irrelevant. The Bib Dup Profile is the one you created in Part I. Make sure that Load Bib/Auth Only is checked in the lower half of the screen (it should be the default). Also, Leave OPAC Suppress Unchanged should be checked. 4

OhioLINK e-book refresh project

OhioLINK e-book refresh project At The University of Akron Mike Monaco Coordinator, Cataloging Services The University of Akron OhioLINK refresh : Why? 1. Are there records for every title? 2. Record quality