Introduction to MarcEdit iskills Workshop Series. University of Toronto. Faculty of Information. Winter 2018. Instructor: R. Antonio Muñoz Gómez. 1 Learning outcomes By the end of this workshop, students will be able to: 1. Understand some of the basic concepts of Machine Readable Cataloguing (MARC) 2. Use MarcEdit to perform simple tasks, such as: a. Obtain bibliographic records using the Z39.50 client b. Apply batch transformations to bibliographic records, like adding, deleting, and/or modifying fields, subfields, and indicators 3. Understand basic principles of regular expressions (regex) and their usefulness for data cleanup Preliminary notes MarcEdit is free software programmed by Terry Reese, librarian at the Ohio State University. You may download a copy from: http://marcedit.reeset.net/ In this workshop, we will use version 6.3 of the software, which was the latest stable version at the time that this workshop and handout were put together. Since then, a new version (MarcEdit 7) has been released. The skills that you will learn in this workshop are perfectly compatible with the newest version of the software. Obtaining MARC records In the workplace you may work with MARC record files that may come from various sources, including: Your own library catalogue Vendor records Records obtained from other libraries using the Z39.50 protocol MarcEdit Z39.50 client In order to obtain records from other institutions, we first need to configure our Z39.50 client, adding the database(s) that we want to query. The Z39.50 Target Directory (http://irspy.indexdata.com/) has the names, addresses, ports, and other technical information required to query library databases and obtain MARC records. 1 This workshop and handout were made possible in part thanks to the collaboration of Sarah Gorman and Jordan Pedersen, first-year students at the Faculty of Information, University of Toronto, and TALint students in metadata and electronic resources at the University of Toronto Libraries.
Introduction to MarcEdit 2 NOTE: Some databases may require authorization credentials for access or additional functionality. Configuring the Z39.50 Client Step 1 : Find a database to query Go to the Z39.50 Target Directory (http://irspy.indexdata.com/) and use the Find a target option to find a library whose database you would like to query. Some examples include: the University of Toronto, University of Alberta, Toronto Public Library. Step 2 : In MarcEdit > Z39.50/SRU Client > Modify Databases > Add Database Step 3 : Copy-paste the relevant information in the corresponding fields: Name Host Database Port Syntax Step 4 : Check Show Database Step 5 : Save Exercise 1 : Adding a database to the Z39.50 client Repeat the above process to add the Toronto Public Library database to your list. Name: Toronto Public Library Host: symphony.torontopubliclibrary.ca Database: unicorn Port: 2200 Syntax: usmarc Exercise 2 : Querying databases to obtain records For this example, we will use the ISBN of a book as our search parameter. Step 1 : In MarcEdit > Z39.50/SRU Client > Search Mode Step 2 : Enter your query and search parameters Search: [type the ISBN for your book] Select the ISBN option from the menu Select the database(s) you want to query Check Retrieve Unicode and select UTF8 character set
Introduction to MarcEdit 3 Step 3 : Click on the green arrow If your search was successful, you should see one or more records in the Results section. Double-click on a record to preview and inspect it. Once you have chosen a record to download, select the preferred folder for download and save the file with the.mrc extension You can repeat this process and either append new records to the same file, or create new files. NOTE: The.mrk extension is used for files that you only intend to use in MarcEdit. The.mrc extension is used for files that will be loaded into your library system. MarcEdit supports both extensions, but only.mrc will work when loading records into your library system. MARC basics MARC stands for MAchine Readable Cataloguing, and it is currently the standard most widely used in libraries to structure and share bibliographic information. At present, the Library of Congress s Bibframe is a likely/potential candidate to replace MARC. MARC is used in combination with other metadata standards and data models, such as the Anglo American Cataloguing Rules, Second Edition (AACR2), Resource Description and Access (RDA), and the Library Reference Model (LRM) MARC records consist of a series of text strings or fields. Each field is identified by a three-digit combination. Fields with a value larger than 008 can further have two indicators (digits) and alphanumeric subfields (varying in number depending on each field). 245 14 $athe Little Mermaid /$cby Hans Christian Andersen Where: 245 = Field 1 = First Indicator 4 = Second Indicator $a = Subfield $c = Subfield Detailed information about each field, indicators, and subfields, is available at: http://www.loc.gov/marc/bibliographic/
Introduction to MarcEdit 4 245 Title Statement (NR) MARC 21 Bibliographic Full October 2008 First Indicator Second Indicator Title added entry Nonfiling characters 0 No added entry 0 No nonfiling characters 1 Added entry 1-9 Number of nonfiling characters Subfield Codes $a Title (NR) $k Form (R) $b Remainder of title (NR) $n Number of part/section of a work (R) $c Statement of responsibility, etc. (NR) $p Name of part/section of a work (R) $f Inclusive dates (NR) $s Version (NR) $g Bulk dates (NR) $6 Linkage (NR) $h Medium (NR) $8 Field link and sequence number (R) FIELD DEFINITION AND SCOPE Title and statement of responsibility area of the bibliographic description of a work. Title Statement field consists of the title proper and may also contain the general material designation (medium), remainder of title, other title information, the remainder of the title page transcription, and statement(s) of responsibility. The title proper includes the short title and alternative title, the numerical designation of a part/section and the name of a part/section. For mixed material, the title statement is defined as the name by which the material is known. The field may contain inclusive dates (subfield $f) and bulk dates (subfield $g) pertaining to a collection. For collections with no formal bibliographic title, subfield $k (Form) is used. Subfield $k may also be used to indicate "form" even if a formal title is given in subfields $a, $b, and $c. MARC notation Library of Congress documentation uses the dollar sign ($) to indicate subfields, and the pound sign (#) to indicate blank data elements. 300 ## $a100 pages :$billustrations ;$c26 cm. MarcEdit uses the equals sign (=) to indicate fields, the dollar sign ($) to indicate subfields, and the backslash (\) to indicate blank data elements. MarcEdit uses two spaces between the field number and the indicator codes. =300 \\$a100 pages :$billustrations ;$c26 cm. Different library systems may use the pipe sign ( ) or the double dagger ( ) to indicate subfields. For some fields (e.g. 008), the pipe sign ( ) may indicate no attempt to code.
Introduction to MarcEdit 5 Exercise 3 Imagine that you are creating a bibliographic record for a book and you want to indicate that this book is a translation. (Field 041 Language Code). Assumptions You will be using MARC language codes in this field. The language of the text you have in your hands is German (MARC language code ger). The language of the original text was Italian (MARC language code = ita). In the table below, please write your 041 field as you would enter it in your library system using Library of Congress notation. Write the field again, this time using MarcEdit notation. Please include any appropriate symbols to indicate blanks. Notation Field code Indicator 1 Indicator 2 Subfield Subfield LC MarcEdit Using the MarcEditor tool To open a file If the file has a.mrk extension: o Double-click on it and it will automatically open in the MarcEditor. If the file has a.mrc extension: o Open the MarcEditor tool o File > Open > Choose the folder where your file is saved o Choose.mrc as the file type and select the file that you want to open Inspecting the file It is usually a good idea to get a general sense of your data before you start making changes to it. In MarcEditor > Reports > Field count This will generate a Field Count report which will show you how many records you have, and how many instances of a particular field are in your records. Take note in particular of any fields that you know you will be making changes to. Basic batch modifications Generally speaking, any fields that contain number 9 in them are local fields (they contain information that only applies to the library who created the record). If you are copying records from other libraries into your own library s database (e.g. through Z39.50) you would normally want to get rid of any fields that are local to the other institution.
Introduction to MarcEdit 6 NOTE: An important exception to this rule is field 490, which contains series information. Field 500 contains general notes about a book which we can assume apply to all manifestations (instances) of that particular book. Your library may create a 590 field to include a note that is specific to the copy of the book that you have (e.g. your copy is missing some pages, or it has the author s autograph). Exercise 4 For this exercise, we will use the file called records001.mrc Step 1 : In MarcEditor > Open the records001.mrc file Step 2 : In MarcEditor > Reports > Field count Step 3 : Make note of all the fields with a number 9 in them, as well as the number of occurrences: Field Occurrences Exercise 5 : Deleting fields Now we are going to delete all the irrelevant fields that we found above. Step 1 : In MarcEditor > Tools > Add/Delete Field OR Keyboard shortcut : F7 Step 2 : Field: (Write the field number that you want to delete) Delete Field
Introduction to MarcEdit 7 Editing indicators Indicators provide additional information about a specific field. For example, the second indicator in field 245 tells us the number of nonfiling characters (characters that will not be used when filing and sorting the field). We use nonfiling characters for definite or indefinite articles at the beginning of a title. The number of nonfiling characters includes any spaces or punctuation immediately following the article. Example 1: Becomes: Title T h e Little Mermaid Nonfiling characters 1 2 3 4 245 14 $athe Little Mermaid Example 2: Becomes: Exercise 6 Title A Nonfiling characters 1 2 Streetcar Named Desire 245 12 $aa Streetcar Named Desire Before we start making transformations, we will examine field 245 to see if any indicators need to be fixed. Step 1 : In MarcEditor > Edit > Find OR Keyboard shortcut: Ctrl + F Find what: =245 (NOTE: Remember that MarcEdit notation uses an = sign for field names) Find All Step 2 : Take a quick look at your results. In particular, focus on any titles that may have an initial article and whether or not the indicators are correct based on what we would normally expect.
Introduction to MarcEdit 8 For this particular exercise, our titles have initial articles A or The, but the indicators are wrong. We are going to fix this. Step 3 : In MarcEditor > Tools > Edit Indicator Data OR Keyboard shortcut : F8 Field : 245 Indicators : 10 Field Data : $athe Replace with indicators : 14 Replace NOTE: You may have noticed that the number of modifications made will be larger than the number of 245 fields in your file. This is because each field has two indicators, so any transformation made to the indicators will be double the number of corresponding fields. Step 4 : Repeat the above instructions to fix the indicators for titles that have initial article A. Working with variable data : Basics of regular expressions Sometimes we want to make transformations that affect multiple similar fields, each of which has unique data. Field 020 normally contains the ISBN for the book in subfield $a. It may also contain qualifying information in subfield $q, such as whether the ISBN applies to the paperback edition or hardcover. 020 ## $a0802142176$qpbk. Subfield $q was introduced in 2013. Prior to that date, qualifying information was contained in subfield $a. It is possible to find 020 fields which look like this: 020 ## $a9780802078285 (pbk.)
Introduction to MarcEdit 9 We would like to fix all 020 fields in our file, so that qualifying information is moved to subfield $q. However there are a few challenges: 1) The length of the ISBN is variable (10 or 13 digits). 2) The qualifying information is also variable: a. It may or may not include parentheses. b. It may appear abbreviated or in full (e.g. paperback OR pbk.) c. Different words or abbreviations may appear as qualifying information (e.g. hardcover, pbk., vol. 1) To address this type of issue, we use regular expressions. Regular expressions identify a general pattern in simplified form, that applies to all the text strings that we want to transform, and then help us find and replace any strings that fulfill our criteria. Exercise 7 : Identifying a pattern What do these 020 fields all have in common? =020 \\$a9780802008435 (bound) =020 \\$a9780802078285 (pbk.) =020 \\$a9781895830835 (Paper) =020 \\$a0195057317 (v. 1) Solution : 1) All of them have a subfield $a 2) All of them have some number of digits from 0-9 3) All of them have a space after the digit string 4) All of them are followed by an opening parenthesis 5) All of them have letters following the parenthesis With the above information, let us begin writing our regular expression: $a 0-9 space ( letters In regular expressions, certain symbols have special meanings, such as $ and (. In order for our regular expression to work, we need to escape these characters if we want them to keep their literal value. We use the backslash (\) to escape characters: \$a 0-9 space \( letters The space is represented by \s: \$a 0-9 \s \( letters
Introduction to MarcEdit 10 Our regular expression includes a range, in this case, of digits between 0 and 9. We indicate a range using square brackets [ ]: \$a [0-9] \s \( letters We don t know exactly how many digits are in our string, it could be 10, it could be 13 (it could be more if there was a typo when entering this information). We tell the computer to look for this range of numbers appearing any number of times using the + symbol: \$a [0-9]+ \s \( letters NOTE: We can also use the * symbol to indicate any number of times. The difference is that * assumes any number of times, including 0 times, whereas + assumes any number of times, at least 1. We don t know exactly which letters appear after the parenthesis, nor do we know if they are uppercase or lower case. We have two options. Option 1 : Use ranges [a-z,a-z]* Option 2 : Use a period to indicate any character whatsoever, including spaces, digits, letters, but excluding line breaks.* Our regular expression will look something like this: \$a[0-9]+\s\(.* Remember that we want to add a subfield $q instead of the blank space. We can do this by grouping parts of our regular expression that we want to keep unchanged, and separate groups for parts we want to change. We use parentheses to group parts of our regular expression: (\$a[0-9]+) : (\s) : We want to keep subfield $a, as well as any number of digits in it. We want to group the blank space by itself, because we will be changing it. (\(.*) : We want to keep the opening parenthesis, as well as any characters after it. Now our regular expression will look like this: (\$a[0-9]+)(\s)(\(.*)
Introduction to MarcEdit 11 We can identify each part of our regular expression using a number starting with 1: (\$a[0-9]+) (\s) (\(.*) 1 2 3 Now we will tell the program to keep part 1, replace part 2 with $q and keep part 3. Here is how we do it: Now let s put everything together. In MarcEditor > Edit field data $1$q$3 Field : 020 Find : (\$a[0-9]+)(\s)(\(.*) Replace : $1$q$3 Options : Check Use Regular Expressions Process Learn more MARC biblilographic standard: http://www.loc.gov/marc/bibliographic/ https://www.oclc.org/bibformats/en.html MarcEdit: http://marcedit.reeset.net/help Recommended: Join the MarcEdit Listserv Regular expressions https://regexr.com/ Muñoz Gómez, R. Antonio (Roberto Antonio), autor Text (visual) : electronic Introduction to MarcEdit / Instructor: R. Antonio Muñoz Gómez. -- Toronto, Ontario, 2018. 1 electronic resource (11 pages + 4 pages of additional exercises). -- (iskills Workshops Series) Handout for an introductory workshop delivered at the Faculty of Information, University of Toronto in the Winter term, 2018. -- Accompanied by two electronic files : records001.mrc and records 002.mrc Also available in print. 1. Libraries--Computer programs. 2. MarcEdit. I. Gorman, Sarah, contributor. II. Pedersen, Jordan, contributor. III. University of Toronto. Faculty of Information. iskills Workshops, host institution. IV. iskills Workshops Series
Introduction to MarcEdit : Additional exercises iskills Workshop Series. University of Toronto. Faculty of Information. Winter 2018. Instructor: R. Antonio Muñoz Gómez. For these exercises we will use the file records002.mrc Exercise 1 Follow the instructions in your handout (page 6) and delete all local fields from the records in the file. Solution: In MarcEditor > Tools > Add/Delete Field OR Keyboard shortcut : F7 Field : (Write the field number that you want to delete) Delete field Exercise 2 Follow the instructions in your handout (pages 8-11) and fix all 020 fields in your records. Solution: In MarcEditor > Edit field data Field : 020 Find : (\$a[0-9]+)(\s)(\(.*) Replace : $1$q$3 Options : Check Use Regular Expressions Process Exercise 3 Add a local note (590) to all records to indicate that the corresponding books are currently in the bindery for repairs. NOTE: You must include the indicators along with the field contents.
Additional exercises 2 Solution: In MarcEditor > Tools > Add/Delete Field OR Keyboard shortcut : F7 Field : 590 Field Data : \\$aitem currently in bindery for repair. Options : Check Add field only if not a duplicate Add Field Exercise 4 After you examined your records, you notice that some of them have information that normally goes on field 300 in an incorrect field 305. Transfer the information from field 305 to field 300. Solution: In MarcEditor > Tools > Copy field Source field : 305 Destination field : 300 Options : Check Delete Source Field Copy Exercise 5 ISBD punctuation rules for field 245 indicate that subfield $b is preceded by colon (:) 245 10$aMacroeconomics :$ba practical handbook Find all the records in your file that have both subfields $a and $b to check that the punctuation is correct. For this exercise, we will use regular expressions. The pattern that we are looking for is: 245 Followed by two spaces and two digits Followed by $a Followed by any text Followed by $b Followed by any text
Additional exercises 3 Solution There are many possibilities for regular expressions, here we have two examples: =245\s\s[0-9]+\$a.*\$b.* Where: =245 : Field number \s\s : Two blank spaces [0-9]+: One or more digits between 0-9 \$a : The text string $a (we use the backslash to escape the $ sign).* : Any number of characters \$b : The text string $b.* : Any number of characters =245.+\$a.*\$b.* Where: =245 : Field number.+ : One or more characters (including blank spaces) \$a : The text string $a (we use the backslash to escape the $ sign).* : Any number of characters \$b : The text string $b.* : Any number of characters In MarcEditor > Edit > Find OR Keyboard shortcut: Ctrl + F Find what: (Either of the above regular expressions, or another one that you may come up with). Search Options: Check Use regular expressions Find All Exercise 6 In your file, some records have incorrect punctuation. =245 10$aMy Tibetan childhood,$bwhen ice shattered stone Using the above regular expressions, fix the punctuation. NOTE: You will need to make some minor adjustments to the expression.
Additional exercises 4 Solution Step 1 : First, we need to add the incorrect punctuation mark to our regular expression. In this example we will focus on comma before $b: =245.+\$a.*,\$b.* Step 2 : Now we need to group the elements of the regular expression, separating what we want to keep from what we want to change. We use numbers for each part of our regular expression. Step 3 : In MarcEditor > Edit > Replace OR Keyboard shortcut: Ctrl + R Find what : (=245.+\$a.*)(,)(\$b.*) Replace with : $1:$3 (=245.+\$a.*) (,) (\$b.*) $1 $2 $3 Keep Change Keep Search Options: Check Use regular expressions Replace All ALTERNATIVE SOLUTION In MarcEditor > Tools > Edit Field Data Field : 245 Find : (\$a.*)(,)(\$b.*) Replace : $1:$3 Options : Check Use regular expressions Process NOTE: If we use the Edit Field Data option, we are already indicating that the field we want is 245, so we MUST remove this from our regular expression. Using this method gives us the OPTION to also get rid of the indicators that were present in our original regular expression.