New Zealand Government Locator Service (NZGLS) Metadata Schema Compliance Study

Transcription

1 New Zealand Government Locator Service (NZGLS) Metadata Schema Compliance Study E-Government Unit, State Services Commission Te Komihana O Ngā Tari Kāwanatanga and National Library of New Zealand Te Puna Mātauranga o Aotearoa December 2001 December

2 Contents Purpose... 3 Background... 3 Methodology... 3 Findings and Recommendations Upgrade of Existing Metadata Creating Metadata for Usability Time Taken to Edit/Create Metadata... 4 Average Time Taken (in minutes) Human Resource Requirements Workflow Support Documentation FONZ, SONZ, and Library of Congress Subject Headings (LCSH)... 7 Appendix One: Issues encountered with the NZGLS Schema... 8 Appendix Two: Data on Time Taken to Create/Edit Metadata... 8 Appendix Three: DC Data Entry guidelines used... 9 Appendix Four: NZGLS guidelines used for the upgrading the NL website DC HTML metatags Appendix Five: Differences between the NZGLS Schema and the Dublin Core Standard Element Qualifier December

3 Purpose A joint pilot study funded by the State Services Commission and the National Library of New Zealand was run to identify the compliance costs and issues involved in extending existing Dublin Core (DC) compliant metatags in a website to make them New Zealand Government Locator Service (NZGLS) compliant. Background The E-Government Unit is developing a new Government Portal that will be based on agency-created metadata for services and resources. One potential major source of relevant metadata is agency websites. The level of current agency website compliance with metadata standards is largely unknown. The National Library s corporate web site contains some basic DC metadata in the HTML metatags. This metadata requires upgrading to both comply with NZGLS and conform to the Library s new DC guidelines. 1 In general these guidelines require a fuller record than NZGLS does. Methodology The study worked on a representative sample of 40 pages from the National Library s corporate web site ( The existing DC metadata was evaluated for quality, upgraded, and NZGLS metadata was added. A contractor familiar with the DC standard, the NZGLS schema, and cataloguing processes was hired for this work. Mandatory and recommended NZGLS elements were added, plus additional elements as required to assist with description and retrieval. The Windows-based Metabrowser tool ( was used for data entry. A CGI Perl script provided by the vendor allows a Metabrowser user to save their changes directly into the HTML pages on a Web server using HTTP. Findings and Recommendations 1. Upgrade of Existing Metadata The existing metadata was of average (typical) quality. The metadata was created by nonspecialists at the time the website was created. While it is Dublin Core (DC) compliant, this is primarily because the DC standard is not particularly prescriptive in the content of its elements. Thus the existing metadata required upgrading in order to both conform to NZGLS, and also to meet the National Library s own DC guidelines. 2 Recommendation: Upgrading existing DC metadata should be factored into planning it is likely to need upgrading as often non-specialists have created it. 1 NL Metadata Standards Framework. 2 See Appendices Three and Four for further information on meeting the National Library s DC guidelines December

4 2. Creating Metadata for Usability In accordance with the criteria in Sections A7-A13 of the NZGLS manual v2, a decision was made to create metadata for all the mandatory and recommended NZGLS elements. It was felt that the description, date and multiple subject elements are likely to contain the keywords the users will search on and also be used to decide whether the resource or service is likely to be relevant. This study concludes that generating the abstract for the description requires more subject analysis and work. But as the description element is an important one for searching and helping users decide if the resource may be useful, it was reasoned that it was time well spent. It became quickly apparent that the most efficient approach was to cut and paste existing text on the website as all this information content had been approved initially for publication on the site. Recommendation: All elements with a Recommended obligation in NZGLS should be included; otherwise the usefulness of the records to users is greatly reduced. Recommendation: Where possible, cut-and-paste sentences from the website page for the description element, as this saves time, and uses text already approved for publication. 3. Time Taken to Edit/Create Metadata In general, pages with Collection aggregation level take much longer to make NZGLS compliant than item level pages, as more subject headings and longer descriptions are required. Providing metadata for services is initially more complicated than for documents. However because the records are shorter, the time taken is roughly comparable. As expected, the average time taken per record dropped over the course of the study. Timesaving was more noticeable in the area of adding new tags rather than editing existing tags. Average Time Taken (in minutes) 3 For a record without a subject encoding scheme besides SONZ For a record with an additional subject encoding scheme (e.g. LCSH) Recommendation: Planning times should be estimated at an average of 13 to 16 minutes for each record without an additional subject-encoding scheme. These times are based on a skilled person working on the metadata. This time could also be applied to creating NZGLS metadata from scratch. The difference between 13 and 16 minutes depends on the quality level the agency chooses to apply to the DC elements (ranging from a reasonable NZGLS level up to the National Library s guidelines level (except for LCSH subject terms)). 3 Note: these times do not include quality assurance. See Appendix Two for further data on times. December

5 If the agency chooses to only create minimal metadata to meet NZGLS minimum (mandatory) requirements, the time will be reduced (in the region of an estimated 3 minutes per record), though the usefulness of the resulting record will also be significantly reduced. An average of 5 minutes per record should be added for Quality Assurance. The NZGLS schema uses some of the DC standard s elements differently (in the description of its scope, the types of values expected, the refinement qualifiers allowed, and the controlled vocabularies expected). This resulted in creating duplicate tags to satisfy them both. For those organisations creating both DC and NZGLS metadata, this means additional time during the initial preparation to compare the two requirements and then deciding which to use, and when. In addition, these organisations will need to duplicate the five elements that have been differently defined in NZGLS. In this study, it was decided to not duplicate the Date element and assume that the W3CDTF scheme (as specified by the DC standard) would be recognised by the NZGLS schema (the W3CDTF scheme is a refinement of the ISO 8601 scheme specified by NZGLS). It was also decided to not duplicate the Language element and assume that DC will recognise the RFC 3066 scheme in the near future. Recommendation: Amend the NZGLS schema so elements that are part of the DC standard are revised to use the descriptions and scope defined by the DC standard 4 so they do not need to be duplicated. 5 Add additional elements and qualifiers to meet NZGLS requirements. This would remove the extra time crosschecking between DC and NZGLS to ensure compliance with each and remove the need for duplication. It would also futureproof the metadata so it is more likely to be compliant with the DC Government (DC-Gov) Application Profile when it is completed and provide inter-operability across international DC searching Human Resource Requirements This study found that demanding mental processes were required when working with two different element sets (DC and NZGLS). The use of different schemes, controlled vocabularies and qualifiers requires a lot of thought. Agencies starting from scratch and only creating NZGLS metadata will not have to work in this dual environment (though the result will not be DC compliant unless the previous recommendation is accepted). However, in either environment it is essential that the person/people an agency selects to do this work be highly PC literate, as the ability to work with multiple windows and programs simultaneously is a great advantage. In this pilot study, a minimum of four windows were used: 4 The Dublin Core Metadata Element Set ANSI/NISO Z , ISSN: See Appendix Five for a list of differences between the NZGLS schema and the Dublin Core standard 6 See Appendix One for Issues encountered with the NZGLS schema December

6 1. Metabrowser 2. Web browser (Internet Explorer) to check other related pages on the web site 3. SONZ pdf 4. FONZ pdf Other windows were also used: 5. Notepad for commonly used phrases to cut-&-paste 6. Controlled vocabulary tool (Te Puna for LCSH subject terms) for searching for terms 7. Word for tracking work undertaken The time taken to create the metadata depends on the staff chosen by the agency to create it. If they are already familiar with the basic structure and elements of DC and NZGLS, the use of subject heading schemes and thesauri, and general cataloguing conventions, it will be a faster process. Otherwise, a longer time will need to be spent familiarising them with the schemas and the cataloguing/indexing process. Recommendation: Use staff members with knowledge, experience, or an interest in metadata and cataloguing/indexing processes to speed the process up. Advanced PC skills are also an advantage. 5. Workflow Working through groups of pages (e.g. a site map approach) saves a lot of time as the subjects and functions are similar. The use of a template (cut-and-pasted) into the Availability field saved a great deal of time, as did cut-and-pasting from the web pages for Description. Adding LCSH subject terms was facilitated by cutting and pasting from Te Puna Search. When the existing metadata is of poor quality, it takes longer to edit the existing metadata than to create new metadata from scratch. Recommendation: Plan workflow to create metadata for resources with similar subjects and functions Recommendation: Agencies with metadata they wish to upgrade should consider creating the revised metadata from scratch, especially if the metadata is of poor quality. Recommendation: When editing or upgrading a record, it is much easier to edit all the existing DC metadata and then add in NZGLS metadata rather than trying to do both things at the same time. Recommendation: If NZGLS compliance work is being combined with other metadata upgrades, it is quicker to upgrade the existing metadata in the record first then add the NZGLS data requirements to the record. December

7 6. Support Documentation The study found that the NZGLS documentation is large and formidable. It is difficult to locate each element quickly (e.g. using section letters rather than a running page number and there is no page for each of the agent elements). Once an element is found, it is difficult to determine its requirements at a glance (the page layout makes scanning difficult Information Mapping principles or tables may make scanning easier). (Note: The contractor in this study had only ever seen the 2 nd version of the documentation). As this study also required the metadata to conform to DC metadata requirements, several external resources (e.g. manuals) needed to be referred to at the same time, which is very time consuming. Agencies with these dual requirements need to incorporate this into their planning. 6.1 FONZ, SONZ, and Library of Congress Subject Headings (LCSH) As full versions of FONZ and SONZ in the metadata-capturing tool were not hierarchical or searchable, it was easier to work with the PDF documents to find terms. The terms had to then be inserted using the tool, rather than typing or pasting them. This creates a risk of inconsistency through spelling errors, typos etc. Recommendation: Further work is needed to make the documentation easier to use, especially in the navigation and presentation (to make page scanning easier). It would be easier if the specification were separate from the instructions so it isn t so formidable in size. Recommendation: Agencies should spend time preparing metadata guidelines specific to their agency. This speeds up the work and improves the consistency and quality of the metadata. A sample template for this may be helpful to agencies. December

8 Appendix One: Issues encountered with the NZGLS Schema Agent/CCP: the manual advises no spaces in Lastname,Firstname for personal names. This is inconsistent with common practice. A space was used in this study. Type: manual advises (rule N.1.11) when DC.type.category=document to repeat the element then select the document type from the list. However, the list is not extensive and it is often difficult to classify as one of these (e.g. National Library Act). An expanded list is needed. Software: It is unclear where to put information about software requirements for using/viewing the resource (e.g. Adobe Acrobat). In this study, the MIME type was placed in DC.Format.medium with the scheme IMT it is assumed an application would recognise if the MIME type (e.g. application/pdf) required special mention. Mandate: It is difficult to know where to obtain this information. It would be helpful for the agency to compile a list of the different mandated resources e.g. Legal Deposit for the National Library. Coverage: Terms giving coverage of the Pacific are harder to find (e.g. NZ Geographic Places Names Database only covers New Zealand and ISO 3166 only covers individual countries). Recommend using the Getty TGN thesaurus. Name vs. Label: The NZGLS documentation only provides labels for the elements, qualifiers, and schemes. It is not clear what machine-readable values to use. For example, should the first letter be capitalised, should spaces be included, are there standardised short codes to be used (e.g. for encoding schemes presented as AGLS agent encoding scheme, New Zealand Government Online Directory Service, or The New Zealand Geographic Place Names Database (LINZ) ). Appendix Two: Data on Time Taken to Create/Edit Metadata The average time taken per record was 20.3 minutes (range 9-37 minutes). This was made up of 11.5 minutes for upgrading the DC data (5-32 minutes) and then 8.8 minutes for adding the NZGLS elements (3-21 minutes). Time taken for Quality Assurance was not included in these figures (this is estimated to be around 5 minutes per record). The individual elements that take the most time are the access points for the record Description, Subject, and Function. On average, the time taken to create the description was 1.4 minutes and the time taken to create the NZGLS Subject and Function data was 2.7 minutes (whereas creating the LCSH subject data took 4.4 minutes on average). Minutes taken to Average Min Max Upgrade existing DC Create/Edit Description December

9 Create LCSH Subject Terms [Nat.Lib. reqt.] Make NZGLS compliant Create Subject (SONZ) and Function (FONZ) Total per record: Perform Quality Assurance [estimate] 5.0 Some of the DC tags are also required for NZGLS compliance and the intellectual effort used creating some of the DC metadata can be re-utilised in the NZGLS metadata (e.g. Subject analysis for DC identifies the themes this resource covers which then makes selecting the SONZ terms for NZGLS easier). So the minimum time it would take to upgrade existing DC and create NZGLS metadata would be 8.8 (NZGLS) (Description) (an estimated half of the remaining DC time) = 13 minutes. A previous finding found the time taken to create DC metadata from scratch is on par with the time taken to upgrade an existing record, so this figure could also be applied to creating NZGLS data from scratch. Appendix Three: DC Data Entry guidelines used These are the guidelines developed for, and used, in this pilot revisions may be necessary for any future use. Guidelines for Dublin Core (DC) usage in NLNZ Websites Follow Dublin Core recommendations (as per references below). Follow MARC and AACR2 in general (e.g. use LCSH Name Authorities in CCP). Catalogue the resource the page represents, not the page itself (i.e. Type is not Web Page). If a resource is an online version of an offline resource, catalogue the online version (e.g. the online version won t have an ISBN so don t include it). Only place metadata in HTML pages for resources we want discovered, e.g. exclude pages describing images used in the site and other trivial pages. Use DC ordering of elements (as appears below). Use a separate element for each entry. Only use Lang attribute where one entry for the element is non-english in which case use Lang for all entries for that element. Use HTML entities for macrons: e.g. ā = ā ē = ē ī = ī ō = ō ū = ū Any DC elements that are added solely to satisfy NZGLS compliance should be added to the end, e.g. DC.Coverage.jurisdiction or DC.Type.category. Do not use AGLS Agent value components in Creator/Contributor/Publisher (CCP). Title Creator Subject Meaningful (readable in hit list) so apply NLA guidelines Always National Library of New Zealand or as per name authority. Include business unit or individual if named individuals will appear in a separate element. Use LCSH (use - [space-dash-space] not --" as separator) or APAIS. Keywords: use no scheme only use meaningful keywords (not Search Engine stacking December

10 keywords), use all in one element separated by semi-colons, and ensure other subjects used are not duplicated. Description Meaningful free text abstract/summary based on the page with added content if needed as per NLA guidelines. No qualifiers. Publisher Always National Library of New Zealand. Contributor Date Type Format Identifier Source Language Relation Coverage Rights As appropriate in separate elements as per name authority. Always use qualifier. Always use W3CDTF scheme [this is non-nzgls compliant currently]. Expect at least a created use for web pages where the date is unknown. Other qualifiers as appropriate, eg. issued for publication date. Always use DCMIType qualifier. Mostly Text or Service, some may be Collection or InteractiveResource. Always use medium qualifier and IMT scheme. Most likely to be text/html there may be multiple types. extent qualifier is only needed for large files (to show Kbytes size). The URI of the resource being catalogued, eg. the PDF file not this HTML page. Always use URL with URI scheme. Always use a scheme even if not listed in DCQ. Often the print version prefer this is placed here than in Relation.isVersionOf. Use the ISBN/ISSN (plus scheme ISBN or ISSN), or the title if no ISxN number. The predominant language of the resource repeat if more than one. Use RFC3066 scheme (en/mi). Always use a qualifier most likely to be ispartof/haspart. Always use scheme URI. Always put one entry with spatial qualifier and value NZ and scheme ISO3166, unless inappropriate. Always use a qualifier. Always use a scheme even if not listed in DCQ. For spatial use in this order: ISO3166, LCSH, LINZ. For Temporal use W3CDTF. Always use and URI scheme. Issues: An HTML page can only contain metadata about one resource this causes a problem if an HTML page contains/describes 2 or more resources we want to catalogue. For now, just describe the HTML page. Future solutions are: 1. Ensure every resource has a separate HTML page, or 2. Store the metadata in linked RDF/XML files then there can be 2 or more metadata descriptions linked from a single HTML page. Metabrowser makes it difficult to place separate DC.Subject entries in separate elements for now just place all (from the same scheme, eg. LCSH, SONZ) in one element separated by semicolons. References: Z Dublin Core Metadata Element Set: Dublin Core Qualifiers: Using Dublin Core: DCMI Frequently Asked Questions (FAQ): December

11 Guidelines for the Creation of Content for Resource Discovery Metadata (July 2001 Draft), National Library of Australia. To be read in conjunction with the response from the National Library of New Zealand. New Zealand Government Locator Service (NZGLS) Metadata Standard and Reference Manual v2.0: Date and Time Formats (W3CDTF ): DCMI Type Vocabulary: Internet Media Types (IMT): ISO 3166 Codes for the representation of names of countries: LCSH Subject Headings/Name Authorities/Geographic Authorities: [need Te Puna password] APAIS thesaurus (NZ Supplement): [need Te Puna password] LINZ New Zealand Geographic Place Names Database: Appendix Four: NZGLS guidelines used for the upgrading the NL website DC HTML metatags This work required the creation of a fuller record (adding more elements where appropriate), adding DC qualifiers and schemes where appropriate, and adding LCSH (Library of Congress Subject Headings a controlled vocabulary) subject terms (for compatibility with its other metadata). These are the guidelines developed for, and used, in this pilot revisions may be necessary for any future use. Guidelines for NZGLS usage in NLNZ Websites DC.Title, DC.Creator, DC.Subject, DC.Publisher, DC.Date, DC.Type and NZGLS.Function must be used. Either DC.Identifier or NZGLS.Availability must be used: Identifier (URI) for an online resource, Availability for a service or offline resource. Use DC.Description, DC.Language, DC.Coverage, NZGLS.Audience and NZGLS.Mandate where required. DC.Title DC.Creator DC.Subject DC.Description DC.Publisher DC.Contributor Repeat DC.Subject element; always use SONZ Repeat element for each subject heading if tool allows December

12 DC.Date DC.Type DC.Format DC.Identifier DC.Source DC.Language DC.Relation DC.Coverage DC.Rights NZGLS.Function NZGLS.Availability NZGLS.Audience NZGLS.Mandate Always use qualifiers and add at the end of DC record Add Type.category; always use NZGLS scheme (i.e. will be agency, service or document; usually document) Repeat Add.Type.category=document: always use NZGLS Portal Vocabulary scheme to select text or image type Add Type.aggregation.Level; always use NZGLS Portal Vocabulary scheme (i.e. will be item or collection; if collection, use Description to provide information about the different parts of the collection that searchers might be trying to find) Always use for online resources [see Availability for services and offline resources] Always use a scheme, as per DC guidelines Always use a qualifier and scheme, as for DC guidelines DC.Coverage.spatial, use ISO3166 with value NZ unless inappropriate DC.Coverage.temporal, use W3CDTF [not currently NZGLS compliant] Add DC.Coverage.jurisdiction at the end of DC record: only use where a resource or service has a particular jurisdiction (not very likely for NL) for resources; can also be used to describe rights to access services [nb. do not confuse with Audience] Always use FONZ Repeat element for each subject heading if tool allows Always use for off-line resources and services: must conform to New Zealand Government Directory once available. In meantime must conform to AGLS Availability Scheme (see Q2 in NZGLS manual). Always use a scheme: NZGLS Portal Vocabulary, ANZSIC, NZSCO99, NZSC: will usually be All from NZGLS Only use where necessary to highlight regulations that require the resource to be created or provided No scheme, but always use generally accepted legal notation Appendix Five: Differences between the NZGLS Schema and the Dublin Core Standard Based on The Dublin Core Metadata Element Set v1.1 (2/7/1999) [the same as ANSI/NISO Z (10/9/2001)], DC Qualifiers v1.0 (11/7/2000), and NZGLS v2.0 (Aug 2001). General differences: Definition and purpose/comments for each element are mostly different December

13 NZGLS adds refinement qualifiers to DC elements that are not defined by DC Qualifiers v1.0 NZGLS presents the elements in a different order to DC In NZGLS, Creator, Publisher, and Contributor may have value components in the data: corporatename, personalname, contact, address, , and jurisdiction All elements and qualifiers are optional in DC. NZGLS defines various elements and qualifiers mandatory, recommended, or optional. DC allows for application profiles which define how DC and other metadata schemes are to be combined for a particular application. They must not alter the source schemes though they can clarify their usage/implementation (NZGLS would be appropriate as an application profile) Major differences in qualifiers: Title.alternative is defined using a capital A in NZGLS Description NZGLS does not recognise the two DC refinement qualifiers. Contributor NZGLS specifies user-defined refinement qualifiers Date and Coverage specify ISO8601 in NZGLS rather than W3CDTF [W3CDTF is a profile of ISO8601] Date NZGLS does not recognise one of the DC refinement qualifiers. Date DC defines the encoding scheme name Period and label DCMIPeriod as different text, NZGLS uses the same text for the name and label Type NZGLS specifies five extra refinement qualifiers Language NZGLS specifies the RFC3066 encoding scheme (the latest RFC DC does not support this currently, but it is in the process of being registered) Relation NZGLS specifies two extra refinement qualifiers Coverage NZGLS specifies an extra refinement qualifier Elements that may need to be duplicated to satisfy both DC compliance and NZGLS compliance: Subject If an encoding scheme other than SONZ is used, any topics selected from SONZ will need another element containing the same topic as expressed in the other encoding scheme Date NZGLS uses a different name for the same encoding scheme Type NZGLS uses different refinement qualifiers and encoding scheme Language NZGLS uses a different encoding scheme Coverage NZGLS uses a different encoding scheme (ie. the recommended encoding scheme) Element Qualifier DC NZGLS Title Refinement alternative alternative Scheme - - Creator Refinement - - Scheme - NZ Govt Online Directory Service AGLS Agent Subject Refinement - - Scheme LCSH MeSH SONZ [others must be registered] December

14 DDC LCC UDC Description Refinement tableofcontents - abstract Scheme - URI Publisher Refinement - - Scheme - Online directory service Contributor Refinement - [user-defined roles] Scheme - Online directory service created valid issued modified Date Refinement created valid issued modified available Scheme Period W3CDTF DCMIPeriod ISO 8601 Type Refinement - category aggregationlevel Scheme DCMIType [NZGLS list] Format Refinement extent medium extent medium Scheme IMT [user-defined] Identifier Refinement - - Scheme URI URI ISBN ISSN Source Refinement - - Scheme URI URI ISBN ISSN Language Refinement - - Scheme Relation Refinement IsVersionOf hasversion isreplacedby replaces isrequiredby requires ispartof haspart isreferencedby references isformatof hasformat ISO639-2 RFC1766 [RFC3066 coming] RFC 3066 RFC 3166 Scheme URI URI ISBN isversionof hasversion isreplacedby replaces isrequiredby requires ispartof haspart isreferencedby references isformatof hasformat isbasedon isbasisfor December

15 Coverage Refinement spatial temporal Scheme Point ISO3166 Box TGN Period W3CDTF ISSN spatial temporal jurisdiction [user-defined] [NZGLS list when using jurisdiction] Rights Refinement - - Scheme - URI December