The need for data custodians to capture meaningful metadata

Transcription

1 The need for data custodians to capture meaningful metadata by Helena Fourie, Department of Rural Development and Land Reform Abstract Have you ever tried sourcing a dataset to solve a spatially oriented problem? How do you know who captures what data and which dataset will be the most appropriate to address your problem? In order to determine which of the available datasets will be most appropriate for solving your specific problem, it is necessary to filter all available datasets through pre-defined acceptance criteria. That is, assuming you can also determine what data is available. Metadata enables us to search and discover datasets according to the key parameters or information entered as metadata values. Without meaningful metadata we cannot know enough about a dataset to distinguish a suitable dataset from an unsuitable dataset. However, thinking that a dataset is suitable may not be sufficient if the credibility of the dataset is questionable. This is especially important when the use of a less reliable dataset could negatively impact the natural environment or affect human lives. The governance of data through responsible custodianship is therefore important to ensure authoritative data sourcing. This paper discusses three primary questions: Who should be custodian of what data? What are the responsibilities of custodians? When is metadata meaningful? The South African Spatial Data Infrastructure (SASDI) should be based on responsible custodianship and the capturing of meaningful metadata. This will greatly improve the access to, sharing and use of authoritative and relevant spatial information to support decision-making in order to achieve our socio-economic development objectives. Keywords spatial information, data custodianship, custodian, metadata, data catalogue Introduction Although the history of GIS in South Africa can generally be traced no further than the mid-1980s, a lot of spatial data has been collected since then. As GIS technology continually develops and GIS applications gain momentum, the amount of data increases exponentially. Spatial data plays a critical role in decision-making. Knowing who has what spatial data, and determining which dataset will be the most suitable for a specific purpose, could save a lot of time and effort. Yet the sourcing and acquisition of relevant data remains one of the biggest challenges. GIS practitioners need to be able to search through an electronic catalogue of available spatial data to discover data suitable for their needs. To make data discoverable the metadata for all available datasets must first be captured and catalogued. However, unless meaningful metadata is captured, datasets may not become as discoverable as the user needs them to be in order to determine fitness for a specific use. This paper discusses the need for data governance through responsible custodianship. Emphasis is placed on the capturing of meaningful metadata to optimise the search and discovery of appropriate data. Without this, the effort put into capturing data is rendered worthless. Background The Spatial Data Infrastructure Act, 2003 The Spatial Data Infrastructure Act (SDI Act, No. 54 of 2003) is an acknowledgement of the value of spatial data as a strategic asset for supporting decision-making to achieve South Africa s national development objectives. The Act established the South African Spatial Data Infrastructure (SASDI), the Committee for Spatial Information (CSI) and an electronic metadata catalogue (EMC). The Act also provides for: the determination of 19

2 standards and prescriptions to facilitate the sharing of spatial information; the avoidance of duplication of data capture; the capture and publishing of metadata; and matters concerned therewith. It applies to organs of state which hold spatial information and to users of spatial information [1]. The South African Spatial Data Infrastructure (SASDI) The SASDI is the national technical, institutional and policy framework to facilitate the capture, management, maintenance, integration, distribution and use of spatial information [1, Sec. 3.(1)]. The words national and institutional imply that SASDI is intended for use by the national geo-information (GI) community and that all state institutions have a spatial data infrastructure (SDI) role to play. The objectives of SASDI are inter alia to [1, Sec. 3.(2)]: Capture spatial information through cooperation among organs of state. Promote effective management and maintenance of spatial information. Promote the use and sharing of spatial information. Facilitate coordination and cooperation among stakeholders. Eliminate data capture duplication. Promote universal access to spatial information. Protect copyright of the state in works relating to spatial information. The EMC, a component of the SASDI, plays an important role in providing users access to spatial information. Custodians have prescribed responsibilities in terms of managing, maintaining and providing access to their data. Data custodianship Defining core (base) data and appointing custodians To implement the SASDI it is necessary to appoint data custodians and apply sound data governance. By prioritising the appointment of base data custodians and ensuring that they publish their metadata on the EMC, users will have access to the core datasets to begin with. Subsequent appointment of other data custodians will ultimately ensure the publishing of metadata for all state owned datasets, thereby providing universal access. The SDI Act defines a base dataset as those themes of spatial information which have been captured or collected by a data custodian [1, Sec. 1]. This definition is ambiguous and does not distinguish base data from other data. A data custodian is defined as an organ of state, or independent contractor or person engaged in the exercise of a public power or performance of a public function, which captures, maintains, manages, integrates, distributes or uses spatial information [1, Sec. 1]. Public power/function implies legislative responsibility. However, this definition is very broad and could be applied to any institution capturing spatial information. For this reason the CSI s Data Subcommittee, in 2011/2012, commissioned research and stakeholder surveys to be conducted to determine the criteria for identifying South Africa s core spatial datasets and responsible custodians [2]. The results were workshopped, consolidated and presented at a CSI Meeting on 1 October The following recommendations were adopted [3]: Recommendation 1: Defining base geospatial datasets. Base geospatial datasets are identified as the minimum set of essential datasets that are widely used, as a reference base, at various administrative levels, to accomplish South Africa s national and international priorities. Recommendation 2: Criteria for base geospatial datasets: Diversity of users from different sectors deriving significant benefit from its use (compulsory). Cannot be substituted easily or generally (compulsory). Sufficient detail and accuracy for widespread use (compulsory). Complete coverage over the area of interest, preferably national (compulsory). Produced as a result of the core mandate of the custodian (conditional). Source for accurately referencing other datasets or for displaying the results of an analysis (conditional). 20

3 Recommendation 3: Criteria for identifying base dataset custodians: Mandated responsibility (compulsory). Sufficient capacity, resources and infrastructure to be custodian (conditional). Requested by the CSI (compulsory). Mandated responsibility for organs of state is defined by legislation, cabinet/departmental strategy or policy to satisfy a distinct service delivery, operational or business need. Mandated responsibility should make provision for capacity, resources and infrastructure, therefore lack of such does not constitute motivation to be pardoned from custodianship responsibilities, unless specifically granted by the Minister in terms of the Act [1, Sec. 20.(1)(c)]. Recommendation 4: Theme, base spatial data coordinator and data custodian(s). This recommendation was adopted on condition that the coordinating custodian will not manipulate the data without informing the base data custodian(s). It was also recommended that forthwith the term coordinating custodian be replaced with base spatial data coordinator. Custodianship could be shared between a base spatial data custodian, responsible for the spatial component, and a base attribute data custodian, responsible for the attribute component, where the spatial and attribute components of the datasets are captured separately. Custodianship could also be shared between multiple custodians, each being responsible for a specific area or field of jurisdiction. In such cases it would be advisable to set up service level agreements to specify terms of reference. Recommendation 5: Piloting ten base spatial data themes. Ten base spatial data themes and associated datasets were proposed for piloting the identification and appointment of custodians and base data coordinators: administrative boundaries, imagery, roads, social statistics, land use, land cover, cadastre, hydrology, geodesy and conservation [4]. Custodianship responsibilities In terms of the Base Dataset Custodianship Policy [5], responsibilities of data custodians include: Authoritative source: Base dataset custodians are regarded as the authoritative source for base datasets in their care. They are responsible for the processes/functions of data capture, validation, maintenance, management, archiving and documenting, as well as for accuracy, currency, metadata, storage and security (Sec ). Cooperation and coordination: Base dataset custodians shall: commit to promoting efficient, economic and effective use of resources by cooperating and coordinating with each other in sharing spatial information; improve accessibility and distribution of spatial data; avoid duplication of data capture and cost in all spheres of government; consult users, in line with Batho Pele principles, when determining data capture/updating needs. Organs of state that have been granted permission by the CSI to capture base datasets must do so in consultation with the relevant base dataset custodian and return such data to the custodian at no cost. Users must report errors found in spatial datasets, by completing Form D [6], for referral to the relevant custodian, who must provide appropriate feedback and, where applicable, supply the user with the corrected spatial dataset at no cost. Where a user is granted permission by a base dataset custodian to update a base dataset, it must be returned to the base dataset custodian at no cost (Sec ). Access: Base dataset custodians shall: capture metadata for their data holdings; supply or disseminate spatial data together with the associated metadata; disclose or make data available to the public on request unless exempted to do so in terms of the Public Access to Information Act (PAIA); protect the privacy of individuals in accordance with applicable legislation; apply security measures to avoid loss, unauthorised access, modification or disclosure of personal information; register data capture projects on the Data Capture Project Register (DCPR) managed by the NSIF; capture their metadata in the EMC (Sec ). Shared base dataset custodianship and governance: Where custodianship is shared the contributing custodians must agree on policies, standards and prescriptions for the base dataset (Sec ). In order to establish governance, a base dataset coordinator may be appointed to manage shared custodianship of a base dataset. Responsibilities include defining policy, standards and prescriptions, as well as coordinating collective inputs and ensuring the compatibility and integration of multiple datasets for consolidation into a base dataset (Sec ). 21

4 Data quality: Data custodians shall: ensure that their data is accurate, current and free from ambiguities in line with defined specifications; ensure that the quality and resolution of their data meets defined user needs; and include in dataset s metadata, quality statements on the accuracy and correctness of data (Sec ). Rights: Data custodians are entitled to charge for their data in accordance with the Policy on Pricing of Spatial Information Products and Services (Sec ) [7]. Data Capture Project Register: All data capture projects, must be registered in the Data Capture Project Register (DCPR) maintained by the NSIF. The NSIF must administrate the following submissions: Data custodians, wishing to undertake data capture projects, must notify the CSI of their intention to capture spatial datasets by completing Form E2 [8]. Non-custodians wishing to undertake data capture projects must request approval from the CSI to capture spatial datasets by completing Form E1 [9]. Users may submit their data capture user-requirements by completing Form F [10], for consideration by the relevant data custodians, when next they capture/update spatial data. Metadata Defining metadata The SDI act defines metadata as a description of the content, quality, condition and other characteristics of spatial information. Metadata describes the who, what, when, where, why and how of geospatial resources. The purpose of metadata Metadata provides for the following functions: Maintenance: To maintain an organisation s geospatial resource investment. Dissemination: To provide information about an organisation s resources to users, distributors, data catalogues or clearing houses. Use: To provide information needed to interpret and process data from external sources Metadata serves four roles [11]: Discovery/availability: Information needed to determine what datasets exist for a given subject or geographic extent/location (elements such as title, geographic extent). Fitness for us: Information needed to determine if a dataset will meet a specified need (elements such as limitations, quality, scale/resolution). Access: Information needed to acquire a dataset from the distributor or owner (elements such as resource site/url, contact details). Data transfer: Information needed to process and interpret a dataset, to be received through transfer from an external source or for data integration (elements such as methodology, spatial reference system). Metadata standards Metadata is usually captured according to a specific metadata standard. Metadata standards are designed to address the needs of particular user communities to define specific resource types. They define the metadata scheme/schema (metadata sections, entities and elements) and how metadata should be captured and represented. The International Standards Organisation (ISO) has a subcommittee for developing standards for geographic information (TC 211). Two international standards for metadata are: ISO :2014 Geographic Information Metadata Part 1: Fundamentals; and ISO 19139: 2007 Geographic Information Metadata XML schema implementation. Other internationally renowned metadata standards include: Dublin Core, comprising only 15 metadata elements for defining a wide range of resources; and Federal Geographic Data Committee (FGDC) standard - FGDC-STD , also referred to as the Content Standard for Digital Geospatial Metadata (CSDGM). Sometimes a country, or a community of practice, adapts a metadata standard to suit their specific needs. This is known as a metadata profile. SANS : 2005 is a South African National Standard (SANS) entitled South 22

5 African spatial metadata standard, Part 1: Core metadata profile. This standard is a profile of the ISO standard adapted to define the schema required for geographic information and services in South Africa. There is also a SANS : 2010 standard entitled Geographic information Metadata Part 2: Extensions for imagery and gridded data. Metadata is mandatory In terms of the SDI Act [1, Sec.12(1)-(2)], a data custodian must capture and maintain metadata for any spatial information held by it and ensure that such metadata is made available to users. Furthermore, in terms of Sec. 14(3), a data custodian or a data vendor supplying spatial information must provide the relevant metadata together with the spatial data/information. In order to comply fully with the SDI Act s metadata requirements organisations must: (a) capture metadata for their data holdings; (b) provide access to their data holdings by making metadata available to users: (i) in the organisation s information manual, in terms of Sec. 14 of the Promotion of Access to Information Act (PAIA, Act No. 2 of 2000); (ii) together with any dataset distributed/disseminated; and (iii) by ensuring that their metadata is published in the EMC. Withholding relevant metadata information could also be regarded as a contravention of the PAIA Act. Elements for metadata capture The CSI Data and Standards subcommittees work on defining core metadata elements and their associated mandatory, conditional and optional obligations. They compiled a Metadata Fields Comparison [12] list, comparing metadata fields (elements) for a range of standards. This work is yet to be finalised and submitted to the CSI for approval. This should however, not deter data custodians or contributors from beginning to document the essences of metadata for their data holdings. The core metadata elements answer the what, where, when and who to contact questions that enable users to search and discover geospatial information. Although the names may vary in different metadata standards, the generic metadata sections, entities and elements in Fig. 1 were adapted from this list and the core metadata for geographic datasets listed in Table 2 of SANS :2005 [13]. The text in blue indicates the minimum metadata required for describing most spatial datasets or resources - extensions for imagery and gridded data is not included. The indicated mandatory (M), conditional (C) and optional (O) obligations should be regarded as a guideline only, as they are yet to be finalised. 23

6 Identification or Citation (what, by whom, why): Title (M) Resource online URL (C) Subject (M) Publication/reference date (M) Author/Responsible party (who to contact): Author/creator responsible party/individual (M) Publisher responsible organisation (M) Address (O) Position/role (O) Contact details (O) Associated party/co-author(s): Contributor individual (C) Contributor organisation (C) Address (O) Position/role (O) Contact details (O) Topic category, keywords (M) Abstract/description (M) Usage, restrictions, constraints(for whom, for what): License and user rights (M) Provenance/lineage statement (M) Methodology (how): Main step(s) (O) Detailed documentation/description (O) Instrumentation (O). Technical parameters (what): Language (M) Character set (M) Scale/resolution (M) Distribution format(s) (M) Format version (C) Data/spatial representation type(m) Spatial reference system (M) and Projection (C) Data Quality (M) Maintenance (m) Data dictionary/schema (what): Object type/name (M) Attribute name(s) (O) Definition, valid values (O) Column/field name(s) (O) Value type (O) Dictionary URL or schema (O) Coverage/dimensions of the data (where, when): Spatial coverages - provide at least one: Geographic identifier/location/name (M) Geographic Reference: Latitude (C); Longitude (C) Bounding coordinates: North (M); South (M); West (M); East (M) Vertical Extent: Minimum (C); Maximum (C); Unit of measure (C); Datum (C) Temporal extent/coverage Begin date (M) End date (M) Physical measurement/observation coverage: Quantity (O) Unit (O) Taxonomic coverage: Rank(s) (C) Value(s) (C) Metadata elements: Online resource: Metadata online URL (C) Protocol (C) Name (C) Description (C) Metadata standard: Metadata standard name (C); Version (C) Language (M) Character set (M) Metadata file identifier (M) Metadata creation date/time stamp (M) Maintenance (m) Custodian (metadata): Metadata contact/individual (M) Organisation (M) Address (O) Position/role (O) Contact details (O) Key: Section, element Entity, element Fig. 1: Generic metadata sections, entities and elements adapted from Metadata Fields Comparison [12] list. Some useful hints for capturing metadata Without metadata, a digital resource may be irretrievable. In an electronic metadata catalogue it is only possible to find a spatial dataset based on those metadata elements that were captured. Regardless of M/C/O obligation as many elements as possible should be captured that are needed to properly define a dataset and to ensure that the dataset is not applied incorrectly. Optional elements should not be disregarded purely because obligation to capture is not implied. Ideally every data custodian/contributor s organisation should have a metadata management plan to specify metadata standard/profile(s) to be used, metadata elements to be captured and responsible parties, and capture and validation procedures. A metadata manager could be appointed to compile templates and metadata capture guidelines, ensure that the capturing of metadata is built into business processes, and consolidate metadata for capturing and publishing on the organisation s metadata catalogue as well as on the SASDI EMC. Once metadata is published to a metadata catalogue it is advisable to do some searches to test the discoverability of datasets and adapt metadata capture guidelines [14]. Practitioners involved in capturing or processing spatial data should provide metadata inputs. Leaving the capture of metadata for later or for those not involved in the data capture process, may lead to incorrect or unreliable information being captured. 24

7 Preferably keywords, abstracts and descriptions should be based on standardised vocabulary. Searches are not efficient if, for example, some practitioners refer to alien vegetation while others refer to invader species [12]. GIS software systems differ in how metadata is captured. In some proprietary software, such as Esri s ArcGIS, the metadata consists of properties and documentation. Properties, such as extent or coordinate system, are inherent characteristics automatically derived from the data source. Documentation, such as purpose and keywords, are entered by the user or data/metadata administrator. This means that not all the metadata is captured automatically and custodians/contributors must manually capture the documentation sections themselves. It should however be noted that, if a property s value is edited manually, it changes categories from property to documentation and will no longer update automatically in the future if the item changes by further data manipulation or processing [14]. SASDI metadata portal hosted by the NRF The SDI Act also establishes an EMC [1, Sec. 4(1)(d)] and requires that data custodians must make their metadata available to the department, in the prescribed manner, for inclusion in the EMC. The department is represented by the NSIF, who is mandated to develop and maintain the EMC. The prescribed manner requires that data custodians are themselves responsible for capturing their metadata on the EMC. The current EMC is the SASDI metadata node on the SEAON system, hosted by the NRF, which may be accessed either directly via the URL: [15] or via the SASDI website: [16], by clicking on the EMC tab. Custodians, or any data contributors, can register on the system, create a custodian/contributor node, and capture metadata by capturing metadata directly onto the EMC, harvesting metadata from the custodian/contributor s own system, or downloading metadata from existing own metadata records. The SAEON portal supports a variety of metadata standards and is designed to support more, if required by its stakeholder community [17]. The standards currently supported are Dublin core, EML 2.x, ISO and 19139, SANS 1878 and FGDC- CSDGM. Wim Hugo will present two workshops on the SAEON system and SASDI electronic metadata portal at the Geomatics Indaba 2015 [18, 19]. Ideally every custodian should have an internal server or SDI that includes a web portal enabling users to browse their metadata and access or acquire their spatial data. This will also simplify the process of harvesting metadata from the custodian s system into the SAEON/SASDI system. However, while metadata on a custodian s SDI/Web portal will only enable searches from amongst data holdings of that custodian s available metadata, searches on the SAEON/SASDI system will enable searches from amongst data holdings of all custodians and other contributors on the SASDI node, as well as any other nodes on the SAEON system. This optimises the ability to find data that is fit for use. The concepts of a collaborative SDI is presented by Ms Lindy-Anne Siebritz at Geomatics Indaba 2015 [20]. The need for meaningful metadata An alarming fact is that, according to a GIS data and capacity survey conducted in 2008 for the office of the Presidency and the Department of Public Service and Administration (DPSA), out of 1114 datasets listed by 60 institutions 42% of the datasets were being maintained while metadata was maintained for only 10% of listed datasets [21]. Although the capturing of metadata for spatial data is mandatory for data custodians, it does not mean that other spatial data stakeholders, vendors or distributors do not need to capture metadata. Metadata lends credibility to spatial resources as well as to the organisation responsible. Metadata is therefore synonymous with marketing. Many organisations assume that the dataset s component metadata file (XML), even if incomplete, is sufficient proof that they have fulfilled their obligation in terms of complying with the SDI Act. It is very frustrating, when opening an XML file, to discover that many documentation sections have not been completed. It also means that the metadata is lacking vital information that deprives users of the ability to adequately choose a dataset most suited to their specific needs. Users are then required to acquire and investigate the dataset itself to determine its fitness for use this is time-consuming and wasteful expenditure. Finally, metadata becomes meaningful when it is properly captured and available for discovery. If sufficient metadata information on the available resources or datasets has been captured users can select a dataset that is fit for use - for a specified subject and purpose within the required geographic extent. Metadata should also provide the necessary information to access or acquire the dataset as well as to guide its transfer and integration with other datasets. By ensuring that metadata is captured, current, freely available and accessible in an electronic metadata catalogue, it will improve the access to and use of spatial data by all. 25

8 References [1] Republic of South Africa, Spatial Data Infrastructure Act, 2003 (No. 54 of 2003). South Africa: Government Gazette No , [2] C Schwabe and S Govender, Stakeholder survey on defining the criteria and identifying core geospatial datasets and data custodians in South Africa, [3] Committee for Spatial Information, Minutes of Committee for Spatial Information (CSI) meeting held on 1 October 2012, CSI, Johannesburg, [4] Committee for Spatial Information Data Subcommittee, Proposed base data custodians and coordinators, [5] Committee for Spatial Information, Base dataset custodianship policy, Government Notice no. 96 in Government Gazette no , Department of Rural Development and Land Reform, 2015, pp [6] National Spatial Information Framework, Form D: Report regarding perceived deficiency in the quality of spatial data. SASDI/DCPR. [7] Committee for Spatial Information, Policy on pricing of spatial information products and services, in Government Notice No. 96 in Government Gazette No.38474, Department of Rural Development and Land Reform, 2015, pp [8] National Spatial Information Framework, Form E2: Inform CSI of intention to capture spatial data. SASDI/DCPR. [9] National Spatial Information Framework, Form E1: Request approval from CSI to capture spatial data. SASDI/DCPR. [10] National Spatial Information Framework, Form F: Submission of spatial information needs. SASDI/DCPR. [11] Federal Geographic Data Committee, Content standard for digital geospatial metadata, Version 2.0. Washington, D.C., [12] Committee for Spatial Information: Standards Subcommittee, Metadata fields comparison v1, [13] South African Bureau of Standards, SANS : 2005 South African spatial metadata standard Part 1: Core metadata profile. SANS, [14] ArcGIS Desktop Help, What documentation to provide, A quick tour of creating and editing metadata, Esri. [15] National Research Foundation and Department of Rural Development and Land Reform, SAEON Electronic Metadata Catalogue (EMC). [Accessed: 1 July 2015]. [16] Department of Rural Development and Land Reform, SASDI website. [Accessed: 4 Jul 2015]. [17] National Research Foundation - SAEON, SAEON/Metadata/Standards support. [Accessed: 5 Jul 2015]. [18] W Hugo, Roadmap for SASDI (SAEON) EMC implementation, in Geomatics Indaba, Workshop, [19] W Hugo, SASDI (SAEON) EMC implementation: Practical registration and metadata capture, in Geomatics Indaba, Workshop, [20] L-A Siebritz, The South African Spatial Data Infrastructure: a Collaborative SDI, in Geomatics Indaba, General Paper, [21] Geospace International, GIS survey final report: GIS data and capacity survey, prepared for Office of the Presidency and Department of Public Service and Administration, Contact Helena Fourie, Department of Rural Development and Land Reform, Tel , helena.fourie@drdlr.gov.za 26