Guidelines for Metadata and Data Directory

Transcription

1 Guidelines for Metadata and Data Directory Prepared for the GRDC SIP09 project teams by: Mohammad Abuzar, Department of Primary Industries (DPI), Tatura, Victoria. Brett Whelan, Australian Centre for Precision Agriculture (ACPA), University of Sydney, NSW. Greg Lyle, Future Farming Systems for Australian Landscape, CSIRO: Land and Water, WA. March 2003 Page 1 of 11

2 CONTENTS INTRODUCTION 3 PROFORMA 4 DEFINITION OF ELEMENTS 5 EXAMPLE OF IMAGERY SET 7 EXAMPLE OF YIELD SET 8 PROPOSED DIRECTORY STRUCTURE FOR STORAGE 10 EXAMPLE: STRUCTURE AND STORAGE 11 Page 2 of 11

3 INTRODUCTION All project teams under GRDC initiative will be required to maintain metadata for the geospatial information collected and generated during the course of project implementation. Metadata will need to meet ANZLIC standards. Data directories will be maintained according to CSIRO (Land and Water) directory structure. The term metadata means summary information about data, recorded in a standard structured format. ANZLIC - the Spatial Information Council is the peak intergovernmental organisation providing guidelines for the collection, management and use of spatial information in Australia and New Zealand ( Under GRDC initiative, all vector data (point, line, and polygon) will be recorded in Geographic (latitude/longitude) coordinates using GDA94 /WGS84 format. All raster (images) will be recorded in MGA94 format. In the metadata proforma below, all the core metadata elements have been listed. Some of these elements are considered essential, shown as mandatory. The other elements are optional. However, the project teams are encouraged to complete the optional elements too. A version of this proforma is available from the ACPA as a MS-Word document with modifiable metadata fields. Page 3 of 11

4 PROFORMA CATEGORY CORE ELEMENT OBLIGATION SET Title Custodian Jurisdiction DESCRIPTION Abstract CURRENCY SET STATUS Search Word(s) Name(s) Polygon(s) Beginning date Ending date Progress Maintenance and update frequency ACCESS Stored Data Format Available format types QUALITY CONTACT INFORMATION Access constraints Lineage Positional accuracy Attribute accuracy Logical consistency Completeness Contact organisation Contact position Mail Address Locality /Suburb/Place State Country DATE ADDITIONAL Postcode Telephone Facsimile Electronic mail address Metadata date Additional Metadata Page 4 of 11

5 DEFINITION OF ELEMENTS Title The commonly used name of the data set. This should also be used for the file name (i.e. Title.doc) which is to be saved in the same directory as the data. Custodian The name of the organisation responsible for the storage, maintenance and distribution of the data set. For farmer gathered data such as yield that has been released for use, the basic farmers details (name and farm) should also be recorded. Should a farmer wish to maintain control of data distribution, the farmers full details need to be recorded under contact organization. Jurisdiction Jurisdiction means the name of the State / Country in which the custodian is based. If the custodian has offices in more than one state, it may choose to write Australia as its jurisdiction. Abstract This is a brief summary of the data set. It shall highlight the important points that help identification of the data set. Abstract shall include the additional information about the following data sets. Imagery: Platform/sensor, spatial resolution, level of processing and the number of bands. Electrical conductivity (EC): Device used (eg EM38), grid spacing, and the GPS type used. Yield: Monitor type, GPS type used Search Word(s) Key words relevant to the identification of the data sets. : Name(s) The ordinary name of the place, region or farm property, describing the location/coverage of the data set. : Polygon(s) Coordinates (Longitude, latitude) of the centre and/or major corners of the area, map projection (if other than Geographic ), and datum (if other than GDA94). Beginning date The earliest date when the content of the data set was recorded. Ending date The latest date when the content of the data set was recorded. Progress The status of the data set whether complete, in progress, planned or not known. Maintenance and update frequency This element describes the frequency with which the data set undergoes changes. It may be weekly, yearly or as required, or not applicable. Page 5 of 11

6 Stored Data Format Information on digital presentation (eg text, table, point, raster or vector), software if relevant, number of records/polygons/arcs (where applicable), number of files/folders and the size of storage. Available format types Available version(s) and format(s) of the data set. Access constraints Any restrictions/legal prerequisites, royalty/licensing agreement and the conditions to assure the protection of privacy and intellectual property. Lineage A history of the source and the processing steps used to produce the data set. Positional accuracy Spatial accuracy/resolution and vertical accuracy/resolution. Attribute accuracy Results of any accuracy assessment or validation process conducted on the attributes of the data set. Logical consistency Results of logical consistency tests if carried out. In an example of logical consistency of a vector data set, tests generally will address questions such as: Are all points labelled? Do lines intersect at nodes? Do lines overshoot or undershoot? Are all polygons labelled? Do any polygons have any duplicate labels? Completeness This element will provide information whether the data set is complete for coverage and content or any parts/attributes are missing. Contact organisation Name of the organisation/unit that can be contacted to obtain the data set. It may be the same as Custodian. Contact position The name/position of the person for the contact. Mail Address Locality /Suburb/Place State State or Union Territory Country Australia Postcode Telephone Area code + Number Electronic mail address Metadata date Date of metadata creation. Facsimile Area code + Number Additional Metadata Title of other metadata if relevant. Page 6 of 11

7 EXAMPLE OF IMAGERY SET CATEGORY CORE ELEMENT DESCRIPTION SET Title St_Arnaud9485_28jun02 (Landsat ETM Image) Custodian Department of Primary Industries (DPI), Victoria Jurisdiction Victoria, Australia DESCRIPTION Abstract Landsat-7 ETM+ Scene 94/85 was acquired on 18 October This orthorectified product has all 8 bands. This image was primarily used for land cover classification and later on for biomass estimation. Search Word(s) St. Arnaud scene, Birchip image, Scene 94/85, satellite image. Name(s) North-Western Victoria, the Mallee. CURRENCY SET STATUS Polygon(s) Beginning date 18 October 2002 Ending date 18 October 2002 Progress Complete Image Format: GDA94 (Also available in MGA format) Map Projection: Geographic Datum: GDA94 Minimum longitude: E Minimum latitude: 35.13S Maximum longitude: E Maximum latitude: S Maintenance and Not applicable. update frequency ACCESS Stored Data Format Raster; Imagine format; img and rrd files, 218MB Available format types Access constraints Imagine Business arrangement with DPI required. QUALITY CONTACT INFORMATION DATE ADDITIONAL Lineage Data was acquired and image processing done by ACRES. Positional accuracy Spatial resolutions: 30m (Bands 1-5,7), 60 m (Band 6), 15 m (Band 8) Attribute accuracy Logical consistency Completeness Contact organisation Contact position Mail Address Not known. Not known. Complete. Locality /Suburb/Place Tatura State Country Department of Primary Industries, Victoria Database Manager GIS Group, Ferguson Road. Victoria Australia Postcode 3616 Telephone (03) Facsimile (03) Electronic mail address..@nre.vic.gov.au Metadata date 17 December 2002 Additional Metadata Not applicable. Page 7 of 11

8 EXAMPLE OF YIELD SET CATEGORY CORE ELEMENT DESCRIPTION SET Title Wheat_Yield_2002 (Johns) Custodian Dean Johns, 4 Wettenhall St, Horsham, VIC 3400 Jurisdiction Victoria, Australia DESCRIPTION Abstract The point data refers to wheat harvested in 2002 on the DEL16 Paddock of Dean Johns using CASE systems. CURRENCY SET STATUS Search Word(s) Name(s) Polygon(s) The property is located near Dooen-Murtoa townships. The following fields are available: Longitude, Latitude, Field, Dataset, Product, Object Id, Track (deg), Swath Width (m), Distance (m), Duration (hr), Elevation (m), Area Count, Diff Status, Time, Crop Flow (kg/s), Moisture (%), Yield Mass (Dry)(tonne/ha), Yield Volume (Dry)(L/ha), Yield Mass (Wet)(tonne/ha), Yield Volume (Wet)(L/ha), Pass Number, and Prod (ha/hr). Wheat yield North-Western Victoria, the Mallee. Point Data Format: GDA94 Map Projection: Geographic, Datum: GDA94 Minimum longitude: E Minimum latitude: S Maximum longitude: E Maximum latitude: S Beginning date 30 November 2002 Ending date 1 December 2002 Progress Complete Maintenance and Not applicable update frequency ACCESS Stored Data Format CASE YLD, text (690KB), shapefile. QUALITY Available format types Access constraints Lineage Positional accuracy Text, shapefile. Permission required from Dean Johns through Birchip Cropping Group (BCG). Yield data was stores originally in CASE systems. Approx. 1m CONTACT INFORMATION Attribute accuracy Not known Logical consistency Not known Completeness Complete Contact organisation Birchip Cropping Group (BCG) Contact position Mail Address P. O. Box 85 Locality /Suburb/Place Birchip State Victoria Country Australia Postcode 3483 Telephone (03) Page 8 of 11

9 DATE ADDITIONAL Facsimile (03) Electronic mail Metadata date 28 January 2003 Additional Metadata Not applicable Page 9 of 11

10 PROPOSED DIRECTORY STRUCTURE FOR STORAGE Yield and Treatment Monitoring Systems Most manufacturers of precision agriculture equipment provide their own custodial software to provide basic display and manipulation operations for the acquired farm and paddock spatial data. Generally, the spatial data is imported from a storage device to a PC and placed in a data format that is accessible from the custodial software. However, all systems provide the user with export capabilities where the original data can be accessed and output as some form of text file. Why Export? In order to gain a greater understanding of the variation/operations occurring within a paddock and improve interpretability, more analytical capabilities are often needed than are currently available in the custodial software. Spatial data needs to be transportable between software packages of varying analytical power. Data Storage Once the spatial data has been exported, the question of storage protocols is raised. A decision to store it under one directory may make the process quite cumbersome and inefficient when numerous years, paddocks, operations and other information are added. Therefore, a file allocation method is proposed to provide users with a logical and easy-to-use way of storing spatial information. The proposed hierarchical directory structure is: Client/Operator, Farm, Paddock, Year, Operation. - where each text string under Client/Operator is a sub-directory of the Client/Operator main directory. The individual operation directories may include data from various sources. When this data is collected, or further information is derived from the existing data, another sub directory can be created under the relevant directory. At this level of the hierarchical structure, the decision of if/where to add another directory really depends on the user but placement should depend on whether the data is time or scale dependant, and on it use. For example, gamma radiometrics may be collected for the whole farm but might be clipped into separate paddocks. Naturally, we would keep both a farm and paddock copy but this information would be kept above the Year directory in the hierarchical structure since it will not be collected every year. Definitions Client/Operator: the Client/Operator directory represents the top of the hierarchical structure. This can be the farmer s or operator s name or could be the name of a farming group. Farm: represents the farm name of the Client/Operator. Paddock: represents the paddock name within the particular farm. Year: All years where the paddock has both spatial and non-spatial information. Operation: Is a conceptual name for the various actions that can be performed on a paddock. For example, the most regular operation will be yield monitoring, but fertiliser application, weed identification, spraying logs etc may be gathered. All annual operations should be sub directories of the particular Year in which they were collected. These directories are the depository of all raw data files collected by the monitoring systems. Page 10 of 11

11 EXAMPLE: STRUCTURE AND STORAGE CLIENT/OPERATOR FARM PADDOCK YEAR OPERATION Represents a directory folder Yield Monitoring Weed Mapping Boundary Etc.. RAW Page 11 of 11