August 2015 Version 3.9. Data Product Description

Transcription

1 Product: Prepared: August 205 Version 3.9 Data Product Description

2 0BStandard This document is based on the AS/NZS ISO 93:2008 Geographic information Data product specifications standard. For more information, refer to 2TUwww.saiglobal.com/onlineU2T. BAcknowledgements PSMA Australia acknowledges the assistance of CGI Group Inc. & Geometry Pty Ltd in the preparation of this guide. 2BDisclaimer PSMA Australia believes this publication to be correct at the time of printing and does not accept responsibility for any consequences arising from the use of information herein. Readers should rely on their own skill and judgement to apply information to particular issues. This work is copyright. Apart from any use as permitted under the Copyright Act 968, no part may be reproduced by any process without prior written permission of PSMA Australia Limited. VERSION 3.9 PAGE i

3 3BContents. B Overview B Data product specification title B Reference date B Responsible party B Language B Topic category B Distribution format B Terms and definitions B Abbreviations and acronyms B Informal description of the data product B Specification Scope B Scope identification B Data Product Identification B Title B Alternate titles BAbstract B Purpose B Topic category B Geographic description B Data Content and Structure B Feature-based data B Feature-based application schema (data model) B Data dictionary B Feature-based content scope B Reference System B Spatial reference system B Temporal reference system B Reference system scope B Data Quality B Positional accuracy B Attribute accuracy B Address mesh block integration B Allocation of Postcodes... 2 VERSION 3.9 PAGE ii

4 6.5 49B Logical consistency B Completeness B Data Capture B Data capture scope B Data Maintenance B G-NAF maintenance pre-processing B G-NAF maintenance software B G-NAF data processing B G-NAF Lite data processing B Maintenance software process flow A Address identifiers and update examples B Update frequency B Maintenance scope B Data Product Delivery B Delivery medium information B Data importer B Licensee installation procedure B Units of delivery B Medium name B Delivery format information B Metadata B PSMA Data BAppendix A Guidelines on using G-NAF to support NAMF BAppendix B Data Models BAppendix C Data Dictionary VERSION 3.9 PAGE iii

5 4BTables Table : Relationship between G-NAF, AS4590 and NAMF fields Table 2: ADDRESS_ALIAS Table 3: ADDRESS_ALIAS_TYPE_AUT Table 4: ADDRESS_ALIAS_TYPE_AUT Table 5: ADDRESS_DETAIL Table 6: PRIMARY_SECONDARY Table 7: PS_JOIN_TYPE_AUT Table 8: PS_JOIN_TYPE_AUT Codes Table 9: ADDRESS_MESH_BLOCK_ Table 0: MB_MATCH_CODE_AUT Table : MB_MATCH_CODE_AUT Codes Table 2: ADDRESS_SITE Table 3: ADDRESS_SITE_GEOCODE Table 4: ADDRESS_TYPE_AUT... 5 Table 5: ADDRESS_TYPE_AUT Codes... 5 Table 6: FLAT_TYPE_AUT Table 7: FLAT_TYPE_AUT Codes Table 8: GEOCODE_RELIABILITY_AUT Table 9: GEOCODE_RELIABILITY_AUT codes Table 2: GEOCODE_TYPE_AUT Table 2: GEOCODE_TYPE_AUT Codes Table 22: ADDRESS_DEFAULT_GEOCODE Table 23: GEOCODED_LEVEL_TYPE_AUT Table 24: GEOCODED_LEVEL_TYPE_AUT Codes Table 25: LEVEL_TYPE_AUT Table 26: LEVEL_TYPE_AUT Codes Table 27: LOCALITY_CLASS_AUT Table 28: LOCALITY_CLASS_AUT Codes Table 29: LOCALITY Table 30: LOCALITY_ALIAS Table 3: LOCALITY_ALIAS_TYPE_AUT Table 32: LOCALITY_ALIAS_TYPE_AUT Codes Table 33: LOCALITY_NEIGHBOUR... 6 Table 34: LOCALITY_POINT... 6 VERSION 3.9 PAGE iv

6 Table 35: MB_ Table 36: STREET_CLASS_AUT Table 37: STREET_CLASS_AUT Codes Table 38: STREET_TYPE_AUT Table 39: STREET_TYPE_AUT Codes Table 40: STREET_LOCALITY Table 4: STREET_LOCALITY_POINT Table 42: STREET_LOCALITY_ALIAS Table 43: STREET_LOCALITY_ALIAS_TYPE_AUT Table 44:STREET_LOCALITY_ALIAS_TYPE_AUT Codes Table 45: STREET_SUFFIX_AUT Table 48: STREET_SUFFIX_AUT Codes Table 47: STATE... 7 Table 48: APPLIED GEOCODE PRIORITY ORDER BFigures Figure : Corner block... 4 Figure 2: Alternative locality... 5 Figure 3: High Level Diagram of G-NAF Maintenance Process... 2 Figure 4: Flow of data through the G-NAF maintenance program Figure 5: Data importer Figure 6: Architectural components Figure 7: Data Importer supported formats VERSION 3.9 PAGE v

7 . BOverview. 22BData product specification title G-NAF and G-NAF Lite Product Description.2 23BReference date August BResponsible party PSMA Australia Limited Level 2, 3 Canberra Avenue, GRIFFITH ACT 2603 Australia T: F: E: 2TUsupport@psma.com.auU2T URL: 2TUhttp:// ABN: BLanguage English.5 26BTopic category Address files for urban, rural and locality areas within Australia..6 27BDistribution format PDF.7 28BTerms and definitions The following table lists the terms and definitions that are used by PSMA Australia. This list defines some of the terms used for G-NAF, but is not necessarily the same definition as used by others. Term Address Address Contributor Address Level Geocode Alias Address (or Alternate Address) Building Name Class Confidence Level Contributor Data Definition A structured label for any place that could deliver or receive a good or service. Supplier of address information to G-NAF. An address in G-NAF that has a parcel level geocode (i.e. a geocode reliability code of 2). Also referred to as a parcel level address Another label for a Principal Address which may differ because of a variation in some or all address components (i.e. numbers, levels, street name, locality name). An Alias Address will share the same location as the Principal Address. Association with Principal Address may be provided by an Address Contributor or determined programmatically. A building or property name that is associated with an address. The name is typically free text and is not unique to any address. Description of a set of objects that share the same attributes, operations, methods, relationships, and semantics [UML]. Note: A class does not always have an associated geometry (e.g. the metadata class). G-NAF is currently built from three address datasets. The confidence level of an address indicates the level of usage of each address by the contributor address datasets. Address datasets held and maintained by organisations external to PSMA Australia that is being made available for the purpose of building and maintaining G-NAF. VERSION 3.9 PAGE

8 Event Feature Feature Attribute Gap Geocode Geocode Geocode Level Type Geocode Priority Geocode Reliability G-NAF Merge Criteria Jurisdiction Locality Locality Alias Other Territories Mesh Blocks Object Package Postcodes Primary Postcodes Primary Address Principal Address PSMA Systems Online Data Delivery Reference Data Characteristic of a feature measured within an object without modifying the associated geometry. Abstraction of real world phenomena. Characteristic of a feature (e.g. name of an area). A geocode created programmatically based on address ranging in the absence of a geocode being allocated to a specific property. This geocode may not necessarily lie within a property polygon. A point feature for an address spatially defined by a coordinate. In G-NAF an address may have multiple geocodes representing various real world features (e.g. parcel, property, building centroid, street centroid, locality centroid etc.) associated with a physical address. The geographic coordinates are expressed as latitude/longitude in decimal degrees. The geocode level type indicates which geocodes have been assigned to an address. Every address within G-NAF must have a locality level geocode. Addresses may also have a street level geocode and a parcel level geocode. G-NAF is able to assign multiple geocodes to an address. The geocode priority is a priority order established by PSMA Australia to enable a default geocode to be assigned to each address which represents the geocode of the highest precision currently assigned to an address. Refers to the geocode precision and is linked to how the geocode was generated. Those components of an address string used by the G-NAF process to uniquely identify an address. Reference to a State or Territory Government. A named geographical area defining a community or area of interest, which may be rural or urban in character. Usually known as a Suburb in an urban area. The localities used in G-NAF are the gazetted localities as provided by the respective jurisdictions. Another recognised name for a gazetted locality name. Could be misspellings, historic authoritative names or unauthoritative names. Refers to external Territories of Australia included within the PSMA Australia datasets, namely Christmas Island, Cocos (Keeling) Islands, Norfolk Islands and Jervis Bay. Mesh Blocks are the smallest geographic region in the Australian Statistical Geography Standard (ASGS), and the smallest geographical unit for which Census data is available. Entity with a well-defined boundary and identity that encapsulates state and behaviour [UML Semantics] Note: An object is an instance of a class. Grouping of a set of classes, relationships, and even other packages with a view to organising the model into more abstract structures. Postcodes are allocated to geographic areas to facilitate the efficient processing and delivery of mail to customers by Australia Post. Primary postcodes are unique integers based on the postcodes used to differentiate between gazetted localities within a jurisdiction that share the same name. They may not necessarily be coincident with the postcode for any area. A Principal Address which does not have a flat number but which matches the Secondary Address in all other respects OR is designated as owning Secondary Addresses by PSMA through an address contributor (e.g. involves private road in complex development, public housing estates etc.) Accepted label for an Address which may have zero, one or more than one associated Alias Address. A suite of applications to store, quality assure and distribute PSMA Australia s datasets. Each address entering G-NAF is tested for to ensure it can be matched against the geospatial region to which it relates: state, locality and street. The datasets used for this geospatial verification are: National Road Centreline Dataset (PSMA Australia Transport & Topography) Gazetted Locality Boundaries for Australia (PSMA Australia Administrative Boundaries) These two datasets are commonly referred to as the Reference Data. VERSION 3.9 PAGE 2

9 Rules Rules are at times applied to contributor addresses as part of the G-NAF processing where errors are identified in the addresses such as the incorrect spelling of street names, incorrect street types or incorrect localities. The application of these rules generates alias street localities or localities depending on the rule applied. Secondary Address A Secondary Address represents a separately addressable location which is located with the same general site as the associated Primary Address (although the Secondary Address may share the same geocode as the Primary Address). In G- NAF generally, it is any address with a flat_number or more literally any address where flat_number_prefix, flat_number or flat_number_suffix is not null OR is designated as being linked to a Primary Address by PSMA through an Address Contributor (e.g. involves complex sites etc.). in some cases, an address with a number_first_suffix may be designated as a secondary address. Street (or Road) An in use name for a street name that exists within PSMA Australia s Road centreline dataset (i.e. reference dataset). Street Locality A Street or Road within a particular Locality e.g. Smith St Melbourne VIC A street is unique to a locality. If it crosses a locality boundary, that segment receives a different street_locality_pid and treated as another street. Street Locality Alias A Street or Road within a particular Locality that is an alternative name for a Street Locality Address. Street Alias In use name for a Street name not existing within PSMA Australia s Road centreline dataset (i.e. reference dataset). Quality Data is of the highest quality reflective at the time and fields provided. Key factors - are timeliness, consistency and completeness. VERSION 3.9 PAGE 3

10 .8 29BAbbreviations and acronyms Term Definition ABS ASGC ASGS AS4590:2006 AS/NZS 489:20 CD DPS FOI Australian Bureau of Statistics Australian Standard Geographical Classification. Australian Statistical Geography Standard Data interchange standard Rural and urban addressing standard Collection Districts (for Census collection) Data Product Specification Features of Interest GDA94 Geocentric Datum of Australia 994 GIDB G-NAF GCCSA IARE ILOC IREG ICSM IDB MB NAMF PID SEB SLA UCL UML UUID A copy of the IDB for use in Data Maintenance in Radius Studio Geocoded National Address File Greater Capital City Statistical Areas Indigenous Areas Indigenous Localities Indigenous Regions Intergovernmental Committee on Surveying & Mapping Integrated Data Base Mesh Blocks National Address Management Framework Persistent Identifier State/Territory Electoral Boundaries Statistical Local Area Urban Centre Localities Unified Modelling Language Universal Unique Identifiers.9 30BInformal description of the data product G-NAF (Geocoded National Address File) is the authoritative address index for Australia. It contains the state, suburb, street, number and coordinate reference or geocode for street addresses in Australia. The G-NAF Data Model (see Appendix A) is derived from the AS 4590:2006 Data interchange standard. The addressing information interchange in Section 5 of the standard has been aligned with AS/NZS 489:20 Rural and urban addressing standard. Names of people are not part of G-NAF. G-NAF does not contain any personal information. VERSION 3.9 PAGE 4

11 G-NAF uses existing and recognised address sources (referred to as contributors) from the state and territory government land records, as well as address data from Australia Post and the Australian Electoral Commission. A rigorous process is used that involves textual address comparison, matching and geospatial validation, to provide both national consistency and national coverage (see Section 8 Data Maintenance for detailed discussion of the build methodology). G-NAF at the time of its introduction in 2004 it was a world first methodology providing a mechanism to create and maintain a national address index of all addresses in use and importantly identifies those considered official. The process is repeatable and the relationships established for access to the address data are long standing to ensure that maintenance of the index can be assured. G-NAF Lite is produced as part of the G-NAF production process. This derivative dataset delivers all the quality benefits of the full G-NAF including principal and alias addresses, confidence levels, complex site addresses and a unique persistent identifier. However, all spatial attributes and links, including geocodes, are removed from G-NAF Lite. The National Address Management Framework (NAMF) is a national, coordinated approach to address management. It is a consistent, standards-based framework which will guide the process for verifying addresses and provide a standard for exchange of address data. More information about NAMF is available from the ANZLIC website - Some guidelines on using G- NAF to meet NAMF requirements is provided in Appendix A. VERSION 3.9 PAGE 5

12 2. 2BSpecification Scope The Feature Based Content, Reference Systems, Data Quality, Data Capture and Data Maintenance also have defined scopes regarding the data accuracy, geometry, metadata and temporal considerations of the data release cycle. 2. 3BScope identification 66BLevel Dataset 67BLevel name G-NAF G-NAF Lite 68BExtent Spatial coverage of Australia s landmass including External Territories and Coastal Islands (including Lord Howe Island). Localities in SA include an unincorporated area which is covered by Mesh Blocks. All data is supplied by the appropriate jurisdiction quarterly. VERSION 3.9 PAGE 6

13 3. 3BData Product Identification 3. 32BTitle G-NAF Dataset and G-NAF Lite Datasets BAlternate titles G-NAF for Australia Geocoded National Address File Geographic National Address File BAbstract The G-NAF Product Description (an ISO 93 compliant description) provides an optimised quality geometric description and a set of basic attributes of the Address Index for Australia. This release of the G-NAF product includes all addresses included in contributor s data that are regarded as complete addresses. G-NAF data will be revised on a regular basis. Geographic Data Files based on GDA94 are used (from CadLite and Transport datasets provided from the Jurisdictions) to relate addresses within cadastre boundaries or along street lines BPurpose A physical address relates to any property or parcel identified by contributors and is used by them for administrative purposes. This means that the vast majority of addresses relate to an existing building or property. However there are a few addresses that relate to: Proposed developments (and therefore will have no buildings) Vacant land that is not ear-marked for development at this time Other addresses that may not receive mail deliveries Addresses required for the operations of the contributors for asset management and other administrative purposes. This means that there may not be a building at each address and mail or goods may not be delivered to those addresses. G-NAF is used within many applications including web based maps. PSMA Australia updates the data every three months but the data in the applications is not necessarily current. The provider of the applications may choose to update far less regularly and therefore supply out of date data in their application. This is beyond the control of PSMA Australia BTopic category G-NAF: Addresses defined by coordinate spatial data (latitude and longitude) with associated textual metadata. G-NAF Lite: Addresses with associated textual metadata. VERSION 3.9 PAGE 7

14 3.6 37BGeographic description The G-NAF datasets cover the addresses within the complete national geography of Australia (AUS). The Bounding Box for this data is as follows; North bounding latitude: -8 South bounding latitude: -45 East bounding longitude: 68 West bounding longitude: 96 This area covers the landmasses of Australia (Geographic Australia), including External Territories and offshore islands. The following quote from the ABS is used to identify the coverage of the data. Geographic Australia means the area as defined by the Acts Interpretation Act 90 as amended by the Territories Law Reform Act No. 04, 992. For the avoidance of doubt, the External Territories of Christmas Island and Cocos (Keeling) Islands are included in Geographic Australia. The spatial domain is described by the polygon: VERSION 3.9 PAGE 8

15 69BGeographic extent name AUSTRALIA INCLUDING EXTERNAL TERRITORIES AUS Australia Australia The States and Territories within Australia are represented by the following: State or Territory Name Abbreviation Character Code New South Wales NSW (or 0) Victoria VIC 2 (or 02) Queensland QLD 3 (or 03) South Australia SA 4 (or 04) West Australia WA 5 (or 05) Tasmania TAS 6 (or 06) Northern Territory NT 7 (or 07) Australian Capital Territory ACT 8 (or 08) Other Territories OT 9 (or 09) VERSION 3.9 PAGE 9

16 4. 4BData Content and Structure G-NAF is a feature-based product. A data model is included (Appendix A) with an associated data dictionary (Appendix B) BFeature-based data Data types and codes are derived from the address standard where applicable. However, in some cases the codes have been extended to handle exceptional cases (e.g. street type of BRETT ). The data model is hierarchical in nature, storing information about streets and localities separate from address sites. Alias addresses are stored in the exact same way as principal addresses. There is simply a mapping table provided to determine which address is an alias of which principal address BFeature-based application schema (data model) The G-NAF data model is set out in Appendix A BData dictionary The G-NAF feature catalogue in support of the application schema is provided in Appendix B. Spatial attributes are added to the feature catalogue in the same manner as other attributes for completeness and conformance to the application schema. Note that spatial attributes are excluded from G-NAF Lite. For ALL tables the Persistent Identifier (_pid), date_created and date_retired fields are governed by the ICSM Policy and Guidelines for Incremental Update. This can be accessed by following the link below. 2TUwww.icsm.gov.au/icsm/harmonised_data_model/model/incremental_up-date_guidelines.pdf U2T 4.4 4BFeature-based content scope All geometry and metadata for polygons and points within the G-NAF Dataset. VERSION 3.9 PAGE 0

17 5. 5BReference System 5. 42BSpatial reference system GDA BTemporal reference system Gregorian calendar BReference system scope The spatial objects and temporal collection periods for the G-NAF Datasets VERSION 3.9 PAGE

18 6. 6BData Quality 6. 45BPositional accuracy G-NAF is a concerted effort to deliver the best possible geocoded national address dataset available. The magnitude of this dataset, the complexity of its content, and the multiplicity of its sources, means that there is ongoing requirement to improve the content, quality and coverage of the G-NAF. Since there is no simple means of determining the absolute accuracy and coverage of the G-NAF, it is not possible to provide global statistically valid indicators of the degree to which G-NAF meets these criteria. Jurisdictional documentation will provide a guide to values within each state/territory. Relative accuracy and coverage can be gauged to an extent through comparison between associated datasets. Inbuilt procedures have been developed to ensure data conformity & accuracy. Note that spatial attributes are excluded from G-NAF Lite BAttribute accuracy Attribute accuracy is determined from rigorous scrubbing processes and matching of the three contributor datasets as well as the matching of addresses against gazetted localities and the roads layer of PSMA Australia s Transport & Topography dataset BAddress mesh block integration G-NAF has been updated to allocate a mesh block ID to every address within G-NAF (except Norfolk Island). This now includes addresses that have a geocode allocated at a street-locality and locality level where a single mesh block cannot be identified. A coding table (MB_MATCH_CODE_AUT) lists the codes describing the level of matching to mesh blocks. The ABS recommends that G-NAF addresses that are associated with locality (and street) level geocode locations can be associated with SA2 level units, ASGS units built up from SA2s. Whilst there is a strong alignment between gazetted suburb and locality boundaries there are situations where SA2 boundaries split localities. In these circumstances an unknown proportion of the addresses associated with the locality centroid will be coded to an incorrect SA2. Note that mesh block ID s are excluded from G-NAF Lite BAllocation of Postcodes Postcodes are included in G-NAF at the address record level (i.e. postcode field in the respective address_detail tables). The postcodes used are those provided by Australia Post where they contribute to the specific address. The postcode is then validated via a spatial intersection from a PSMA created (and maintained) postcode polygon dataset with boundary information sourced from the Commonwealth. Some realignment of the supplied boundaries has been untaken to align them to locality boundaries BLogical consistency The G-NAF Data Model (see Appendix A) is derived from the AS4590:2006 Data Interchange Standard. The addressing information interchange provided in Section 5 of the Standard has been aligned with AS/NZS 489:2003 Geographic information-rural and urban addressing standard. VERSION 3.9 PAGE 2

19 The dataset data structure has been tested for conformance with the data model. The following have been tested and confirmed to conform: File names Attribute names Attribute lengths Attribute types Attribute domains Attribute order in file. Compulsory attributes populated BCompleteness Completeness is an assessment of the extent and range of the dataset with regard to completeness of coverage, completeness of classification and completeness of verification. 70BDataset coverage National (for the incorporated data note that the Localities Theme for South Australia has some unincorporated areas). 7BFeatures 82BAddresses included G-NAF aims to include all physical addresses in circulation, postal addresses and PO boxes are not included. However, due to the lack of rural address information in some states, roadside mail box (RMB) numbers, Lot numbers and Block & Section numbers (South Australia) have been included. All addresses have been allocated a postcode. 83BAlias Addresses Alias addresses are addresses, other than the principal address, that refer to the same physical location as another address record. The inclusion of aliases in G-NAF is not an endorsement of the use of unofficial addresses, but as an aid to assist licensees in matching to their existing address databases. The inclusion of alias information greatly enhances the usability of G-NAF by supporting addresses in popular use regardless of their official status. PSMA Australia recognises that G-NAF has a role to play in progressing usage of official, gazetted addresses. However, it is also acknowledged that the issue cannot be forced and in some cases, it will take generational change to see alias or incorrect addresses taken out of everyday usage. VERSION 3.9 PAGE 3

20 It is also considered that the benefits of the inclusion of aliases outweigh the costs; particularly in the application of G-NAF by emergency services. There are three levels of aliases in the G-NAF schema: Alias Address- where an individual address is also known by another name Locality Address - where a locality does not exist in the reference data and is the synonym or incorrect spelling of a locality that does exist Street/Locality Address - where a street/locality pair does not exist in the reference data and is the synonym or incorrect spelling of a street/locality pair that does exist. Consider the following examples: Figure : Corner block VERSION 3.9 PAGE 4

21 Figure 2: Alternative locality 84BAlias locality Similar to the address aliases, the locality aliases are used to determine those addresses that refer to the same physical location as another address record, where the locality is different (as shown Figure 2: Alternative locality). Where it is identified that the locality in an address from a contributor was incorrect (e.g. spelling error or vanity addressing), a rule is created to manipulate the data during the scrubbing process. An example of a rule is provided below: <sql_update> <sequence>2</sequence> <statement>update address_details_scrub set locality_name = 'CITY' where locality_name = 'CANBERRA CITY' </statement> </sql_update> The example locality CITY will exist in the LOCALITY table and an entry for CANBERRA CITY will exist in the LOCALITY_ALIAS table. When validating an address, if the locality of the address is CANBERRA CITY it should be changed to CITY prior to checking the address in G-NAF. VERSION 3.9 PAGE 5

22 85BAlias street/locality Similar to the other aliases, the street/locality aliases are used to determine those addresses that refer to the same physical location as another address record, where the street/locality is different (as shown in the Alternative Locality diagram above). Where it is identified that the street/locality in an address from a contributor was incorrect (e.g. spelling error or vanity addressing) a rule (see below) is created to manipulate the data during the scrubbing process. <sql_update> <sequence>4</sequence> <statement>update address_details_scrub set street_name='aitkens',street_type = 'ST', </sql_update> locality_name = 'KAMBAH' where street_name = 'ATKINS' and street_type = 'ST' and locality_name = 'KAMBAH' </statement> The street AITKENS ST will exist in the STREET table and locality KAMBAH will exist in the LOCALITY table and an entry for ATKINS ST will exist in the STREET_LOCALITY_ALIAS table. In this way, a link exists to the principal address even if the licensee is presented with an alias. 72BUsing alias datasets When using G-NAF to validate an address, the steps are: Is there a principal address for this address? 2 Is there an alias address for this address? 3 Is there an alias locality for the locality of the address? a. This can be determined by checking the locality name of the address against the LOCALITY_NAME field in the LOCALITY_ALIAS table, the locality_pid is then used to determine the correct locality_name from the LOCALITY table. b. The next step would be to retry steps & 2 with the new locality_name. 4 Is there an alias street/locality for the address? a. This can be determined by checking the street name of the address against the street_name, street_type, street_suffix fields in the STREET_LOCALITY_ALIAS table, the street_pid is then used to determine the correct street_name from the STREET table. b. The next step would be to retry steps, 2 & 3 with the new street name. 73BAttribute completeness All mandatory attributes for each object are populated. Some attributes are not populated but have been included in the data model to assist with the alignment with relevant standards. Temporal accuracy is applicable to most of the current release. The G-NAF data model allows attributes to be stored for each address attribute. An overview of these is provided below. 86BAddress usage Each address and geocode can be related to the dataset that contained it, which in turn can be related to the contributor who provided it. This feature is essential to being able to supply the information back to the address contributors. However, for Value Added Resellers (VARs) and end users of G-NAF, the VERSION 3.9 PAGE 6

23 address custodian identifier has been removed. Instead, the VAR/end user is provided address level metadata indicating how many source datasets provided each address. The Address Usage is reflected in the Confidence field included in the ADDRESS_DETAIL table and is expressed as a number representing the number of datasets the address was found in, less one. Given G-NAF has been built with three Contributor datasets, the Address Usage (Confidence Level) possibilities are as follows: Confidence Level Confidence level = 2 Confidence level = Confidence level = 0 Description This reflects that all three contributors have supplied an identical address. This reflects that a match has been achieved between only two contributors. This reflects that a single contributor holds this address and no match has been achieved with either or the other two contributors. Confidence level = - This reflects that none of the contributors hold this address in their address dataset anymore. An Archival Policy has been instituted for G-NAF. Addresses with an attribute of - will remain with the G-NAF release database for a period of 4 updates, after which time they will become tagged as -2 s and be archived. The -2 tagged records stay within the production database and are available on request. 87BGeocoding Multiple geocodes and multiple types of geocodes can be stored for each address. All geocodes are stored in G-NAF in the GDA94 spatial reference system. Whilst this capability exists in the G-NAF model, at this stage addresses with multiple geocodes only exist for some addresses in South Australia. Note that geocodes are excluded from G-NAF Lite. 88BGeocode level type Every address within G-NAF must have a locality level geocode, it may also have a street level geocode and a parcel level geocode. The table GEOCODE_LEVEL_TYPE_AUT indicates which of these Geocode Level Types are associated with an address in accordance with the table below: Geocode_Level_Type Description 0 No Geocode Parcel Level Geocode Only (No Locality or Street Level Geocode) 2 Street Level Geocode Only (No Locality or Parcel Level Geocode) 3 Street and Parcel Level Geocodes (No Locality Geocode) 4 Locality Level Geocode Only (No Street or Parcel Level Geocode) 5 Locality and Parcel level Geocodes (No Street Level Geocode) 6 Locality and Street Level Geocodes (No Parcel Level Geocodes) 7 Locality, Street and Parcel Level Geocodes Note: LEVEL_GEOCODED_CODE field within the ADDRESS_DETAIL table refers to the CODE field within the GEOCODE_LEVEL_TYPE_AUT. 89BGeocode reliability Reliability of a geocode refers to the geocode precision and is linked to how the geocode was generated. VERSION 3.9 PAGE 7

24 Every geocode in G-NAF has a reliability level. The levels and their descriptions are stored in the table GEOCODE_RELIABILITY_AUT. These descriptions and an example are given in the table below. Reliability Level Description Example Geocode resolution recorded to appropriate surveying standard 2 Geocode resolution sufficient to place geocode within address site boundary or access point close to address site boundary 3 Geocode resolution sufficient to place geocode near (or possibly within) address site boundary 4 Geocode resolution sufficient to associate address site with a unique road feature 5 Geocode resolution sufficient to associate address site with a unique locality or neighbourhood 6 Geocode resolution sufficient to associate address site with a unique region Address level geocode was manually geocoded with a GPS a) Address level geocode was calculated as the geometric centre within the associated cadastre parcel b) Geocode for access point identified for a rural property c) Calculated geocode based on centre setback from road within cadastre parcel d) Geocode for approximate centre of building Address level geocode was automatically calculated by calculating where on the road the address was likely to appear based upon other bounding geocoded addresses Street level geocode automatically calculated by using the road centreline reference data Locality level geocode automatically calculated to the geometric centre within the gazetted locality for this address Locality level geocode derived from topographic feature Note: RELIABILITY_CODE field within the ADDRESS_SITE_GEOCODE table refers to the CODE field within the GEOCODE_ TYPE_AUT. Every geocode has a reliability level. These levels are stored with the geocodes in the following tables: STREET_LOCALITY_POINT LOCALITY_POINT ADDRESS_SITE_GEOCODE Provision has also been made for G-NAF to cater for multiple types of geocodes for an address. Where the geocode types are nominated by the jurisdiction these are reflected in the geocode type field. Where the geocode type is not provided, a default value is used that reflects the majority of addresses. The full list of allowed geocode types is included in Table 22 (i.e. Geocode AUT table) of the Data Dictionary in Appendix C 90BGeocode Priority A priority order has been developed and applied during G-NAF production to provide a single geocode for all G-NAF addresses. The priority order developed places an emphasis on identifying locations associated with emergency management access, buildings on a site and then other locations which are associated with the land management process. This order has been developed with a view towards assisting users in general and will not be suitable for all user business needs. The priority order applied is included in the relevant table in Appendix C. It should be noted that whilst the data model and respective geocode types have been listed, in the vast majority of cases there is no current national data sources identified to populate these additional codes. The priority order has been applied in the ADDRESS_DEFAULT_GEOCODE table. VERSION 3.9 PAGE 8

25 74BQuality scope The attribute accuracy is in scope for all areas in the G-NAF product. VERSION 3.9 PAGE 9

26 7. 7BData Capture The G-NAF data contributors include: The mapping agencies and land registries of each of the state and territory governments Australia Post Australian Electoral Commission. 7. 5BData capture scope Data for changed objects within the current release time period. VERSION 3.9 PAGE 20

27 8. 8BData Maintenance Maintenance activities are triggered by PSMA Australia receiving updated address data from the data custodians according to a pre agreed schedule for data delivery. At present, this schedule defines a quarterly update process. Users that supply corrections to the dataset are reviewed by PSMA Australia and then passed onto the contributor where appropriate. During the maintenance phase, contributed addresses are analysed and compared to existing records in G-NAF. This analysis and comparison gives rise to new records being inserted and existing records being updated or retired. Figure 3: High Level Diagram of G-NAF Maintenance Process illustrates a high-level view of the G- NAF system including G-NAF Maintenance Pre-processing, G-NAF Maintenance Software and G-NAF outputs. Figure 3: High Level Diagram of G-NAF Maintenance Process Reference Reference Data Reference Files Data Files Data Files Contributor Contributor Data Contributor Files Data Files Data Files G-NAF Maintenance Pre-Processing Rule Generation G-NAF Maintenance Software see below Log Files PSMA Geocoded National Address File (G-NAF) Database G-NAF Lite Data Reports Data Extraction G-NAF Data 8. 52BG-NAF maintenance pre-processing The G-NAF Maintenance Pre-process as indicated in Figure 3: High Level Diagram of G-NAF Maintenance Process takes the input files from the PSMA Australia reference datasets and VERSION 3.9 PAGE 2

28 Contributor data and performs processing prior to data being processed by the G-NAF Maintenance Software. Pre-processing is used to describe the following activities: Conversion of file format (e.g. MapInfo to G-NAF files) Mapping from Contributor model to G-NAF model (with parsing as necessary) Creation of an incremental file from a full re-supply (if required) Application of rules that make corrections to misspellings, abbreviations and erroneous characters. These business rules are all supplied back to the Custodians. Where it is not evident that an element of an address is erroneous no change is made without reference back to the relevant jurisdiction Application of updates to suburb data and road names propagating the changes through all effected parts of the data BG-NAF maintenance software The G-NAF Maintenance Software, as indicated in Figure 3: High Level Diagram of G-NAF Maintenance Process, processes data passed to it from the pre-process stage. The G-NAF Maintenance Software then stores verified data in a database and log files are created which contain those records that could not be verified. These log files are then reviewed and pre-processing rules are created that are used by the pre-process function. This process is iterative and the cycle continues until all logged records are accepted or confirmed as invalid and not repairable BG-NAF data processing The data is processed with the G-NAF Maintenance Software in the seven main steps listed below: Address scrubbing State-locality validation and geocoding Street validation Street geocoding Address geocoding Merging Post Merge processing (including validation processes) BG-NAF Lite data processing G-NAF Lite follows the G-NAF data processing steps with the addition of a further step to remove all spatial attributes and links from the final product. VERSION 3.9 PAGE 22

29 8.5 56BMaintenance software process flow Figure 4: Flow of data through the G-NAF maintenance program NSW Jurisdiction ADDRESS_ DETAILS_CHAR AEC (NSW) ADDRESS_ DETAILS_CHAR Australia Post (NSW) ADDRESS_ DETAILS_CHAR G-NAF_NSW Database G-NAF_ AEC_NSW Database G-NAF_ AUSP_NSW Database SCRUB Process- Correct spelling, Principal address, Correct Synonym. Make corrections to incorrect entries Make corrections to incorrect entries Make corrections to incorrect entries LOCALITY MATCH Process- Compare all contributor localities Check that Locality details match Locality Reference files Check that Locality details match Locality Reference files Check that Locality details match Locality Reference files STREET MATCH Process- Compare all contributor street addresses etc Check that Street details match to Street Reference files Check that Street details match to Street Reference files Check that Street details match to Street Reference files Street Geocoder Process Geocode the Address to the centre of the street in a locality Geocode the Address to the centre of the street in a locality Geocode the Address to the centre of the street in a locality Address Geocoder Process Use the Geocode supplied to locate the Address within the property Match the Address and select the property Geocode Match the Address and select the property Geocode Merge all data Post Merge Processes e.g validation PSMA G-NAF Database Filter out spatial attributes PSMA G-NAF Lite Database VERSION 3.9 PAGE 23

30 8.6 A Address identifiers and update examples ADDRESS_DETAIL_PID Each table in the G-NAF data model (Appendix B) contains a primary key. The primary key field from the central ADDRESS_DETAIL table is the address_detail_pid. This key is often referred to as the address identifier (or G-NAF ID). The value (and ongoing persistance) for the address_detail_pid is based on many of the fields within the ADDRESS_DETAIL table (plus data from the LOCALITY and STATE tables), which describe the address. G-NAF users should note that there is no reference to feature geometry used in the creation nor maintenance of the address_detail_pid. Specifically the address_detail_pid is based on the concatenation (i.e. joining) in order, of a number of fields (each fields separated by a pipe ) referred to the G-NAF merge criteria. The fileds comprising the G-NAF merge criteria are (note exception below); STATE_ABBREVIATION LOCALITY_NAME PRIMARY_POSTCODE STREET_NAME STREET_TYPE STREET_SUFFIX NUMBER_FIRST_PREFIX NUMBER_FIRST NUMBER_FIRST_SUFFIX NUMBER_LAST_PREFIX NUMBER_LAST NUMBER_LAST_SUFFIX FLAT_NUMBER_PREFIX FLAT_NUMBER FLAT_NUMBER_SUFFIX LEVEL_NUMBER Exception: Addresses without a number first. Where a contributed address is not supplied with a number_first, consideration is given as to whether the address contains a lot_number. An address without a number_first but with a lot_number will be added to G-NAF. Where an address does not contain a number_first but it does contain a lot_number the merge criteria is changed to; STATE_ABBREVIATION LOCALITY_NAME PRIMARY_POSTCODE STREET_NAME STREET_TYPE STREET_SUFFIX LOT_NUMBER_PREFIX LOT_NUMBER LOT_NUMBER_SUFFIX NUMBER_LAST_PREFIX VERSION 3.9 PAGE 24

31 NUMBER_LAST NUMBER_LAST_SUFFIX FLAT_NUMBER_PREFIX FLAT_NUMBER FLAT_NUMBER_SUFFIX LEVEL_NUMBER A G-NAF ID (i.e. address_detail_pid) relates to a unique combination of these fields. Addresses which share the merge criteria from the different suppliers are merged into a single record (i.e. address_detail_pid) with associated metadata which highlights the number of contributing organisations (confidence attribute). This address_detail_pid will persist with the address whilst it remains in the dataset. Where values in fields, which are not included in the merge criteria (from the ADDRESS_DETAIL table) changes in consecutive product releases, the address_detail_pid will not change however the associated date_last_modified field will. This example shows building name "PONDEROSA" being changed to "EL RANCHO". The existing G-NAF record with (address_detail_pid = GAVIC4744) was created on: 29/04/2004. Because BUILDING_NAME is a non merge criteria field, the existing record (address_detail_pid = GAVIC4744) is maintained. Scenario (date_created/date_retired) Existing G-NAF Record ADDRESS_DET AIL_ PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENCE DATE_ CREATED DATE_ RETIRED DATE_ LAST_ MODIFIED GAVIC4 744 UNIT 3 PONDEROSA 2 SMITH STREET BURWOOD 2 29/04/ 2004 Updated G-NAF Records ADDRESS_DET AIL_PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENCE DATE_ CREATED DATE_ RETIRED DATE _LAST_ MODIFIED GAVIC4 744 UNIT 3 EL RANCHO 2 SMITH STREET BURWOOD 2 29/04/ /06/ 2004 A change in the confidence value across consecutive G-NAF releases would result in an update of this type as confidence is not a merge criteria value. An example of a confidence level change is provided below: Scenario 2 (date_last_modified) Existing G-NAF Record GNAF_ PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENCE DATE_ CREATED DATE_ RETIRED DATE _LAST_ MODIFIED GAVIC4 744 UNIT 3 PONDEROSA 2 SMITH STREET BURWOOD 2 29/04/ 2004 New Incoming G-NAF Record Address has been removed from one of the contributor s data. Updated G-NAF Records VERSION 3.9 PAGE 25

32 GNAF_ PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENCE DATE_ CREATED DATE_ RETIRED DATE _LAST_ MODIFIED GAVIC4 744 UNIT 3 PONDEROSA 2 SMITH STREET BURWOOD 29/04/ /06/ 2004 Merge Criteria Changes When any element comprising the merge criteria changes, the new record is treated as a new address and inserted into G-NAF as such. The example below shows Unit 3 2 Smith Street Burwood (address_detail_pid = GAVIC4744) being changed to Unit 3 2 Brown Street Burwood. Because of the street name change it is no longer possible to match the new incoming record to an existing G-NAF record, so a new G-NAF record (address_detail_pid = GAVIC ) is created. In this example it is assumed that Unit 3 2 Smith Street Burwood is provided by only one contributor and so has a confidence of 0. An example of a street name change is provided below: Scenario 3 (merge criteria changed) Existing G-NAF Record GNAF_ PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENC E DATE_ CREATE D DATE_ RETIRED DATE _LAST_ MODIFIE D GAVIC4 744 UNIT 3 PONDEROSA 2 SMITH STREE T BURWOOD 2 29/04/ 2004 Updated G-NAF Records GNAF_ PID FLAT _TYPE FLAT_ NUMBER BUILDING _NAME NUMBER _FIRST STREET _NAME STREET _TYPE LOCALITY _NAME CONFI DENC E DATE_ CREATE D DATE_ RETIRED DATE _LAST_ MODIFIE D GAVIC4 744 UNIT 3 PONDEROSA 2 SMITH STREE T BURWOOD * 29/04/ /06/ 2004 GAVIC UNIT 3 PONDEROSA 2 BROW N STREE T BURWOOD 0 4/06/ 2004 * The confidence value of the original record will be reduced because the Data Manager is creating retire/insert records of this type. * This example is where the contributor has changed the street name for the previously supplied address and no longer supports Smith Street Burwood therefore the confidence is decremented. Address Duplication By virtue of the fact that multiple contributors supply data nominally covering the same area which is used in the production of G-NAF, there is a possibility that there are duplicate addresses which represent the same addressable location. The above example simplistically demonstrates how this could occur. PSMA has developed a sophisticated series of production process to counter issues such as this through the matching of locality, street and address level duplication in the data it receives. The official locality and road datasets provided by state and territory jurisdictions are paramount in enabling this to occur along with feedback generated through the production process and from address users. PSMA continually uses this information to improve the quality of G-NAF. VERSION 3.9 PAGE 26

33 Primary Postcode Changes and the Merge Criteria Whilst described as a primary postcode and despite the fact that many values in the table reflect a postcode, this value is intended to be a unique integer, which separates localities that share a name within a state/territory. The primary postcode field is included in the address merge criteria. On some occasions (e.g. when a duplicate locality name has been changed and the primary postcode is associated with the original locality name which has not changed) the primary postcode may be removed from a locality which will cause a new address_detail_pid to be generated for every address inside the locality. Changes in location and impacts on G-NAF As mentioned above, the identifier often referred to as the G-NAF ID (i.e. address_detail_pid) is the primary key in the core table, which contains most of the address elements but not any explicit location or spatial information. Every address in G-NAF is assigned an address site identifier, which is the link between the attributes and spatial elements of the address. Addresses with an address/parcel level geocode (i.e. level_geocoded_code in [,3,5,7]) reference the same address_site_pid identifier in the associated spatial table (i.e. ADDRESS_SITE_GEOCODE). The address_site_pid persists across G-NAF releases and can be used to identify addresses which have had their location move from one release to another. This requires a query which identifies records from the address_site_geocode table, which has the same address_site_pid across consecutive releases but has a geometry change BUpdate frequency PSMA Australia currently releases G-NAF every quarter in the months of February, May, August and November BMaintenance scope Data for existing objects with changed geometry and/or attributes as well as data for new objects within the release time period are included in the release. VERSION 3.9 PAGE 27

34 9. 9BData Product Delivery PSMA Australia is the crucial link between the supply and demand sides of the market for the fundamental national spatial datasets that it offers under the banner of PSMA Data. The organisation eliminates the difficulties of negotiating multiple license agreements with Australian governments and the problems of integrating the data into a seamless consistent national dataset. Furthermore, the existence of PSMA Australia minimises the duplication of effort within the market for organisations wishing to access national data. Access to PSMA Data is enabled through a network of value-added resellers who are licensed by, and work closely with PSMA Distribution, the wholly owned subsidiary of PSMA Australia. Value-added resellers create many powerful and varied applications that use PSMA Data. PSMA Distribution provides strategic support to value-added resellers to ensure that both the public and private sectors obtain the maximum benefit from the use of PSMA Data. PSMA Australia s website 2TUwww.psma.com.auU2T provides a value-added reseller directory to assist those interested in accessing PSMA Data. Current users of PSMA Data should contact their value-added reseller for clarification or guidance before contacting PSMA Distribution. For further information on accessing PSMA Data, or becoming a value-added reseller contact: PSMA Distribution Unit 6, 3 Canberra Avenue, Griffith ACT 2603 T: F: E: 2TUenquiries@psma.com.au U2TW: 2TUwww.psma.com.au U2T 9. 60BDelivery medium information PSMA Systems is a cutting-edge data platform that has been developed to hold, quality assure and distribute PSMA Australia s suite of national spatial datasets. It streamlines PSMA Australia s data delivery. The core of PSMA Systems is the Integrated Database (IDB), which holds our suite of datasets in one location and within a single environment. PSMA Australia provides data updates to licensees through data download or on disk. This service is supported by a detailed user guide BData importer The purpose of the Data Importer is to make loading the G-NAF address data into a database a simple process. The Data Importer is licensed separately to G-NAF and VARs are encouraged to onlicence it to their clients. Please see Data Importer technical documentation for further details. 9BOverview The Data Importer provides a freely available Open Source relational database and a schema/data loading utility along with G-NAF data. The user is given the opportunity to select whether they install the free database and load the data into this or whether they load the data into their own database. The data load utility handles either option. The only requirement of the data loading utility is that there VERSION 3.9 PAGE 28

35 is a JDBC driver available for the database of choice. JDBC drivers for all popular databases are provided with the Data Importer. Data is extracted from the G-NAF Build database into binary files which can be read by the data loading utility. A binary file format has been selected because of efficiencies in file size and processing speeds. There will typically be a single process for the licensee to run to install the G-NAF data. This process will initiate the schema creation utility and the data loading utility. Once the G-NAF data is loaded into the user s database they will be able to link it directly to their internal datasets or applications as required. Figure 5: Data importer Once the Open Source database software is installed at the users site, there will be no need to do this again; and creating the G-NAF schema in the database is a one-off task. Each subsequent supply of data to the licensee will simply require the execution of a single process distributed along with the data to refresh the users G-NAF database. 92BImport Architectural Components The import process requires a utility to create the G-NAF schema and load the G-NAF data. The DBMS Installer is the installer for the free relational database and is an optional step for installing the G-NAF data. The following diagram shows the components of the import process. Figure 6: Architectural components VERSION 3.9 PAGE 29

36 9.3 62BLicensee installation procedure To install the G-NAF database at their site the licensee will need to follow these steps: Install the Open Source database software or provide connection details for their own database 2 Execute a utility to create the G-NAF schema in the database 3 Load the G-NAF data into their database. Installers are provided for each of the above steps with the emphasis on making the installation process as simple as possible for the licensee. Subsequent re-supplies of the G-NAF data will require the licensee to only run the data load process only. The Java application uses JDBC to load the G-NAF to the databases specified below: Figure 7: Data Importer supported formats Database Type MySQL Oracle Microsoft SQL2000 Server Microsoft SQL2005 Server Comments A free ANSI SQL-92 compliant database. For more information see: 2TUwww.mysql.com ANSI SQL-92 compliant production-level database from Oracle. For more information see HU2TUwww.oracle.com/databaseUU2T H. A production-level database from Microsoft. Is not ANSI SQL-92 compliant. For more information see U2TUwww.microsoft.com/sql/default.aspUU2T H. A production-level database from Microsoft. Is not ANSI SQL-92 compliant. For more information see: U2TUwww.microsoft.com/sql/default.aspUU2T H. PostgreSQL A free open-source ANSI SQL-92 compliant database. For more information see : 2TUhttp:// PostgreSQL 8. to 8.4 have been used successfully with the Importer. ODBC Database Connection The destination database must be ANSI SQL-92 compliant. 93BSupported Platforms of Version.3 of the import utility Oracle MySQL SQLServer 2000 and PostgreSQL Any ODBC connection as long as the destination database is ANSI compliant BUnits of delivery Datasets as prescribed in the License agreement brokered by PSMA Distribution BMedium name Online via PSMA Australia s website or on DVD. VERSION 3.9 PAGE 30

37 9.6 65BDelivery format information 78BPSV files 94BFormat Name: Pipe Separated Value files 95BSpecification: PSV files may be used in relational data base applications (They may be viewed in spreadsheets also). This format provides files with the following extension *.psv 96BLanguage: English 79BLYNX Proprietary Binary files 97BFormat Name: LYNX binary file format 98BSpecification: This format includes files with the following extension: *.lynx 99BLanguage: English 80BOracle Dump 00BFormat Name: Oracle g Dump Format 0BSpecification: 02BA binary-format file created by the Oracle Export utility. 03BLanguage: English 8BOracle Data Pump 04BFormat Name: Oracle g Data Pump Format 05BSpecification: 06BThe Data Pump (dump) file set is made up of one or more files that contain table data, database object metadata, and control information. More information is available from 2TUOracleU2T 07BLanguage: English VERSION 3.9 PAGE 3

38 9.7 Delivery file information PSMA provides 2 types of extracts; Full and Incremental. Full extracts contain all active records for a release. Incremental extracts contain all records that have a DATE_CREATED (active) or DATE_RETIRED (retired) later than the maximum date of the last release. The incremental extracts are only available in the Standard profile. PSMA Australia has implemented the ICSM policy for incremental updates - The quantity and names used are different between those available in the full extracts compared to the incremental extracts. For example, the Suburbs/Localities files associated with NSW for a release would have the following files available in the Standard profile for each type of extract: Full extract NSW_LOCALITY_POLYGON_shp.shp NSW_LOCALITY_shp.dbf Incremental extract NSW_INC_ACTIVE_LOCALITY_POLYGON_shp.shp NSW_INC_RETIRED_LOCALITY_POLYGON_shp.shp NSW_INC_ACTIVE_LOCALITY_shp.dbf NSW_INC_RETIRED_LOCALITY_shp.dbf Date Used For the Incremental Extracts When it is time to create an incremental extract the processing undertaken will extract all records with a DATE_CREATED or DATE_RETIRED later than the maximum date of the PREVIOUS release of this layer. For example, when creating an incremental extract for Suburbs/Localities February 205 it would use the maximum date for Suburbs/Localities November 204 (the previous release). The incremental extract date and the last published date of a release The incremental extract date cannot use the last published date of a release because processing may have commenced on the next release of a layer before the current release is published. This will more than likely happen for layers that get processed first, such as Suburbs/Localities. If PSMA were to use the date the last release was published, then records would be missed that had changed before the last release was published in the incremental extract files. Due to the relationships between the different products means that the same process needs to be applied across all products. VERSION 3.9 PAGE 32

39 Extracts Example NSW November 204 Suburbs/ Localities Extracts NSW_LOCALITY_POLYGON_shp. shp Profile: Standard Jurisdiction: NSW Table: LOCALITY_POLYGON Full/ Incremental: Full NSW_INC_ACTIVE_LOCALITY_POLY GON_shp.shp Profile: Standard Jurisdiction: NSW Table: LOCALITY_POLYGON Full/ Incremental: Incremental (active records) NSW_INC_RETIRED_LOCALITY_POL YGON_shp.shp Profile: Standard Jurisdiction: NSW Table: LOCALITY_POLYGON Full/ Incremental: Incremental (retired records) VERSION 3.9 PAGE 33