Approaching Data Quality

Transcription

1 USAF 2004 GeoBase Compass Conference 18 Aug 2004 Approaching Data Quality Mr. Nathan Lowry HQ AFSPC/CEXI AFSPC GIO

2 Approaching Data Quality Overview DoD Standards and Flexibility Extending Standards to meet needs Values and Objectives Spatial Data Accuracy Interpreting the SDSFIE Metadata Development Quality Assurance Procedures Consumption and Cost Effectiveness 2

3 DoD Installation Visualization Tool (IVT) Supports Base Realignment and Closure (BRAC) Explicit spatial, attribute, and metadata specifications Quality assured and audited for accuracy DRAFT DELIBERATIVE DOCUMENT - FOR DISCUSSION PURPOSES ONLY DO NOT RELEASE UNDER FOIA 3

4 4

5 Standards and Flexibility Advantages Provides a common interpretation of metadata and attribute values Consistent Comparable Allows implementation within a short time period Captures information of greatest interest to a specific audience Measurable quality Disadvantages Interpretations may not have considered all alternatives Language may not be clear, concise, or easily understood May lack some information needed to communicate to a generic audience Standard interpretation may evolve over time 5

6 Extending Standards to Meet Needs Establish local, MAJCOM, or Service-level guidance to address immediate needs Installation Boundaries Generic uses and distribution Lower sensitivity risks associated with BRAC Strategic visualization: Address customer needs Garrison mapping: Address generic needs Address data in many dimensions: Completeness Currency Accuracy Positional Accuracy Attribute Accuracy Geometric representation Logical consistency Metadata Geospatial Data Registration File Format 6

7 Understanding Values and Objectives Values of the customer Efficiency Effectiveness Quality of Service Sensitivity of information Perceptions on Cost Objectives of the customer Intended uses of the data Policy requirements Timeline constraints Desired end-state Term of success Will drive Level of detail needed Level of accuracy needed Amount and length of time 7

8 Measuring Spatial Data Accuracy Deductive estimate Statements based on knowledge of errors occurring during process steps, with assumptions and references to calibration tests Internal evidence Tests based on repeated measure and redundancy Comparison to source Graphic inspections, citing geometric tolerances and registration methods used Independent Source of Higher Accuracy The preferred test for positional accuracy From Basic Implementation of SDSFIE, CADD-GIS Technology Center, ERDC, USACE, July 2004) 8

9 Measuring Accuracy Using the NSSDA Select well-defined points from Dataset being tested for accuracy Independent source of higher accuracy (ie. NSRS) Measure distances (x,y and/or z) Calculate statistics: Standard Error * Root Mean Square Error 9

10 So Accuracy expressed using NSSDA = Standard Error * Root Mean Square Error, or * ( 1 ) 2 + ( 2 ) ( n ) n n in the horizontal plane, and * ( 1 ) 2 + ( 2 ) ( n ) n n in the vertical. 10

11 Stating Accuracy Using the NSSDA Tested to meet meters horizontal accuracy at 95% confidence level." Tested to meet 3.3 feet vertical accuracy at 95% confidence level Compiled to meet 10m vertical and horizontal accuracy at 95% confidence level. 11

12 Establishing Thresholds The critical value in the NSSDA i.e. shall be tested meet or exceed 1m horizontal accuracy at 95% confidence level. Required minimum value for accuracy Cost of Accuracy Value of Accuracy Project or agency defined standards for spatial accuracy Feature-specific Utilities survey vs. biological sample site

13 Interpreting the SDSFIE Understand definitions Read them Understand structure Entity Set, Class, Type, etc. Vendor-specific Understand relationships Refer to data models Understand limitations of an evolving standard Approaching Comprehesive Look for the denotative meaning and intent Recognize that there may be occasional errors Browse by Alias Closest approximation to common names of items Alias to be shown in applications required by the GeoBase Architecture CIP written in CONOPS using alias names 13

14 Implementing the SDSFIE Identify the data to be collected & stored in the database Step # Browse the SDS to identify the Features needed (Browser) 1 2 Create a correlation matrix between your data and SDSFIE Build a filter containing the Features (Filter Maker) 3 4 Generate a SDS-compliant Access Database (Generator) Populate the Access Database (Migrate) Quality check the migration and add metadata Final products (queries, maps, etc.) Step # From Basic Implementation of SDSFIE, CADD-GIS Technology Center, ERDC, USACE, July 2004) 14

15 The Correlation Matrix The AFSPC CIP (Draft) 15

16 Metadata Development Strategies Introduction to GeoBase Technologies Metadata Development Approach (NGA) Inventory All source materials, prioritizing spatial data sets and related files for documentation Identify Whose good will and cooperation you ll need in order to gather and verify all the information for your metadata Select Software that will meet your needs Standardize Your business rules and re-use components via boilerplates Implement and Evaluate Metadata maintenance and development progress From Introduction to GeoBase Technologies, National Geospatial-Intelligence Agency (NGA) (DMS Course #4m/41-716) 16

17 Metadata Development Strategies Margo Berendsen s Metadata for the Rest of Us Approach The Good News There is really only one section of the Standard that is going to take a significant amount of time: Data Quality. The other sections can usually be recycled with minor changes. The Bad News If you do take the time to really look into your data's quality, you'll discover you wish you hadn't But look at it this way: better now than later 2) Data Quality Accuracy Lineage (sources) Process steps (procedure) From Metadata for the Rest of Us, Margo Berendsen, Spatial Data and Visualization Center, University of Wyoming 17

18 Metadata Development Strategies Nathan s Pragmatist s Guide to FGDC Metadata 1) Gather and investigate all the information that you can on the data (documents, subject-matter experts, etc.) 2) At some point, recognize that YOU are the data Expert. 3) Write down (ie. in WordPad) what you feel is the most essential and relevant information for the next guy to know in a coherent narrative. THEN cut-and-paste from your narrative into the metadata editor. 18

19 Metadata Development Strategies Organize your information Write your metadata file Review your file Have someone else review Revise Publish Six Steps for Writing Quality Metadata. From Introduction to FGDC Metadata Workshop,FGDC/NOAA-CSC, Denver Federal Center, Denver, CO, June

20 Metadata Development Strategies FGDC Workbook Metadata entry tool Chocolate Coffee Sense of Humor! 20

21 Metadata Development Strategies Get organized Document your data as you go Write so others can understand Always review your document Keep your readers in mind Write simply but completely Document for a general audience Be consistent in style and terminology Define all acronyms Avoid using jargon Clearly state data limitations From Introduction to FGDC Metadata Workshop,FGDC/NOAA-CSC, Denver Federal Center, Denver, CO, June

22 Quantitative and Qualitative Metadata Quantitative Measures National Standard for Spatial Data Accuracy (NSSDA) Qualitative Value Structure vs. Content Tested with thresholds Tested to meet 0.5 meter horizontal accuracy at 95% confidence level Content driven What is good metadata? Good writing 22

23 Quality Assurance Procedures Develop plans that assure standards for quality are met Consistency Repeatable process Teamed Effort Product Assurance Process Assurance SDSFIE Compliance Written procedures NSSDA Testing Process steps Metadata Requirements Re-evaluate processes Checklist approach Measurable Meets USAF culture Independent Verification Risk-reduction An evolving process 23

24 Consumption and Cost Effectiveness $ Increase Consumption = Increased Accuracy Reduced Cost Cost Benefits of Garrison Mapping Increased Wing Capability Increased quality for Strategic requirements % Accuracy 24

25 References Spatial Data Transfer Standard, USGS, 1988 Geospatial Positioning Accuracy Standards (FGDC-STD ) Content Standard for Digital Geospatial Metadata (CSDGM) Version 2.0 (FGDC-STD ) FGDC Metadata Workbook Version 2.0 (FGDC-STD ) USAF GeoBase Enterprise Architecture, Version 1.0, Headquarters Air Force Geo Integration Office, Jan 2003 USAF Garrison Mapping CONOPS, Version 2.0, Headquarters Air Force Geo Integration Office, Jun 2003 DoD Installation Visualization Tool, Quality Assurance Plan, Version 1.1, IVT Program Office, Headquarters Air Force Geo Integration Office, 31 Dec 2003 DoD Installation Visualization Tool Checklist, Version 1.1, Headquarters Air Force Geo Integration Office, Mar 2004 Introduction to GeoBase Technologies, National Geospatial Intelligence School (NGS), NIMA (DMS Course #4m/41-716) Error, Accuracy, and Precision, Kenneth E. Foote and Donald J. Huebner, Department of Geography, University of Texas at Austin, 1995 Managing Error, Kenneth E. Foote and Donald J. Huebner, Department of Geography, University of Texas at Austin, 1996 Basic Implementation of SDSFIE, CADD-GIS Technology Center, ERDC, USACE, July 2004 Introduction to FGDC Metadata Workshop,FGDC/NOAA-CSC, Denver Federal Center, Denver, CO, June 2004 Train the Metadata Trainer Workshop, FGDC/NOAA-CSC, Denver Federal Center, Denver, CO, June 2004 Metadata Education Project: Data Quality / Error and Uncertainty, University of Wyoming, (date and author not cited) Metadata for the Rest of Us, Margo Berendsen, Spatial Data and Visualization Center, University of Wyoming A Classification System for the National Standards for Spatial Data Accuracy (NSSDA), Gregory Tilley, VARGIS, LCC,(date not cited, available upon request). Development of the National Spatial Reference System, David R. Doyle, National Geodetic Survey (NGS), National Oceanic and Atmospheric Administration (date not cited). How Good Are Your Maps?, Bruce Joffe, GIS Consultants, GeoWorld, February 2002 Map Error and Root Mean Square, Joseph M. McCollum, USDA Forest Service, (date not cited) Metadata in Plain Language, Peter N. Schweitzer, U.S. Geological Survey, Reston, VA Frequently-asked Questions on FGDC Metadata, USGS Conversations with Mr. John Conroy, USGS/NMP Standards Team, August 2, 2004 Conversation with Mr.Joe Chumbley, CH2MHill, Colorado PLS, June, 2004 Conversations with Ed Riegelmann, Scott Beattie, and Joe Goutch, CH2MHill, Conversation with Rick Johnston, AFMC Strategic GeoBase Coordinator,