GEOINDEX - A NEW GEOLOCATION METHODOLOGY

Transcription

1 GEOINDEX - A NEW GEOLOCATION METHODOLOGY Matthew Hudnall Presentation to the 2014 GIS-T Conference Burlington, VT May 8, 2014

2 CENTER FOR ADVANCED PUBLIC SAFETY Works with: State, federal and local agencies Commercial enterprises Researchers Information technology innovation About 125 software developers About 80 different projects Mix of faculty, students and professional staff Many projects have a GIS component

3 CAPS TECHNOLOGY Numerous live production systems: Electronic citations (AL, MS, AR) Electronic vehicle crash reporting Electronic I/O reports Electronic ambulance run reports Commercial vehicle enforcement systems Criminal data query systems (NCIC, LETS) Data analysis and geographic information systems (several states) Authentication management for all AL law enforcement State Vehicle Titling System State Vehicle Registration Repository State Vehicle Online Insurance Verification System State Food Assistance Application System State Children s Insurance Application System Many other systems deployed to date

4 MOBILE OFFICERS VIRTUAL ENVIRONMENT MOVE ecitation ecrash ULTRA Uniform Crime Reporting LETSgo NCIC Mobile Automatic Vehicle Locator (AVL) License Plate Reader (TRACE) ASPEN Commercial Vehicle Information Field Reporting/ SAR Computer Mag Strip/Bar Code Scanner Printer GPS

5 IOS & ANDROID DEVELOPMENT DMV application Personalized tags Find DMV MOVE for ios/android Nlets/NCIC prototype Mobile mapping software Moving toward ecite, ecrash and other formsbased applications

6 SOME USAGE STATISTICS Mobile officer laptop suite (managed installs): MOVE 16,288 ecite 13,420 ecrash 15,177 ULTRA 11,546 Vehicle registrations: Titles: 90M records Growing by 520,000 per month 9M applications 23M titles 150,000 titles per month

7 DATA ANALYTICS Major applications areas Enforcement Engineering Both GIS and non-gis analytics

8 ADVANCE Law enforcement dashboard ecite ecrash eweight Crime Reporting Deep Analysis & Search Mapping HTML 5 Runs on any platform, including mobile

9 CRASH DASHBOARD

10 CRASH DATA POINTS

11 CRIME HEAT MAP

12 GEOCODING Geocoding is the process of deriving a geographic coordinate from textual input, typically a street address.

13 GEOCODING Continual Batch As data comes into a system, addresses are Geocoded Nightly, quarterly or otherwise bulk processing of addresses

14 GEOCODING METHODS Local Processing Data is processed server side High volume / throughput Web Service / Remote Processing Data is transmitted to a remote server for Geocoding. Centralized/Cloud/SOA approach, but requires lower volumes of transaction sizes

15 PROPER GEOCODING Steps for good Geocoding: Correct the input address Validate the corrected address Geocode the corrected/validated address against the best line work available Store the latitude, longitude and quality of the geocode match

16 GEOCODING PROBLEMS Results are stored in isolated data silos on a per-source/application basis For large datasets, processing can take hours or days Effort is duplicated dozens of times across multiple systems for the same addresses

17 COMMON RAW ADDRESS SOURCES Incident/Offense Reports Crime Address Victim Address Perpetrator Address Witness Address Vehicle Data Registrant Address Owner Address Crash Reports Crash Address Driver(s) Address DL Address Sex Offenders Probation/Parole Corrections Etc.

18 WEB SERVICE SPEEDS

19 LOCAL PROCESSING SPEEDS Architecture Interactive Batch ArcGIS Desktop Subsecond 2,250,000/hour (625 per second) ArcGIS Server Web client ArcGIS Server Local client < 2 seconds 15,000/hour per process (4.1 per second) Subsecond 2,000,000/hour per process (555 per second)

20 GEOINDEX Optimize Geocoding times by caching past results Distribute cache to other agencies/partners Geocoding Engine is NOT needed by partners, only the cache is needed Results of past Geocodes are made available to all Geocoding by SQL Joins to cache on raw address Can Geocode millions of records per second!

21 GEOINDEX ALL addresses are stored, even the bad ones The index gets very large, but is easily handled by modern data storage means For Alabama, 40 Million addresses cover: Every vehicle registered in past 10 years Every vehicle title in the past 10 years All drivers license records

22 GEOINDEX FORMAT RawAddress HouseNumber PreDir StreetName StreetSuffix PostDir City State ZipCode Zip4 County Latitude Longitude GeoData MatchDate Source MatchScore Column Format Varchar(100) Primary Key Varchar(10) Varchar(10) Varchar(30) Varchar(10) Varchar(10) Varchar(30) Varchar(2) Varchar(5) Varchar(4) Varchar(20) Varchar(18) Varchar(18) Geography Datetime Varchar(20) Int

23 GEOINDEX RAW FORMAT Street, City, State, Zip 500 North Ripley Street, Montgomery, AL, North Ripley St, Montgomery, AL, N Ripley Street, Montgomery, AL, N Ripley St, Montgomery, AL, North Ripley, Montgomery, AL, North Ripley, Mont, AL, Etc.

24 SINGLE GEOCODE PROCESS Raw Address Data Address is sent to GeoIndex Web Service Service checks GeoIndex cache Match on Raw data AccuMail Correction ESRI Geocode on Navteq data Addresses corrected, validated, geocoded and stored Address can now be mapped Data is cached server and client side

25 BATCH GEOCODE PROCESS Nongeocoded Data Addresses corrected, validated, geocoded and stored Data is rejoined to SQL SQL Join to GeoIndex Unmatched Addresses sent to geocoder Data can now be mapped

26 CENTRAL MAPPING ENGINE Central Mapping Server Only ONE server needed per state Accumail address correction/validation Sync services for client agencies Core Geolocation web service Agency Local Copy of DB Local Match Service Local DB Sync Service Agency Local Copy of DB Local Match Service Local DB Sync Service Agency Local Copy of DB Local Match Service Local DB Sync Service

27 GEOINDEX SUMMARY A GeoIndex is simple to implement and can drastically improve geocoding performance for future geocodes. GeoIndexes can also be distributed to partner agencies for instant geolocating without the need for a geoprocessing engine.

28 THANK YOU Q&A SESSION Matthew Hudnall, Senior Associate Director