Real-Time GIS: Applying Real-Time Analytics

Transcription

1 Real-Time GIS: Applying Real-Time Analytics RJ Sunderman GeoEvent Server Product Engineer, Esri Ken Gorton Real-time/Big Data Product Engineer, Esri

2 Agenda Performing Analysis in Real-Time Use Case 1: Identifying Conditions Use Case 2: Finding Patterns in Data Use Case 3: Workforce Tracking Summary & Resources

3 1 Performing Analysis in Real Time

4 ArcGIS Enterprise Web Device Desktop with real-time capabilities Ingest high velocity real-time data into ArcGIS. live features live & historic aggregates & features Perform continuous analytics on events as they are received. Store observations in a spatiotemporal big data store. stream service visualization map & feature service Visualize high velocity & volume data: - as an aggregation - or as discrete features. ingestion analytics ArcGIS Enterprise Notify about patterns of interest. GeoEvent Server spatiotemporal big data store storage

5 Applying real-time analytics GeoEvent Services A GeoEvent Service configures the flow of GeoEvents - The Filtering and GeoEvent Processing steps performed - The input(s) data comes from and the output(s) to which results are sent

6 Out of the Box Esri Gallery Inputs Outputs Applying real-time analytics processors and filters Perform continuous analytics on GeoEvents as they are received using processors and filters. GeoEvent Extension GeoEvent Services Buffer Creator Geotagger Add XYZ Range Fan Convex Hull Creator Difference Creator Envelope Creator Incident Detector Intersector Projector Bearing You can create Ellipse your own processors. Event Volume Control Reverse Geocoder Service Area Creator Symbol Lookup Field Calculator Simplifier Extent Enricher Track Idle Detector Field Enricher Symmetric Difference Field Grouper Unit Converter Field Mapper Track Gap Detector GeoNames Lookup Visibility Field Reducer Union Creator Motion Calculator Query Report

7

8 Fleet Asset Tracking Pseudo Service real-time analytics design

9 Processors Some work only on geometry Some work on geometry and/or attributes Some respond to spatial and/or attribute conditions and generate new messages Processors typically transform the event record being processed in some way

10 Filters Allow event records to pass only if a conditional expression evaluates TRUE The expression can use attributes or geometries you have established as geofences

11 Filters You can use Boolean logic (AND, OR, NOT) to combine expressions

12 Spatial Operators geofence selection vs. spatial operator scope Don t confuse ANY and ALL with the regular expression pattern used to select a set of geofences GEOMETRY INTERSECTS ALL.*/.* GEOMETRY DISJOINT ANY.*/.*

13 Spatial Operators overlapping geofences Intersects Any geofence Intersects All geofences

14 Spatial Operators Spatial operators at 10.2: inside enter outside exit Spatial operators at 10.3 and beyond: intersect touches crosses overlaps disjoint contains equals within

15 GeoFences feature service synchronization Import from a feature service - Reads once (good for static geofences) Synchronize with a feature service - Periodically refreshes to update geofences Event GeoFences Category (e.g. RestrictedZone ) o SecurityPavilion o LoadingDock o ObservationHanger Filtered Event Import Synchronize Feature Service

16 GeoFences stream service synchronization GeoFences are constantly updated - Allows geofences to become dynamic Requires active management and purging of geofences as they expire Event GeoFences Category (e.g. ActiveTracks ) o A12345 o G65190 o K91263 Filtered Event Synchronize Stream Service

17 Dynamic GeoFences

18 Dynamic GeoFences

19 2 Identifying Conditions (demo)

20 Identifying conditions with real-time data real-time analytics design

21 Identifying conditions with real-time data real-time analytics design

22 Field Mapper processor Use a Field Mapper when you need to: - Change the schema or structure of an event record - Translate from one GeoEvent Definition to another - Specify how attribute values map from an inbound to an outbound event record

23 Field Mapper processor TrackID J7890 TrackID J7890 Date Date Sensor 2 Geometry , BatteryLevel Medium Latitude Longitude Incoming Event Mapping Translated Event Category AnkleBraceletGPS Distance 0.01 DurationMin 1.03 SpeedMPH 0.62 CourseDeg Geometry , Category AnkleBraceletGPS

24 Identifying conditions with real-time data real-time analytics design

25 Field Enricher processor Use a Field Enricher when you need to: - Enrich an event record with new attribute values from an secondary source An attribute join is used to retrieve values from the secondary table. The processor retrieves specified data values and then enriches an event record by either appending new fields or writing the data to new fields.

26 Field Enricher processor TrackID V10987 Date BatteryLevel Low Distance Speed 1.2 Course geometry , TrackID NoContact NoEntry K90123 Event Feature Service Temecula gangland V10987 F65432 Pass Christian School Enriched Event TrackID V10987 Date BatteryLevel Low Distance Speed 1.2 Course geometry , NoContact F65432 NoEntry Pass Christian School

27 Identifying conditions with real-time data real-time analytics design

28 Buffer Creator processor Use a Buffer Creator when you need to: - Construct a polygon around an event s point, polyline, or polygon geometry - Enrich an event record by adding a new geometry field - Replace an event record s geometry with a derivative

29 Buffer Creator processor TrackID S90909 Date Buffer Creator TrackID S90909 Date BatteryLevel High Distance 0.2 Event Enriched Event BatteryLevel High Distance 0.2 geometry , geometry , buffer rings" : [ [ [ , ],[ , ] ]]

30 Identifying conditions with real-time data real-time analytics design

31 Intersector processor Use an Intersector processor when you need to: - Generate a geometry representing the intersection between a event record s geometry and a set of specified geofences Geometry processors can either enrich an event record by adding a new geometry field or replace an event record s geometry with a derivative geometry.

32 Intersector processor TrackID S90909 Date Intersector TrackID S90909 Date BatteryLevel High Distance 0.2 Event Enriched Event BatteryLevel High Distance 0.2 geometry rings" : [ [ [ , ],[ , ] ]] geometry , intersection rings" : [ [ [ , ],[ , ] ]]

33 3 Finding Patterns in Data (demo)

34 Finding patterns in real-time data real-time analytics design

35 Finding patterns in real-time data real-time analytics design

36 GeoTagger processor Use a GeoTagger when you need to: - Enrich an event record with the name of a geofence with which the event has a spatial relationship The processor uses a spatial expression to identify related geometries. The name or unique identifier of the related geofences is appended to the event record. A GeoTagger is essentially a spatial join.

37 GeoTagger processor TrackID J7890 TrackID J7890 Date Date Sensor 2 Sensor 2 BatteryLevel Medium Latitude Event Expression Enriched Event BatteryLevel Medium Latitude Longitude Longitude Distance 0.01 Distance 0.01 DurationMin 1.03 DurationMin 1.03 SpeedMPH 0.62 SpeedMPH 0.62 CourseDeg CourseDeg Geometry , Geometry , Category AnkleBraceletGPS Category AnkleBraceletGPS Temecula gangland IsInside Temecula gangland

38 Finding patterns in real-time data real-time analytics design CrimeIDs 19382,21472,39820,

39 Finding patterns in real-time data real-time analytics design

40 Field Calculator processor Use a Field Calculator when you need to: - Calculate new values using data from a received event record - An expression is evaluated and used to calculate the new values - Results can be written to a new field or used to update an existing attribute Expressions can be mathematical, string operations, or function invocations which use regular expressions.

41 Field Calculator processor TrackID V10987 TrackID V10987 Date Date BatteryLevel Low Distance Event Expression Enriched Event BatteryLevel Low Distance 0.02 Speed 1.2 Course geometry , Convert from Feet to Miles Expression: Speed 1.2 Course geometry , Distance / 5280

42 Finding patterns in real-time data real-time analytics design

43 Finding patterns in real-time data real-time analytics design

44 4 Workforce Tracking (demo)

45 Workforce Tracking real-time analytics design

46 Workforce Tracking real-time analytics design

47 Field Splitter processor Use a Field Splitter when you need to: - Split a list of delimited into multiple event records Lists are normally a single delimited string (e.g. comma delimited values). A multicardinal version of this processor can handle more complex lists.

48 Field Splitter processor TrackID V10987 Date BatteryLevel msgcode 1 freetext Low stopped for fuel and lunch Addresses , admin@inspections.com" geometry , Event Expression Enriched Event TrackID V10987 Date BatteryLevel Low msgcode 1 freetext stopped for fuel and lunch Addresses geometry , TrackID V10987 Date BatteryLevel Low msgcode 1 freetext stopped for fuel and lunch Addresses admin@inspections.com geometry ,

49 Workforce Tracking real-time analytics design

50 Track Gap Detector Use a Track Gap Detector when you need to: - Detect the absence of event record reporting - Alert or notify someone that expected data was not received

51 Track Gap Detector Gap Closed Gap Detected TrackID TrackID V10987 V10987 Date Date TrackID TrackID V10987 TrackID V10987 Geometry TrackID V10987 geometry Date V , Date Date Date geometry , geometry , Geometry , trackid trackid V10987 trackid V10987 trackid V10987 gaptrackid V10987 truev10987 gap gap true lastreceived gap true lastreceived false lastreceived lastreceived geometry lastreceived geometry , , geometry , Geometry , geometry ,

52 Workforce Tracking real-time analytics design

53 Incident Detector Use an Incident Detector when you need to: - Detect that a condition has occurred and monitor its duration Uses a filter expression to specify an opening and (optional) closing condition. Maintains state for the duration of the incident, closing the incident when a specific closing condition is observed. Incidents may also close (expire) when no further events are received for the TRACK_ID associated with the incident.

54 Incident Detector g g g g g g TrackID TrackID J7890 TrackID J7890 Date TrackID J7890 Date TrackID J7890 Date TrackID J7890 BatteryLevel Date TrackID J7890 BatteryLevel Date TrackIDMedium J7890 BatteryLevel Date TrackIDMedium J7890 Distance BatteryLevel Date TrackIDMedium J7890 Distance 0.01Medium J7890 BatteryLevel Date Distance BatteryLevel Date 0.01Medium DurationMin Distance BatteryLevel Date DurationMin Medium Distance BatteryLevel DurationMin Distance BatteryLevel Medium 1.03 SpeedMPH DurationMin Distance BatteryLevel 0.01Medium SpeedMPH DurationMin Medium Distance SpeedMPH DurationMin Distance Geometry SpeedMPH DurationMin Distance 0.01 Geometry SpeedMPH , DurationMin , Geometry SpeedMPH DurationMin , Geometry SpeedMPH DurationMin , Geometry SpeedMPH , Geometry SpeedMPH , Geometry SpeedMPH , Geometry , Geometry , Geometry , g g g g Incident Detector Incident Ended GEOMETRY INSIDE AnkleBraceletGPS-Geofences/Temecula Gangland AND TrackID = J7890 Temecula gangland Incident Started id id c982db54- -3bbb61211eb6 id c982db54- -3bbb61211eb6 name id c982db54- -3bbb61211eb6 name id Geofence c982db54- -3bbb61211eb6 Geofence c982db54- -3bbb61211eb6 violation name violation type name Geofence violation type name Cumulative Geofence Cumulative Geofence violation violation type status type Cumulative status type Started Cumulative Ongoing Cumulative status alerttype status Ongoing alerttype status Warning Ongoing Warning Ended alerttype opencondition alerttype Warning opencondition alerttype (INSIDE(AnkleBraceletGPS/Tem Warning (INSIDE(AnkleBraceletGPS/Tem Warning opencondition opencondition ecula (INSIDE(AnkleBraceletGPS/Tem ecula gangland) gangland) AND AND (TrackID (TrackID = opencondition (INSIDE(AnkleBraceletGPS/Tem = J7890)) ecula (INSIDE(AnkleBraceletGPS/Tem J7890)) ecula gangland) AND (TrackID = J7890)) ecula gangland) gangland) AND AND (TrackID (TrackID = = closecondition J7890)) closecondition J7890)) closecondition description closecondition description closeconditionstarted Ongoing at for 7/12/14 last 10 10:54 seconds. AM description Ongoing for last 20 seconds. timestamp description timestamp description Ongoing timestamp Ended at for 7/12/14 last 30 10:54 seconds. AM and definitionname timestamp definitionname incident lasted for seconds definitionname incident incident definitionowner definitionname timestamp definitionowner com.esri.ges.processor/incident incident definitionowner com.esri.ges.processor/incident definitionowner definitionnamedetector/ com.esri.ges.processor/incident Detector/ com.esri.ges.processor/incident incident Detector/ trackid trackid definitionowner J7890 Detector/ trackid J7890 com.esri.ges.processor/incident J7890 geometry trackid geometry , J7890 Detector/ , geometry , duration geometry trackid duration , J duration dismissed duration geometry dismissed false , dismissed false false assignedto dismissed duration false assignedto assignedto note assignedto dismissed false note note note assignedto note

55 Workforce Tracking real-time analytics design

56 Reverse Geocoder processor A Reverse Geocoder is an example of a custom processor - Enriches an event record with the street address nearest the event s location Uses a point geometry from a received event to enrich the event record with new attribute field(s) representing a matched address.

57 Reverse Geocoder processor TrackID J7890 Date TrackID J7890 Date Reverse Geocoder Sensor 2 BatteryLevel Medium Sensor 2 BatteryLevel Medium Event Enriched Event SpeedMPH 0.01 Geometry , SpeedMPH 1.79 Address Avenida Sonrisa Geometry , REST Reverse Geocoder Service Neighbor City Beaumont Subregion Region California Postal PostalExt CountryCode USA Match_addr Avenida Sonrisa, Beaumont, California, Loc_name USA.PointAddress

58 5 Summary & Resources

59 Summary GeoEvent Server real-time analytics for your ArcGIS Enterprise ArcGIS is a dynamic platform that enables continuous analytics and real-time visualization for better understanding of our world The ArcGIS GeoEvent Server allows you to: - know what is happening, as it happens - react and make smarter decisions faster - be notified when interesting events occur

60 Summary self-paced training and resources Step-by-Step Tutorials, free to download - Introduction - Stream services - Spatiotemporal Big Data Store - Notifications Blogs and discussions on the forum - Video recordings of technical workshops -

61 Please Attend Our Other Sessions! GeoEvent Server: An Introduction GeoEvent Server: Applying Real-Time Analytics Real-Time and Big Data GIS: Best Practices ArcGIS and the Internet of Things (IoT) Real-Time and Big Data GIS: Leveraging the Spatiotemporal Big Data Store Tue, 2:30-3:30 pm, Primrose B Tue, 5:30pm-6:30 pm, Primrose A Wed, 10:30-11:30 am, Primrose B Fri, 1:00-2:00 pm, Catalina/Madera Wed, 2:30-3:30 pm, Primrose B Wed, 4:00-5:00 pm, Primrose A Developing Real-Time Web Apps with the ArcGIS API Thu, 9:00-9:30 am, Demo Theater 1: Oasis 1-2 for JavaScript Real-Time GIS: Road Ahead GeoEvent Server: Creating Connectors and Processors Using the GeoEvent SDK Thu, 4:00-5:00 pm, Pasadena/Sierra/Ventura Fri, 8:30-9:30 am, Mesquite B

62 Please Take Our Survey! Download the Esri Events app and find your event Select the session you attended Scroll down to the Feedback section Complete Answers, add a Comment, and Select Submit

63 Esri.com Proceedings page Where users can find PDF'ed slides after DevSummit

64 Questions / Feedback? RJ Sunderman GeoEvent Server Product Engineer, Esri rsunderman@esri.com Ken Gorton Real-time/Big Data Product Engineer, Esri kgorton@esri.com