Standardizing the Geospatial Farm

Transcription

1 Standardizing the Geospatial Farm ESRI User Conference / Jim Koob and Gavin Rehkemper / Version 1

2 Agenda Bayer Crop Science The End Motivation Standardizing the Experiment Sort of Standardizing the Ingest JSON Automating the Analytics Pipeline Service Object Interceptor Page 2

3 Bayer Crop Science Science For a Better Life A world leader in Agricultural products Seeds, Seed Treatments and Crop Protection (synthetic and organic) Studying Effects of Crop Protection products in real world situations Trials on 30 X 30 foot plots does not translate into ½ mile X ½ mile fields Answering questions on product effectiveness: To help growers be more profitable and then grow more crops To help growers be more sustainable lowering or eliminating harmful effects of due to runoff. Page 3

4 The End A Field Trial Management System Systematic collection of quality information directly from growers and agronomists on full scale farms. Providing a point of standardization for data needed to understand the effectivity of Bayer products. Data needed to be standardized for analysis across multiple farms. Page 4

5 Motivation Managing Real World Research Communicating the Trial Design (Protocol) Previously only done at the beginning of the trial Protocols were not sufficiently detailed Managing the Trial Lack of visibility to protocol execution (workflow) Difficult to adapt to changes from environmental conditions Timely Collection of Quality Data Most data did not show up until the end of the season (when growers and agronomists had time to deal with the extra workload) Data from non-study fields was sometimes provided by mistake Data files were missing information (not set on the equipment) Page 5

6 Motivation Water, Water Everywhere, But. Manually Collected Data Field Boundary Field Scouting Soil Cores Soil Chemical Analysis Crop Tissue Samples Equipment Generated Data Planting (Seeding) As-Applied Fertilizer As-Applied Herbicide As-Applied Fungicide As-Applied Pesticide Harvest (Yield) Public Sector Data Multiple Manufacturers used on a Field. Elevation Points Soil Maps Imaging (mostly aerial) RGB and Multispectral NDVI Page 6

7 Motivation Not a Drop to Drink We had more data then we knew what to do with But still did not have enough We were missing context (metadata) to the data files being auto-generated Was it raining the day before harvest? Was the field harvested with one combine or two? If more than one were they calibrated by the same person? What was the target planting rate (if not entered into the planter computer)? Did the grower till this year even though practicing no-till (and for how long)? What type of irrigation is used on the field?. Page 7

8 Motivation And also Supporting a Global Community Different User Types Different Contractual Agreements Different Data Privacy Regulations went with the most restrictive (Europe) Different Languages Page 8

9 Standards Page 9

10 Protocol workflow Organization Lead Agronomist Agronomist 1 Agronomist 2 Farm Owner Tenant Farm Manager 1 Farm Manager 2 Master Data Year Crop Treatment Protocol Author Protocol Organization Farm Field Job 1 Job 2 Job Step 1 Agronomist 1 Farm Mgr 1 Job Step 1 Job Step 2 Agronomist Protocol Author Protocol Template Protocol Instance Page 10

11 Standardizing the Experiment Template everything The protocol (Experiment Design) is a template Protocol Copy Protocol Reuse allows flexibility for each experiment while enabling standardization Page 11

12 Standardizing the Experiment Template everything Each part of the protocol is a template: The Job is a template The job step is a template Reuse allows flexibility for each experiment while enabling standardization Page 12

13 Standardizing the Ingest Templates for Interpolation Each data layer requires a different type of interpolation methodology Creating a Form Optimal interpolation methods can change So the interpolation template needed to be easily configurable Wanted to give researchers the ability to tailor their interpolation methods to the research objectives and source of data collection. Page 13

14 Standardizing the Ingest Adding Context Geospatial Data is not Enough Filling the Form Contextual data varies depending on the job step The seed variety may be very important for understanding the harvest Equipment manufacturer will be important for application, but not for soil sampling. Needed a flexible way to collect any data without making changes to the data model Page 14

15 Standardizing the Ingest Templates for Layer Display Each layer type (feature class) uploaded has a standardized template for the attributes. The template allows data from any source to be compared with similar data from a different source The attributes used to display information for that layer on the base map are indicated by the layer template The template also includes symbology. Page 15

16 Forms Standard JSON JSON is used for: Context Forms templates Context ingestion and storage Layer Mapping templates Localization Support Why: Primary programming language for the application is java script JSON easily parsed by java script PostgreSQL with jsonb extension used for data storage JSON interpreted as a table by PostgreSQL using SQL. Page 16

17 Automating the Analytics Pipeline Interpolation Templates in Action A simple ArcGIS tool (using python) developed to leverage the interpolation templates Analyst selects the field(s) to analyze Sets the interpolation polygon shape and size (square or hexagon) Each data set is then processed using the interpolation template that was chosen for that data set for that step of the protocol All interpolation results are stored into a new feature class that is ready for statistical analysis in R or SAS. Savings in data preparation of 1 to 3 business days depending on the number of data sets in the protocol. Freeing the data scientist to spend their time analyzing instead of preparing data. Page 17

18 Server Object Interceptor Providing Row Level Security Challenge: Providing custom row-level security Previously: Clients making requests Custom Middleware ArcGIS Server Page 18

19 Server Object Interceptor Challenge: Providing custom row-level security Now: Clients making requests ArcGIS Server with SOI Page 19

20 Summary Standardize the experiment by making each protocol a future template Standardize the ingest using a mapping template Standardize the display using the ingest map + standardized color ramps Standardize the context (metadata) using forms for each ingest type Same form is used for the same job step everywhere the protocol is used. Standardize data preparation using templeted interpolation algorithms Standardize the coding and data access using json Page 20

21 Thank you!

22 Server Object Interceptor Page 22

23 Server Object Interceptor Page 23

24 Textslide Text in Arial Regular 18pt First level Second level Third level Fourth level Page 24

25 Textslide Subheadline Text in Arial Regular 18pt First level Second level Third level Fourth level Text in Arial Regular 18pt First level Second level Third level Fourth level Page 25

26 Textslide Text in Arial Regular 18pt First level Second level Third level Fourth level Text in Arial Regular 18pt First level Second level Third level Fourth level Page 26

27 Textslide Subheadline Text in Arial Regular 18pt First level Second level Third level Fourth level Text in Arial Regular 18pt First level Second level Third level Fourth level Page 27

28 Textslide Text in Arial Regular 18pt First level Second level Third level Fourth level Text in Arial Regular 18pt First level Second level Third level Fourth level Page 28

29 Headline Subheadline Page 29

30 Zukunftsgerichtete Aussagen Diese Website / Presse-Information / Präsentation kann bestimmte in die Zukunft gerichtete Aussagen enthalten, die auf den gegenwärtigen Annahmen und Prognosen der Unternehmensleitung von Bayer beruhen. Verschiedene bekannte wie auch unbekannte Risiken, Ungewissheiten und andere Faktoren können dazu führen, dass die tatsächlichen Ergebnisse, die Finanzlage, die Entwicklung oder die Performance der Gesellschaft wesentlich von den hier gegebenen Einschätzungen abweichen. Diese Faktoren schließen diejenigen ein, die Bayer in veröffentlichten Berichten beschrieben hat. Diese Berichte stehen auf der Bayer-Webseite zur Verfügung. Die Gesellschaft übernimmt keinerlei Verpflichtung, solche zukunftsgerichteten Aussagen fortzuschreiben und an zukünftige Ereignisse oder Entwicklungen anzupassen. Page 30

31 Forward-Looking Statements This website/release/presentation may contain forward-looking statements based on current assumptions and forecasts made by Bayer management. Various known and unknown risks, uncertainties and other factors could lead to material differences between the actual future results, financial situation, development or performance of the company and the estimates given here. These factors include those discussed in Bayer s public reports which are available on the Bayer website at The company assumes no liability whatsoever to update these forward-looking statements or to conform them to future events or developments. Page 31

32

33 Content area and guides 5.03 Please restrict your content to this area Page 33