Automating Distributed Raster Analysis using the Image Server REST API. Jie Zhang Zikang Zhou Demo Theater 2 - Oasis 1

Transcription

1 Automating Distributed Raster Analysis using the Image Server REST API Jie Zhang Zikang Zhou Demo Theater 2 - Oasis 1

2 What is Distributed Raster Analysis? From 10.5, ArcGIS has a new way to create and execute spatial analysis models and image processing chains which leverage distributed storage and analytics - Raster Analysis works with your existing GIS data and imagery - register your data and go, importing existing data to distributed storage is not mandatory - Raster Analysis can optimize your data for distributed analytics - import your data into ArcGIS distributed storage which further improves the scalability of distributed analytics - Raster Analysis is designed to scale with your organization s demands - scale up to get the job done, scale down when resources are no longer needed

3 Raster Analysis Foundational Concept Use optimized raster format for distributed reading/writing Complete analysis task for data too big for single desktop machine - Meet time constraint (Month -> Weeks -> Days -> Hours -> Minutes) Dynamic Raster Models Geoprocessing Models Server-based Distributed Raster Analytics with Distributed Raster Data Storage Web GIS Layers Portal (persistent) distributed analytics with optional distributed storage for even greater scalability

4 Image Server and Raster Analysis Deployment Assign Raster Analysis role to federated Image Server ArcGIS Pro Webmap Portal Hosting image services Raster data store Image Server (Raster Analysis) Write cloud raster format raster datasets Hosting Server ArcGIS Data Store (Relational and Tile Cache) Write and publish feature output File share storage Cloud storage

5 Image Server and Raster Analysis Cloud Deployment Deploy Raster Analysis on premise or in the Cloud - Cloud Formation Templates for Amazon AWS - Manual federation and assign role - Use S3 as Raster Data Store - Cloud Builder for Microsoft Azure - Use Azure Blob Storage as Raster Data Store ndex.html

6 Distributed Raster data format and Raster Data Store NOT part of ArcGIS Data Store! Raster data store is used by Raster Analysis services to store output imagery in Cloud Raster Format (CRF) - File share - Amazon S3 - Microsoft Azure Blob Storage If multiple raster store configured, random one is picked for a task Feature service output will be stored in Hosting Server s relational database

7 Raster Analysis System Services Use Raster Analysis Tools GP Service REST API

8 Raster Analysis REST API Raster Processing and Analysis modeling achieved using Raster Function JSON object.

9 Other Raster Analysis Clients Raster Analysis services are also accessible through: - ArcGIS Pro/Desktop - Portal Webmap - ArcGIS API for Python ArcGIS Pro Webmap Desktop ArcGIS API for Python Portal through Geoprocessing environment settings

10 Raster Analysis REST API Demo Zikang Zhou

11 Let s submit a Raster Analysis RESTful request first

12 ArcGIS Enterprise (Portal for ArcGIS federated with ArcGIS Server) Fx Slope + Clip

13 ArcGIS Enterprise (Portal for ArcGIS federated with ArcGIS Server) How to submit a raster analysis job to the Raster Analysis server to run, e.g., Generate Raster: - Input: - (functionarguments) Parameters that will be used in rasterfunction, e.g., an input image service or a Portal s Imagery layer item - Output: - (outputname) An existing Portal s imagery layer item or corresponding Portal s hosted image service; can also be a new name for Portal s imagery layer item and Portal s hosted image service - How to process in a Generate Raster job: - (rasterfunction) A raster function or raster function template in JSON - (rasterproperties) Raster key metadata properties to be set on output - (context) Processing environment settings, e.g., output raster spatial reference - (token) If authentication is at Portal for ArcGIS tier, token should be generated under a 2 nd level Publisher or Administrator or customized roles with certain privileges enabled Reference:

14 ArcGIS Enterprise (Portal for ArcGIS federated with ArcGIS Server) What you can expect after submitting a raster analysis job: - A JSON object containing the jobid and current jobstatus Keep querying job status with jobid : Have you finished? Have you finished?... Once finished, access the output: putraster - A JSON object containing the output Portal s imagery layer itemid and corresponding Portal s hosted image service URL

15 rasterfunction parameter: How to construct a JSON raster function chain Raster Analysis service is powerful: Math Abs Arithmetic Band Arithmetic Calculator Divide Exp Exp10 Exp2 Float Int Ln Log10 Log2 Minus Mod Negate Plus Power Round Down Round Up Conditionals Con Set Null Square Square Root Times Bitwise And Bitwise Left Shift Bitwise Not Bitwise Or Bitwise Right Shift Bitwise Xor Boolean And BooleanNot Boolean Or Boolean Xor Equal To Greater Than Greater Than Equal Is Null Less Than Less Than Equal Not Equal ArgStatistics Cell Statistics Statistics ACos ACosH ASin ASinH ATan ATan2 ATanH Cos CosH Sin SinH Tan TanH Correction Apparent Reflectance Geometric Correction Speckle Filtering (Lee,Frost,Kuan) Data Management & Conversion Raster to Vector Vector to Raster Colormap Colormap To RGB Complex Grayscale Remap / Reclass Spectral Conversion Unit Conversion Vector Field LAS to Raster LAS Dataset to Raster Clip Composite Extract Bands Mask Mosaic Rasters Rasterize Features Reproject Visualization & Appearance Contrast and Brightness Convolution Pansharpening Resample Statistics and Histogram Stretch Interpolation Natural Neighbor Nearest Neighbor Inverse Distance Weighted Empirical Bayesian Kriging Swath Surface Generation & Analysis Aspect Curvature Elevation Void Fill Hillshade Shaded Relief Slope Viewshed Analysis: Density Kernel Density Analysis: Overlay Weighted Sum Weighted Overlay Analysis: Zonal Zonal Statistics Analysis: Band Math & Indices NDVI / NDVI Colorized SAVI / MSAVI / TSAVI GEMI GVI (Landsat TM) PVI Tasseled Cap (Kauth-Thomas) Binary Thresholding Analysis: Image Segmentation & Classification Segmentation (Mean Shift) Training (ISO, ML, Support Vector Machine,Random Trees) Classification Python Custom Algorithms Submit a job based on a single function or a function chain combining many functions Reference:

16 rasterfunction parameter: How to construct a JSON raster function chain Name of the raster function, e.g., Clip All parameters for Clip raster function: input raster + how to clip it Reference: The Clip function s input raster is the result from Slope function More nested raster functions can be chained here in parallel with, e.g., zfactor. Parameter name is still Raster. If omitted, it uses the raster in functionarguments specified when submitting RA job.

17

18 Please Take Our Survey! Download the Esri Events app and go to DevSummit Select the session you attended Scroll down to the Feedback section Complete Answers, add a Comment, and Select Submit