Using and Managing Raster Data in Server Applications

Transcription

1 Using and Managing Raster Data in Server Applications Peter Becker Feroz Abdul-Kadar ESRI Developer Summit

2 Schedule 75 minute session minute lecture minutes Q & A following the lecture Cell phones and pagers Please complete the session survey we take your feedback very seriously!

3 Introductions Who are we? ArcGIS Image Server (Product Manager and Developer) Who are you? ArcGIS Image Server Developers? Not Developers, but interested in automating service creation? Experience with image processing? Experience with programming?

4 Topics ArcGIS Image Server: Brief Overview Customization Options Raster Process Definition Custom Raster Types and Formats Custom Raster Processes ESRI Developer Summit

5 ArcGIS Image Server - Overview Fast, Dynamic Image Distribution and Processing No need to pre-process process imagery On The Fly Mosaicking and Processing Create multiple imagery products Author: Image Service Definition Add Rasters Build, Compile ISCDef Serve: Use: Publish ISDef ArcGIS Desktop 3 rd Party CAD/GIS/Imaging ESRI Developer Summit

6 Image Services New to ArcGIS Server at 9.3 Desktop Arc Engine 3 rd Party Web WebMap.NET/Java Google Maps Microsoft Virtual Earth OGC WCS Enables Raster Datasets to be directly published 1 Raster Dataset = 1 Image Service More efficient than Map Service Client can change compression Capabilities: WMS, WCS, REST, KML Image Service can be used as Rasters for processing etc Arc Map Arc Globe SOAP ArcGIS Explorer SOAP REST Mashup WMS WCS Image WMS WCS KML Image Service ArcGIS Server OGC WMS KML Google Earth Raster Mosaic Mosaic Tile Rectified Scene ISCDef (req. Image Server) ESRI Developer Summit

7 ArcGIS Image Server 9.3 Desktop Arc Engine 3 rd Party Web WebMap.NET/Java Google Maps Microsoft Virtual Earth OGC WCS As an Extension (+stand alone) Publish ISCDef as an Image Service Provides OTF processing and mosaicking Large Numbers of rasters New: Recompute Footprint Seamline Generation Tiled Raster Faster JPEG2000. Arc Map RPC Arc Globe SOAP ArcGIS Explorer SOAP REST Mashup WMS WCS Image WMS WCS KML IS Direct Image Service Connect ArcGIS Server Image Server Rectified Scene None Rectified Mosaic Tile OGC WMS KML Google Earth Raster Mosaic 7

8 Customization options Why customize? You have existing catalog or table with georeferencing etc The satellite you use is currently not supported You want to automate a workflow You have proprietary image format You have your own image processing to be implemented You have your own sensor model Being Covered today: Raster Types: Defining all properties of a raster Raster Formats: How pixels are read Radiometric Processes: Processes that do not change pixel location Geometric Processes: Processes that do change pixel location ESRI Developer Summit

9 ArcGIS Image Server Process Chain Core of how Image Server works Output Image Pixels to be processed identified Pixels read Passed though processes Enhance Georeference Mosaicked Passed through Service process Apply codec Transmitted Service Process Mosaic Process Mosaicked Image Processed Raster Processed Raster Processed Raster Raster Process Raster Process Raster Process Input Raster Input Raster Input Raster

10 Authoring Image Service Definition Extract raster properties Optimize service overviews Compile Use Serve Image Server \Servicename.ISDef Properties Boundary ArcMap Service Editor Author Compiled Service Definition Footprint Service Definition Service Overviews Process Definitions Seamline Raster Data

11 Raster Process Definition (RPDef( RPDef) Encapsulates raster dataset properties Pointer to Pixels (only reference) Metadata Processing Instructions Footprint Geometry ESRI Developer Summit

12 Processes within an RPDef RPDef may contain multiple rasters (e.g. overviews) Selected based on pixel size Read using Format Reader Pixels pass through raster + dataset processes Processed Raster Dataset Raster Dataset Process Query PS Raster Process Raster Process Raster Process Format Reader Format Reader Format Reader Raster 1 Raster 2 Raster 3 ESRI Developer Summit

13 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

14 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

15 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

16 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

17 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

18 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

19 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

20 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

21 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

22 Structure of an RPDef XML based Raster Dataset Properties Footprint SRS Metadata Process Chain Raster Source Properties Process Chain Format info

23 Adding Rasters Options Use Existing Raster Type Create RPDef XML Implement an RPDef Generator Use a Template RPDef ESRI Developer Summit

24 Using Existing Raster Type Demo ESRI Developer Summit

25 Creating RPDef XML files RPDef data structure is relatively simple Steps Use scripting tools to generate RPDef XML files Use Raster Process Definition Raster Type in Service Editor Advantages Minimal programming effort required Use existing scripting tools Disadvantages Need to know all parameters Not an integrated workflow ESRI Developer Summit

26 Adding RPDef XML files Demo ESRI Developer Summit

27 Implementing an RPDef Generator Write.NET code to handle typical flow: Prompt for raster files Prompt for processing parameters Parse input file list to generate RPDefEntry objects Use RPDefEntry objects and input parameters to generate RPDef XML (in batches of 100) Advantages Fully integrated workflow in Service Editor Disadvantages Programming effort required Writing RPDef XML files might slow the adding process for very large numbers of rasters ESRI Developer Summit

28 Implementing an RPDef Generator Steps: Create a.net DLL (in VB, C++, C#) Implement the IRPDefGenerator interface Install the signed DLL in GAC Create an RPDefGen XML file to describe the raster type Install RPDefGen in <ArcGIS-Home>\Image Server\XADefs\RPDef Generators\. ESRI Developer Summit

29 IRPDefGenerator Interface

30 The RPDefGen XML Simple XML to define the RPDef Generator First point of interaction between Image Service Editor and the Raster Type ESRI Developer Summit

31 Service Editor Raster Type Dialog RPDefGen XML RPDef Generator Factory RPDef Generator RPDef ESRI Developer Summit

32 Implementing an RPDef Generator Demo ESRI Developer Summit

33 Using Template RPDefs Footprint table points to RPDef Use single RPDef with variables Steps Define variables in an RPDef created by Service Editor Create Footprint table with required fields Raster source field points to template RPDef Advantages No programming effort required Very useful for extremely large datasets Provides a database-centric view Disadvantages Need to hand-modify RPDef XML Not an integrated workflow ESRI Developer Summit

34 Using Template RPDefs Demo ESRI Developer Summit

35 Footprint Geometry Dataset footprint Defined by the GML in the RPDef (only when adding) Then defined by the geometry in the footprint layer Update Footprint Geometry Recompute footprint by geometry Computes ground coordinate for pixels around edge Recompute footprint by radiometry Compute ground coordinates by analyzing NoData areas of raster How to use it: Service Editor toolbar (9.3) ISCommand ISCommand (ComputeFootprint*.ISCmd) Note: Documented API not available at 9.3 ESRI Developer Summit

36 Updating Footprint Geometry Demo ESRI Developer Summit

37 Raster Formats Reads pixel blocks Completely independent of associated geometry Optimized Fast Minimize memory Minimize reading of headers Defined by nodes in RPDef XML: <FormatName>: Raster Format to use <FormatInfo>: Nodes passed to Format <Source>: Source path/string ESRI Developer Summit

38 Raster Formats Options RAW Many formats are raw pixels stored on disk Optimized for accessing raw pixel data (in BIP, BIL, or BSQ) Set: AORaster Set: NCols, NRows, SkipBytes, PixelType, BitDepth, NBands,, Interleave, ColorSpace, TileSize, ByteOrder, TreatAsFloat, ApplyOffsetScale, FloatOffset, FloatScale, ApplyConvert Works with most ArcGIS-supported formats Generally slower than GDAL or a custom format AIS 9.3 doesn t t fully support overviews via AORaster GDAL Wide range of supported formats Cannot write custom GDAL drivers for use in AIS (in 9.3 BETA) Custom Raster Formats Write DLL to specific interface ESRI Developer Summit

39 Custom Raster Formats Designed specifically for Image Server Simpler interface, less development work compared to writing a custom ArcGIS raster format or a GDAL driver Needs programming expertise and effort ESRI Developer Summit

40 Custom Raster Formats Steps Implement a DLL exporting C-callable C functions as described in: <ArcGIS-Home>\Image Server\Developer Kit\Raster Formats\Interface\FormatCoreSDK.h" Place DLL in: <ArcGIS-Home>\Image Server\Raster Formats\Plugins\ Place XML describing the implementation in: <ArcGIS-Home>\Image Server\Raster Formats\Plugins\ ESRI Developer Summit

41 Custom Raster Formats Demo ESRI Developer Summit

42 TiledRaster New in ArcGIS Image Sever 9.3 Enables large number of tiles as single raster Defined by TRDef (Tiled Raster Definition) Rasters must form a grid May overlap slightly Structure: -SRS - Layers - Layer : Pixel Size Range - Tiles - Tile : Individual Tile

43 Raster Processes What s s a process? Radiometric / Geometric Process Chain Every process is a part of a chain Processed data and transformed geometry pulled across Optimized for in-memory processing and transfer of BIP data How do you add a process? ESRI Developer Summit

44 Raster Processes Demo adding processes to a dataset or a raster Demo adding processes to the service. Demo changes in the RPDef and ISDef XML. ESRI Developer Summit

45 Custom Radiometric Process Steps Implement a DLL exporting C-callable C functions as described in: <ArcGIS-Home> Home>\Image Server\Developer Kit\Raster Processes\Interface Interface\CustomProcess.h" Place DLL in: <ArcGIS-Home> Home>\Image Server\Raster Processes\ Create Process.XADef XML and place it in: <ArcGIS-Home> Home>\Image Server\XADefs XADefs\Raster Processes\" ESRI Developer Summit

46 Custom Process UI What it does: Allows users to modify processing parameters Keeps UI separate from the core implementation of process Allows for an integrated workflow in Service Editor Steps: Create a.net DLL Implement an object based on IRasterProcessUI Install the signed DLL in GAC Modify corresponding Process.XADef with info required to create this object. ESRI Developer Summit

47 Custom Radiometric Process Demo Watermarking custom process and UI. ESRI Developer Summit

48 Custom Geometric Process Steps Implement a GeoTransformer DLL by exporting C-C callable functions as described in: <ArcGIS-Home> Home>\Image Server\Developer Kit\GeoTransformers GeoTransformers\ Include\GeoTransformers.h GeoTransformers.h" Place DLL in a location visible to the LoadLibrary Func. Use the GeoTransDef XML in the Warp Process. Warp is only valid as a dataset or raster process. ESRI Developer Summit

49 Summary Multitude of methods to import raster data into an image service definition. Separate roles with clear UE / performance objective: Raster Type Raster Format Raster Radiometric Process Raster Geometric Process Raster Process UI ESRI Developer Summit

50 In Conclusion Other recommended sessions and meetings Developing with Rasters in ArcGIS Thu 8:30am Smoketree A - E 2 nd Offering: Using and Managing Raster Data in Server Applications Thu 1:30pm Primrose A Check out ArcGIS Image Server Help <ArcGIS-Home>\Image Server\Developer Kit\Help\ImageServer_DeveloperGuide.chm All sessions are recorded and will be available on EDN Slides and code will also be available Please fill out session surveys! Tech Talk Still have questions? Straight After This Session! 1.Tech talk, Demo Theatres, Meet the Team 2. Ask a Developer link on web page ESRI Developer Summit