Enterprise GIS Architecture

Transcription

1 Enterprise GIS Architecture Sandy Milliken (Enterprise Systems Architect) Ty Fabling (Enterprise Systems Architect) 19 February, 2009 FedUC

2 Enterprise GIS Architecture ESRI Enterprise Consulting Group: Ty Fabling (Enterprise Systems Architect) FedUC

3 Primary GIS Architectural Goals (prioritize & balance) Functionality Performance Security Policy Standards COTS Versions Make it Work Make it Faster Make it Secure Make it Compliant FedUC

4 Other GIS Architectural Issues (prioritize & balance) Scalability (Ease of handling more users or higher volumes) Mobility (Field Editing Disconnected Data) Availability (Dependence on both Internal and External Network) Reliability (HA / Eliminate single pts of failure / Apps, Data & Network) Quality (of Service, Data, etc) Stability (Software bug free / Network up-time time) Lowest Cost Maintainability (Central vs. Distributed Applications vs. Data) Flexibility (Total Life-Cycle Ease of Upgrading to Future Versions) FedUC

5 Technology Maturation Process Productivity Gains Standardization More Manage Risk Less Innovation FedUC

6 Basic Architectures Single Tier Presentation Business Logic Data Management Multi Tier Client Presentation Application Server Business Logic Data Server Data Management FedUC

7 GIS Common Solution Architecture Presentation Tier Desktop Web/Rich Internet Applications Mobile Services Tier Exposure of information Enterprise Messaging Messaging methods, protocols Application Tier Session management Business Logic User interaction Persistence Tier Databases, Files Enterprise Messaging FedUC

8 Data Center Architecture Production Environment Develop and test Environment Firewall Technology Web Access Tier Enterprise Tier Component Application Tier Enterprise Web Application Servers Windows Terminal Server ArcGIS Desktop Desktop Applications Proxy Servers Edge Servers Network Traffic Load Balancing Traffic Accelerators Websphere Enterprise Applications Weblogic Enterprise Applications Enterprise Portal Technology Optional Firewall Technology Optional Firewall Technology Web Application Servers ArcIMS ArcGIS Server Image Server Switched IP Network Data Tier Storage Network Oracle Sql Server DB2 Informix Development File Server Geodatabase DBMS Servers Switched Storage Network Spatial Files - Coverages -Shapes- -Images Geodatabase - SDE Schema - User Schema - Vector Data - Image Data Attributes -Customers - Facilities -Doc Development -Develop -Training -Demo Test Servers Test FedUC

9 What demands does GIS place on computing infrastructure? Processing Intensive (CPU) Graphics Intensive (GPU) Memory Intensive (RAM) Network I/O Intensive Disk I/O Intensive FedUC

10 GIS Business Planning Business Processes (User workflows) Application Needs (User productivity tools) Infrastructure Requirements Data Requirements (Information resources) The three key architecture factors are the location of: Users Applications GIS Data FedUC

11 Enterprise GIS Architectures Key Components: Users Applications Data Only Two Primary Architectures: Centralize in the Data Center (Web Services, Citrix) Distribute to Regional Offices (Data Replication) Hybrid Solution: New Mobile ADF SmartClient FedUC

12 Three Elements & Two Networks Three Major Elements: 1. Users 2. Applications 3. Data Where do each reside (Data Center, Regions, etc)? What is the network connectivity like between these three? LAN vs. WAN Network Bandwidth and Latency Frequency of Outages or Drops 1 Users 3 Applications 5 Data Thin Connection High Bandwidth 2 User-to-App Connectivity 4 App-to-Data Connectivity FedUC

13 Centralized Infrastructure Centralized Systems Centralized Application near Centralized Data (Web Apps & Citrix) User User User Program Office Regional Office Thin Client Connections Thin Client Connections Data Center Application Data Field Office Thin Client Connections User Thin Client Connections Citrix Server Farm Web Servers Oracle ArcSDE Lab Field User FedUC

14 Distributed Infrastructure Distributed Systems Distribute Replicated Data (9.2 Replication) Regional Office Field Office User & App. User & App. Local Server Data ( Replica ) User & App. Replicate Deltas Only Replicate Deltas Only Data Center Data ( Master ) Replicate Deltas Only User & App. User & App. Region Field User Master Database File Server FedUC

15 Centralized vs. Distributed Systems Centralized Systems Uses Citrix or Web Service apps Single point-of of-access for data, centralized management Dependant upon network for Application & Data Reduced WAN & LAN traffic Increased O&S at Data Centers Decreased O&S at Region Distributed Systems Uses desktop apps on local workstation Multiple replicas of data at Regions, synched w/ master Less dependant upon WAN Increased LAN traffic Decreased O&S at Data Centers Increased O&S at Regions High Points: Minimizes resource requirements at regional centers Centralized database and system administration integrity Low Points: Network is a single point of failure High Points: Editing data offline (during outages) Data distribution (i.e. outsourcing to data vendors, outside agencies) Low Points: Increased resources, O&S support maintaining client software and data at regional centers, etc. Posting/synchronizing local data with master database(s) data currency FedUC

16 Two Geo-Enterprise Information System Approaches Enterprise GIS GIS Centric Core GIS technology Spatially-enabled Information System GIS Enabled Enterprise Applications that include GIS Information Geo-centric workflows Spatially-enabled lines of business-centric workflows Advanced GIS applications Data creation and maintenance, analysis and modeling IT-based Simple GIS functionality Data exploitation, routing, geocoding, simple map IT-based Examples Land records, asset management, facility sitting, corridor analysis Run by GIS and IT professionals Examples Financial system, facility management, executive information system, customer care, first responder Run by IT professionals FedUC

17 Enterprise Architecture GIS Centric GIS Enabled Desktop B r o w s e r Clients Application Servers GIS ERP Web Server CRM CIS SFA Data Servers FedUC

18 Expanding Deployment Options Remote Access Client Options Mobile ADF Supports Windows XP and Mobile Operating Systems Requires development ArcPad Complete Application Browsers ArcGIS Server and ArcIMS ArcEngine Rich Client Development environment Can Consume web services Windows Terminal Services / Citrix Full Rich Client Functionality Typically utilized by Authors ArcMap Full Rich Client Functionality Can consume web services and new GeoData disconnected Editing Line between Mobile Users and Remote Clients is blurring Technology Phone PDA Tablet Laptop Workstation Network Increasing Bandwidth / Connectivity / Richness Software Mobile ADF ArcPad Browser ArcEngine Citrix ArcMap FedUC

19 The SmartClient Paradigm ArcGIS Mobile SmartClient Paradigm For Sometimes Connected Systems.NET based replication Masters & Replicas with only deltas moving across the wire FedUC

20 Remote Access Client Options ArcGIS Mobile in the Enterprise Mobile Client Mobile Server Existing Systems Map, MapActions Web services GDB Replication MapCache Staging of data SDE ServerConnection ArcMobile GPRS EDGE HSDPA Cradle ArcGIS Server Enterprise Systems User Experience Business Logic Data & Web Svc Management Security Connectivity Web Services Web Services & Data Server Manager FedUC

21 Remote Access Client Options Mobile ADF Application and Data Deployment Application Development ArcGIS Server Data Extraction Visual Studio Web Service MapCache Extractor Posts Edits Refresh Data Connection Deployment Mobile Applications Day to Day synchronization from the Field Editing Application Viewing Application FedUC

22 GIS Is Deployed In Many Ways Departmental Enterprise 1 Centralized GIS 2 Traditional File Based Distributed Data 3 4 Emerging FedUC

23 GIS Software Evolution What are the lessons learned and where is the technology going? >> 10x Faster Hardware >> 10x Faster Hardware 1980 s 1990 s 2000 s 2010 s > 2x Process Load > 2x Process Load > > 2x 2x Process Process Load Load >> 10x Faster Technology Change >> 10x Faster Technology Change >> >> 10x 10x Faster Faster Technology Technology Change Change Tightly Scripted Software Code Object Relational Architecture Service Oriented Architecture ARC/INFO 1982 ArcView GIS 1992 Coverages Librarian Shapefiles ArcStorm Platform Performance Change Internet ArcGIS Desktop Web Applications - ArcInfo ArcEditor ArcView 2000 MapObjects ArcGIS Server 2004 ArcIMS ArcGIS Engine 2004 Spatial Database Engine (SDE) ArcSDE Simple non-versioned file types - Geodatabase Versioning (ArcSDE) Mobility - Distributed Geodatabase Replication Scripts Objects Services Departmental Centralized Federated FedUC

24 SOA - GIS Functional View Desktop Web Map Explorer Mobile Open Standards Enterprise Integration Use Serve Author ArcGIS Desktop Users FedUC

25 SOA - GIS Functional View 3-D Viewers Client Visualization Consume Services 2-D Viewers COP Presentation Tier (multiple viewers) Enterprise Messaging Services Server Geoprocessing Visualization Catalog Services Globe Services Map & Chart Services Geo- Processing Services Image- Processing Services Open Web Services Tracking & RSS Services Serving/ Publishing Tier Desktop Geodata Management Data Editing Analysis Published as a Service Authoring Tier FedUC

26 ArcGIS Platform for a Complete GIS ESRI Enterprise GIS Architecture Client Visualization Consume Services 2-D Viewers 3-D Viewers COP Mobile Presentation Tier (Any Viewer) Use GPT Server Geoprocessing Visualization Metadata Catalog Services Map & Chart Services Geo- Processing Services Image Services Open Web Services Tracking & RSS Services Serving & Publishing Tier Desktop Geodata Management Data Editing Analysis Published as a Service Authoring Tier ArcGIS Desktop (ArcMap, ArcCatalog) Image Service Definition Editor FedUC

27 SOA Overview Definition A method of building business applications that utilize common services to support business functions Service Directory Find / Details Publish Service Consumer Bind / Invoke Service Provider FedUC

28 Web Services Overview Core Components WSDL SOAP UDDI These 3 technology standards, coupled with service-oriented design principles, form a basic XML-driven SOA Service Description WSDL Web Services Description Language Defines all details about a service UDDI Service Protocol SOAP How to format XML documents for transmission between applications Service Discovery UDDI Universal Description, Discovery, and Integration Not a requirement Well proven for single Enterprise systems WSDL Enables discovery of Describes Binds to Web Services Is accessed using Enables Communication between SOAP FedUC

29 What Supports Web Services? 2nd Generation Web Services Many still being finalized or have competing standards WS-Security Security and BPEL are exceptions Ultimately incorporated into the other 3 solutions UDDI Provides a Protocol for Enables discovery of Is accessed using WSDL Binds to SOAP Describes the Service for Describes the Service for WS- Coordination Describes Manages Context Across uses Web Services Enables Communication between Provides End-to to-end Security for uses WS-Security Provides a Protocol for Manages Context for Provides a Protocol for Enables Distributed Transactions for Provides a Protocol for Provides Guaranteed Delivery for uses BPEL Provides a Protocol for WS- Transaction WS-Reliable Messaging uses WS-Policy orchestrates governs FedUC

30 SOA Infrastructure Connects Service Consumers with Service Providers May be used to communicate with Service Directories May be implemented using a variety of technologies FedUC

31 Enterprise SOA Framework Clients Multi-Channel Access Types of Applications Client Apps Web Applications Web Services Data Sources (Producers) Unifying Components Web Portals Common Viewing Interface ESB Common Messaging Framework Objects Common System Components Different Communication Paths may be utilized FedUC

32 GIS SOA Components FedUC

33 How does ESRI fit into SOA? Other Web Services and many others FedUC

34 How does ESRI fit into Enterprise SOA? ArcGIS Explorer FedUC

35 Secure Implementation Fundamental Tradeoff Secure You get to pick any two! Usable Cheap FedUC

36 Simplifying Geodatabases ArcGIS New Support for File, Personal and Work Group Geodatabases New GDBs are: Fast Simple Scalable Open Cross Platform* ArcGIS Clients **Replication: SDE 9.2 Files 9.3 (One One-way Read-Only Only) Personal (JET Engine) Shapefiles New GDBs 9.2+ File-based* 9.2+ ArcSDE** Desktop Edition *FGDB Supported on Windows, Linux or UNIX 9.2+ ArcSDE** Workgroup Edition Embeds ArcSDE w/ SQL 9.2 & Oracle 9.3 ArcSDE** Enterprise Edition ArcSDE** SQL Server Oracle DB2 9.3) Informix 9.3 FedUC

37 Managing Distributed Data ArcGIS Replication of Geographic Information Changes Only Updates (Deltas) Synchronized Periodically (Configurable) Works on the Web or Courier (i.e.( CD) Client to Client (not DB to DB) Only Supported on SDE Versioned RDBMS Supports Collaborating Agencies Co-Data Producers Central & Regional Offices Mobile Units Central Hierarchy Read / Write One Way Two Way Read / Write National State Read Only ArcIMS Local... Key Technology For GeoWeb FedUC

38 ArcGIS 9.2 Supports Oracle Spatial Three Ways with New Oracle SQL_Type ArcSDE is How ArcGIS Accesses And Maintains Data in RDBMS ArcGIS SDE Can Be Deployable as App Server (Performance/Scalability) *SQL *Any GIS application can now query GIS features stored using the new ISO SQL_Type directly without requiring ArcSDE Other DBMS Binary SDO Type Oracle ISO Type Three Storage Types are Supported Binary SDO Type ISO SQL Type Using Multiple Methods & Techniques for Many Large Customers FedUC

39 Integration of Time New Ways to Manage, Analyze & Visualize Geography Extended Data Model Tools for Manipulation Query Change Analysis Iterative Processing Visualization Animation Charting Tracking Analysis Current Date View Control Room Model Simulation Multi Dimension Data Sets (NetCDF) Files T x y Archive/History Date Retire Real Time Sensor Network T 1 Future History / Archiving Past Present Mobile Stationary... New Opportunities FedUC

40 ArcSDE Components ArcGIS Application ArcSDE Driver ArcSDE Executable DBMS SDE Schema User Schema(s) ArcSDE Schema License Code Note: ArcSDE executables are included in Direct Connect API FedUC

41 ArcSDE Components ArcGIS Application ArcSDE API 9.1 ArcSDE Executable DBMS SDE Schema User Schema(s) ArcSDE Schema License Code Note: ArcSDE executables are included in Direct Connect API FedUC

42 ArcSDE Components ArcGIS Application ArcSDE API 9.2 ArcSDE Executable DBMS SDE Schema 1 User Schema(s) ArcSDE Schema SDE Schema (n) User Schema(s) License Code Note: ArcSDE executables are included in Direct Connect API FedUC

43 Multi-Tiered SDE Architecture Two-tier Two-tier Three-tier SDE geodatabase Server SDE Direct Connect SDE Application Server Client Tier or ArcObject or ArcIMS ArcObject or ArcIMS SDE (DC) ArcObject or ArcIMS SDE Connect DB Client SDE Connect Application Server Tier Direct Connect SDE GIOMgr SDE** [gsrvr] DB Client ** Remote Connection SDE GIOMgr Database Server Tier SDE* [gsrvr] (ASC) RDBMS [SDE schema] [user schema] RDBMS [SDE schema] [user schema] ASC - Application Server Connect * Local Connection FedUC

44 ArcIMS Component Architecture Web Applications WA Connectors Web Server Applications (ColdFusion/ASP/.NET/.Servlet Engine...) CF ActiveX.NET Link Servlet Java WMS WFS CS-W OAI-PWH Service Manager SM Map Request Image Virtual Server Map Request Map Request Application Server Feature Virtual Server ArcMap Virtual Server Map Request Spatial Services Data Source SS DS Threads Threads Threads Spatial Server Image Feature ArcMap Extract Query Geocode Database Connections Data Server FedUC

45 ArcGIS Server Component Architecture Web Applications WA Web Server Web Application Server Applications (.Net or JSP) Web Services (SOAP) Application Developer Framework (ADF) runtime AGS Licensable Components ADF Map Request Map Request Map Request Map Request Service Manager SM Server Object Manager (load controller) SOM SOC SOC Spatial Services Data Source SS DS Pooled Non Pooled Instances Container Machine MapServer GeocodeServer ArcGIS Server Objects ArcSDE Connects Data Server (Geodatabase Server) SOC SDE FedUC

46 Single-tier Platform Configuration Standard Configuration High-Availability Configuration Network Load Balancing GIS Server GIS Server GIS Server WA WA WA SM SM SM Load Balancing SM SS SS SS SS SS DS DS DS Workgroup Server with SQL Express Enterprise Server with SQL Express FedUC

47 Two-tier Platform Configuration Enterprise Server with Supported Database Standard Configuration High-Availability Configuration Network Load Balancing GIS Server WA GIS Server GIS Server WA WA SM SM SM Load Balancing SM SS SS SS SS SS Cluster Address DS DS DS Primary Data Server Failover Data Server Common Storage Array FedUC

48 Three-tier GIS Platform Configurations Separate Web and Spatial Servers with Supported Database High-Availability Configuration Standard Configuration Network Load Balancing Web Server WA Web Server WA Web Server WA SM SM SM Load Balancing SM SS Map Server Container Machine SM Load Balancing SS Map Server Container Machine SS SS Map Server Container Machine Cluster Address SS SS Map Server Container Machine DS DS DS Primary Data Server Failover Data Server Common Storage Array FedUC

49 Web Services Components in DMZ WA Access Web Server WA Map Server Container Machine SM Client Browser WWW Data Replication Data Server SS Data Server DS Secure Network DMZ FedUC

50 Map/Data Server on Secure Network (Service Manager inside Firewall) HTTP Access** SOAP/XML Map Server Container Machine SM SS Data Server WA SM Access Web Server WA Output File WA Access Client Browser DS Disk Mount* WWW Secure Network DMZ * Disk mount can be replaced with output streaming ArcIMS: Output streaming requires JSP or.net client ** HTTP service required for ArcGIS Server to avoid DCOM port administration. FedUC

51 Web Services with Proxy Server in DMZ Web Server WA Map Server Container Machine Output File WA Access Proxy Server Reverse Proxy SM SS PS Access Client Browser Data Server DS WWW Secure Network DMZ FedUC

52 ESRI Enterprise Support Resources System Design Strategies On-Line Document on ESRI Web Site ( Dave Peters Book: Building a GIS - published Aug 2008 CPT- Capacity Planning Tool (Excel Worksheet) Enterprise Consulting Group System Architecture Design Classes (Available around the World) Enterprise Architecture Consulting Services EEAP ESRI Enterprise Advantage Program BPAP ESRI Business Partner Advantage Program FedUC

53 Enterprise GIS Governance Application Working Group Application Specialist Chairman Application Development Team Steering or Policy Committee Executive Sponsor, Department Managers Status review, general oversight, policy formulation, finance and budget decisions, growth and expansion of participant base, approval of special license agreements, product pricing, provide direction to the GIS departments, legal obligations. Technical Coordinating Committee GIS Manager, Technical Lead, Application Specialist, Data Specialist, IT Lead (platforms and networks) Provide technical direction and standards, recommend projects and studies, recommend database/application development, address system integration/expansion issues, conduct user training. Power User Group Power user representatives Data Management Working Group GIS Data Specialist Chairman Data Management Team End User Group End user representatives IT Infrastructure Working Group IT Lead Chairman IT Operations Team FedUC

54 Building a Qualified GIS Staff FedUC

55 Managing the Enterprise GIS Maintaining Agile Systems Strategic Plan Strategic Plan Deployment Authorization Deployment Authorization Spiral Spiral An Enterprise GIS is never done! FedUC