Automating ArcGIS Deployments Using Chef

Transcription

1 Automating ArcGIS Deployments Using Chef Cherry Lin Bill Major David Cordes Slides:

2 Overview What, Why, When? Esri s Chef Cookbooks Easy Cooking Demo Best Practices Case Study: Esri

3 What, Why, When

4 What is Automation and Why The linking of disparate systems and software in such a way that they become self-acting or self-regulating. Reliable, repeatable and testable deployments and upgrades - DEV TEST PROD - Moving between Data Centers - Faster Disaster Recovery Quick configuration with disposable machines and rapid auto-scaling Automate and become more efficient at administration - Focus more on GIS tasks - Prepare for larger scale and clouds - Ready-to-use tools with minimum requirements to start.

5

6 Operate Software Operation (HA, DR) Org Administration Software Configuration Content Management Software Installation Analytics

7 re Automation Software Operation (HA, DR) Software Configuration Software Installation Software Setups, Enterprise Builder, Chef, Cloud formation, Cloud Builder

8 Automation Org Administration Content Management Analytics Python API for ArcGIS, WebGISDR, Admin API, ArcPy, R-bridge for ArcGIS

9 re Automation Software Operation (HA, DR) Software Configuration Software Installation Software Setups, Enterprise Builder, Chef, Cloud formation, Cloud Builder

10 ArcGIS Enterprise Software Setups = ArcGIS Enterprise ArcGIS Web Adaptor Portal for ArcGIS ArcGIS Server ArcGIS Data Store

11 ArcGIS Enterprise Software Setups No need to learn 3-party technology. Deploy exact the way you want Other Esri software Time consuming More manual steps Manual configurations following different docs One time task For customized deployments Not recommended for highly available and scalable deployments

12 ArcGIS Enterprise Builder ArcGIS Web Adaptor Portal for ArcGIS ArcGIS Web Adaptor Hosting Server ArcGIS Data Store (relational + tile cache)

13 ArcGIS Enterprise Builder All-in-one deployment One GUI setup, 5 or 6 parameters Easy to use A lot less configurations One deployment pattern for one machine quick start Not for a system that needs quickly redeployed Not a DR deployment?

14 ArcGIS In the Cloud Enterprise Software Setups AWS - Esri AMIs - Esri CloudFormation Templates - Python and Powershell scripts Microsoft Azure - Esri Images - Cloud Builder Highly recommend using our Images and tools - Customization based on our Images and tools - Details: AWS and Azure sessions

15 ArcGIS Esri ArcGIS Enterprise Chef Cookbooks Software Setups IT standard automation framework Automate all that you can manually do Deploy exact the way you want. You have full control. Other Esri software Require small up-front investments Reliable, repeatable and testable - Documented deployments - High Availability and Disaster Recovery deployments - Move between Data Centers - Potential scale-ups - Automate upgrades

16 re Automation ArcGIS Enterprise Software Setups Enterprise Builder Software Setups Chef ArcGIS Cookbooks Quick start Easy to use Typical deployment pattern Simple deployment Limited control Not for repeatable deployments Reliable Repeatable Testable Documented deployment Full control

17 Esri s Chef Cookbooks

18 What is Chef Chef - One of the most popular IT automation platforms - Supported on Windows and Linux - Provide tools to take on configuration files Chef Components - Cookbook - Attribute - Chef-Server vs. Client How it works - Install ready-to-use cookbooks and define the configuration - Run chef with the configuration file

19 ArcGIS Cookbook Recipe Recipe - Attributes - Action Property Settings: Attributes Run-list

20 Define You Own Attributes

21 Define You Own Run List

22 Steps to get started with a single machine deployment with Chef Check out Esri Github Chef Cookbooks website v.3.1 just released - Install the Chef Client (v or later) Copy ArcGIS Cookbooks to server and unzip to c:\chef - Stage Setups and License Files Edit JSON configuration file to meet your own needs - c:\chef\roles\webgis-windows.json Run chef-solo from command prompt As Administrator - chef-solo -j C:\chef\roles\webgis-windows.json

23 Easy Cooking Demo

24 Supported ArcGIS Software ArcGIS Enterprise - Portal for ArcGIS -ArcGIS Server - Enabled Roles: Hosting, Image Server, GeoAnalytics, GeoEvent Server -ArcGIS Data Store - General, spatialtemporal -ArcGIS Web Adaptor And other Esri Software: - ArcGIS Pro - ArcGIS Insight - ArcGIS Desktop - ArcGIS License Manager

25 Support ArcGIS Enterprise Deployment Patterns

26 Best Practices

27 ArcGIS Distributed Enterprise DeploymentSoftware Setups Running Chef on different machines in order from a workstation machine - Different property files - Runs in order as required by ArcGIS Enterprise Chef Server - Central repository - Assign roles to every machine - Run the recipes/updates accordingly - Setting up -By yourself or -Hosted Solution

28 ArcGIS Upgrade to Enterprise 10.5 or Software Setups Required properties -New Version # -New Setups -New License File Recommend to use Chef deploy before upgrade -Easier to get the properties right

29 ArcGIS Work with Enterprise Other Automation Software ToolsSetups Esri ArcGIS Chef Cookbooks is ArcGIS deployment tool ArcGIS Infrastructure part of your automation Work with GIS workflow automation tools -Python API for ArcGIS -WebGISDR tool Other software deployment automation tools Whole system infrastructure automation -High Availability and Disaster Recovery deployment -Moving to different data centers

30 ArcGIS Chef Deployment Enterprise Options Software Setups

31 ArcGIS Disconnected Enterprise Environment Software Considerations Setups Make sure you have proper Esri license files staged for use Software setups in a common location for access Setting up your own Chef Server Having Chef Client installation staged for installation

32 ArcGIS Tricks and Enterprise Tips Software Setups Consider creating a base image for some environment - Have some processes done, e.g, setups - Have some components baked in, e.g. base map data, security updates, etc. Chef Client version 12 Set password in an environment variable -l debug for debug Attributes value of True or False, no. e.g. configure_autostart : true ArcGIS Cookbooks support enabling hosting server role and GeoAnalytics Server role. Support security configuration for ArcGIS Server.

33 ArcGIS Key Points Enterprise about ArcGIS Software Chef Automation Setups Automate Your Manual Work Doesn t require programming skills Need plan ahead -Design -Configuration -Testing Repeatable -Success in production environment -Less downtime in production -Faster recovery

34 Case Study : Esri

35 ArcGIS Enterprise : New Software Daily Many software changes each day. Daily testing prevents accumulation complexity.

36 Test Automation : Not just automated functional testing Configuration automation Software installed and configured many times per build Many platforms Many software components Many architectures Limited resources

37 Hundreds of Scripts Every tester had their own configuration script - Python -.Net - Java - Perl - Batch files - Etc Wasted effort Hard to maintain

38 ArcGIS Cookbooks to the Rescue Solution: Use our cookbooks Tested Reliable Maintained

39 Challenge #1: But I already have a script. Not invented here Resistance to moving away from existing scripts. Time eventually removed this barrier from most of our test teams.

40 Challenge #2: But I don t have time to learn a new technology... Training In groups One-on-one Troubleshooting alias

41 Outcome: Successful adoption, focus on the functionality Has taken time but we spend Less time maintaining and troubleshooting scripts. More time focused on functional testing.

42 Multi-Machine Configuration Step 1: Prepare the image

43 Multi-Machine Architecture Command Machine Tomcat + Web Adaptor test-datastore.dev.geocloud.com Portal test-portal.dev.geocloud.com Hosting Server test-server.dev.geocloud.com Registered Data Store test-datastore.dev.geocloud.com

44 Baked an Image in Three Easy Steps Install Chef Copy Cookbook System Requirements

45

46

47

48

49

50

51

52 Multiple Machine Automation Step 2: Set the Parameters

53 One Deployment - Many Machines - One Set of Parameters Portal Server Data Store Web Server Parameters

54

55

56

57

58

59

60

61

62

63

64 Multiple Machine Automation Step 3: Create Role-Specific Runlists

65 Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters

66

67

68

69

70 Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters

71

72

73

74

75 Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters

76

77 Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters

78

79 Run Lists - Federation - Back to Portal Portal Server Data Store Web Server Parameters

80

81 Multiple Machine Automation Step 4: Create Shell Script (Batch File)

82 Script Writing for Multiple Machines Command Machine Scripts stored on command machine Copied to test machines at deploy time Run with ssh

83

84

85

86

87

88

89

90 Multiple Machine Automation Step 5: Run Deployment Script

91 Run: Multi-Machine Configuration in 30 minutes

92 Q&A and Slides Slides:

93