TAMING THE MODERN DATA CENTER. A Hybrid Talk for a Hybrid World

Transcription

1 TAMING THE MODERN DATA CENTER A Hybrid Talk for a Hybrid World

2 SETH

3

4 DC EVOLUTION How did we get here?

5 RISING DATACENTER COMPLEXITY DC

6 RISING DATACENTER COMPLEXITY DC

7 RISING DATACENTER COMPLEXITY DC VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM

8 RISING DATACENTER COMPLEXITY DC VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM

9 RISING DATACENTER COMPLEXITY DC DNS CDN Database

10 RISING DATACENTER COMPLEXITY DC-01 DC-02

11 RISING DATACENTER COMPLEXITY DC-01 DC-02 VM VM VM VM VM VM VM VM

12 RISING DATACENTER COMPLEXITY IaaS PaaS SaaS

13 RISING DATACENTER COMPLEXITY

14 TAMING THE DC Deployment + Maintenance

15 PREVIOUSLY The APUD cycle

16 ACQUIRE PROVISION UPDATE DESTROY

17 ACQUIRE PROVISION UPDATE DESTROY U G VENDOR

18 ACQUIRE PROVISION UPDATE DESTROY U G U U VENDOR DC OPS

19 ACQUIRE PROVISION UPDATE DESTROY U G U U U U VENDOR DC OPS SYSADMIN

20 ACQUIRE PROVISION UPDATE DESTROY U G U U U U U U VENDOR DC OPS SYSADMIN DC OPS

21 ACQUIRE PROVISION UPDATE DESTROY c c c c WEEKS DAYS DAYS DAYS VENDOR DC OPS SYSADMIN DC OPS

22 PRESENTLY The elasbc compute and _aas era

23 ElasBc Compute ACQUIRE PROVISION UPDATE DESTROY c c c c WEEKS DAYS DAYS DAYS

24 ElasBc Compute ACQUIRE PROVISION UPDATE DESTROY c c c c WEEKS DAYS DAYS DAYS

25 ElasBc Compute ACQUIRE PROVISION UPDATE DESTROY c c c c MINUTES DAYS DAYS SECONDS

26 CapEx # OpEx #

27 _aas

28 ConfiguraBon Management ACQUIRE PROVISION UPDATE DESTROY c c c c MINUTES DAYS DAYS SECONDS

29 ConfiguraBon Management ACQUIRE PROVISION UPDATE DESTROY c c c c MINUTES DAYS DAYS SECONDS

30 ConfiguraBon Management ACQUIRE PROVISION UPDATE DESTROY c c c c MINUTES MINUTES SECONDS SECONDS

31 SaaS ProliferaBon ACQUIRE PROVISION UPDATE DESTROY ACQUIRE PROVISION UPDATE DESTROY

32 RISING DATACENTER COMPLEXITY DC DNS VM VM VM VM CDN VM VM VM VM Database

33

34 WHY? What was our original goal?

35 EFFECTIVELY DELIVER AND MAINTAIN APPLICATIONS

36 MOVE FAST AND DON T BREAK THINGS

37 h-ps:// d250n10lsq5j8r.cloudfront.n et/assets/atlas/runame- develop- icon- color- ebf4ccf5f2ca3513e495bccb Consistent e9320d45.png h-ps:// d250n10lsq5j8r.cloudfront.n et/assets/atlas/runame- deploy- icon- color- 1c39f618b3b5ead204 Start and configure servers dda4646bb2410e.png / services DEVELOP DEPLOY MAINTAIN Update servers or applications Shareable Readily Available High production parity Deploy and run application Reconfigure, feature flag Monitor health Orchestrate complex changes

38 h-ps:// d250n10lsq5j8r.cloudfront.n et/assets/atlas/runame- develop- icon- color- ebf4ccf5f2ca3513e495bccb e9320d45.png h-ps:// d250n10lsq5j8r.cloudfront.n et/assets/atlas/runame- deploy- icon- color- 1c39f618b3b5ead204 dda4646bb2410e.png DEVELOP DEPLOY MAINTAIN

39 TERRAFORM Build, Combine, and Launch Infrastructure

40 MOTIVATION Why Terraform?

41 How do I provision resources? compute? storage? network?

42 How do I manage resource lifecycles?

43 How do I balance service providers providing core technology for my datacenter?

44 How do I enforce policy across all these resources?

45 How do I automate and share those configurabons?

46 TERRAFORM'S GOAL

47 PROVIDE A SINGLE WORKFLOW

48 WITH A UNIFIED VIEW

49 USING INFRASTRUCTURE AS CODE

50 THAT CAN BE ITERATED AND CHANGED SAFELY

51 CAPABLE OF COMPLEX N-TIER APPLICATIONS

52 DIGITAL OCEAN DROPLET WITH DNS USING DNS SIMPLE resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }

53 DIGITAL OCEAN DROPLET WITH DNS USING DNS SIMPLE resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }

54 DIGITAL OCEAN DROPLET WITH DNS USING DNS SIMPLE resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }

55 DIGITAL OCEAN DROPLET WITH DNS USING DNS SIMPLE resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }

56 HUMAN-FRIENDLY CONFIG* * JSON-COMPATIBLE FOR NON-HUMANS

57 VCS-FRIENDLY FORMAT

58 ENTIRE INFRASTRUCTURE... IN A SINGLE TEXT FILE

59 TERRAFORM GRAPH

60 COMPOSE N-TIERS / PROVIDERS

61 TERRAFORM PROVIDERS

62 SINGLE INTEGRATION POINT

63 EXPOSE ("PROVIDE") A RESOURCE

64 EXPOSE ("PROVIDE") A RESOURCE aws_instance dnssimple_record

65 CRUD API

66 PLUGGABLE FOR INTEGRATIONS

67 C "LAYER CAKE" C VM VM VM PROVIDER PER LAYER IAAS (OpenStack, etc) UNIFIED CONFIGURATION PHYSICAL

68 $ terraform apply

69 All logos and trademarks are copyright their respective owners

70 TERRAFORM PLAN

71 $ terraform plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>"

72 $ terraform plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>"

73 $ terraform plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>"

74 $ terraform plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>"

75 backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>" ttl: "" => "<computed>" type: "" => "A" value: "" => "${digitalocean_droplet.web.ipv4_address}"

76 backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>" ttl: "" => "<computed>" type: "" => "A" value: "" => "${digitalocean_droplet.web.ipv4_address}"

77 SHOWS YOU WHAT WILL HAPPEN

78 EXPLAINS CERTAIN ACTIONS

79 PREVIOUSLY?

80 STILL UNCERTAINTY

81 FUTURE OPS Managing tomorrow s infrastructure

82 DEPLOY IMMUTABLE INFRASTRUCTURE

83 Mutable Infrastructure CONFIDENCE CHANGES

84 Mutable Infrastructure CONSISTENCY INTERATIONS

85 Immutable Infrastructure CONSISTENCY INTERATIONS

86 IMMUTABLE INFRASTRUCTURE IS FASTER

87 IMMUTABLE INFRASTRUCTURE ALLOWS FOR GREATER PARITY

88 IMMUTABLE INFRASTRUCTURE NEEDS AUTOMATION

89 PACKER Automated Golden Images (and more)

90 MACHINE IMAGES

91 YUCK IMAGES?

92 WHY HAVE WE BEEN GENERALLY AGAINST MACHINE IMAGES?

93 GOLDEN IMAGES USED TO BE THE WAY

94 QUARTERLY, UNCHANGED, AND BLESSED IMAGE

95 CHANGES WERE SLOW AND FRUSTRATING

96 TOOLING WAS NOT MATURE COMPARED TO TODAY

97 MODERN CONFIG MANAGEMENT CHANGED THAT

98 OPS WITHOUT MACHINE IMAGES IS LIKE APPLICATIONS WITHOUT BINARIES

99 ApplicaBon Lifecycle

100 Source Code Binary ApplicaBon Lifecycle

101 Source Code Binary liba 1.0 libb 1.0 libc 1.0 ApplicaBon Lifecycle

102 Source Code Binary liba 2.0 libb 1.0 libc 1.0 ApplicaBon Lifecycle

103 Source Code Failure liba 2.0 libb 1.0 libc 1.0 ApplicaBon Lifecycle

104 Mutable Server Lifecycle

105 New Server Ready Server Mutable Server Lifecycle

106 New Server Ready Server Packages Network CM Changes Mutable Server Lifecycle

107 New Server Ready Server Package Changes Network CM Changes Mutable Server Lifecycle

108 New Server Failure Package Changes Network CM Changes Mutable Server Lifecycle

109 New Server Failure Packages Network Unreliable CM Changes Mutable Server Lifecycle

110 New Server Failure Packages Network CM Changed Mutable Server Lifecycle

111 Machine Image Lifecycle

112 New Server Ready Server Machine Image Lifecycle

113 New Server Ready Server Machine Image Lifecycle

114 PACKER BUILD

115 EMBRACES CONFIG MANAGEMENT

116 TRANSITIONS FAILURES FROM RUN-TIME TO BUILD-TIME

117 ENFORCES PARITY WITH STAGING

118 AND EVEN DE ELOPMENT

119 WHAT DOES THE FUTURE LOOK LIKE?

120 OPERATORS M PRODUCTION DNS Load Balancer AMI DEVELOPERS [] Deploy

121 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] Deploy

122 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] Deploy

123 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] Deploy

124 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] Deploy

125 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] U Deploy

126 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] U Deploy

127 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] U Deploy

128 OPERATORS M U PRODUCTION DNS Load Balancer AMI DEVELOPERS [] U

129 Service Discovery HTTP + DNS

130 Host & Service Level Health Checks

131 K/V Store HTTP API

132 SETH QUESTIONS?