Docker & Chef. Containers. Containers and Docker 8/4/14. Linux containers (LXC) Single operating system Multiple isolated, working configurations

Size: px

Start display at page:

Download "Docker & Chef. Containers. Containers and Docker 8/4/14. Linux containers (LXC) Single operating system Multiple isolated, working configurations"

Theodore Poole
5 years ago
Views:

configurations Containers and Docker Open-source

1 Docker & Chef Containers Linux containers (LXC) Single operating system Multiple isolated, working configurations Containers and Docker Open-source engine for container management Docker registry for sharing containers 1

Virtual Machines vs. Docker https://www.docker.

2 Virtual Machines vs. Docker Chef & Docker Create & Deploy Docker Containers Configure and protect containers Manage host platform Docker & Chef knife container docker build build a Docker container docker init setup Dockerfile 2

3 Docker & Chef Demo Do I still need configuration management? Yes! Image sprawl is a thing Typical Infrastructure Graphite Nagios Rails App Memcache Postgres Slaves Postgres Master 3

4 New Compliance Mandate! Graphite Nagios Rails App Move SSH off port 22 Lets put it on 2022 Memcache Postgres Slaves Postgres Master 6 Golden Image Updates Graphite 1 2 Nagios 3 Rails App 4 Memcache 5 Postgres Slaves edit /etc/ssh/sshd_config 6 Postgres Master 12 Instance Replacements Graphite 1 2 Nagios Rails App Delete, launch Repeat Typically manually Memcache Postgres Slaves 12 Postgres Master 4

5 Done in Maintenance Windows Graphite 1 2 Nagios Rails App 8 9 Memcache Don t break anything! Bob just got fired =( 12 Postgres Slaves Postgres Master Different IP Addresses? Graphite Nagios Rails App Memcache Postgres Slaves Invalid configs! Postgres Master Chef & Docker Create & Deploy Docker Containers Configure and protect containers Manage host platform 5

6 Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Test Driven Development For your infrastructure code! Typical Development Workflow Typical Software Development Workflow: Write Source Compile Source Deploy Artifacts Write Tests Run Tests 6

Typical Development Workflow Source Code The recipe for a computer program Edited directly Managed via Source Control Software Typical Development Workflow Compiler Takes source code and converts it

7 Typical Development Workflow Source Code The recipe for a computer program Edited directly Managed via Source Control Software Typical Development Workflow Compiler Takes source code and converts it into executable programs Artifact Typical Development Workflow Artifacts are executable programs created by compilers. Compiled artifacts cannot be edited directly. Source code must be changed and recompiled to produce a new 7

Typical Development Workflow Testing Write tests to verify that code works as intended Run tests at different stages of the code lifecycle to ensure correctness Typical Development

8 Typical Development Workflow Testing Write tests to verify that code works as intended Run tests at different stages of the code lifecycle to ensure correctness Typical Development When developing Workflow software, most time isn t actually spent coding 10-second changes to source code can take minutes to vet Compiling code Deploying code Writing & Running tests 8

9 Typical Development Workflow Things that are fun: Fun! Designing programs! Writing source code! Boring! Things that are boring: Compiling code Deploying artifacts Running tests Cookbooks Are Source Code Chef Cookbook source code is just like the source code for other programs. Changes are tracked and versioned with source control software such as git Code is inert until it has been deployed Code can be used to re-create the desired system configuration from scratch Cookbooks Aren t Pre- Compiled Because Ruby is an interpreted language, Chef code is not compiled until it is executed by the Chef-Client There is no lengthy compilation step between committing Chef Cookbooks and deploying them X 9

Depending entirely on context, Chef Cookbooks can be either source code OR artifacts. This means something! This is important!

10 Cookbooks Are Artifacts Chef Cookbook artifacts are just like the artifacts for other programs. Cookbooks are deployed Cookbooks are released with versions Cookbooks are immutable once deployed Cookbooks are packaged for distribution X Cookbooks can depend on one another Cookbooks Are Artifacts Depending entirely on context, Chef Cookbooks can be either source code OR artifacts. This means something! This is important! Close Encounters of the Third Kind, copyright 1977, Columbia Pictures Cookbooks Are Artifacts This sounds obvious, but it s easy to overlook: Treating cookbooks only as source code or only as artifacts will lead to long-term problems when using Chef cookbooks to manage infrastructure. Because Chef cookbooks are source code and artifacts, depending on the context, they have all the properties of each. 10

11 Cookbooks Are Artifacts A good Chef workflow treats cookbooks appropriately Cookbooks depending are on context Cookbooks are source! Versioned Stored in a source repository Inert while in development artifacts! Versioned Deployed to a Chef Server Executable by Chef-client once deployed Keeping Repos in Sync A brief aside about deploying cookbook artifacts: Common question: How do I automatically keep my Chef Code Repository and my Chef Server Artifact Repository in sync? Answer: The same way that one would automatically keep a code repo and an artifact repo in sync with any other software released to production. Deploy with care! Cookbooks as Source Code Cookbooks are Source! knife cookbook create Code: recipes/default.rb Version & dependencies: metadata.rb git commit git tag X 11

rb chef-client Testing Cookbooks Three things necessary to test a Chef cookbook: 1) A place to store the

12 Cookbooks as Artifacts Cookbooks are Artifacts! knife cookbook upload Immutable once deployed, can only be replaced Version & dependencies: metadata.rb chef-client Testing Cookbooks Three things necessary to test a Chef cookbook: 1) A place to store the cookbook artifact. 2) A Chef-client with access to the cookbook. 3) A target server running the same OS as production. X Traditional Cookbook Development: First Chef Run 12

Traditional Cookbook Development: Subsequent Runs Boooooooring Too much time doing paperwork Less Fun! THIS SUCKS! X More Boring! vi recipes/ something.

13 Traditional Cookbook Development: Subsequent Runs Boooooooring Too much time doing paperwork Less Fun! THIS SUCKS! X More Boring! vi recipes/ something.rb knife cookbook upload sudo pkill -USR1 chefclient #%$#%$ something broke, let me do that all again Not enough time doing fun stuff! Writing recipes Developing for Chef: Rapid Iteration More Fun! X Less Boring! Less time waiting around for cookbook deploys and Chef runs More frequent testing Better code Business needs met more quickly 13

14 Test Kitchen Serverspec describe "awesome appliance repair" do! it "has a webserver listening on port 80" do!! end! expect(port('80')).to be_listening! it "does not display the default apache home page" do! expect(command("curl return_stdout /Ubuntu/!! end! it "displays the home page" do! expect(command("curl return_stdout /Awesome/! end!! end! ChefSpec require 'chefspec'!! describe 'awesome_appliance_repair_chef::default' do! let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }!! it 'installs foo' do! expect(chef_run).to install_package('apache2')! end! end!! 14

15 Foodcritic $! foodcritic.! FC002: Avoid string interpolation where not required:./recipes/ default.rb:104! Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Chef DK The Chef Development Kit 15

16 Chef DK Chef ohai chef-client chef-apply chef-solo knife Berkshelf Test Kitchen ChefSpec Foodcritic Chef the command line utility chef generate cookbook recipe attribute template file lwrp chef gem chef verify chef exec Chef DK The Workflow Install Vagrant Install Chef DK Generate cookbook, recipe, etc. Use default test-kitchen 16

17 Demo Time Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Chef Metal machine resource and provisioners 17

18 Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Knife 18

19 Demo Time Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Chef vs How does Chef compare to? 19

Chef vs http://ronblacks.com/wp-content/uploads/2013/07/153359_60_redo_web.jpg Choose Your Own Adventure What is DevOps?

Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven

20 Chef vs Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Infrastructure as Code Programmaticall y provision and configure components

21 Infrastructure as Code Treat like any other code base 91 Infrastructure as Code Reconstruct business from code repository, data backup, and compute resources 92 Infrastructure as Code Programmatically provision and configure components Treat like any other code base Reconstruct business from code repository, data backup, and compute resources

22 Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs Version Control Which Version Control System Do You Use? cp foo foo.bak cp foo{,.`date +%Y%m%d%H %M`} 22

23 Which Version Control System Do You Use? cp foo foo.bak cp foo{,.`date +%Y%m%d%H %M`-`$USER`} Use Tools that Reflect & Reinforce Your Culture Open Communication Incentive and Responsibility Alignment Respect Trust Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 23

24 Incident Management and Post Mortems Shit Happens! Have a plan Identify an Incident Commander First responder Decides what to do next Coordinates resources Can hand off command Communicates Status Not about rank Post Mortems Learning Opportunity Blameless here to learn, not blame EVERYONE is invited Establish the timeline Identify contributing factors Describe customer impact Open tickets for remediation tasks Prioritize those tickets 24

8/4/14 Post Mortems There is no such thing as a root cause Focus on MTTD and MTTR MTBF holds much less weight Choose Your Own Adventure What is DevOps?

25 8/4/14 Post Mortems There is no such thing as a root cause Focus on MTTD and MTTR MTBF holds much less weight Choose Your Own Adventure What is DevOps? Docker & Chef Infrastructure as Test Driven Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Podcasts & Podcasting Ask me anything Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 25

Stages of Learning Shuhari First learn, then detach, and finally transcend shu - "obey" - traditional wisdom ha - "detach" - break with tradition ri - "separate" - transcend Choose Your Own Adventure

26 Stages of Learning Shuhari First learn, then detach, and finally transcend shu - "obey" - traditional wisdom ha - "detach" - break with tradition ri - "separate" - transcend Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 26

27 Hiring DevOps Who here cares? Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 27

28 Podcasts & Podcasting DevOps-related Podcasts Other podcasts I listen to The Ruby Rogues Ruby5 Software Defined Talk This American Life The Truth 28

29 Podcasting How do we do it on the Food Fight Show? Google Hangouts Hangouts on Air The Ruby Rep Libsyn.com Food Fight Show Downloads 115 Total Episodes 155,700+ Total Downloads Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 29

30 One repo or many? WTF, community?! How many repos do I need? One repo! N + 1 repos! Choose Your Own Adventure What is DevOps? Infrastructure as Code Version Control Quit your job! Incident Management Stages of Learning How to hire DevOps Docker & Chef Test Driven Development Chef DK Chef Metal Chef for OSX Chef Search Knife One repo or many? Chef vs 30

31 This slide intentionally left blank 31

This tutorial provides a basic understanding of the infrastructure and fundamental concepts of managing an infrastructure using Chef.

This tutorial provides a basic understanding of the infrastructure and fundamental concepts of managing an infrastructure using Chef. About the Tutorial Chef is a configuration management technology developed by Opscode to manage infrastructure on physical or virtual machines. It is an open source developed using Ruby, which helps in