DevOps and Microservices. Cristian Klein

Size: px

Start display at page:

Download "DevOps and Microservices. Cristian Klein"

Erik Whitehead
6 years ago
Views:

1 DevOps and Microservices Cristian Klein

2 Agenda Mo*va*on DevOps: Defini*on, Concepts Star as Code DevOps: Caveats Microservices DevOps and Microservices 2

3 Why DevOps from 50,000 feat DevOps and Microservices 3

4 Google Search Example Business Hypothesis: Users want 30 results instead of 10 Build (A/B tes*ng): Users A served 30 results Users B served 10 results Measure amount spent on Google Search Measure: Users A spent less 3me on Google Search. Learn: Displaying 30 results takes longer than 10 results Display only 10 results OR make 30 results faster DevOps and Microservices 4

5 Some Ideas of Lean Startup Faster learning => more compe33ve Discover requirements Instead of deba*ng them DevOps and Microservices 5

6 DevOps and Microservices 6

7 DevOps and Microservices 7

8 Misalignment of Incen*ves DevOps and Microservices 8

9 Wall of Confusion DevOps and Microservices 9

10 DevOps and Microservices 10

11 Agenda Mo*va*on DevOps: Defini3on, Concepts Star as Code DevOps: Caveats Microservices DevOps and Microservices 11

12 DevOps: Defini*on "DevOps" is an emerging set of principles, methods and prac1ces for communica3on, collabora3on and integra3on between development (applica*on/sozware engineering) and IT opera3ons (systems administra*on/infrastructure) professionals DevOps and Microservices 12

13 DevOps Pipeline DevOps and Microservices 13

14 DevOps (cont.) Acknowledges the interdependence of sozware development and IT opera*ons Aims to help organiza*ons rapidly produce quality sozware products and services Responds to the demands of stakeholders for an increased rate of produc*on releases Supports the use of agile development processes Aligns Dev and Ops incen*ves DevOps and Microservices 14

15 Down*me Budget Example Google s Site Reliability Engineers: SRE is what happens when you ask a sozware engineer to design an opera3ons team. A is allowed to be down 15 minutes in a month. Down*me budget spend => no more new features Devs are incen*vized to: invest more in tes3ng, resilience, automa3on,... Ops: get paged less o@en only for truly new issues DevOps and Microservices 15

16 State of DevOps Report Annual report by Puppet Survey from 2016 Companies that uses DevOps vs they who don t. Lead 1me - 1me between commit and produc1on 16

17 DevOps and Microservices 17

18 Apart from that? Worker 2.2 *mes more likely to recommend their organiza*on to a friend as a good workplace 18

19 Agenda Mo*va*on DevOps: Defini*on, Concepts Star as Code Micro-services DevOps and Microservices 19

20 Pipeline as Code Infrastructure as Code Configura3on as Code Star as Code 20

Lessons Learned from 597 outages 2009-2015 Gunawi, 2017-05-11

Microservices Lessons from Hundreds of Service Outages.

21 Lessons Learned from 597 outages Gunawi, Haryadi S., et al. "Why Does the Cloud DevOps Stop and Computing? Microservices Lessons from Hundreds of Service Outages. 21 Proceedings of the Seventh ACM Symposium on Cloud Computing. ACM, 2016.

22 Mo*va*on Many DevOps tasks are: Dull Error-prone Stressful Time-consuming Such as: Server SoZware configura3on Deploying code Solu*on: Automate! DevOps and Microservices 22

23 Why as code? Code is: human readable machine executable Can be: versioned audited commented tested scaled (configure 10,000 servers) DevOps and Microservices 23

24 Playbooks instead of Tribal Knowledge DevOps and Microservices 24

25 Approaches to as code Impera*ve (how?) Copy this file Install this package Declara*ve (what?) Ensure this file is up-to-date Ensure this package is installed and up-to-date Most tools offer both DevOps and Microservices 25

$Pipeline as Code: Jenkins stage('checkout'){ checkout scm } stage('deploy'){ echo 'Push to Repo' sh './dockerpushtorepo.sh' stage('test'){ env.node_env = "test } sh 'ssh deploy@xxxxx.com dockerrun.$

26 Pipeline as Code: Jenkins stage('checkout'){ checkout scm } stage('deploy'){ echo 'Push to Repo' sh './dockerpushtorepo.sh' stage('test'){ env.node_env = "test } sh 'ssh deploy@xxxxx.com dockerrun.sh } sh 'node -v' sh 'npm prune' sh 'npm install sh 'npm test stage('build Docker'){ sh './dockerbuild.sh } stage('cleanup'){ sh 'npm prune' } sh 'rm node_modules -rf' mail body: 'project build successful', from: 'xxxx@yyyyy.com', replyto: 'xxxx@yyyy.com', subject: build successful to: 'yyyyy@yyyy.com DevOps and Microservices 26

Pipeline as Code: Travis sudo: required language: ruby services: - docker before_install: - docker pull carlad/sinatra - docker run -d -p 127.0.

27 Pipeline as Code: Travis sudo: required language: ruby services: - docker before_install: - docker pull carlad/sinatra - docker run -d -p :80:4567 carlad/sinatra /bin/sh -c "cd /root/sinatra; bundle exec foreman start;" - docker ps -a - docker run carlad/sinatra /bin/sh -c "cd /root/sinatra; bundle exec rake test" script: - bundle exec rake test DevOps and Microservices 27

28 Configura*on Management as Code Describe systems once Predictable results Handle differences in environments Dev vs. staging vs. produc*on No Snowflakes! Server configura*on *me Seconds DevOps and Microservices 28

29 Infrastructure as Code: Pets vs. Cakle DevOps and Microservices 29

30 IaC and CMaC Tools Declara*ve configura*on language Plain-text configura*on in source control Fully programma*c, no manual interac*ons DevOps and Microservices 30

31 Puppet overview 1. Describe what you want to be configured 2. (Don t care how it is done) 3. Describe dependencies file package service types win *nix deb rpm POSIX win providers Puppet DSL DevOps and Microservices 31

32 Puppet example: Apache server Manifests Resources Name Attributes and values Ordering and dependencies package { "httpd": name => "httpd.x86_64", ensure => "present", } file { "http.conf": path => "/etc/httpd/conf/httpd.conf", owner => root, group => root, mode => 0644, source => "puppet:///modules/apache/httpd.conf", require => Package["httpd"], } service { "httpd": ensure => running, enable => true, subscribe => File["http.conf"], } DevOps and Microservices 32

33 Puppet building blocks Manifests Variables and (custom) facts Node declarations Classes and Modules Defined resource types Templates Console if $operatingsystem == 'CentOS' node 'www1.example.com' { include common include apache } node 'db1.example.com' { include common include mysql } file { "http.conf": path => "/etc/httpd/conf/httpd.conf", owner => 'root', group => 'root', mode => '0644', content => template('config/httpd.erb'), } DevOps and Microservices 33

34 A Puppet run 1. Retrieve plugins from server 2. Get facts on client and send them to master 3. Compile catalog and send it to the client 4. Apply catalog on client 5. Process report DevOps and Microservices 34

35 Challenge: Idempotence Applying the same playbook twice MUST give the same results Hummer, Waldemar, et al. "Testing idempotence for infrastructure as code. Distributed Systems Platforms and Open Distributed Processing DevOps and Microservices 35

36 DevOps: Dangers To err is human, but to err and deploy it to all systems simultaneously is DevOps A. CockcroZ, prev. Neolix Architect 2014 Gmail outage: "The incorrect configura*on was sent to live services over the next 15 minutes, caused users requests for their data to be ignored, and those services, in turn, generated errors Amazon WS S3 outage: Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended. Don t roll out system-wide un*l it works DevOps and Microservices 36

37 DevOps and Microservices 37

38 Con*nuous Monitoring DevOps and Microservices 38

39 Agenda Mo*va*on DevOps: Defini*on, Concepts Star as Code Microservices DevOps and Microservices 39

40 Microservices (One) Defini*on: Loosely coupled service oriented architectures with bounded contexts Loosely coupled: asynchronous service updates Bounded context: Limited knowledge of surrounding services DevOps and Microservices 40

41 Microservices: More about coupling Organiza3onal coupling Centralized database schemes Enterprise service bus centralized messaging DevOps and Microservices 41

42 Microservices: Conway s Law Conway s law (1967): Any organiza*on that designs a system (defined broadly) will produce a design whose structure is a copy of the organiza*on s communica*on structure DevOps and Microservices 42

43 Microservices: Avoiding Organiza*onal Coupling DevOps and Microservices 43

44 Microservices: Data Coupling DevOps and Microservices 44

45 Microservices: Proper*es Polyglot Mul*ple languages, plaoorms, etc. Immutable code pakern Keep old code / exis*ng service Deploy new version Eventually, re-route traffic A/B tests, feature flags, version rou*ng Canary tes*ng Try out new features with developers and testers only Add a few real users, measure improvements Make default for all users, but keep old version DevOps and Microservices 45

46 Microservices: Benefits Small, easy to understand code base Easy to scale (next slide) Easy to throw away Easy to deploy Easy to use different tools (polyglot) Resilient systems DevOps and Microservices 46

47 Microservices: Scaling DevOps and Microservices 47

48 Microservices: Drawbacks Higher communica*on overhead Nanoservices an*-pakern Integra3on tes*ng more complicated Moving responsibility more difficult Needs communica*on between different teams DevOps and Microservices 48

49 Conclusions Need to deploy code rapidly and reliably DevOps bridges the gap between development and opera*ons Automa3on key to success Many tools/services out there to help Microservices: preferred architecture for DevOps Hot research area! Make beker use of opera3ons feedback during development DevOps and Microservices 49

From Continuous Integration To Continuous Delivery With Jenkins

From Continuous Integration To Continuous Delivery With Cyrille Le Clerc, Solution Architect, CloudBees About Me @cyrilleleclerc CTO Solu9on Architect Open Source Cyrille Le Clerc DevOps, Infra as Code,