Salt A Scalable Systems Management Solution for Datacenters

Transcription

1 Salt A Scalable Systems Management Solution for Datacenters Open Source Data Center Conference April 26-28, 2016 Sebastian Meyer Linux Consultant & Trainer B1 Systems GmbH meyer@b1-systems.de B1 Systems GmbH - Linux/Open Source Consulting, Training, Support & Development

2 Introducing B1 Systems founded in 2004 operating both nationally and internationally nearly 100 employees provider for IBM, SUSE, Oracle & HP vendor-independent (hardware and software) focus: consulting support development training operations solutions B1 Systems GmbH Salt Scalable Systems Management 2 / 47

3 Areas of Expertise B1 Systems GmbH Salt Scalable Systems Management 3 / 47

4 Salt Introduction B1 Systems GmbH Salt Scalable Systems Management 4 / 47

5 Yet Another Systems Management Solution? takes inspiration from Puppet, Chef or Ansible focuses on the entire system life cycle easily scalable to a few thousand systems convenient and easy to learn configuration management and remote execution B1 Systems GmbH Salt Scalable Systems Management 5 / 47

6 Salt Concept B1 Systems GmbH Salt Scalable Systems Management 6 / 47

7 Master & Minions B1 Systems GmbH Salt Scalable Systems Management 7 / 47

8 Scalability: Masters, Syndics & Minions B1 Systems GmbH Salt Scalable Systems Management 8 / 47

9 High Availability: Multiple Masters& Minions B1 Systems GmbH Salt Scalable Systems Management 9 / 47

10 Salt Modes minions pull from master master pushes to Minions minions apply states locally master applies states on minions via SSH B1 Systems GmbH Salt Scalable Systems Management 10 / 47

11 Remote Execution System B1 Systems GmbH Salt Scalable Systems Management 11 / 47

12 Salt Command B1 Systems GmbH Salt Scalable Systems Management 12 / 47

13 Grains B1 Systems GmbH Salt Scalable Systems Management 13 / 47

14 Configuration Management B1 Systems GmbH Salt Scalable Systems Management 14 / 47

15 States ID: module.function: - name: name - argument1: value - argument2: - value1 - value2 B1 Systems GmbH Salt Scalable Systems Management 15 / 47

16 Top File base: * : - monitoring - ssh - syslog *lan* : - ntp.lan *dmz* : - ntp.dmz - firewall all servers: monitoring ssh config syslog servers in LAN: ntp config servers in DMZ: ntp config firewall B1 Systems GmbH Salt Scalable Systems Management 16 / 47

17 Pillars B1 Systems GmbH Salt Scalable Systems Management 17 / 47

18 Pillar Data Pillar Example ntp: {% if grains[ id ].startswith( myntpserver ) %} ntpservers: ["0.us.pool.ntp.org","1.us.pool.ntp.org"] comment: {% else %} ntpservers: [" "," "] comment: myinternalservers {% endif %} Source: B1 Systems GmbH Salt Scalable Systems Management 18 / 47

19 Pillars and States States top.sls base: * : - monitoring - ssh - syslog - ntp *dmz* : - firewall Pillar top.sls base: * : - monitoring - ssh - syslog *lan* : - ntp.lan *dmz* : - ntp.dmz - firewall B1 Systems GmbH Salt Scalable Systems Management 19 / 47

20 Deploying the State Master pushes to minions salt * state.highstate salt * state.sls mystate Minions pull from master salt-call state.highstate salt-call state.sls mystate B1 Systems GmbH Salt Scalable Systems Management 20 / 47

21 Reusing States: Formulas reusing existing code roughly the same as Puppet modules/ansible roles collection of States and files github.com/saltstack-formulas/ for "official" formulas B1 Systems GmbH Salt Scalable Systems Management 21 / 47

22 Using Formulas directly from VCS or local extendable via include configurable via Pillar data variables mapped via Jinja map requirements across Formulas possible B1 Systems GmbH Salt Scalable Systems Management 22 / 47

23 Demo B1 Systems GmbH Salt Scalable Systems Management 23 / 47

24 Returners salt * disk.usage --return redis_return B1 Systems GmbH Salt Scalable Systems Management 24 / 47

25 Salts Event Driven Infrastructure B1 Systems GmbH Salt Scalable Systems Management 25 / 47

26 Overview actions trigger events events are communicated via the event bus reactors execute trigger actions responding to events B1 Systems GmbH Salt Scalable Systems Management 26 / 47

27 Event Bus B1 Systems GmbH Salt Scalable Systems Management 27 / 47

28 Actions & Events master# salt salt-minion-01 disk.percent /srv salt-minion-01: 11% B1 Systems GmbH Salt Scalable Systems Management 28 / 47

29 Actions & Events { [...] } salt/job/ /new { "_stamp": " T14:32: ", "arg": [ "/srv" ], "fun": "disk.percent", "jid": " ", "minions": [ "salt-minion-01" ], "tgt": "salt-minion-01", "tgt_type": "glob", "user": "root" } B1 Systems GmbH Salt Scalable Systems Management 29 / 47

30 Actions & Events salt/job/ /ret/salt-minion-01 { "_stamp": " T14:32: ", "cmd": "_return", "fun": "disk.percent", "fun_args": [ "/srv" ], "id": "salt-minion-01", "jid": " ", "retcode": 0, "return": "11%", "success": true } B1 Systems GmbH Salt Scalable Systems Management 30 / 47

31 Events in a State b1/mystate/status/update: event.send: - data: status: "Installation done!" B1 Systems GmbH Salt Scalable Systems Management 31 / 47

32 Beacons hook into system on minion create events inotify, diskusage, load, journald... B1 Systems GmbH Salt Scalable Systems Management 32 / 47

33 Beacons - Example inotify Beacon beacons: inotify: /etc/motd: mask: - modify B1 Systems GmbH Salt Scalable Systems Management 33 / 47

34 Reactors B1 Systems GmbH Salt Scalable Systems Management 34 / 47

35 Calling Reactors on Events Reactor Example reactor: - salt/minion/*/start : - /srv/reactor/start.sls - b1/mystate/status/* : - salt://reactor/status.sls B1 Systems GmbH Salt Scalable Systems Management 35 / 47

36 Demo B1 Systems GmbH Salt Scalable Systems Management 36 / 47

37 Use Cases? load-balancing job automation alerting B1 Systems GmbH Salt Scalable Systems Management 37 / 47

38 Salt Cloud B1 Systems GmbH Salt Scalable Systems Management 38 / 47

39 Overview B1 Systems GmbH Salt Scalable Systems Management 39 / 47

40 Providers Amazon EC2 Provider Example my-ec2: driver: ec2 id: MYEC2ID key: adsfrf453fmykeyasdsadg43 private_key: /etc/salt/my_key.pem keyname: my_key securitygroup: default minion: master: saltmaster.example.com B1 Systems GmbH Salt Scalable Systems Management 40 / 47

41 Profiles profile name provider image or template options for the instance minion options B1 Systems GmbH Salt Scalable Systems Management 41 / 47

42 Profiles LXC Profile Example myfancyprofile: provider: lxc-host01 lxc_profile: template: ubuntu options: release: trusty password: test123 B1 Systems GmbH Salt Scalable Systems Management 42 / 47

43 Maps Mapfile profile1: - instance_name_1 - instance_name_2 profile2: - instance_name_3: grains: mykey: myvalue - instance_name_4 Execute Mapfile salt-cloud -m /path/to/mapfile B1 Systems GmbH Salt Scalable Systems Management 43 / 47

44 Bootstrapping a New Salt Environment Mapfile profile1: - instance_name_1: make_master: True minion: master: myoldmaster local_master: True - instance_name_2 - instance_name_3 - instance_name_4... B1 Systems GmbH Salt Scalable Systems Management 44 / 47

45 Saltify Existing Machines 1/2 Saltify Provider saltify-all-machines: driver: saltify minion: master: mysaltmaster Saltify Profile salt-machine: provider: saltify-all-machines ssh_username: root key_filename: /etc/salt/pki/master/ssh/salt-ssh.rsa B1 Systems GmbH Salt Scalable Systems Management 45 / 47

46 Saltify Existing Machines 2/2 Mapfile salt-machine: - first-machine: ssh_host: second-machine: ssh_host: third-machine: ssh_host: B1 Systems GmbH Salt Scalable Systems Management 46 / 47

47 Thank You! For more information, refer to or +49 (0) B1 Systems GmbH - Linux/Open Source Consulting, Training, Support & Development