Best Prac:ces + New Feature Overview for the Latest Version of Splunk Deployment Server

Legal No:ces During the course of this presenta:on, we may make forward- looking statements regarding future events or the expected performance of the company. We cau:on you that such statements reflect our current expecta:ons and es:mates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward- looking statements made in this presenta:on are being made as of the :me and date of its live presenta:on. If reviewed awer its live presenta:on, this presenta:on may not contain current or accurate informa:on. We do not assume any obliga:on to update any forward- looking statements we may make. In addi:on, any informa:on about our roadmap outlines our general product direc:on and is subject to change at any :me without no:ce. It is for informa:onal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obliga:on either to develop the features or func:onality described or to include any such feature or func:onality in a future release. Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respeccve owners. 2013 Splunk Inc. All rights reserved. 2

About Me Gen: Zaimi! Splunk Professional Services, West! April 2010! 237! Splunk Support - > Splunk PS 3

Agenda! About Splunk s Deployment Server! The Big Picture! Configura:on Overview! Advanced Configura:ons! New in Splunk Enterprise 6! Tips & Tricks! Ques:ons and Comments 4

About Splunk s Deployment Server! What it is not: Deployment server is not IT automa:on sowware Deployment server is not a comprehensive provisioning tool Deployment server cannot be used for installa:on or upgrade of Splunk components Puppet/Chef! What it is: Deployment server is Splunk's configura:on management tool Deployment server can be used for pushing out configura:ons and content updates to distributed Splunk instances Deployment server cannot be used for modifying just a single file, instead the whole app gets updated/installed 5

The Big Picture! In a Splunk deployment, you use a deployment server to push out content and configura:ons to deployment clients, grouped into server classes 6

The Big Picture! A deployment server is a Splunk instance that acts as a centralized configura:on manager, collec:vely managing any number of Splunk instances, called "deployment clients! Any Splunk instance even one indexing data locally can act as a deployment server 7

The Big Picture! A deployment client is a Splunk instance remotely configured by a deployment server; a Splunk instance can be both a deployment server and client at the same :me (:ered DS) but is discouraged! Each Splunk deployment client can belong to one or more server classes 8

The Big Picture! A server class is a set of deployment clients, grouped by some set of configura:on characteris:cs, so that they can be managed as a unit! Clients can be grouped by: Applica:on OS Type of data Or any other feature of your Splunk deployment 9

The Big Picture 10

Configura:on Overview! Designate or setup a Splunk instance as the deployment server! Group the deployment clients into server classes! Create a serverclass.conf file on the deployment server! Create a deploymentclient.conf for your deployment clients! Create your deployment apps 11

Configura:on Overview Designate or Setup a Splunk Instance as The Deployment Server! What should serve as a deployment server? < 50 clients - any Splunk instance can be used. > 50 clients - a designated Splunk instance should be used. ê Any mixture of du:es possible? License master, cluster master and deployment server = OK? 12

Configura:on Overview Group The Deployment Clients Into Server Classes! A server class defines a deployment configura:on shared by a group of deployment clients; it defines both the criteria for being a member of the class and the set of content to deploy to members of the class! You can define different server classes to reflect the different requirements, OSes, machine types, or func:ons of your deployment clients 13

Configura:on Overview Group The Deployment Clients Into Server Classes! Example: Windows and Linux universal forwarders All forwarders get outputs.conf Different inputs.conf files for Windows vs. Linux Serverclass "all forwarders distributes outputs.conf file to all forwarders Separate Windows and Linux serverclasses distribute the different inputs.conf files 14

Configura:on Overview Group The Deployment Clients Into Server Classes! Example (cont) serverclass.conf: [global]! restartsplunkd = false!! # ALL FORWARDERS! [serverclass:all_forwarders]! whitelist.0 = *! blacklist.0 = searchhead.acme.com! blacklist.1 = indexer.acme.com! restartsplunkd = true!! [serverclass:all_forwarders:app:acme_all_outputs]! #windows forwarders! [serverclass:all_win_forwarders]! # Match only Windows machines! machinetypesfilter=windows-intel!! #nix forwarders! [serverclass:all_nix_forwarders]! # Match only Unix machines! machinetypesfilter=linux-i686, linux-x86_64!! [serverclass:all_win_forwarders:app:acme_all_win_inputs]! [serverclass:all_nix_forwarders:app:acme_all_nix_inputs]!!! 15

Configura:on Overview Create The Directories for Your Deployment Apps! Default loca:on is: $SPLUNK_HOME/etc/deployment- apps! Each app must have its own subdirectory as specified in serverclass.conf! Naming conven:on should be as granular as possible 16

Configura:on Overview Create The Directories for Your Deployment Apps! Configure the repositorylocabon auribute in serverclass.conf! Configure the base config directories acme_all_indexers_base ê indexes.conf ê inputs.conf ê web.conf ê server.conf acme_all_searchheads_base ê similar to above, define indexes, web, server configura:on files ê note: no inputs.conf here 17

Configura:on Overview Create The Directories for Your Deployment Apps! Dis:nguish between organiza:on configura:ons & apps vs. Splunk out of the box configura:on & apps acme_all_forwarders_outputs ê outputs.conf TA- nix ê out of the box configura:on files downloadable from splunk.com 18

Configura:on Overview Create The Directories for Your Deployment Apps! Dis:nguish between internal departments acme_it_db_inputs ê inputs.conf acme_finance_db_inputs ê inputs.conf 19

Configura:on Overview Create a deploymentclient.conf for Your Deployment Clients! Sample deploymentclient.conf [deployment-client] [target-broker:deploymentserver] # Change the targeturi & port if needed targeturi= deploymentserver.acme.com:8089! 20

Configura:on Overview Create a deploymentclient.conf for Your Deployment Clients! Create an acme_all_deploymentclients app which Contains deploymentclient.conf configura:on file Is installed on all deployment clients Is located in deployment- apps/ on the deployment server! Sample serverclass.conf [serverclass:all_forwarders:app:acme_all_deploymentclients]! Warning: Only change this app in the deployment server if you intend to change deployment server loca:ons 21

Advanced Configura:ons! Deployment server with clustering and search head pooling! Change phonehomeintervalinsecs in deploymentclients.conf! Configure mul:ple deployment server instances (tenants.conf)?! Tiered deployment server configura:on?! Mul:ple deployment servers (rsync way)? 22

Advanced Configura:ons Deployment Server with Clustering and Search Head Pooling! In a clustered configura:ons you cannot use deployment server to push apps to the cluster peers (i.e. indexers)! In a search head pooling configura:on, you can use deployment server to push apps to the pool 23

Advanced Configura:ons Change phonehomeintervalinsecs in deploymentclients.conf! Default interval is 60 seconds! Changing to 600 can ensure less deployment :me in large environments with many thousands numbers of clients Sample stanza:![deployment-client]!!phonehomeintervalinsecs = 600!!![target-broker:deploymentServer]!!targetUri= deploymentserver.acme.com:8089! 24

Advanced Configura:ons Configure Mul:ple Deployment Server Instances (tenants.conf)! Deprecated in Splunk Enterprise 6! Has not been tested! Is not supported! Will almost certainly not work 25

Advanced Configura:ons Tiered Deployment Server Configura:on! Has not been tested in Splunk Enterprise 6! Is undocumented! Is not supported! Will almost certainly be buggy 26

Advanced Configura:ons Mul:ple Deployment Servers (rsync Way)! Has not been tested in Splunk Enterprise 6! Is undocumented! Is not supported! Will probably work 27

Advanced Configura:ons Mul:ple Deployment Servers (rsync Way)! It is best prac:ce to use mul:ple deployment servers, each located on its own Splunk server where All changes are made within the master deployment server All slave deployment servers receive their apps via r- sync from the master 28

Advanced Configura:ons Mul:ple Deployment Servers (rsync Way) 29

New in Splunk Enterprise 6! More clients, less resources According to internal performance tes:ng, Splunk deployment server should be able to: ê Handle more clients ~ 7000 tested (Linux only) @ 50MB ê Fast distribu:on :me (T (minutes) = 0.0075 * C + 1.85) ê Less resource u:liza:on Open files (FD) Load average, memory, cpu! Forwarder management UI!!!! 30

Tips & Tricks! Just, how does the conversa:on between the server and the client go? DS DC m0: <- - - - - - - - - "here are my apps' MD5s"- - - - - - - - - - - - - - m1: - - - - - - - - - "your apps' MD5s should be" - - - - - - - - - - > If actual MD5s == expected MD5s, stop. Else m2: <- - - - - - - - - - - - - - - - "give me app A" - - - - - - - - - - - - - - - - - - - - m3: - - - - - - - - - - - - - - - - - - "here is app A - - - - - - - - - - - - - - - - - - - - > m4: <- - "report: app=a accon=install result=ok" - - - 31

Tips & Tricks! Configure a deployment server as early as possible in your installa:on process It s beneficial to do so right awer sowware install! To refresh serverclasses you don t need a Splunk restart./splunk reload deploy-server!! List all your deployment clients and find out the last :me they were successfully synced./splunk list deploy-clients! 32

Tips & Tricks! On a client, list the deployment server, if any./splunk list deploy-poll! See what serverclasses are loaded ls $SPLUNK_HOME/var/run/tmp! See what apps are loaded within a serverclass; look for the.bundle file ls $SplunkHome/var/run/tmp/all_forwarders! Troubleshoo:ng: Increase logging when needed # deployment server & client category.dc:handshakereplyhandler=info! 33

Ques:ons & Comments 34

Next Steps 1 2 3 Download the.conf2013 Mobile App If not iphone, ipad or Android, use the Web App Take the survey & WIN A PASS FOR.CONF2014 Or one of these bags! Go to Best PracHces and Lessons Learned from Splunk s Professional Services Team Room: Brera 2&3, Level 3 Today, 1:45-2:45pm 35

THANK YOU