Islands of Splunk. MulJple Splunk as a Service Architecture and ImplementaJon

Transcription

1 Copyright 2014 Splunk Inc. Islands of Splunk MulJple Splunk as a Service Architecture and ImplementaJon Michael de Buin, Schuberg Philis mdebruin@schubergphilis.com Gert Kremer, Schuberg Philis gkremer@schubergphilis.com Dani Flexer, Splunk dflexer@splunk.com

2 Disclaimer During the course of this presentajon, we may make forward looking statements regarding future events or the expected performance of the company. We caujon you that such statements reflect our current expectajons and esjmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward- looking statements made in the this presentajon are being made as of the Jme and date of its live presentajon. If reviewed ater its live presentajon, this presentajon may not contain current or accurate informajon. We do not assume any obligajon to update any forward looking statements we may make. In addijon, any informajon about our roadmap outlines our general product direcjon and is subject to change at any Jme without nojce. It is for informajonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligajon either to develop the features or funcjonality described or to include any such feature or funcjonality in a future release. 2

3 Agenda! The MulJ- Splunk- as- a- Service (MSaaS) framework! MSaaS Schuberg Philis! DemonstraJon 3

4 MSaaS Architecture

5 Background! Splunk administrators are increasingly required to provision Splunk as a service offering to muljple customers Commonly requires provisioning a Splunk instance to each customer! MSaaS is a conceptual framework designed to help deliver such an offering! Schuberg- Philis have implemented this framework using Chef on Apache CloudStack 5

6 FuncJonal Requirements! AutomaJc deployment and update of muljple Splunk instances! Packaged binaries, scripts and configurajons! Modular! Each instance can scale from very small to as large as necessary! Each instance is customized as needed! System must funcjon independently without external resources Except for authenjcajon, a datacentre automajon (DCA) tool and an opjonal license manager to fulfill its purpose! Archiving, backup and resilience requirements are defined per customer 6

7 FuncJonal Requirements! Resilience Disaster recovery (DR), no single point of failure, indexing Jer resilience, storage resilience! Strict data segregajon! Different network jurisdicjons isolated from each other with no shared resources nor shared informajon! Cross- jurisdicjon access possible when explicitly enabled! JurisdicJon hierarchy supported A jurisdicjon can include other jurisdicjons 7

8 FuncJonal Requirements! Indexed data can be shared, subject to jurisdicjon, and indexes copied between instances! MulJple licensing models supported central license manager and license pools per- Island license manager and muljple keys assigned on deployment global license! Roles defined independently for each instance! All credenjals maintained in the enterprise idenjty management system and allocated at deployment 8

9 Architecture Concepts: Island! A complete Splunk deployment! No informajon sharing between Islands! Own set of users and roles! Manages a set of Forwarders! Forwarders can send data to many Islands but are managed by a single Island 9

10 Architecture Concepts! Bridges Islands without indexing capabilijes that enable search on muljple Islands! Deployment unit An independently deployed collecjon of Splunk components! Customer An independent user of Splunk a business unit or customer! Island service agributes ReplicaJon factor, search factor, DR requirements, security, backup, storage Jer, performance, retenjon plan, daily volume! AdministraJon Island monitors the other Islands 10

11 MSaaS Deployment Server! A centralized system responsible for installing, configuring, and updajng the Islands! Maintains all binaries, applicajons, configurajons, apps and roujng informajon in a version control system (VCS)! Updates the Islands binaries when necessary! Each Splunk DS maintains its Island s components ConfiguraJon files supplied by the MSaaS DS and propagated by Island DS 11

12 ApplicaJons Each Island has any number of Splunk applicajons a.k.a. apps Apps are Managed centrally and deployed with the Island Versions are maintained in the VCS Customized for the MSaaS as a whole or for a subset of the Islands! MSaaS administrajon apps reside on a dedicated Island Monitors the other Islands for usage, security, charge- back, health and need for maintenance! Standard apps deployed with each Island S.o.S Splunk on Splunk App for Unix or for Windows Infrastructure as appropriate 12

13 MSaaS

14 My Company and I Our customers: Gert Kremer Mission CriJcal Engineer since 2007

15 Engineering the MSaaS Architecture! Dual datacenter setup, no data backups (RF=SF=2)! Maximum 100 GB/day per Island! Centralized license server! AcJve- standby search heads (rsync- ed)! Not implemented: Splunk Deployment Server AdministraJon Island Dedicated Job Servers 15

16 Ingredients Requirements M- SaaS Cloud(Stack) Github Splunk Chef & use Meta cases enterprise Architecture DescripJon 16

17 Configura)on HP DL380G7; CPU: 2 6 Xeon 2.67GHz; Memory: 12GB; OS: Linux 64- bit, Fedora 14 (*) Linux on EC2: c1.xlarge 800 piops (*) Will it blend? Indexing: Avg KBPS Indexing: Avg EPS Search: Avg First Event (sec) Search: Avg Search (sec) 22,400 79, ,410 43, SBP: 4 CPU/16GB 12,449 43, SBP: 8 CPU/32GB 14,715 51, (*) hgp://blogs.splunk.com/2013/06/06/splunkit- v results- ec2- storage- comparisons/ 17

18 Prior art on Splunk, Chef and Cloud! Best Buy Splunk cookbook hgps://github.com/bestbuycom/splunk_cookbook! OpsCode Splunk cookbook hgps://github.com/opscode- cookbooks/chef- splunk! Splunk Storm (Splunk as a Service) hgp:// 18

19 Scope of AutomaJon! Island creajon! Server instanjajon! Splunk sotware installajon! Cluster configurajon! Data disk and indexes: creajon and management! Data replicajon between Search Heads! Security (firewall rules, SSL setup)! Monitoring (process, connecjvity, cluster health, Splunk alerts)! Single- sign- on (SSO)! Splunk applicajons 19

20 Splunk Enterprise at Schuberg Philis Datacenter 1 Datacenter 2 Search head Search head Cluster master Indexer Indexer License server

21 Splunk Enterprise at Schuberg Philis AuthenJcaJng proxy Bridge (SIEM) Island Customer Island Customer Island Customer SupporJng Systems Search head Search head Search head Search head Search head Search head Search head Search head Cluster Indexer master Cluster Indexer master Cluster Indexer master Indexer Indexer Indexer License server AcJve Directory SupporJng Cloud Infrastructure

22 Island InstanJaJon Proxy Servers Datacenter 1 Datacenter 2 1. Configure island 2. Deploy island 3. Integrate island Search Head Search Head Cluster Master Indexer Indexer License Server

23 Island InstanJaJon Proxy Servers Datacenter 1 Datacenter 2 1. Configure island 2. Deploy island 3. Integrate island Search Head Search Head Cluster Master Indexer Indexer License Server

24 Returning the favor! Generalize available Chef Cookbooks! Splunk monitoring Nagios plugin available in Nagios Exchange! Splunk deployment best pracjces and tools 24

25 DemonstraJon by Michael

26 THANK YOU

27 Process Requirements! Service Request When customers request the service, a process is triggered that results in a deployed instance of Splunk that implements the customer s use- cases and the other agributes of the service requested.! Charge Back The cost of the service to its operators can be charged back to its customers based on the actual cost of provisioning the service.! Easy to Onboard The process of incorporajng data sources into the system is well defined and simple 27

28 Data RouJng! Islands use- cases can overlap requiring them to share data and data- sources! Data roujng is maintained in a global roujng table! On update, the roujng table is converted into Splunk configurajon elements suitable for inserjon into the transforms.conf file that is then propagated to the data collecjon Jer by the Master Deployment Server and Island Deployment Servers! Heavy Forwarders are required to support data roujng, otherwise a Universal Forwarder is used

29 Island Service Agributes! Agributes are defined when a service is requested by a customer! The provisioning process implements the agributes and deploys the Island! Agributes Resilience DR License volume Data segregajon and availability Storage volume RetenJon requirements Data sources and use cases

30 Customers! Each customer is an area of the enterprise or an external organizajon that requires Splunk! A customer consists of one or more use- cases! MulJple customers can be consolidated into a single MSaaS Island segregajon requirements permizng

31 Source Types and Data Sources! In addijon to the default set of source types provided by Splunk, MSaaS implements addijonal source types that are used as needed by any Island! When an Island is implemented, new source types are defined as required and the pre- defined MSaaS source types extended! Data sources indexed in a given Island are parjjoned into specific indexes based on security, ownership, visibility constraints, retenjon requirements, and resilience requirements.! These agributes can differ between Islands for common data sources, depending on the requirements of the use- cases implemented by this Island

32 Common Namespace and InformaJon Model! Named Splunk enjjes are named in a common namespace so different Islands can share informajon, apps, indexes etc.! Requires MSaaS to maintain a centrally controlled global name space