Amazon AMI Build Guide January Doc Version 1.5

Transcription

1 Amazon AMI Build Guide January 2018 Doc Version 1.5 1

2 TABLE OF CONTENTS 1 Authors Preface Introduction Build Installation Architecture Overview Version 3 Happy Snap Features Provided CF Template Installation Key Information Pre-Requisites Cloudformation Stack Deployment Web Application Setup User SETUP User Roles Site Administration Site LDAP Settings Operational Overview Log View Shell Interaction VPN Access Skedler Licensing

3 1 AUTHORS PREFACE In 2015, one of our corporate clients told us of their frustrations with the exorbitant licensing costs of commercial Security Information and Events Management (SIEM) products. The customer light heartedly asked whether we could build them an open source SIEM to get rid of these annual license fees. We thought that was a great idea and set out so to develop a SIEM product for Managed Security Service Providers (MSSP s) and Security Professionals. This product is called SIEMonster. SIEMonster Version 1 was released in late April of 2016 and a commercial release in November The release has been an astounding success without over 100,000 downloads of the product. We have assisted individuals and companies integrate SIEMonster into small medium and extra-large companies all around the world. SIEMonster with the help of the community and a team of developers have been working hard since the Version1 release incorporating what the community wanted to see in a SIEM as well as things we wanted to see in the next release. Along the way we have signed up MSSP s from around the world who have contributed to the rollout of SIEMonster and in return they have assisted us with rollout scripts, ideas and things we hadn t even considered. We are now proud to release the latest Version 3.0 Beta, and finalized in February 2018 for Alpha Release. We have added the following features to this release ELK Stack updated to version 5.5 Built in Searchguard open source RBAC & encrypted node to node transport Wazuh HIDS system with Kibana plugin and OpenSCAP options & simplified agent registration process Simplified installation process for both Rancher Docker orchestration & SIEMonster web application All new dashboard with options for 2fa, site administration with user role based access and faster load times Built in parsers for most proprietary devices Preloaded Minemeld threat intel feeds integrated with log ingest out of the box. COREOS with NFS support We have also automated correlation with Palo Alto MineMeld Open Source Threat Intelligence and added two factor authentication and easier rollouts. The transition has now been completed to a full containerize all aspects of the SIEMonster application pool using the popular Docker system. This allows us to run on any hardware, cloud or operating system. It also provides the architecture for docker containers to be moved to other servers during downtime without affecting the SIEM. We welcome you to try out our fully functional SIEM product, and if you wish to upgrade to our Premium version with Advanced Correlation, Reporting, Auditing and support please contact sales@siemonster.com. 3

4 2 INTRODUCTION SIEMonster Version 3 is built on the best open source components and custom develop from a wish list from the SIEMonster community. This document will cover the architecture, the features and the open source components that make up SIEMonster, so that all security professionals can run a SIEM in their organisations with no budget. If you would like more information about the architecture please see our High-Level Design. SIEMonster is built on CoreOS, Docker with Rancher, Kubernetes orchestration. The product comes in Vbox, VMware, Bare-metal or Cloud install on AWS/Azure. SIEMonster can scale horizontally and vertically to support any enterprise client. Some of these features include. OSINT from PaloAlto Minemeld. OSSEC Wazuh fork. Full integration with OSSEC Wazuh fork for Host Intrusion Detection and PCIDSS ruleset incorporated into Elastic. 411 demonstrated at DEFCON. Instant Incident Alerting via or SMS or Console view via a secure portal and integration with Slack / PagerDuty / Jira using 411 Streams. Open Source AuditIT by Opmantek. Open Source Incident Response. Alerts maybe escalated as tickets to other operators or a whiteboard to show night shift analysts current issues. Elastalert, Event Monitor Alerting from the Guardian Newspaper. Data Correlation UI, community rulesets and dashboards, community and open source free plugins that make the SIEM. Incorporate your existing Vulnerability Scans into the Dashboard, (OpenVAS, McAfee, Nessus etc.) We have also developed and built in LDAP integration, advanced correlation and two factor authentication. 4

5 3 BUILD INSTALLATION ARCHITECTURE OVERVIEW SIEMonster V3 cloud deployment is a modular Docker container system which will run on all operating systems supporting Docker. Architecturally this was chosen for portability across platforms, supporting not only most container platforms such as AWS ECS, Azure etc. but also VMWare, VirtualBox and bare metal installs used by our corporate customers. This will provide simplified upgrade paths and scaling potential as well as high availability. Flexible deployment solutions include most cloud container platforms such as AWS, Azure, Digital Ocean etc. Also, options are available for VMware ESX and bare metal installs. For AWS deployment, the platform chosen is the open source container management system provided by Rancher Labs. Rancher supplies the entire software stack needed to manage containers in production. Rancher software consists of four major components: 1. INFRASTRUCTURE ORCHESTRATION Rancher takes in raw computing resources from any public or private cloud in the form of Linux hosts. Each Linux host can be a virtual machine or physical machine. Rancher does not expect more from each host than CPU, memory, local disk storage, and network connectivity. From Rancher s perspective, a VM instance from a cloud provider and a bare metal server are indistinguishable. Rancher implements a portable layer of infrastructure services designed specifically to power containerized applications. Rancher infrastructure services include networking, storage, load balancer, DNS, and security. Rancher infrastructure services are typically deployed as containers themselves, so that the same Rancher infrastructure service can run on any Linux hosts from any cloud. 2. CONTAINER ORCHESTRATION AND SCHEDULING Many users choose to run containerized applications using a container orchestration and scheduling framework. Rancher includes a distribution of all popular container orchestration and scheduling frameworks today, including Docker Swarm, Kubernetes, and Mesos. The same user can create multiple Swarm or Kubernetes clusters. They can then use the native Swarm or Kubernetes tools to manage their applications. In addition to Swarm, Kubernetes, and Mesos, Rancher supports its own container orchestration and scheduling framework called Cattle. Cattle was originally designed as an extension to Docker Swarm. As Docker Swarm continues to develop, Cattle and Swarm started to diverge. Rancher will therefore support Cattle and Swarm as separate frameworks going forward. Cattle is used extensively by Rancher itself to orchestrate infrastructure services as well as setting up, managing, and upgrading Swarm, Kubernetes, and Mesos clusters. 3. APPLICATION CATALOG Rancher users can deploy an entire multi-container clustered application from the application catalog with one click of a button. Users can manage the deployed applications and perform fully automated upgrades when new versions of the application become available. Rancher maintains a public catalog consisting of popular applications contributed by the Rancher community. Rancher users can create their own private catalogs.w ith this deployment, custom Rancher catalog applications have been created for the SIEMonster stack. Using the Rancher network overlay, the SIEMonster container application loads have been evenly balanced across four nodes. 4. ENTERPRISE-GRADE CONTROL Rancher supports flexible user authentication plugins and comes with pre-built user authentication integration with Active Directory, LDAP, and GitHub. Rancher supports Role-Based Access Control (RBAC) at the level of environments, allowing users and groups to share or deny access to, for example, development and production environments. 5

6 4 VERSION 3 HAPPY SNAP FEATURES All new mobile friendly interface 6

7 Updated fast loading dashboard Pre-Configured Dashboards 7

8 Role based access control with LDAP integration 8

9 Customizable Dashboards Raw Log searches 9

10 Full Stack Monitoring Alerting 10

11 Wazuh HIDS Integration Threat Intel Vulnerability Management 11

12 Event Monitor Reporting 12

13 Audit and Discovery Upgrade to Premium for more advanced features including full reporting, customizations, upgrades and support 13

14 5 PROVIDED CF TEMPLATE The SIEMonster CloudFormation template provides the means to build a scalable cluster comprising the base build for all 5 servers required. Also included within this template: Internal & external load balancer options Auto-scaling groups Security Groups RDS Backend Database DNS Zone VPN server for cluster access Multi AZ private network + VPC s Auto Mounted EFS The five servers are comprised of Proteus (Application Server) Capricorn (Application Server) Kraken (Elasticsearch) Tiamat (Elasticsearch) Makara (Rancher / Orchestration Server / Ingestion Server) For production deployment, the minimum recommended AWS instance types are as follows: SIEMonster Server Amazon Instance Makara Rancher Server /Ingest/Web c5.2xlarge Capricorn Application Server c5.2xlarge Proteus Application c5.2xlarge Kraken Elasticsearch Data Node c5.4xlarge Tiamat Elasticsearch Data Node c5.4xlarge For development purposes, these instances may be deployed with the following AWS instance types: SIEMonster Server Amazon Instance Makara Rancher Server /Ingest/Web c5.xlarge Capricorn Application Server c5.xlarge Proteus Application c5.xlarge Kraken Elasticsearch Data Node c5.xlarge Tiamat Elasticsearch Data Node c5.xlarge 14

15 6 INSTALLATION 6.1 KEY INFORMATION PRE-REQUISITES Registered domain on AWS AWS generated SSL Certificates for chosen subdomains, e.g. siemportal.corp.clientname.com (see Appendix for further details). 6.2 CLOUDFORMATION Goto the SIEMonster website and click on Download to register: Download the latest Cloudformation zip for Amazon AMI - SIEMonster. Open the AWS CloudFormation area within the AWS console Unzip the downloaded template Create Stack Choose a template Upload a template to Amazon S3 Choose file Choose file downloaded SIEMonster CF template Next Flexible options are provided. Mandatory options are indicated below, see Appendix A for further details: Note: The VPN Password, Hostname is required to connect into the SIEM remotely into AWS. This VPN Hosts removes the requirement for having the large firewall rule set from the previous version on SIEMonster. 15

16 16

17 In the above example, the web application will be deployed at: For Public DNS Zones, choose internet-facing Stack Type. For Private DNS Zones choose internal These details will be required in Section 6.3 Stack Deployment The Rancher Server will be deployed to: The VPN server will be deployed at to: SSH credentials to the hosts is rancher/s13m0nsterv3 17

18 Click on Next when finished. Options - Add IAM role if applicable, Next. Review and Create, after acknowledging IAM resource confirmation. Stack creation will take around 10 minutes to complete - refresh page to monitor status. Indication will be given on completion. Check the outputs for Rancher Server login URL. 18

19 Allow an additional 5 minutes for the Rancher cluster creation. Open the output URL and login with the credentials provided during the setup phase. (default admin/ s13m0nsterv3) Infrastructure Hosts At this stage access to the internal cluster hosts is via VPN only. Open the EC2 dashboard and identify the created instances. 19

20 6.3 STACK DEPLOYMENT The SIEMonster V3 application catalog item is pre-loaded. Navigate to the V3 catalog and click View Details for the SIEMonster V3 App. Choose V3 Under New Stack, substitute projectname for the required application name. This name will be used for your site domain in the next step Using the example details entered in section 6.2 Cloudformation: siemonster-project-projectname change this to siemportal siemonster-project-siemportal Under Configuration Options, substitute projectname for the name chosen previously in the CloudFormation template For example Name: siemonster-project-siemportal will become Site domain name: siemportal.corp.clientname.com (domain name must have 4 names) 20

21 Before After Next set the Elasticsearch JAVA HEAP SIZE per the AWS instance type deployed. For Elasticsearch Data Nodes, this should be set to a value half of the available system RAM. If you have selected C5.xlarge for a demo leave these settings as default. For the Master & Client nodes, the heap sizes can be left as default as these can be modified to suit at any time post-install. 21

22 Set the administrator address for the SIEMonster Web interface. This should be the same that will be used in Chapter 7 Web Application Setup. The remaining application passwords should be changed from the defaults, see Appendix B for change management table. Aside from the CertAuth, Truststore & KeyStore passwords, all passwords can be changed post-install if required. The SITE_ID option should be left at default, as initially the Logstash Heap Size If Gmail alert relaying is required set the appropriate values. It is recommended to setup a Gmail account specifically for this purpose. Finally, click on Launch. The stack will take around 5-10 minutes to build. The status can be viewed under Stacks User On completion, the status will turn to green for all items: Leave a few minutes for the DNS to propagate and the system health checks to complete before opening the web application URL, e.g. from the example shown previously. 22

23 7 WEB APPLICATION SETUP For the Root Domain, enter the domain name used in Section 6 e.g. siemportal.corp.clientname.com The Admin User address should be the same as that entered in section 6.3 Stack Deployment Strong passwords are enforced and must be 8 Characters in Length, upper and lower-case letters, at least 1 number, at least 1 symbol Click Setup on completion. On successful setup, a sign in page will appear: 23

24 Sign in with the credentials entered during the above Setup phase. Note that the Authentication Code for 2FA if required, can be setup after initial login. 24

25 8 USER SETUP For each logged on user there is an option available under the user menu, top right, to modify the users profile. This includes changing the display name, changing the password or adding two factor authentication. 8.1 USER ROLES User Roles are used to allow access to different components within the SIEM. Two roles are preconfigured during deployment admin and user. The admin role contains all default role options for frames (home page tiles) and dashboards (Kibana). New frames may also be added using the Create Frame option: Similarly, after creating new dashboards within Kibana, menu links to these items may be added using the Create Dashboard option. 25

26 Using the Settings option, the frame can be modified if required and an image used to reflect the properties of the frame. Similarly, the default Dashboard URLs may be modified to suit if required. 26

27 The users role is designed for new users who have been allocated login credentials without a specific role. This is useful when allocating members of an LDAP group. A single support access tile is provided. New roles may be added using the Create Role option. Access to relevant frames can be enabled and settings modified if required. If the Dashboards frame is enabled, a Dashboard settings section will appear, providing options to enable or disable dashboards specific to the role.. 27

28 9 SITE ADMINISTRATION Under the Profile option is the Site Administration option. This is used to setup site settings, new local or LDAP users, roles and custom dashboard setup for each user. 9.1 SITE settings are configured to use Mailgun, for which a free account can be setup at This mail account is for the web application only, which will send out notifications when a user logs on to the SIEM. 9.2 LDAP SETTINGS LDAP settings can be used to setup Active Directory users. It is recommended to create a group within the AD and then add users to this group who will require access. Once completed, click on Save LDAP Settings. The entered details will first be confirmed correct before being saved. LDAP users in the chosen group will now be able to login using their corporate address and active directory password. 28

29 10 OPERATIONAL OVERVIEW 10.1 LOG VIEW The Rancher Server - Platform Orchestration Management System can be accessed at the URL designated during the Cloud Formation deployment, e.g. using the credentials that were supplied. The logs for each container can be viewed within the Rancher Server UI as follows: First click on a container Next click on the menu to the right and choose View Logs: This is useful for diagnostics and maintenance, the logs for any container can be viewed in this manner. 29

30 10.2 SHELL INTERACTION Following the above steps and choosing the Execute Shell option, a terminal may be opened to each container if any maintenance is required. For access to the configuration files, rules, etc. see the following section VPN access. If any changes have been made, the container can be restarted on the main screen: 30

31 10.3 VPN ACCESS VPN via the Bastion host provides connectivity for terminal/log access to the private subnets hosting the platform. To download the client certificate first browse to the allocated URL on port 943, e.g. Change the access option to Login and use the credentials provided in the CF template. If the VPN client application is required, it may be downloaded in the next screen. The connection profile for the client can now be downloaded as highlighted below: This profile can now be imported into the VPN client. For Windows, use the import from local file option. Once imported a VPN session can be established by using the Connect as admin option from OpenVPN Connect to the IP address shown. Once connected, SSH access will be available to the SIEMonster cluster using the SSH key provided during CloudFormation deployment and username core. Host IP addresses can be found in the Rancher Server UI Infrastructure Hosts 31

32 11 SKEDLER LICENSING Reports - Menu Click on Activate License Use the provided trial license key fill out the details to activate the license. Configure the and Time Zone settings as appropriate. Options are also available for setting a proxy, Slack messages and uploading a custom logo. 32

33 33

34 Appendix A: AWS SSL Certificate requirements AWS Requirements Request a certificate for the Rancher server, e.g. for a chosen domain name of corp.clientname.com request a certificate for this domain name plus and Additional name of *.corp.clientname.com Request a certificate for the SIEMonster web application, e.g. for a chosen domain name of siemportal.corp.clientname.com request a certificate for this domain name with an Additional name of *. siemportal.corp.clientname.com Key AWS Cloud Formation Settings ZoneName AWS hosted domain name ZoneID Zone ID for the AWS hosted domain name WebAppSSLSertARN Under Certificate Management, the ARN for the SSL certificate issued for the SIEMonster web application WebAppDomain the first 2 octets of the domain name for the web application, e.g. siemportal.corp SSLCertArn The ARN for the SSL certificate issued for the Rancher Server SSHKeyName Predefined AWS key RancherUIDomain The chosen subdomain for the SIEMonster platform, e.g. corp RancherSvrInstanceClass the AWS instance type required, e.g. C5.2xlarge (should match RancherAgentsInstanceClass for basic non-ha deployments) RancherSvrVolumeSize The GP2 root volume size for Makara, minimum of 30GB RancherAgentsVolumeSize The GP2 root volume size for the remaining instances, minimum of 30GB RancherAgentsInstanceClass The AWS instance type required, e.g. C5.2xlarge RancherAdminPassword Strong password required BastionHostName VPN hostname allocation, e.g. bastion-host BastionAdminPassword Strong password required for VPN admin 34

35 Appendix B: Password Table for change management Use only Alphanumeric passwords, e.g. Ys3CretpAss624 Application Username Password Grafana (Health) admin admin Web App Mongo siemuser01 s13m0nsterv3 Mongo Hash Salt N/A 6b44d8edb86b4ca8bb8f3aaa35ddaf7d RabbitMQ admin s13m0nsterv3 Wazuh API siemonster s13m0nsterv3 Logstash logstash s13m0nsterv3 CA N/A s13m0nsterv3 411 admin admin IR admin admin Minemeld admin mimemeld Truststore N/A s13m0nsterv3 Keystore N/A s13m0nsterv3 Elastic elastic s13m0nsterv3 Beats beats s13m0nsterv3 Skedler skedler s13m0nsterv3 MySQL fouronone s13m0nsterv3 MySQL Root root s13m0nsterv3 Rancher admin s13m0nsterv3 SSH rancher s13m0nsterv3 35