Eyes Everywhere: Monitoring Today's Borderless Landscape

Transcription

1 SESSION ID: CMI1-R09 Eyes Everywhere: Monitoring Today's Borderless Landscape Bill Shinn Principal Security Architect Amazon Web

2 What we ll cover today Event & Finding Reference Architecture Generating Events and Findings Old and New Design Patterns for Collection/Ingestion/Aggregation Approaches to Processing Analysis and Workflow Call to Action 2

3 Event & Finding Reference Architecture

4 Simple Definitions - Events & Findings Events - one time record - or series of records that fire - where you can't change the state of what happened. Examples: record of user activity, details of a network flow, parameters of an API call request/response 4

5 Simple Definitions - Events & Findings Events - one time record - or series of records that fire - where you can't change the state of what happened. Examples: record of user activity, details of a network flow, parameters of an API call request/response Finding - longer-lived information which reflect the state of something that can be changed. Examples: vulnerability scan result, patch state, software defect, build status, threat level, undesireable state of user entitlements 5

6 Event & Finding Reference Architecture Event & Finding Generation Event & Finding Collection/Ingestion Aggregation Event & Finding Processing Event & Finding Analysis & Workflow 6

7 Event & Finding Generation

8 Event Generation Traditional Sources Source code you write (e.g log.debug( foodebug ); ) Source code you configure (e.g. log4j.properties ) Source code you configure with json, inf, xml files - DEBUG, WARN, INFO, etc.) 8

9 Event Generation Traditional Sources Source code you write (e.g log.debug( foodebug ); ) Source code you configure (e.g. log4j.properties ) Source code you configure with json, inf, xml files - DEBUG, WARN, INFO, etc.) Operating System log configuration (*nix syslog.conf, Windows Event Log properties) Network devices (router/switch log configuration, firewall changes and drop/accept configuration, IDS/IPS signature set/severity configuration) Security services (application source code scanner job status, vulnerability scanner job status) 9

10 Finding Generation - Traditional Sources Traditional Sources Patch evaluation task results missing patches Vulnerability scanner results open CVEs Unit test results failed classes/code base + error, failed build Application security assessments (software defects) Questionnaires s of vendor due diligence output Access/Entitlement review results (elevated privileges, affirm/reapprove) 10

11 Demo #1 Events vs Findings from a CVE Assessment Tool 11

12 Streaming Reference Architecture Assessment/CVE Agent (eg Amazon Inspector) Source Code Repository CI/CD Task Build Target Server Image Factory flow logs 12

13 Streaming Reference Architecture Assessment/CVE Agent (eg Amazon Inspector) Events (ticketing to ops on failure) Source Code Repository CI/CD Task Build Target Findings (backlog or Jira issue to devs or server build team) Server Image Factory flow logs 13

14 Event Generation Modern Sources Cloud platform logs (entitlement management events, infrastructure events such as instance launches or network configuration changes) Server-less application logs/streams (cloud-based functions, object storage notification) PaaS & managed database services logs (not in a file, but accessible only via API or table) SaaS logs delivered via download or API feed 14

15 Demo #2 Cloud Service Provider Platform Logging 15

16 Demo #3 Server-less Architecture Logging 16

17 Demo #3 (reference architecture) 17

18 Event & Finding Collection/Ingestion/Aggregation

19 Event Collection Traditional Forms Files & remote syslog (ultimately another file, but with some filtering on the way) Database tables Windows Event Log 19

20 Finding Collection Traditional Forms Static reports (console, csv/excel, pdf) Findings entered into a database table 20

21 Event & Finding Collection Modern Forms Event Streams Findings available via an API 21

22 Demo #4 Event Streams 22

23 Streaming Reference Architecture Amazon EC2 Agent-based Logging Amazon CloudWatch (Log Group/Log Stream) Subscription AWS Lambda Amazon Elasticsearch Service flow logs 23

24 Streaming Reference Architecture Amazon EC2 Agent-based Logging Amazon CloudWatch (Log Group/Log Stream) Subscription AWS Lambda Subscription Amazon Elasticsearch Service Network Traffic Stream Amazon EC2 Elastic Network Interface Amazon VPC Flow Logs Amazon CloudWatch (Log Group/Log Stream) flow logs 24

25 Event & Finding Processing

26 Event & Finding Processing Rules-engine application MapReduce tasks Elasticsearch cluster 26

27 Demo #5 Cloud-based rules engine 27

28 Rules Engine Reference Architecture AWS CloudTrail Event Selector Amazon CloudWatch Events Rule Target AWS Lambda Output of function flow logs 28

29 Event & Finding Analysis and Workflow

30 Event & Finding Analysis Commercial log/event search tools and product consoles SIEM tools Kibana (part of ELK stack) 30

31 Analyzing Ephemeral Events & Findings What happens when you log something from a server that no longer exists? Even more with serverless architectures, what happens when you log something from a function call that didn't exist as a running set of code loaded into a servlet container or container until it was invoked? How? Track the configuration, not the configuration item Correlate events from no-longer-existing assets with platform events 31

32 Event & Finding Enrichment Meta data on objects Resolver groups Data classification Correlation or joins on other sources Extract fields and key off critical key/value pairs to perform lookups on related data such as last change, related objects 32

33 Event & Finding Workflow Ticketing or case management system DevOps collaboration platforms (Slack, HipChat, etc.) 33

34 Demo #6 Cloud-based event workflow 34

35 Cloud-based Workflow Reference Architecture AWS CloudTrail Event Selector Amazon CloudWatch Events Rule Target AWS Lambda flow logs 35

36 Applied Architecture Call to action weeks Document event & finding flow for 2 critical applications Keep it to super simple documentation like this and check into revision control: Java properties -> flat file -> file monitor agent -> aggregation microservice -> object storage -> Elasticsearch) Java properties -> flat file -> file monitor agent -> aggregation microservice -> stream processing rules engine-> ticketing API) Pair up security analysts with developers to understand log generation and events of interest in 1-2 critical applications 36

37 Applied Architecture Call to action - 3 months Stand up a centralized log ingestion platform Create micro-services to integrate event/finding processing & analysis tiers with actionable workflow systems 37