Tetration Analytics - Network Analytics & Machine Learning Enhancing Data Center Security and Operations

Size: px

Start display at page:

Download "Tetration Analytics - Network Analytics & Machine Learning Enhancing Data Center Security and Operations"

Vernon Crawford
6 years ago
Views:

2 Tetration Analytics - Network Analytics & Machine Learning Enhancing Data Center Security and Operations Mike Herbert, Principal Engineer, INSBU BRKDCN-2040

3 Okay what does Tetration Mean? Tetration (or hyper-4) is the next hyperoperation after exponentiation, and is defined as iterated exponentiation It s bigger than a Google [sic] (Googol) And yes the developers are a bunch of mathematical geeks BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 3

4 Tetration Analytics Platform Introduction

5 We Are at the Cusp of a Major Shift TRADITIONAL DATA CENTRE CLOUD DATA CENTRE HYBRID CLOUDS Adoption Curve Efficiency We are here AUTOMATION IT as a Service IaaS PaaS SaaS XaaS Flexible Consumption Models CONSOLIDATION VIRTUALISATION EFFICIENCY SIMPLICITY SPEED DIGITAL EXPERIENCES The Next 5+ Years BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 5

Modern data centers are getting increasingly complex Big and

east-west traffic Expanded attack surface Open source Zero

Continuous development Application mobility Micro services

6 Modern data centers are getting increasingly complex Big and fast data Hybrid cloud Rapid app deployment Increase in east-west traffic Expanded attack surface Open source Zero trust model Multi cloud orchestration Application portability Continuous development Application mobility Micro services BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 6

What if you could actually look at every data packet header that has ever traversed the network

8 Tetration Analytics Platform Every Packet, Every Flow, Every Speed Network Policy Cisco Tetration Analytics Pervasive Visibility and Forensics Compliance Application Insight BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 8

9 Cisco Tetration Analytics Application Insights Policy Simulation and Impact Assessment Automated Whitelist Policy Generation Forensics: Every Packet, Every Flow, Every Speed Policy Compliance and Auditability BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 9

10 Cisco Tetration Analytics Pervasive Sensor Framework Provides correlation of data sources across entire application infrastructure Enables identification of point events and provides insight into overall systems behavior Monitors end-to-end lifecycle of application connectivity BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 10

11 Datacenter Wide Traffic Flow Visibility Detail information about the flow Information about Consumer Provider and type of traffic BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 11

12 Application Discovery and Endpoint Grouping BM VM VM VM BM Bare-metal, VM, & switch telemetry VM BM Cisco Nexus 9000 Series Network-only sensors, host-only sensors, or both (preferred) VM VM BM VM VM BM Brownfield Bare-metal & VM telemetry Cisco Tetration Analytics Platform BM VM VM BM Bare metal and VM VM BM BM VM VM BM On-premises and cloud workloads (AWS) VM telemetry (AMI ) Unsupervised machine learning Behavior analysis VM BM BM BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 Whitelist Policy Recommendation Application Discovery Web Tier App Tier DB Tier Storage Storage Whitelist Policy Recommendation (Available in JSON, XML, and YAML) Policy Enforcement (Future Roadmap) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 13

14 Real-Time and Historical Policy Simulation BM VM VM BM VM VM VM BM VM VM VM Cisco Tetration Analytics Platform VM VM Validating policy impact assessment in real time Simulating policy changes over historic traffic View traffic outliers for quick intelligence Audit becomes a function of continuous machine learning BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 14

and update whitelist policy with one click Policy lifecycle management

15 Policy Compliance BM VM VM BM VM VM VM VM BM VM VM VM VM VM BM VM Cisco Tetration Analytics Platform Identify policy deviations in real-time Review and update whitelist policy with one click Policy lifecycle management BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 15

16 Application Dependency Application Performance Compliance Enforcement Infrastructure Behavioral Anomalies Tetration Analytics Servers Network flows Buffer Stats Automation & Ecosystem Partners Process User Compute Application Insights Policy Forensics Network Tetration Analytics Engine PB Scale Secure Appliance BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 16

17 Tetration Analytics Platform Architecture - Sensors

18 Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors VM Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 18

19 Pervasive Sensors Host Sensors NW Sensors 3 rd Party Linux VM Windows Server VM Bare Metal (Linux and Windows Server) Hypervisors Containers Nexus 9200-X Nexus 9300-EX Geo Whois IP Watch Lists Load Balancers Available at FCS Next Generation 9K switches Future releases 3rd party Data Sources Low CPU Overhead (SLA enforced) Highly Secure (Code Signed, Authenticated) Low Network Overhead (SLA enforced) Every flow (No sampling), NO PAYLOAD BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 Traditional Monitoring Is Showing Its Age Not suited for Modern Network and Security Operations Where Data Is Created Where Data Is Useful SNMP SNMP Server Non Real time Syslog Syslog Collector Storage & Analysis CLI Scripts Strong burden on back-end Normalize different encodings, transports, data models, timestamps BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 20

21 Streaming Telemetry is a game changer Monitoring becomes a big data problem Where Data Is Created Where Data Is Useful Removing limitations and complexity Real time Streaming paradigm Dense Sensor Framework Increased Data Granularity Update on every event Multiple Data Sources Volume Scale of Data Velocity Analysis of Streaming Data Variety Different Forms of Data Big Data and Machine Learning Problem BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 21

23 Tetration Sensors Locations Hardware Sensor Packet and Flow Events Buffer and Switch State 92160CY-X 93180Y-EX 9732C-EX LC Software Sensor Processes & Socket Packet and Flow Events HYPERVISOR HYPERVISOR HYPERVISOR Tetration Cluster BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 23

Hardware Sensor Embedded Module (Flow Cache) Nexus 92160CY-X Nexus 93180Y-EX & 9732C-EX Line Cards Extracts Meta-Data from the forwarding pipeline

24 Hardware Sensor Embedded Module (Flow Cache) Nexus 92160CY-X Nexus 93180Y-EX & 9732C-EX Line Cards Extracts Meta-Data from the forwarding pipeline No latency impact, no performance impact Flow Cache PRX LUA LUB LUC BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 24

25 Software Sensor Not in the data path Sits in User Space Designed by Kernel Developers Secure Code Signed SLA Enforcement CPU and BW throttling FCS availability Windows 2008 / 2008 R2 / 2012 / 2012 R2 Linux RedHat (5.3+, 6.x) CentOS (5.11+, 6.x) Ubuntu (12.04, 14.04, 14.10) Tetration Sensor libpcap Application Network Stack Driver NIC BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 PKI within the Cluster/Sensor Tetration Cluster runs an internal PKI Root CA is per cluster, inserted at Image creation Not accessible outside the cluster Cannot connect to an external PKI Certificate based authentication is performed for the Control Channel CN of the certificate is the IP address Certificates are rotated every 60 days Sensors are code signed Signature Authority is Cisco s code signing certificate Code Signature is validated at process start BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 26

27 How Sensor Communicate with the Cluster the First Time? Register with web server via ssl Assign UUID Rails Sensor Register with web server via ssl Download config Config Server Send meta data to collectors Collector BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 Components & Communication Hardware Sensor NXOS Agent Control Channel TCP/443 Guest Shell ASIC Cisco Nexus 9000 Agent Communication Unix Socket Sensor Data UDP/5640 Tetration Cluster BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 28

29 Components & Communication Software Sensor Software Sensor Control Channel TCP-SSL 443 Sensor Data TCP-SSL 5640 Agent Communication Unix Socket Tetration Cluster LINUX/Windows/ BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 29

30 Currently Supported Platforms Windows 2008 Datacenter, Enterprise, Essentials, Standard Windows 2008 R2 Datacenter, Enterprise, Essentials, Standard Windows 2012 Datacenter, Enterprise, Essentials, Standard Windows 2012 R2 Datacenter, Enterprise, Essentials, Standard RedHat Enterprise Server 5.3 & above 6.x CentOS 5.11 & above 6.x Ubuntu This list will grow based on what you need and ask for BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 30

33 Tetration Analytics Platform Architecture - Sensor Data

34 Looking Beyond Connectivity Application Processes and Sockets Socket > 1023 Socket = 443 Chrome NGINX Consumer Process Provider/Service Process Application developers implement business logic as code that runs as processes and threads TCP/IP which forms a foundation of the Internet was designed to allow these application processes to interact via sockets Application logic can be viewed on one level as the interaction between a group of processes and their associated sockets Understanding the inter-process communication and mapping that directly to the infrastructure provides a direct correlation between the application and the infrastructure BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 34

35 Looking Beyond Connectivity Application Processes and Sockets Socket > 1023 Socket = 80 Chrome NGINX Consumer Process Provider/Service Process #create an INET, STREAMing socket s = socket.socket( socket.af_inet, socket.sock_stream) #now connect to the web server on port 80 # - the normal http port s.connect((" 80)) #create an INET, STREAMing socket serversocket = socket.socket( socket.af_inet, socket.sock_stream) #bind the socket to a public host, # and a well-known port serversocket.bind((socket.gethostname(), 80)) #become a server socket serversocket.listen(5) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 35

36 What do we mean by Application Visibility Internet Stack Application Application Process Process Process Process Sockets Sockets Transport Transport Network Network Network Network Data Link Data Link Data Link Data Link Physical Physical Physical Physical BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 36

37 What Does Tetration Sensor Collect Socket Connectivity, the data flows Application Application Process Process Process Process Sockets Sockets Transport Transport Network Network Network Network Data Link Data Link Data Link Data Link Physical Physical Physical Physical BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 What does the Sensor Collect Context Application Process Information: Which process is it, who started it, etc. Device Information: Buffer/ACL Drops, etc. Application Process Process Process Process Sockets Sockets Transport Transport Network Network Network Network Data Link Data Link Data Link Data Link Physical Physical Physical Physical BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 38

39 Sensor Data Process Information Host Sensor collects information about the consumer and provider processes /proc runtime system information (e.g. system memory, devices mounted, hardware configuration, etc). BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 39

40 Additional Context External Data Sources CMDB, DNS, whois, Talos (future), etc. Application Application Process Process Process Process Sockets Transport Sockets Transport Network Network Network Network Data Link Data Link Data Link Data Link APIC Physical Physical Physical Physical Pervasive Sensors Tetration Analytics Engine BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 40

41 What does the Sensor Collect Socket Level Flow Information + Context Information Understanding of what happens TO and INSIDE a flow Distributions (packet sizes, TCP windows ) Burstiness Anomaly detection Latency (application and network) Events VXLAN information Per Packet Variations Length 66 Length 9000 Accumulated Flow Information (Volume ) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 41

43 Full vs. Sampled Reasons and Use Cases for Both Sampled Sampling has it s use cases, in SP environments for example High Volume, no behavioral analysis Sampling provides a good statistical model For Trends For Traffic Visibility For Volume Indication Full Depending on the number of flows and type of flows Mice flows can go completely unseen Connection Oriented flows may not be tracked properly (missed flags) Accuracy of the flow increases with the packet count Type of sampling and quality of entropy Entropy is very important BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 43

44 Tetration Examines every packet Full Packet Stream Variability within the flow Variability between the flows Changes within the flow BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 44

45 Collects the Meta-Data not the Packet Meta-Data Including Overlay VXLAN/GRE/IPinIP Encapsulated Header Ethernet Header IP Header UDP Header VXLAN Header Ethernet Header IP Header TCP Header Payload Ethernet Header IP Header TCP Header Payload Ethernet Header IP Header UDP Header Payload Privacy Risk BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 45

46 Sensor Data Flow Data Forwarding COS Overlay Type (Native, 802.1q / 802.1p, VXLAN, ivxlan, NVGRE, NSH, other) Source TEP or Port ID Destination TEP Disposition (RPF or Port Security failure, Policy drop, redirect or span) Port type (spine to leaf or leaf to host) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 46

47 Sensor Data Accumulated Flow Information Bytes, Packet Count IP options present IP length error DF bit set Fragment seen Last TTL Accumulated TCP flags Last ACK / SEQ Sampled Packet length Sampled Packet ID BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 47

48 Sensor Data Histogram Bins #1 82 bits #2 82 bits #3 0 bits #4 165 bits Flow Cache has the notion of bins to build histograms TCP options length (8 bits) Payload length (12 bits) Receive window (6 bits) This means more visibility on the activity of flow #5 82 bits #6 82 bits #7 130 bits #8 165 bits Export Bin sizes are configurable Bins don t need to be of equal size (but need to be contiguous) Last bin will capture the configured size and above = Histogram of the flow Export BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 48

49 Sensor Data Burst Measure the burstiness of a flow Current Burst Max Burst Burst Index Flowlets Burst are measured in 32k interval Each export period is divided by 128 Flowlets are activity after a silence period (configurable) Current 128 Max 128 Burst Index - 0 Current 256 Max 256 Burst Index - 3 Current 32 Max 256 Burst Index - 3 Current 1024 Max 1024 Burst Index - 80 Current 0 Max 1024 Burst Index Flowlet #1 Silence Flowlet #2 Max Burst occurred at 62.5ms with a value of 1024 and 2 flowlets BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 49

50 Sensor Data Anomaly List TTL changed IP reserved flags are not 0 DF bit has changed Ping of death Fragment is too small to contain L4 header (TCP, UDP and SCTP) TCP SYN and FIN are set TCP SYN and RST are set TCP FIN, PSH and URG are set TCP flags are zero d TCP SYN with data TCP FIN with no ACK TCP RST with no ACK TCP SYN, FIN, RST and ACK zero d URG set but no URG pointer URG pointer with no URG flag TCP seq outside the expected range TCP seq is less than expected (rexmit) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 50

51 Sensor Data Events RTT Sample Way of approximating the RTT based on specific packet characteristics Preset ACK & SEQ Approximation as this includes the OS network stack Uses sampling, sample taken every 8192 bytes (by default, configurable) Tracks ACK for these specific SEQ and creates an event for each By using this global configuration, if return path is via another switch the ACK is still tracked BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 51

52 RTT Sample Example Event Triggered Event time stamped Event time stamped TCP SEQN 100 TCP ACK 100 TCP SEQN 8192 TCP ACK 8192 RTT = Event ACK TS Event SEQ TS BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 52

53 Sensor Data Events Mouse Packet Export the first n packets of a flow (configurable) Analytics Changed A parameter of the flow has changed (bit mask comparison), 1 mask configurable Packet Value Match A packet field contains a specific value, 1 field configurable (mask + value) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 53

54 Sensor Data Example Let s take this Web page request as an example Assumption is that it s the first connection, this is a new flow One flow is created per direction Flow Export Flow A B A A B B A A B B A BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 54

59 Pervasive Visibility Flow Search and Forensics

65 Tetration Analytics Platform Architecture - Cluster

66 Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors VM Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 66

67 The Analytics Cluster Components Hadoop Based Platform Self managed One touch deployment Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Messaging Bus API Access Front End Compute (Data Cleaning and Analytics) Caching (Search) Long Term Storage (Data Lake) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 67

68 The Analytics Cluster Appliance The Analytics Cluster operates as an appliance Avoids the need for in house Big Data, Analytics expertise Supported by Cisco TAC Self Monitoring The cluster leverages a sensor architecture to track it s state and provides event based notifications for Software upgrades and full install are all automated BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 68

Cluster Monitoring and Maintenance BRKDCN-2040 2016 Cisco

75 Analytics Engine The Platform Hadoop Based Platform Self managed One touch deployment Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Messaging Bus API Access Front End Compute (Data Cleaning and Analytics) Caching (Search) Long Term Storage (Data Lake) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 75

76 Front End GUI, RESTful API, Messaging BUS Servers hosting front end processes GUI and Operational Interfaces RESTful API (post FCS) Messaging BUS (post FCS) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 76

Scaled to Millions of events per second BRKDCN-2040 2016

77 Data Processing Pipeline Data Ingest and Processing Multiple Pipelines for different processing activities Scaled to Millions of events per second BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 77

Caching Layer Natural Language Search BRKDCN-2040 2016

Caching Layer Search Caching Layer provides a large in memory and flash based data store for real time searches e.g. 16 weeks of policy delta data accessible for real time search BRKDCN-2040 2016 Cisco and/or its affiliates.

79 Caching Layer Search Caching Layer provides a large in memory and flash based data store for real time searches e.g. 16 weeks of policy delta data accessible for real time search BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 79

Data Lake HDFS Storage Long Term Storage for collected observations, for pipeline processing tasks, etc Usage is based on Time Based Retention Space Based Retention Greedy Retention Max possible

80 Data Lake HDFS Storage Long Term Storage for collected observations, for pipeline processing tasks, etc Usage is based on Time Based Retention Space Based Retention Greedy Retention Max possible Retention period will depend on cluster size and observation rate K hours of available capacity at the current collection rates (587 days) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 80

81 Standard Data Analytics Pipeline Tetration Data Analysis Various Pipelines (e.g. ADM) process the data to derive appropriate insights Data Aggregation Automated Data Discovery& Evaluation Data Prep & cleansing Statistical Analysis & Prediction Tools Reporting, Visualization or Alerts Sensor Collectors De-duplication, unification of unidirectional flows into bi-directional, annotate flows with context information, etc. GUI, REST API, Kafka, Policy Export, BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 81

82 Data Collection Sensor to Collector Data Aggregation Automated Data Discovery& Evaluation Data Prep & cleansing Statistical Analysis & Prediction Tools Reporting, Visualization or Alerts BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 82

83 Process Process Process Process Data Prep Application Sockets Transport Network Data Link Physical Network Data Link Physical Network Data Link Physical Application Sockets Transport Network Data Link Physical Collector Collector De-duplication, unification of unidirectional flows into bi-directional, annotate flows with context information, etc. Data Aggregation Automated Data Discovery& Evaluation Data Prep & cleansing Statistical Analysis & Prediction Tools Reporting, Visualization or Alerts BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 83

84 Analyzing the Data Endpoints are iteratively compared with each other to find which profiles are most similar Sensor Data: Ports provided and consumed, Addresses sent and received from, Properties of network flows, Running processes, Process originating flow, Hostname, External Context: Load balancers / DNS / route tags Human approved clusters from current or other workspaces and base cluster definition This is an example of where we use machine leaning Data Aggregation Automated Data Discovery& Evaluation Data Prep & cleansing Statistical Analysis & Prediction Tools Reporting, Visualization or Alerts BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 84

Machine Learning Cognitive Computing - Finding and remembering all the relationships between data, querying the matrix of relationships (Watson) Machine Learning - Remember what has happened before

85 Machine Learning Cognitive Computing - Finding and remembering all the relationships between data, querying the matrix of relationships (Watson) Machine Learning - Remember what has happened before and then look at new data coming in that context to try and find patterns, build up a body of knowledge and then use that data to make a decision based on the new data. Can machines remember and apply what they remember to new data Deep Learning - Not trying to maintain data and relationships over time but analyze that data through better representations and create model to learn these representations from large scale unlabeled data. Succession analysis BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 85

Machine Learning A "Field of study that gives computers the ability to learn without being explicitly programmed Arthur Samuel (1959) The programmers construction of algorithms that can learn from

86 Machine Learning A "Field of study that gives computers the ability to learn without being explicitly programmed Arthur Samuel (1959) The programmers construction of algorithms that can learn from and make predictions on data (as opposed to static programming instructions). 7:00 am = 65 degrees 8:00 am = 75 degrees 9:00 am = 85 degrees 77.5 degrees How warm will it be at 8:30 am tomorrow? Supervised learning: Linear regression, Logistics regression, SVMs Unsupervised learning: K-means, PCA, Anomaly detection BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 86

88 K-means Algorithm Finding the Clusters Randomly initialize cluster centroids Repeat { for = 1 to := index (from 1 to ) of cluster centroid closest to for = 1 to := average (mean) of points assigned to cluster } BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 88

98 Silhouetting Validation of the Cluster The silhouette value is a measure of how similar an object is to its own cluster (cohesion) compared to other clusters (separation) Produces a higher degree of probability that the clustering is representational BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 98

101 Analyzing the Data Fitting the Curve Every data set (e.g. flow) is examined to find the best function that describes it s behaviour Comparison within and between flows can be used to find outlier or anomaly conditions Data Aggregation Automated Data Discovery& Evaluation Data Prep & cleansing Statistical Analysis & Prediction Tools Reporting, Visualization or Alerts BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 101

102 Visual Query with Flow Exploration Replay flow details like a DVR Information mapped across 25 different dimensions Thick lines indicate common flows Faint lines indicate uncommon flows BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 102

103 Outliers Switch on Outlier view to highlight uncommon flows What does ot look like it fits Outlier dimension is highlighted with purple circle BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 103

104 Tetration Application Insight

Why This Approach Is Different App Insight derived based on actual communication Automated grouping of similar endpoints in a cluster Keep your App Insight

105 Why This Approach Is Different App Insight derived based on actual communication Automated grouping of similar endpoints in a cluster Keep your App Insight up-to-date based on application evolution Flexibility of using hardware or software sensors BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 105

all the parts of a service that should be migrated together to the

application ACI application profiles, end point groups, and

108 Why should I understand dependencies? Identify a single point of failure that should be replicated Find all the parts of a service that should be migrated together to the cloud Replace infrastructure components of an undocumented application ACI application profiles, end point groups, and contracts based on applications BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 108

Understand the communication Load Balancer Database App BRKDCN-2040

$Enforcement Anywhere Whitelist policy Cisco Tetration Analytics Data Whitelist policy { "src_name": "App", "dst_name": "Web", "whitelist": [ {"port": [ 0, 0 ],"proto": 1,"action": "ALLOW"}, {"port":$

115 Enforcement Anywhere Whitelist policy Cisco Tetration Analytics Data Whitelist policy { "src_name": "App", "dst_name": "Web", "whitelist": [ {"port": [ 0, 0 ],"proto": 1,"action": "ALLOW"}, {"port": [ 80, 80 ],"proto": 6,"action": "ALLOW"}, {"port": [ 443, 443 ],"proto": 6,"action": "ALLOW"} ] } Amazon Web Services Public Cloud Microsoft Azure Google Cloud Linux and Microsoft Windows Servers and VM Cisco ACI and Cisco Nexus 9000 Series Standalone Cisco ACI EGP/Contract Integration via Cisco ACI Toolkit Traditional Network ACL Firewall Rules Host Firewall Rules BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 115

116 Application Centric, Okay but how do I get there?

117 Policy Creation Flow Cisco Tetration Analytics Application Policy APIC Export Clusters and Policies in JSON/XML format Data ACI Toolkit Network Policy Import Policy using ACI Toolkit Automatic creation of EPGs and Contracts Nexus 9K BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 117

currently Intuitive names Not full functionality, most common Focused primarily on configuration APIC Preserves ACI

118 ACI Toolkit Simple toolkit built on top of APIC API Set of simple python classes Python Library Used to generate REST API calls Runs locally NX-OS like CLI Linux Commands ACI Toolkit Custom Python Scripts Small number of classes ~30 currently Intuitive names Not full functionality, most common Focused primarily on configuration APIC Preserves ACI basic concepts Tenants, EPGs, Contracts, etc. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 118

119 Configpush Application Runs as a command line tool or a REST service. The initial expected usage is as a command line tool. Command line tool is here: tool.py Takes the JSON provided by Tetration and pushes to the APIC. It requires the APIC credentials and which tenant/app profile to place the EPGs. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 119

120 Configpush Application Syntax python apic_tool.py -h usage: apic_tool.py [-h] [--maxlogfiles MAXLOGFILES] [--debug [{verbose,warnings,critical}]] [--config CONFIG] [-u URL] [-l LOGIN] [-p PASSWORD] [--displayonly] [--tenant TENANT] [--app APP] optional arguments: -h, --help show this help message and exit --maxlogfiles MAXLOGFILES Maximum number of log files (default is 10) --debug [{verbose,warnings,critical}] Enable debug messages. --config CONFIG Configuration file -u URL, --url URL APIC IP address -l LOGIN, --login LOGIN APIC login ID. -p PASSWORD, --password PASSWORD APIC login password. --displayonly Only display the JSON configuration. Do not actually push to the APIC. --tenant TENANT Tenant name for the configuration --app APP Application profile name for the configuration BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 120

121 Policy Simulation and Compliance

125 Publish, export, and enforce Policy App Load Balancer Database Load Balancer Provides Port 3306 Database Provides Port 3306 BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 125

132 Policy Compliance Verification & Simulation What was seen on the network that was out of Policy Permitted Traffic Seen on the network BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 132

133 Summary

134 ACI Architecture ACI Intent (May) Traffic Analysis Lots of Data Configuration Analysis Very Large State-Space Analytics (Did) ADM Security Forensics Guarantees Compliance Consistency Assurance (Can) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 134

135 Summary VM Pervasive flow telemetry that supports infrastructure for multiple data centers at scale Ready-to-use solution to address critical data center operational use cases Self-monitoring and eliminate the need for in-house big data expertise Open platform and northbound APIs enable transparent integration Accelerated adoption and comprehensive Solution support with Services BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 135

136 Q & A

Complete Your Online Session Evaluation Give us your feedback and receive a Cisco 2016 T-Shirt by completing the Overall Event Survey and 5 Session Evaluations.

137 Complete Your Online Session Evaluation Give us your feedback and receive a Cisco 2016 T-Shirt by completing the Overall Event Survey and 5 Session Evaluations. Directly from your mobile device on the Cisco Live Mobile App By visiting the Cisco Live Mobile Site Visit any Cisco Live Internet Station located throughout the venue T-Shirts can be collected Friday 11 March at Registration Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 137

138 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Lunch & Learn Meet the Engineer 1:1 meetings Related sessions BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 138

139 Thank you

140

Self-driving Datacenter: Analytics

Self-driving Datacenter: Analytics George Boulescu Consulting Systems Engineer 19/10/2016 Alvin Toffler is a former associate editor of Fortune magazine, known for his works discussing the digital revolution,