Self-driving Datacenter: Analytics

Transcription

1 Self-driving Datacenter: Analytics George Boulescu Consulting Systems Engineer 19/10/2016

2 Alvin Toffler is a former associate editor of Fortune magazine, known for his works discussing the digital revolution, communication revolution, and technological singularity

3 Define Security The conscious or unconscious acceptance of a risk in relation of the probability of this becoming to be reality in a delta Time

4 Datacenter Evolution

5 Datacenter Evolution

6 We Are at the Cusp of a Major Shift TRADITIONAL DATA CENTRE CLOUD DATA CENTRE HYBRID CLOUDS Adoption Curve We are here Efficiency AUTOMATION IT as a Service IaaS PaaS SaaS XaaS Flexible Consumption Models CONSOLIDATION VIRTUALISATION EFFICIENCY SIMPLICITY SPEED DIGITAL EXPERIENCES The Next 5+ Years 6

7 Modern data centers are getting increasingly complex Big and fast data Hybrid cloud Rapid app deployment Increase in east-west traffic Expanded attack surface Open source Zero trust model Multi cloud orchestration Application portability Continuous development Application mobility Micro services

8 What if you could actually look at every data packet header that has ever traversed the network without sampling? 8

9 ACI Architecture Analytics POLICY ACI Intent (May) Traffic Analysis Lots of Data Configuration Analysis Very Large State- Space Analytics (Did) ADM Security Forensics Guarantees Compliance Consistency Assurance (Can) 9

10 Tetration Analytics Platform Every Packet, Every Flow, Every Speed Network Pervasive Policy Visibility Cisco Tetration Analytics and Forensics Compliance Application Insight 10

11 Cisco Tetration Analytics Application Policy Automated Forensics: Policy Insights Simulation Whitelist Policy Every Packet, Compliance and Impact Generation Every Flow, and Assessment Every Speed Auditability 11

12 Cisco Tetration Analytics Pervasive Sensor Framework Provides correlation of data sources across entire application infrastructure Enables identification of point events and provides insight into overall systems behavior Monitors end-to-end lifecycle of application connectivity 12

13 Application Discovery and Endpoint Grouping BM BM Bare-metal,, & switch telemetry BM Cisco Nexus 9000 Series Network-only sensors, host-only sensors, or both (preferred) BM BM Brownfield Bare-metal & telemetry Cisco Tetration Analytics Platform BM BM Bare metal and BM BM BM On-premises and cloud workloads (AWS) telemetry (AMI ) Unsupervised machine learning Behavior analysis BM BM 13

14 Whitelist Policy Recommendation Application Discovery WebTie r AppTier DB Tier Storage Storage Whitelist Policy Recommendation (Available in JSON, XML, and YAML) Policy Enforcement (Future Roadmap) 14

15 Real-Time and Historical Policy Simulation BM BM BM Cisco Tetration Analytics Platform Validating policy impact assessment in real time Simulating policy changes over historic traffic View traffic outliers for quick intelligence Audit becomes a function of continuous machine learning 15

16 Policy Compliance BM BM Cisco Tetration BM BM Analytics Platform Identify policy deviations in real-time Review and update whitelist policy with one click Policy lifecycle management 16

17 Tetration Analytics Servers Network flows Buffer Stats Application Dependency Application Performance Automation & Compliance Enforcement Infrastructure Behavioral Anomalies Ecosystem Partners Process User Compute Application Insights Policy Forensics Network Tetration Analytics Engine PB Scale Secure Appliance 17

18 Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events 18

19 Pervasive Sensors Host Sensors NW Sensors 3 rd Party Linux Windows Server Bare Metal (Linux and Windows Server) Hypervisors Containers Nexus 9200-X Nexus 9300-EX Geo Whois IP Watch Lists Load Balancers Available at FCS Next Generation 9K switches Future releases 3rd party Data Sources ü Low CPU Overhead (SLA enforced) ü Highly Secure (Code Signed, Authenticated) ü Low Network Overhead (SLA enforced) ü Every flow (No sampling), NO PAYLOAD 19

20 Traditional Monitoring Is Showing Its Age Not suited for Modern Network and Security Operations Where Data Is Created Where Data Is Useful SNMP SNMP Server Non Real time Syslog CLI Syslog Collector Scripts Storage & Analysis Strong burden on backend Normalize different encodings, transports, data models, timestamps 20

21 Streaming Telemetry is a game changer Monitoring becomes a big data problem Where Data Is Created Where Data Is Useful Removing limitations and complexity Real time Streaming paradigm Dense Sensor Framework Increased Data Granularity Update on every event Multiple Data Sources Volume Scale of Data Velocity Analysis of Streaming Data Variety Different Forms of Data Big Data and Machine Learning Problem 21

22 Why Multiple Sensors? Example monitoring temperature in a room Lamp Sensor Plug Sensor Heater 22

23 Tetration Sensors Locations 9732C-EX LC Hardware Sensor Packet and Flow Events Buffer and Switch State Software Sensor Processes & Socket Packet and Flow Events 92160CY-X 93180Y-EX HYPERVISOR HYPERVISOR HYPERVISOR Tetration Cluster 23

24 Hardware Sensor Embedded Module (Flow Cache) Nexus 92160CY-X Nexus 93180Y-EX & 9732C-EX Line Cards Extracts Meta-Data from the forwarding pipeline No latency impact, no performance impact Flow Cache PRX LUA LUB LUC 24

25 Software Sensor Not in the data path Sits in User Space Designed by Kernel Developers Secure Code Signed SLA Enforcement CPU and BW throttling FCS availability Windows 2008 / 2008 R2 / 2012 / 2012 R2 Linux RedHat (5.3+, 6.x) CentOS (5.11+, 6.x) Ubuntu (12.04, 14.04, 14.10) Tetration Sensor libpcap Application Network Stack Driver NIC 25

26 Methods to deploy the sensor 26

27 Coming soon to a GitHub near you github.com/datacenter 27

28 Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events 28

29 The Analytics Cluster Components Front End Compute (Data Cleaning and Hadoop Based Platform Self managed One touch deployment Analytics) Caching (Search) Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Long Term Storage Messaging Bus API Access (Data Lake) 29

30 The Analytics Cluster Appliance The Analytics Cluster operates as an appliance Avoids the need for in house Big Data, Analytics expertise Supported by Cisco TAC Self Monitoring The cluster leverages a sensor architecture to track it s state and provides event based notifications for Software upgrades and full install are all automated 30

31 FCS Analytics Cluster Configurations 4 x 3-Phase PDU 22.5 KW Peak Power 4 x 1-Phase PDU 11.5 KW Peak Power 31

32 Options for Future Cluster Models 32

33 Analytics Engine The Platform Front End Compute (Data Cleaning and Hadoop Based Platform Self managed One touch deployment Analytics) Caching (Search) Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Long Term Storage Messaging Bus API Access (Data Lake) 33

34 Front End GUI, RESTful API, Messaging BUS Servers hosting front end processes GUI and Operational Interfaces RESTful API (post FCS) Messaging BUS (post FCS) 34

35 ACI Architecture ACI Intent (May) Traffic Analysis Lots of Data Configuration Analysis Very Large State- Space Analytics (Did) ADM Security Forensics Guarantees Compliance Consistency Assurance (Can) 35

36 Summary Pervasive flow Ready-to-use Self-monitoring and Open platform and Accelerated telemetry that solution to address eliminate the need northbound APIs adoption and supports critical data center for enable transparent comprehensive infrastructure for operational in-house big data integration Solution multiple data use cases expertise support with centers at scale Services 36

37