Self-driving Datacenter: Analytics George Boulescu Consulting Systems Engineer 19/10/2016
Alvin Toffler is a former associate editor of Fortune magazine, known for his works discussing the digital revolution, communication revolution, and technological singularity
Define Security The conscious or unconscious acceptance of a risk in relation of the probability of this becoming to be reality in a delta Time
Datacenter Evolution
Datacenter Evolution
We Are at the Cusp of a Major Shift TRADITIONAL DATA CENTRE CLOUD DATA CENTRE HYBRID CLOUDS Adoption Curve We are here Efficiency AUTOMATION IT as a Service IaaS PaaS SaaS XaaS Flexible Consumption Models CONSOLIDATION VIRTUALISATION EFFICIENCY SIMPLICITY SPEED DIGITAL EXPERIENCES 2000 2010 2015 The Next 5+ Years 6
Modern data centers are getting increasingly complex Big and fast data Hybrid cloud Rapid app deployment Increase in east-west traffic Expanded attack surface Open source Zero trust model Multi cloud orchestration Application portability Continuous development Application mobility Micro services
What if you could actually look at every data packet header that has ever traversed the network without sampling? 8
ACI Architecture Analytics POLICY ACI Intent (May) Traffic Analysis Lots of Data Configuration Analysis Very Large State- Space Analytics (Did) ADM Security Forensics Guarantees Compliance Consistency Assurance (Can) 9
Tetration Analytics Platform Every Packet, Every Flow, Every Speed Network Pervasive Policy Visibility Cisco Tetration Analytics and Forensics Compliance Application Insight 10
Cisco Tetration Analytics Application Policy Automated Forensics: Policy Insights Simulation Whitelist Policy Every Packet, Compliance and Impact Generation Every Flow, and Assessment Every Speed Auditability 11
Cisco Tetration Analytics Pervasive Sensor Framework Provides correlation of data sources across entire application infrastructure Enables identification of point events and provides insight into overall systems behavior Monitors end-to-end lifecycle of application connectivity 12
Application Discovery and Endpoint Grouping BM BM Bare-metal,, & switch telemetry BM Cisco Nexus 9000 Series Network-only sensors, host-only sensors, or both (preferred) BM BM Brownfield Bare-metal & telemetry Cisco Tetration Analytics Platform BM BM Bare metal and BM BM BM On-premises and cloud workloads (AWS) telemetry (AMI ) Unsupervised machine learning Behavior analysis BM BM 13
Whitelist Policy Recommendation Application Discovery WebTie r AppTier DB Tier Storage Storage Whitelist Policy Recommendation (Available in JSON, XML, and YAML) Policy Enforcement (Future Roadmap) 14
Real-Time and Historical Policy Simulation BM BM BM Cisco Tetration Analytics Platform Validating policy impact assessment in real time Simulating policy changes over historic traffic View traffic outliers for quick intelligence Audit becomes a function of continuous machine learning 15
Policy Compliance BM BM Cisco Tetration BM BM Analytics Platform Identify policy deviations in real-time Review and update whitelist policy with one click Policy lifecycle management 16
Tetration Analytics Servers Network flows Buffer Stats Application Dependency Application Performance Automation & Compliance Enforcement Infrastructure Behavioral Anomalies Ecosystem Partners Process User Compute Application Insights Policy Forensics Network Tetration Analytics Engine PB Scale Secure Appliance 17
Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events 18
Pervasive Sensors Host Sensors NW Sensors 3 rd Party Linux Windows Server Bare Metal (Linux and Windows Server) Hypervisors Containers Nexus 9200-X Nexus 9300-EX Geo Whois IP Watch Lists Load Balancers Available at FCS Next Generation 9K switches Future releases 3rd party Data Sources ü Low CPU Overhead (SLA enforced) ü Highly Secure (Code Signed, Authenticated) ü Low Network Overhead (SLA enforced) ü Every flow (No sampling), NO PAYLOAD 19
Traditional Monitoring Is Showing Its Age Not suited for Modern Network and Security Operations Where Data Is Created Where Data Is Useful SNMP SNMP Server Non Real time Syslog CLI Syslog Collector Scripts Storage & Analysis Strong burden on backend Normalize different encodings, transports, data models, timestamps 20
Streaming Telemetry is a game changer Monitoring becomes a big data problem Where Data Is Created Where Data Is Useful Removing limitations and complexity Real time Streaming paradigm Dense Sensor Framework Increased Data Granularity Update on every event Multiple Data Sources Volume Scale of Data Velocity Analysis of Streaming Data Variety Different Forms of Data Big Data and Machine Learning Problem 21
Why Multiple Sensors? Example monitoring temperature in a room Lamp Sensor Plug Sensor Heater 22
Tetration Sensors Locations 9732C-EX LC Hardware Sensor Packet and Flow Events Buffer and Switch State Software Sensor Processes & Socket Packet and Flow Events 92160CY-X 93180Y-EX HYPERVISOR HYPERVISOR HYPERVISOR Tetration Cluster 23
Hardware Sensor Embedded Module (Flow Cache) Nexus 92160CY-X Nexus 93180Y-EX & 9732C-EX Line Cards Extracts Meta-Data from the forwarding pipeline No latency impact, no performance impact Flow Cache PRX LUA LUB LUC 24
Software Sensor Not in the data path Sits in User Space Designed by Kernel Developers Secure Code Signed SLA Enforcement CPU and BW throttling FCS availability Windows 2008 / 2008 R2 / 2012 / 2012 R2 Linux RedHat (5.3+, 6.x) CentOS (5.11+, 6.x) Ubuntu (12.04, 14.04, 14.10) Tetration Sensor libpcap Application Network Stack Driver NIC 25
Methods to deploy the sensor 26
Coming soon to a GitHub near you github.com/datacenter 27
Tetration Analytics Architecture Overview Data Collection Analytics Engine Visualization and Reporting Host Sensors Tetration Telemetry Web GUI Network Sensors Cisco Nexus 92160YC-X Cisco Nexus 93180YC-EX Cisco Tetration Analytics Platform REST API 3rd-Party Metadata Sources Configuration Data Push Events 28
The Analytics Cluster Components Front End Compute (Data Cleaning and Hadoop Based Platform Self managed One touch deployment Analytics) Caching (Search) Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Long Term Storage Messaging Bus API Access (Data Lake) 29
The Analytics Cluster Appliance The Analytics Cluster operates as an appliance Avoids the need for in house Big Data, Analytics expertise Supported by Cisco TAC Self Monitoring The cluster leverages a sensor architecture to track it s state and provides event based notifications for Software upgrades and full install are all automated 30
FCS Analytics Cluster Configurations 4 x 3-Phase PDU 22.5 KW Peak Power 4 x 1-Phase PDU 11.5 KW Peak Power 31
Options for Future Cluster Models 32
Analytics Engine The Platform Front End Compute (Data Cleaning and Hadoop Based Platform Self managed One touch deployment Analytics) Caching (Search) Tiered System Heavy Compute for Machine Learning Caching for light speed queries Extensibility (future) Long Term Storage Messaging Bus API Access (Data Lake) 33
Front End GUI, RESTful API, Messaging BUS Servers hosting front end processes GUI and Operational Interfaces RESTful API (post FCS) Messaging BUS (post FCS) 34
ACI Architecture ACI Intent (May) Traffic Analysis Lots of Data Configuration Analysis Very Large State- Space Analytics (Did) ADM Security Forensics Guarantees Compliance Consistency Assurance (Can) 35
Summary Pervasive flow Ready-to-use Self-monitoring and Open platform and Accelerated telemetry that solution to address eliminate the need northbound APIs adoption and supports critical data center for enable transparent comprehensive infrastructure for operational in-house big data integration Solution multiple data use cases expertise support with centers at scale Services 36