ANALYTICS IN VIRTUAL & PROGRAMMABLE NETWORKS. Big Data Analytics for Telecom 2.0

Transcription

1 ANALYTICS IN VIRTUAL & PROGRAMMABLE NETWORKS Big Data Analytics for Telecom 2.0

2 Big Data for the Big Leagues 4 of the top 5 NA mobile operators 3 of the top 5 NA IP / MPLS backbones 80% of NA cable MSOs PB data/day 250B records/day 2.5M transactions/second 50% US mobile traffic PB data/day 500B records/day 6M transactions/second 70% US mobile traffic 2014 Guavus. All rights 2 reserved.

3 About Guavus Market Imperative Exploding usage, revenue pressures, and zero-sum competitive landscape is necessitating rapid, Operational Intelligence - centric decision making New data analytics architectures are emerging to address new scale, multiple data silos and data streams Customers Growing Customer Traction 4 of the top 5 North American Mobile Operators & expanding Internationally 3 of the top 5 IP / MPLS backbone carriers & CDN Networks 80% of Tier 1 NA Cable Operators Leading Partners 2014 Guavus. All rights 3 reserved.

4 Guavus Enables Holistic Views of Operations ERP/CRM End-to-End View Streaming Operational Intelligence VMs + Devices Performance Metrics Anomaly detection CAPEX deferral Cloud Physical Infrastructure Capacity Profiling & Prediction Operational automation OPEX reduction Better user experience Security risk mitigation Security Systems Applications & Users Future scalability 2014 Guavus. All rights 4 reserved.

5 Reflex 2.0 Real-Time Platform Evolution Batch stream processing pipeline < 5 to 60 minutes Collector Custom Adapter 1 Compute Cluster Analytics Cluster Cache Cluster Machine Learning Anomaly Query Engine DB Query Engine Rules Engine Cache UI Query Handler Adapter 2 Map Reduce Base Cubes Business Cubes Real-Time distributed stream processing < 1 to 60 seconds Stream Analytics High-Level Rules Engine State DB Message Broker Triggers Alerts Recommend Policy Streaming Analytics & Machine Learning at carrier scale and reliability Per sub and per network element granularity Dynamic rules & behavioral notification / alerts Ad hoc data exploration & search capabilities Virtualization & OpenStack integration The Guavus Reflex platform continues to adopt the latest open source technologies to maintain industry leadership 2014 Guavus. All rights 5 reserved.

6 Machine Learning & Data Science support Operational Intelligence Analytics Framework Data Streams Feature Engine Anomaly Engine RCA Engine Contextual Data Stores Data fusion Metrics creation Variables selection Causality inference Multivariate analysis Outlier Detection Statistical learning Clustering Pattern identification Itemset mining Targeted Actions 2014 Guavus. All rights reserved. 6

7 TELECOM ANALYTICS Big Data Analytics for Telecom 2.0

8 Objective Reduce service delivery cost by enabling delivery of % services running on 99% reliable servers. Advanced Anomaly Detection, Machine Learning and Streaming Analytics Techniques used to identify service impacting events Agentless data collection of logs, flow, and state data enables: Performance change detection of applications enabling the service, Identification of likely sources of service degradation Integrate with orchestration systems to automate service integrity management and outage prevention Analytics should be considered an NFV/SDN Primitive 2014 Guavus. All rights reserved. 8

9 What Role Does Analytics Play? Simple Usage Put a name to a bit every bit Inter-Machine Flows Layer 1 to 7 Inter-Application Flows Monitor the behavior of flows between each application in a service Classifies all flows traversing the network Bandwidth Busy Hours QoE Requirements 2014 Guavus. All rights reserved. 9

10 Analytics Based SDN/NFV Control The key to successful SDN & NFV is to deploy on lowcost, less reliable hardware Optical SDN Controller NFV Controller Carrier Ethernet SDN Controller Restful API MANO Data Center SDN Controller VNFs & hardware will push logs, flow data and alerts to the analytics system Real-Time Usage Models Demand Modeling Path Optimization Guavus Analytics Platform Application Affinity Map Off-Load Calculations In-Service QA Guavus will apply anomaly detection and continuous machine learning algorithms to predict component failures Mobile Network Data Source: DPI RAN CDR FDR CDN Data Source: Syslog sflow Usage Logs Video Services Data Source: Syslog Usage Logs IP/MPLS Core Data Source: BGP Netflow CSP Data Center Data Source: DPI Log Data BGP Netflow sflow CSP Test Systems Data Source: Logs Results Enterprise Data Center Data Source: DPI Log Data BGP Netflow sflow Enterprise Customer Data Source: BGP Netflow sflow Guavus eliminates the need for high-speed failover and state-awareness as the system can prepare for, rather than react to, failure 2014 Guavus. All rights reserved. 10

11 Analytics Based Network Design and Operations The Challenge The Solution Provisioning of the network is segmented and done in absence of true application behaviors Due to complexity, overbuilding networks is all to common. SDN controllers do not have global context from Layer 1 to Layer 7 WDM WDM Fixed CPE, CE, Mobile Device, Managed CPE RAN, DSL, PON, TDM, Services Core IP/MPLS WDM WDM WDM WDM WDM The collection and fusion of data from optical transport to customer applications. Analytics formulates application affinities, demand, and performance based topologies. Develop unified configurations across multiple SDN/NFV layers. Guavus provides accurate real-time state of the access, transport, virtual domain, and service quality simultaneously Guavus. All rights reserved. 11

12 Time Operational Process Access Management and Control Analytics IP and Transport Management and Control APIs, Exposable Service and Assets Service Management and Control Integrated Network and Control VNF Products and Service Management Cross-Domain Orchestration, OSS/BSS Virtualization and Abstraction VNF CRM and CEM Policy Control VNF Charging and Billing Cloud Management VNF OpenStack 1. Data Collection: Application Logs Server Logs Flow Records Syslog VM Logs Test Sets Net Flow DPI Etc. Virtual Networking SDN Controller OpenDaylight 3. Visualization Clustering of Anomalies, Pattern of Life Visuals Syslog Server Logs Netflow/DPI Storage Logs & Alerts VM Logs & Alerts Test Systems 2. Anomaly Detection performed on all data sets Root Cause Analyses Services 4. Automation Scale-Up/Down Trigger VNF Build Flow Capture Policy Control Operator Alerting 12

13 Signaling and Control TBD Data Collection RAN KPIs Location Signaling and Control Data Collection DPI/FDR Phase 1 - Services Care Records CDR Repository Customer Profile Data Collection, Analytics, and Signaling System SON MTSO/RNC/MNE GGSN/PGW/HA/DPI Video Optimization VoIP Content Caching Rate Shaping TCP Optimization Load Balancing DNS DPI Traffic Steering Toll Free Data Internet Analytics Guavus Data Collection Operational Reports & Dashboards Tablet UI Web Based UI Policy Control Cost Modeling SDN Contextream Gi Cloud VAS Service Enablement TCP Optimization DPI Rate Shaping & Clamping Service Chaining Toll Free Data CDN(s) Video Optimization Skyfire ABR Rate Control Real-Time Transcoding Video Policy Enforcement Video QoE Optimization Video Analytics Interstitial Advertising 2014 Guavus. All rights reserved. 13

14 Mobile Operator Gi/SGi-LAN steering Challenges All functions sized for peak Slow new service deployment Separate load balancers needed per function Service Rails Advantages No need to send all traffic through all devices no overprovisioning Agility add new services without affecting other services With SDN 2014 Guavus. All rights reserved. 14

15 NFV PoC #15 - Subscriber Aware SGi/Gi-LAN Participants Proof of concept ETSI areas: 1. Subscriber aware service chaining method (VNFG) 2. Service chaining intrinsically provided by NFVI 3. Selection of a specific function element instance to service user within (or across data centers) 4. Subscriber aware virtualization enabling programmability of network from a subscribers self service portal and ease rapid introduction of new functions Test Concepts: 1. Add/remove a VNF instance, programming the virtual network via. OpenFlow 2. Use self service portal to provision policy-driven subscriber aware steering to functions 3. Perform automated steering in the Virtual Network Infrastructure to load balance across instances of VNFs 4. Instantiate open source inline services within the NFV framework

16 ENTERPRISE ANALYTICS Analyses to Drive Operational Excellence

17 Challenges Massive volumes of data make problems hard to see Too many elements to manage individually Massive volume, always evolving datasets Red light fatigue ; knowing what s really broken while utilizing full capacity Constant churn makes pinpointing issues hard Automated tools are needed, but most fail at Hyperscale When objects are regularly created & destroyed, you stop tracking them Placement and scheduling issues Initial placement uses current load, not predicted load Recomputing best-placement is hard at scale Scheduling contiguous physical resources is also hard at scale Orchestration issues Problems automating the entire Infrastructure Lack of automated live migration and embargo & evacuate abilities 2014 Guavus. All rights reserved. 17

18 Use Case: Cloud Analytics & Control Gain end-to-end visibility for your distributed, Hyperscale Cloud Business Problem Too many objects to manage individually Alerting via threshold doesn t scale well in dynamic virtual environments Resource balancing methods are not scaling Lack of visibility & control means provisioning is at less than 1:1 efficiency Solution Collect stats & logs from servers, storage, network, Hypervisors Fuse with data from CMDB & Orchestration layer, also lookup tables detailing release calendar & other notable date patterns Build state map of object relationships Outcome Creates stack visibility, probable cause identification and resource projections Utilize end-to-end visibility and control to move from current subscription rates to an oversubscription model, deferring CAPEX and OPEX costs while increasing overall capacity 18

19 Use Case: Software Build Management Track Build environment health and SCM responsiveness to build a comprehensive picture of builds and help keep NetApp within SLA Business Problem Build SLA are not always met When builds slip, releases slip Multi-system distributed environment makes true root cause sometimes hard to find Prototype Gather statistics and logs from Build servers & P4 Proxies Gather network flow data between machines and for NAS/iSCSI traffic Use Perfstat/OnCommand/API to get Array statistics Fuse with lookup tables detailing release calendar & other notable date patterns Outcome End-to-end visibility into Build environment health with KPI tracking Identification of hard-to-pinpoint trouble spots Predictive analytics show marginal resources before they fail, allows embargo & evacuate to remediate affected resources 19

20 Use Case: Resource Scheduling & Placement Hyperscale churn plus modern business cycles mean tight scheduling control is more important than ever in today s Data Center Business Problem Current placement mechanisms offer onetime advice and no rebalancing mechanism Physical resource checkout takes into account current capacity, not future need Planned future efficiency gains through resource oversubscription will require smart placement strategies Prototype Gather stats & logs from servers, storage, network, Hypervisors Fuse with data from CMDB & Orchestration layer, also lookup tables detailing release calendar & other notable date patterns Build model of checkout patterns and needs over time Alert on resources checked-out but probably idle Outcome Allow smart rebalancing of virtual resources through Orchestration Allow smarter placement & checkout Enable shorter delays checking out the largest environments 20

21 SECURITY ANALYTICS Detecting Threats Before Service Degradation

22 The New Security Threat Advanced attackers now exploit these critical gaps in visibility and control inside the data center, with the average living inside enterprise networks for 243 days before detection Guavus. All rights reserved. 22

23 Integrated User Profile Human Resources Performance and Policy Payroll Network Activity Flow Analysis Web Activity Corporate Apps LDAP Logs Employee Comms /chat Voice Physical Security Access logs LBS Behavioral Analysis Life patterns All devices 2014 Guavus. All rights reserved. 23

24 The Cyber Battlefield Enterprise Network Internet Office Environment Data Center Network Gateway Cyber Threat Detection Data Center elements Servers APT Detection Network Systems Insider Threat Detection IDS/IPS DDOS Mitigation Detect threats before they hit the IDS or FW Office Environment Insider Threat Detection Physical Security Logs Firewall The combination of Insider Threat Detection and APT Detection create unprecedented views into network risk. Rather than gunpowder, the most used and, increasingly powerful, weapon is cyber. Financial, Telecom, and Infrastructure are prime targets Guavus. All rights reserved. 24

25 Anomaly Based Packet Capture An SDN approach to network analytics allows the WAN to use an OpenFlow overlay or passive optical taps for traffic replication to the analytics network, the analytics network to use OpenFlow for traffic filtering and replication to multiple analytics tools, and a network analytics application that resides on the SDN controller Guavus. All rights reserved. 25

26 Conclusion Guavus utilizes advanced data science to drive real business value from your network data. 26