Diego R. López Telefónica I+D

Transcription

1 Diego R. López Telefónica I+D

2 CogNet is building a network management solution based on machine-learning Relying on SDN and the NFV architecture framework Focused on 5G use cases Committed to produce and demonstrate a general framework for smart, data-driven management Able to support different application scenarios Monitoring and SLA enforcement Traffic identification and security Resource management Reliability enhancement E2E service management <Your favourite scenario here> And for sure Relying on open source Willing to contribute back to the community Seeking for opportunities of further engagement 2

3 CogNet Policy Engine CogNet Smart Engine Batch Engine CogNet Collector Collector NFV/SDN-based Environment Real Time Engine Policies Policy Engine Scores Network Management Existing Solutions CogNet Solution 3

4 A data-driven management engine: CSE Tuned by the application of ML models Two streams, inbound Policies, outbound adaptors preprocessing Policy optimization A double closed-control loop With the CSE at the intersection 4

5 Pre-processing Automated Selection User inputs Feature Selection and Extraction User inputs Hardware Analysis set Analysis Comparative Analysis Selection Distortion Evaluation Batch Processing Engine Training Scoring (Semi) Supervised & Unsupervised ML Normalisation, Transformation Distributed File System Storage (Near) Real-time Processing Engine User inputs E2E inputs Frozen Collector Generator Measurements (Network and Infrastructure) Adaptors Cleaning, Filtering, Feature Extraction Pull Training Scoring Online Learning and Scoring or Score using model from Batch Layer MANO Block Event Instance Name (Events Values) Policy Engine Thresholds & Events Values Tenant SDN Controller VTN EM(s) Resources and Services NFVO Managment LCSE Thresholds & Events Values Policy Recommender VNF(s) Servcie, VNF and interface descriptions VNF Manager(s) Proxy Recommendations Policy Distribution Policy Repository Policies NFVi Virtualized Infrastructure State & Config Info Policies Compute Storage Network Virtualized Infrastructure Manager(s) Virtualization Layer Physical Hardware Resources Infrastructure SDN Controller Action Parame ters Policy Optimizer Functional Components Cognitive Elements Sources Communication ML Scores Policies Policy Engine Sub-Components Raw Collection Processed Preprocessing Optional Components External Input Optional Mandatory 5

6 Based on open source Monasca, Kafka and InfluxDB for data collection PANDAS, Spark MLlib, Hive and Hadoop for the machine learning process Drools as policy engine ODL as tenant controller OSM for orchestration OPNFV Brahmaputra as NFVI And open data models JSON for (most) interfaces SUPA for policy definitions DATA COLLECTOR (MONASCA) Inbound API (REST+ JSON) System X VNFs OSCON MONASCA DATA STORAGE (INFLUXDB) INTERNAL DATA BUS (KAFKA) Outbound API (REST+JSON) (KAFKA) DATA BUS (KAFKA) demox metrics DATA BUS (KAFKA) KAFKA CogNet Common Infrastructure DOCKER [L]CSE ([PYTHON ] SPARK HIVE) ML Algorithms for System X ML Algorithms for System X ML Algorithms for Systen Y ML Algorithms for System Y demox DATA BUS (KAFKA) POLICY ENGINE (JAVA) Policies set for System X Policies set for System X (SUPA) Policies for System X Policies set for System Y Policies set for System Y (SUPA) Policies for System Y POLICY newpolicy 6

7 There is a serious lack of training datasets as an asset Privacy concerns None or limited tagging Working on the production of synthetic datasets Apps & clients Browser Video Cloud Illegal Network VNFs ML Algorithms CSP & Internet Web Server Video stream Cloud File Provider Illegal Synthetic traffic laboratory Generates traffic samples in a controlled way Configurable mixes of synthetic and real traffic Different scenarios, from high loads to security threats A way worth exploring in the future OSS App 2 App 1?? Manageable traffic Unknown MouseWorld.. OSS App 2 App 1 Browsing Video Cloud Illegal Attacks CogNet Mouseworld HoneyPot Manageable traffic Open synthetic datasets Open synthesizing tools 7

8 -driven management is the natural evolutionary step for network management Is not only about ML or AI The same NFV is not only about cloud Several challenges remain The dynamic, complex nature of networks The availability of useful datasets The challenge of traceability in a multi-tenant, multi-user environment We are moving along this path A pile of fun ahead 8