Data Driven Networks

Transcription

1 Data Driven Networks

2 Is it possible for to Learn the control planes of networks and applications? Operators specify what they want, and the system learns how to deliver

3 CAN WE LEARN THE CONTROL PLANE OF THE NETWORK? DRONE ATC 4K VIDEO VIRTUAL REALITY Applications with Access to Network Control Northbound Application Interfaces Operator Policies Operator Operator & Policies, Intents Intent Policies, & SLA Intent & SLA Real-time Network Telemetry Control & Orchestration Plane automatically synthesized by learning from telemetry data Southbound Infrastructure Interfaces Edge Cloud RAN Infrastructure Control Points You specify what you want, and the network figures out how to deliver it! Result: Dynamic per user/flow control implemented to meet the policy and deliver on the SLA

4 DATA DRIVEN CONTROL LOOP Operator Policies Intent & SLAs Service KPI Apply Controls Realtime Data Prediction Network Telemetry Will Current Controls Meet KPI? Y Recommend Current Control Settings N What-If Adjust Control Knobs Will Adjusted Controls Meet KPI? Y Meets Operator Policies? Y Recommend Adjusted Control Settings N N

5 TWO LEARNING APPROACHES: Classical Machine Learning Deep Reinforcement Learning Reward Function Neural Network Reinforcement Learning (Prediction & What-if) Requires extensive domain knowledge Largely blind, truly approximates intent based system design

6 Classical Machine Learning Deep Reinforcement Learning Build Rules-based Algorithm / Model Learning Agent Deploy State Reward Actions Evaluate Results Application and Network state variables Reward values to achieve desired outcomes Controllable actions Tune / Update Algorithm / Model Environment Millions of Training Cycles per hour

7 DEEP RL IS A GREAT FIT FOR NETWORK CONTROL LEARN DIRECTLY FROM EXPERIENCE NO MODELS, BRITTLE ASSUMPTIONS POLICIES ADAPT TO ACTUAL ENVIRONMENT & WORKLOAD OPTIMIZE A VARIETY OF OBJECTIVES END-TO-END PROVIDE RAW OBSERVATIONS OF NETWORK CONDITIONS EXPRESS GOALS THROUGH REWARD SYSTEM DOES THE REST NETWORK CONTROL DECISIONS ARE OFTEN HIGHLY REPETITIVE LOTS OF TRAINING DATA SAMPLE COMPLEXITY LESS OF A CONCERN

8 LEARNING & CONTROL PIPELINE Inference Prediction Control Machine Learning Deep Learning Feature Extraction Neural Networks

9 INFERENCE Inference Understand how known network input variables affect network KPI (e.g throughput) Leverage traditional supervised machine learning and statistical analysis (random forest, etc.) to develop function f(x t+1 ) to be used as input for deep learning agent X = Average Throughput per user, per cell Current and past collisions per cell (a) Current and past number of users per cell (b) Current and past signal strength per user, per cell (c) X t = f(a, b, c)

10 PREDICTION Prediction Extracting features from Training Data (offline) Network State Variables a, b, c Input Features from realtime data feed Prediction Neural Network (real-time) X t+1 Throughput KPI Prediction

11 CONTROL Control Network State Variables Prediction Neural Network (real-time) Deep Learning (unsupervised, reinforced learning) Operator Policies Client Conditions App specific inputs Control Decision

12 USE CASE: VIDEO STREAMING Network Assisted Mobile Video Streaming CDN STREAM Optimization Objective: Maximize video quality and Minimize Stall Time in challenging network scenarios Next Chunk ABR Result: 50-75% Higher Video Quality 75%-100% Lower Stall Time Policy: Maintain minimum average throughput of 500Kbps per user for non-video traffic Data: Real-time Network State Data Feed Real-time Mobile Application State Data Control: Next Chunk Adaptive Bit Rate (ABR)

13 API SYSTEM ARCHITECTURE CDN Video Stream ABR Request LTE Video Encoded for ABR Streaming API Real-time Network Conditions ABR guidance request Network Assisted ABR Video ABR recommendation for next chunk 13

14 INFER Video QoE depends on user throughput. What does user throughput depend on? 58% error USER SESSION # 40% error USER SESSION # Throughput bits per time 12% error USER SESSION # Spectral efficiency bits per resource element Resource allocation rate resource elements per time CQI Rank MCS X Σ Active users Σ Pkt rate, size Cell bandwidth Cell contention User&total demand? User performance can be modeled fairly accurately using features extracted from real time network data feeds

15 remainder trend seasonal data PREDICT Predicting the network state variables and then throughput Actual Foreca st 4% error 00:00 01:00 02:00 03:00 04:00 05: time Number of users on a cell Test for seasonality and trend, learning ARIMA using a neural network

16 Predict + Control inputs Hidden layers past b_app ts conv1d relu past b_seg ts RSRP/RSRQ ts CELT ts relu relu linear thpt prediction video buffer states B R C A relu softmax ABR Policy LSTM thpt forecaster A3C ABR

17 BUFFER (seconds) BUFFER (seconds) BITRATE (kbps) BITRATE (kbps) WHAT BEHAVIOR DOES THE RL AGENT FOR CONTROLLING VIDEO ABR LEARN? VIDEO ON DEMAND LIVE VIDEO ON DEMAND VIDEO (buffer limit up to 4 minutes) LIVE VIDEO (buffer limit less than 30s) Build a safe buffer, then safely play the highest quality without stalling :52 13:53 13:54 13:55 13:56 LOCAL TIME :14 11:15 11:16 11:17 11:18 LOCAL TIME Track throughput predictions to play as high quality as possible without stalling :52 13:53 13:54 13:55 13:56 LOCAL TIME 11:14 11:15 11:16 11:17 11:18 LOCAL TIME Network predictions become more valuable as buffer limit grows smaller

18 OPEN DRBS CUMULATIVE STALL (seconds) (Mbps) WHY FINE-GRAINED NETWORK PREDICTION MATTERS FOR QOE No context awareness at the start. Huge :39 17:40 17:41 17:42 17:43 17:44 17: startup time! :39 17:40 17:41 17:42 17:43 17:44 17:45 colour colour BITRATE THP CUM STALL Cell congestion can spike transiently (~10s of seconds). Huge stalls! TYPE ERAB HANDOVER NORMAL :39 17:40 17:41 17:42 17:43 17:44 17:45

19 LINK QUALITY (db) CUMULATIVE STALL (seconds) (Mbps) WHY FINE-GRAINED NETWORK PREDICTION MATTERS FOR QOE colour 1.0 BITRATE :11 13:12 13:13 13:14 13:15 13:16 THP colour CUM STALL 3 Downlink interference can spike transiently (~10s of seconds). Huge stalls! 0 13:11 13:12 13:13 13:14 13:15 13: colour -16 NEIGHBOR SERVING :11 13:12 13:13 13:14 13:15 13:16

20 USE CASE: CONTENT PREPOSITIONING Content Pre-Positioning CDN CONTENT Optimization Objective: Maximize content downloads, while maintaining operator defined minimum throughput requirements When and how much data to download Result: More than double data downloaded on existing infrastructure with deterministic impact on other subscribers Policy: Maintain minimum average throughput of 500Kbps per user Data: Real-time Network State Data Feed Control: When and how much data to download

21 USE CASE: MOBILE NETWORK LOAD BALANCING RAN Load Prediction RAN Controller A SIGNAL LOAD SIGNAL 1 RAN Handoff Control B LOAD RAN Load Balancing Optimization Objective: In cases where signal strength is sufficient on two or more cells, manage connectivity to balance load across available cells. Policy: Maintain minimum average throughput of 500Kbps per user Data: Real-time Network State Data Feed 2 Control: When and which user to handoff to neighboring cell?

22 API API API PLATFORM ARCHITECTURE DEEP LEARNING ENGINE Dynamic Network Conditions Uhana Managed Service (Cloud Infrastructure) Application Control Points App Specific Neural Networks App Control CTR/CTUM Feed Over-the-Air Real-time Per-User Fine-Grained Monitoring Deep Learning Engine App Control Fine Grained Mobile Network Visibility Operator App Specific Intents (Desired Outcomes) Operator Policies (Constraints) Varied based on: User, Subscription Level, Device Type, Content Type, etc. 22

23 THE INFER-PREDICT-CONTROL PRIMITIVES OFFLINE INFER e.g., link throughput What network metrics affect app QoE metrics? e.g., avg bitrate, stalls, switches REAL TIME PREDICT CONTROL e.g., link quality, cell congestion etc. How will network metrics look in the future? e.g., link throughput in the next 5s What should be the app control for the best app QoE? e.g., ABR for the next video chunk e.g., buffer, app throughput

24 PLATFORM FEEDBACK: PROCESSING LATENCY DICTATES PREDICTION REQUIREMENTS INFER PREDICT CONTROL INGEST PROCESS FORECAST Real-time consumption of network data Real-time computation of network metrics Real-time prediction of network metrics Lower the latency of processing, lower the requirement on prediction horizon (easier the prediction)

25 CONCLUSION & LESSONS LEARNED Network control is a Big Control problem Learning techniques can lead to automated control and intent based system design Infer + Predict + Control is a general framework Applies to systems beyond mobile networks Streaming analytics pipeline latencies are hard to predict right now Direct implication on hardness of prediction Analytics pipeline system performance requires careful manual tuning Hard to iterate and keep performance predictable Online learning is still an unsolved problem, both in learning techniques as well as system design