Towards (More) Data Science in Communication Networks NetSys KuVS Preisverleihung - Presentation

Transcription

1 Towards (More) Data Science in Communication Networks NetSys KuVS Preisverleihung - Presentation Andreas Schmidt, MSc Telecommunications Lab Saarland Informatics Campus - Saarbrücken March 15, 2017

2 A Formula for Improving Network Performance Higher Availability Better Utilization Improved Network Performance Decreased Loss 2 / 16

3 A Formula for Improving Network Performance offers Measurement Higher Availability Better Utilization Software Defined Networking Improved Network Performance offers Orchestration Decreased Loss 2 / 16

4 A Formula for Improving Network Performance Graph Analysis Statistics Higher Availability offers Measurement exploits query mechanisms Better Utilization Software Defined Networking + = Network Analysis Solution Improved Network Performance offers Orchestration supports manual / enables automated Decreased Loss Data Mining Visualization Machine Learning Forecasting 2 / 16

5 A Formula for Improving Network Performance Graph Analysis Statistics Higher Availability offers Measurement exploits query mechanisms Better Utilization Software Defined Networking + = Network Analysis Solution Improved Network Performance offers Orchestration supports manual / enables automated Decreased Loss Data Mining Visualization Machine Learning Forecasting That I had in mind when I worked on my thesis in / 16

6 Data Science for Networks - A Hot Topic 3 / 16

7 Motivation and Requirements In theory, networked systems still have a great potential for optimization. TCP inefficiencies 1. Practical performances far from information-theoritical bounds. 1 Wang, Bing, et al. Multimedia streaming via TCP: An analytic performance study. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 4.2 (2008): / 16

8 Motivation and Requirements In theory, networked systems still have a great potential for optimization. TCP inefficiencies 1. Practical performances far from information-theoritical bounds. Communication meta-data can be used to apply optimization. Consider flows (sequences of packets) for analysing traffic. Optimize delays, redundancy overhead, etc. 1 Wang, Bing, et al. Multimedia streaming via TCP: An analytic performance study. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 4.2 (2008): / 16

9 Motivation and Requirements In theory, networked systems still have a great potential for optimization. TCP inefficiencies 1. Practical performances far from information-theoritical bounds. Communication meta-data can be used to apply optimization. Consider flows (sequences of packets) for analysing traffic. Optimize delays, redundancy overhead, etc. Methodologies, frameworks and tools from the area of data science support this. 1 Wang, Bing, et al. Multimedia streaming via TCP: An analytic performance study. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 4.2 (2008): / 16

10 Motivation and Requirements In theory, networked systems still have a great potential for optimization. TCP inefficiencies 1. Practical performances far from information-theoritical bounds. Communication meta-data can be used to apply optimization. Consider flows (sequences of packets) for analysing traffic. Optimize delays, redundancy overhead, etc. Methodologies, frameworks and tools from the area of data science support this. Open source technology allows wide-spread adoption and interoperability. 1 Wang, Bing, et al. Multimedia streaming via TCP: An analytic performance study. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 4.2 (2008): / 16

11 A Master Thesis on Network Analytics 5 / 16

12 Solution Architecture Services Statistics Infrastructure Topology polls information Floodlight Sensor Transmissions Reliability contains sensors executes tasks contains views Controller (Floodlight) Observer (Python) Analyzer (Python) Visualization (NodeJS) exposes exposes writes to write / read NetAPI (Python) Store (PostgreSQL) read from Existing Component check status / trigger run Own Contribution Network Analytics Presentation Legend Available on GitHub: 6 / 16

13 Network Data Model Internet- Address Sample- Timestamp Report (Content, Sampling) n : m Flow (Match, Cookie) n : 1 Node (Device ID, Type) 2 : n Link (Direction, Type) 1 : n LinkSample (Rx, Tx, Loss, Delay, Centrality) 1 : n 1 : n 1 : n FlowSample (Counts, Durations) NodeSample (Centrality) Port (Name, Number) 1 : n PortSample (Rx, Tx, Errors) Types: Network Entities Entity Samples Auxiliary 7 / 16

14 Visualization Available on GitHub: 8 / 16

15 OpenNetworking Project NN- Client NN- Node UdS- Node3 UdS- Node5 MU- Node MU- Client UdS- Node4 HIIT- Client HIIT- Node UdS- Node1 UdS- Send1 UdS- Recv1 SF- Client UdS- Node2 UdS- Node6 SF- Node SF- Relay Entities: Node Host Locations: Saarland University (Saarbrücken, DE) Nokia Networks (Munich, DE) Smart Factory (Kaiserslautern, DE) Helsinki Institute of Technology (Helsinki, FI) Manipal University (Manipal, IN) Project Website: 9 / 16

16 Data Science in Networks - A Brief Overview 10 / 16

17 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. 10 / 16

18 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. Graph Analysis to determine network stability, traffic flows, etc. 10 / 16

19 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. Graph Analysis to determine network stability, traffic flows, etc. Machine Learning helps training algorithms to make decisions on routing, resource reservations, traffic blackholing, etc. 10 / 16

20 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. Graph Analysis to determine network stability, traffic flows, etc. Machine Learning helps training algorithms to make decisions on routing, resource reservations, traffic blackholing, etc. Execute SDN enables to dynamically change the network s architecture and operation. 10 / 16

21 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. Graph Analysis to determine network stability, traffic flows, etc. Machine Learning helps training algorithms to make decisions on routing, resource reservations, traffic blackholing, etc. Execute SDN enables to dynamically change the network s architecture and operation. NFV is a tool to interact with communication in a direct manner. 10 / 16

22 Data Science in Networks - A Brief Overview Analyze Statistical methods for estimating network parameters and giving guarantees. Graph Analysis to determine network stability, traffic flows, etc. Machine Learning helps training algorithms to make decisions on routing, resource reservations, traffic blackholing, etc. Execute SDN enables to dynamically change the network s architecture and operation. NFV is a tool to interact with communication in a direct manner. Configuration Management solutions (SaltStack, Chef, Puppet,...) improve system administration, e.g. to deploy NFVs. 10 / 16

23 Transparent Transmission Segmentation (TTS) E2E Segment NetFlix Low Loss High Delay Wireless Router High Loss Low Delay Tablet 11 / 16

24 Transparent Transmission Segmentation (TTS) E2E Segment NetFlix Low Loss High Delay Wireless Router High Loss Low Delay Tablet Split Segment Relay Split Segment 11 / 16

25 Transparent Transmission Segmentation (TTS) E2E Segment NetFlix Low Loss High Delay Wireless Router High Loss Low Delay Tablet Split Segment Relay Split Segment End-to-End Paradigm and its Implications End-to-End Arguments in System Design [Saltzer 84] A Critical Review of End-to-End Arguments in System Design [Moors 02] RFC3531: Performance Enhancing Proxies [IETF 03] A multitude of papers on TCP proxies, Split-TCP, CCAs, etc. [ 02-09] 11 / 16

26 TTS - Early Results Improving Multimedia Streaming Performance from the Network s Core [ICCE-Berlin 16, Schmidt, Herfet] Systematic investigation which domains (loss, congestion, buffers,...) and network functions (error, flow, congestion control,...) are affected by TTS. First promising results for TCP, using simulated environments (mininet). Round-trip times (interactivity) in high-loss scenarios reduced. (Mean RTT: 4%, Jitter: 16%, Applied Loss: 3%) 12 / 16

27 TTS - Early Results Improving Multimedia Streaming Performance from the Network s Core [ICCE-Berlin 16, Schmidt, Herfet] Systematic investigation which domains (loss, congestion, buffers,...) and network functions (error, flow, congestion control,...) are affected by TTS. First promising results for TCP, using simulated environments (mininet). Round-trip times (interactivity) in high-loss scenarios reduced. (Mean RTT: 4%, Jitter: 16%, Applied Loss: 3%) Approaches for Resilience- and Latency-Aware Networking [NetCPS 16, Schmidt, Herfet] Effect analysis using non-parametric metrics. Depending on link parameters, TTS can reduce delivery times compared to E2E, being superior in 68 91% of the cases. 12 / 16

28 TTS - Recent Investigations In Progress / Completed Consolidation and in-depth analysis of TTS effects. Details on implementation and deployment inside SDNs in [NetSoft 17, Schmidt, Herfet] (accepted for publication, presentation in Jul 17). Relay deployment using Docker, leading to small latency penalties. Investigation of TTS effects with RTP in a recent thesis [Birtel, 01/17]. Application-independent error control with minimal overhead ( 2ms). 13 / 16

29 TTS - Recent Investigations In Progress / Completed Consolidation and in-depth analysis of TTS effects. Details on implementation and deployment inside SDNs in [NetSoft 17, Schmidt, Herfet] (accepted for publication, presentation in Jul 17). Relay deployment using Docker, leading to small latency penalties. Investigation of TTS effects with RTP in a recent thesis [Birtel, 01/17]. Application-independent error control with minimal overhead ( 2ms). Planned Investigations on TTS with the Predictably Reliable Real-time Transport (PRRT) protocol, developed at the TC Lab. Evaluating TTS with novel congestion control algorithms, e.g. Google s BBR 2. 2 Jacobsen V., et al. BBR: Congestion-Based Congestion Control. ACM Queue 14.5 (2016): / 16

30 TTS - Why Data Science? Open Question: Where and how many relays to place for optimal performance? 14 / 16

31 TTS - Why Data Science? Open Question: Where and how many relays to place for optimal performance? Where can Data Science be used in this context? Answer clearly depends on precise and continuous measurements of the network. Analysis solutions have to find this answer quickly, ideally in real-time, so that transmissions can be immediately segmented for optimal performance. 14 / 16

32 Latency- And Resilience-aware Networking (LARN) Organization DFG Schwerpunktprogramm 1914 Cyber-Physical Networking (CPN). Kickoff in October 2016, two funding periods of 3 years each. Cooperation with Prof. Wolfgang Schröder-Preikschat, Chair for Distributed and Operating Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg 15 / 16

33 Latency- And Resilience-aware Networking (LARN) Organization DFG Schwerpunktprogramm 1914 Cyber-Physical Networking (CPN). Kickoff in October 2016, two funding periods of 3 years each. Cooperation with Prof. Wolfgang Schröder-Preikschat, Chair for Distributed and Operating Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg Research Objectives Develop a latency- and resilience-aware atomic unit for cyber-physical networks. Optimise the cross-layer communication while maintaining separability. Validate the approach in theory and experiments, e.g. on the ON testbed. 15 / 16

34 Latency- And Resilience-aware Networking (LARN) Organization DFG Schwerpunktprogramm 1914 Cyber-Physical Networking (CPN). Kickoff in October 2016, two funding periods of 3 years each. Cooperation with Prof. Wolfgang Schröder-Preikschat, Chair for Distributed and Operating Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg Research Objectives Develop a latency- and resilience-aware atomic unit for cyber-physical networks. Optimise the cross-layer communication while maintaining separability. Validate the approach in theory and experiments, e.g. on the ON testbed. Exact knowledge about current network parameters through measurements is crucial for guarantees provided for CPS communication. Applying intelligence gathered through analysis is can provide better performance regarding latency and resilience. 15 / 16

35 Conclusion Towards (More) Data Science in Communication Networks Data Science, often colloquially refered to as Big Data, is an important emerging discipline to be applied in many areas (manufacturing, finance,...). Networking is not yet exploiting the potential of data science. SDN, NFV, together with Configuration Management bring the required monitoring and orchestration facilities to enable this. One task that can be tackled is the optimal placement of segmentation points for transparent transmission segmentation. Only by constantly measuring and changing the networks, we can provide the servies needed by e.g CPS and other future applications. 16 / 16

36 Conclusion Towards (More) Data Science in Communication Networks Data Science, often colloquially refered to as Big Data, is an important emerging discipline to be applied in many areas (manufacturing, finance,...). Networking is not yet exploiting the potential of data science. SDN, NFV, together with Configuration Management bring the required monitoring and orchestration facilities to enable this. One task that can be tackled is the optimal placement of segmentation points for transparent transmission segmentation. Only by constantly measuring and changing the networks, we can provide the servies needed by e.g CPS and other future applications. Thank you for your attention. Questions? 16 / 16