Performance investigation and comparison between virtual networks and physical networks based on Sea-Cloud Innovation Environment Website: http://scie.ac.cn E-mail: scie@cstnet.cn CANS 2015, Chengdu, Sep 21, 2015
Outline Background Architecture Software & Hardware Deployment Demonstration Performance investigation 9/22/15
Background Sea-Cloud Innovation Environment, a national wide testbed supported by the Strategic Priority Research Program - New Information and Communication Technology (SPRP-NICT) of the Chinese Academy of Sciences, is aiming to build an open, general-purpose, federated and largescale shared experimental facility to foster the emergence of new ICT.
Background Objective Providing shared and sliceable experimental facilities for academia and industry to bridge the gap between visionary research and large-scale experimentation. Establishing and practicing the methodology of experimentally -driven innovation for the clean-slate architecture of ICT. Evaluating and validating new protocols, devices and research achievements of SPRP-NICT.
Outline Background Architecture Software & Hardware Deployment Demonstration Experimentation 9/22/15
Architecture Experiment Topology Requests SCIE portal Scie.ac.cn Resource control framework Experiment measurement system SDN/VLAN-based network slicing
Outline Background Architecture Software & Hardware Deployment Demonstration Performance investigation 9/22/15
Software--Overview Topology Editor Experiment Playground Resource Management Authorization & Accounting SCIE Portal Experiment Service System Resource Management & Control System Experiment Measurement System Control Center Resource Control module Site manager Measurement Module Site manager. Resource Site
Software SCIE Resource Control Architecture Distributed resource control framework with one control center and many site managers Defining resource control interfaces, measurement interfaces to integrate different resource Light-weight VM management tool
Software SCIE measurement System External VM measurement without any plug-in in VMs AMQP based control message & measurement data transfer sflow based network traffic measurement MongoDB as Storage Engine
Software Experiment Service System Experiment life cycle Management Java & Python based experiment control library Topology and experiment process visualization
Hardware Smart-Flow Switch OpenFlow 1.2 GRE tunnel QoS supported 24*GE 1*10GE Four slots Line Card & UTM Card
Hardware SCIE Rack Integrated network, computing and storage Built-in site management module Virtualization Dynamic scheduling
Outline Background Architecture Software & Hardware Deployment Demonstration Performance investigation 9/22/15
Deployment Four contries&seven cities &22 sites Data plane via GRE tunnel; Control plane via L3 network 2234 cores, 1510TB storage, 512TB experimental data
Outline Background Architecture Software & Hardware Deployment Demonstration Performance investigation 9/22/15
Outline Background Architecture Software & Hardware Deployment Demonstration Performance investigation 9/22/15
Performance investigation and comparison between virtual networks and physical networks based on an advanced testbed network
Three cities & 4 sites VM based on KVM Data transmission via GRE tunnel built based on OVS Deployment VM1 VM1 Xinjiang, China Beijing, China VM2 GRE Tunnel VM2...... Beijing, China VM8 VM8 Michigan, US
Scenario single-thread vs. multi-thread Deployment For each scenario the intra-domain case, from Beijing to Xinjiang in China the inter-domain case, from Beijing in China to Michigan in US Extensive performance evaluation tests UDP and TCP traffic in idle and non-idle period Key performance metrics For UDP traffic round trip time (RTT), throughput, packet loss, and jitter For TCP traffic RTT and throughput
Experimental results and analysis UDP traffic throughput Throughput (Mbits/second) 60 VM(single- thread,idle) 50 PHY(single- thread,idle) 40 VM(multi- thread,idle) 30 20 10 0 (a) UDP Bandwidth BJ-XJ (Idle) Throughput (Mbit/s) 60 50 40 30 20 10 0 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle) (b) UDP Bandwidth BJ-XJ (Non-Idle) Throughput (Mbit/second) 10 9 8 7 6 5 4 3 2 VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) (c) UDP Bandwidth BJ-MI (Idle) Throughput (Mbit/second) 10 9 8 7 6 5 4 3 2 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle) (d) UDP Bandwidth BJ-MI (Non-Idle) The virtual network is very similar to single-thread physical network ü very similar to single-thread physical network scheme in intra-domain and in inter-domain ü the deviation Ø less than 0.35% in intra-domain Ø about 0.21% in inter-domain The deviation is stable (multi-thread virtual network vs. single-thread physical network)
Experimental results and analysis UDP traffic packet loss rate Packet Lost Rate (%) 20 VM(single- thread,idle) PHY(single- thread,idle) 15 VM(multi- thread,idle) 10 5 0 (a) UDP Packet Loss BJ-XJ (Idle) Packet Lost Rate (%) 20 VM(singel- thread,non- Idle) 15 PHY(singel- thread,non- Idle) VM(multi- thread,non- Idle) 10 5 0 (b) UDP Packet Loss BJ-XJ (Non-Idle) Packets Lost Rate (%) 20 15 10 5 VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) Packets Lost Rate (%) 20 15 10 5 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle) 0 (c) UDP Packet Loss BJ-MI (Idle) (d) UDP Packet Loss BJ-MI (Non-Idle) Single-thread virtual network and single-thread physical network are better than multi-thread virtual network scheme in all cases The deviation is less than 1% (multi-thread virtual network vs. singlethread physical network) ü 0.17% in intra-domain ü about 0.23% in inter-domain 0
UDP traffic jitter Experimental results and analysis Jitter (ms) Jitter (ms) 0.8 0.6 0.4 0.2 0 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 (a) UDP Jitter BJ-XJ (Idle) (c) UDP Jitter BJ-MI (Idle) (b) UDP Jitter BJ-XJ (Non-Idle) (d) UDP Jitter BJ-MI (Non-Idle) The jitter of inter-domain environment was higher than that of intradomain environment with about 13.5% Jitter in Multi-thread virtual network is higher than single-thread physical network ü 2.6% higher in intra-domain environment ü 10.2 % higher in inter-domain environment The distance is the main factor affecting the jitter VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) 0 VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) Jitter (ms) Jitter (ms) 0.8 0.6 0.4 0.2 0 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 VM(single- thread,non- Idle) PHY(single- thread, non- Idle) VM(multi- thread, non- Idle) 0 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle)
Experimental results and analysis UDP traffic RTT RTT (ms) 70 65 60 55 50 (a) a UDP RTT BJ-XJ (Idle) VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) RTT (ms) 70 65 60 55 50 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle) (b) UDP RTT BJ-XJ (Non-Idle) RTT (ms) 280 270 260 250 240 230 220 210 200 (c) UDP RTT BJ-MI (Idle) (d) UDP RTT BJ-MI (Non-Idle) Single-thread physical network is more stable and smaller than single-thread and multi-thread virtual network RTT is most stable and lowest in all cases in single-thread physical network The RTT of single-thread physical network is about 3.3% smaller than singlethread and multi-thread virtual network schemes The RTT is not so stable in non-idle scenario, the deviation between idle and non-idle cases is 0.1% The background traffic is the key influence factor of RTT and the performance of network experiment in virtual network environment VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) RTT (ms) 340 320 300 280 260 240 220 200 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle)
TCP traffic throughput Experimental results and analysis Throughput (Mbit/second) Throughput (Mbit/second) 300 250 200 150 100 50 0 100 80 60 40 20 0 VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) 1 11 21 31 41 51 (a) TCP Throughput BJ-XJ (Idle) (c) TCP Throughput BJ-MI (Idle) (b) TCP Throughput BJ-XJ (Non-Idle) (d) TCP Throughput BJ-MI (Non-Idle) Throughput in Idle case is better than non-idle case Throughput in intra-domain environment is higher than that in interdomain environment After slow start, the throughput of single-thread physical network is higher than that of single-thread and multi-thread virtual network scheme The throughput of single-thread virtual network is the most stable and is very similar to single-thread physical network in non-idle case Samples 1 11 21 31 41 51 Samples VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) Throughput (Mbit/second) Throughput (Mbit/second 300 250 200 150 100 50 0 100 80 60 40 20 0 1 11 21 31 41 51 Samples VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle) 1 11 21 31 41 51 Samples VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle)
Experimental results and analysis TCP traffic RTT RTT (ms) 70 68 VM(single- thread,idle) 66 PHY(single- thread,idle) 64 VM(multi- thread,idle) 62 60 58 56 54 52 50 (a) TCP RTT BJ-XJ (Idle) 70 68 VM(single- thread,non- Idle) 66 PHY(single- thread,non- Idle) 64 VM(multi- thread,non- Idle) 62 60 58 56 54 52 50 RTT (ms) (b) TCP RTT BJ-XJ (Non-Idle) RTT (ms) 300 290 280 270 260 250 240 230 220 (c) TCP RTT BJ-MI (Idle) (d) TCP RTT BJ-MI (Non-Idle) Stability of RTT ü single-thread physical network > single-thread virtual network > multi-thread virtual network In idle traffic case, the deviation: ü 0.23%(single-thread physical network vs. single-thread virtual network) ü 0.55%(single-thread physical network vs. multi-thread virtual network) In non-idle traffic case, the deviation: ü 3.7% (single-thread physical network vs. single-thread virtual network) ü 4.5% (single-thread physical network vs. multi-thread virtual network) VM(single- thread,idle) PHY(single- thread,idle) VM(multi- thread,idle) RTT (ms) 300 290 280 270 260 250 240 230 220 VM(single- thread,non- Idle) PHY(single- thread,non- Idle) VM(multi- thread,non- Idle)
Experimental results and analysis Conclusion 1) the RTT and jitter of virtual networks have little deviation from physical networks, 2) the throughput and packet loss rate of virtual networks are similar to physical networks, 3) the performance of single-thread virtual network is more similar to the existing physical network than the multi-thread virtual networks, 4) the multi-thread virtual networks have certain deviation from physical networks, but the deviation is stable and shows certain characteristics. Thus, it is possible to get the performance of real physical networks in virtual networks