Supercomputer Networking via IPv6. Xing Li, Congxiao Bao, Wusheng Zhang

Transcription

1 Supercomputer Networking via IPv6 Xing Li, Congxiao Bao, Wusheng Zhang

2 Outline Sunway TaihuLight Supercomputer CERNET and CERNET2 Challenges Solution Performance Q&A Remarks 2

3 National Supercomputing Center in Wuxi (NSCC-Wuxi) PetaFlops No.1 among the TOP500 List Published at ISC High Performance, June 20,

4 Sunway TaihuLight: System Overview The first China homegrown many-core processor: SW cores per chip divided into four core groups (CGs) 3 Tflops The first system in the world that provides over 100 Pflops performance with over 10 million cores theoretical peak 125 Pflops, 2.5 times improvement over before LINPACK performance 93 Pflops, 3 times improvement over before High energy-efficiency of the overall system 6.05 Gflops/Watt, 3 to 6 times improvement over before Co-operated by Jiangsu Gov., Wuxi Gov., andtsinghua University 4

5 10M-core system 163,840 processes 65 threads racks chips core-groups cores total number of cores 1,024x 40 x 260-core Chip Rack System 5

6 Wind Resource Estimation (National Climate Center, located at Beijing) 4020 simulation cases Data generation rate: ~1.5TB/hour Need stable and high bandwidth network connection 80m_wind(m/s) T2(K) 6

7 Dark Matter Particle Explorer (WuKong) Purple Mountain Observatory Chinese Academy of Sciences, located at Nanjing Observation data need to be transfered from Nanjing to Wuxi. Processing result feed back to Nanjing. ~19 hundred million particles per year. 7

8 CERNET and CERNET2 IPv4 IPv6 100G 100G TaihuLight connected to CERNET/CERNET2 via 100G 8

9 Challenges High performance Single flow TCP BGP multi-homing Both to academic and commodity networks University firewall Performance bottleneck Traffic based charging Distinguish the traffic Security VPN IPv4-only applications Upgrade IPv6 takes time BGP optimization MPLS SDN? Try something differently 9

10 Solution Infrastructure Wuxi supercomputer center: dual stack CERNET2: 100G IPv6 Campus network: dual stack Building block Stateless IPv4/IPv6 translation technology 10

11 Stateless translation (RFC7915) 11

12 Address mapping (RFC6052) 12

13 Translation example 13

14 Double translation (4aaS) IPv4 prefix1 IPv6 prefix1 IPv6 prefix2 IPv4 prefix2 IPv4 IVI core IPv6 IVI edge IPv4 IPv6 IPv4 IPv4 IPv6 IPv4 Decomposition: User IPv4 prefix Network IPv6 prefix 14

15 Features The network can by fine-toned using IPv6 prefixes More flexible and controllable than IPv4 BGP More cost effective than MPLS The network O&M is very simple End to end address transparency, no encapsulation is required No need to upgrade the application at this stage The application can still be IPv4 The different charging model can be applied via difference IPv6 prefix 20Gbyte/user/month global Internet access Unlimited limited domain (Wuxi) IPv6 prefix 15

16 Topology IVI-B IPv4 RFC1918 Wuxi TUNET IPv4 IVI-A CERNET2 IPv6 IVI-B IPv4 RFC1918 IPv4 IPv6 IPv4 IPv4 IPv6 IPv4 IPv6 100G 100G 10G 16

17 Performance testing max= G max= G max= G max= G max= G TCP single flow 17

18 Q&A Why not use IPv4 VPN? Performance, not transparent Why not use IPv6-only? Need to upgrade the applications Why not use IPv4 over IPv6? Network O&M need decapsulation Why not use stateful NAT64? Cannot support bidirectional initiated communication Why not use 464xlat? Cannot support bidirectional initiated communication Why not use stateless single translation? Need to upgrade the applications 18

19 Traffic control 2001:da8:ff3a:c8fb:300:: -> M (blue) IPv6 null route > 2001:da8:ff3a:cbfb:200:: (w/ rate-limit) > 2001:da8:ff3a:cbfb:300:: (w/o rate-limit) Beginning of ratelimit End of ratelimit 19

20 Next steps Link more end sites 20+ International sites Scheduling mechanism (central control) More supercomputer centers Wuxi: Sunway TaihuLight(1) Guangzhou Tianjin Shenzhen Jinan Changsha Qingdao etc 20

21 International collaboration Sunway TaihuLight Supercomputer is an open facility operating by Tsinghua University Team It can provide services for the international partners Contact: 21