Performance Benchmarking/Testing of CARC Network, and Connectivity (CRF<- >RSC<- >Penguin)

Transcription

1 Performance Benchmarking/Testing of CARC Network, and Connectivity (CRF<- >RSC<- >Penguin) CARC Hussein Al- Azzawi June 2, 2014

2 Introduction The purpose of this document is to summarize testing and benchmarking results from the evaluation of CARC computer network connectivity to the Cancer Research Facility over private fiber connection, which serves as a seed for the future UNM Science DMZ network. All tests were performed during the last week of May 2014, by: Jason Byars (CRF). Network setup and configuration: Hussein Al-Azzawi (CARC/Psychology), Jason Byars (CRF), Susan Wilson (CARC), and Ben Archuleta (CARC). Network consulting: Edward May (IT Networks). Summary and Results Testing and benchmarking results confirmed 10G network connectivity all the way to CARC over the IT-Core dedicated fiber link coming from CRF, the end-to-end connectivity between CRF, CARC, and Penguin also shows 10G network utilization. Based on the testing results, its possible to transfer 2 TB/hr over the CRF <--> CARC <--> Penguin path. We used two methods for testing: direct Ceph file transfer and perfsonar NPAD test. The results from Ceph file transfer were consistently above 500 MBps (4000 Mbps) on a 160GB transfer, perfsonar testing results were about 6580 Mbps, when MTU size is 9000 bytes. File Transfer Testing We transferred files of 160GB over the link between CRF, RSC, and Penguin, using Ceph file transfer; the results were consistently above 500 MBps (4000 Mbps). These results came as expected from such type of network, which is at the read limit of the SSD on the CRF end. perfsonar Testing: We run the perfsonar NPAD test 17 times, changing the MTU size starting with 1448 [Standard size] bytes, increments of 500 bytes, up to 8948 bytes [Jumbo frames]. As we can see from the results, as we increase the MTU size the maximum data rate increases, this makes sense because the as we are utilizing the 10G fiber link more and more, and we are reducing frame header overhead. The total loss is also increasing; the reason behind this is having more data being transmitted during each test cycle. 2

3 CARC Internal perfsonar_server CRF_10G_Client 1. NPAD test run 1: a. RTT: 20 msec b. MTU: 1448 bytes. c. Target rate: 1000 Mbps d. The maximum data rate was Mbps. e. Total loss rate %. 2. NPAD test run 2: b. MTU: 1448 bytes. c. Target rate: 2000 Mbps d. The maximum data rate was Mbps. e. Total loss rate %. 3. NPAD test run 3: b. MTU: 1948 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 4. NPAD test run 4: b. MTU: 2448 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 5. NPAD test run 5: b. MTU: 2948 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 6. NPAD test run 6: b. MTU: 3448 bytes. d. The maximum data rate was Mbps. 3

4 e. Total loss rate %. 7. NPAD test run 7: b. MTU: 3948 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 8. NPAD test run 8: b. MTU: 4448 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 9. NPAD test run 9: b. MTU: 4948 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 10. NPAD test run 10: b. MTU: 5448 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 11. NPAD test run 11: b. MTU: 5948 bytes. d. The maximum data rate was Mbps. e. Total loss rate %. 12. NPAD test run 12: b. MTU: 6448 bytes. 4

5 d. The maximum data rate Mbps. e. Total loss rate %. 13. NPAD test run 13: b. MTU: 6948 bytes. d. The maximum data rate Mbps. e. Total loss rate %. 14. NPAD test run 14: b. MTU: 7448 bytes. d. The maximum data rate Mbps. e. Total loss rate %. 15. NPAD test run 15: b. MTU: 7948 bytes. d. The maximum data rate Mbps. e. Total loss rate %. 16. NPAD test run 16: b. MTU: 8448 bytes. d. The maximum data rate Mbps. e. Total loss rate %. 17. NPAD test run 17: b. MTU: 8948 bytes. d. The maximum data rate Mbps. e. Total loss rate %. 5

6 Glossary CARC: UNM Center for Advanced Research Computing [2]. CARC_Cuda02: Two identical machines, each has: 8 Core i7-2600k 3.40GHz, Myri-10G Dual-Protocol NIC, 16GB Ram, 2 x 1TB drives, 60GB SSD, 2 x MSI N560GTX-Ti [2]. CARC_perfSONAR_Server: Internal perfsonar server, 10G connection direct to the RSC 10G switch, used for internal network testing and troubleshooting, and secured external accessed through CARC s firewall by request, hosting server: CARC_Cuda02, IP address: /8, connected to VLAN-15, access link: [2]. CRF: UNM Cancer Research Facility. MMR: CARC main machine room [2]. NPAD: Allows end-users to test limited portions of the network path and attempts to determine if there are issues that would adversely affect longer paths. The user provides a target data rate and round-trip-time (RTT) and NPAD attempts to determine if that should be possible, given the infrastructure on that limited portion of the path [1]. perfsonar: Is a web services-based infrastructure for collecting and publishing network performance monitoring. A primary goal of perfsonar is making it easier to solve end-to-end performance problems on paths crossing several networks. It contains a set of services delivering performance measurements in a federated environment [1]. RSC: Research Storage Consortium, storage system size is 0.56 Petabyte, lives in CARC MMR [2]. RTT: Stands for round trip time, the required time of a packet to travel from the source to destination, and the time of the acknowledge to travel back from the destination to the source. 6

7 References [1] Brian Tierney, Martin Swany, and Je Boote. perfsonar: Instantiating a Global Network Measurement Framework. ACM SIGOPS Operating Systems Review, 43(4):50 53, January [2] Systems Group, CARC Internal Documentation, CARC. 7