Amazon S3 Cloud-storage Service

Size: px

Start display at page:

Download "Amazon S3 Cloud-storage Service"

Brooke Harvey
5 years ago
Views:

1 On the Network Performance of Amazon S3 Cloud-storage Service Valerio Persico*, Antonio Montieri, Antonio Pescapè*, *University of Napoli Federico II (Italy) NM2 srl (Italy) IEEE CloudNet 2016 Pisa, Italy, October 4th 2016

2 (Public) Cloud Paradigm An increasing number of services and applications rely on Public Clouds XaaS: everything as a service Pay-as-you-go resources No upfront investments Real-time provisioning Ubiquitous access Autoscaling COMPUTATION STORAGE NETWORKING Cloud infrastructures backed by huge investments from the providers Complex and continuously evolving infrastructures Hot research topic ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 2

Cloud networking and traffic growth Cloud network performance is critical Without high-performance networks, there would be no such thing as cloud computing* The network can be the bottleneck for

3 Cloud networking and traffic growth Cloud network performance is critical Without high-performance networks, there would be no such thing as cloud computing* The network can be the bottleneck for computation (e.g. scientific computing, video processing, etc.) Poor network provisioning can severely compromise user experience Cloud traffic is rapidly growing Since 2008, most of the IP traffic has originated or terminated in a datacenter More than 83% of datacenter traffic will be cloud traffic by 2019** Public cloud is growing faster than private cloud *Mogul and Popa, 2012 **Cisco Global Cloud Index: Forecast and Methodology ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 3

4 Cloud storage (Storage-as-a-Service) On-line services for archiving and backup data Simple and clean user interface available Complexities of direct HW management abstracted away High availability and reliability guaranteed by Service Level Agreements (SLAs) Cloud network heavily impacts the Quality of Service (QoS) experienced by the customers of cloud storage services The performance of cloud-to-user network is hard to predict Cloud providers rarely make promises or expose details about expected performance ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 4

5 Public cloud network monitoring Effective, efficient, and accurate monitoring is constantly needed to manage increasing complexity Providers rarely make promises or expose details about the network Customers have to cope with the limited awareness about cloud networking environments Performance unpredictability Performance variability Hidden datacenter topology and exact service location ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 5

6 Public cloud network monitoring Effective, efficient, and accurate monitoring is constantly needed to manage increasing complexity Providers rarely make promises or expose details about the network Customers have to cope with the limited awareness about cloud networking environments Performance unpredictability Performance variability Hidden datacenter topology and exact service location Non-cooperative monitoring approaches do not require access to any restricted information adopt the point of view of the general consumer Customers are able to validate and extend the information supplied by the provider perform informed choices among different services or different providers ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 6

7 Contribution Large-scale distributed campaign Analysis of the performance of the cloud-to-user network associated with Amazon S3 General assessment of the performance of the service Evaluation of the impact on the performance of a number of configuration choices Usage guidelines for customers ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 7

8 Methodology EU US AP SA 4 Cloud Regions North Virginia (US) Ireland (EU) Singapore (AP) Sao Paulo (SA) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 8

(SA) 77 vantage points (VPs) United States (US) Europe (EU) Asia-Pacific (AP) Central-South

9 Methodology EU US CSA ZA AP Project 4 cloud BISmark regions Platform Router-based North Virginia (US) Nodes Ireland voluntarily (EU) hosted by home Singapore users(ap) Sao Paulo (SA) 77 vantage points (VPs) United States (US) Europe (EU) Asia-Pacific (AP) Central-South America (CSA) South Africa (ZA) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 9

10 Methodology 477 cloud vantage regions points (VPs) Experimental campaign and dataset North 7 United daysvirginia States (US) Ireland 40 Europe downloads (EU) (EU) every 2 hours Singapore Different Asia-Pacific file (AP) sizes (AP) Sao Different South Paulo Africa storage (SA) (SA) classes Standard Central-South America (CSA) Content Delivery Network (CDN) Enabled ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 10

Methodology 477 cloud vantage regions points (VPs) Experimental campaign and dataset North 7 United daysvirginia States (US) Ireland 40 Europe downloads (EU) (EU) every 2 hours Singapore Different

11 Methodology 477 cloud vantage regions points (VPs) Experimental campaign and dataset North 7 United daysvirginia States (US) Ireland 40 Europe downloads (EU) (EU) every 2 hours Singapore Different Asia-Pacific file (AP) sizes (AP) Sao Different South Paulo Africa storage (SA) (SA) classes Standard Central-South America (CSA) Content Delivery Network (CDN) Enabled ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 11

12 Impact of the file size 2500KiB/s, on average when downloading 100 MiB files -33% w 16MiB -80% w 1MiB Goodput is impacted by TCP dynamics Performance depends on the file size Impact on user experience Impact on monitoring activities ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 12

13 Impact of the cloud region US: 3562 KiB/s EU: 2791 KiB/s SA: 1445 KiB/s AP: 2018 KiB/s US region is the best global choice from both the performance and the economic point of view US and EU 45% better than AP and SA, on average Counterintuitively AP and SA are associated to higher costs ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 13

14 Impact of the placement of the VP Best performance obtained for US VPs retrieving objects from the same region, on average VPs Worst performance observed for AP VPs retrieving objects from SA and vice versa (CSA from AP) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 14

15 Impact of the placement of the VP Best performance obtained for US VPs retrieving objects from the same region, on average VPs Worst performance observed for AP VPs retrieving objects from SA and vice versa (CSA from AP) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 15

16 Impact of the placement of the VP Best performance obtained for US VPs retrieving objects from the same region, on average VPs Worst performance observed for AP VPs retrieving objects from SA and vice versa (CSA from AP) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 16

The performance grows by 44%, on average, considering VP and cloud bucket placed

17 How much does the performance improve if users rely on buckets placed in the same geographical region? The performance grows by 44%, on average, considering VP and cloud bucket placed in the same geographical regions AP is the region with the major improvement (+77%) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 17

18 Adoption of the CDN service Better performance in terms of goodput Markedly higher improvement for smaller sizes 100MiB +104%, on average 1MiB +274%, on average NO CDN 1MiB CDN 1MiB NO CDN 100MiB CDN 100MiB Lower performance variability -28 % in terms of Coefficient of Variation (CoV) CDN ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 18

19 User to CDN node mapping 38/54 CDN nodes encountered CDN nodes placed in the same geographical region of the VP in the US region VPs For only 18/48 VPs the best performing node has the highest occurrence frequency ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 19

20 User to CDN node mapping 38/54 CDN nodes encountered CDN nodes placed in the same geographical region of the VP in the US region VPs For only 18/48 VPs the best performing node has the highest occurrence frequency ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 20

21 User to CDN node mapping 38/54 CDN nodes encountered CDN nodes placed in the same geographical region of the VP in the US region VPs For only 18/48 VPs the best performing node has the highest occurrence frequency ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 21

22 CDN doesn t lead always to better performance than S KiB/s, on average ( %) In 30% of the cases average improvement lower than 100 KiB/s x Goodput gain with CDN In 119 out of 924 cases the CDN leads to worse performance Worst case: % degradation Goodput gain with CDN ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 22

23 Conclusion Cloud-storage high level management interface hides implementation details and performance figures Distributed experimental campaign leveraging the BISmark platform General assessment of the performance in terms of goodput US and EU cloud regions offer better performance (+45%, on average) at lower cost Enabling the CDN service leads to +144% improvement A number of cases found exposing a detrimental effect of the CDF service (up to -44%) ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 23

CloudSurf a platform to monitor public-cloud networks Non-cooperative approaches - adoption of the point of

- different providers - different use scenarios Predictability of experimental costs - computation cost -

specific monitoring skills needed Public availability CloudSurf User http://traffic.comics.unina.

24 CloudSurf a platform to monitor public-cloud networks Non-cooperative approaches - adoption of the point of view of the general customer - no need for access to information restricted to the provider Comprehensiveness - different providers - different use scenarios Predictability of experimental costs - computation cost - network-usage cost Results sharing - community results repository Ease of use - on demand measurements - no specific monitoring skills needed Public availability CloudSurf User Master Results Repository Cloud probe Management API Virtual machine Cloud credentials Up-to-date Pricing Information

25 ? Questions?

27 Impact of the placement of the VP For some VPs the performance is not significantly affected by the cloud region (e.g., AP12, EU7, US13) In other cases, a non-negligible discrepancy is measured on changing buckets (e.g., AP11, EU8) The globally optimal choice could be dramatically outperformed by local optimum choices ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 28

28 Impact of the placement of the VP For some VPs the performance is not significantly affected by the cloud region (e.g., AP12, EU7, US13) In other cases, a non-negligible discrepancy is measured on changing buckets (e.g., AP11, EU8) The globally optimal choice could be dramatically outperformed by local optimum choices ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 29

29 Impact of the placement of the VP For some VPs the performance is not significantly affected by the cloud region (e.g., AP12, EU7, US13) In other cases, a non-negligible discrepancy is measured on changing buckets (e.g., AP11, EU8) The globally optimal choice could be dramatically outperformed by local optimum choices ON THE NETWORK PERFORMANCE OF AMAZON S3 CLOUD-STORAGE SERVICE 30

On the Network Performance of Amazon S3 Cloud-storage Service

On the Network Performance of Amazon S3 Cloud-storage Service Valerio Persico, Antonio Montieri 2, Antonio Pescapè,2 University of Napoli Federico II (Italy) and 2 NM2 s.r.l. (Italy) {valerio.persico,