Implications of Cache Asymmetry on Server Consolidation Performance

Transcription

1 Implications of ache Asymmetry on Server onsolidation Performance Presenter: Omesh Tickoo Padma Apparao, Ravi Iyer, Don Newell *Hardware Architecture Lab Intel orporation IISW

2 Outline Server onsolidation Asymmetric aches Performance Implications Measurement-Based Analysis onclusions / Future Work IISW

3 Server onsolidation Motivation Virtualization and consolidation are a growing trend in datacenters Majority of servers expected to run consolidated workloads within few years Workload Workload 1 Workload 2 Workload 3 Single O/S Guest OS Guest OS Guest OS VMM or Hypervisor Server Server Problem Performance analysis of consolidation scenarios is challenging Focus Different virtualization overheads depending on VMM & platform virtualization support Resource contention (core, cache, memory, etc) between VMs affects performance Server consolidation performance as a function of cache contention & asymmetry IISW

4 Why study asymmetry? MP platforms today have symmetric caches But space in cache is asymmetrically allocated depending on demand from virtual machines => Virtual Asymmetry Future MP platforms may have asymmetric caches Asymmetry to reduce cache space domination of die area Asymmetry due to process variability / faults => Physical Asymmetry IISW

5 ache Asymmetry Task1 Task2 Taskx Taskn Task1 Task2 Taskx Taskn ache ache ache ache ache ache (a) Symmetric Private aches of Equal Size (b) Virtually Asymmetric Shared aches of Equal Size Task1 Task2 Taskx Taskn Task1 Task2 Taskx Taskn ache ache ache ache ache ache (c) Physically Asymmetric Private aches of Different Size (d) Virtually & Physically Asymmetric Shared aches of Different Size What are the implications IISW 2008 on server consolidation performance? 5

6 onsolidation Benchmark vonsolidate 5 VMs -- SPEjbb VM -- Sysbench VM -- Webbench VM -- MailServer VM -- Idle VM VM/Workload Vcpus onfiguration Memory onfiguration in MB Java/SPEjbb (bops/sec) Database/Sysbench (Tx/sec) Web/Webench (Tx/sec) Mail/Exchange (hits/sec) Idle IISW

7 Platform onfiguration Hardware Intel Xen 5400 series Quadcore per socket 6MB+6MB $ per socket Used 4MB, 3MB, 2MB cache configs also to create physical asymmetry VMM Xen3.1 vonsolidate VM VM VM Xen 3.1 LL LL LL LL Memory IISW

8 Analyzing Implications Four Key onfigurations 1 Virtual Machine On physically symmetric cache On physically asymmetric cache Multi-Virtual Machine On physically symmetric cache But virtually asymmetric On physically asymmetric cache But virtually asymmetric also IISW

9 1VM / Symmetric aches Virtual Machine (no sharing) OR Virtual Machine (w/ sharing) All LLs of same size LL LL LL LL Memory SPEjbb Performance (Symmetric aches) Webbench Performance (Symmetric aches) Sysbench Performance (Symmetric aches) Metric normalized to 2MB MB 3MB 4MB 6MB Thruput PI MPI Metric normalized to 2MB MB 3MB 4MB 6MB Thruput PI MPI Metric normalized to 2MB MB 3MB 4MB 6MB Thruput PI MPI SPEjbb2005 most sensitive to cache 50% perf improvement from 2MB to 6MB Sysbench and Webbench show less than 10% improvement IISW

10 Multi-VM / Virtually Asymmetry onsolidated Virtual Machines (von) All LLs of same size LL LL LL LL Memory Metric normalized to when running alone SPEjbb Performance with Virtual ache Asymmetry (6MB) 1.60 JBBalone JBB+JBB JBB+Sysbench 6MB Thruput 6MB PI 6MB MPI JBB+Webbench JBB in von Metric normalized to when running alone SPEjbb Performance with Virtual ache Asymmetry (4MB) MB Thruput 4MB PI 4MB MPI JBBalone JBB+JBB JBB+Sysbench JBB+Webbench JBB in von onsolidation causes causes ~30% loss in performance ache Interference => 20% ore Inteference => 9% IISW

11 1VM / Physical Asymmetry Individual Virtual Machine LL LL LL LL LLs are 6M size Memory LLs are smaller (4M, 3M or 2M) Metric normalized to 6MB-6MB SPEjbb (Physically Asymmetric aches) Thruput PI MPI Metric normalized to 6MB-6MB Webbench (Physically Asymmetric aches) Thruput PI MPI Metric normalized to 6MB-6MB Sysbench (Physically Asymmetric aches) Thruput PI MPI SPEjbb2005 is affected the most Sysbench and Webbench are not affected much IISW

12 Multi-VM / Virtual+Physical Asymmetry Metric normalized to 6MB-6MB Metric normalized to 6MB-6MB SPEjbb Performance onvirtual+physical asymmetry Thruput PI MPI Sysbench Performance on Virtual+Physical Asymmetry Thruput PI MPI WebBench Performance onvirtual+physical Asymmetry LL LL LL LL LLs are 6M size onsolidated Virtual Machines (von) Memory LLs are smaller (4M, 3M or 2M) SPEjbb is affected the most (as expected) Sysbench and Webbench are not affected much Metric normalized to 6MB-6MB Opportunity to move Sysbench and Webbench to smaller cache cores => can improve performance of SPEjbb? Thruput PI MPI IISW

13 Inferences Asymmetry-Aware Scheduling Virtual Asymmetry Monitor usage and interference Modify VMM scheduler to take this into account Physical Asymmetry Monitor usage in large and small cores Modify VMM scheduler to affinitize ache-sensitive VMs to large-cache-cores ache-insensitive VMs to small-cache-cores IISW 2008 Affinitization Experiment: Affinitize one vcpu to large core Leave the other vcpu floating vcpu0 (affinitized vcpu1 JBB to 6MB) (floating) % benefit PI % MPI % vcpu0 (affinitized vcpu1 Sysbench to 6MB) (floating) % benefit PI % MPI % Webbench vcpu0 (6MB cache) vcpu1 (floating) % benefit PI % MPI % Allows for detection of sensitivity for Improved scheduling 13

14 Summary Presented cache asymmetry Symmetric Virtual Asymmetry Physical Asymmetry Virtual + Physical Asymmetry Studied the implications of cache asymmetry on a consolidation workload Using vonsolidate & asymmetric MP platform Showed cache contention overheads and overall cache sensitivity Discussed the potential for asymmetry-aware scheduling IISW