Custom Memory Allocation For Free

Transcription

1 Custom Memory Allocation For Free Alin Jula and Lawrence Rauchwerger Texas A&M University November 4, 2006

2 Motivation Memory Allocation Matters Goals Speed Data locality Fragmentation Current State of Affairs General Allocators Portable, Fast Limited Locality, Context-oblivious Custom Allocators Improved Locality, Fast, Context-aware Not portable across applications, Hard to develop

3 Motivation Custom Memory Allocation for Free C++ Standard Template Library (STL) provides context portability Performance Context - container specific Size, Neighbors or Proximity, Container Dynamism, Traversal Locality Improvement Productivity Portability - integration in STL containers Applications need not change a single line of code Generic Interface Selection of Partition and Allocation strategies a In Latin, defero means to communicate

4 Motivation Custom Memory Allocation for Free C++ Standard Template Library (STL) provides context portability Performance Context - container specific Size, Neighbors or Proximity, Container Dynamism, Traversal Locality Improvement Productivity Portability - integration in STL containers Applications need not change a single line of code Generic Interface Selection of Partition and Allocation strategies Defero a = Container-Centric Memory Allocation a In Latin, defero means to communicate

5 Motivation Example allocated allocated allocated allocated malloc(8)

6 Motivation Example allocated allocated allocated allocated malloc(8)

7 Motivation Example allocated allocated allocated tail Link* node=malloc(8) tail next=node; Semantic Context

8 Motivation Example allocated allocated allocated Link* node=malloc(8) tail next=node; Semantic Context tail

9 Partition Design Strategy 1 Select an equivalence relation 2 Partition the memory address space 3 Allocation = search for the target equivalence class Select any equivalent address 4 Deallocation = search for its equivalence class Insert into the class Invariant Memory is always partitioned in equivalence classes, for any allocation pattern

10 Partition Allocation in a Space of Equivalence Classes Search in an ordered space of equivalence classes Allocation Predicate guides the search target class binary search algorithm

11 Partition Generic Interface Increased Productivity for Custom Memory Allocation Development Flexible Design Generic Partition Interface Equivalence Class Partition Generic Allocation Predicate Allocation Strategy Code Example defero<equivalence-relation, Order-Predicate, Allocation-Predicate>

12 Partition K-Bit Objective: allocate close to a target, at all times K-Bit equivalence relation a KBit b iff first K higher-order bits are the same. K-class Example K=8 0xFF Bit 0xFF Partition based on address K (0, 32) adjustable

13 Partition Other Equivalence Relations Segregated-Lists Objective: optimize the common sizes - 90% of the allocated objects are small x y iff (size(x) 128 size(y) 128) or ( size(x) 8 = size(y) 8 ) otherwise. Partition based on size Cache Sets Objective: Reduce conflict misses - Spread the cache contention Partition based on cache sets

14 Partition Other Equivalence Relations Segregated-Lists Objective: optimize the common sizes - 90% of the allocated objects are small x y iff (size(x) 128 size(y) 128) or ( size(x) 8 = size(y) 8 ) otherwise. Partition based on size Cache Sets Objective: Reduce conflict misses - Spread the cache contention Partition based on cache sets Focus of this talk K-Bit

15 Partition K-Bit Implementation Equivalence Classes Trees for K-classes Need to search Equivalent Elements Lists for equivalent elements

16 Partition Defero s Structure 8 Composition of Partitions 1 Segregated-Lists 2 K-Bit 2-D allocation (size,address) Selects a size-class and then a K-class Deallocation - returns a chunk to its size-class and K-class Complexity O(K)

17 Allocation Predicates First-Fit Allocation Predicate Selects first K-class (root) Guarantees consecutive allocations from the same K-class Favors temporal locality Allocation O(1), deallocation O(K)

18 Allocation Predicates Best-Fit Allocation Predicate Closest available address to a target Binary search performed on a tree K dictates how close Favors spatial locality and irregular allocation patterns Allocation/deallocation O(K).

19 Allocation Predicates Path - a Novel Allocation Predicate for trees Problem Rebalancing hurts locality Example Rotate-right and Rotate-left Solution allocate similar values together 20 Tree Container

20 Allocation Predicates Path - a Novel Allocation Predicate for trees Problem Rebalancing hurts locality Example Rotate-right and Rotate-left Solution allocate similar values together 20 Tree Container

21 Allocation Predicates Path - Example Insert 43 Left 0, Right 1 Path: Tree Container

22 Allocation Predicates Path - Example Insert 43 Left 0, Right 1 Path: 0 Tree Container

23 Allocation Predicates Path - Example Insert 43 Left 0, Right 1 Path: 0 1 Tree Container

24 Allocation Predicates Path - Example Insert 43 Left 0, Right 1 Path: Tree Container

25 Allocation Predicates Path - Example Allocate for value 43 K-Bit Tree Partition Left 0, Right 1 Path: 0 1 0

26 Allocation Predicates Path - Example Allocate for value 43 K-Bit Tree Partition Left 0, Right 1 Path: 0 1 0

27 Allocation Predicates Path - Example Allocate for value 43 K-Bit Tree Partition Left 0, Right 1 Path: 0 1 0

28 Allocation Predicates Path - Example Allocate for value 43 K-Bit Tree Partition Left 0, Right 1 Path: 0 1 0

29 Allocation Predicates Allocation Predicate Comparison Strengths and Weaknesses K-Bit with First Best Path Advantage + Temporal Locality + Spatial Locality + Spatial Locality + Fast + Container aware + Tree aware + Fixes worst case allocation patterns Disadvantage - Not container aware - Slower - Slower

30 Allocation Predicates Code Example //1. A l l o c a t e 4 b y t e s with F i r s t p r e d i c a t e i n t y= d e f e r o : : a l l o c a t e ( 4, F i r s t ) ; //2. A l l o c a t e z near x, or the c l o s e s t i n t z= d e f e r o : : a l l o c a t e ( 4, Best ( x ) ) ; //3. D e a l l o c a t e z d e f e r o : : d e a l l o c a t e ( z ) ; //3. L i s t with Defero (K Bit, F i r s t ) l i s t <i n t, d e f e r o <i n t, Kbit <12> >, F i r s t > s m a r t l i s t 1 ; //4. L i s t with Defero (K Bit, Best ) l i s t <i n t, d e f e r o <i n t, Kbit <12> >,Best> s m a r t l i s t 2 ;

31 Integration in STL STL + Defero Communication Container semantic context increased locality Memory Allocator Allocation context aware STL containers list, vector,deque, set, multi-set, map, multi-map, dynamic containers implementation

32 Integration in STL STL + Defero Communication Container semantic context increased locality Memory Allocator Allocation context aware STL containers list, vector,deque, set, multi-set, map, multi-map, dynamic containers implementation

33 Integration in STL STL + Defero Communication Container semantic context increased locality Memory Allocator Allocation context aware STL containers list, vector,deque, set, multi-set, map, multi-map, tree dynamic containers implementation

34 Integration in STL STL Containers List Context Proximity - Next and Prev Linear traversal Allocation predicates: First-fit, Best-fit Tree Context - more complex than list s Proximity : Parent, sibling, value distribution Traversal Element s value participates in the allocation process Allocation predicates: First-fit, Best-fit, Path

35 Selecting K Selecting K Is there a magic value for K? High K good locality, but slower Low K fast, but poor locality We want BOTH Virtual Page Size K (PageSize 4, PageSize + 4)

36 Selecting K Container s Dynamism How dynamic is a dynamic container? Methods 1 Modifying operations (insert, erase) M 2 Non-modifying operations (access) NM Dynamism D= NM+M, D (0,1) M High correlation with memory behavior High dynamism - fast memory allocation Low dynamism - locality improving memory allocation D and K - inversely proportional K (PageSize 4, PageSize + 4)

37 Evaluation Setup Intel(R) Xeon(TM) 3.00 GHz, 1GB memory g with -O3 Average over three runs Compared Defero against Doug Lea s allocator (DL) - best overall memory allocator GNU STL allocator - segregated lists Native new and malloc Defero - STL = K-Bit All allocators had the same internal fragmentation

38 Lists List - Synthetic Kernels Setup Normalized Time to STL GNU Normalized Time to STL GNU Create First Best DougLea GNU STL K bit Precision For each ++ First Best DougLea GNU STL K bit Precision Normalized Time to STL GNU Normalized Time to STL GNU Clear First Best DougLea GNU STL K bit Precision Sort First Best DougLea GNU STL K bit Precision

39 Trees Tree - Synthetic Kernels Normalized Time to STL GNU Normalized Time to STL GNU Create First Best Path DougLea GNU STL K bit Precision Find First Best Path DougLea GNU STL K bit Precision Normalized Time to STL GNU Normalized Time to STL GNU First Best Path DougLea GNU STL Clear K bit Precision For each First Best Path DougLea GNU STL K bit Precision Setup

40 Molecular Dynamics Molecular Dynamics Execution Time Hardware Counters Normalized to GNU STL Kbit First Kbit Match 0.8 DougLea Malloc New STL Kbit Precision Normalized to GNU STL L1 m L2 m TLB m Instr 24bit First 24bit Best Doug Lea New Malloc STL

41 Motivation Design & Implementation STL + Defero Polaris Polaris Perfect Benchmarks Compilation Time - below good, above bad Performance consistency 10 % faster vs. Original 5 % faster vs. Doug Lea s Results

42 Polaris Polaris Spec 89 Benchmarks Compilation Time - below good, above bad Performance consistency 20 % faster vs. Original 10 % faster vs. Doug Lea s

43 Polaris Summary Container-Centric Memory Allocation Improves data locality automatically STL + Defero Custom Memory Allocation for Free Portability and Performance K-Bit - Novel and adjustable partition Path - novel allocation predicate for trees... More Partitions... More Allocators

44 Polaris Thank You