DAL: A Locality-Optimizing Distributed Shared Memory System

Size: px

Start display at page:

Download "DAL: A Locality-Optimizing Distributed Shared Memory System"

Valentine Richardson
5 years ago
Views:

1 DAL: A Locality-Optimizing Distributed Shared Memory System Gábor Németh, Dániel Géhbeger and Péter Mátray Ericsson Research {gabor.a.nemeth, daniel.gehberger, peter.matray}@ericsson.com HotCloud ' Page 1

Introduction Critical applications moving to the cloud Telecom applications Industrial IoT, cloud controlled collaborative systems Tight end-to-end

2 Introduction Critical applications moving to the cloud Telecom applications Industrial IoT, cloud controlled collaborative systems Tight end-to-end latency requirements Handling continuous influx of data Heavy state sharing and cross communication Various data access patterns HotCloud ' Page 2

3 Idea behind DAL For a fast data store, transport costs dominate response times Handle the locality of each data item separately & seamlessly move data to the worker process accessing it Flexible state externalization for critical applications HotCloud ' Page 3

4 High-level architecture Local data access Lock-free shared memory IPC Direct memory read Remote data access Local server acts as a proxy DPDK is used to bypass the kernel UDP based transport HotCloud ' Page 4

5 Locality optimized Key-value access Single key sharding Two-phase lookup: key location value Location is cached Distributed key space Server roles: key/data/combined Value can be moved separately of key location Automatic move to dominant accessor Key to location service Get location 1 Cache the location 2 DAL lib Client Value 3 Access HotCloud ' Page 5

6 Data access patterns Key-value Messaging PUB-SUB Watch for changes of a value 1 Write Value 2 Notify Application B Application C HotCloud ' Page 6

7 Data access patterns Key-value Messaging PUB-SUB Group PUB-SUB Watch for changes of a value Multiple parallel groups are all notified Load sharing in groups - Round-robin Application B 1 Write Value 2 Notify Application C HotCloud ' Page 7

8 Data access patterns Key-value Messaging PUB-SUB Group PUB-SUB Watch for changes of a value Multiple parallel groups are all notified Load sharing in groups - Round-robin - Sticky Value Application B Application B Application B 1 Write 2 Notify Application C Application C HotCloud ' Page 8

9 Data access patterns Key-value Messaging PUB-SUB Group PUB-SUB Request-response communication Combined with load sharing Notify on write 4 Response 3 Response Application 1 Request Request 2 Notify on write Service HotCloud ' Page 9

10 Data access patterns Key-value Messaging PUB-SUB Group PUB-SUB Request-response communication Decoupled Producers & Consumers Key-value Application C Write Value Messaging Application B HotCloud ' Page 10

11 Evaluation Random access of 1 million keys with 100 byte values Throughput 1.6M local write 0.9M remote read or write RAMCloud Open source key-value store Comparable remote access No local access Elementary data operations 50% [µs] 99% [µs] Local read Local write Remote read or write Comparison Local [µs] Remote [µs] RAMCloud using InfiniBand [3] RAMCloud in our lab Redis HotCloud ' Page 11 [3] Ousterhout et.al.: The RAMCloud Storage System (ACM TOCS (3), 7)

12 Stateless application Single session Application with geographic partitioning Sticky DAL messaging State externalization Host 1 p1, hash(p1) emulator Host 2 p1 ++ blue p1 HotCloud ' Page 12

13 Stateless application Single session Application with geographic partitioning Sticky DAL messaging State externalization Host 1 emulator Host 2 p1 blue p1 ++ green HotCloud ' Page 13

14 Stateless application Single session Application with geographic partitioning Sticky DAL messaging State externalization Seamless state handover Host 1 emulator Host 2 p1 ++ blue green p1 HotCloud ' Page 14

15 Stateless application Single session Host 1 emulator Host 2 p1 ++ blue green p1 HotCloud ' Page 15

16 Stateless application Mass evaluation Startup Stable Scaling Stable HotCloud ' Page 16

17 Conclusions DAL achieves 1 µs data access Per key automatic locality optimization Shared memory communication State externalization for latency sensitive applications Telecom session handling, industrial IoT Discussion / current focus areas Locations have to be cached: key-space must be partitionable Asynchronous replication More advanced optimization algorithms Data & key server scaling More use cases? HotCloud ' Page 17

18 HotCloud ' Page 18

DAL: A Locality-Optimizing Distributed Shared Memory System

DAL: A Locality-Optimizing Distributed Shared Memory System Gábor Németh Dániel Géhberger Péter Mátray Ericsson Research Abstract Latency-sensitive applications like virtualized telecom and industrial