SLIPSTREAM: AUTOMATIC INTERPROCESS COMMUNICATION OPTIMIZATION. Will Dietz, Joshua Cranmer, Nathan Dautenhahn, Vikram Adve

Transcription

1 SLIPSTREAM: AUTOMATIC INTERPROCESS COMMUNICATION OPTIMIZATION Will Dietz, Joshua Cranmer, Nathan Dautenhahn, Vikram Adve

2 Introduction 2 Use of TCP is ubiquitous Widely Supported Location Transparency Programmer-friendly but not always ideal Faster IPC exists for local communication Goal: Best of TCP generality with fast performance locally

3 Motivating Example 3 (Java, C, Python, ) Database

4 Motivating Example 4 (Java, C, Python, ) Database Optimization opportunity: Local communication could use UDS

5 Motivating Example Manually Optimized 5 (Java, C, Python, ) Database UDS vs TCP: up to 2x ops/sec! Avoid TCP layers

6 Motivating Example Manually Optimized 6 (Java, C, Python, ) Can we do this automatically? Database UDS vs TCP: up to 2x ops/sec! Slipstream: Automatic IPC Optimization Avoid TCP layers

7 What s Coming 7 Slipstream Overview How Slipstream Works Selected Results Performance Compatibility Docker Conclusions

8 8 Slipstream Overview

9 Design Principles: Automatic and Useful 9 (Java, C, Python, ) Database No App. Modifications Language Agnostic No Modifications No Network Assumptions Backwards Compatible

10 Slipstream Architecture: What is it? 10 (Java, C, Python, ) Database

11 Example + (libipc) 11 (Java, C, Python, ) Database libipc libipc libipc libipc: Shim library between application and OS

12 Example + (libipc + ipcd) 12 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd ipcd: Coordinating daemon for all libipc on host

13 Example + Slipstream 13 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd

14 14 How Slipstream Works

15 How Slipstream Works: Overview 15 Track TCP State Detect Local Communication Switch to UDS

16 Tracking TCP State 16 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

17 1) Tracking TCP State 17 read write listen (Java, C, Python, ) libipc Database libipc: Transparently monitors TCP operations ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

18 1) Tracking TCP State 18 read write listen fork (Java, C, Python, ) libipc exec dup Database libipc: Transparently monitors TCP operations ipcd libipc libipc ipcd libipc: Maintains state throughout execution Track TCP State Detect Local Communication Switch to UDS

19 Detecting Local Communication 19 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

20 Detecting Local Communication 20 (Java, C, Python, ) ipcd: Uses information from libipc to perform endpoint Database matching libipc ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

21 Detecting Local Communication 21 (Java, C, Python, ) libipc ipcd: Uses information Endpoint from Matching: libipc to identify local Memory 1. Store {SRC IP:Port, communication DEST Database IP:Port} 2. N-Byte Checksum of Stream Data 3. Timing Windows ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

22 2) Detecting Local Communication 22 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Local communication detected! Track TCP State Detect Local Communication Switch to UDS

23 Switching to UDS 23 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

24 Switching to UDS 24 (Java, C, Python, ) libipc Database ipcd: Generate optimized socketpair, send to libipc instances ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

25 Switching to UDS 25 (Java, C, Python, ) libipc: Transparently switch Database to new transport libipc ipcd libipc libipc ipcd Track TCP State Detect Local Communication Switch to UDS

26 Switching to UDS 26 (Java, C, Python, ) libipc: Transparently switch Database to new transport libipc ipcd libipc libipc: ipcd Preserve expected socket behavior for compatibility Track TCP State Detect Local Communication Switch to UDS

27 Switching to UDS 27 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Slipstream: up to 2x ops/sec Track TCP State Detect Local Communication Switch to UDS

28 28 Selected Results

29 Performance vs Local TCP 29 Memcached Up to +80% ops/sec! At least +40% ops/sec!

30 Performance vs Manual UDS 30 Netperf UDS Slipstream automatically achieves performance similar to manual UDS!

31 Performance Overhead 31 Netperf ~3.5% overhead Memcached < 3% overhead

32 Software Compatibility 32 Supports many popular server applications MySQL, PostgreSQL, Redis, Memcached, Apache, Jenkins, Language agnostic in practice Java, C, Python, All work correctly with Slipstream All are optimized when communicating locally Supports use with Docker!

33 Example with Docker 33 (Java, C, Python, ) Database Put app in container Container per component Slower comm., extra layers

34 Example with Docker + Slipstream? 34 (Java, C, Python, ) Database Did someone say they need Automatic IPC Optimization? Put app in container Container per component Slower comm., extra layers

35 Slipstream with Docker 35 (Java, C, libipc Python, ) libipc Database libipc per container

36 Slipstream with Docker 36 (Java, C, libipc Python, ) ipcd libipc Database libipc per container Connect to common ipcd

37 Slipstream with Docker 37 (Java, C, libipc Python, ) ipcd libipc Database libipc per container Connect to common ipcd ipcd can run in a container too

38 Slipstream with Docker 38 (Java, C, libipc Python, ) ipcd libipc Database Optimize across containers!

39 Slipstream with Docker 39 (Java, C, libipc Python, ) ipcd libipc Database Optimize across containers! 2x-3x ops/sec over Docker Same perf. as without Docker

40 Docker with NetPIPE-C 40 Slipstream shows large speedups Same perf. with or without Docker! Baseline Docker is significantly slower

41 Conclusions: 41 Automatically use faster IPC mechanism when available Easy to deploy, highly compatible No OS modifications No modifications to application Language-agnostic Backwards-compatible (partial deployment supported) ~2x bandwidth improvement for host-local communication Low overhead when optimization is not possible Freely Available at Thank you! Questions?

42 (End of presentation) 42

43 43 FAQ

44 Q: Why use UDS instead of other local IPC? 44 Simply: Socket interface of UDS facilitated prototyping Any local IPC mechanism should be possible (Patches welcome! ) Others could avoid kernel interaction for even better performance See ipc-bench paper Slipstream is not UDS-specific UDS implementation detail, idea is faster performance leveraging locality In fact, netperf sometimes slower with UDS (see paper).

45 Q: Why not have application do this itself? 45 Legacy: many applications don t do this today (Knowledge that UDS faster than TCP is not new!) They shouldn t have to, Slipstream! Achieves performance of manual optimization Low overhead when optimization not possible Applications have better things to do than transport selection!

46 Q: Why userspace solution, not in-kernel? 46 Mostly: Legacy Ease of use, ease of deployment Suitable for use on other systems (FreeBSD, etc.) May not be appropriate for kernel Correctness concerns (Opt-in approach likely appropriate) Possibility for using even more efficient transports than UDS Can t cut out kernel with a kernel-based solution

47 Q: What about frequent short-lived connections? 47 Current implementation not well-suited for this, but could be! Would not benefit from optimization only applied to longer-lived connections In fact, initial connection latency over localhost increased Not by design, result of implementation choices (but nevertheless) Probably better off with custom is local decision mechanism Ex: Is destination IP Would allow optimizing before connection, get faster connect() times Thanks for the question, Joshua.

48 48 Extra Slides

49 Related Work 49 Category Name Application Transparency OS Transparency OS Feature VM-VM Userspace Networking Stack Userspace Shim Library Windows Fast-Path Solaris TCP Fusion AIX fastlo Linux TCP Friends? XWay XenSocket XenLoop mtcp Sandstorm Fable Java Fast Sockets Universal Fast Sockets FastSockets Slipstream

50 Building on Slipstream 50 Alternative transports (shared memory, etc.) (see ipc-bench) Select best transport for workload/configuration Support multiple locality detection mechanisms Provided one is suitable for many use-cases If can assume network topology, can directly determine this More exhaustive testing, performance investigation

51 Try it out today! 51 No configuration, simple deployment Big performance gains for many applications (Docker support!) Low overheads when optimization is not possible Makes most of partial deployment Open-source, CRAPL (don t judge, it s research!) Download today!

52 53 Deploying Slipstream

53 Example + Slipstream: Deployment 54 (Java, C, Python, ) Database

54 Example + Slipstream: Deployment 55 (Java, C, Python, ) Database libipc libipc libipc libipc deployed with application (LD_PRELOAD or /etc/ld.so.preload)

55 56 Challenges and Solutions

56 Example + Slipstream: Key Challenge 57 (Java, C, Python, ) Challenge: Which connections are communicating locally? Database libipc ipcd libipc libipc ipcd libipc only sees what goes through it Solution: Endpoint matching algorithm to identify local communication

57 Endpoint Matching Overview 58 It s the orange one! (Java, C, Python, ) libipc works with ipcd which has more information Database libipc ipcd libipc libipc ipcd Observed behavior from both libipc instances used by ipcd to find match

58 Endpoint Matching Overview 59 It s the orange one! (Java, C, Python, ) libipc works with ipcd which has more information Database libipc ipcd libipc libipc ipcd What information can be used to help pair endpoints? Observed behavior from both libipc instances used by ipcd to find match

59 Endpoint Matching Overview 60 Src/Dst IP and Port (Java, C, Python, ) libipc works with ipcd which has more information Database Data Sent/Recv libipc Timing Details ipcd libipc libipc ipcd and it s orange. Observed behavior from both libipc instances used by ipcd to find match

60 Endpoint Matching Overview 61 Src/Dst IP and Port (Java, C, Python, ) libipc works with ipcd which has more information Database Data Sent/Recv libipc Timing Details ipcd libipc libipc ipcd More details in paper! Observed behavior from both libipc instances used by ipcd to find match

61 Endpoint Matching Overview 62 (Java, C, Python, ) libipc works with ipcd which has more information Database libipc ipcd libipc libipc ipcd Observed behavior from both libipc instances used by ipcd to find match

62 Endpoint Matching Overview 63 (Java, C, Python, ) libipc works with ipcd which has more information Database libipc ipcd libipc libipc ipcd Observed behavior from both libipc instances used by ipcd to find match Local endpoints found!

63 Endpoint Matching: Complete 64 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Local endpoints found!

64 Automatic IPC Optimization 65 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd Transparently switches to UDS

65 Automatic IPC Optimization 66 (Java, C, Python, ) Database libipc ipcd libipc libipc ipcd UDS vs TCP: up to 2x ops/sec

66 Without Modifying Applications? 67 (Java, C, Python, ) Implementation Challenge: libipc must preserve expected socket behavior Database for compatibility libipc ipcd libipc libipc ipcd Solution: Engineering!

67 68 Results, with slides. Source:

68 Faster than TCP? NetPIPE Microbenchmarks 69 NetPIPE-C NetPIPE-Java 1.4x to 2.5x improvement! Works with both C and Java!

69 Faster than TCP? Memcached x to 1.8x ops/sec

70 Faster than TCP? PostgreSQL 71 PostgreSQL TPC-B PostgreSQL - Select No improvement when CPU/IO bound Small improvement for Select-only

71 Faster than TCP? PostgreSQL 72 PostgreSQL TPC-B PostgreSQL - Select Application benefit depends on how much network is the bottleneck No improvement when CPU/IO bound Small improvement for Select-only

72 Slower than TCP? Slipstream Overhead 73 Netperf Memcached ~3.5% overhead for Netperf < 3% overhead for Memcached

73 Slower than TCP? Slipstream Overhead 74 Netperf Memcached Overhead incurred is low in even network-intensive applications ~3.5% overhead for Netperf < 3% overhead for Memcached

74 Slower than manual UDS? Netperf Microbenchmark 75 Relative to fixed benchmark (Default UDS does poorly)

75 Slower than manual UDS? Netperf Microbenchmark 76 Relative to fixed benchmark Automatically achieves performance close to manually using UDS! (Default UDS does poorly)

76 Docker with NetPIPE-C 77 Slipstream shows large speedups Same perf. with or without Docker! Baseline Docker is significantly slower

77 Docker with Memcached 78 Memcached with Docker Curious: Memcached without with Docker Slipstream performs better across Docker containers Baseline Docker has lower performance

78 Software Compatibility 79 Supports many popular server applications MySQL, PostgreSQL, Redis, Memcached, Apache, Jenkins, Language agnostic in practice Java, C, Python, And of course our microbenchmarks: NetPIPE, netperf, iperf, lmbench All work correctly with Slipstream All are optimized when communicating locally

79 Slipstream Overview Multiple Applications 80 ipcd: One instance per host Web App libipc libipc libipc ipcd libipc: One instance per process