Enabling Hybrid Parallel Runtimes Through Kernel and Virtualization Support. Kyle C. Hale and Peter Dinda

Transcription

1 Enabling Hybrid Parallel Runtimes Through Kernel and Virtualization Support Kyle C. Hale and Peter Dinda

2 Hybrid Runtimes the runtime IS the kernel runtime not limited to abstractions exposed by syscall interface opportunity for leveraging privileged HW features 2 HPDC 15

3 CURRENT OS/R MODEL 3 PARALLEL APP user-mode PARALLEL RUNTIME kernel-mode GENERAL PURPOSE OS HARDWARE

4 THE HYBRID RUNTIME 4 PARALLEL APP user-mode PARALLEL RUNTIME kernel-mode GENERAL PURPOSE OS PARALLEL APP HARDWARE HYBRID RUNTIME kernel mashup of OS and runtime HARDWARE

5 5

6 NESL NDPC Racket Legion 6

7 7 1.4 Xeon Phi x64 Speedup over Linux Legion Processor Count (Cores)

8 8 1.4 Xeon Phi x64 Speedup over Linux x speedup over linux Legion Processor Count (Cores)

9 9 1.4 Xeon Phi x64 Speedup over Linux x speedup over linux 1.37x speedup over linux Legion Processor Count (Cores)

10 TWO ENABLING TOOLS 10

11 kernel framework 11

12 12 HVM THE HYBRID VIRTUAL MACHINE bridge HRT with legacy OS

13 OUTLINE Background/Overview Nautilus Deployment Models Hybrid Virtual Machine Multiverse & Future Work 13

14 NAUTILUS 14 user-mode kernel-mode PARALLEL APP RUNTIME THREADS SYNC aerokernel PAGING EVENTS HW INFO BOOTSTRAP TIMERS IRQS HARDWARE Nautilus primitives & utilities (HRT can use or not use any of them) CONSOLE

15 15 Nautilus under the hood

16 16 Parallel Runtime System Nautilus

17 17 kernel primitives should be SIMPLE and FAST runtime developer can easily reason about them!

18 18 start with familiar interfaces threads condition variables mutexes/locks memory management fork/join parallelism

19 19 map runtime s logical view of machine Parallel Runtime System onto physical HW logical CPUs

20 threads 20 unified, shared address space threads can operate preemptively or cooperatively* *for runtimes that require more determinism

21 21 thread creation is FAST Cycles Nautilus Linux (pthreads) x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

22 22 thread creation is FAST Cycles Linux (pthreads) x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

23 23 thread creation is FAST ~3ms Cycles Nautilus Linux (pthreads) ~90µs x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

24 software events user-mode software events are SLOW 24 lower is better trigger latency (cycles) cond. var. futex hardware (IPI) x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

25 nautilus events triggers are FASTER trigger latency (cycles) naut. cond. var. futex cond. var. x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM naut. cond. var. w/ipi hardware (IPI)

26 26 IPI Nemo TRIGGER LATENCY (CYCLES)

27 27 IPI ~100 cycles Nemo TRIGGER LATENCY (CYCLES)

28 memory 28

29 29 static identity map bootup... page size... 2MB physical memory 1GB

30 30 eliminates expensive page faults reduces TLB misses + shootdowns increases performance under virtualization

31 31 NUMA first-touch NUMA domain NUMA domain runtime has FULL control over thread placement and memory layout stack

32 philix 32

33 33 virtual console philix host utility host Intel MPSS stack Xeon Phi custom OS

34 34 Nautilus is fairly small Nautilus 25,000 lines of mostly C Legion 43,000 lines of C++ Additions for Legion 800 lines of C/C++ Additions for Xeon Phi 1350 lines of C

35 What do we lose? 35

36 familiar environment 36

37 driver ecosystem 37

38 protection/isolation 38

39 OUTLINE Background/Overview Nautilus Deployment Models Hybrid Virtual Machine Multiverse & Future Work 39

40 40 DEDICATED APPLICATION HRT full HW access HARDWARE

41 41 PARTITIONED LINUX STACK APPLICATION HRT full HW access CORE 0 CORE K-1 HARDWARE CORE K CORE N-1

42 VM 42 PARALLEL APP LINUX STACK HRT VIRTUAL MACHINE 0 VIRTUAL MACHINE 1 VMM HARDWARE

43 43 Hybrid Virtual Machine LINUX STACK PARALLEL APP HRT regular OS vcores HVM HRT vcores VMM HARDWARE

44 OUTLINE Background/Overview Nautilus Deployment Models Hybrid Virtual Machine Multiverse & Future Work 44

45 regular OS (ROS) PARALLEL APP legacy functionality from ROS via HVM performance path 45 PARALLEL RUNTIME GENERAL PURPOSE OS user-mode kernel-mode user-mode kernel-mode PARALLEL APP HRT NODE HARDWARE GENERAL VIRTUALIZATION MODEL HVM NODE HARDWARE SPECIALIZED VIRTUALIZATION MODEL NODE HARDWARE

46 ROS/HRT setup 46 ROS vcore ROS vcore IPI IPI HRT vcore IPI HRT vcore IPI Virtual Machine

47 ROS/HRT setup 47 ROS vcore HRT vcore ROS vcore ROS visible memory HRT vcore VM memory Virtual Machine

48 merged address space 48 higher half ROS kernel higher half Aerokernel merged app + runtime code & data ROS vaddr space HRT vaddr space

49 ROS/HRT communication 49 shared data page Address LINUX space HRT HRT merger function reboot invocation request HRT HYPERVISOR + HVM

50 ROS/HRT communication 50 shared data page comm. area LINUX Synchronous operation request HRT HYPERVISOR + HVM

51 ROS/HRT communication 51 Cycles Time Addr space merge ~33K 15µs Asynch function invocation Synch function invocation (remote socket) Synch function invocation (same socket) ~25K 11µs ~ ns ~ ns

52 ROS boot 52 BOOTSTRAP ROS BSP INIT-SIPI-SIPI BOOTSTRAP INIT-SIPI-SIPI ROS APs BOOTSTRAP INIT-SIPI-SIPI

53 HRT boot 53 GDT IDT HVM sets boots up registers, cores up in TSS system long simultaneously mode* tables REGISTERS *no need for INIT-SIPI-SIPI sequence INIT PAGE TABLES

54 LINUX FORK + EXEC ~ 714µs HVM + HRT CORE BOOT ~ 61µs 54

55 55 how do we take a legacy runtime to the HRT + X model? PORT

56 porting a runtime/app 56 to an a new OS environment is DIFFICULT TIME-CONSUMING ERROR-PRONE

57 development cycle: 57 do { ADD FUNCTION REBUILD BOOT HRT } while (HRT falls over)

58 58 much of the functionality is NOT ON THE CRITICAL PATH

59 59 we want to make this easier

60 OUTLINE Background/Overview Nautilus Deployment Models Hybrid Virtual Machine Multiverse & Future Work 60

61 61 give us your legacy runtime

62 we automatically 62 transform it to run as an HRT in kernel mode bridged with a legacy OS (AUTOMATIC HYBRIDIZATION)

63 63 LEGACY APP/RUNTIME rebuild with our toolchain

64 64 AEROKERNEL BINARY runs like kernel-mode normal LEGACY APP/RUNTIME LINUX application MULTIVERSE RUNTIME LAYER LEGACY APP/RUNTIME MULTIVERSE RUNTIME LAYER BOOTED AEROKERNEL HYPERVISOR + HVM

65 65

66 66 4 parallel runtimes (Legion, Racket, NDPC, NESL) Hybrid Virtual Machine bridge HRT with legacy OS Multiverse automatic transformation: legacy app+runtime HRT

67 thank you 67

68 68 my webpage: halek.co lab: presciencelab.org download nautilus: nautilus.halek.co v3vee project: v3vee.org

69 backups 69

70 thread fork 70

71 interrupt driven execution 71

72 memory and paging 72

73 73 static identity map bootup... page size... 2MB physical memory 1GB

74 74 eliminates expensive page faults reduces TLB misses + shootdowns increases performance under virtualization

75 75 log scale Unified TLB Misses

76 76 Linux Nautilus Unified TLB Misses

77 77 thread creation is PREDICTABLE log scale lower is better 1e8 1e7 cycles 1e Linux (pthreads) kernel threads Nautilus x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

78 78 binding to physical processor is GUARANTEED

79 79 RUNTIME control over physical virtual CPU

80 80 interrupts & events

81 81 asynchronous notifications

82 82 trigger latency EVENT COMPLETES trigger DEPENDENT TASK EXECUTES

83 user-mode software events are SLOW 83 lower is better trigger latency (cycles) cond. var. futex hardware (IPI) x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM

84 nautilus events triggers are FASTER trigger latency (cycles) naut. cond. var. futex cond. var. x86_64 Opteron: 64 cores, 4 sockets, 8 numa zones, 128GB RAM naut. cond. var. w/ipi hardware (IPI)

85 85 handle_event() handle_event()

86 86 asynchronous, IPI-based execution

87 87 NOT POSSIBLE IN LINUX USERSPACE

88 lower is better 88 IPI Nemo TRIGGER LATENCY (CYCLES)

89 89 IPI Nemo TRIGGER LATENCY (CYCLES)

90 90 IPI ~100 cycles Nemo TRIGGER LATENCY (CYCLES)

91 91 this gives us an incremental path for creating HRTs

92 92 start out with a working HRT system

93 93 then pull functionality into the HRT for hot spots

94 94 RACKET most widely used Scheme implementation downloaded ~300 times/day complex runtime with JIT

95 ROS/HRT communication 95 AEROKERNEL BINARY LEGACY APP/RUNTIME MULTIVERSE RUNTIME LAYER LINUX system call system call LEGACY APP/RUNTIME MULTIVERSE RUNTIME LAYER BOOTED AEROKERNEL HYPERVISOR + HVM

96 96

97 Broadcast wakeups on x min = max = e+06 µ = σ = Cycles to Wakeup min = max = e+06 µ = σ = min = 3258 max = µ = σ = min = 7842 max = µ = σ = Nautilus events pthread condvar futex wakeup Aerokernel condvar Aerokernel condvar + IPI min = 1252 max = µ = σ = broadcast IPI

98 Wakeup deviation on x CDF pthread condvar futex broadcast Aerokernel condvar Aerokernel condvar + IPI broadcast IPI σ

99 Wakeup deviation on x user-mode events CDF HW lower bound Nemo events Nautilus (kernel-mode) events 2x 0.2 pthread condvar futex broadcast Aerokernel condvar Aerokernel condvar + IPI broadcast IPI σ

100 Broadcast wakeups on x min = max = e+06 µ = σ = Cycles to Wakeup min = max = e+06 µ = σ = pthread condvar futex wakeup min = 1252 max = µ = σ = broadcast IPI

101 Unicast IPIs on phi th percentile = 746 cycles CDF Cycles mesaured from BSP (core 224)

102 Roundtrip IPIs on phi th percentile = 2105 cycles CDF Cycles mesaured from BSP (core 224)

103 Multicast IPIs on phi th percentile σ = 31 cycles CDF σ

104 Wakeup deviation on x CDF pthread condvar futex broadcast broadcast IPI σ

105 Wakeup deviation on x HW lower bound user-mode events 0.6 CDF 0.4 ~70x 0.2 pthread condvar futex broadcast broadcast IPI σ

106 thread context switch on x Cycles min = 1352 max = 2438 µ = σ = min = 1176 max = 1391 µ = σ = Linux (pthreads) Aerokernel threads

107 thread create + launch on phi Cycles Linux (pthreads) Linux (kernel threads) Nautilus kernel threads

108 thread create + launch 108 (many threads) on x Linux pthreads Linux kernel threads Nautilus threads Cycles Threads

109 thread create + launch 109 (many threads) on phi Linux pthreads Linux kernel threads Nautilus threads Cycles Threads

110 Unicast IPIs on x th percentile = 1728 cycles CDF Cycles mesaured from BSP (core 0)

111 Roundtrip IPIs on x th percentile = 4640 cycles CDF Cycles mesaured from BSP (core 0)

112 Multicast IPIs on x th percentile σ = 2812 cycles CDF σ

113 Unicast IPI vs memory polling unicast IPI synchronous event (mem polling) CDF Cycles mesaured from BSP (core 0)

114 Unicast IPI vs memory polling unicast IPI synchronous event (mem polling) coherence network CDF interrupt network ipi sync cdf (this one Cycles mesaured from BSP (core 0)

115 Unicast IPI vs memory polling unicast IPI synchronous event (mem polling) projected remote syscall CDF Cycles mesaured from BSP (core 0)

116 Unicast IPI vs memory polling unicast IPI synchronous event (mem polling) projected remote syscall CDF cost of dest. handling Cycles mesaured from BSP (core 0)

117 Single wakeup on phi 117 min = max = µ = σ = min = max = µ = σ = min = 732 max = 761 µ = σ = pthread condvar futex wakeup unicast IPI

118 Wakeup deviation on phi CDF pthread condvar futex broadcast broadcast IPI σ wakeup phi many

119 Wakeup deviation on phi HW lower bound user-mode events 0.6 CDF 0.4 ~79000x 0.2 pthread condvar futex broadcast broadcast IPI σ

120 Single wakeup on x64 (CDF) CDF Cycles to Wakeup U. mode condvar Futex wakeup Oneway IPI

121 Single wakeup on phi (CDF) CDF Cycles to Wakeup U. mode condvar Futex wakeup Oneway IPI

122 Single wakeup on phi Cycles to Wakeup min = max = µ = σ = min = 5528 max = µ = σ = min = max = µ = σ = min = 5953 max = 7305 µ = σ = pthread condvar futex wakeup Aerokernel condvar Aerokernel condvar + IPI min = 732 max = 761 µ = σ = unicast IPI

123 Many wakeups on phi 123 Cycles to Wakeup pthread condvar min = max = e+07 µ = e+06 σ = e+06 futex wakeup min = max = e+06 µ = e+06 σ = e+06 Aerokernel condvar min = 7199 max = e+06 µ = e+06 σ = e+06 min = 8267 max = e+06 µ = σ = min = 1016 max = 1225 µ = σ = Aerokernel condvar + IPI broadcast IPI

124 Wakeup deviation on phi CDF 0.4 pthread condvar 0.2 futex broadcast Aerokernel condvar Aerokernel condvar + IPI broadcast IPI σ wakeup phi many wk

125 Wakeup deviation on phi CDF 0.6 HW lower bound user-mode events 4x 0.4 Nemo events (kernel-mode) pthread condvar 0.2 futex broadcast Aerokernel condvar Aerokernel condvar + IPI broadcast IPI σ

126 Broadcast wakeups on phi 126 min = max = e+07 µ = e+06 σ = e+06 min = max = e+06 µ = e+06 σ = e+06 min = 1016 max = 1225 µ = σ = pthread condvar futex wakeup broadcast IPI

127 Single wakeup on x64 (CDF) CDF Cycles to Wakeup U. mode condvar Futex wakeup K.mode condvar K.mode condvar + IPI Oneway IPI

128 Single wakeup on phi (CDF) CDF Cycles to Wakeup U. mode condvar Futex wakeup K.mode condvar K.mode condvar + IPI Oneway IPI

129 racket multiverse overheads Native Virtual Multiverse Runtime (s) fannkuch-redux binary-tree-2 fasta fasta-3 nbody spectral-norm mandelbrot-2

130 system call overheads 130 1x10 8 1x10 7 Virtual Multiverse Runtime (cycles) 1x ~40µs 100 getpid gettimeofday fwrite stat read getcwd open close mmap