AND THEN THERE WERE NONE

Transcription

1 ND THEN THERE WERE NONE Stall-Free Real-Time Garbage Collector for Reconfigurable Hardware David F. acon Perry Cheng Sunil Shukla IM Research

2 IMPLEMENTING PROGRMMING LNGUGE Program Source Code Interpreter Instruction Set Processor Circuit

3 IMPLEMENTING PROGRMMING LNGUGE Program Program Compiler Compiler Source Code Interpreter Machine Code Instruction Set Processor Instruction Set Interpreter Circuit Circuit

4 IMPLEMENTING PROGRMMING LNGUGE Program Program Program Compiler Compiler Hardware Compiler Source Code Interpreter Machine Code Instruction Set Processor Instruction Set Interpreter Circuit Circuit Circuit Layout Circuit

5 PROGRMMING RECONFIGURLE HRDWRE (FPGS) Programmed at very low level of abstraction same as designing custom circuits (SICs) Verilog, VHDL prevail: bits and bit arrays are main abstraction

6 PROGRMMING RECONFIGURLE HRDWRE (FPGS) Programmed at very low level of abstraction same as designing custom circuits (SICs) Verilog, VHDL prevail: bits and bit arrays are main abstraction HIGH LEVEL LNGUGE

7 PROGRMMING RECONFIGURLE HRDWRE (FPGS) Programmed at very low level of abstraction same as designing custom circuits (SICs) Verilog, VHDL prevail: bits and bit arrays are main abstraction HIGH LEVEL LNGUGE GRGE COLLECTION

8 SYSTEM = PPLICTION + COLLECTOR HND-WRITTEN HDL COLLECTOR & MEMORY

9 RECONFIGURLE HRDWRE CKGROUND

10 CONFIGURLE LOGIC UP TO 300K SLICES = 2.4M FLIP-FLOPS

11 PROGRMMLE ROUTING NETWORK SOURCE: WIKIMEDI (CC) 2007

12 LOCK-RM MEMORIES (RMS) R/W ddress Data In Data Out

13 LOCK-RM MEMORIES (RMS) R/W ddress Data In Data Out R/W ddress Data In Data Out

14 LOCK-RM MEMORIES (RMS) R/W R/W ddress Data Out ddress Data In Data Out Data In R/W ddress Data In Data Out R/W ddress Data In Data Out 36 KIT 36K X 1 18K X 2 1K X RM OR FIFO

15 LOCK-RM MEMORIES (RMS) R/W R/W ddress Data Out ddress Data In Data Out Data In R/W ddress Data In Data Out R/W ddress Data Out Data In

16 R/W ddress Data In Data Out R/W ddress Data In Data Out R/W ddress Data In R/W Data Out R/W ddress Data In Data Out ddress Data In R/W ddress Data In Data Out Data Out

17 R/W ddress Data In Data Out R/W ddress Data Out Data In R/W ddress Data In Data Out R/W ddress Data In Data Out

18 WHT WE UILT

19 COLLECTOR IN HRDWRE FOR HRDWRE

20 COLLECTOR IN HRDWRE FOR HRDWRE Complete garbage collector NOT hardware-assist instructions (eg zul, Lisp Machine) For on-chip FPG memory NOT for large, general-purpose CPU DRM With fixed object geometry (2 pointers + data) NOT for arbitrarily sized/shaped objects Snapshot-at-the-eginning lgorithm [Yuasa 1990]

21 lloc ddr lloc d ddr to Read/Write Pointer to Write Pointer Value Memory Subsystem

22 lloc ddr lloc d ddr to Read/Write Pointer to Write Pointer Value llocator Mark Engine Memory Subsystem Sweep Engine Memory

23 lloc ddr lloc d ddr to Read/Write Pointer to Write Pointer Value llocator Mark Engine Memory Subsystem Sweep Engine Memory GC Snapshot Engine ROOT

24 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

25 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

26 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

27 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

28 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

29 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear 5 Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

30 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear 5 Stack Top Free Stack Pointer Memory ddress llocated MLLOCTOR (INCL. 1 MEMORY COLUMN )

31 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated WRITING (POINTER) VLUE

32 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated WRITING (POINTER) VLUE

33 lloc ddr to Free ddr to Read/Write Pointer to Write Pointer Value ddr lloc d ddress to Clear Stack Top Free Stack Pointer Memory ddress llocated WRITING (POINTER) VLUE

34 THE TRCE ENGINE 3 OPERTIONS (a) Get a root pointer and mark it (b) Deque a pointer from mark queue and mark it (c) Perform write barrier and mark overwritten pointer

35 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue Pointer Memory MUX Pointer to Trace

36 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg 1 Mark Queue MUX Pointer Memory MUX 3 Pointer to Trace

37 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue Pointer Memory MUX 3 3 Pointer to Trace

38 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue Pointer Memory MUX 3 3 Pointer to Trace

39 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 3 Pointer Memory MUX 3 Pointer to Trace

40 (a) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 3 Pointer Memory MUX 3 Pointer to Trace

41 (b) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 3 Pointer Memory MUX Pointer to Trace

42 (b) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 3 1 Mark Queue Pointer Memory MUX Pointer to Trace

43 (b) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue Pointer Memory MUX 5 5 Pointer to Trace

44 (b) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 5 Pointer Memory MUX 5 Pointer to Trace

45 (c) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 5 Pointer Memory MUX Pointer to Trace

46 (c) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd 3 Mark Map arrier Reg MUX 1 Mark Queue 5 Pointer Memory MUX Pointer to Trace

47 (c) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd 3 Mark Map arrier Reg MUX 1 Mark Queue 5 Pointer Memory MUX Pointer to Trace

48 (c) ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map arrier Reg MUX 1 Mark Queue 5 Pointer Memory MUX Pointer to Trace

49 RESULTS

50 EVLUTE 3 SYSTEMS (a) Malloc llocator Memory (b) Stop-the-World GC llocator Sweep Engine Mark Engine Memory (c) Real-Time GC llocator Sweep Engine Mark Engine Memory Snapshot Engine

51 EVLUTE SYSTEMS IN 3 CONTEXTS COLLECTOR & MEMORY (a) Collector in isolation (no application)

52 EVLUTE SYSTEMS IN 3 CONTEXTS INRY TREE (HND-WRITTEN HDL) COLLECTOR & MEMORY (a) Collector in isolation (no application) (b) Collector with inary Tree benchmark

53 EVLUTE SYSTEMS IN 3 CONTEXTS DEQUEUE INRY TREE (HND-WRITTEN HDL) HDL) COLLECTOR & MEMORY (a) Collector in isolation (no application) (b) Collector with inary Tree benchmark (c) Collector with Double-ended Queue benchmark

54 LOGIC (SLICE) USGE - NO PPLICTION Xilinx Virtex-5 LX330T 51,840 Slices Tiny fraction of chip STW almost as complex as RTGC

55 SYNTHESIZED CLOCK FREQUENCY - NO PPLICTION Frequency goes down with design complexity Malloc is faster, but advantage narrows

56 EXECUTION TIME - DEQUEUE

57 EXECUTION TIME - DEQUEUE RTGC uniformly faster than STW Malloc is faster, but not by that much (almost even for inary Tree)

58 CONCLUSIONS First complete garbage collector in hardware First garbage collector that NEVER pauses mutator Greatly expands expressiveness of hardware programs RTGC is faster, smaller, and cooler than STW RTGC in hardware is MUCH SIMPLER than in software Is something wrong with our processor designs?

59 Questions?

60 Questions? Suggestions: You only have 2 microbenchmarks. Isn t that bogus? Isn t a fixed object layout totally bogus? Can determinism be preserved with a more complex heap? Could this technique be applied to general-purpose systems? I don t believe you never stall. Do you have a proof? Don t you lose performance by reserving one of the ports? What unique hardware features made stall-freedom possible?

61 CKUP

62 ROLE OF THE GRGE COLLECTOR COLLECTOR & MEMORY

63 ROLE OF THE GRGE COLLECTOR PPLICTION COLLECTOR & MEMORY

64 ROLE OF THE GRGE COLLECTOR PPLICTION HND-WRITTEN HDL COLLECTOR & MEMORY

65 ROLE OF THE GRGE COLLECTOR PPLICTION LIME TSK HND-WRITTEN HDL COLLECTOR & MEMORY

66 PIPELINES IN THE LIME LNGUGE var pipeline = task worker1 => task worker2 => task worker3; port-to-stream connection port-to-stream connection char char int int[[5]] worker1( ) { } worker2( ) { } worker3( ) { } compound filter

67 PIPELINES IN THE LIME LNGUGE var pipeline = task worker1 => task worker2 => task worker3; port-to-stream connection port-to-stream connection char char int int[[5]] worker1( ) { } worker2( ) { } worker3( ) { } compound filter

68 GRGE COLLECTING LIME TSKS var pipeline = task worker1 => task worker2 => task worker3; char char int int[[5]] worker1( ) { } worker2( ) { } worker3( ) { }

69 GRGE COLLECTING LIME TSKS var pipeline = task worker1 => task worker2 => task worker3; char char int int[[5]] worker1( ) { } worker2( ) { } worker3( ) { }

70 W_EN DT_IN DT_OUT DT_IN W_EN 101 Mutator Register DT_OUT REGISTER MODULE

71 W_EN DT_IN DT_OUT GC ROOT_OUT W_EN DT_IN DT_IN W_EN Mutator Register DT_OUT Shadow Register DT_OUT REGISTER MODULE + SNPSHOT COMPONENT

72 W_EN DT_IN DT_OUT GC ROOT_OUT 101 W_EN DT_IN DT_IN 101 W_EN Mutator Register DT_OUT Shadow Register DT_OUT REGISTER MODULE + SNPSHOT COMPONENT

73 W_EN DT_IN DT_OUT GC ROOT_OUT 101 W_EN DT_IN DT_IN W_EN Mutator Register DT_OUT Shadow Register DT_OUT REGISTER MODULE + SNPSHOT COMPONENT

74 GC Push/Pop Push Value Pop Value Root to dd Write Reg Read Reg Stack Top MUX Shadow Register Scan Pointer Mutator Stack Mutator Register

75 ddr to Clear ddr to Read/Write Pointer to Write Pointer Value Root to dd Mark Map 000 arrier Reg MUX 1 Mark Queue Pointer Memory MUX Pointer to Trace

76 lloc GC ddr lloc d ddr to Clear ddress to Free =10? Stack Top Free Stack Used Map MUX ddress llocated Sweep Pointer Mark Map

77 GC W_EN PUSH DT_IN (PUSH) DT_OUT (POP) ROOT_OUT 1 DT_IN_ +/- W_EN_ DDR_IN_ DT_OUT_ DT_IN W_EN 101 Top of Stack 1 - MUX 0 W_EN_ DDR_IN_ DT_OUT_ MUX Mutator Stack State Machine 101 Scan Index

78 ENLERS FOR STLL-FREEDOM Dual-ported Memory Read-before-Write Memory and Registers Simple, uniquitous synchronization (clock edge) Forward reasoning about remote states (clock cycles) Determinism

79 EXECUTION TIME IN CYCLES - DEQUEUE

80 EXECUTION TIME IN CYCLES - DEQUEUE STW burns cycles while stopping the world Malloc pays (a little) for explicit free operation Malloc can run in a smaller heap (but not as bad as software)

81 FIELD PROGRMMLE GTE RRYS

82 FIELD PROGRMMLE GTE RRYS

83 FIELD PROGRMMLE GTE RRYS

84 FIELD PROGRMMLE GTE RRYS

85 FIELD PROGRMMLE GTE RRYS IO

86 CPU ackend GPU ackend Node ackend Verilog ackend bytecode binary binary bitfile CPU GPU PowerEN FPG

87 THE LIQUID METL PROGRMMING LNGUGE Lime CPU ackend GPU ackend Node ackend Verilog ackend Lime Compiler bytecode binary binary bitfile CPU GPU PowerEN FPG

88 EXECUTION, COMMUNICTION, ND REPLCEMENT LVM

89 EXECUTION, COMMUNICTION, ND REPLCEMENT LVM

90 EXECUTION, COMMUNICTION, ND REPLCEMENT LVM

91 EXECUTION, COMMUNICTION, ND REPLCEMENT LVM

92 STTEFUL TSKS var averager = task verager().avg; instance variables (local state) primitive filter double double total; long count; double double avg(double x) { total += x; return total/++count; }

93 VIRTULIZTION OF DT MOVEMENT =>

94 INTERPRETTION VERSUS COMPILTION PROGRM getfield invokevirtual INTERPRETER MOV LR INSTRUCTION SET PROCESSOR

95 INTERPRETTION VERSUS COMPILTION PROGRM getfield invokevirtual INTERPRETER MOV LR INSTRUCTION SET PROCESSOR

96 GRGE COLLECTION Frees programmer from managing memory Simpler interfaces, easier debugging, memory safety Invented 1960 for IM 704 with 18K Current large FPGs have memory commensurate with a VX 11/780 Recent results: We built a garbage collector for data in on-chip RMs ble to handle a memory op each cycle without ever stalling Cost in slices and energy is ~0; cost in frequency and RM is small lgorithmically simpler than SW GC, yet achieves vastly better results Potential game-changer in scope of synthesizable code