Yet Another Implementation of CoRAM Memory

Size: px

Start display at page:

Download "Yet Another Implementation of CoRAM Memory"

Tyrone Curtis
5 years ago
Views:

Dec 7, 2013 CARL2013@Davis, CA Py Yet Another Implementation

Takamaeda-Yamazaki, Kenji Kise, James C.

1 Dec 7, 2013 CA Py Yet Another Implementation of Memory Architecture for Modern FPGA-based Computing Shinya Takamaeda-Yamazaki, Kenji Kise, James C. Hoe * Tokyo Institute of Technology JSPS Research Fellow * Carnegie Mellon University

2 Agenda Background Py Overview Py Microarchitecture Evaluation Conclusion Dec 7, 2013 Shinya T-Y. Tokyo Tech 2

3 Background Dec 7, 2013 Shinya T-Y. Tokyo Tech 3

VHDL Modeled in new modeling tool Bluespec, AutoESL, Chisel, DDRx DRAM interface PCI-express

4 FPGA as SoC Put together various components on a single FPGA CPU core Microblaze (Soft-macro) Cortex-A9 (Hard-macro) CPU Hardware accelerator logic Modeled in traditional RTL Verilog HDL, VHDL Modeled in new modeling tool Bluespec, AutoESL, Chisel, DDRx DRAM interface PCI-express Ethernet, t Ether FPGA HW Acc Interconnect t DRAM I/F HW Acc PCI-E Dec 7, 2013 Shinya T-Y. Tokyo Tech 4

5 Portability Issue of Application Design How to support various FPGA platforms? Different logic size, memory interface, peripherals and I/O property Digilent Atlys (Xilinx Spartan-6 LX45) ScalableCore System (our FPGA system) (Xilinx Spartan-6 LX node) Xilinx ML605 (Xilinx Virtex-6 LX240T) Dec 7, 2013 Shinya T-Y. Tokyo Tech 5

IP-core Based System Development To build a system, add IP-cores and

interconnect EDK automatically generates an on-chip interconnection and

6 IP-core Based System Development To build a system, add IP-cores and connect them IP-cores are connected through a standard on-chip interconnect EDK automatically generates an on-chip interconnection and (some) device-dependent interfaces IP-core List No (or few) annoying steps! FPGA IP-core Instances CPU HW Acc Interconnect HW Acc Interconnect Ether DRAM I/F PCI-E DRAM Xilinx Platform Studio (XPS) Dec 7, 2013 Shinya T-Y. Tokyo Tech 6

7 Abstract Memory System for FPGAs (Connected RAM) [FPGA 11] High-level abstraction for memory management Decoupling computing logics and memory access behaviors Memory access patterns in software model (C language) g Read/Write Communication FIFOs (Registers) Channel Abstracted On-chip Memories Read/Write HW Kernels (Computing Logics) Read Write Co oram Me emory Off-chip Memory Manage Control Threads (Memory Access Pattern in C) Dec 7, 2013 Shinya T-Y. Tokyo Tech 7

8 RTL conversion Tutorial From What 13 Core Logic SRAM Control thread programs Architecture Microarchitecture Control Logic Control Logic Fab bric Cluster DMA Fab bric Cluster DMA High-level synthesis from C to RTL using LLVM FPGA Network on Chip CONNECT Memory Translation (TLBs) Memory Interfaces and Caches NoC generator 6/19/2013 Tutorial 18 Dec 7, 2013 Shinya T-Y. Tokyo Tech 8

9 Py Overview Dec 7, 2013 Shinya T-Y. Tokyo Tech 9

10 Motivation: for EDK Integration of memory architecture for modern EDK-based development flow with standard IP-cores Portable application design with Cooperation with standard IP-cores Accelerator logic Abstraction Standard IP-core Standard On-chip Interconnect CPU core Device-dependent Interfaces Dec 7, 2013 Shinya T-Y. Tokyo Tech 10

11 Py Python-based implementation of memory architecture for modern FPGA EDKs memory abstraction for EDK development flow Key features Control Thread in Python We developed Python-to-Verilog HLS Compiler from scratch AMBA AXI4 Interconnect for on-chip interconnect For IP-core based development on Xilinx Platform Studio (XPS) Parameterized RTL Design Support for User-logic Generate-statement and Parameter-statement analyzed by our original Verilog analysis tool-chain (Pyverilog) Dec 7, 2013 Shinya T-Y. Tokyo Tech 11

12 Comparison with Original Py Language for Control-Thread C Python Supported (Blocking/Non-Blocking) (Blocking/Non-Blocking) Memory Operations Read/Write Read/Write On-chip Interconnect CONNECT NoC [FPGA 12] AMBA AXI4 FSM Granularity in Control Thread Generate Statement Support for User logics Supported FPGAs # Lines of Code LLVM-IR No Xilinx ML605 Altera Terasic DE-4 11,682 lines (w/o CONNECT) Python AST Node Yes Any FPGAs supporting AXI Bus 4,922 lines (w/o Pyverilog) FSM: Finite State Machine LLVM-IR: Low Level Virtual Machine Intermediate Representation AST: Abstract Syntax Tree Dec 7, 2013 Shinya T-Y. Tokyo Tech 12

13 Py Development Flow Py generates an IP-core package from user-logic RTLs and control thread scripts in Python Each part can be replaced with the original s component User-logic (Verilog HDL) Control Threads (Python) RTL Conversion IP-core generation with AXI4 Interface Top design synthesis with AXI4 Logic Control IP-core Hierarchy Signal Integration Analysis Insertion on EDK Python-to- Verilog Compilation Control Signal Port Addition IP-core Packing (RTL,.mpd, and.pao) Synthesis FPGA Bit File Portable Application Design Python-to-Verilog HLS Py Tool-chain Vendor EDA Flow Dec 7, 2013 Shinya T-Y. Tokyo Tech 13

14 FPGA Accelerator with Py IP-core FPGA Py IP (Application) HW Kernels (Computing Logics) Channel Contro l Thread Other AXI IP-core or CPU Memory DMA Cluster DMAC Memory Memory DMA Cluster DMAC Memory Stream DMAC Stream DMAC FSM AXI I/F AXI I/F AXI I/F AXI I/F AXI4 Interconnect DRAM Controller DRAM (Off-chip) Dec 7, 2013 Shinya T-Y. Tokyo Tech 14

15 Py Microarchitecture Dec 7, 2013 Shinya T-Y. Tokyo Tech 15

16 Py Microarchitecture (Logical View) GPIO User User Logic Contro I/O l Register Thread Channel Memory DMAC Stream DMAC FSM Dec 7, 2013 Shinya T-Y. Tokyo Tech 16

17 Py Microarchitecture (Logical View) GPIO Modeled in RTL (Verilog HDL) User User Logic Contro I/O l Register Thread Channel Memory Access Pattern in Python Memory DMAC Stream DMAC FSM Dec 7, 2013 Shinya T-Y. Tokyo Tech 17

18 Py Microarchitecture (Physical View) GPIO Py IP User User Logic Contro I/O l Register Thread Channel Memory DMAC Stream DMAC FSM AXI I/F AXI I/F AXI4 Interconnect FPGA DRAM Controller Dec 7, 2013 Shinya T-Y. Tokyo Tech 18

19 Py Microarchitecture (Physical View) GPIO Parameterized RTL Design Support Py IP User User Logic Contro I/O l Register Thread Memory Channel Stream FSM Control Thread in Python DMAC DMAC AXI I/F AXI I/F AXI4 Master Interface AXI4 Interconnect FPGA DRAM Controller Dec 7, 2013 Shinya T-Y. Tokyo Tech 19

20 Control Thread in Python Operations for objects To/from Memory Data movement pattern with DMA operations between on-chip memory and DRAM To/from Channel Token communication action between user-logic and control thread 0 def calc_sum(times): 1 ram = CoramMemory(idx=0, datawidth=32, size=1024) 2 channel = CoramChannel(idx=0, datawidth=32) 3 addr = 0 4 sum = 0 5 for i in range(times): ram.write(0, addr, 128) channel.write(addr) sum += channel.read() addr += 128 * (32/8) print( sum=, sum) calc_sum(8) User I/O Memory DMAC User Logic # Transfer (off-chip DRAM to BRAM) # Notification to User-logic # Wait for Notification from User-logic # $display Verilog system task Dec 7, 2013 Shinya T-Y. Tokyo Tech 20 Channel Control Thread FSM

21 objects in User Logic objects as standard BRAM or FIFO Very similar interface to the standard memory components User-logic can use their contents in them in the same way Essential parameters to define object characteristics Thread name, ID, data width, address length, CoramMemory1P #(.CORAM_THREAD_NAME( NAME("thread_name name") ),.CORAM_ID(0),.CORAM_ADDR_LEN(ADDR_LEN),.CORAM_DATA_WIDTH(DATA_WIDTH) ) inst_memory (.CLK(CLK),.ADDR(mem_addr),.D(mem_d),.WE(mem _ we),.q(mem_q) ); CoramChannel #(.CORAM_THREAD_NAME("thread_name"),.CORAM_ID(0),.CORAM_ADDR_LEN(CHANNEL_ADDR_LEN),.CORAM_DATA_WIDTH(CHANNEL_DATA_WIDTH) WIDTH) ) inst_channel (.CLK(CLK),.RST(RST),.D(comm_d),.ENQ(comm_enq),.FULL(comm_full),.Q(comm_q),.DEQ(comm _ deq),.empty(comm_empty) ); (a) Memory (b) Channel Dec 7, 2013 Shinya T-Y. Tokyo Tech 21

22 AXI4 Master Interface DMA controller works as AXI4 master IP-core interface HW Kernels (Computing Logic) Addr WrData RdData WrEn Addr WrData RdData WrEn Empty Deque RdData AlmFull Enque WrData Memory (BRAM) Memory (BRAM) DMA Cluster Channel Addr WrData RdData WrEn Addr WrData RdData WrEn Empty Deque RdData AlmFull Enque WrData DMA Controller BramAddr DramAddr Size Control Thread WrEn RdEn Busy Ready FSM Empty Deque RdData AlmFull Enque WrData Ready RdEn RdEn Size Addr AXI Master Interface (Protocol Conversion) AWREADY AWVALID AWLEN AWADDR WREADY WVALID WDATA BVALID ARREADY ARVALID ARLEN ARADDR RVALID RREADY RDATA Write Address Channel Write Data Channel Read Address Channel AXI4 Interconnect Read Data Channel Dec 7, 2013 Shinya T-Y. Tokyo Tech 22

23 For Parameterized RTL design support Generate-statement support by advanced RTL analyzer Not supported by the original compiler Pyverilog: Python-based Tool-chain for Verilog HDL Design Parser Dataflow Analysis Optimization RTL Code Generation Control flow Analysis Graphical Output Dataflow State Machine Dec 7, 2013 Shinya T-Y. Tokyo Tech 23

24 Evaluation Dec 7, 2013 Shinya T-Y. Tokyo Tech 24

Evaluation Point: Maximum memory bandwidth utilization Py is a memory

DDR2-800 DRAM 128MB (1.2GB/s*) AXI4 128-bit, 100MHz (1.

512MB (6.4GB/s) AXI4 256-bit, 200MHz (6.4GB/s) Xilinx Platform Studio (14.

25 Evaluation Point: Maximum memory bandwidth utilization Py is a memory abstraction framework Setup 2 FPGA boards EDK Digilent Atlys Spartan-6 LX45 DDR2-800 DRAM 128MB (1.2GB/s*) AXI4 128-bit, 100MHz (1.6GB/s) Xilinx ML605 Virtex-6 LX240T *Due to 300MHz operation DDR3-800 DRAM 512MB (6.4GB/s) AXI4 256-bit, 200MHz (6.4GB/s) Xilinx Platform Studio (14.6) Digilent Atlys (Xilinx Spartan-6 LX45) Xilinx ML605 (Xilinx Virtex-6 LX240T) Dec 7, 2013 Shinya T-Y. Tokyo Tech 25

26 Evaluation: Application Array-sum: calculate summation value of an array Two memories as Double-buffered Varying SIMD width (=# simultaneous ops) to check the effect to the memory bandwidth utilization 4, 8, 16, 32, 64 (bytes) sum + Output MUX s3 s2 s1 s D[3] D[2] D[1] D[0] MUX Memory 0 D[3 ] D[2 ] D[1 ] D[0 ] D[3 ] D[2 ] D[1 ] D[0 ] Memory 1 from DMA Controller 0 from DMA Controller 1 Dec 7, 2013 Shinya T-Y. Tokyo Tech 26

27 Memory Bandwidth Utilization Good bandwidth utilization Atlys: 85.5% (at 16-byte) ML605: 84.9% (at 64-byte) Degradation reasons Sequential (single) transaction for each DMA controller Memory latency directly affects the performance adversely Bandwidth Utilization Atlys (Spartan-6) SIMD size [byte] Bandwidth Utilization ML605 (Virtex-6) SIMD size [byte] Dec 7, 2013 Shinya T-Y. Tokyo Tech 27

28 Conclusion and Dec 7, 2013 Shinya T-Y. Tokyo Tech 28

29 Conclusion Py: Python-based implementation of memory architecture for modern FPGA EDKs Future work Further evaluation on more realistic applications AXI4 slave feature for control thread Tutorial slide Portable application design with Cooperation with standard IP-cores Accelerator logic Abstraction Standard IP-core CPU core Standard On-chip Interconnect Device-dependent Interfaces Automatically managed by EDK Dec 7, 2013 Shinya T-Y. Tokyo Tech 29

30 Py and Pyverilog are ready for public! Py (0.7.0-public) b / /P CRAM Pyverilog (0.6.0-public) 0-public) g y g Thanks! Dec 7, 2013 Shinya T-Y. Tokyo Tech 30

PyCoRAM: Yet Another Implementation of CoRAM Memory Architecture for Modern FPGA-based Computing

PyCoRAM: Yet Another Implementation of CoRAM Memory Architecture for Modern FPGA-based Computing Py: Yet Another Implementation of Memory Architecture for Modern FPGA-based Computing Shinya Kenji Kise Takamaeda-Yamazaki Tokyo Institute of Technology Tokyo Institute of Technology Tokyo, Japan 152-8552