27 March 2018 Mikael Arguedas and Morgan Quigley

Transcription

1 27 March 2018 Mikael Arguedas and Morgan Quigley

2

3

4

5

6

7

8 Separate devices: (prototypes 0-3) Unified camera: (prototypes 4-5) Unified system: (prototypes 6+) USB3 USB Host USB3 USB2 USB3 USB Host PCIe root PCIe Camera Camera IMU FPGA Imager IMU Imager FPGA Imager Imager IMU

9

10 Global-shutter 1.3 MPix imagers, 20cm baseline FPGA+DRAM+USB3 on daughterboard InertialSense µimu-2

11 DRAM 40 Imagers Imagers 30 / FPGA Imagers capable of ~2 Gbit pixel rate (each) DRAM buffer required since USB3 = 4Gbit - scheduling Artix-7: MHz DDR = ~13 Gbit - overhead 40 / USB3 PHY 8 / Any PC USB3-based design PCIe-based design Imagers Imagers 30 / PCIe x2 bandwidth is similar to (low-end) FPGA DRAM bus! FPGA PCIe PHY PCIe root on SBC 12 (NVIDIA TX2) / PCIe Gen 2 x2 = 8 Gbit full-duplex

12 Designed around TX2 FPGA is PCIe endpoint Self-contained computer vision: "just supply power"

13

14

15 Imager active area is not centered Use 3d-printed lens holders PLA is OK. Carbon-fiber is better Heat-set inserts for mounting + lens lock

16 Stereo systems need to be very stiff PCB is clamped to carbon-fiber tube

17

18 always-on MCU waits for TX2 boot During TX2 boot: MCU loads stage-1 FPGA image TX2-FPGA PCIe link established TX2 loads stage-2 FPGA image over PCIe sensors initialized over PCIe MMIO UART TX2 MCU SPI PCIe FPGA IMU Imager Imager

19 FPGA IMU sync PCIe decimate DRAM Image timing via IMU sync Image sensors TX2 PCIe trigger DMA write arbiter

20 FPGA PCIe DMA arbiter FIFO image sensors deserialize decode framing fix column ordering TX2 PCIe DRAM

21 Extreme close-up of typical indoor navigation feature (sprinkler pipe joint)

22 7x7 circle corner

23 7x7 circle (discretized) corner

24 7x7 circle (discretized) corner "unrolled" discretized circle

25 "unrolled" circle subtract center threshold 1) Find (in parallel) if there is a contiguous sequence of >= 9 pixels above threshold value. 2) For non-max suppression, find (in parallel) "how far" the sequence is above the threshold

26 Imagers produce 4 pixels per clock. Solution: search in parallel.

27 An example of FPGA reducing latency in (simple) pixel-wise operations 100's of operations per clock: 8-bit subtractions, comparisons, etc Deterministic timing, keeps up with pixel rate Many other algorithms are FPGA-friendly: pyramids, gradients,...

28 FPGA SPI IMU SPI sequencer PCIe IMU BRAM TX2 PCIe sync decimate control register BRAM DRAM CPU Image sensors DMA write arbiter trigger SPI clock sync data / 4 deserializer serializers pixel array ADC register file link train image image and c decoders pixel pixel FIFOs FIFOs pixel corner FIFOs detectors image stats GPU

29

30 Initialization allocate PCIe-visible RAM block configure FPGA core, imager SPI registers, IMU registers Every frame FPGA writes pixels via DMA to TX2 RAM, sends interrupt kernel re-syncs CPU caches kernel unblocks userland thread in ROS node ROS node copies image into ROS message, sends it downstream DMA Imager Imagers FPGA PCIe MSI TX2 RAM kernel module ROS driver image image image consumer consumer consumer nodes nodes nodes

31 Dynamic distributed message-passing framework Huge collection of open-source nodes Tools to parameterize, configure, and debug nodes sensor hardware actuator hardware

32

33 camera camera driver evil data sniffing prevented by encryption vision node publish evil image data evil node prevented by authorization downstream nodes

34

35

36

37 PCIe root SSD GbE USB TX2 (Tegra) SoC System Fabric Flash GPU USB (etc) "Traditional" system all peripherals on PCIe PCIe cannot reset / re-enumerate PCIe devices ready within 100ms GbE USB GPU USB PCIe root FPGA TX2-based system only the FPGA hangs off PCIe PCIe kernel driver can be reloaded FPGA configure/reconfigure at any time elaborate "fast" configuration not needed

38 all connectors on same side no configuration MCU FPGA upgrade (?) stack boards to reduce footprint

39

40 For more information: Morgan Quigley