PCIe interface firmware and software

Transcription

1 PCIe interface firmware and software Filippo Costa Sanjoy Mukherjee Tuan Mate Nguyen ALICE -O 2 CERN

2 OUTLINE PCIe interface : ) data flow 2) control interface 3) DCS interface (SC and monitoring) 2

3 PCIe FIRMWARE, based on ALTERA DMA ) x DMA channel - PHYSICS data will be moved over this channel 2) PCIe BAR access - this will allow the communication and configuration with all the blocks implemented in the firmware 3

4 PCIe data flow, general firmware -HDL - -HDL - -HDL - 2 -HDL - 23 DATA MOVER + LOAD BALANCE PCIE PCIE 4

5 PCIe data flow ( st approach), PCIe -HDL - -HDL - -HDL - 2 data mover PCIE -HDL

6 Actual implementation (simulation and hw) It has been tested on C-RORC (called G-RORC) as the interface for the ARRIAX changed again, but the idea works for both implementation receiver receiver framer framer load balance PCIe FIFO 2 4 MHz small FIFO MHz bigger FIFO (each one can store 4 Kb per ) TDT ARRIA X : it is MHz 6

7 Some simulation case studies (general firmware) receiver receiver framer framer load balance PCIe FIFO Development started with 2 (actually 4) receivers and changing the read clock frequency of the PCIe FIFO to simulate different cases Release of the busy once the buffers are empty I send out one buffer after the other 7

8 Some real case studies (TOF) receiver receiver framer framer load balance PCIe FIFO I have implemented in hardware the TOF readout configuration (for their tests) sending events with 32 KB size The system can read-out the event at 25 KHz (using current DATE system) 8

9 PCIe data flow ( st approach) trigger mode Data from each is buffered for every trigger and the data is sent over the PCIe one buffer after the other -HDL - -HDL - -HDL - 2 data mover PCIE -HDL

10 PCIe data flow ( st approach) continuous mode Similar to trigger mode, but using the DROP schema instead the BUSY -HDL - -HDL - -HDL - 2 data mover PCIE -HDL

11 PCIe data flow ( st approach), 2 x PCIe -HDL - -HDL - -HDL - 2 data mover load balance PCIE -HDL PCIE

12 Control over PCIe Different range of registers to configure: ) logic: - DMA control - registers - TTC interface - DCS block - 2) Detector specific logic: 3) FEE configuration and monitoring 2

13 Control over PCIe Software tools: - _READ_REG xadd - _WRITE_REG xadd xvalue - _WRITE_BUFFER xstart_add BUFFER (32 bit word) 3

14 Software interface PDA (Portable Driver Architecture) - PDA: a library for programming microdrivers - Move large parts of a device driver code to the user space without loss of speed - The device driver code is compatible with at least 28 kernel releases, easier to maintain 4

15 DMA with PDA: allocating memory buffer - dop = DeviceOperator_new( pci_ids ); - DeviceOperator *dop; - DeviceOperator_getPciDevice(dop, &device, ) - PciDevice *device - Protected struct - PciDevice_allocDMABuffer(device, i, dma_buffer_size, &buffer) - DMABuffer *buffer - Protected struct - PciDevice_getDMABuffer(device, i, &buffer) - DMABuffer_getSGList(buffer, &sglist) - DMABuffer_SGNode *sglist - Protected struct 5

16 DMA with PDA: mapping the BARs - PciDevice_getBar(device, &bar, i) - Bar *bar; - i-th BAR - Bar_getMap(bar, (void**)&bar_address, &length) All the software and tests will are done in SLC6, but we are moving the system in CENTOS7 (next CERN OS release) PDA just installed and tested and it is working, we will continue testing the system in the new OS 6

17 Software tools to test the hardware and the links FEE versatile link Different data loopback: ) Inside the (test only PCIe), 2) Inside up to the (test the data flow inside the ) 3) Up to the FEE (test the link and the installed in the FEE if possible using the chip) 3 DONE WILL BE DONE POSSIBLE? 2 7

18 Software tools to test the hardware and the links FIFO data generator PCIE DRIVER SW the software check the data consistency on both channels Done to exercise the data taking + dcs monitoring data 8

19 DCS interface FLP -SCA controller PCIe DRIVER SW DIM Firmware simulation is almost done a few weeks work to finalize it and do first test in hw Software part, first prototype in place, DIM services have been developed DCS - FRAME WORK 9