BES-III off-detector readout electronics for the GEM detector: an update

Transcription

1 BES-III off-detector readout electronics for the GEM detector: an update The CGEM off-detector collaboration ( INFN/Univ. FE, INFN LNF, Univ. Uppsala ) 1

2 Outline Reminder Update on development status Off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card VME-based data collector card (GEM-DC) Off-detector readout system (GEM-ODR) WBS update 2

3 Reminder: proposed system layout BES-III trigger / timing system 10x off-detector Low Voltage readout cards crate (GEM-ROC) BES-III spectrometer 10x off-detector readout cards (GEM-ROC) GEM detector ASICs Low Voltage crate GbEthernet port for monitoring and standalone debugging Amphenol micro-ribbon twist&flat (17 pairs) ~ length: 10m patch panel ~ length: 10m ASICs Core VDD ASICs IO VDD GND 2x VME (6U) data collector cards (GEM-DC) 3 VME based BES-III DAQ system bi-directional fiber optic links for ASIC configuration and readout

4 Reminder: hardware locations GbEthernet port for monitoring and stand-alone DAQ VME based BES-III DAQ system data collector cards (GEM-DC) 2 x VME 6U units bi-directional fiber optic links for ASIC configuration and readout?? Low Voltage crate GEM-ROC 10 units?? GEM-ROC 10 units patch panel GEM ROC VCC ASICs Core VDD ASICs IO VDD GND? ASICs carrier PCBs 4

5 Design baseline for off-detector electronics: off-detector readout card (GEM-ROC) Reminder: trigger/timing connectors Finisar SFP transceiver KEL KEL KEL KEL LVDS / CMOS LVDS / CMOS LVDS / CMOS LVDS / CMOS Board to Board HSMC connector ALTERA ArriaV GX development kit GbE port Each board handles 4 ASIC PCBs <-> 8 ASICs The unit is based on an ALTERA development board for an ArriaVGX FPGA coupled to the GEM-ROC motherboard through an HSMC high performance connector. The motherboard provides the electrical and physical interfaces to the ASIC carrier PCBs and to the GEM-DC (Data Collectors). GEM-ROC mother board 5

6 Update on development status: off-detector readout card (GEM-ROC) Block diagram of the GEM-ROC FPGA design BEING TESTED ON ARRIA V GX DEVELOPMENT KIT Fast Control Interface module 2Gbps transceiver Data Collector Interface module Clock generator SIMULATION OK EMULATION OK SIMULATION OK EMULATION OK SIMULATION OK TESTED IN VIVO OK (1 INSTANCE) transceiver Rst/Cfg 4x ASIC Cfg Link 4x ASIC Read In Link GbE Port NIOS-II (soft) microprocessor Latency buffer for data (internal RAM) BES-III Specific HDL modules SIMULATION OK BEING TESTED in vivo TESTED IN VIVO OK TESTED IN VIVO OK Stefano s Ethernet MAC SDRAM DDR3 BUFFER High bandwidth link 6

7 Update on development status: off-detector readout card (GEM-ROC) Current status of top level of the GEM-ROC FPGA design High performance Ethernet MAC developed by Stefano Chiozzi (INFN-Ferrara) has been implemented in the target ARRIA V FPGA. It features: an 8 bit port toward the NIOS-II processor for the exchange of low bandwidth configuration and status information with the host PC a port of configurable width (set to 64 bit) toward the BES-III specific HDL modules for high bandwidth transmission of CGEM data in stand-alone mode for debugging / monitoring purposes GbE Port NIOS-II (soft) microprocessor TESTED in vivo: OK Stefano s Ethernet MAC on-chip memory High bandwidth port 7

8 Update on development status: off-detector readout card (GEM-ROC) Block diagram of the GEM-ROC FPGA design Fast Control Interface module 2Gbps transceiver Data Collector Interface module Clock generator SIMULATION OK EMULATION OK SIMULATION OK EMULATION OK SIMULATION OK TESTED IN VIVO OK (1 INSTANCE) transceiver Rst/Cfg 1 x ASIC Cfg Link 1 x ASIC Read In Link GbE Port TIGER test assembly NIOS-II (soft) microprocessor Latency buffer for data (internal RAM) BES-III Specific HDL modules TESTED IN VIVO OK TESTED IN VIVO OK Stefano s Ethernet MAC SDRAM DDR3 BUFFER High bandwidth link TESTED ON ARRIA V GX DEVELOPMENT KIT 8

9 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card GEM-ROC FPGA card Single TIGER card (INFN- TORINO) 9

10 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER configuration The Tiger test assembly module receives configuration commands and data for the controlled ASIC from the on-board processor accessed via Ethernet though a dual port memory which is also used to return the result of the configuration command (status only in case of a WRITE command, status and data in case of a READ command. The Master side of the DPRAM is connected to the NIOS-II processor data bus (Avalon) 10

11 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly simulation Test conditions: two rx 200MHz, SDR ( corresponding to a 75kHz max hit rate per channel) 11

12 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly real signals Injected test pulse TIGER data (only channel 0 shown) Current test conditions: two rx 160 MHz, SDR ( corresponding to a 60kHz max hit rate per channel) 12

13 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly real signals TIGER data (HeartBeat word) Current test conditions: two rx 160 MHz, SDR ( corresponding to a 60kHz max hit rate per channel) signals activity inside the GEM-ROC FPGA 13

14 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly triggerless mode TESTED OK Current test conditions: test log file console output of TIGER configuration interface program (a.c.r) running on the NIOS-II soft microprocessor in FPGA Stefano s terminal_udp program output, capturing the UDP stream two rx 160 MHz, SDR ( corresponding to a 60kHz max hit rate per channel) dump of Heart Beat and Counter words from the TIGER ASIC collected by the GEM-ROC FPGA card 14

15 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly trigger match mode Higlights: data is stored in a circular buffer with some extra information (data word is extended to 72 bits) to detect whether data is stale (i.e. no new data is being received due to HW issues) data is trigger 200MHz (now being by reading a fixed number of words from the buffer which include the window of interest ( 8.6us latency, 1.6us window-centered) a formatted packet is created for each event with header and trailer allowing event identification (see simulation log below) the first received L1 trigger will actually serve to synch reset all TIGER chips: it will produce trigger matched packet with no data in it Current test conditions: two rx 160 MHz, SDR ( corresponding to a 60kHz max hit rate per channel) fine tuning of the trigger matching algorithm is in progress 15

16 Update on development status: off-detector readout card (GEM-ROC) Interconnection tests with single TIGER card : TIGER test assembly trigger match mode Parts of the TIGER test assembly modules (ACR) that are being tuned are: data receiver: to re-lock in automatically to the input serial streams when a synchronous reset is sent to the TIGER (at the start of a run for instance) adjustment of the size and access mode of the circular buffer for the TIGER data in order to better account for the latency of transmission from the TIGER to the GEM-ROC FPGA - left picture: TIGER TX units are reset by a synch reset - right picture: time to collect test pulse data for all 64 channels of TIGER Current test conditions: two rx 160 MHz, SDR ( corresponding to a 60kHz max hit rate per channel) 16

17 Update on development status: VME-based data collector card (GEM-DC) Reminder: (picture and description from ATLB/Data Collector documentation by Pawel Marciniewski) GEM-DC cards are available A VME crate with one GEM-DC card is ready 17

18 Update on development status: VME-based data collector card (GEM-DC) Firmware development: The GEM-DC hardware is ready; the firmware must be tuned to the BES-III specific application so that the off-detector readout system may be operated throughout the VME based GEM-DC card. All optical link and VME interface HDL modules are ready. The protocol of the communication through optical link between the GEM-ROC card and the GEM DC card is being implemented. Some details are given in the following. Mapping of the configuration and data buffer onto the GME-DC VME memory space will follow. 18

19 Update on development status: VME-based data collector card (GEM-DC) Firmware development: GEM_DC to GEM_ROC protocol: TIGER Global Configuration register programming The TIGER GLOBAL configuration (GCFG) register has 169 bit. These are organized in 6 32-bit words as shown above. The word format reflects the current implementation of the GEM-ROC FPGA design in which the serial configuration controller receives the data to be used from a DPRAM memory set by the on board microcontroller. Routines written in C language are available to prepare the 7-word packet from the desired values of the TIGER Global Configuration parameters. Word 0 is a command word containing the following fields: - CMD CODE : either CMD_WRGCFG = 0x8 for GCFG register write or CMD_RDGCFG = 0x9 for GCFG register read - TIGER ID: the ID of one of the 8 ASICs controlled by one GEM_ROC card - WR-nRD (redundant) According to this proposal the GEM-DC would send a command packet and will receive an echo packet back, with the same structure as the sent packet except for the field TIGER REPLY BYTE (which contains error flags reported by the TIGER for the current SPI configuration) 19

20 Update on development status: VME-based data collector card (GEM-DC) Firmware development: GEM_DC to GEM_ROC protocol: TIGER Channel Configuration register programming The TIGER CHANNEL configuration (CHCFG) registers have 125 bit. These are organized in 4 32-bit words as shown above. The word format reflects the current implementation of the GEM-ROC FPGA design in which the serial configuration controller receives the data to be used from a DPRAM memory set by the on board microcontroller. Routines written in C language are available to prepare the 5-word packet from the desired values of the TIGER Global Configuration parameters. Word 0 is a command word containing the following fields: - CMD CODE : either CMD_WRCHCFG 0x0 for CHCFG register write or CMD_RDCHCFG = 0x1; for CHCFG register read - TIGER ID: the ID of one of the 8 ASICs controlled by one GEM_ROC card - WR-nRD (redundant) - ToALL_Enable: if 1 then all channels are configured with the parameters received with the command packet - Target Channel ID According to this proposal the GEM-DC would send a command packet and will receive an echo packet back, with the same structure as the sent packet except for the field TIGER REPLY BYTE (which contains error flags reported by the TIGER for the current SPI configuration) If ToALL_Enable is set then the GEM-ROC will send 64 echo packets back to the GEM-DC 20

21 Update on development status: VME-based data collector card (GEM-DC) Firmware development: GEM_DC to GEM_ROC protocol: trigger matched data packet format The proposed format for the TRIGGER MATCHED DATA PACKET transmitted by the GEM-ROC at every L1 pulse received is outlined above. The number of 32-bit words transmitted is * N, where N is the number of trigger matched TIGER timestamp words. The packet is assembled with: - TRIG_HEAD word TRIG_HEAD = 100 L1_header_status_bit s; current definition: tm_dfifo_almostfull, tm_dfifo_full, "0, "0, "0" local_l1_count (31 downto 0) : count of L1 trigger pulses received by the GEM-ROC local_l1_framenum (7 downto 0): 8 LSB of local counter of Frame periods (equivalent to TIGER FAST clock cycles) local_l1_ts (15 downto 0) : time stamp of L1 Trigger measured on the GEM_ROC with the time resolution of the TIGER FAST clock - DATA word DATA_TAG = 000 TIGER ID LAST TIGER FRAMENUMBER (3 downto 0) : LSBs of the last TIGER FRAME NUMBER received; included for off-line checks on data consistency TIGER RAW DATA ( 53 downto 0 ): see TIGER documentation for details - TRIG_TAIL word TRIG_TAIL = 101 Count of Trigger matched hits ( 4 downto 0 ) local_l1_count (7 downto 0) : LSBs of count of L1 trigger pulses received by the GEM-ROC; repeated here for consistency checks local_l1_framenum (23 downto 8): 16 MSBs of local counter of Frame periods (equivalent to TIGER FAST clock cycles) LAST COUNTER DATA RECEIVED FROM TIGER [ 31 : 0 ] : it contains the last value transmitted by the TIGER in the COUNTER field 21

22 Update on development status: Off-detector readout system (GEM-ODR) WBS update 2016: o Definition of the baseline architecture for the GEM-ROC modules o Procurement of the basic building blocks (FPGA development kits, for instance) needed for prototyping o Development of the firmware for the GEM-ROC FPGA: o development of the interface to the TIGER chip starting from the ASIC model and testbench provided by the ASIC designers (Ricardo Bugalho) o Test the interface to the VME Data Concentrator card o validation of firmware interface to the TIGER prototype carrier PCB o development of the GEM-ROC motherboard (details on the available FCS receiver / fanout modules are needed to complete this task) 2017: o commissioning of the GEM-ROC operation by reading out in stand-alone mode (Ethernet based) a prototype C-GEM equipped with TIGER ASICs o delivery of a prototype GEM-ROC module to BES-III to test stand-alone operation in the experimental hall (for eventual investigation of background radiation effects) o see next slide Development of GEM-ROC: milestones 22

23 Update on development status: Off-detector readout system (GEM-ODR) WBS update The project is moving on as outlined in the WBS, roughly on-time. Some highlights: further details on how the BES-III Fast Control signal will be delivered to the offdetector readout systems are still needed (possible impact on design schedule for the GEM-ROC card) the location for the off-detector electronics parts of the system is still to be defined (impact on cable length and bandwidth, not on the schedule for the moment) interconnection test with the TIGER is still on going but outcome is positive. (ok for schedule) design of the schematic for the GEM-ROC motherboard not finished (we shoud still meet delivery date for GEM-ROC prototype motherboard) early installation of prototype GEM-ROC card (and possibily GEM-DC) is being planned according to schedule 23

24 Thank You! 謝謝 24