The Trigger and Data Acquisition system for the NA62 experiment at CERN

Transcription

1 The Trigger and Data Acquisition system for the NA62 experiment at CERN M. Sozzi University of Pisa and INFN 11 th Pisa Meeting on Advanced Detectors

2 Introduction for the unaware In year 2009 a.d. CERN was eagerly waiting for LHC to start. All physicsists were set for a long-awaited attack at the energy frontier. Actually not all of them For one group of undomitable physicists still held out at the precision frontier: the NA62 collaboration NA62 CERN a LHC occupatum 2762 ab Urbe Condita

3 The ultra-rare decays K πνν K πνν processes are unique as extremely accurate and clean probes for the nontrivial flavour structure of New Physics No uncertainty from hadronic matrix element and long distance terms (ρ,η) α K + π + νν Best way to measure V td They can provide an indipendent precise measurement of the unitarity triangle BR ~ γ (0,0) K 0 π 0 νν β (1,0) The K L π 0 νν mode (so far undetected) is searched for at KEK and J-PARC

4 K + π + νν Long experimental history, low-e, decays at rest BR(K + πνν) TH =(8.22±0.84) BR(K + πνν) EXP =( ) (7 candidates: BNL E787+E949) New approach: decay in flight Proposal 2005 R&D CERN final approval years construction + 1 year commissioning High energy (75 GeV/c ± 1%) unseparated hadron beam (6% K) from SPS π + backward in CM GOAL: 0(100) events (~10% bkg) in O(2 years) data-taking taking

5 Non -kinematically constrained bkg PID and Veto Single-track signature Both K + and π + momentum measurement ~50MHz ~800MHz Single track signature π Non -kinematically constrained bkg Ultra-rare signal ν ~10MHz ν Both K + and π + momentum measurement PID and Veto High intensity beam and trigger / DAQ system Ultra-rare signal High intensity beam and trigger / DAQ

6 Trigger/DAQ: key requirements Offline control Flexibility, minimize hardware Rare decay experiment Not limited by proton flux Crucial vetoing power Background control Other physics Integrated, fully digital trigger + DAQ Fully monitored system: inefficiency and flow control recording Hardware L0 and software L1/L2 High trigger efficency, deadtimeless Bandwidth scalability Low random veto probability: high online time + double pulse resolution Undetected acquisition losses < 10-8 Readout without zero suppression of candidate events Simple, controllable trigger cuts Flexibility, configurability

7 The Kaon environment (SPS is not LHC) No tight space constraints, longish setup O(150 m) No serious radiation issues Electronics rather close to electronics (but scattered) Fixed-target experiment: no bunch structure, duty cycle O(25-35%), long O(20 s) inter-spill period usable for data retrieval and processing

8 NA62 TDAQ - OVERVIEW RICH MUV CEDAR STRAWS LKR O(10 MHz) LAV 2K PC 1K 1 MHz PC PC PC 1 MHz L0 CTP O(<1ms) 200 GbE switch PC PC PC PC 8K 1 MHz 1 MHz PC PC PC PC PC PC L0 L1/2 13K 2.5K 60K L0 trigger Trigger primitives PC PC PC PC CDR Data O(KHz) EB

9 The size of the project, the size of the collaboration, the financial envelope and the time-scale of the experiment required intense R & D Re-use & Duplicate GOALS: Achieve maximum possible uniformity for most sub-detectors Exploit as much as possible existing solutions (either commercial or developed for HEP experiments)

10 Clock issues Free running master clock (use 40 MHz to exploit LHC developments) LHC standard TTC system used for master clock distribution and L0 trigger distribution (with trigger type info): TTCci module by CMS (to be tested) CERN QPLL everywhere to reduce clock jitter Still performance might not be good enough (e.g. 6.4ps RMS for 10b FADC aperture jitter in LKr calorimeter sampling): keep (existing) excellent NA48 clock (80 MHz) in parallel 40 MHz SPS L0 TTC TTCrx QPLL Sub-detector boards 40 MHz clock (50 ps jitter) burst reset L0 trigger + info

11 LHCb TELL1 board Common infrastructure: TELL1, general-purpose acquisition board developed by EPFL Lausanne for LHCb (300 boards deployed) CUSTOM CUSTOM CUSTOM CUSTOM VME-9U format (power only) Up to 4 custom daughter-cards Up to 24 Gb/s input bandwidth 5 FPGA for processing 384 MB DDR RAM for buffering Embedded diskless PC TTC connection for trigger and clock 4x GbE output links Somewhat oldish design Proposed as baseline common solution O(100) boards needed

12 TDC daughtercard Most sub-detectors need TDCs High-integration TDC TELL1 daughtercard developed to provide 100ps time digitization (CERN HPTDC) 1 TDC card = 4 TDCs (128 channels), 4 TDC cards per TELL1 = 512 channels / VME 9U FPGA (Altera Stratix II) + 1 MB SRAM for data monitoring and preprocessing Two protoype versions existing, final version by end of the year Development by INFN Pisa 11th Pisa Meeting - 27/5/2009 Used by: RICH, straws, MUV, CEDAR, LAV, vetoes LVDS inputs QPLL on board Miniaturized connectors (both sides) The NA62 TDAQ system I2C slave and JTAG master Low noise electronics + filters Clock jitter < 40 ps M. Sozzi

13 TELL1 fully TDC-equipped TDC board PC RAM HPTDC TTC SL-FPGA TC-FPGA PP-FPGA 11th Pisa Meeting - 27/5/2009 The NA62 TDAQ system GbE M. Sozzi

14 TDCB and TELL1 layout Clock Data Controls Packets

15 TDC: under test Test system performances match the requirements so far Mechanical issues with cables and connectors require different solution Time resolution <250 ps (single hit) Favourable comparison with commercial TDC VME modules (CAEN V1290A) based on same HPTDC (smaller number of channels, smaller max frequency due to VME bandwidth) TDCs are readout continuously Commercial TDC/TELL1

16 TDC: more tests New RICH prototype test (400 PM) ongoing now at CERN, with real (low-intensity) hadron beam, on the NA62 beam line Goals: RICH optimization, rate testing (present limitation O(200 khz)), parasitic trigger primitive generation, choice of final cables and connectors th 11 Pisa Meeting - 27/5/2009 The NA62 TDAQ system M. Sozzi

17 TDC/TELL1 for L0 trigger TELL1 can be used both for readout and generation of trigger primitives. RICH Sub-detector FE connected to TELL1 through a suitable daughter card (e.g. TDC board) Trigger primitives can be collected with a daisy chain or a tree connection to the L0 central trigger processor (L0TS) using GbE link(s) The trigger decision (~1 MHz) will be sent by L0TS using TTC R/O is in parallel through other GbE links Data from FE electronics Data to L1 trigger/daq TELL1 TELL1 TELL1 TELL1 Data from FE electronics RICH TELL1 TELL1 TELL1 TELL1 TTC LOTS TTC Tree L0 trigger Data to L1 trigger/daq LOTS Daisy chain L0 trigger

18 Example: RICH multiplicity trigger Each 25 ns time window is subdivided in N=8 bins Pipelined filling of fine-time distribution, with time realignment and hit reordering If one of the bins is over threshold then the trigger primitive is set Merge 128-ch histograms into 512-ch histograms in SL-FPGA Merge 512-ch histograms into 2048-ch histograms along L0 chain Send time of interest tag to L0TS for comparison with other sub-detectors PP0 Implementation: Pisa and Perugia PP1 PP2 SL Similar approach for LAV (LNF/Roma 1), MUV (Protvino) PP3 25 ns

19 LKr calorimeter readout The existing LKr readout was designed for a 10 khz trigger rate Most crucial (and expensive) element are CPD (Calorimeter Pipeline Digitizer) analog cards: custom ASIC for shaping & gain-switching, FADC for cells, analog energy sums (over 2x8 cells) The 10+2 bit 40 MHz FADCs produce ~ 1 TB/s (without zero suppression)

20 New LKr calorimeter readout Keep the analog cards Redesign readout part and motherboard to have continuous fast readout (32x1.2 Gbps) in parallel All data stored in very large memories (1GB/ch DDR2) (4+1)GbE links/board will send the data to the readout system upon request (can stretch over inter-spill period too) CARE: Calorimeter REcorder Prototype presently under development at CERN

21 New LKr calorimeter readout Reduced data set (heavily zero-suppressed) can be delivered at L0 rate Complete non zero-suppressed events readout during the inter-spill period 8.3 s deep memory: latency not an issue Large GbE switch will send data to a dedicated PC farm The super-cells (8x2) analog sums will be used in the LKr trigger system

22 New LKr calorimeter trigger channels for the readout 864 channels (2x8 cells) for the trigger Existing (NA48) trigger system (Vienna-Pisa-CERN) based on 40 MHz FADCs, custom peak-counting boards, custom FPGA-based boards (8 crates) Keep the existing analog inputs, redesign the digitization and peak-counting using again TELL1 as underlying MB Develop 8-channel 40 MHz 10-bit FADC daughter card Develop 6xGbE receiver card to interconnect TELL1s Roma Tor Vergata INFN + University CPD-CARE SYSTEM 32 supercells TTC ADC card 28 boards TELL1 32 ch 1st layer: Trigger board (pulse reconstruction, trigger primitive generation,...) 3 boards 16 ch L0TS Ethernet card TELL1 2st layer: Concentrator board TTC

23 New LKr calorimeter trigger 6x GbE receiver card under development This allows flexible interconnection of TELL1 boards on copper for any purpose FPGA design in advanced stage Ethernet link tests (with development kit) starting

24 New LKr calorimeter trigger Goal: perform crude (super-cell) cluster-counting with 1-2 ns time resolution Locate peaks in space and time, perform time interpolation, take care of edge effects Algorithm implemented and dimensioned on actual FPGA: peak-finder + peak-processor

25 L0 central trigger processor (L0TS) Tasks: merge time of interest/veto lists re-synchronize L0 trigger to drive TTC provide trigger data for readout Different solutions under consideration: High-performance PC TELL1 w. GbE receiver daughter cards Full custom board One possible component exists (developed in Roma Tor Vergata): PCI Express Readout Card (12 layers, DDR2, StratixII) IN (optical): Gb/s IN(copper): Gb/s OUT: PCI-Ex gen.1 4x2.5Gb/s Can be used as L1 PC receiver or for L0TS receiver in the PC scenario

26 L0 processor (L0TS) and more? Recent development: assess the possibility of exploiting GPUs to handle the L0TS task Well-suited architecture for embarassingly parallel tasks (exploding interest in GPGPU): such as independent event processing Main obstacles so far: (1) Architecture-dependent coding: no longer true (2) Huge latencies: now approaching the interesting range O(100us) Aim to measure real-time capabilities of 4 TFlop system with huge bandwidth-to-memory, assess possibility of use directly at L0 (e.g. RICH) Work started in Pisa

27 Several items not discussed here: The silicon pixel Gigatracker project: two ASIC projects under design (CERN and Torino) for integrated 100ps TDC. Readout with custom electronics + possibly a downstream TELL1 The L1/L2 and event-building PC farm presents a challenge in itself to achieve the required rate reduction with a limited amount of processors Conclusions NA62: a new experiment, challenging under several aspects NA62 at La Biodola 3 years from now The trigger/daq system, is one challenging item Hard work ahead, but the final goal is well worth it! (even more if LHC 11 th Pisa surprises Meeting - 27/5/2009 us) The NA62 TDAQ system M. Sozzi N. Thakar