HCAL Trigger Readout

Transcription

1 HCAL Trigger Readout HTR Status and Clocking Issues D. Baden, T. Grassi 1

2 FE/DAQ Electronics S-Link: MHz Trigger Primitives READ-OUT Crate CAL REGIONAL TRIGGER Rack CPU S B S C L K D C C H T R H T R H T R 12 HTRs per Readout Crate, 2 DCC TTC MHz FRONT-END RBX Readout Box (On detector) HPD QIE QIE CCA MHz GOL Shield Wall QIE QIE QIE QIE CCA CCA GOL Fibers at 1.6 Gb/s 3 QIE-channels per fiber FE MODULE 2

3 HTR Principal Functions 1. Receive front-end data for physics running Synchronize optical links Data validation and linearization Form TPG s and transmit to Level 1 at 40 MHz Pipeline data, wait for Level 1 accept Upon receiving L1A: Zero suppress, format, transmit to the concentrator (no filtering) Handle DAQ synchronization issues (if any) 2. Calibration processing and buffering of: Radioactive source calibration data Laser/LED calibration data 3. Support a VME data spy monitoring 3

4 Readout VME Crate BIT3 board Slow monitoring over VME Commercial VME/PCI Interface to CPU FanOut board Takes TTC stream in Clone and Fanout timing signals HTR (HCAL Trigger and Readout) board D C C Spy output over VME FE-Fiber input TPG output (s) to CRT DAQ/TP Data output to DCC DCC (Data Concentrator Card) board Input from HTRs Spy output Output to DAQ TTC fiber DAQ B I T 3 F a n O u t D C Front End Electronics H T H T Fiber 1.6 Gb/s... C R R VME CRATE 20m Copper 1.2 Gb/s Calorimeter Regional Trigger 4 H T R H T R

5 Board organized around 2 identical sets of circuitry: Optical inputs 1.6 GHz, 8B/10B frames, 3ch/link Dual LC detectors and drivers TLK2501 Deserializers Crystal RefClk TTC 80MHz backup Xilinx Virtex FPGA XCV1000E 24 channels each TPG signals Sent to over backplane, LVDS s mounted 6 to a transition board Level 1 accept output to DCC LVDS output VME Altera FPGA and firmware Old HTR Design (Summer 2002) OLD DESIGN 5

6 HTR Functional Experience What was tested: VME fully tested and working Some changes necessary to conform to CMS VME standards Optical links and synchronization No indication of any problems. Big success here was a real worry LVDS to DCC Tested, working (Will change cable/connector to Cat 6/RJ45) Fanout of timing signals on two Cat5 cables Plan to change to a single Cat6 or Cat7 cable (very low cross-talk) Firmware full tests of: Pipeline and L1A triggering capability In-line histogramming for source calibration TTCrx Not working at all (4 bad on 4 tested). What was not tested: Anything to do with TPG 6

7 HTR Board Experience Produced ~12 boards Several bare boards were delivered warped Many opens under FPGA after assembly (~9 boards) Some fixed after reflow (a few) Some worse after reflow (shorts) X-rayed a few boards, sometimes inconclusive Some opens on VME side Non BGA FPGA, indicates bad vias Few other various open circuits Finally got ~8 boards to work Questionable reliability 7

8 Modifications Change board from using white-tin to gold traces This process was sold to us by the board maker. Our mistake. Used only for very high volume, cost competitive products, very difficult and expensive to control. Gold is flatter and not very much more expensive (~$50/board), better for FPGAs Change assembly house Insufficient Quality Control on current assembler they are fired. We visited 2 high-end assemblers Modern Machines Step up and step down oven temp control. In-line X-ray for BGA QC Manufacturability Review Add stiffeners to HTRS Flexability of 9U VME boards was underestimated Worry: fine-line BGA (FBGA) can pop connections 8

9 Modifications (cont) Change from FBGA to BGA FBGA 1.0mm pitch, change to BGA 1.27mm pitch No additional expense, plenty of available real estate, no need to push We are just being very conservative here JTAG capabilities added Will help with debugging By making these changes We have profited from the summer We have reduced our production risk considerably 9

10 HTR Design Changes transition board issues: Worries about so many LVDS signals over backplane for old design Routing is too complicated Many signals going to same backplane location Requires multi-layer routing with many vias TPG cables very thick Mechanical issues are very worrisome changes needed (e.g. height reduced after ECAL redesign ) Solution: move s to HTR motherboard Benefits: Mechanically attach s to HTR front panel for mechanical stability Eases routing requirements, reduces board and assembly risks, cheaper too Change from Xilinx VirtexE to Virtex2 More resources, block ram, hardware multipliers Big cost reduction (save $180k) More modern chip for long-term maintenance Clock synchronization Decouple 80MHz crystal from FPGA system clock Will allow us to use crystal to maintain synchronization of serdes This gives us 2 solutions for our 40ps jitter requirement issue 10

11 New HTR Conceptual Design Fibers to DCC 8-way 8-way LVDS LVDS FPGA Xilinx XC2V LC LC LC LC VME FPGA P1 to Level 1 Cal Trigger from Fanout RJ45 TTC FPGA Xilinx XC2V LC LC LC LC No P3! P2 11

12 Clocking Schematic Start with Fanout card TTCrx Maryland mezzanine card or CERN TTCrm daughterboard QPLL Fanout on Cat6/7 quad twisted pair TTC, BC0, 40MHz, 80MHz In HTR: Send TTC signal to TTCrx mezzanine board, access to all TTC signals Send 80MHz clean clock (cleaned by QPLL) to mux Select 80MHz clean clock OR crystal to deserializers Cat 6/7 quad cable (allows PECL) TTC Fanout Board TTCrx QPLL TTC BC0 80MHz 40MHz 80MHz BC0 80 MHz LVPECL Crystal 40MHz 1 to 8 Fanout BC0 40 MHz 80 MHz 1 to 8 Fanout 1 to 8 Fanout (16) F P G A Single width VME TTC TTC mezz TTC broadcast bus 12

13 Fanout Card TTCrx HCAL TRIDas Clock Scheme QPLL Cat6/7 RJ45 TTC BC0 CC40 CC80 4 twisted pair ( CC means Clean Clock) RJ45 TTC TTCMezz TTC broadcast, L1A, BCR, EVR, CLK40 CC40 CC80 Xilinx BC0 HTR Board 13

14 Fanout HTR scheme TTC fiber Notes: s require fanout of CLK40, BC0. FE-link possibly requires CLK80. PECL fanout was tested in TB2002. One Cat6E cable (low x-talk) replaces the 2 Cat5 cables used in TB2002. TTC and BC0 remain LVDS as in Weiming s board. HTR needs Broadcast bus, BCntRes and L1A: from TTCrx if we get it to work, otherwise we have to fan them out. TTCrx (or daughter card) Fanout Board TTC Brdcst<7:2>, BrcstStr QPLL AN1568/D Fig 11 Onsemi.com FPGA CLK40 LVDS CLK80 LVDS Brdcst<7:2>, BrcstStr, BCntRes, CMOS L1A Fanout buffer TTC BC0 Fanout x 15 TTC Low-jitter Fanout x 15 PECL fanout PECL fanout LVDS or diff PECL LVDS O/E NB100LVEP221 is LVDS compatible ~Fifteen RJ45 connectors connectors on bottom layer? Cat6E or Cat7 cable 2 Test Points for CLK40 and BC0 15 Cables & Connectors tbd 9U Front-panel space = 325 mm ; => space per connector ~ 21.5 mm RJ TTC LVDS RX_BC0 LVDS CLK80 3.3V-PECL DS90LV001 e.g. DS90LV110 LVDS Fanout x 8 Q1 Q2 CLK40 Q3 3.3V-PECL Q4 Q5 Q6 Q7 Q8 MC100LVE V PECL MHz 3.3V crystal Diff. PECL MC100LVEL37 CK CK CK/2 CK/2 Diff. to 6 s Single-end to 2 xilinx To 6 s Diff. to 2 Xilinx + termin. IN IN_b TTC daughter card PCK953 LVPECLto-LVTTL Fanout (top layer) PCK953 LVPECLto-LVTTL Fanout (top layer) 8 clks to TLKs + TPs Brdcst<7:2>, BrcstStr, L1A, BCntRes to xilinx and s 8 clks to TLKs HTR Tullio Grassi <tullio@physics.umd.edu> 14

15 Very simple card: 2 PMC connectors TTCrx chip TTC signal driver on motherboard Will be sent out for prototype next week Used by HTR, DCC, and Fanout cards TTCrx Mezzanine card 15

16 TTC Distribution Fanout Card Currently HCAL has 6 TTC partitions: Each partition requires TTCvi and TTCex Each HCAL VME crate will have a single TTCrx receiving data directly from TTCex in a single VME card (Fanout Card) Fanout TTC signal to HTR mezzanine card with TTCrx chip Use quad twisted pair CAT6/7 cable allows PECL fanout TTC raw, BC0, 40MHz clean, 80MHz clean fanout Cost savings and simplification TTC monitoring by Fanout card over VME Count resets, etc 16

17 Random Latency Issue Texas Instruments TLK2501 Serdes Run with 80MHz frame clock 20 bits/frame, 1.6GHz bit clock 625ps bit time Latency from data sheet: ~20ns variation (overall latency between 47 and 67ns) Fiber to fiber alignment could cross a 40MHz bucket boundary. How to fix? knows this latency we will read it out after each reset HCAL LED fast rise time Can pulse during abort gap and align channels Requires LED pulsing alignment FE will send BC0 signal on all fibers Will measure this alleged latency with new HTR boards 17

18 TPG Alignment TPG alignment performed in Necessary: All HTRs will send common BC0 to s within each of 16 VME crates Calibration procedure to be performed for crate-crate alignment Initial alignment with LEDs, laser, etc. Final alignment with LHC first beam data Use 1-bucket running to check everything This will ensure successful alignment 18

19 DAQ Alignment DAQ data must also be aligned Must know L1A bucket for zero suppression Only if we will do ZSP on 1 or 2 HCAL channels centered on L1A bucket If ZSP done with sum over 5 channels, then this alignment is not critical Solution: discussed in previous slide Read from FE sending known ID after with fixed offset relative to BC0 during abort gap Comparison of the two for error checking DAQ check on BC0 in DCC for alignment Will send BC0, BCN, and EVN with the data to DAQ 19

20 MISC Errors What happens if DCC finds mismatch in EVN? DCC will then issue resynch request to atts system Details not yet defined but is fully programmable Fiber Link/synchronization errors (GOL/) Work out protocols to inform DCC Reset requests to atts as well FE Clock/GOL PLL link errors If GOL loses synch, then transmitter will send out IDLE characters IDLE characters are illegal in a pipelined system! HTR will trap on IDLE as a signal that FE/GOL is having trouble 20

21 Schedule O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D Firmware Board layout Fab/assembly 20 boards will be built but not assembled Pre-production HTR board Checkout Board layout if needed Fab/assembly if needed Production prototype Checkout Production Testbeam? Vertical Slice? 21