The Application of DAQ-Middleware to the J-PARC E16 Experiment

Transcription

1 Track 1 Session: #3 (Data acquisition and electronics) April 14, :00 Village Center The Application of DAQ-Middleware to the J-PARC E16 Experiment E Hamada 1, M Ikeno 1, D Kawama 2, Y Morino 1, W Nakai 3, 2, Y Obara 3, K Ozawa 1, H Sendai 1, T N Takahashi 4, M M Tanaka 1, S Yokkaichi 2 1 High Energy Accelerator Research Organization (KEK) 2 RIKEN Nishina Center 3 The University of Tokyo 4 Research Center for Nuclear Physics, Osaka University 2015/4/14 CHEP2015 1

2 Outline 1. DAQ-Middleware overview and features 2. DAQ System at E16 Experiment requirements and architecture 3. DAQ System Performance method and result 4. Summary and Future Plan 2015/4/14 CHEP2015 2

3 1. DAQ-Middleware 2015/4/14 CHEP2015 3

4 What DAQ-Middleware Is A framework for network based DAQ software Easy to use, configure and develop Target Medium-scale experiments Test benches (sensors, electronics) 2015/4/14 CHEP2015 4

5 DAQ-Middleware Architecture Develop DAQ system by configuring DAQ components XML System Configuration PC Daq Operator software unit that achieve DAQ functions Command / Status PC Logger Read-out -modules Gatherer Dispatcher Monitor 2015/4/14 CHEP2015 5

6 DAQ Component Service Port (command/status) Service Port (command/status) Data + = Logics Data Logics InPort OutPort DAQ-Middleware provides (for data handling) users writes InPort (for data handling) OutPort 2015/4/14 CHEP2015 6

7 DAQ Component Features Flexibility Scalability READOUT READOUT READOUT READOUT Users can flexibly change DAQ component combination. Reusability Users can improve performance by adding new PCs and deploying DAQ components. Ring Buffer logic Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer. 2015/4/14 CHEP2015 7

8 List of DAQ-Middleware Users Experiments [19 experiments] - Material and Life Science - J-PARC MLF Neutron/Muon - DAQ system of Depth-resolved XMCD (KEK PF) - Elementary Particle/Nuclear Physics - J-PARCE16 Experiment - CANDLES - SuperNEMO (planning) DAQ-Middleware Working Test benches for sensors and electronics [9 test benches] Liquid Argon TPC SOI Pixel Detector ILC CCD Vertex ADC-SiTCP 2D gaseous detector with readout ASIC using printing technologies J-PARC MLF Neutron beam lines and so on. 2015/4/14 CHEP2015 8

9 2. DAQ System at E16 Experiment 2015/4/14 CHEP2015 9

10 Requirements Estimation of data transfer to DAQ PCs Data rate per spill Trigger rate 660MB/spill (average) 1kHz 660MB data receive 660MB data receive (max) 2kHz 1spill Instantaneous data rate (average) 330MB/s (max) 660MB/s not receive not receive time [s] Trigger rate fluctuates due to beam rate variation. Event size per one event is almost constant. 2015/4/14 CHEP

11 System Architecture Read-out-module Network switch Storage PC HDD Function Read data Save data on each HDD Send data of a part of event to Monitor PC Network switch Monitor PC HDD HDD Function Event build Analyze data & show the result 2015/4/14 CHEP

12 DAQ Component Configuration Gatherer Storage PC Logger Monitor PC Merger Dispatcher Merger Eventbuilder Monitor Filter Component Function Component Function Gatherer Merger Read data Receive data from multiple Gatherers Merger Eventbuilder Receive data from multiple Filters Event build Dispatcher Logger Send data to Logger and Filter Save data on each HDD Monitor Analyze data & show the result Filter Send data of a part of event to Merger of Monitor PC 2015/4/14 CHEP

13 DAQ Component Features Flexibility Scalability READOUT READOUT READOUT READOUT Users can flexibly change DAQ component combination. Reusability Users can improve performance by adding new PCs and deploying DAQ components. Ring Buffer logic Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer. 2015/4/14 CHEP

14 Advantage of DAQ Component Features Flexibility & Scalability If data volume increase by detector upgrade, we add Storage PC. Reusability Using DAQ component which is prepared, we saved time and effort for development. Sample Dispatcher, Logger Gatherer, Monitor DAQ system Ring Buffer We can use no event time effectively. receive process receive process time [s] 2015/4/14 CHEP

15 3. DAQ System Performance 2015/4/14 CHEP

16 Purpose & Method Purpose Evaluation of a total throughput using a DAQ-Middleware for the J-PARC E16 experiment Method Emulator Emulator Emulator Emulator Emulator Emulator Emulator Emulator Emulator test data Storage PC HDD HDD send data once every 10 events Network switch Monitor PC Emulators run with 1 cycle per 6 seconds. During one cycle, emulators send test data for 2 seconds and do not send test data for 4 seconds. Monitor PC shows data value of a part of data regularly. We changed the number of emulators to change transfer data volume and measured processing data speed. 2015/4/14 CHEP

17 Result 2kHz HDD write limit 1kHz requirement throughput (Trigger rate is always max [2kHz]. ) requirement throughput (Trigger rate is always average [1kHz]. ) During sending test data, one event data size per one emulator is 14kB. The result met the requirements of the E16 experiment! 2015/4/14 CHEP

18 4. Summary and Future Plan 2015/4/14 CHEP

19 Summary and Future Plan DAQ-Middleware is a framework for network based DAQ software. DAQ component has following features. Flexibility Scalability Reusability Ring Buffer We have developed DAQ system for E16 experiment by using DAQ-Middleware. The requirements from E16 experiment have been met. In the future, we are going to connect DAQ system to read out module, and evaluate the system. 2015/4/14 CHEP

20 Backup 2015/4/14 CHEP

21 J-PARC E16 Experiment DAQ system overview 2015/4/14 CHEP

22 Evaluation Environment CPU PC1 PC1 PC2 HDD HDD Intel(R) Xeon(R) 2.67GHz 6Cores CPU Cisco Catalyst 2960G PC2 Intel(R) Xeon(R) CPU E GHz 6Cores CPU PC3 PC3 Intel(R) Xeon(R) CPU E GHz Memory 24GB Memory 32GB Memory 8GB Network 1Gbps x 5 Network 1Gbps x 5 Network 2Gbps OS Scientific Linux 6.4 OS Scientific Linux 6.6 OS Scientific Linux 6.6 HDD Hitachi HDS724040ALE6 4TB HDD Hitachi HDS724040ALE6 4TB 2015/4/14 CHEP

23 HDD write speed check 2015/4/14 CHEP

24 SRS (Scalable Readout System) SRS is general purpose multi-channel readout system. User can choose variety of frontend chips. SRU Eurocrate FEC Front-End Card FEC Front-End Card FEC Front-End Card Front-End ASIC Front-End ASIC Front-End ASIC ADC FEC 2015/4/14 CHEP

25 Evaluation by using SRS SRS ADC/EFC test data Storage PC HDD send data once every 10 events Network switch Monitor PC HDD ADC/FEC send test data. Maximum transfer speed of test data is 1Gbps + 1Gbps. We configured that DAQ PC could process all of data. 2015/4/14 CHEP

26 Storage PC Performance Emulator emulator emulator emulator emulator test data trigger rate = 2kHz 1 event size = 14kB write data on SSD Storage PC CPU Intel(R) Xeon(R) 2.67GHz 6Cores Memory 24GB Network 1Gbps x 10 OS Scientific Linux 6.4 SSD Intel SSD520Series 240GB Test PC only receive data Throughput of 1 Storage PC is 1000MB/spill. 2015/4/14 CHEP

27 DAQ Component & Configuration Example Service Port (command/status) Service Port (command/status) Data InPort OutPort DAQ-Middleware provides + = Data Logics (for data handling) user writes InPort Logics (for data handling) OutPort Examples of DAQ component combination DaqOperator DaqOperator DaqOperator Logger Gatherer Dispatcher Monitor Online Monitor & Logging Gatherer Logging only Logger Gatherer Monitor Online Monitor only 2015/4/14 CHEP