Bruno Soares Gonçalves

Transcription

1 Advanced Control and Data acquisition systems for fusion experiments Why ATCA? Bruno Soares Gonçalves On behalf of IPFN Control and Data Acquisition Group Instituto de Plasmas e Fusão Nuclear Instituto Superior Técnico Lisbon, Portugal B. Gonçalves Paris, February 2, 2009 NI - Big Physics Round Table

2 Large-scale experiments: control and data aquisition challenges The next generation of physics experiments will be highly complex raise new challenges in the field of control and automation systems demand well integrated, interoperable set of tools with a high degree of automation deliver and process data at a rate of up to hundred GBytes/s. deploy and integrate systems with different degrees of complexity and provenience 2

3 Large-scale experiments: control and data aquisition systems New projects prominently feature solutions adopted from other laboratories, hardware and software standards industrial solutions. Are they any different from commercial systems? Although sharing a large degree of architectural commonality, given their unique requirements, traditionally, such systems have been purpose built. significantly greater amount of I/O capability required between computational elements unique and disparate I/O requirements imposed on their interfaces 3

4 Large-scale experiments: R&D on control and data aquisition R&D activities target Self-triggered front-end electronics with adequate output bandwidth and data processing MIMO controllers with efficient resource sharing between control tasks on the same unit Massive parallel computing capabilities. Commercial technology will likely meet the basic requirements on which physics experiments can leverage for building future control systems 4

5 Large-scale experiments: control and data aquisition (more) challenges But future systems are envisioned to be more than an order of magnitude larger than those of today More challenging will be providing a robust, fault tolerant, reliable, maintainable, secure and operable control systems 5

6 Is ITER any different? ITER CODAC is a challenging endeavour ITER will generate a huge quantity of experimental data 150 plant systems diagnostic channels slow control channels fast control channels 40 CODAC systems 5 Gb/s data 3Pb/year data (e.g. 12 IR cameras in a 10 minutes discharge: Tbytes) 6 In addition... ITER will require a far higher level of availability and reliability than previous/existing tokamaks.

7 Aim of Control Systems in fusion: JET as an example JET has a wide range of Plasma Control Systems PF Coils GAS + Pellets NBI ICRH LHCD TAE / EFCC Wall Load Coil Protection Shape & Current Control (PPCC) plasma Simulink code R-T Controller R-T Signal Server Magnetics Interferom/Polarim Neutron X-ray etc. ECE Te (R) CXS Ti (R) LIDAR Ne&Te(R) MSE pitch (R) X-ray Ti (0) Comms network ATM, some analogue VUV impurities Vis Da, Brem, ELM Vis H/D/T Confinement q profile Flux surfaces EQX EQX kinetic map Pale blue = Diagnostic, Sky blue = Analysis, Red = Heating / Fuelling / Magnets & Power, Yellow = PPCC (XSC), Green = RTMC To reach and reproduce scenarios which cannot be programmed Quasi-Steady State Experiments Magnetic and Kinetic Profile & ITB Experiments Mode Conversion Experiments Radiation and Impurity Experiments MHD Experiments To protect Plasma and Machine NB Shinethrough (WALLS, PEWS and NBLM) LHCD Launcher (Monitor Iron and Radiation) Avoid Disruptions Mimize waste (Neutrons, Tritium) 7

8 Control and data acquisition in fusion: JET Vertical Stabilization as example Elongated plasmas are vertically unstable If no corrective action is taken the plasma column moves vertically until it reaches the vessel protecting tiles. The plasma then rapidly shrinks and eventually disrupts discharging all its energy to the machine structures imparting large impulsive forces. Forces of hundreds of tons were measured at JET VS system designed to make the plasma vertically stable so that the Shape Controller can successfully control the plasma position and shape. 8

9 Control and data acquisition in fusion: ITER Vertical Position Control Loss of vertical plasma position control in ITER will cause thermal loads on PFC of MJ/m 2 for ~0.1s. PFCs cannot be designed to sustain (repetitive) thermal loads like those quoted above. VDE also generates the highest electromagnetic loads A phenomenological extrapolation of horizontal forces from worst JET cases implies horizontal loads ~45MN on ITER vacuum vessel. MHD wetted kink model developed to simulate the horizontal loads predicts ~20MN. Vertical loads ~90MN Plasma vertical position in ITER must be robust & reliable to ensure a vertical plasma position control loss is a very unlikely event 9

10 JET Vertical Stabilization: Specifications of the MIMO architecture Reduction of loop delay on the signal acquisition/generation endpoints (down to 10 µs) on the data interconnect links from and to the processing unit on the analogue signal path (analogue filters); High processing power 10 on the acquisition/generator endpoints on the system controller; improvement of MIMO algorithm performance Synchronization of all digitizer/generator endpoints; Architecture designed for maintainability, upgradeability and scalability; Low cost per channel; Low risk of implementation and testing.

11 ATCA: a possible solution ATCA is the most promising architecture to substantially enhance the performance and capability of existing standard systems It is designed to handle tasks such as event building, feature extraction high level trigger processing. TeraOPS of processing power in a single sub-rack. First commercial open standard designed for high throughput High availability High interest to data acquisition physics and attractive for experiments requiring a very high up-time 11

12 Why ATCA? ATCA platform is gaining traction in the physics community because of Advanced communication bus architecture (serial gigabit replacing parallel buses) very high data throughput options and its suitability for real-time applications Agnostic backplane that accepts several serial switch network protocols Scalable shelf capacity to 2.5Tb/s Scalable system availability to % Robust power infrastructure (distributed 48V power system) and large cooling capacity (cooling for 200W per board) High levels of modularity and configurability Ease of integration of multiple functions and new features The ability to host large pools of DSPs, NPs, processors and storage The ability to host multiple controllers and storage on a shelf High security and regulatory conformance Reliable, full redundancy support Reliable mechanics (serviceability, shock and vibration) Hardware management interface 12

13 JET Vertical Stabilisation Vertical Stabilization Controller (input signals >190): x86-based ATCA controller Up to 12 DGP cards (PCIe links through the ATCA full mesh backplane) Each board has bit ADC channels, each separately isolated Parallel execution on FPGAs for MIMO signal processing (Control loop delay < 50 µs, aim < 10 µs) Freeware operating system RTAI The Aurora and PCI Express communication protocols allow for data transport, between modules, with expected latencies below 2 µs. 13

14 JET Vertical Stabilization 192 input signals Front view Rear view 14

15 JET Vertical Stabilization: the star of the show 15

16 Does it work? VS controller in action - Vertical kicks experiment 16

17 Who else is using ATCA? The group of experimenters includes several major laboratories representing different fields of use and a range of applications. Active programs are showing up most notably at DESY for XFEL and JET Other laboratories ILC, IHEP, KEK, SLAC, FNAL, ANL, BNL, FAIR, ATLAS at CERN, AGATA, large telescopes, Ocean Observatories Investigating ATCA solutions for future upgrades Both the CMS and ATLAS detectors Setting up prototype experiments to test its potential ILC and ITER ATCA is being adapted without significant change as a platform for generic data acquisition processors requiring high throughput and bandwidth. Most of these programmes put the emphasis on High Availability 17

18 Control and Data Acquisition in fusion: towards ITER relevant solutions VME PCI ATCA (R&D) ISTTOK TCV TJ-II TCA/Br MAST JET ETE JET ISTTOK Compass ITER JET ITER LIDAR conceptual design Control and Data acquisition activities build upon Fusion specific needs 18

19 Finding the bridge with NI ATCA based powerful control and data acquisition platform leveraging on Intel multicore technology for high performance and low cost development and LabviewRT Multicore flexibility 19 Author s xx, 2007 B. Gonçalves Paris, Februaryname 2, 2009 Place, NI - Big Month Physics Round Table Event

20 High performance Plant System: possible implementation 20 LabView RT programming tools Experiment - Sensors - Actuators 2 U Gigabit ethernet 26 U 14 U PCIe optical links High performance networks NI Controller Multicore Processor Blade ATCA shelf High Performance System 2 U User ATCA-MIMO-ISOL ISOL 32 ADC 8 DAC 8 DIO 2MSPS (alternative) SR-TR-ATCA 8 ADC bit bit bit NI multicore processor controller ATCA ADC/DAC/DIO boards with MIMO capability at different data rates LabviewRT programming tools Local control and autonomous operation Local data storage Timing, (synchronous) messages and high performance network interfaces ATCA will add the required robustness to the platform

21 Who could benefit from it? Accelerators and Detectors Fusion Reactors and Experiments Light sources and Experiments Astrophysics Instruments and Experiments Astronomy Instruments and Experiments Medical Imaging Instruments and Experiments Nuclear Physics Instruments and Experiments But, vertical solutions of great complexity are only a subset of the potential market 21 ATCA is suitable for horizontal Control and Data Acquisition solutions

22 What exists? Ad Hoc Committee on xtca for Physics Advance Call for Participation to Physics Community ongoing The many options available in ATCA and AMC hardware and software require making a down-selection of standards for specific user-supplier communities advantages of a common standard inter-operability of functional modules and support systems. User-supplier groups to identify and adopt best choices for their particular group. The Physics community encompasses many different kinds of applications but generally builds very similar controls and data acquisition systems with similar requirements 22 the best interests of this community will be served by standardizing on a subset of the many available options. best done in a Working Group collaboration including designers of special functional modules, system designers, users, and industry providers who make the finished designs available to the entire community at reasonable cost.

23 Closing remarks These days, building the best system isn t enough. That s the price of entry. Ann Livermore, HP The remaining price should be paid on robustness fault tolerance reliability maintainability Security operability 23 ATCA fits the bill!

24 Backup slides 24

25 Market trend Convergence of computer systems and communication technologies are moving to switched highperformance modular system architectures on the basis of high-speed switched interconnections Traditional parallel bus system architectures (VME/VXI, cpci/pxi) are moving to new higher speed serial switched interconnections. Traditional bus architectures have a relatively straightforward programming model, but they have limited effectiveness in multiprocessor systems, especially when a low-latency, deterministic response is required. Bandwidth is one limitation of bus implementations, but even more important is contention between multiple processors for use of a shared bus. Predictable, deterministic response times are not possible when processors must wait to access a bus. Switch fabric architectures offer a much better basis for multiprocessor systems, and provide several performance and usability benefits. Several high-performance switch fabric standards have been developed. PCIexpress, 10 Gigabit Ethernet, and RapidIO are the most viable choices for high availability and high speed applications better overall backplane throughput with low-latencies and deterministic delays. 25

26 Issues for Physics xtca Standards While some specific issues can only be discovered over time by implementing new applications, some are apparent and need to be addressed immediately. What will be the standard communications protocols for ATCA and AMC modules? There are seven possible protocols that could be supported. The answer is probably Ethernet at the shelf level and PCIe at the backplane level, but this needs to be decided. Can some of the AMC data transmission lines be re-allocated for analog or digital I/O use? What strategies are possible to achieve high or low bandwidth rear interconnects in the ATCA RTM area? In the MicroTCA package? Note that rear entry is most desirable to support hot-swap of both ATCA and AMC cards. What features of Shelf Management IPMI need to be implemented besides the standard ones that support power management and hot-swap? Analog and digital modules for data acquisition and control have features that may need special monitoring and management. How can we best implement high precision timing and synchronization in an xtca shelf? What scalable redundancy strategies need to be employed in controls systems versus data acquisition systems? How can we implement xtca standard power system strategies and shelf management inside of large detectors? Can we develop a few generic FPGA-based data acquisition and control card along with a common firmware kernel and EPICS support that can easily adapt to various module functions such as ADC s, DAC s, power supply control, beam monitoring instruments, etc.? 26

27 JET: Gamma ray spectroscopy Intelligent modules (with FPGAs) are used for real time pulse processing: Pulse height analyzer; Pile-up rejection; and Pulse shape discriminator 27 Digitizer modules centered around an FPGA, which: controls ADCs, Controls local memory, provides the gigabit interconnections and runs DSP algorithms Concurrent algorithms can be implemented on the FPGA and each one can be parallelized (e.g. 4 pipes at 250 MSPS 1 GSPS) Data reduction rate of at least 80% attainable with pulse height analysis

28 ATCA: Controller 28