Optimization of HOM Couplers using Time Domain Schemes Workshop on HOM Damping in Superconducting RF Cavities Carsten Potratz Universität Rostock October 11, 2010 10/11/2010 2009 UNIVERSITÄT ROSTOCK FAKULTÄT INFORMATIK UND ELEKTROTECHNIK
Overview Introduction Comparison of selected numerical time domain schemes Application Example: Optimization of the filter characteristics of a preliminary HOM coupler design with SPL dimensions. Conclusions 2
Introduction - Numerical Optimization Design Results Geometrical parameters Model Simulation scattering properties thermal load... new set assessment optimization algorithm Criterions met? Optimized Design Simulation is (in general) the most time consuming part Use as few simulations as possible and a suited numerical scheme! 3
a(t) Time Domain Computation of S-Parameters #1 b(t) response excitation b(t) response t b(t) response t t t 4
b(t) Time Domain Computation of S-Parameters #2 s21db 20 fhz 1.0 10 9 1.5 10 9 2.0 10 9 2.5 10 9 3.0 10 9 a(t) t FFT Normalization 40 60 80 100 120 140 args21 3 2 1 t 1 2 1.0 10 9 1.5 10 9 2.0 10 9 2.5 10 9 3.0 10 fhz 9 3 5
Introduction Comparison of selected numerical time domain schemes Application Example: Optimization of the filter characteristics of a preliminary HOM coupler design with SPL dimensions. Conclusions 6
Numerical Schemes - Stencil Approach FDTD/FIT Commonly used on regular grids (orthogonality!) Compute fields/fluxes at a location by surrounding fields/fluxes (linear operators L1,L2) Explicit update equation for discrete field vectors (e,h): d e dt h = L1 (e,h) L 2 (e,h) HOM section and cartesian grid Explicit but limited geometric flexibility. 7
FEM Approach HOM section discretized unstructured tetrahedral grid (coarse) Allows for unstructured grids => suited for complex curved structures and a reasonable number of elements/dofs Project fields on a finite function space, compute inner product with test functions (commonly used approach: Galerkin) Problem: Leads to implicit semi-discrete formulation (unless problem is sufficiently small and mass matrices can be inverted) d dt M e 1 M 2 h = S h e Global matrices Geometrical flexibility but implicit. 8
Discontinous Galerkin (DG) - FEM Approach #1 Allows for unstructured grids => suited for complex curved structures Support of basis and test functions is limited to the individual corresponding elements Adjacent elements connected by boundary fluxes All matrices defined element wise => small Matrix inversion is feasible => explicit Parallel by design d e dt h = M 1 S ε 1 h µ 1 e + M 1 F ε 1ˆn f H µ 1ˆn f E HOM section discretized (12k elements, second order) element wise matrices coupling of adjacent elements Geometrical flexibility and explicit. 9
Discontinous Galerkin (DG) - FEM Approach #2 Our S-Parameter code is based on NUDG* framework NUDG* framework (open source) implements basic operators, time integrator,... We added: boundary conditions (broadband waveguide excitation and absorption), improved PML, modal analysis,..., pre- and post processing => on graphic card (GPU - NVIDIA CUDA based) HOM section discretized (12k elements, second order) Why graphic cards? * www.nudg.org 10
Why GPUs? Modern GPUs heavily outperform modern CPUs! Up to 1.5 TFlops / GPU (single precision) cheap (350 /Unit) Highly scalable => multiple GPU/Workstation or GPU clusters Well suited for highly parallel algorithms like Discontinuous Galerkin FEM Comparison theoretical GFLOP/s GPU vs. CPU (source: NVIDIA ) 11
Introduction Comparison of selected numerical time domain schemes Application Example: Optimization of the filter characteristics of a preliminary HOM coupler design with SPL dimensions. Conclusions 12
Application Example: HOM coupler with SPL geometric properties Coax (without connector) Port 1 Tuning of filter characteristics beam pipe Notch effect @ 704.4 MHz (SPL fundamental mode) Port 2 Port 3 Which scheme to use for optimization? (Preliminary) Model of a HOM coupler/beam pipe section with SPL specs. 13
Filter Characteristics Logs21dB 0 fhz 1.0 10 9 1.5 10 9 2.0 10 9 2.5 10 9 3.0 10 9 20 In this example: region of interest 40 60 The interesting things happen below cutoff. 80 100 120 140 TM01 fcut Filter effect for different geometric parameter combinations Selected transmissions from TM01 to TEM (computed with DG-FEM) with different geometric parameters 14
First Simulation - CST MW Studio Port 1 LogsdB fhz 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 20 40 Port 2 Port 3 60 80 s31 100 120 s21 HOM section discretized (528k meshcells total, first order) Transmission of TM01 to TEM, Computational Time 1700s on 4 Cores @ 3.2 GHz 15
Then - GPU Accelerated DG-FEM (NUDG*) Port 2 Port 1 LogsdB 0 20 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz Port 3 40 60 80 100 120 s31 s21 Transmission of TM01 to TEM, Computational Time 400s on GPU GTX 470 HOM section discretized (12k elements) *www.nudg.org 16
LogsdB Comparison of both schemes (accuracy vs. time) 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz 20 40 Results of well established commercial code MWS matches DG-FEM computed S- parameters very well. 60 4 graphs! Match very well. 80 100 120 400 s (GPU) vs. 1700 s (4 CPU Cores) Transmission of TM01 to TEM, Comparison of computed transmissions Speedup by a factor of 4 => use DG-FEM for further optimization! 17
Interlude - Further Speedup? MWS: Use more Cores? 1700 s (4 CPU Cores) => 1610 s (8 CPU Cores) (Performance is limited by memory bandwidth, not arithmetic operations count!) DG - FEM: - Local timestepping => Implementation in development (Expected gain 2...3) - Multiple GPUs => coming soon: GPU Cluster } Reduces optimization time by at least one magnitude 18
Text example #1 Logs21dB 0 20 40 60 80 100 120 140 Logs31dB 0 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz Improved coupling, but detuning! 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz 20 40 Lowering the hook in the beam pipe 60 80 19
Text example #2 Rotation around axis Logs21dB 0 20 40 60 80 100 120 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz 140 0 45 90 Logs31dB 0 fhz 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 20 40 60 80 100 120 140 180 135 90 20
Text example #3 r Logs21dB 0 20 40 60 80 100 120 140 Logs31dB 0 20 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz r 8.0 10 8 1.0 10 9 1.2 10 9 1.4 10 9 fhz Scaling of the hook s radius 40 60 80 r 100 21
Conclusion Systematic optimization (= extensive parameter sweeps) is required for HOM coupler design Therefore, a suited numerical scheme is essential Based on an (preliminary) application geometry: S-Parameters computed with DG-FEM are in very good agreement with well established code (MWS) Computation time (and thus optimization time) can be reduced significantly by GPU accelerated DG-FEM 22