Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes

Transcription

1 Advanced Methods for Space Simulations, edited by H. Usui and Y. Omura, pp c TERRAPUB, Tokyo, Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes Daniel W. Swift Geophysical Institute, University of Alaska, Fairbanks, Alaska , U.S.A. This paper describes the elements for writing hybrid and electromagnetic plasma simulation codes in generalized curvilinear coordinates. The coordinate system is described by a table giving the three-dimensional Cartesian coordinate positions of a structured curvilinear coordinate mesh. The paper shows how to calculate the geometric coefficients of the differential operators describing the electromagnetic field and how to calculate the motion of particles across the curvilinear grid. In addition, a new algorithm for calculating the particle current sources for the electromagnetic field is described that conforms naturally to the curvilinear grid. Measures are described for checking and ensuring accuracy of the code. Extension of the methods for multiple, discontinuously joined coordinate patches is also outlined. 1 Introduction There are two main reasons for using curvilinear coordinates for space physics applications. One is to conform to the particular geometry of the problem. A good example is that of magnetosphere-ionosphere (MI) coupling. The Earth s magnetosphere is in the shape of a comet, but one set of boundary conditions may be in the ionosphere, which is a spherical shell. It might therefore be desirable to formulate coordinate system with a spherical coordinate surface near the Earth, but which gradually deforms to roughly to a plane face of a box containing the simulation domain at large distances. Figure 1 shows an example of such a coordinate system. The other main reason is to accommodate disparate size scales. Again in the case of MI coupling, one may want to resolve distances of tens of kilometers near the Earth. Yet processes driving field-aligned currents near the Earth take place many Earth Radii out in space, where it would be computationally expensive to maintain that resolution. Also, one might want a relatively high degree of spatial resolution across the plasma sheet and in the region of the bow shock and dayside magnetopause, yet low resolution in the large volume occupied by the lobes of the magnetotail. An example is shown if Fig. 2, which shows a global view in the midnight meridian plane of the dawn-dusk component of the magnetic field in the top panel. The bottom panel shows the field-aligned currents near the Earth. Much of this presentation is based on an earlier publication (Swift, 1996) where most of the techniques were introduced. The Swift (1996) paper, which focused on the hybrid code, also describes how to include a cold ionospheric population in the fluid approximation in the same spatial volume occupied by the discrete particles 77

2 78 D. W. Swift Fig. 1. An example of a coordinate system conforming to the surface of the Earth at the apex and deforming to a plane boundary at large distances. and how to subcycle the update of the fields to the particle push. This paper will, however, focus on the curvilinear methods for doing field and particle operations that are common to hybrid and electromagnetic codes and that can also be applied to MHD codes. For the field operations, we will describe operations such as taking the curl of a vector field, taking cross products and converting fields back and forth between curvilinear and Cartesian coordinates. We shall also describe how to trace the motion of a particle across a curvilinear grid and to calculate the current sources of particles. The last section describes extension of curvilinear methods to multiple, discontinuously joined coordinate patches. 2 Specification and Setup of the Grid The coordinate grid is specified by a table giving the x, y, z-positions of the coordinate points, q x (i, j, k, m), q y (i, j, k, m), q z (i, j, k, m), where the q s specify the x, y, z-positions of the curvilinear coordinate point denoted by the index i, j, k. With this formulation, the grid can be changed simply by changing the defining table. The last index, m takes of values of 1 or 2. The set corresponding to m = 1defines the location of the divergence of the magnetic field, B, and the other set of points is the location of the divergence of the electric field, E. Also, the particle charge density resides at the E-grid points. The grid contains no constraints to orthogonality, because it has proven difficult to generate a grid that is both orthogonal and adapts well to the geometrical constraints of the simulation. Figure 3 shows a grid cell and the orientation and position of the electric and magnetic field components. This particular cell is termed a B-cell, since the divergence of

3 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 79 Fig. 2. Above shows the dawn-dusk component of the magnetic field in the midnight meridian plane. The figure below shows a fine-scale view of the field-aligned currents near the Earth. Fig. 3. A coordinate cell showing position and orientation of curvilinear components of the magnetic and electric fields. B is at the center of the cell. The components of B are on the cell faces. The E-grid points are at the corners of the cell. If the center of the cell is located at the grid point i, j, k, then the E-grid points are located at i 1/2, j 1/2, k 1/2. In this picture, the

4 80 D. W. Swift Fig. 4. Schematic illustration of finite charge element carried by a particle on the grid. components of E are located on the cell edges. In a view showing a cell with the E-grid point at the center, the components of E would be on cell faces. The E- and B-grid points are nested so differences are automatically centered. 3 Field Operations Let us now look at how the field equations are set up. As a first example, consider the update of the magnetic field, B from Faraday s Law. B = E. t (1) Use Stokes Theorem to convert the curl operation to a line around a cell face. B = t E dl. (2) In curvilinear coordinates, B is represented by B = B 1 e 1 + B 2 e 2 + B 3 e 3 (3) where the e i s are unit vectors parallel to cell edges. Making use of (2) and (3), the curl operation can be evaluated. The result applied to the right-hand face of Fig. 1 is [( ) ] B 1 n+1 e 1 A 1 = ( E n+1 ) l 2 t i+ 1 2, j,k ( E n+1 ) l i+ 1 2, j,k ( E n+1 ) l 3 i+ 1 2, j+ 1 2,k ( E n+1 l 3 )i+ 1 2, j 1 2,k where (l 2 ) i+ 1 2, j,k+ 1 = r 2 i+ 1 2, j+ 1 2,k+ 1 r 2 i+ 1 2, j 1 2,k+ 1 2 (4)

5 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 81 where the grid point indices are specified by i, j, k and n is the time level. The x-component of l 2 can be computed from the table specifying the location of the coordinate points and is given by l 2x = q x (i, j + 1, k, 2) q x (i, j, k, 2). (5) The unit basis vectors, and the cell-face area, A 1 are given in terms of the tangent basis vectors l i, i = 1, 2, 3 are given by e 1 = l 1 /l 1, A 1 = l 2 l 3. (6) Using this formulation of the curl operation yields a vector that is exactly divergenceless. This can be seen from Fig. 1. The divergence at the center of the cell can be shown to be the sum of the line integrals around all the faces of the cell. The line integrals along cell edges on adjacent faces cancel each other. In a non-orthogonal coordinate system, there are two sets of basis vectors. One is the tangent basis vector, which is given above. The other basis set is normal to cell faces. Thus the vector in (3) can also be represented B = e 1 B 1 + e 2 B 2 + e 3 B 3. (7) Where e 1 l 2 l 3 = (l 1 (l 2 l 3 )) l 1, e i e j = δ j i. (8) Those who have some familiarity with tensor analysis will recognize the raised indices are related to contravariant components, while the lowered indices are covariant components. The difference is that we choose to use dimensionless unit basis vectors. It can be seen that the components of B with raised indices are cell-face normal components, while those with lowered indices are cell-edge tangent components. The component indices can be raised by taking the dot-product of the expression in (7) with the cell-face normal basis vectors in (8) and lowered by taking the dot-product of the expression in (3) with the cell-edge basis vectors in (6). The equations involved in the hybrid code require the taking of cross products of vectors with cell-face (raised index) components. For example, u B = [e 1 (e 2 e 3 )] [e 1 ( u 2 B 3 u 3 B 2) +e 2 ( u 3 B 1 u 1 B 3) +e 3 ( u 1 B 2 u 2 B 1) ]. (9) Notice, this cross product yields a vector with cell-edge components, as required for the curl operation. We also need to convert back and forth between Cartesian and curvilinear components. From the relation B = 1 x B x + 1 y B y + 1 z B z = e 1 B 1 + e 2 B 2 + e 3 B 3. (10) The components are easily derived, for example, B x = 1 x e 1 B x e 2 B x e 3 B 3 B 1 = e 1 (1 x B x + 1 y B y + 1 z B z ). (11)

6 82 D. W. Swift The above has provided a sampling of the types of operations that go into development of a hybrid code. All the coefficients are derived from the tangent vectors given in (4) and (5). 4 Particle Motion The particle phase space coordinates are the position specified in curvilinear coordinates and the velocity specified in Cartesian components. P = ( q 1, q 2, q 3,v x,v y,v z ). (12) Where the q s are the curvilinear position of a particle, and the v s are the Cartesian components of the velocity. Topologically, the curvilinear grid is a cubic lattice with unit distance between grid points. Thus, the index of the cell containing the particle is obtained by simply truncating the particle index. For example in FORTRAN notation I = INT(Q1) + 1, and similarly for the other indices. The velocity is represented in Cartesian coordinates in order to avoid explicit calculation of derivatives of the metric coefficients given (4). Those familiar with tensor analysis would recognize that those derivatives would comprise elements of the three-index Christoffel coefficients. The relation between the rate of change of the curvilinear position and the particle velocity can be derived from v = ( 1 x v x + 1 y v y + 1 z v z ) = ( l1 q 1 + l 2 q 2 + l 3 q 3). (13) From which the components of curvilinear velocity are readily obtained q i = ei (v ) x 1 x + v y 1 y + v z 1 z. (14) l i The Cartesian components of ei are obtained (8) making use of (4) and (5). 5 Field Sources The customary method of calculating field sources from particle positions and velocities is the particle-in-cell weighting in which a fraction of the particle charge (PIC) is assigned to each of the eight grid points on the corners of the cell. The field sources in both hybrid and electromagnetic codes are the electric currents derived from particle fluxes. This procedure yields Cartesian components of the flux at the cell corners, which has to be converted to cell-face normal curvilinear components. In addition, the current components have to be interpolated from cell corners to cell faces. Experience indicates this procedure works satisfactorily in a hybrid code; but this procedure is not very satisfactory in electromagnetic codes. The reason is that charge is not well conserved. This inaccuracy can result in an intolerable level of noise. In strictly Cartesian coordinates with regular boundaries, one can do a divergence clean on the electric field by inversion of the Laplace operator. This is very difficult in Curvilinear coordinates, where the Laplace operator has variable coefficients.

7 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 83 Villasenor and Buneman (1992) have proposed an algorithm that conserves charge exactly and that conforms naturally to a curvilinear coordinate grid and places the current on cell faces where it is needed in the field equations. The scheme is illustrated schematically in Fig. 2. As mentioned previously, the grid is topologically equivalent to a square or cubic lattice. Each particle carries with it a square or cubic charge element, and the particle transports this charge as it moves through the grid. The current is simply the amount of charge that moves across a cell face in a time step divided by the time step The current comes out in cell-face components exactly as needed for updating the field in curvilinear coordinates. The algorithm as described in the original Villasenor-Buneman (1992) paper describes a very complicated ten-branch logic tree. In the electromagnetic code, the time step is constrained by the Courant condition with respect to the velocity of light, and a particle is constrained by relativity from moving faster than the speed of light. Therefore, the number of cell boundaries a particle charge element can cross is constrained in the electromagnetic code. There is no such rigid constraint in the hybrid code. The hybrid code we have developed uses a modified and conceptually much simpler variation of the Villasenor-Buneman algorithm. In the subroutine where the particles are moved, the time step for each is broken into variable increments which end when a particle charge element encounters or leaves a new cell boundary. The current contributions in each of these increments is summed. In most instances, the particle will cross fewer than ten cell boundaries, so this represents a saving over the original algorithm, but the algorithm also has the flexibility to accommodate the occasional fast particle that cuts across cell corners. 6 Tests and Other Practical Considerations The most common field boundary condition used is the specification of two tangential components of the electric field and the normal component of the magnetic field. These conditions require that the E-grid points lie on the outer boundary. Thus the generation of the grid would begin with specification of the E-grid points. The B-grid point would then be placed in the center of cells formed by the eight E-grid points. Thus the advancement of B from Faraday s law would have second order spatial accuracy. The problem is that in a variable grid, the E-grid points are not necessarily in the center of B-grid cells, so the computation of curl B would not be second-order accurate. This can be corrected by a relaxation process of centering the E-grid points in the B-grid. Very close to second-order accuracy can be achieved by about two cycles of grid relaxation, while keeping the E-grid boundary points fixed. It can be seen that the accuracy is dependent on the smoothness of the grid, so it is wise to avoid sudden changes in orientation of the grid lines or changes in grid density. If the original has large changes, multiple relaxation cycles will serve to smooth the grid. Although the methods developed have accommodated non-orthogonal grids, experience has shown that the code performs poorly if the angle between tangent vectors becomes less than 45. There are several accuracy checks on the various modules that should be per-

8 84 D. W. Swift Fig. 5. A meridian-plane cut showing the use of multiple coordinate patches for a global-scale simulation of the magnetosphere. formed in the process of developing a curvilinear code. The most critical test consists of taking the a vector like A = 1x y. Convert it to a cell-face normal curvilinear representation. Then convert the result to a cell-edge tangent representation and take the curl. Next convert the result back to Cartesian. The result should be the unit vector 1z. Experience indicates satisfactory code performance with a cumulative accuracy of 1%. Another check is to trace the force-free motion of a particle across the grid. That should be a straight line. One can use the coordinate tables and PIC weighting to compute the instantaneous particle position in Cartesian coordinates that can be compared to an analytically calculated position. The Villasenor-Buneman conservative algorithm should be checked by tracing the density of a single moving particle. This is done by taking the divergence of the particle flux and integrating with respect to time. If the density is initialized to the density calculated by PIC weighting at t = 0, the result is the density calculated by PIC weighting of the particle charge at the final position of the particle. Finally, those who have worked with hybrid codes know they can be quite temperamental. Hybrid codes can become unstable or otherwise produce inaccurate results if values are not properly centered for second-order accuracy. One aspect that has been glossed over is that of interpolation that is necessary between parts of a cell in converting between cell-face and cell-edge components of a vector, and in converting between curvilinear and Cartesian representations. Care must be taken to maintain second-order accuracy. Second-order accuracy also requires calculation of dual sets of tangent basis vectors formed by separately differencing between E- and B-cell

9 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 85 Fig. 6. The coordinate grid on the ionospheric shell of the Earth. The center is occupied by the North polar patch, which is surrounded by the day, night, dawn and dusk patches. Notice the small overlap between patches. grid points so that the vectors are centered on the line between grid points. Cell-face areas must also be carefully centered on the cell face. 7 Extension to Multiple Coordinate Patches Figure 1 shows a coordinate patch suitable for simulating a segment of the magnetosphere that extends from the ionospheric outward to fifteen Earth radii. A truly global simulation would require the joining of several coordinate patches to cover the region surrounding the Earth and to include the magnetotail. Figure 5 shows how several of the coordinate patches shown in Fig. 1 may be joined for a global scale simulation. This figure shows a meridian plane cut of the dayside, nightside, north and south polar and tail patches. Not shown are the dawn and dusk coordinate patches. Six of the patches bound on the ionospheric shell. The multiple patch simulation is considerably more complicated than one based on a single curvilinear coordinate system. The major complication arises from the fact that the coordinate system undergoes discontinuous changes from one coordinate patch to another. This means that components of the field change in crossing a patch boundary. The reader will also note there is a greater density of grid points on the dayside patch than on the neighboring patches. This provision is added to capture details of the bow shock and magnetopause. Figure 5 shows the B-grid coordinate lines The outer B-grid coordinate surfaces must coincide with the outer B-grid coordinate surfaces of the neighboring patches. As mentioned in the previous section, boundary conditions are imposed on the surfaces defined by the outer layer of E-grid points. On the interface between patches, these points lie inside the domain of a neighboring patch. This overlap is illustrated in Fig. 6, which shows the E-coordinate grid on the ionospheric surface. Notice the

10 86 D. W. Swift small overlap between patches. Although the grid lines normal to the patch boundary do not necessarily coincide, the density of grid surfaces is constrained to be constant across patch boundaries, so that the outer E-grid surface nearly coincides with next to outermost surface on the adjacent patch. Boundary conditions are imposed by interpolating field data from points on the outer B-grid and the next-to-last E-grid points, called source points, to the outer E-grid points of the neighboring patch, which will be called target points. To impose inter-patch boundary conditions, it is first necessary to identify the closest source points surrounding a given target point. The strategy for doing this is to a large extent dependent on the details of the coordinate system and will not be described here. However, as mentioned above, the vector fields must be interpolated in both position and direction. To do this, we realize that a vector field can be expanded in Cartesian coordinates about a target point (x 0, y 0, z 0 ) E = 1 x [c 1 (x x 0 ) + c 2 (y y 0 ) + c 3 (z z 0 ) + c 4 ] + 1 y [c 5 (x x 0 ) + c 6 (y y 0 ) + c 7 (z z 0 ) + c 8 ] + 1 z [c 9 (x x 0 ) + c 10 (y y 0 ) + c 11 (z z 0 ) + c 12 ]. (15) The coefficients, c i may systematically be determined by computing the curvilinear components at twelve source points surrounding the target point. For example. E 1 = e 1 E (16) where e 1 is given by (8). Upon taking the scalar product of E in (15) with e 1 at the source point (x i, y i, z i ), we get the relationship E 1 = T 1x [c 1 (x i x 0 ) + c 2 (y i y 0 ) + c 3 (z i z 0 ) + c 4 ] + T 1y [c 5 (x i x 0 ) + c 6 (y i y 0 ) + c 7 (z i z 0 ) + c 8 ] + T 1z [c 9 (x i x 0 ) + c 10 (y i y 0 ) + c 11 (z i z 0 ) + c 12 ]. Where, for example, T 1x = e 1 1 x. This process is repeated for all three components (E 1, E 2, E 3 ) using four source points for each component. This generates twelve equations for the twelve coefficients c j. Care has to be taken that the source points form a non-degenerate tetrahedron encompassing the source point. The coefficients c i can then be found by matrix inversion. The target field component can then be determined by application of an operation like in (16) to the expression in (15). The coefficients c i depend only on the geometry, of the coordinate system which does not change with changing values of the source field. Therefore, it is possible to set up a simple linear relationship between the source point fields and target point fields, E i t E i t = j=1 H i j f s i. (17) Where f s is one of the E k source points, and the H s are precomputed.

11 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 87 There are instances in which the boundary conditions require edge-aligned components. In that case, one should take the scalar product of the expression in (15) with e i, as given in (6). One can similarly apply the expression in (15) to compute the coefficients c i if the source fields are known in terms of edge-aligned components. There is an additional complication when field target points are on the edge of a coordinate patch that is not part of an external boundary. In that case, source data must be taken from two coordinate patches simultaneously. However, the above-described interpolation procedures still apply. When a particle crosses a coordinate patch boundary, it s curvilinear coordinate position must be changed in a way that its actual spatial position remains unchanged. This requires a table lookup and interpolation in a manner not dissimilar to the procedure for identifying field source points with target points. The velocity components, being Cartesian, do not change. However, use of the Villasenor-Bunemann current algorithm requires care in order to maintain charge conservation. As mentioned above, there is an overlap between coordinate patches. Say, a particle is passing from coordinate Patch 1 to Patch 2. A particle enters the domain of Patch 2 before it leaves Patch 1. At this point the particle is placed in Patch 2, so it resides simultaneously in both patches and is tracked in both patches. It is finally eliminated from Patch 1 when it passes through the outer E-grid surface of Patch 1. There will also be instances when a particle passes close to a coordinate grid edge. In this case, the procedure allows particles to exist in three patches simultaneously. A global-scale hybrid code is of such a size that it requires use of a massively parallel computer. It is therefore logical that particle and fields residing in different coordinate patches be processed by separate processors. The coordinate system outlined in Fig. 5 consists of seven coordinate patches. It is desirable to use a much larger number of processors. The number of particles in each of these patches is significantly different. Since most computer time is devoted to pushing of particles, it is desirable to have the flexibility to independently partition the various coordinate patches into a varying number of segments to achieve particle balance among processors without. It should also be possible to independently repartition the coordinate patches without having to do any reprogramming. This is accomplished by doing the message passing is done in two stages. First, data is passed among segments interior to coordinate patches. This is easily done since within a patch there is a common coordinate system topologically equivalent to a cubic lattice. Next, each segment that resides on a coordinate patch boundary passes its field data to a buffer which handles data from all segments on the coordinate patch face. The field interpolation procedures described above for an entire coordinate patch face are carried out, and the data is passed to a buffer on a processor in the neighboring coordinate patch. Then processors for each of the segments on the patch face claim its own section of data. Similarly, particles crossing a patch boundary are put in a common buffer for an entire patch face and then passed to a buffer on for the adjoining patch. The particles are then distributed to segments according to their position. The testing procedure is much as described in Section 6. The curl of a vector field

12 88 D. W. Swift like A = 1 x y should yield a constant unit vector everywhere, including points on and near patch boundaries. The force-free trajectory of a particle should be a straight line, even when crossing coordinate patch boundaries, and the total charge carried by a particle should remain the same as it crosses coordinate patch boundaries. Finally, a word about diagnostics: Each processor writes out files containing field data within its spatial domain. Standard routines are available for converting the fields to Cartesian components. Commerical visualization packages contain modules for plotting volumetric data from unstructured grids. Many of these procedures for unstructured grids are impossibly slow for any grid size approaching grid points. Then there is the problem of collating data and preparing displays for regions that span volumes occupied by many processors. There are much faster procedures that take advantage of the structured nature of the curvilinear grid and integrate data from all processor domains within a predetermined volume. The number of desired Cartesian grid points and the resolution of the output grid can also be specified. A processing program can be set up to loop through the files written by the various processors. The program then loops through each of the curvilinear grid points represented in a processor file. For each curvilinear hexahedral cell, the program determines the maximum and minimum x, y and z-values of the box containing the curvilinear cell and then determines the indices of the Cartesian grid points within that rectangular box. The program then loops trough the indices of Cartesian points within the box and finds the curvilinear points closest to each Cartesian point. Next, it determines whether the Cartesian point is within the curvilinear cell. If it is, it then does a four-point linear interpolation from the closest curvilinear point and three neighboring corner points on the cell. An alternative version does an eight-point tri-linear interpolation utilizing all corner points defining the curvilinear cell. This procedure scales linearly with the number of points. Moreover, it has the flexibility to allow the user to render data in any predetermined box at any resolution. Thus t is possible to see details in the data near the ionosphere in one view and to look at larger scale structure in more distant parts of the magnetosphere. 8 Concluding Remarks Curvilinear coordinates are obviously more complicated than a Cartesian grid, and the computational resources needed to update the fields and push the particles is certainly much greater. Curvilinear coordinates have an advantage in large-scale problems where high spatial resolution is needed in only some regions. Curvilinear coordinates avoids the necessity of carrying a high density of grid points were the spatial resolution is not needed. Since the grid is represented by a table, the grid can be changed by changing the defining table without the necessity of reprogramming. This opens the way for adaptive meshes. However, a change in the underlying grid in the course of a simulation will require the repositioning of the particles and interpolation, and possibly re-orientation of the field components, to the old to new grid. The other and more flexible alternative to the curvilinear grid is the unstructured

13 Generalized Curvilinear Coordinates in Hybrid and Electromagnetic Codes 89 mesh. Curvilinear coordinates have the connectivity of a cubic lattice, so that that every cell knows its nearest neighbors and every particle knows which grid cell it is in. In situations of where there are many sharp boundaries, the unstructured mesh may be a necessity. However, the conceptual simplicity of the curvilinear coordinate offers a clear advantage in space physics problems where boundaries may be represented by a small number of smooth surfaces. Curvilinear coordinate techniques are now being extended to systems composed of multiple, discontinuously joined, coordinate patches. The multiple coordinate patches, among other things, makes it possible to cover the entire surface of the Earth in the coordinate grid without singularities. The multiple-patch technique is being used in a global-scale hybrid code model of the Earth s magnetosphere. Acknowledgments. This work was supported by the USA National Science Foundation under Grant Computer resources are provided by the Arctic Region Supercomputer at the University of Alaska. References Swift, D. W., Use of a hybrid code for global-scale plasma simulation, J. Comput. Phys., 126, , Villasenor, J. and O. Buneman, Rigorous charge conservation for local electromagnetic field solvers, Comput. Phys. Commun., 2, , 1992.