Anisotropic Patch-Based Adaptive Mesh Renement for Finite-Volume Method

Transcription

1 Anisotropic Patch-Based Adaptive Mesh Renement for Finite-Volume Method Joshua Christopher, Xinfeng Gao and Stephen M. J. Guzik Computational Fluid Dynamics and Propulsion Laboratory Colorado State University, Fort Collins, CO 80525, USA We propose an anisotropic, patch-based adaptive mesh renement algorithm for the nite-volume method solving partial dierential equations on Cartesian and mapped grids. For large ow gradients, such as shock waves and ame fronts, that are aligned with one grid direction, anisotropic renement can provide a similar reduction in error using much less grid cells compared to isotropic renement. To enable anisotropic renement, additional logistics are needed to track anisotropic patches, and new operations for transferring data between patches of anisotropic renement are introduced. In this study, we focus on developing these new operators associated with a single-level grid and explore the solutions to issues associated with the operators across grid levels. For example, methods for tagging regions for anisotropic renement and modications to the Berger-Rigoutsos grid generation algorithm are considered. The savings in cell count are expected to more than oset the extra program logistics. Validation of the anisotropic algorithm is achieved by comparison to the existing isotropic algorithm for an unsteady shock problem. I. Introduction sotropic adaptive mesh renement has been established as a means to reduce the computational costs Iwhile maintaining accuracy through ner spatial discretization in regions of large solution gradients. 1 The proposed patch-based anisotropic adaptive mesh renement (AMR) methodology expands upon established isotropic AMR algorithms, allowing for an algorithm to rene computational grids in individual directions. Our motivation primarily stems from a need to regularize a moving mapped grid that becomes either highly compressed or highly stretched in one direction, such as a mesh representing the volume of a piston-engine combustion chamber. As a starting point, in this work we explore the performance gain from anisotropic renement of ow discontinuities. We predict anisotropic AMR will provide further computational savings by reducing the number of cells needed to resolve grid-aligned ow features such as shock waves and ame fronts. Methodologies for block-based anisotropic mesh renement have been described by Williamschen 2 and Zhang, 3 in which they have shown a signicant reduction in the number of cells used. Williamschen and Zhang both expressed concerns about the accuracy of the solution, suggesting that the criteria used for anisotropic renement marking needs to be more sophisticated. Cell-based anisotropic mesh renement has been described by Ham et al., 4 who claim signicant reduction in the number of cells used and an even more accurate result in the shock cube test than with isotropic results. In this study, we will focus on patchbased anisotropic renement. Unlike block- and cell-based AMR, patch-based AMR does not require a tree data structure to describe connectivity of patches, instead determining connectivity through patch indices. Additionally, block-based AMR embeds its more rened blocks, whereas patch-based AMR overlaps more rened patches, which leads to more complex conservation enforcement. On the other hand, patch-based renement does allow for closer tting of rened regions to ow features than block-based methods, requiring fewer rened cells. 5 Graduate Research Assistant, joshuac@rams.colostate.edu, Student Member AIAA Assistant Professor, xinfeng.gao@colostate.edu, Member AIAA Assistant Professor, stephen.guzik@colostate.edu, Member AIAA 1 of 16

2 Figure 1. An isotropic nested grid mesh renement. The goal of this research is to implement the anisotropic methodology into Chombo, 6 a parallel AMR library developed at Lawrence Berkeley National Laboratory. As such, Chombo's infrastructure and methodologies are used. However, Chombo was designed from its inception for isotropic grids, and so complications arise from interactions between dierently rened patches and between mesh renement levels. Therefore, additional functionalities are required on nearly all core components of the base library to support anisotropic grids. II. Isotropic Patch-Based Adaptive Mesh Renement First we briey review the existing isotropic patch-based AMR in order to facilitate the discussions on the new operators. We make use of the Chombo library for parallel AMR 6 and follow the strategies used therein. The problem domain is discretized using a grid Γ Z D that is a bounded subset of the integer lattice dened by the points (i 0,..., i D 1 ) = i Z D marking the cell centers of the control volumes. The control volumes take the form ( ) V i = ih, (i + I)h, i Z D, (1) where h is the grid spacing, Z D is the integer space, and I is dened as the vector whose components are all equal to one. Adaptive mesh renement calculations are performed on a hierarchy of nested meshes Ω l Γ l, with Ω l C n l ref (Ω l+1 ) where n l ref denotes the renement ratio between levels l and l + 1 and C is a coarsening operator. The grid levels are considered as overlapping as opposed to embedded as in the case of block-based AMR. At level l, we label all cells inside Ω l as being valid and all cells outside Ω l (e.g., ghost cells) as being invalid. We assume that there are a sucient number of cells on level l separating the level l + 1 cells from the level l 1 cells such that interpolations to ll invalid ghost cells on ner levels can be independently performed. We will refer to grid hierarchies that meet this condition as being properly nested. Figure 1 demonstrates a properly nested hierarchy of three levels. Typically, Ω l is decomposed into a disjoint union of rectangles, called patches, in order to perform calculations in parallel. Any relationships between patches on the same level, or between dierent levels, are known simply through the vectors describing the corner locations on the integer lattice. Consequently, there is no need for tracking connectivity between patches (although data-motion patterns are cached when the grids change for better eciency). The typical work-ow for advancing level l on an isotropic mesh is: 1. Regrid levels ner than l if required. This involves: (a) Tag all cells which should compose the ner levels, often based on the magnitudes of solution gradients. 2 of 16

3 (b) Construct a new, properly-nested mesh hierarchy. (c) In regions where new ne cells appear, the solution is interpolated from the coarser level. (d) In regions where ne cells persist, copy the solution from the old mesh hierarchy. 2. Advance level l with a chosen time-stepping algorithm. In this work, a classical fourth-order Runge- Kutta method is used. 3. Interpolate to the invalid ghost cells surrounding level l+1. The interpolation need not be conservative because the resulting values in the ghost cells are only used to reconstruct the ux on the faces of the valid cells. 4. Start level l+1 at step 1. Level l+1 is rened in time (sub-cycled) with a time step t l+1 = t l /n l ref. 5. Average the solution from l + 1 and correct uxes at coarse-ne interfaces to ensure conservation. 7, 1 Considerations required to revise each of the steps listed above to support anisotropic AMR include: 1. Regrid ner levels: (a) Tagging operations now need to supply separate list of tags for each spatial dimension. (b) The AMR mesh generator is modied to consider the direction-based tagging. (c) A region with persistent ne cells may change renement orientations, requiring a non-trivial copy. 2. Advancing the level in time requires exchanging valid ghost cell data between dierently rened patches, requiring a resolution scheme to change the data from one renement orientation to another. On an isotropic grid, exchanges are a simple copy. 3. Interpolating the invalid ghost cells surrounding a patch on level l + 1 adds the possible complication of variations in resolution of coarse-grid cells. 4. Averaging the solution to coarser levels requires adjustments for anisotropic renements, and ux corrections at the interface between dierent cell resolutions. In addition to the above, several modications must be made to accommodate anisotropic operations on a single grid level. These are discussed in Section III. Section IV focuses on data transfer between anisotropic levels. The results are presented in Section V, and conclusions are drawn in Section VI. III. Operators Associated with Anisotropic Single-Level Grid in Space As mentioned earlier, there is a major dierence on the single-level grid operators between the anisotropic and isotropic algorithms. New concepts and operators, such as per-patch ratios and ghost structures to facilitate data transfer, must be introduced for the anisotropic single-level grid operations. They are described in detail, along with the implementations considered. III.A. Patches and Ratios As in the case of isotropic renement, anisotropic patches are known by vectors describing their corners on the integer lattice, and are organized into levels by their renement. The patches in an isotropic mesh on the same level all have the same renement ratio n l ref. An anisotropic mesh can have patches on the same level in many dierent orientations, from fully anisotropic in one direction to fully isotropic renement and any possibility in between. As such, an anisotropic patch is dened as belonging to a level l if is rened to the resolution of the level, given by the single integer l 1 k=0 nk ref, in at least one direction. From a temporal perspective, all patches on a level advance with the same time step t l. Due to the dierent patches on the same level having dierent renement orientations, there is a need to track each patch's renement level independent of other patches on the same level. Each patch stores its renement in an integer vector of length equal to the number of spatial dimensions. The renement ratio vector, n l ref, measures the ratio of renement in each direction relative to the base grid. The base grid is the coarsest grid Ω 0 and is isotropic. 3 of 16

4 y y y 3 2 (2,2) 3 2 (5,2) (2,5) 1 0 (1,1) x 1 0 (2,1) x (1,2) x (a) Isotropic patch to be rened. (b) Patch rened in x. (c) Patch rened in y. Figure 2. The dierent indices and integer lattices of the same patch rened in dierent orientations. A renement ratio of 1 in a direction would denote the same resolution as the base grid; a ratio of 2 would indicate twice as many cells as the base grid, and so forth. The integer lattice that underlies an isotropic patch becomes stretched when an anisotropic renement is applied in order to maintain one cell per Cartesian coordinate. When a patch has an anisotropic renement or coarsening operation applied to it, the indices of the patch are adjusted to t the target anisotropic lattice, as shown in Figure 2. An isotropic patch spanning cells (1, 1) to (2, 2) is shown in Figure 2a. If the isotropic patch is rened in the x-direction, the underlying integer lattice still marks one cell per integer and the cells describing the corners of the patch are rened to (2, 1) and (5, 2) as shown in Figure 2b. If instead the patch is rened in the y-direction, the cells describing the corners are rened to (1, 2) and (2, 5) as shown in Figure 2c. Frequently, operations between two patches of dierent renements ratios occur, which then requires a method to resolve the diering ratios. The resolution schemes, detailed in later sections, often need to nd the least coarse isotropic ratio to which both patches can average their data. This isotropic ratio is called the common coarse ratio for these two patches, and is determined by nding the minimum renement for each direction. This is described by n coarse ref,d = min( n A ref,d, n B ref,d), which compares the ratios of patches A and B in direction d. Likewise, some operators may require the patch indices in isotropic form. If both patches are on the same level, they share the same common ne ratio. Converting an anisotropic renement ratio to an isotropic renement ratio is done by applying the maximum renement in any direction to all directions with n ne ref = max( n ref,d ). Computing the solution of cells on a level l with anisotropic grids is largely the same as isotropic grids. The anisotropic grids use an integer lattice in the same way as the isotropic grids, with one cell per coordinate. Additionally, each patch is able to independently solve the governing equations over its domain once the ghost cells are lled. As such, the application of the governing equations to anisotropic patches is the same as isotropic patches, with the exception that dx dy dz. III.B. Neighbor Patch Finding and Disjointedness Due to the eect of anisotropic renement on the integer lattice, adjacent patches may not have adjacent indices as demonstrated in Figure 3. This causes problems with the neighbor patch nding routines, as they rely on searching through adjacent indices to nd adjacent patches. Since all patches on the same level must share the same maximum renement (otherwise they would be on a dierent level), they can temporarily be rened to their isotropic state before comparison. This operation puts the two patches into the same coordinate space and allows for utilization of the existing neighbor nding algorithm. 4 of 16

5 y (5,2) (4,5) 2 1 (2,1) (3,2) Figure 3. Non-adjacent indices for an adjacent anisotropic patches. x In addition to nding neighbors, this scheme is used to ensure that patches on the same level do not overlap, which is important to the conservation scheme of the algorithm. In order to check this, sets of patches are checked for disjointedness before being used. Disjointedness is determined by the comparison of the low and high indices of the patch, where if the x-direction of the low index of a box is greater than the x-direction of the high index of another box then the patches do not overlap, and so on for each coordinate direction. With anisotropic patches, there are cases where patches rened in dierent directions may appear to overlap under the existing comparison scheme, when in fact they do not. This case is also shown in Figure 3, where the left patch has a high x-coordinate of 5 and the right patch has a low x-coordinate of 3, despite being adjacent and not overlapping. To correct for this case, the patches are again rened to the isotropic renement of their level before comparison. III.C. Averaging Operator from Fine Cells to Coarse Cells Averaging is a process in which multiple rened cells have their nominal values averaged into a single coarse cell. First, the renement ratio dierence is calculated from the renement ratios of the coarse patch and ne patch. The renement ratio dierence denes the number of cells the ne patch has in each direction relative to the coarse patch, and is given by n = n l+1 ref /nl ref patch, and n l ref the renement ratio dierence,, where nl+1 ref is the renement ratio of the ne is the renement ratio of the coarse patch. From there the scaling factor is the product of S = D n d, d=1 where D is the number of spatial dimensions. It should be noted that the scaling factor is the number of ne patch cells overlapping each coarse cell. The value of the coarse cell is then ϕ l = 1 S S i=1 ϕ l+1 i. (2) Equation 2 is repeated for each coarse cell that is being averaged to. When working with cell-averaged data, the averaging operator is exact and introduces no error. III.D. Interpolation Operator from Coarse Cells to Fine Cells Linear interpolation is a process in which the solution data on a ner patch is reconstructed from a coarser patch. To start, the dierence between the renement ratios of the coarse and ne patch is calculated. The renement ratio dierence denes the number of cells the ne patch has in each direction relative to the coarse patch, and is given again by n = n l+1 ref /nl ref. A piecewise constant interpolation of the coarse data to the ne data provides an initial estimate of the ne patch's data. The linear interpolator will provide a correction in a later step. The constant interpolation visits each cell in the ne patch and copies the value from the overlapping coarse patch. 5 of 16

6 Needed ghost cells Figure 4. The isotropic patch on the right requests two layers of ghost cells. Once the constant interpolation step has nished, the slope of the reconstructed function is calculated at each cell of the ne patch. In the middle of the domain a centered slope is used, and near the boundaries a one-sided slope is used. The slopes are calculated for each dimension, so for dimension d the centered slope is calculated by ϕ l i,d = 1 ( ϕ l 2 i+d ϕ l ) i d, where ϕ l i,d is the slope of the coarse patch at location i in direction d, and ϕl i+d is the solution data from the course patch once cell oset from location i in direction d. The one sided slope for the low-side is given by ϕ l i,d = ϕ l i ϕ l i d, and the high-side is ϕ l i,d = ϕ l i+d ϕ l i. After the slope has been determined for each location, the correction may be added to the ne patch. Each cell in the ne patch is visited, and the dierent cell spacing, h d, for each direction is calculated from h d = ( n d 0.5) n d, with n d being the value of the dierence ratio in direction d. Finally the correction is added in to ne patch with ϕ l+1 i = ϕ l+1 i + h d ϕ l i, where the ϕ l+1 i III.E. is the constant interpolation value from earlier in the linear interpolation operation. Exchange Operator for Filling Ghost Cells on the Same Level When patches on a level Ω l have renements of dierent orientations, exchanging valid ghost cells becomes more complex. Transferring data from one arbitrary renement orientation to another requires averaging to a temporary common coarse patch before interpolating to the destination patch. In addition there may be multiple source patches for a given set of ghost cells, so the common coarse patch must be of the common coarse renement between the patch of interest and all of its neighboring patches. This scenario is shown in Figure 4, where the isotropic patch on the right requires information from the two dierently rened patches on the left. In this example, the isotropic patch has a ratio of (2, 2), and the anisotropic patches have ratios of (2, 1) and (1, 2) from top to bottom, so the common coarse ratio in this case would be (1, 1). For reference, an overview of the exchange algorithm is shown by the owchart in Appendix A, Figure A.1. A comparison of the renement ratio vectors of the source and destination patches determines the number of cells to transfer. The number of cells is dependent on the number of ghost cells needed by the physics algorithms, and the interpolation stencil radius (which is one cell for the linear interpolator used in this work). The patches never directly send information to one another. Instead, a coarser intermediate patch is 6 of 16

7 Ghost cell overlap Interpolation radius Figure 5. The source region identied by the Motion Item. constructed for this purpose. The number of cells to transfer is specied in relation to the renement ratio of the intermediate patch, n l 1 ref,intm. The number of cells a source patch needs to transfer in a direction d to the intermediate patch is determined from c source,d = nl 1 ref,intm,d n l (r g + r s ), ref,source,d with r g being the ghost cell radius and r s being the interpolation stencil radius. Four steps must be executed for the anisotropic exchange operation. I. Allocate the temporary structures needed for transferring data. Exchanging data between anisotropic patches requires an algorithm that can resolve the dierences in cell size and cell shape. To ll data with dierent source and destination renements on the same level, a temporary intermediate patch is used. Each patch in a level has its own corresponding intermediate patch, with a renement that will let the patch and all its neighbors perform only an averaging operation to move data to the intermediate patch, the common coarse ratio dened above. The intermediate patch encapsulates its corresponding patch (including its ghost cells) and is grown by the number of cells needed for the interpolation stencil. II. Identify and cache the source and destination patches, and the regions of the source and destination to transfer. The regions of each patch that must be transferred from one patch to another is cached into motion items. Each motion item stores the destination region, the ghost cells in the destination patch that is to be lled as marked in Figure 4. Also stored is the source region, the core cells of the source patch that overlap with the destination ghost cells and the extra cells needed for the interpolation radius, marked in Figure 5. To accommodate anisotropic boxes, the common-coarse ratio associated with the destination patch is added to the motion item. Also allocated and cached at this time are the intermediate common-coarse data structures. These motion items are valid for the grid as long as it is unchanged, and must be re-cached upon regridding. III. Linearize the data from the source patch and transfer the data to the correct processor. When data is exchanged between processors, it is linearized into a generic array buer. The linear buer will be lled by the source patch with the source region as marked in Figure 5, but needs to be coarsened to the same renement as the destination patch's common-coarse intermediate patch. If the source cells and the coarsened-ne structure have the same renement ratio, then the existing isotropic linearization functions can be used. In cases where the coarsened-ne structure has a more coarse ratio, then the data must be averaged as it is transferred from the source patch to the linear buer. 7 of 16

8 Needed ghost cells Interpolation radius Figure 6. All cells to be averaged to the common coarse patch. Once the linear buers are lled, they are transferred using MPI to the processor specied by the motion items. On their destination processor, the data is delinearized into either the destination patch or the intermediate patch. If the destination patch and the intermediate patch have the same renement ratio, then the intermediate patch can be skipped entirely and the data delinearized directly into the destination patch. If, instead, the destination patch is more rened than the intermediate patch, the data must be delinearized into the intermediate patch. In both cases, the data was converted into the intermediate structure's ratio during the linearization step and so the delinearization step is direct copy to the same cell shapes. IV. Unpack the data on the destination processor, and store the information in the ghost cells. III.F. If the delinearization step was able to move the data from the linear buer directly into the destination patch, then this section is skipped as the data is already in the nal location. Otherwise, once all of the motion items have been processed, the coarsened-ne intermediate structure is now lled with the ghost cell and interpolation cell data from the source patch. Before the data can be interpolated, additional data must be moved from the destination patch to the intermediate patch. This is because the interpolator will operate on data from the intermediate structure, which has the core cells underlying the destination patch undened. Another averaging operation is used to move data from the destination patch to the intermediate patch. Figure 6 continues the example from above, showing the cells in the destination patch needed for interpolation. Shown in Figure 7 is how the intermediate patch would look overlaying the interpolation region. With the intermediate patch lled with all needed data, a linear interpolation scheme is used to move the data to the destination patch. The interpolation will populate all ghost cells shown by a diamond symbol in Figure 7 (same cells as marked in Figure 4). Refer to Section III.D for details on the linear interpolator. Flux Correction Between Anisotropic Patches on the Same Level In a nite-volume method, the ux at cell interfaces must be single-valued. However, problems arise at the interface between cells of dierent resolution; the sum of the ux calculated from the ne cells may not equal the ux calculated from the coarse cell. This ux correction needs to be applied at the interfaces of anisotropic patches at each time step. There already exists functionality for ux correction at level boundaries, and these methods are adapted to apply to anisotropic patch boundaries on the same level. IV. Operators Associated with Anisotropic Adaptive Renement in Space When a single-level grid is adaptively rened anisotropically, more complexity is added. For example, the existing tagging method for the isotropic algorithm must be adjusted to provide renement tags in each 8 of 16

9 Figure 7. A pseudo-isometric view of the common coarse patch overlaying the interpolation region. Figure 8. A nested grid hierarchy with transition on level 2. spatial coordinate. Other operators, such as inter-level interpolation and averaging and the ux corrections, will be revised accordingly and discussed next. IV.A. Tagging and Regridding Cell tagging is an operation specic to the case being run. For normalizing a moving mesh, cells may be tagged in order to re-normalize the grid. In the case of the shock example, cells are tagged for renement based on the undivided gradient of density. When the gradient of the solution exceeds a threshold specied by the user, the cell is tagged with spatial coordinates in which the gradient exceeds the threshold. The regridding operator is then be provided with this list of tags. We initially adopt a very simple approach for constructing the new mesh hierarchy. We follow the procedure outlined by Berger and Rigoutsos 8 to create an isotropic mesh. Then, we visit each of the newly designated patches and coarsen the patch to meet proper nesting constraints. We expect this approach to be excessively conservative (tending towards isotropic) and in future work will explore more advanced methods for generating new grid levels. Proper nesting in an AMR scheme ensures that enough cells are available to the Ω l+1 level so that the 9 of 16

10 y Cell to be interpolated 2 Interpolation radius 1 0 x Figure 9. Two level case with level l + 1 anisotropically rened in x. invalid ghost cells may be interpolated from without requiring expansion or further renement of the Ω l level. Proper nesting in isotropic schemes imposes two constraints: 1. The grids are properly blocked such that Ω l k = C 1 p (C p (Ω l k )) where p = nl 1 ref. 2. There is at least one level l cell separating level l+1 cells from level l 1 cells: G(C n l ref (Ω l+1 ), 1) Γ l Ω l. Figure 8 illustrates an anisotropic grid with three levels. Despite the anisotrophy in the renement, the grid still satises proper nesting. To the nesting constraints imposed above, we add a third: 3. The resolution change between adjacent cells from levels level l and level l + 1 can be no greater than n l ref. Additionally, the resolution change between adjacent cells on level l + 1 can be no greater than n l ref. This last constraint is satised automatically by constraint 2 when the grid is isotropically rened. The grid in Figure 8 also satises this last constraint. Note that the grid on level 1 transitions to satisfy constraint 3 as the level 2 grid changes from isotropic renement to fully-anisotropic moving from right to left. This intermediate box is required to allow the x-direction renement ratio of the boxes on level 2 to transition from 4 to 1 relative to the base grid. IV.B. Interpolating Data from Coarse Level to Fine Level Signicant changes were made to the inter-level data transfer algorithms to support anisotropic patches. Since a level is now made up of multiple patches rened in dierent orientations, there is not a single ratio between a level Ω l and Ω l+1. This introduces complications when transferring data between levels as a patch on level Ω l+1 may interact with several patches on level Ω l of dierent orientations. Before the calculations begin, halos of ghost cells must be lled with solution information in order to perform the calculations. Valid ghost cells overlap core cells on the same level; they are lled by an exchange operation described in detail in Section III.E. Invalid ghost cells overlap the core cells of a coarser level; they are lled by the interpolation operation described in Section IV.D. One focus in this section is the lling of the core cells of Ω l+1 with data from the core cells of Ω l, an operation that must be performed when new ne cells are created. A linear interpolation method with three points is used, so this is a second order operation. A patch on the ne level may overlay several coarse patches, and those coarse patches may be rened in dierent 10 of 16

11 Intermediate patch Level l + 1 patch Cell to be interpolated Interpolation radius Patch 1 Patch 2 Figure 10. A top-down view of a level l + 1 patch with corresponding intermediate patch, and cells marked for interpolation and interpolation radius on the level l patches. directions. A method to resolve the dierences in renement and provide a consistent space is needed to perform interpolation. An intermediate patch is used, which is a coarsening of the ne patch, and grown by some number of cells. It is similar to the coarsened-ne data structures mentioned in Section III.E. This intermediary patch is then lled with data from source Ω l patches. The data is then interpolated from the intermediary patch to the destination Ω l+1 patch. The purpose of the intermediary patch is two fold. First, the data from the Ω l level may be in dierent patches on dierent processors. This step allows for the collection of the data from these multiple sources into a single source patch. Since the intermediary patch is only as big as needed for the destination patch, the memory footprint is smaller than if the entirety of the source patches are transferred. Second, the intermediary patch standardizes the source data into a single renement ratio. The source patches may have dierent renement directions and be from dierent levels, so resolving these dierences into a single patch allows for a single application of the interpolation scheme when transferring data to the destination patch. Since the averaging scheme is an exact method and the interpolation scheme introduces error, it is critical for the algorithm to reduce the number of interpolations needed when resolving cell dierences. Careful choice of the shape of the intermediary patch can help reduce the number of operations needed. Most generally, the intermediary cells should match the source patch. This is possible when there is a single underlying patch, as shown in Figure 9. When there are two or more underlying patches, as in Figure 10, a method is needed to decide on which coarse patch to use as the base. With two source patches, we choose to use the coarser patch as the source. Using the coarsest patch eliminates the need for interpolating from the coarsest source patch to the nest coarse patch, and thus reduces the stacking of error. The renement ratio vector that should be used for the intermediary patch is found from the piecewise comparison of the sources' renement ratio vector, n lh/2 ref = min( n l ref,k), for each direction. After the intermediate patch has been created, data must be transferred from the coarse level to the ne level. Figure 11 demonstrates the case where data must be transferred from two patches on level l to the intermediate patch. As shown in the gure, the data on level l may be either at the same renement (patch 1) or more rened (patch 2) than the intermediate patch. In the case with the same renement, transferring the data is a simple copy operation. In the case where the level l patch is more rened, the data is averaged to the intermediate patch using the method described in Section III.C. 11 of 16

12 Intermediate patch Level l + 1 patch Level l Patch 1 Patch 2 Cell to be interpolated Interpolation radius Figure 11. A pseudo-isometric view showing a level l+1 patch and corresponding intermediate patch. Once the coarsened-ne patch is completely lled with coarse data, the ne patch may be interpolated to using the method described in Section III.D. IV.C. Averaging Data from Fine Levels to Coarse Levels At the end of a time step, data must be averaged from ner levels to coarser levels. While the data will be less resolved, averaging is an exact operation. A ner patch may overlay several coarse patches, and since those coarse patches may be rened in dierent directions, the anisotropic averaging algorithm must dier from the isotropic algorithm. Due to the requirements of proper nesting, transferring data from ner levels to coarser levels is always strictly an averaging operation with no stencil and so an intermediate patch like the one used for interpolation is not needed. Which coarse patches underlie a particular ne patch must be known for the averaging operation. Since the patch layout remains the same between regridding operations, the knowledge of the underlying patches can be cached until the next regrid. During this operation each ne patch is visited, the intersect of the ne and each underlying coarse patch is identied, and the averaging operator is called. For details on the averaging operator, see Section III.C. IV.D. Filling Data in the Invalid Ghost Cells The invalid ghost cells surrounding level l + 1 need to be lled with data from coarser level l data before calculations can be performed in level l + 1. This process uses a similar method to lling core cells on level l + 1: intermediate common coarse patches are created, data is transferred to the intermediate patch, then interpolated to the destination patch. There are a couple additional concerns when lling invalid ghost cells. For each patch in level l + 1, the patch's common coarse ratio with the underlying patches is calculated, the common coarse patch is allocated, and its ghost cells are identied. Calculating the common coarse ratio and allocating the common coarse patch have been discussed previously in this paper. As there will be an interpolation in this process, the common coarse patch include both the ghost cell radius and the interpolation stencil radius. The invalid ghost cells are identied by originally including all of the ghost cells on a patch, then removing ghost cells from the set if they are found to be valid ghost cells (ghost cells that 12 of 16

13 Figure 12. A 3-level grid showing the solution transfer being interpolated from the coarse grid to anisotropic ne grids. overlap other patches on this level). Identication of the valid ghost cells for an anisotropic level requires the same considerations as when checking for disjointedness in an anisotropic level, as described in Section III.B. The invalid ghost cells for each patch are iterated over and interpolated in time before being interpolated in space from the intermediate common coarse patch in the process described by Section IV.B. IV.E. Inter-level Flux Correction As with the ux corrections at cell interfaces with renement orientation changes on a single level, the ux at inter-level cell interfaces must be corrected from ner resolutions in order to maintain conservation of the system. Methods already exist in the isotropic algorithm to correct the ux between dierent levels, but these methods need to be adjusted to anisotropic ratios. In brief, the ux correction occurs by comparing the face ux on the coarser grid with the sum of the ner cell uxes that meet at the face, and correcting the coarse ux to equal that of the sum of the ner uxes. Anisotropic levels will change the number of ne cells per coarse cell, depending on the two patches at the interface and the orientation of renements. Research is ongoing into inter-level ux corrections. V. Results Having implemented the infrastructure for anisotropic mesh renement, we now demonstrate the capability using two test cases. The rst case tests the inter-level data interpolator from coarse levels to ner levels. The second test is a single level 2D shockbox with anisotropic patches which shows the anisotropic exchange operator and other single-level operations. For the interpolator test case, a 2D coarse grid of ve by nine isotropic cells is created and lled with the linear test function, f(x, y) = 4x +.04y This coarse isotropic grid is the base grid of the solution, Ω 0, labeled Level 0 in Figure 12, and encapsulates the entire problem domain. Level Ω 0 comprises a single patch covering the entire domain with a renement ratio of (1, 1). Figure 12 shows a level Ω 1 grid that is overlaid on the base grid (marked Level 1 in the gure). There 13 of 16

14 Figure 13. Density plot for 2D shockbox with anisotropic renement on top left and bottom right patches, isotropic renement on bottom left and top right patches. Solution at t = 6.1 ms. are three patches on the Ω 1 grid, with anisotropic renement in the y-direction on the left side of the level, transitioning into isotropic renement in the middle, and nally to anisotropic renement in the x-direction on the right side of the level. These locations and renements were chosen to be representative of all the cases anticipated to be layered on the base grid. Anisotropic renement in both coordinate directions are represented, as well as the isotropic renement. The data was then interpolated from the coarse grid to the Ω 1 grid using the inter-level data transfer method described in Section IV.B, although a piecewise constant interpolator was used instead of a piecewise linear interpolator. The piecewise constant interpolation exactly copies the underlying grid's value onto the ner grid, and is an O(h) error operation. Lastly in Figure 12 a second level, Ω 2, is layered on top of the Ω 1 grid (marked Level 2 in the gure). This new region tests the proposed method for having multiple underlying coarse grids as a source, and demonstrates proper nesting on anisotropic grids. The second test case demonstrates the single level operations in a 2D shockbox, shown in Figure 13. In this shockbox case, the low pressure region was placed in the bottom left corner, and the high pressure region in the remaining three quadrants. The shockbox would have an isotropic grid of 64 by 64 cells, but anisotropic patches are introduced in the top left and bottom right regions. The anisotropic patch in the top left, rened in y, has a ratio of (1, 2), as opposed to the (2, 2) ratio of the isotropic patches, and contains 16 cells in the x-direction and 32 cells in the y-direction. The anisotropic patch in the bottom right 14 of 16

15 is the opposite, with a ratio of (2, 1) and 32 cells in the x-direction with only 16 cells in the y-direction. Replacing the two isotropic patches with anisotropic patches reduces the cell count from 4096 cells down to 3072 cells. The exchange operator transfers solution data between anisotropic and isotropic patches, and between anisotropic patches in the center of the domain. The shockbox was run for a total time of t = 6.1 ms with a t = ms for a total of 2440 time steps. Figure 13 shows the solution at the end time. The performance of the anisotropic case (3072 cells) was compared against the full isotropic case (4096 cells). The run times are presented in Table 1. Even with a very modest reduction in cell count, the anisotropic algorithm outperforms the isotropic case with a runtime decrease of approximately 8%. Thus with additional AMR levels and spatial dimensions, signicant improvements to performance are expected. Table 1. Run times for isotropic and anisotropic solution of shock problem. Renement Time (s) Isotropic 73 Anisotropic 67 VI. Conclusions and Future Work In this work we have created the infrastructure to perform single-level and inter-level anisotropic patchbased operations. The functionality of these operations were demonstrated through a single-level shockbox simulation and an inter-level interpolation from coarse to ne patches. Further work needs to be done on validating the results of anisotropic renement for both performance of the algorithm and accuracy of the solution. VII. Acknowledgment Research at the Colorado State University was supported by Department of Energy under contract number DE-EE References 1 Berger, M. J. and Colella, P., Local Adaptive Mesh Renement for Shock Hydrodynamics, J. Comput. Phys., Vol. 82, No. 1, May 1989, pp Williamschen, M., Parallel Anisotropic Block-Based Adaptive Mesh Renement Algorithm For Three-Dimensional Flows, Master's thesis, University of Toronto, Zhang, J., Parallel Anisotropic Block-Based Adaptive Mesh Renement Finite-Volume Scheme, Master's thesis, University of Toronto, Ham, F., Lien, F., and Strong, A., A Cartesian Grid Method with Transient Anisotropic Adaptation, Journal of Computational Physics, Vol. 179, No. 2, 2002, pp Gao, X., A parallel solution-adaptive method for turbulent non-premixed combusting ows, Ph.D. thesis, University of Toronto, Adams, M., Colella, P., Graves, D. T., Johnson, J. N., Johansen, H. S., Keen, N. D., Ligocki, T. J., Martin, D. F., McCorquodale, P. W., Modiano, D., Schwartz, P. O., Sternberg, T. D., and Straalen, B. V., Chombo Software Package for AMR Applications - Design Document, Tech. Rep. LBNL-6616E, Lawrence Berkeley National Laboratory, Berger, M. J., On Conservation at Grid Interfaces, SIAM J. Numer. Anal., Vol. 24, No. 5, 1987, pp Berger, M. J. and Rigoutsos, I., An algorithm for point clustering and grid generation, IEEE Transactions Systems, Man, and Cybernetics, Vol. 21, No. 5, 1991, pp A. Appendix A Figure A.1 illustrates the process used to exchange ghost cells between dierently rened patches on the same level. 15 of 16

16 Begin Exchange Ghost and interpolation stencil size Find common coarse ratio Begin linearization process Grow destination by ghost and interpolation radius Allocate common coarse data structures Intersect with source patch Cache destination and source regions Common coarse ratio Allocate common coarse linear buer Compare common coarse & source ratio Common coarse greater No Same Begin delinearization process Pass Messages Yes All patches linearized? Fill linear buer Average to common coarse ratio Compare common coarse & destination ratios Same Unpack CFLB to CFBF Interpolate from common coarse data structure to destination Common coarse greater Interpolate to dest ratio Figure A.1. Procedure for exchanging ghost cells 16 of 16