Algorithms for construction of Element Partition Trees for Direct Solver executed over h refined grids with B-splines and C 0 separators

Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 108C (2017) 857 866 International Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland Algorithms for construction of Element Partition Trees for Direct Solver executed over h refined grids with B-splines and C 0 separators Bartosz Janota 1, Maciej Paszyński 1 AGH University of Science and Technology, Kraków, Poland Abstract We propose a way of performing isogeometric finite element method (IGA-FEM) computations over h refined grids with B-spline basis functions. Namely, we propose to use the B-spline basis functions defined over patches of elements with C 0 separators between the refinement levels. Our definition of the B-splines and C 0 separators allows introduction of arbitrary order B-splines over 2D grids refined towards singularities. We also present an algorithm for construction of element partition trees (EPT) over h refined grids with modified B-splines. The EPT allows generating an ordering which gives a linear computational cost of the multi-frontal solver over 2D grids refined towards a point or an edge singularity. We present the algorithm for transforming the EPT into an ordering. We also verify the linear computational cost of the proposed method on grids with point and edge singularity. We compare our method to h-adaptive finite element method (h-fem) computations with Lagrange polynomials. 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the International Conference on Computational Science Keywords: multi-frontal direct solver, element partition trees, b-spline, ordering 1 Introduction The main idea of the isogeometric analysis [10] (isogeometric finite element method IGA-FEM) is to connect computational methods with computer aided design tools, such as AutoCAD. Before the IGA-FEM has been invented, the geometry of the computational domain had often been defined in Computer Aided Design (CAD) tools using B-splines or their rationalized version called Non-Uniform Rational B-splines (NURBS). The computations, however, had been performed in finite element method (FEM) tools, using hierarchical Lagrange polynomials, and so the mesh and the basis functions were converted from the CAD world to the FEM world. Since the invention of the IGA-FEM, both steps are integrated. The IGA-FEM has multiple applications in shear deformable shell theory [5], phase field models [18, 19], phase-separation simulations with application to cancer growth simulations [21, 22], wind turbine aerodynamics [24], incompressible hyper-elasticity [20], turbulent flow 1877-0509 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the International Conference on Computational Science 10.1016/j.procs.2017.05.064

858 Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 simulations [8], transport of drugs in cardiovascular applications [23] or the blood flow simulations itself [3, 2, 7]. We introduce rationalized B-splines and C 0 separators between same refinement levels elements, which allows extending the IGA-FEM with B-splines into grids refined toward a point or an edge singularities. Alternative approach is the application of T-splines [28], which requires the introduction of the T-junction extensions to obtain the analysis suitable grids, but in this paper we focus on the B-spline basis functions. We also propose an algorithm for construction of orderings to be used by the multi-frontal direct solvers, based on the concepts introduced in [15, 16] for standard finite element method. When comparing classical FEM and IGA-FEM on regular grids, the classical FEM is only C 0 on the borders of elements, while C (p 1) inside particular elements, whereas IGA-FEM ensures C p 1 regular solutions if one uses B-splines of the order p. Of course, there are some limitations, such as the IGA-FEM solution increases the computational cost, as described in [11]. The application of B-spline basis functions results in O(N 1.5 p 3 ) computational complexity on homogeneous 2D meshes and O(N 2 p 3 ) on homogeneous 3D meshes, where N is the number of unknowns over the regular mesh and p is the B-spline order. The computational complexity is p 3 times higher in comparison to standard FEM with Lagrange polynomials. The computational complexity can be reduced down to O(Np 2 )in2d or O(N 4 /3p 2 )in3d, by using shared-memory [13] or distributed memory solvers [14]. In the second case, the communication complexity is also of the same order as the computational complexity. However, when we switch into the computational costs, the constants in front of the computational complexity come into the picture. As it is shown in [17], the computational cost of IGA-FEM for the fixed number of elements becomes lower than the cost of FEM. However, when we utilize patches of higher order elements with C 0 separators between them, the resulting method (called riga) is cheaper than both IGA-FEM and FEM. In [17] the higher order elements are grouped into patches over the regular grids. In this paper we rather focus on adaptive, h-refined grids. In this paper, we propose to identify patches of elements with same refinement levels and introduce C 0 separators between them. 2 B-splines over refined meshes Our idea of the introduction of B-splines over the 2D mesh with singularities is to introduce C 0 separators between the refinement levels surrounding the singularities and to rationalize the B-splines. The rationlization is necessary to fulfill the partition of unity property. In this case, we only have C 0 continuity between the refinement levels. However, we still have higher order continuity inside the patches of elements, located inside the refinement levels. In the following considerations, for simplicity, we will focus on two-dimensional meshes with point and edge singularity, presented in Figure 1. We truncate the B-spline basis functions with the C 0 separators defined over the mesh, and rationalize them, to obtain the partition of unity feature Ri k(x) = B n i,k(x)ω i j=1 B j,k(x)ω j. The drawback of introductions the C 0 sepparators is that some of the supports of basis functions have rectangular shape, and some other are non-rectangular ones. In Figures 2 and 3 we can see some examples of basis functions spanned over the 3 rd depth level 2D mesh. In the case of non-rectangular supports, the green elements belong to the support of the basis functions located on the yellow point. The same situation exists when we want to span basis functions from the blue points, except the most center ones. Each non-rectangular basis function is built from two pieces, as it is illustrated in Figure 4. Red basis function is a 3/4 of the regular basis function and this is the first piece. The second one is

Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 859 Figure 1: 2D mesh, refined towards the point singularity (a) Regular 4x4 mesh, 1 st level depth, no separators (b) Regular 4x4 mesh, 4 th level depth, double lines yet mean C 0 separators Figure 2: 3 rd depth level 2D mesh with some sample basis functions combined from two regular basis functions (dark and light blue) and the black one. The black basis function is a sum of two regular basis functions and one linear function. Thanks to them, we have continuity on the edges between red basis function and all the others and a zero value in the most central corner of that basis function. The most important thing is that the red value between two blue basis functions is zero so that basis function perfectly fits into 3. 3 Element Partition Tress Element Partition Tree (EPT) can be built based on the mesh, as shown in Section (2), and the basis functions spanned over that mesh. EPT is an essential base for the ordering (see Section 3.1). Ordering resulting from EPT may turn the not-nice matrix into the one that is multi-diagonal or almost multi-diagonal, and the resulting cost of the factorization will be linear or quasi-linear. On Algorithm 1 you can see EPT generation algorithm. The algorithm cuts layer by layer, getting deeper into the mesh. Thanks to that, we are sure, that the new sub-tree (created from cut part) is not too deep. We create those sub-trees recursively. As you can see on Figure 5, EPT is deep and unbalanced. EPT can be also generated for other refined grids, like for the edge singularity case. EPT is going to be used as a base for ordering.

860 Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 Figure 3: Non-rectangular support Figure 4: Non-rectangular support basis function Figure 5: Complete EPT 4

Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 861 Algorithm 1 EPT generation procedure GenerateEptNodes(rootNode) for i =1todepth do There is an example I refer to, see 5 ProcessEptNode(left) red on 5 ProcessEptNode(right) blue on 5 ProcessEptNode(top) green on 5 ProcessEptNode(bottom) purple on 5 end for ProcessEptNode(innerMost16Nodes) orange on 5 end procedure procedure ProcessEptNode(node) AddEptNode(node) if node can be split then ProcessEptNode(nodeF irsthalf) ProcessEptNode(nodeSecondHalf) end if end procedure procedure AddEptNode(node) EPT EPT + node end procedure 3.1 Ordering Figure 6: Original FEM matrix for depth =8 Figure 6 presents the original FEM matrix A generated for the regular mesh with point singularity (depth = 8) with the natural ordering of elements. This matrix is not multi-diagonal at all (it is sparse but the non-zero entries are spread around the whole matrix and it has some blocks of non-zero entries far from the diagonal, which results in a large number of additional non-zero entries generated during the factorization), so the cost of the elimination is non-linear.

862 Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 We want to reduce the bandwidth as much as possible and to achieve that we need to permute the rows somehow to make numerical factorization more efficient. We can do that thanks to that very basic matrix transformation: B = P 1 AP (1) where P is the permutation matrix based on elimination ordering and B is the outcome matrix that gives as the linear cost of the multi-frontal computations. The permutation matrix P and the ordering can be obtained by post-order traversal of EPT. This is illustrated in Figure 7 for the exemplary EPT for 1/4 of 5. The ordering is generated by Algorithm 2. If we meet any B-spline that has not been added yet, we should add it to the ordering list. Main of that algorithm is to eliminate each of the basis functions as deep in the EPT as it is possible, so this is why we eliminate 1, 2, 3, 4, 5, 6 and 7 in the leaves, and not earlier. 9 and 11 could be eliminated in the root node, but they exist deeper as well, so we add them to the ordering list once we meet them there. There are some basis functions which do not exist deeper in the EPT than in the root, so, in the last step we add 8, 10 and 12 to the ordering list. Figure 7: EPT ordering, EPT for 1/4 of 5, last step of 2 Figure 8: 8 th, 12 th and 10 th basis function added, we skip all of the remaining basis functions, because they are already added, final ordering list: [1, 2, 9, 3, 4, 5, 7, 6, 11, 8, 12, 10]

Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 863 Algorithm 2 Ordering generation procedure TraverseEptPostOrder(currentNode) TraverseEptPostOrder(leftSon) traverse the left subtree TraverseEptPostOrder(rightSon) traverse the right subtree if orderinglist does not contain any B-splines spanned over currentnode then orderinglist orderinglist + newly encountered B-splines end if end procedure Our matrix A after the application of the generated ordering is presented in Figure 9. We can see that the matrix now is multiple-diagonal, and the cost of its factorization will be linear. Figure 9: Multi-diagonal matrix, after permutation of 6 with a use of ordering (7) 4 Results evaluation Our theoretical estimation of computational cost can be proved with the results shown below. To do that, we measured multi-frontal direct solvers computational cost and we express that cost in number of floating-point operations (FLOPS). We are using ordering (obtained from EPT) to control MUMPS solver. We run our experiment for the two-dimensional mesh refined towards point and edge singularity. We compare the mesh with B-spline basis functions and C 0 separators with the mesh with Lagrange polynomials. In the first case, we generate the ordering by using the EPT and submit it to MUMPS solver. In the second case we use the automatic ordering proposed by MUMPS, and the grid obtained from hp2d code. We consider the square grids and 2 nd order B-splines.

864 Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 Figure 10: FLOPS estimate for point singularity with respect to the degrees of freedom (log scale), for B-splines with C 0 separators, ordering from EPT, passed to MUMPS solver, versus Lagrange polynomials and MUMPS with automatic ordering Figure 11: FLOPS estimate for edge singularity with respect to the degrees of freedom (log scale), for B-splines with C 0 separators, ordering from EPT, passed to MUMPS solver, versus Lagrange polynomials and MUMPS with automatic ordering The experiments are presented in Figures 10 and 11. For the case of point singularity, both costs are linear with respect to the number of degrees of freedom, and the costs are almost identical. For the case of edge singularity, the B-splines with C 0 separators with EPT ordering give linear cost, while Lagrange polynomials with MUMPS with automatic ordering gives quasi-linear ordering.

Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 865 5 Conclusions In this paper we have shown that introducing C 0 separators between refinement levels and modifying B-splines by rationalizing and truncating them with the separators, results in a linear computational cost reduction when compared to classical FEM with Lagrange polynomials. Namely, for point singularity the costs are comparable, and for the edge singularity in 2D the B-splines give linear cost while Lagrange polynomials give quasi-linear cost. We proved that rationalized B-splines span over patches of elements separated with C 0 separators are the keys. They can be used as the basis functions for grid refinements both for point and edge singularity. For the generation of the ordering with our B-splines and C 0 separators we performed a postorder traversal of the element partition tree. We showed that our approach allows to reduce the computational cost when comparing to classical FEM with Lagrange polynomials. The future work will involve implementation in three-dimensions [25] and parallel computations in two and three-dimensions [26, 27]. 6 Acknowledgements This work is supported by National Science Centre, Poland grant no. 2016/21/B/ST6/01539. References [1] Y. Bazilevs, L. Beirao da Veiga, J.A. Cottrell, T.J.R. Hughes, and G. Sangalli, Isogeometric analysis: Approximation, stability and error estimates for h-refined meshes, Mathematical Methods and Models in Applied Sciences, 16 (2006) 1031 1090. [2] Y. Bazilevs, V.M. Calo, J.A. Cottrell, T.J.R. Hughes, A. Reali, G. Scovazzi, Variational multiscale residual-based turbulence modeling for large eddy simulation of incompressible flows, Computer Methods in Applied Mechanics and Engineering 197 (2007) 173-201. [3] Y. Bazilevs, V.M. Calo, Y. Zhang, T.J.R. Hughes: Isogeometric fluid-structure interaction analysis with applications to arterial blood flow, Computational Mechanics 38 (2006). [4] Y. Bazilevs, V. M. Calo, J. A. Cottrell, J. A. Evans, S. Lipton, M. A. Scott, T. W. Sederberg, Isogeometric analysis using T-splines, Computer Methods in Applied Mechanics and Engineering, 199 (2010) 229-263. [5] D.J. Benson, Y. Bazilevs, M.-C. Hsu and T.J.R. Hughes, A large-deformation, rotation-free isogeometric shell, Computer Methods in Applied Mechanics and Engineering, 200 (2011) 1367-1378. [6] M. Bubak, J. Kitowski, K. Wiatr, EScience on Distributed Computing Infrastructure: Achievements of PLGrid Plus Domain-specific Services and Tools, vol. 8500 (2014) Springer [7] V.M. Calo, N. Brasher, Y. Bazilevs, T.J.R. Hughes, Multiphysics Model for Blood Flow and Drug Transport with Application to Patient-Specific Coronary Artery Flow, Computational Mechanics, 43(1) (2008) 161 177. [8] K. Chang, T.J.R. Hughes, V.M. Calo, Isogeometric variational multiscale large-eddy simulation of fully-developed turbulent flow over a wavy wall, Computers and Fluids, 68 (2012) 94-104. [9] B. Janota, P. Lipski, and M. Paszyński, Computational complexity of isogeometric analysis with T-splines and B-splines over 2D grids refined towards singularities, Procedia Computer Science, proceedings of International Conference on Computational Science ICCS, (2016). [10] J. Cottrell, and T. Hughes, and Y. Bazilevs, Isogeometric Analysis: Toward Integration of CAD and FEA, (2009).

866 Bartosz Janota et al. / Procedia Computer Science 108C (2017) 857 866 [11] N. Collier, D. Pardo, L. Dalcin, M. Paszyński, and V. Calo, The cost of continuity: A study of the performance of isogeometric finite elements using direct solvers, Computer Methods in Applied Mechanics and Engineering, (2012), 213, 353-361. [12] L. Piegl, and W. Tiller, The NURBS Book (Second Edition), Springer-Verlag New York, Inc., (1997). [13] M. Woźniak, K. Kuźnik, M. Paszyński, V. Calo, and D. Pardo, Computational cost estimates for parallel shared memory isogeometric multi-frontal solvers, Computers & Mathematics with Applications, 67(10) (2014) 1864-1883. [14] M. Woźniak, M. Paszyński, D. Pardo, L. Dalcin, V. Calo, Computational cost of isogeometric multi-frontal solvers on parallel distributed memory machines, Computers Methods in Applied Mechanics and Engineering, 284 (2015) 971-987. [15] H. AbouEisha, M. Moshkov, V. Calo, M. Paszyński, D. Goik, and K. Jopek, Dynamic Programming Algorithm for Generation of Optimal Elimination Trees for Multi-frontal Direct Solver Over H-refined Grids, Procedia Computer Science, 29 (2014) 29, 947-959. [16] M. Paszyński, Fast Solvers for Mesh-Based Computations, CRC Press, Taylor & Francis, (2016). [17] D. Garcia, D. Pardo, L. Dalcin, M. Paszyński, N. Collier, V. Calo, The value of continuity: Refined isogeometric analysis and fast direct solvers, Computer Methods in Applied Mechanics and Engineering (2017) DOI: 10.1016/j.cma.2016.08.017 [18] L. Dedè,T.J.R. Hughes, S. Lipton, V.M. Calo, Structural topology optimization with isogeometric analysis in a phase field approach, USNCTAM2010, 16th US National Congree of Theoretical and Applied Mechanics. [19] L. Dedè, M. J. Borden, T.J.R. Hughes, Isogeometric analysis for topology optimization with a phase field model, ICES REPORT 11-29, The Institute for Computational Engineering and Sciences, The University of Texas at Austin (2011). [20] R. Duddu, L. Lavier, T.J.R. Hughes, V.M. Calo, A finite strain Eulerian formulation for compressible and nearly incompressible hyper-elasticity using high-order NURBS elements, International Journal of Numerical Methods in Engineering, 89(6) (2012) 762-785. [21] H. Gómez, V.M. Calo, Y. Bazilevs, T.J.R. Hughes, Isogeometric analysis of the Cahn-Hilliard phase-field model, Computer Methods in Applied Mechanics and Engineering 197 (2008) 4333 4352. [22] H. Gómez, T.J.R. Hughes, X. Nogueira, V.M. Calo, Isogeometric analysis of the isothermal Navier- Stokes-Korteweg equations. Computer Methods in Applied Mechanics and Engineering 199 (2010) 1828-1840. [23] S. Hossain, S.F.A. Hossainy, Y. Bazilevs, V.M. Calo, T.J.R. Hughes, Mathematical modeling of coupled drug and drug-encapsulated nanoparticle transport in patient-specific coronary artery walls, Computational Mechanics, doi: 10.1007/s00466-011-0633-2, (2011). [24] M.-C. Hsu, I. Akkerman, Y. Bazilevs, High-performance computing of wind turbine aerodynamics using isogeometric analysis, Computers and Fluids, 49(1) (2011) 93-100. [25] A. Paszyńska, E. Grabska, M. Paszyński, A Graph Grammar Model of the hp Adaptive Three Dimensional Finite Element Method. Part II., Fundamenta Informaticae, 114(2) (2012) [26] M. Paszyński, A. Paszyńska, Graph transformations for modeling parallel hp-adaptive Finite Element Method, Lecture Notes in Computer Science, 4967 (2008) 1313-1322 [27] M. Paszyński, On the Parallelization of Self-Adaptive hp-finite Element Methods Part I. Composite Programmable Graph Grammar Model, Fundamenta Informaticae, 93(4) (2009) 411-434 [28] L. Beirao da Veiga, A. Buffa, G. Sangalli, R. Vazquez, Analysis-suitable T-splines of arbitrary degree: definition and properties, Mathematical Models and Methods in Applied Sciences, 23(11) 2013