Truncated Newton-based multigrid algorithm for centroidal Voronoi diagram calculation

Transcription

1 NUMERICAL MATHEMATICS: Teory, Metods and Applications Numer. Mat. Teor. Met. Appl., Vol. xx, No. x, pp (200x) Truncated Newton-based multigrid algoritm for centroidal Voronoi diagram calculation Zicao Di 1, Maria Emelianenko 1,, Stepen G. Nas 2 1 Department of Matematical Sciences, George Mason University, Fairfax, VA Systems Engineering and Operations Researc Department, George Mason University, Fairfax, VA Abstract. In a variety of modern applications tere arises a need to tessellate te domain into representative regions, called Voronoi cells. A particular type of suc tessellations, called centroidal Voronoi tessellations or CVTs, are in big demand due to teir optimality properties important for many applications. Te availability of fast and reliable algoritms for teir construction is crucial for teir successful use in practical settings. Tis paper introduces a new multigrid algoritm for constructing CVTs tat is based on te MG/Opt algoritm tat was originally designed to solve large nonlinear optimization problems. Uniform convergence of te new metod and its speedup comparing to existing tecniques are demonstrated for linear and nonlinear densities for several 1d and 2d problems, and O(k) complexity estimation is provided for a problem wit k generators. AMS subject classifications: 15A12; 65F10; 65F15 Key words: Centroidal Voronoi tessellation, optimal quantization, truncated Newton metod, Lloyd s algoritm, multilevel metod, uniform convergence 1. Introduction A Voronoi diagram can be tougt of as a map from te set of N-dimensional vectors in te domain Ω N into a finite set of vectors {z i } k i=1 called generators. It associates wit eac z i a nearest neigbor region tat is called a Voronoi region {V i } k i=1. Tat is, for eac i, V i consists of all points in te domain Ω tat are closer to z i tan to all te oter generating points, and a Voronoi tessellation refers to te tessellation of a given domain into te Voronoi regions {V i } k i=1 associated wit a set of given generating points {z i } k i=1 Ω [1, 35]. Wit a suitably defined distortion measure, an optimal tessellation is given by a centroidal Voronoi tessellation, wic is constructed as follows. For a given Corresponding autor. addresses: zdi@gmu.edu(zicao Di), memelian@gmu.edu (Maria Emelianenko), snas@gmu.edu(stepen Nas) ttp:// 1 c 200x Global-Science Press

2 2 Zicao Di, Maria Emelianenko, Stepen G. Nas density function ρ defined on Ω, we may define te centroids, or mass centers, of regions {V i } k i=1 by 1. z i = yρ(y) dy ρ(y) dy V i V i A centroidal Voronoi tessellation (CVT) is ten a tessellation for wic te generators of te Voronoi diagram coincide wit te centroids of teir respective Voronoi regions, in oter words, z i = z i for all i. Given a set of points {z i } k i=1 and a tessellation {V i} k i=1 of te domain, we may define te energy functional or te distortion value for te pair ({z i } k i=1, {V i} k i=1 ) by {z i } k i=1, {V i} k i=1 = k i=1 V i ρ(y) y z i 2 dy. If te Voronoi tessellation {V i } k i=1 is determined from {z i} k i=1 ten we write {z i } k i=1 {z i } k i=1, {V i} k i=1. (1.1) Te minimizer of necessarily forms a CVT wic illustrates te optimization property of te CVT [7]. Tis functional appears in many engineering applications and te relation of its minimizers wit CVTs is studied, for instance, in [18, 19, 38]. For instance, it provides optimal least-squares vector quantizer design in electrical engineering applications. Te CVT concept also as applications in diverse areas suc as astronomy, biology, image and data analysis, resource optimization, sensor networks, geometric design, and numerical partial differential equations [2,7 10,12,13,21,22,30,40,42]. In [7,11], extensive reviews of te modern matematical teory and diverse applications of CVTs are provided, and tis list is constantly growing. Te most widely used metod for computing CVTs is te algoritm developed by Lloyd in te 1960s [28]. Lloyd s algoritm represents a fixed-point type iterative algoritm consisting of te following simple steps: starting from an initial configuration (a Voronoi tessellation corresponding to an old set of generators), a new set of generators is defined by te mass centers of te Voronoi regions. Te domain is re-tessellated and a new set of centroids is taken as generators. Tis process is continued until some stopping criterion is met. For oter types of algoritms for computing CVTs we refer to [1, 11, 14, 17]. It was sown tat Lloyd s algoritm decreases te energy functional ({z i } k i=1 ) at every iteration, wic gives strong indications of its practical convergence. Despite its simplicity, proving convergence of Lloyd s algoritm is not a trivial task. Some recent work [6, 16] as substantiated earlier claims about global convergence of Lloyd s algoritm, altoug single-point convergence for a general density function ρ is still not rigorously justified. For modern applications of te CVT concept in large scale scientific and engineering problems suc as data communication, vector quantization and mes generation, it is crucial to ave fast and memory-efficient algoritms for computing te CVTs. Variants of Lloyd s algoritm ave been recently proposed and studied in many contexts for different

3 Truncated Newton-based multigrid algoritm for CVT calculation 3 applications [9,19,27,37]. A particular extension using parallel and probabilistic sampling was given in [22] wic allows efficient and mes free implementation of Lloyd s algoritm. However, te issue of finding a better alternative remains critical for many applications, since Lloyd s algoritm and its variants are at best linearly convergent. Moreover, te standard Lloyd algoritm slows down as te number of generators gets large, wic renders many practical calculations proibitively expensive. Several alternatives ave been proposed, including Newton-based metods [3, 25] and GPU extensions [36, 41]. Multilevel algoritms present a desirable framework in te context of large-scale applications, since tey do not suffer from te deterioration of convergence as te problem size grows a typical pitfall of most iterative metods. Te idea of multigrid and multilevel implementation of Lloyd s algoritm (referred to later as Multilevel-Lloyd metod ) as been recently introduced in [4], were multigrid metod was used as an outer" sceme, wit Lloyd s algoritm playing te role of a relaxation at eac level. Te metod was rigorously sown to be uniformly convergent for all smoot perturbations of a constant density in [5] and was extended to te 2-dimensional setting in [15], were it was successfully applied to a pysical data binning application problems. Despite its uniform convergence, te metod as a rater big computational cost for medium-sized problems and te construction of te interpolation operators is far from straigtforward. An alternative approac was introduced by Yavne et al. in [24] tat used a FAS (full approximation sceme) implementation of te Lloyd-Max sceme metod based on minimizing te residual between generators and centroids. We refer to tis algoritm as Multigrid Lloyd-Max metod later in te text. Altoug successful in te 1-dimensional setting, tis approac was not generalizable to iger dimensions [23]. Here we propose a new multilevel formulation for constructing CVTs in te 1- dimensional and 2-dimensional cases tat is based on a different type of FAS sceme. Our approac is distinctive because te algoritm uses a fine-tuned version of te off te self tecniques instead of special purpose approaces. A distinctive feature of te new metod is its close relationsip wit optimization problems. Tis is accomplised by formulating Lloyd s algoritm as a minimization problem in terms of te energy functional ({z i } k i=1 ). We ten apply a multilevel optimization framework called MG/Opt to tis functional. MG/Opt is a general framework for optimization wen a ierarcy of approximate models is available. MG/Opt was originally developed in te context of optimal control problems wit PDE constraints, but is applicable to a broader class of problems. By coosing appropriate interpolation operators, we design a new sceme wic enjoys uniform convergence wit respect to te problem size similar to te aforementioned algoritms. Te advantage comparing to te Multilevel-Lloyd metod comes in a form of significant reduction of computational costs, wile lower convergence factors and generalizability to iger dimensions favorably distinguises it from te Multigrid Lloyd-Max metod. Altoug MG/Opt is inspired by FAS, it is not equivalent to FAS. In te simplest case, i.e., wen MG/Opt is applied to an unconstrained problem, te update to te variables is te same as if FAS were applied to te first-order optimality conditions. However MG/Opt includes a line searc wic guarantees convergence in te sense tat lim i = 0. Even if a line searc is added to FAS, ten FAS is only guaranteed to find a local minimizer

4 4 Zicao Di, Maria Emelianenko, Stepen G. Nas of, wic is not guaranteed to be zero. Tus MG/Opt as stronger convergence properties tan FAS. In addition, MG/Opt (but not FAS) can be applied to optimization problems wit constraints, suc as bounds on te variables or more complicated constraints [33]. We expect tis to be an important factor wen applying MG/Opt to two- and igerdimensional problems. Additional reasons for preferring MG/Opt to FAS are discussed in [26]. Te rest of te paper is organized as follows. In section 2, we discuss te MG/Opt algoritm wic forms te basis for te new CVT construction metod. Its connection wit te CVT formulation is te subject of Section 3, were we also introduce te new FAS implementation and define te corresponding operators. Te results of numerical experiments and comparison wit oter metods are presented in Section 4, and final conclusions and discussion are found in Section Te Multilevel Optimization Algoritm We are applying a multilevel optimization algoritm called MG/Opt to te CVT problem. MG/Opt was originally developed to solve unconstrained optimization problems [32], wit an empasis on discretized optimization models. It as been extended to constrained problems [33] and te approac can be applied to problems not based on discretization. MG/Opt is designed to accelerate a traditional optimization algoritm applied to a ig-fidelity problem by exploiting a ierarcy of coarser models. In its raw form it is an optimization framework, not an algoritm, since it depends on an underlying (traditional) optimization algoritm, ere labeled OPT. Tere is considerable flexibility in selecting te underlying optimization algoritm. Of course, te performance of MG/Opt will depend on te coice of OPT. MG/Opt is based on te ideas of te full approximation sceme (see, e.g., [29]) but is not equivalent to te full approximation sceme. Before giving a description of MG/Opt, it is necessary to say more about te underlying optimization algoritm OPT. It is assumed tat OPT is a convergent optimization algoritm in te sense tat, if appropriate assumptions on te objective function are satisfied ten lim (zj ) = 0 j were z j are te iterates computed by OPT. (See, for example, [20].) We will write OPT as a function of te form z + Opt( ( ), v, z, k) wic applies k iterations of te convergent optimization algoritm to te problem min (z) vtrz z wit initial guess z to obtain z +. If te parameter k is omitted, te optimization algoritm continues to run until its termination criteria are satisfied. If OPT is applied by itself to

5 Truncated Newton-based multigrid algoritm for CVT calculation 5 te ig-fidelity model, ten v = 0. In te context of te multilevel algoritm MG/Opt, non-zero coices of v will be used. To describe an iteration of MG/Opt we make reference to a ig-fidelity model and a low-fidelity model H. Te letters and H are used repeatedly to identify te igand low-fidelity information. Also required are a downdate operator I H and an update operator I H tat map sets of generators z from one level to anoter. Unlike te algoritm in [33], te version of MG/Opt ere uses a separate downdate operator Î H to map values of from te fine level to te coarse level. Here is a description of MG/Opt: Given an initial estimate of te solution z 0 on te fine level, set v = 0. Select non-negative integers k 1 and k 2 satisfying k 1 + k 2 > 0. Ten for j = 0, 1,..., set z j+1 MG/Opt( ( ), v, z j ) were te function MG/Opt is defined as follows: Coarse-level solve: If on te coarsest level, ten solve te optimization problem: Oterwise, z j+1 Opt( ( ), v, z j ). Pre-smooting: z Opt( ( ), v, z j, k 1) Recursion: Compute z H = I H z v = Î H v + H ( z H ) Î H ( z ) Apply MG/Opt recursively to te surrogate model: z + H MG/Opt( H( ), v, z H ) Compute te searc directions e H = z + H z H and e = I H e H. Use a line searc to determine z + = z + αe satisfying (z + ) ( z ). Post-smooting: z j+1 Opt( ( ), v, z +, k 2) Because te integers k 1 and k 2 satisfy k 1 +k 2 > 0, eac iteration of MG/Opt includes at least one iteration of te convergent optimization algoritm OPT. Tis fact, in combination wit te line searc to determine z +, makes it possible to prove tat MG/Opt is guaranteed to converge in te same sense as OPT [33].

6 6 Zicao Di, Maria Emelianenko, Stepen G. Nas In tis paper, we ave cosen OPT to be te truncated-newton algoritm TN [31]. Wen applied to an optimization problem of te form min z f (z), at te j-t iteration a searc direction p is computed as an approximate solution to te Newton equations 2 f (z j )p = f (z j ) were z j is te current approximation to te solution of te optimization problem. Te searc direction p is computed using te linear conjugate-gradient algoritm. Te necessary Hessian-vector products are estimated using finite differencing. Te TN algoritm only requires tat values of f (z) and f (z) are computed. TN as low storage requirements, and as low computational costs per iteration, and ence is suitable for solving large optimization problems [34]. 3. Applying MG/Opt to te CVT Formulation To make sure we are dealing wit te same underlying problem on different levels, we scaled te objective function value by k 2 so tat te optimal energy value would be same on every level for te uniform density. Tis is also approximately true for oter density functions. Te coarser grids were obtained by standard coarsening, i.e. doubling grid size at every level: N H = 1 2 N, were N and N H are te numbers of grid points at te finer and coarser grids, respectively. To define our algoritm for solving te CVT problem, we need to adapt te formulation above to te CVT context and specify te appropriate coices of te operators and algoritm parameters MG/Opt setup It turns out tat in 1-dimensional case te following coice of te transfer operators works best in te CVT MG/Opt context. Transfer of te solution from finer to coarser grid (error restriction) is done via simple injection, i.e. given a vector v on te fine level, te downdate operator I H samples v at te even indices: [I H v ] i = v 2i, i = 1, 2,..., k/2. Te corresponding downdate operator Î H for te gradient (gradient restriction) tat maps values of from te fine level to te coarse level is given by [Î H v ] i = 1 2 v2i 1 + v 2i v2i+1, i = 1, 2,..., k/2. Tis form corresponds to te a standard coice of a full weigting (FW) restriction operator scaled by a factor of 2, wic comes from scaling te objective function at eac level.

7 Truncated Newton-based multigrid algoritm for CVT calculation 7 Te update (interpolation) operator I H is ten given by a standard bilinear interpolation operator, wic satisfies I H = 2(FW )tr (see p.61 of [39] for example), as is te case wen solving an optimization problem based on a discretized PDE in one dimension. For our coice of te gradient restriction we obtain te relationsip I H = (Î H )tr, tat we rely on in our implementation. Similarly to te 1D, transfer of te solution from finer to coarser grid in 2D can be computed by injection. However, te oter transfer operators ave to be cosen differently due to te differences in te CVT geometry. For te gradient downdate operator te following coice as been sown to work: [Î H v ] i = α i j v j, were α i i = 1 and α i j = 1 2 for any j s.t. z j is a fine node saring an edge wit z i in te fine level triangulation. In te calculations provided in Section 4, we fix tese weigts according to te equilibrium configuration. Altoug tis setup is specific to te situation wen te exact solution as a ierarcical structure wit a nested set of grids (like in te case of triangular domain wit ρ = 1), it can be generalized to arbitrary domains via recomputing te connectivity matrix at eac step and re-assigning te interpolation weigts accordingly. Te exact procedure will be provided in a fortcoming publication. Te corresponding update operator for te error can be provided by te formula analogous to te 1-D case: I H = 4(Î H )tr. In bot dimensions, for a V 1,1 cycle implemetation, one pre-smooting and one postsmooting relaxations were used (k 1 = k 2 = 1), wic corresponds to applying one iteration of OPT in eac of tese steps. We ave tested several convergence criteria. As we demonstrate in Section 4, te best convergence was acieved wen bot OPT and MG/Opt were terminated due to saturation of te energy gradient (z) (z ) 10 8, were z is te solution to te CVT problem. In 1-dimensional nonlinear density case, te estimation of te exact solution z was obtained by running OPT wit a tigt convergence tolerance. Exact solution was computed analytically for te case of constant density in bot dimensions. It is also wort mentioning tat we reordered te generators at eac energy evaluation, since te order is not necessarily preserved by te optimization procedure described above. Te symmetry of te CVT energy wit respect to re-orderings of te points {z i } k i=1 assures tat te objective function is not suffering from tis modification. j 4. Numerical Experiments All numerical experiments ave been performed on 2-core Intel Duo CPU E GHz platform wit 3.25 GB of RAM and te running times migt differ for oter configurations.

8 8 Zicao Di, Maria Emelianenko, Stepen G. Nas We did not use any preprocessing to generate te starting configuration z 0 for our algoritm, wic was simply obtained from a random sampling in te interval [0, 1]. In Matlab, we reset te random generator seed eac time and supplied te initial generators by means of te commands rand( state,0),v0 = sort(rand(k,1)) to eliminate te effect of initial configuration wen bencmarking algoritm performance. Te coarse grid consisted of no more tan 2-3 generators in all numerical tests presented below dimensional examples All 1-dimensional examples ave been computed on te [0, 1] domain. In te 1- dimensional case, te objective function (1.1) and its gradient can be integrated analytically in te case of ρ = 1 and require numerical approximation for more complicated cases. For te types of nonlinear densities cosen in our numerical experiments we ave tested several quadratures, including Simpson s and 15-point Gauss-Kronrod rules, wit no significant cange in algoritm performance. In te numerical experiments below we compare te multilevel algoritm MG/Opt wit te single level counterpart (OPT) given by te Truncated Newton (TN) metod. Since te TN algoritm is minimizing te energy at every step, te approac is comparable to te Lloyd algoritm (see [6]), wic is te usual bencmark used for CVT algoritms. We measure te computational cost of OPT by counting te number of fine-level gradient evaluations of, wic estimates te dominant cost of using OPT. For MG/Opt we count te number of equivalent fine-level gradient evaluations. Tat is, we determine (for eac level) te relative cost of a gradient evaluation compared to an evaluation on te fine level. Our first test uses te uniform density ρ(y) = 1 wit 512 variables. Tis is an easy problem. MG/Opt converges at a fast linear rate, about 12 times as fast as OPT (see Figure 1a). Next we use te density ρ(y) = 6 y 2 e 2y3. Te results are in Figure 1b. In tis case te performance of MG/Opt is almost te same, but OPT converges more slowly. Again we are able to acieve fast linear convergence using te multilevel metod. Te convergence factor ere is computed as follows: (z k+1 ) (z ) 1 k+1 C = (z 1 ) (z, (4.1) ) were (z ) is te approximation of te exact solution precomputed by running OPT until saturation. In Figure 2 we analyze te performance of MG/Opt and OPT as te size of te problem increases. In tese tests we used te uniform density ρ(y) = 1 and averaged te results over 6 independent runs wit random initial guesses. In te left plot we display te number of iterations needed for MG/Opt and OPT to compute te objective function value to a specified accuracy. In te middle figure we display te time needed to do tis. Bot MG/Opt and TN are expected to converge at a linear rate, and te rate constant ( convergence factor ) can be estimated from te results of eac test. Te rigt plot displays te rate constants for bot algoritms for different problem sizes. As can be seen, for MG/Opt te rate constant is insensitive to te problem size, wereas for OPT te rate constant

9 Truncated Newton-based multigrid algoritm for CVT calculation 9 (a) density ρ(y) = 1 (b) density ρ(y) = 6 y 2 2 y3 e Figure 1: Comparison of MG/Opt versus OPT: (a) for density ρ(y) = 1, (b) density ρ(y) = 6 y 2 e 2 y3. Circle: MG/Opt; Star: OPT; Dot: Lloyd. deteriorates as te problem size increases. Exact numerical values for bot number of cycles, convergence factor and elapsed time are given in Table 1. Bot MG/Opt and OPT algoritms ave been stopped wen te residual reaces a tolerance tresold of Figure 2: Solving problems of increasing size, MG/Opt versus OPT (ρ( y) = 1). Blue- Square: MG/Opt, Red-Circle: OPT; Cycle numbers and Time elapsed are log scaled Since te CVT algoritms in general sow sensitivity to te coice of te initial configuration, we ran several tests wit an intentionally bad coice of te initial guess, wit all generators clustered near te origin in te [0, 1] region. As Figure 3 demonstrates, no significant difference in performance as been noted. Figure 4 sows te performance of MG/Opt wen different stopping criteria are used. Again we use te uniform density ρ(y) = 1. From te plot, we can see tat te common stopping criterion based on te improvement in te residual (wic is related to te difference between te positions of generators in te approximated and exact solution) is not te ideal coice ere since it leads to a sligt increase in te iteration count as te problem size increases. For te oter two criteria based on te energy function, bot te number of cycles and te convergence factors in MG/Opt are not sensitive to

10 10 Zicao Di, Maria Emelianenko, Stepen G. Nas convergence factor number of cycles time(sec) k MG/Opt OPT MG/Opt OPT MG/Opt OPT Table 1: Comparing performance of MG/Opt wit 1-level optimization for ρ = 1 Figure 3: Performance of MG/Opt wit different initial configurations. Red-Circle: OPT; Green-Star: MG/Opt wit bad initial condition; Blue-Square: MG/Opt wit random initial condition. Number of cycles and elapsed time axes are log-scaled. problem size. Te oter reason to prefer te criterion basen on energy values rater tan positions is te abundance of local minima in te large-scale CVT problems, wic makes it impossible to pinpoint te exact solution te metod converges to. Te energy value provides a reasonable measure for te closeness to a particular exact solution. However, as demonstrated by Figure 4, bot criteria are acceptable and performance differences are minor. It is also wort noting tat based on te time complexity sown ere te algoritm is linearly scalable a desirable feature of te multigrid formulation tat allows for an efficient parallelization for modern large-scale computing applications. Figure 5 demonstrates te performance of MG/Opt wen applied to problems wit different density functions. In particular we consider ρ 1 (y) = y and ρ 2 (y) = 6 y 2 e 2 y3. In te left plot, we can see tat te iteration count for ρ 1 stays constant wit te increase of te problem size, and increases only sligtly for ρ 2. For bot densities te convergence factor exibits insensitivity to te problem size, altoug te convergence factor is sligtly larger for ρ 2, wile remaining bounded from above by 0.2.

11 Truncated Newton-based multigrid algoritm for CVT calculation 11 Figure 4: Performance of MG/Opt wit different stopping criteria (ρ( y) = 1). Blue-Dot: (z) (z ) 10 8 ; Red-Circle: (z) (z ) ; Green-Square: Figure 5: Performance of MG/Opt wit different density functions. Circle: ρ( y) = y. Star: ρ(y) = 6y 2 e 2y3 Figure 6 compares te performance of MG/Opt to tat of te multilevel optimization based metod introduced in [5]. Altoug te latter was also based on te idea of formulating an optimization problem based on te CVT energy (1.1), it as significant differences wit te MG/Opt approac. Essentially, te previous sceme represented a successive correction algoritm wit a suitably defined domain decomposition, wit te Lloyd iteration used as a relaxation on eac level ence te name Multilevel-Lloyd we use to refer to it in tis work. Comparison of performance of bot metods for constant and linear densities sows tat MG/Opt yields sligtly lower convergence factors, wile sowing significant advantage in terms of te time complexity. Table 2 compares te performance of MG/Opt wit tat of te OPT algoritm and te Multigrid Lloyd-Max (MLM) algoritm introduced in [24], wic is based on a FAS formulation of te Lloyd-Max iterative process. Te same form of te nonlinear density y = 6x 2 e 2x3 as te one tested in [24] as been cosen for fair comparison. Te column MG [24] sows te convergence factors reported in [24], were te convergence factors

12 12 Zicao Di, Maria Emelianenko, Stepen G. Nas (a) density ρ(y) = 1 (b) density ρ(y) = y Figure 6: Convergence factors for MG/Opt vs. Multilevel-Lloyd [5] for (top) uniform density ρ( y) = 1; (bottom) linear density ρ( y) = y. Notations: Dot:MG/Opt; Star:Multilevel/Lloyd were computed using C = (zk+1 ) (z ) (z k ) (z ) (4.2) Te column MG/Opt 1 lists te convergence factors for our algoritm MG/Opt computed te save way. Te column MG/Opt 2 lists te convergence factors for MG/Opt computed according to te formula (4.1), as we did in our earlier discussion. Tis latter formula is not as sensitive to te performance of te algoritm at te final iteration. It is also te formula used in our oter numerical experiments wen computing te convergence factor. Notice tat te convergence factors for MG/Opt are significantly better tan for MLM [24]. Te fact tat convergence factors stay very small even for muc larger problems suggests tat te new metod as te potential to outperform earlier formulations, even for igly nonlinear densities dimensional examples Below are some preliminary results tat ave been obtained in te 2-dimensional case for te triangular domain. Te coice of te domain is motivated by te availability of eas-

13 Truncated Newton-based multigrid algoritm for CVT calculation 13 convergence factor number of cycles k MG/Opt 2 MG/Opt 1 MLM [24] OPT MG/Opt 2 OPT Table 2: Comparing performance of MG/Opt wit 1-level optimization (OPT) and te Multilevel Lloyd-Max (MLM) metod of [24] for a nonlinear density: y = 6x 2 e 2x3. MG/Opt 1 denotes te MG/Opt algoritm wit te convergence factor computed according to (4.2), MG/Opt 2 uses te geometric mean convergence factor (4.1); MLM [24] is te result from [24] wit te same convergence factor formula as MG/Opt 1, refers to data not provided in [24] convergence factor number of cycles k MG/Opt OPT Lloyd MG/Opt OPT Lloyd Table 3: Comparing performance of 2-dimensional MG/Opt wit 1-level optimization (OPT) and Lloyd metod for constant density in triangular domain. ily computable exact solution and te absence of boundary effects inerent to rectangular saped regions. In Figure 7, we provide convergence results for MG/Opt and its competitors wen te initial guess is taken to be relatively close to te local minimizer. It is clear tat MG/Opt maintains its superiority over 1-level metods. Moreover, MG/Opt as a significantly lower convergence factors independent of te problem size, similar to te beavior observed in 1-dimensional case. Table 3 provides numerical data for te performance of MG/Opt, OPT and Lloys metods for te case of initial guess taken furter away from te minimizer. Again, tere is a clear advantage in using MG/Opt in tis case. It as also been noted tat te tendency to move points outside of te domain typical for te regular 1-level OPT metod is significantly reduced in te 2-dimensional MG/Opt implementation. Overall, wile te above estimates are only a first step in a compreensive analysis of

14 14 Zicao Di, Maria Emelianenko, Stepen G. Nas Figure 7: Comparison of performance of MG/Opt and OPT for te case of a triangular 2-d region wit ρ = 1. Red-Circle: OPT, Blue-Square: MG/Opt. te 2-dimensional version of te algoritm, and more work needs to be done to make te metod work efficiently for general 2-d domains and density functions, tey give a reason to believe MG/Opt will be competitive to oter metods. Analysis of te performance of te 2-d metod for different coices of transfer operators, and te creation of a robust and versatile general-purpose solver are te focus of current investigations. 5. Complexity of MG/OPT for 1-d CVT problem In tis section, we analyze te computational complexity of a V 1,1 cycle of te MG/Opt algoritm for CVT, i.e. k 1 = k 2 = 1. At eac level, MG/Opt consists of Pre-smooting and Post-smooting wic amount to two iterations of te Truncated Newton (TN) metod, plus te cost of transferring te solution from one grid to anoter (downdate and update) and line searc. Leaving te TN part aside, let us first compute te work required for all oter parts of te algoritm, similar to te analysis performed in [34]. Notice tat: (a) downdating te variables is carried out by means of simple injection, so no additional computational cost is involved; (b) te update operator is applied to alf of te points in te grid during te refinement process; (c) te process of downdating and finding local searc directions is only applied to alf of te points in te coarsening process; (d) tese steps are performed in te beginning and at te end of eac cycle. Te setup is performed once per outer iteration. Wit tese observations in mind, we obtain te estimate of computational cost for all of tese operations to be less tan 9k for te problem of size k, as outlined in Table 4. To estimate te complexity of te TN part, notice tat if we only do one iteration of TN in pre-smooting and post-smooting, respectively, te following computations are to be performed: 1 infinity-norm calculation, 1 vector addition and 2 function-gradient evaluations Conjugate-gradient (CG) iteration wit cost per iteration given by: 1 function-gradient evaluation,

15 Truncated Newton-based multigrid algoritm for CVT calculation 15 Operations per coordinate on te fine level Downdate(variable) 0 0 Downdate(residual) 3 3k/2 Searc Direction 1 k/2 Update 3 3k/2 Line searc 1 k Table 4: Te complexity of various parts of MG/Opt measured in floating point operations. 4 inner product and 5 vector + constant vector" operations. On average, we do 5 CG iterations per TN iteration. Based on te above estimate, one TN iteration requires 1 infinity norm, 20 inner product, 25 vector + constant vector", 1 vector add calculations and 7 function/gradient evaluations. If te cost of infinity norm calculation is estimated to be no more tat k flops (floating-point operations, wic include additions, subtractions, products and divisions), wit k being te number of grid points at te finest level, and te cost of eac te inner product and vector + constant vector" calculations is taken to be 2k flops, te overall cost of one TN iteration amounts to k( ) = 47k flops, plus 7 function-gradient evaluations. Since in eac V 1,1 cycle we do 2 TN iterations at eac level and te contribution from coarser grids is estimated from above as log k i=0 2 i < 2, we multiply tis estimate by a factor of 4, and add te additional costs as specified above. We conclude tat te typical cost of a MG/Opt V-cycle is ( )k = 197k, plus 28k function-gradient evaluations. Tis proves te fact tat te above algoritm as O(k) complexity. Tis bound can be lowered by performing less safeguarding as part of te TN algoritm at eac step, owever, we feel tese steps are needed in order to obtain te most robust implementation. 6. Comments and Conclusions In conclusion, we ave introduced a novel way of computing CVTs by means of an O(k) complexity algoritm based on te MG/Opt strategy originally used only for large-scale nonlinear optimization problems. Te main advantage of te new metod is its superior convergence speed wen compared to oter existing approaces. Even toug te computational complexity of eac cycle is not te lowest, te fact tat te convergence factors are on average muc lower tan tose of its counterparts makes tis formulation attractive wen it comes to solving large CVT problems. Te simplicity of its design and te results of preliminary tests presented ere suggest tat te metod is not only generalizable to iger dimensions but retains its superior convergence properties wit te appropriate modification of parameters. Te compreensive study of te algoritm s performance for arbitrary 2-dimensional domains and arbitrary density functions, as well as te design of optimal transfer operators are te subject of current investigations. Rigorous convergence

16 16 Zicao Di, Maria Emelianenko, Stepen G. Nas teory development for tis type of a metod is nontrivial but possible, and will be addressed in a future publication. Future work also includes application of tis tecnique to various scientific and engineering problems, including image analysis and grid generation. 7. Acknowledgements Te work of Zicao Di and Stepen G. Nas was supported by te U.S. Department of Energy under Award DE-SC Maria Emelianenko acknowledges te support from te ORAU Ralp E. Powe Junior Faculty Enancement Award. References [1] F. AURENHAMMER, Voronoi diagrams. a survey of a fundamental geometric data structure, ACM Computing Surveys, 23 (1990), pp [2] J. CORTES, S. MARTINEZ, T. KARATA, AND F. BULLO, Coverage control for mobile sensing networks, IEEE Transactions on Robotics and Automation, 20 (2004), pp [3] Q. DU AND M. EMELIANENKO, Acceleration of algoritms for te computation of centroidal Voronoi tessellations, Numerical Linear Algebra and its Applications, 13 (2006), pp [4], Uniform convergence of a multilevel energy-based quantization sceme, in Lecture Notes in Computer Science and Engineering, O. B. Widlund and D. E. Keyes, eds., vol. 55, Springer, Berlin, 2007, pp [5], Uniform convergence of a nonlinear energy-based multilevel quantization sceme via centroidal Voronoi tessellations, SIAM Journal on Numerical Analysis, 46 (2008), pp [6] Q. DU, M. EMELIANENKO, AND L. JU, Convergence properties of te Lloyd algoritm for computing te centroidal Voronoi tessellations, SIAM Journal on Numerical Analysis, 44 (2006), pp [7] Q. DU, V. FABER, AND M. GUNZBURGER, Centroidal Voronoi tessellations: applications and algoritms, SIAM Review, 41 (1999), pp [8] Q. DU AND M. GUNZBURGER, Grid generation and optimization based on centroidal Voronoi tessellations, Applied Matematics and Computation, 133 (2002), pp [9] Q. DU, M. GUNZBURGER, AND L. JU, Mesfree probabilistic determination of points, support speres, and connectivities for mesless computing, Computational Metods in Applied Mecanics and Engineering, 191 (2002), pp [10], Constrained centroidal Voronoi tessellations on general surfaces, SIAM Journal on Scientific Computing, 24 (2003), pp [11] Q. DU, M. GUNZBURGER, AND L. JU, Advances in studies and applications of centroidal voronoi tessellations, NMTMA, 3 (2010), pp [12] Q. DU AND D. WANG, Tetraedral mes generation and optimization based on centroidal Voronoi tessellations, International Journal for Numerical Metods in Engineering, 56 (2002), pp [13], Recent progress in robust and quality Delaunay mes generation, Journal of Computational and Applied Matematics, 195 (2006), pp [14] R. DWYER, Higer-dimensional Voronoi diagrams in linear expected time, Discrete and Computational Geometry, 6 (1991), pp [15] M. EMELIANENKO, Fast multilevel CVT-based adaptive data visualization algoritm, Numerical Matematics: Teory, Metods and Applications, 3 (2010), pp

17 Truncated Newton-based multigrid algoritm for CVT calculation 17 [16] M. EMELIANENKO, L. JU, AND A. RAND, Weak global convergence of te Lloyd metod for computing centroidal Voronoi tessellations in R d, SIAM Journal on Numerical Analysis, 46 (2008), pp [17] S. FORTUNE, A sweepline algoritm for Voronoi diagrams, Algoritmica, 2 (1987), pp [18] A. GERSHO, Asymptotically optimal block quantization, IEEE Transactions on Information Teory, 25 (1979), pp [19] R. GRAY AND D. NEUHOFF, Quantization, IEEE Transactions on Information Teory, 44 (1998), pp [20] I. GRIVA, S. NASH, AND A. SOFER, Linear and Nonlinear Optimization, SIAM, Piladelpia, [21] S. HILLER, H. HELLWIG, AND O. DEUSSEN, Beyond stippling metods for distributing objects on te plane, Computer Grapics Forum, 22 (2003), pp [22] L. JU, Q. DU, AND M. GUNZBURGER, Probabilistic metods for centroidal Voronoi tessellations and teir parallel implementations, Parallel Computing, 28 (2002), pp [23] Y. KOREN AND I. YAVNEH, Adaptive multiscale redistribution for vector quantization, SIAM Journal on Scientific Computing, 27 (2006), pp [24] Y. KOREN, I. YAVNEH, AND A. SPIRA, A multigrid approac to te 1-D quantization problem, tec. report, [25] B. LÉVY AND Y. LIU, L p centroidal Voronoi tessellation and its applications, in ACM Transactions on Grapics, SIGGRAPH conference proceedings, vol. 24, Article 119. [26] R. LEWIS AND S. NASH, Model problems for te multigrid optimization of systems governed by differential equations, SIAM Journal on Scientific Computing, 26 (2005), pp [27] Y. LINDE, A. BUZO, AND R. GRAY, An algoritm for vector quantizer design, IEEE Transactions on Communications, 28 (1980), pp [28] S. LLOYD, Least square quantization in PCM, IEEE Transactions on Information Teory, 28 (1982), pp [29] S. MCCORMICK, Multilevel Projection Metods for Partial Differential Equations, Society for Industrial and Applied Matematics, [30] A. MENDES AND I. THEMIDO, Multi-outlet retail site location assessment, International Transactions in Operational Researc, 11 (2004), pp [31] S. NASH, TN/TNBC software. ttp://www-fp.mcs.anl.gov/otc/guide/softwareguide/ Blurbs/tn_tnbc.tml, [32], A multigrid approac to discretized optimization problems, Journal of Computational and Applied Matematics, 14 (2000), pp [33], Convergence and descent properties for a class of multilevel optimization algoritms, tec. report, Systems Engineering and Operations Researc Department, George Mason University, Fairfax, VA, [34] S. NASH AND J. NOCEDAL, A numerical study of te limited memory BFGS metod and te truncated-newton metod for large scale optimization, 1 (1991), pp [35] A. OKABE, B. BOOTS, AND K. SUGIHARA, Spatial Tessellations; Concepts and Applications of Voronoi Diagrams, Wiley, Cicester, [36] G. RONG, Y. LIU, W. WANG, X. YIN, X. GU, AND X. GUO, GPU-assisted computation of centroidal Voronoi tessellation, IEEE Transactions on Visualization and Computer Grapics, 16 (2010). to appear. [37] J. SABIN AND R. GRAY, Global convergence and empirical consistency of te generalized Lloyd algoritm, IEEE Transactions on Information Teory, IT 32 (1986), pp

18 18 Zicao Di, Maria Emelianenko, Stepen G. Nas [38] A. TRUSHKIN, On te design of an optimal quantizer, IEEE Transactions on Information Teory, 39 (1993), pp [39] U.TROTTENBERG, C.W.OOSTERLEE, AND A.SCHULLER, Multigrid, (2001), pp [40] S. VALETTE AND J. CHASSERY, Approximated centroidal Voronoi diagrams for uniform polygonal mes coarsening, Computer Grapics Forum, 23 (2004), pp [41] C. N. VASCONCELOS, A. SÁ, P. C. CARVALHO, AND M. GATTASS, Lloyd s algoritm on GPU, in ISVC 08: Proceedings of te 4t International Symposium on Advances in Visual Computing, Berlin, Heidelberg, 2008, Springer-Verlag, pp [42] C. WAGER, B. COULL, AND N. LANGE, Modelling spatial intensity for replicated inomogeneous point patterns in brain imaging, Journal of Royal Statistical Society B.