Alternating Minimization Algorithm with Iteratively Reweighted Quadratic Penalties for Compressive Transmission Tomography

Transcription

1 Alternating Minimization Algorithm with Iteratively Reweighted Quadratic Penalties for Compressive Transmission Tomography Yan Kaganovsky a, Soysal Degirmenci b, Shaobo Han a, Ikenna Odinaka a, David G. Politte c, David J. Brady a, Joseph A. O Sullivan b and Lawrence Carin a a Department of Electrical and Computer Engineering, Duke University, Durham, NC, USA b Electrical and Systems Engineering, Washington University in St. Louis, MO, USA c Mallinckrodt Institute of Radiology, Washington University in St. Louis, MO, USA ABSTRACT We propose an alternating minimization (AM) algorithm for estimating attenuation functions in x-ray transmission tomography using priors that promote sparsity in the pixel/voxel differences domain. As opposed to standard maximum-a-posteriori (MAP) estimation, we use the automatic relevance determination (ARD) framework. In the ARD approach, sparsity(or compressibility) is promoted by introducing latent variables which serve as the weights of quadratic penalties, with one weight for each pixel/voxel; these weights are then automatically learned from the data. This leads to an algorithm where the quadratic penalty is reweighted in order to effectively promote sparsity. In addition to the usual obect estimate, ARD also provides measures of uncertainty (posterior variances) which are used at each iteration to automatically determine the trade-off between data fidelity and the prior, thus potentially circumventing the need for any tuning parameters. We apply the convex decomposition lemma in a novel way and derive a separable surrogate function that leads to a parallel algorithm. We propose an extension of branchless distance-driven forward/back-proections which allows us to considerably speed up the computations associated with the posterior variances. We also study the acceleration of the algorithm using ordered subsets. Keywords: Alternating minimization algorithms, automatic relevance determination, reweighted L2, posterior variance estimation, separable surrogates, edge preserving priors, image reconstruction, helical CT.. Statement of the Problem. INTRODUCTION We consider x-ray transmission computed tomography (CT) with Poisson noise. The statistical model for the observations y N n is given by p(y x) = Pois[η exp( Φx)], () where denotes an elementwise product; each row in the matrix Φ R n p + represents the lengths of intersection ofaraywitheachpixel/voxelandapproximatesalineintegral;x R p + arethelinearattenuationcoefficientstobe determined and η are the mean of the measurements in the absence of the obect. Image reconstruction methods based on penalized log-likelihood can be interpreted from a Bayesian perspective as finding the maximum-a posteriori (MAP) solution (Type I) for the parameters x. Often the penalty is chosen to promote sparsity of the pixel/voxel differences. More specifically, the MAP solution can be written as x map = argmax x [ logp(y x) + logp(x) ] where p(y x) is the likelihood function, given by Eq. (), and p(x) is the prior. Further author information: s: yankagan@gmail.com, s.degirmenci@wustl.edu, shaobohan@gmail.com, ikenna.odinaka@duke.edu, politted@wustl.edu, dbrady@duke.edu, ao@wustl.edu, lcarin@duke.edu.

2 .2 Automatic Relevance Determination (ARD) A different approach to sparse estimation is automatic relevance determination (ARD), where sparsity is promoted by treating x as latent variables with prior p(x γ), where γ are newly introduced hyperparameters, and then seeking the γ that maximizes the marginal (Type II) likelihood p(y γ). A special case of ARD, called sparse Bayesian learning (SBL), 2 is based on a Gaussian noise model for y where each parameter x is assigned a zero-mean Gaussian prior with variance γ. Learning the hyperparameters γ from the data results in γ 0 for some, so the corresponding parameters are effectively pruned from the model. Searching for the maximum of the ARD Type II likelihood p(y γ) is intractable. For Gaussian likelihoods, an expectation-maximization (EM) algorithm 2 is used. For non-gaussian likelihoods, the E-step is not tractable and one solution is to use the Laplace approximation 2 where the posterior distribution is approximated by a Gaussian with mean equal the MAP solution and covariance matrix equal to the inverse of the Hessian at this point. Unfortunately, the resulting EM-like algorithm is not based upon a consistent obective function, which prevents monitoring convergence. In addition, it requires expensive inversions of the Hessian matrix, which is not practical for large-scale problems..3 Extending ARD We propose a computationally efficient extension of ARD which can be used for Poisson noise models and that can be applied in practice to large-scale problems such as x-ray CT. We propose the prior p(x;γ) = N(x;0,(Ψ T Γ Ψ) ), Γ = diag(γ), (2) which as opposed to the original SBL, 2 promotes sparsity in the transform domain s = Ψx R p, i.e., we desire s to have many near-zero elements, but not necessarily x. For now, we assume that Ψ is a square matrix but this requirement will be relaxed later. We propose an alternating-minimization (AM) algorithm to compute the hyperparameters in (2). The algorithm will be shown to promote sparsity in the pixel/voxel differences domain. The proposed AM algorithm provides an image of the obect estimate, which can be compared to standard image reconstruction methods. It also provides a variance image which reflects the confidence in the estimated quantities. The latter is not computed automatically by standard image reconstruction algorithms. Importantly, we will show that the variances have a key role in automatically determining the balance between the data-fit and the prior, leading to a tuning-free algorithm. We term this framework variational automatic relevance determination (VARD). The obective of this paper is to study the performance of VARD for helical CT and medical applications. We also propose a method to compute new types of forward/back-proections that are required by VARD and are associated with the posterior variances. The method is an extension of branchless distance-driven forward/backproections proposed by Basu and De Man. 3,4 In addition, we study ordered subsets implementations ALTERNATING-MINIMIZATION ALGORITHM FOR ARD 2. Extending the EM-ARD Algorithm to Poisson Likelihoods First, we address the intractability of the E-step in the EM algorithm for the Poisson model in (). We use a well-known alternative view of the EM algorithm based on minimizing the free variational energy (FVE) 6 where E : q (t+) = argmin q F[q(x),γ (t) ] (3) M : γ (t+) = argmin γ F[q (t+) (x),γ], (4) F[q(x),γ] = E q(x) log[q(x)/p(y,x γ)], (5) and E q denotes the expectation with respect to the probability distribution q. In the EM algorithm, q (t+) (x) = p(x y,γ (t) ), i.e., the exact posterior distribution, and F is the free variational energy (FVE). Instead of finding

3 the posterior distribution during the E-step (which is intractable), we limit the minimization of the FVE to a tractable parametric family of distributions q(x;m,v) = N(x ;m,v ), (6) where the mean m represents our estimate for the attenuation image and the entries in v are the estimates of the posterior variances which provide confidence intervals about m. We consider a factorized distribution in (6) for computational efficiency. Substituting p(y, x γ) = p(y x)p(x γ) into (5) and limiting q according to (6), we obtain the following alternating-minimization (AM) algorithm Backward : (m (t+),v (t+) ) = arg min m 0,v 0 E q(x;m;v) [ logp(y x) ] +DKL [q(x;m,v) p(x;γ (t) )] (7) Forward: γ (t+) = argmin γ D KL [q(x;m (t+),v (t+) ) p(x;γ)], (8) where D KL denotes the Kullback Leibler (KL) divergence. The first term of the right side of (7) does not depend on γ and was omitted in (8). In the B-step in (7) we update the approximation to the posterior q by searching for (m,v). Note that the B-step is equivalent to minimizing the KL divergence between q and the true posterior p(x y), so for a given γ, this q is the best approximation to the true posterior in the KL sense. In the F-step in (8) we update the generative (forward) model by finding γ. By definition, the FVE is reduced at each iteration and the two steps are repeated until convergence. Importantly, as shown below, the iterations of the AM algorithm lead to near-sparse solutions, i.e., in the s = Ψx domain, q of (6) is highly concentrated about zero for many s i. In contrast, a single B-step in (7) for the ARD model does not promote sparsity. For additional discussions on the sparsity promoting mechanism of VARD we refer the reader to Kaganovsky et al. 7 The obective function in (7) can be written explicitly as F = F (m,v)+f 2 (m,γ) (9) F (m,v) = i [ {}}{ y i φ i m + η i exp( φ i m + ] φ 2 i v /2) i T2 {}}{ F 2 (m,γ) = γi ( ψ i m ) 2 + γi ψi v logv + logγ i, () 2 2 i where ψ i and φ i denote the entry on the ith row and th column of the matrices Ψ and Φ, respectively; we have marked the terms which are not additively separable by T and T2 for future reference. The solution to the F-step in (8) is obtained in closed form by solving γ D KL [q p] = 0 and is given by T i (0) γ (t+) i = ( ψ i m ) 2 + ψ 2 iv. (2) One choice for Ψ in (2) is logp(x γ) = i γ (x i χ i x / χ ) 2 +Const, where χ i is the neighborhood of the ith voxel and χ is the number of neighbors. To make Ψ invertible, one can enforce zero differences at the boundaries (Neumann boundary conditions). This prior will promote sparsity in the differences between each mean voxel m and the average of its neighboring mean voxels. However, we have found in some preliminary experiments 7 that the image quality improves if instead we use the following Ψ Ψ = [Ψ x ;Ψ y ;Ψ z ], (Ψ x,y,z ) k = { if = k if {,k} Θ x,y,z 0 otherwise, (3) where [; ] denotes concatenation, and Θ x, Θ y and Θ z define pairs of neighboring voxels located along the x, y and z directions, with corresponding matrices Ψ x,ψ y, and Ψ z respectively. We assign the same weight to each voxel difference, so γ is replaced by [γ;γ;γ]. In this case, the F-step in (8) can no longer be interpreted as minimizing KL divergence, but the obective function in (9) () remains the same as before.

4 2.2 Separable Surrogates for Parallel Computing First, we make several definitions to simplify the following derivations, p (t) i φ i m (t), p(t) i φ 2 i v(t), (4) µ (t) [ i E q(x;m (t),v (t) ) ηi exp( φ i x ) ] = η i exp( p (t) i /2)exp( p (t) i ), (5) b y i φ i y i, b (t) i φ i µ (t) i, b(t) i φ 2 iµ (t) i /2, (6) where y i and η i are defined in (). To distinguish between quantities associated with the mean and variance, we denote any quantities associated with the variance using a tilde. In (4), p (t) i is the proection of the posterior mean m (t) along the ith line, which we shall call a mean-type proection; p (t) i is a proection of the posterior variance v (t) along the ith line using an elementwise squared forward operator, which we term a variance-type proection. In (5), µ (t) i is the predicted Poisson rate for the ith measurement, based on the expectation with respect to the posterior distribution q computed at iteration t. In (6), b y is the back-proectionof measurements y to the th pixel; b (t) and b (t) are the back-proections of the predicted Poisson rate in (5) to the th pixel, using the backward (adoint) operator and the squared backward operator, respectively, which we call mean-type and variance-type backproections. We modify the B-step in (7) by applying the convex decomposition lemma 8 to the non-separable terms in (0) () marked by T and T2. This leads to a separable surrogate obective S which has the following properties, and it is given by F(m,v γ (t) ) S(m,v;m (t),v (t) ) (7) F(m (t),v (t) γ (t) ) = S(m (t),v (t) ;m (t),v (t) ), (8) S(m,v;m (t),v (t) ) = S m (m ;m (t),v (t) )+ S v (v ;m (t),v (t) )+h (t), (9) S m (m ;m (t),v (t) ) = b y m + b(t) exp [ Z (m m (t) Z )] +f (t) (m m (t) S v (v ;m (t),v (t) ) = )+ g(t) (m m (t) )2, (20) b(t) Z exp [ Z (v v (t) ) ] +ξ (t) v /2 logv /2, (2) where b (t) (t), b are given in (6) and the rest of the parameters are defined by Z = max i g (t) φ i +φ 2 i/2, Z 2 k ψ k /2γ (t) k, d (t) k Z 2 = max k ψ k m (t), f(t) k ψ k, ξ (t) k ψ k d (t) k /γ(t) k, (22) ψ 2 k/γ (t) k, h(t) k (d (t) k )2 /2γ (t) k. (23) Note that the surrogate function S in (9) is separable with respect to the components of both m and v. This yields a modified B-step where S is minimized by simultaneous (parallel) line search procedures, each with respect to a different component m or v. The iterations are defined as m (t+) v (t+) = argmins m (m ;m (t),v (t) ) (24) m 0 = argmins(v v ;m (t),v (t) ). (25) v 0

5 Combining the modified B-step in (24) (25) with the F-step in (2), we obtain the parallel VARD algorithm described in Algorithm. Each iteration requires a mean-type forward-proection p (t) (line 4 in Algorithm ) and a variance type forward-proection p (t) (line 5 in Algorithm ) which can be computed in parallel. It also requires a mean-type backproection b (t) (line 8 in Algorithm ) and a variance-type backproection (line 9 in Algorithm ), which also can be computed in parallel. For proof of convergence we refer the reader to Kaganovsky et al BRANCHLESS DISTANCE-DRIVEN VARIANCE-TYPE PROJECTION AND BACKPROJECTION Basu and De Man 3 proposed a variation of their distance-driven proection and backproection method 4 that is highly parallelizable and amenable to vectorization in highly pipelined architectures. They consider the usual proection and backproection operations where the system matrix Φ and its transpose are implicitly applied on a vectorized image. We use their method for the mean-type proections/backproections required by VARD (lines 4 and 8 in Algorithm ). However, in addition to these operations, the VARD algorithm also requires variance-type proection and backproection (see lines 5 and 9 in Algorithm ) where the system matrix is squared element-wise and then applied to a vector. Next, we propose an extension of the branchless distance-driven method 3 that allows us to perform the variance-type operations while maintaining the same computational advantages as for the mean-type operations. Note that ray-tracing-based proections are easily extendable to these variance-type operations, however, as explained in Basu and De Man, 3,4 the distance-driven approach allows one to eliminate artifacts that are associated with ray-based proections and can allow more efficient memory access patterns leading to reduced computational time in highly pipelined architectures. For simplicity, we present the algorithm for the 2D case, since the extension to 3D is straightforward. To simplify the presentation we shall present the original method by Basu and De Man 3 used for the mean type operations and then state our modification for the variance type operations. Distance-driven proection and backproection consist of two maor steps: () proections of detector and pixel boundaries onto a common axis (see Fig. ); (2) application of the overlap kernel. The overlap kernel for the mean operations can be described in an abstract way as follows. Suppose we are given a set of points p i on the x axis for i {,...,I} that are sorted p i < p i+. Associated with each successive pair of points {i,i+} is a signal value p i,i+. We are also given a second set of ordered points d for {,.,,,J} such that d < d + which define intervals [d,d + ) on which we wish to compute the value d,+ associated with this interval. Each signal value p i,i+ is weighted accordingto the overlapbetween the intervals[p i,p i+ ) and [d,d + ), as defined in (30). In the case of forward proection, d,+ is the proection along a ray and [p i,p i+ ) [d,d + ) /(d + d ) b (t) Algorithm VARD with Separable Surrogates : Initialize m (),v (),γ () 2: Compute b y in (6) % backproection of data % 3: for t = to N do % AM iterations % 4: p (t) Φm (t) % mean-type proections % 5: p (t) (Φ Φ)v (t) % variance-type proections % 6: µ (t) η exp( p (t) /2) exp( p (t) ) 7: Compute g (t), ξ (t) and f (t) defined in (22) (23) 8: b (t) Φ T µ (t) % mean-type backproections of µ (t) % 9: b(t) (Φ Φ) T µ (t) % variance-type backproections of µ (t) % 0: for = to p do % executed in Parallel % : m (t+) argmin m 0S m(m ;b (t),f(t),g (t) ) % see (20) % 2: v (t+) argmin v 0 Sv (v ; b (t),ξ(t) ) % see (2) % 3: γ (t+) [Ψm (t) ] [Ψm (t) ]+[Ψ Ψ]v (t) % see (2) %

6 p,2 p 2,3 x p p 2 p 3 x d d 2 d d 2 d,2 d,2 Figure. Left: an example of the proections of detector boundaries onto the x axis; Right: enlarged view of two of the proected detector boundaries shown on the left (red circles) with the relevant pixel boundaries (magenta circles). The pixel values are denoted by p,2 and p 2,3; the proection corresponding to the considered detector is denoted by d,2. The expressions for calculating the proections for mean and variance are given in (32) and (33), respectively. represents the ith entry in the system matrix. Similarly, we can define the overlap kernel for the variance-type operations, where the coefficients of the system matrix are squared. This corresponds to the overlap kernel defined in (3) with the weight of each signal value squared. Examples corresponding to Fig. are shown in (32) and (33). The main idea behind the branchless method by Basu and DeMan is to express the proections/backproections as resampling operations. They define the integral in (34), which is then expressed as a difference between two integrals; their values are determined by linear interpolation from the knowledge of the integral values on a chosen grid (exploiting the piecewise nature of the function in (35)). Here we follow a similar route, but we need to define a two-dimensional (2D) piecwise constant function in (37) and a 2D integral in (36), which is then expressed as d+ d+ d,+ = (d + d ) 2 p v (x,y)dxdy (26) d d = (d + d ) 2[P(d +,d + ) 2P(d +,d )+P(d,d )], (27) where we defined P(τ,τ 2 ) = τ τ2 p v (x,y)dxdy +C, (28) and used the symmetry in p v (x,y), with C being an arbitrary constant. The value of P(τ,τ 2 ) is obtained by a bilinear (quadratic) interpolation from its values on the pixel grid, which are given by P(i, ) = min(i,) k= p k,k+, (29) where = p i+ p i is the size of the grid. Note that the function p v (x,y) in (37) is non-zero only in diagonal blocks in the (x,y) space, where x and y belong to the same pixel. To summarize, the steps for a forward variance-type proection are: () Calculate the partial accumulations of the image (Eq. 29); (2) Bilinear interpolation; (3) Evaluation of Eq. (27). The backproection is obtained by performing the transposed operations in reverse order RESULTS USING CLINICAL DATASET Next we present some preliminary results. We note that optimization of the algorithm/software is still in progress. The latest images will be presented at the conference; they will also be available through the author s

7 Mean Overlap Kernel: Variance Overlap Kernel: d,+ = I i= pm i,i+ [p i,p i+ ) [d,d + ) (d + d ) (30) d,+ = I i= pv i,i+ [p i,p i+ ) [d,d + ) 2 (d + d ) 2 (3) Example (see Fig. ) d,2 = pm,2 (p 2 d )+p m 2,3 (d 2 p 2 ) d 2 d (32) Overlap Kernel can be written as d,+ = d+ p m (x)dx (34) (d + d ) d Example (see Fig. ) d,2 = pv,2(p 2 d ) 2 +p v 2,3(d 2 p 2 ) 2 (d 2 d ) 2. (33) Overlap Kernel can be written as d,+ = d+ d+ (d + d ) 2 p v (x,y)dxdy (36) d d where I ( ) p m x (pi+ +p i )/2 (x) = p i,i+ π p i= i+ p i (35) where p v (x,y) = I ( ) ( ) x (pi+ +p i )/2 y (pi+ +p i )/2 p i,i+ π π p i+ p i p i+ p i i= see definition of π(x) below (37) π(x) = for 0.5 x < 0.5, and zero otherwise Table. Definitions of the overlap kernels used in the distance-driven proections with mean and variance operations shown on the left and right, respectively. The kernel for mean-type operations has been introduced by Basu and DeMan. The kernel for variance-type operations is proposed here as a modification to their method. (37) personal webpage. 9 We used the HECTARE (Helical CT Advanced Reconstruction Engine) software package implemented in C++ by Daniel B. Keesing 0 and modified it to include the proposed VARD algorithm and the proposed variance-type branchless distance-driven operators. With approval from the Washington University Institutional Review Board (IRB), patient data was acquired on a Siemens Sensation 6 scanner at St. Louis Childrens Hospital using a standard abdominal imaging protocol with contrast agent. The Sensation 6 is a third-generation multi-detector-row computed tomography (CT) scanner and was operated in spiral scanning mode. The x-ray source orbits around the isocenter of the system every 0.5 seconds at a distance of 570 mm. The number of uniformly-spaced views per rotation is 60. The source is collimated to a width of.5 cm at the isocenter of the system. The detectors are arranged in 6 rows of 672 detectors, each on a cylindrical arc with radius 040 mm and centered at the source. Each detector subtends an arc of radians, so the entire fan subtends approximately 52. The center of the detector array was offset by /4 of a detector to improve sampling. The voltage and current of the x-ray tube were 20 kv and 50 mas, respectively. The patient bed traveled 24 mm per rotation of the gantry at a uniform speed. Figure 2 shows examples of reconstructed mean (left column) and variance (right column) images using VARD. Each column in Fig. 2 has three figures, showing the axial, coronal, and sagittal cuts through the center of the volume. The mean image was initialized using the FDK algorithm after negative values have been set to zero. The variance image was initialized to 0 2 cm 2 (the square of the average expected attenuation value) and the penalty weights (γ ) were initialized to 0 6 cm so that at the first mean update most of the weight is on the data-fit term. We used 29 ordered subsets; for each subset we updated the mean and variance images, followed by an update of the penalty weights before proceeding to the next subset. It can be seen that the highest values of the variance are obtained around boundaries of organs or bones, where the attenuation values change considerably, i.e., voxels with the highest neighborhood differences. This is mainly due to the learned prior variances γ (hyperparameters) which assign low/high values to low/high voxel differences, respectively.

8 Importantly, γ determines the neighborhood penalty weights locally (with a different weight for each voxel difference) and mitigates the need for tuning global parameters that control the balance between the data-fit term and the penalty. Another potential use of the variances could be image segmentation as can be seen from Fig. 2. Figure 3 shows the reconstruction using the FDK algorithm. By comparing Fig. 3 and Fig. 2 one can notice a slight loss of contrast in the mean images produced by VARD. A similar effect can occur using standard MAP reconstruction algorithms when the global tuning parameters are not chosen correctly. In fact, MAP reconstruction is quite sensitive to the choice of global tuning parameters and can result in considerable artifacts for some values of the tuning parameters. 7 In contrast, VARD does not require any manual tuning of global parameters so one can expect a certain trade-off between automation and quality. There is still room for improvement and we are currently studying different penalties/neighborhoods for VARD that might improve the quality of the mean images. We are also investigating cases where the data is undersampled, in which case FDK is known to produce aliasing artifacts. In addition, the variance images are a unique feature of the VARD algorithm and are interesting by themselves. Acknowledgments This work was supported by the Department of Homeland Security, Science and Technology Directorate, Explosives Division, through contract HSHQDC--C We thank Daniel B. Keesing for providing us the HECTARE software package. REFERENCES [] Neal, R. M., [Bayesian Learning for Neural Network], Springer-Verlag, New York (996). [2] Tipping, M. E., Sparse Bayesian learning and the relevance vector machine, J. Mach. Learn. Res., (200). [3] Basu, S. and De Man, B., Branchless distance-driven proection and backproection, In Proc. Electronic Imaging, SPIE 6065 (2006). [4] De Man, B. and Basu S., Distance-driven proection and backproection in three dimensions, in Phys. Med. Biol. 49, (2004). [5] Erdogan, H. and Fessler, J. A., Ordered subsets algorithms for transmission tomography, Phys. Med. Biol. 44(), (999). [6] Neal, R. M. and Hinton, G. E., A view of the EM algorithm that ustifies incremental, sparse, and other variants, [Learning in graphical models], Springer Netherlands, (998). [7] Kaganovsky, Y., Han, S., Degirmenci, S., Politte, D. G., Brady, D. J., O Sullivan, J. A., and Carin, L., Alternating minimization algorithm with automatic relevance determination for transmission tomography under Poisson noise, arxiv preprint, (204). [8] O Sullivan, J. A. and Benac, J., Alternating minimization algorithms for transmission tomography, IEEE Trans. Med. Imaging 26(3), (2000). [9] [0] Keesing, D. B., O Sullivan, J. A., Politte D. G. and Whiting, B. R., Parallelization of a fully 3D CT iterative reconstruction, In Biomedical Imaging: Nano to Macro, 3rd IEEE International Symposium on, (2006). [] Feldkamp, L. A., Davis, L. C. and Kress, J. W., Practical cone-beam algorithm, J. Opt. Soc. Am. A (6), (984).

9 x x x Figure 2. Cross sections through the reconstruction of pediatric patient data after 0 iterations with 29 ordered subsets (290 updates of all images). The mean image was initialized with FDK while setting negative values to zero. The axial, coronal, and sagittal views are shown from top to bottom. The left column shows the mean images, i.e., reconstruction of the linear attenuation coefficients; values from cm to 5cm (approximately bone) are shown in the range from black to white, respectively. The right column shows the variance images (Bayesian confidence intervals) with values in units of cm 2. The image volume is voxels, with voxel dimensions of mm by mm by 2 mm. We used a neighborhood penalty with 6 neighbors for each voxel (2 neighbors along each direction).

10 Figure 3. Reconstruction using the FDK algorithm. The cross sections are the same as in Fig. 2.