Algebraic Iterative Methods for Computed Tomography

Size: px

Start display at page:

Download "Algebraic Iterative Methods for Computed Tomography"

Juniper Park
5 years ago
Views:

Mathematics and Computer Science Technical University of

1 Algebraic Iterative Methods for Computed Tomography Per Christian Hansen DTU Compute Department of Applied Mathematics and Computer Science Technical University of Denmark Per Christian Hansen Algebraic Iterative Methods 1 / 32

FBP: Filtered Back Projection This is the classical method for 2D reconstructions. There are similar methods for 3D, such as FDK. Many year of use lots of practical experience.

2 FBP: Filtered Back Projection This is the classical method for 2D reconstructions. There are similar methods for 3D, such as FDK. Many year of use lots of practical experience. The FBP method is very fast (it uses the Fast Fourier Transform)! The FBP method has low memory requirements. With many data, FBP gives very good results. Example with 3% noise: Per Christian Hansen Algebraic Iterative Methods 2 / 32

FBP Versus Algebraic Methods Limited data, or nonuniform distribution of projection angles or rays artifacts appear in FBP reconstructions. Difficult to incorporate constraints (e.g., nonnegativity) in FBP.

3 FBP Versus Algebraic Methods Limited data, or nonuniform distribution of projection angles or rays artifacts appear in FBP reconstructions. Difficult to incorporate constraints (e.g., nonnegativity) in FBP. Algebraic methods are more flexible and adaptive. Same example with 3% noise and projection angles 15, 30,..., 180 : Algebraic Reconstruction Technique, box constraints (pixel values [0,1]). Per Christian Hansen Algebraic Iterative Methods 3 / 32

4 Another Motivating Example: Missing Data Irregularly spaced angles & missing angles also cause difficulties for FBP: Per Christian Hansen Algebraic Iterative Methods 4 / 32

5 Setting up the Algebraic Model The damping of the ith X-ray through the object is a line integral of the attenuation coefficient χ(s) along the ray (from Lambert-Beer s law): b i = χ(s) dl, i = 1, 2,..., m. ray i Assume that χ(s) is a constant x j in pixel j. This leads to: b i = a ij x j, a ij = length of ray i in pixel j, j ray i where the sum is over those pixels j that are intersected by ray i. If we define a ij = 0 for those pixels not intersected by ray i, then we have a simple sum n b i = a ij x j, n = number of pixels. j=1 Per Christian Hansen Algebraic Iterative Methods 5 / 32

6 A Big and Sparse System If we collect all m equations then we arrive at a system of linear equations A x = b with a very sparse system matrix A. Example: 5 5 pixels and 9 rays: A really big advantage is that we only set up equations for the data that we actually have. In case of missing data, e.g., for certain projection angles or certain rays in a projection, we just omit those from the linear system. Per Christian Hansen Algebraic Iterative Methods 6 / 32

7 The System Matrix is Very Sparse Another example: pixels and 180 projections with 362 rays each. The system matrix A is 65, , 536 and has elements. There are 15, 018, 524 nonzero elements corresponding to a fill of 0.35%. Per Christian Hansen Algebraic Iterative Methods 7 / 32

The Simplest Algebraic Problem One unknown, no noise: Now with noise in the data compute a weighted average: We know from statistics that solution s variance is

8 The Simplest Algebraic Problem One unknown, no noise: Now with noise in the data compute a weighted average: We know from statistics that solution s variance is inversely proportional to the number of data. So more data is better. Let us immediately continue with a 2 2 image... Per Christian Hansen Algebraic Iterative Methods 8 / 32

solutions, with k R: (There is an arbitrary component in the null

9 A Sudoku Problem Four unknowns, four rays system of linear equations A x = b: Unfortunately there are infinitely many solutions, with k R: (There is an arbitrary component in the null space of the matrix A.) Per Christian Hansen Algebraic Iterative Methods 9 / 32

10 More Data Gives a Unique Solution With enough rays the problem has a unique solution. Here, one more ray is enough to ensure a full-rank matrix: The difficulties associated with the discretized tomography problem are closely linked with properties of the coefficient matrix A: The sensitivity of the solution to the data errors is characterized by the condition number κ = A 2 A 1 2 (not discussed today). The uniqueness of the solution is characterized by the rank of A, the number of linearly independent row or columns. Per Christian Hansen Algebraic Iterative Methods 10 / 32

11 Algebraic Reconstruction Methods In principle, all we need to do in the algebraic formulation is to solve the large sparse linear system A x = b: Math: x = A 1 b, MATLAB: x = A\b. How hard can that be? Actually, this can be a formidable task if we try do use a traditional approach such as Gaussian elimination. Researchers in tomography have therefore focused on the use of iterative solvers and they have rediscovered many methods developed by mathematicians... In tomography they are called algebraic reconstruction methods. They are much more flexible than FBP, but at a higher computational cost! Per Christian Hansen Algebraic Iterative Methods 11 / 32

12 Some Algebraic Reconstruction Methods Fully Sequential Methods Kaczmarz s method + variants. These are row-action methods: they update the solution using one row of A at a time. Fast convergence. Fully Simultaneous Methods Landweber, Cimmino, CAV, DROP, SART, SIRT,... These methods use all the rows of A simultaneously in one iteration (i.e., they are based on matrix multiplications). Slower convergence. Krylov subspace methods (rarely used in tomography) CGLS, LSQR, GMRES,... These methods are also based on matrix multiplications. Per Christian Hansen Algebraic Iterative Methods 12 / 32

13 Matrix Notation and Interpretation Notation: r 1 A = c 1 c 2 c n =., r m The matrix A maps the discretized absorption coefficients (the vector x) to the data in the detector pixels (the elements of the vector b) via: b 1 r 1 x b 2 b =. = A x = x r 2 x 1 c 1 + x 2 c x n c }{{ n = }.. linear combination of columns b m r m x The ith row of A maps x to detector element i via the ith ray: n b i = r i x = a ij x j, i = 1, 2,..., m. j=1 Per Christian Hansen Algebraic Iterative Methods 13 / 32

Example of Column Interpretation A 32 32 image has four nonzero pixels with intensities 1, 0.8, 0.6, 0.4.

14 Example of Column Interpretation A image has four nonzero pixels with intensities 1, 0.8, 0.6, 0.4. In the vector x these four pixels correspond to entries 468, 618, 206, 793. Hence the sinogram, represented as a vector b, takes the form b = 0.6 c c c c 793. Per Christian Hansen Algebraic Iterative Methods 14 / 32

15 Geometric Interpretation of A x = b r 1 x = a 11 x 1 + a 12 x a 1n x n = b 1 r 2 x = a 21 x 1 + a 22 x a 2n x n = b 2. r m x = a m1 x 1 + a m2 x a mn x n = b m. Each equation r i x = b i defines an affine hyperplane in R n : x 2 R 2 a i1 x 1 + a i2 x 2 = b i x 1 x 3 R 3 x 2 x 1 a i1 x 1 + a i2 x 2 + a i3 x 3 = b i Per Christian Hansen Algebraic Iterative Methods 15 / 32

16 Geometric Interpretation of the Solution Assuming that the solution to A x = b is unique, it is the point x R n where all the m affine hyperplanes intersect. Example with m = n = 2: x 2 R 2 a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 x 1 Per Christian Hansen Algebraic Iterative Methods 16 / 32

17 Kaczmarz s Method = Algebraic Reconstruction Technique A simple iterative method based on the geometric interpretation. In each iteration, and in a cyclic fashion, compute the new iteration vector such that precisely one of the equations is satisfied. This is achieved by projecting the current iteration vector x on one of the hyperplanes r i x = b i for i = 1, 2,..., m, 1, 2,..., m, 1, 2,... Originally proposed in 1937, and independently suggested under the name ART by Gordon, Bender & Herman in 1970 for tomographic reconstruction. Per Christian Hansen Algebraic Iterative Methods 17 / 32

18 Orthogonal Projection on Affine Hyperplane r i H i = {x R n r i x = b i } z Pi (z) O The orthogonal projection P i (z) of an arbitrary point z on the affine hyperplane H i defined by r i x = b i is given by: P i (z) = z + b i r i z r i 2 2 r i, r i 2 2 = r i r i. In words, we scale the row vector r i by (b i r i z)/ r i 2 2 and add it to z. Per Christian Hansen Algebraic Iterative Methods 18 / 32

19 Kaczmarz s Method We thus obtain the following algebraic formulation: Basic Kaczmarz algorithm x (0) = initial vector for k = 0, 1, 2,... i = k (mod m) end x (k+1) = P i ( x (k) ) = x (k) + b i r i x (k) r i 2 2 Each time we have performed m iterations of this algorithm, we have performed one sweep over the rows of A. Other choices of sweeps: Symmetric Kaczmarz: i = 1, 2,..., m 1, m, m 1,..., 3, 2. Randomized Kaczmarz: select row i randomly, possibly with probability proportional to the row norm r i 2. r i Per Christian Hansen Algebraic Iterative Methods 19 / 32

20 Convergence Issues The convergence of Kaczmarz s method is quite obvious from the graph on slide 17 but can we say more? Difficulty: the ordering of the rows of A influences the convergence rate: x = The ordering is preferable: almost twice as fast. Per Christian Hansen Algebraic Iterative Methods 20 / 32

21 Convergence of Kaczmarz s Method One way to avoid the difficulty associated with influence of the ordering of the rows is to assume that we select the rows randomly. For simplicity, assume that A is invertible and that all rows of A are scaled to unit 2-norm. Then the expected value E( ) of the error norm satisfies: ( ) ( E x (k) x ) k n κ 2 x (0) x 2 2, k = 1, 2,..., where x = A 1 b and κ = A 2 A 1 2. This is linear convergence. When κ is large we have ( 1 1 ) k n κ 2 1 k n κ 2. After k = n steps, corresp. to one sweep, the reduction factor is 1 1/κ 2. Note that there are often orderings for which the convergence is faster! Per Christian Hansen Algebraic Iterative Methods 21 / 32

22 Cyclic Convergence So far we have assumed that there is a unique solution x = A 1 b that satisfies A x = b, i.e., all the affine hyperplanes associated with the rows of A intersect in a single point. What happens when this is not true? cyclic convergence: m = 3, n = 2 Kaczmarz s method can be brought to converge to a unique point, and we will discuss the modified algorithm later today. Per Christian Hansen Algebraic Iterative Methods 22 / 32

23 From Sequential to Simultaneous Updates Karzmarz s method accesses the rows sequentially. Cimmino s method accesses the rows simultaneously and computes the next iteration vector as the average of the all the projections of the previous iteration vector: x (k+1) = 1 m ( P i x (k) ) = 1 m (x (k) + b i r i x (k) ) m m r i=1 i=1 i 2 r i 2 = x (k) + 1 m b i r i x (k) m r i 2 r i. 2 i=1 P H 1 (x (k) ) 2 x (k) x (k+1) H 1 P 2 (x (k) ) Per Christian Hansen Algebraic Iterative Methods 23 / 32

24 Matrix formulation of Cimmino s Method We can write the updating in our matrix-vector formalism as follows x (k+1) = x (k) + 1 m = x (k) + 1 m m b i r i x (k) i=1 ( r 1 r r m r i 2 2 T = x (k) + 1 r 1 r m. r i r m r m 2 2 = x (k) + A T M 1( b A x (k)), ) b 1 r 1 x (k). b m r m x (k) r 1... b. r m 2 2 r m where we introduced the diagonal matrix M = diag ( m r i 2 2). x (k) Per Christian Hansen Algebraic Iterative Methods 24 / 32

25 Cimmino s Method We thus obtain the following formulation: Basic Cimmino algorithm x (0) = initial vector for k = 0, 1, 2,... x (k+1) = x (k) + A T M 1( b A x (k)) end Note that one iteration here involves all the rows of A, while one iteration in Kaczmarz s method involves a single row. Therefore, the computational work in one Cimmino iteration is equivalent to m iterations (a sweep over all the rows) in Kaczmarz s basic algorithm. The issue of finding a good row ordering is, of course, absent from Cimmino s method. Per Christian Hansen Algebraic Iterative Methods 25 / 32

26 The Optimization Viewpoint Karczmarz, Cimmino and similar algebraic iterative methods are usually considered as solvers for systems of linear equations. But it is more convenient to consider them as optimization methods. Within this framework we can easily handle common extensions: We can introduce a relaxation parameter or step length parameter in the algorithm which controls the size of the updating and, as a consequence, the convergence of the method: a constant ω, or a parameter ω k that changes with the iterations. We can also, in each updating step, incorporate a projection P C on a suitably chosen convex set C that reflects prior knowledge, such as the positive orthant R n + nonnegative solutions, the n-dimensional box [0, 1] n solution elements in [0,1]. We can introduce other norms than the 2-norm 2, which can improve the robustness of the method. Per Christian Hansen Algebraic Iterative Methods 26 / 32

27 Example: Robust Solutions with the 1-norm The 1-norm is well suited for handling outliers in the data: m min A x b 1, A x b 1 = r i x b i. x Consider two over-determined noisy problems with the same matrix: A = , A x = 34 57, b = , b o = Least squares solutions: x LS and x o LS ; 1-norm solutions: x 1 and x o 1 : x LS = , x o LS = , x 1 = , x o 1 = i=1 Per Christian Hansen Algebraic Iterative Methods 27 / 32

28 The Least Squares Problem Revisited The objective function and its gradient F(x) = 1 /2 b A x 2 2, F(x) = A T (b A x). The method of steepest descent for min x F(x), with starting vector x (0), performs the updates x (k+1) = x (k) ω k F(x) = x (k) + ω k A T (b A x), where ω k is a step-length parameter that can depend on the iteration. Also known as Landweber s method corresponds to M = I in Cimmino. Cimmino s method corresponds to the weighted problem: F M (x) = 1 /2 M 1/2 (b A x) 2 2, F M (x) = A T M 1 (b A x). Per Christian Hansen Algebraic Iterative Methods 28 / 32

29 Incorporating Simple Constraints We can include constraints on the elements of the reconstructed image. Assume that we can write the constraint as x C, where C is a convex set; this includes two very common special cases: Non-negativity constraints. The set C = R n + corresponds to x i 0, i = 1, 2,..., n. Box constraints. The set C = [0, 1] n (n-dimensional box) corresponds to 0 x i 1, i = 1, 2,..., n. Per Christian Hansen Algebraic Iterative Methods 29 / 32

30 The Projected Algorithms Both algorithms below solve min x C M 1/2 (b A x) 2. Projected gradient algorithm (ω k < 2/ A T MA 2 ) x (0) = initial vector for k = 0, 1, 2,... x (k+1) = P C ( x (k) + ω k A T M 1 (b A x (k) ) ) end Projected incremental gradient (Kaczmarz) algorithm (ω k < 2) x (0) = initial vector for k = 0, 1, 2,... i = k (mod m) ( ) x (k+1) = P C x (k) b i r i x + ω k r i 2 r i 2 end Per Christian Hansen Algebraic Iterative Methods 30 / 32

31 Iteration-Dependent Relaxation Parameter ω k The basic Kaczmarz algorithm gives a cyclic and non-convergent behavior. Consider the example from slide 22 with: ω k = 0.8 (independent of k) and ω k = 1/ k, k = 0, 1, 2,... The rightmost plot is a zoom of the middle plot. With a fixed ω k < 1 we still have a cyclic non-convergent behavior. With the diminishing relaxation parameter ω k = 1/ k 0 as k the iterates converge to the weighted least squares solution x LS,M. Per Christian Hansen Algebraic Iterative Methods 31 / 32

32 A Few References T. Elfving, T. Nikazad, and C. Popa, A class of iterative methods: semi-convergence, stopping ryles, inconsistency, and constraining; in Y. Censor, Ming Jiang, and Ge Wang (Eds.), Biomedical Mathematics: Promising Directions in Imaging, Therapy Planning, and Inverse Problems, Medical Physics Publishing, Madison, Wisconsin, P. C. Hansen and M. Saxild-Hansen, AIR Tools A MATLAB package of algebraic iterative reconstruction methods, J. Comp. Appl. Math., 236 (2012), pp G. T. Herman, Fundamentals of Computerized Tomography Image Reconstruction from Projections, Springer, New York, M. Jiang and G. Wang, Convergence studies on iterative algorithms for image reconstruction, IEEE Trans. Medical Imaging, 22 (2003), pp A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, Per Christian Hansen Algebraic Iterative Methods 32 / 32

Algebraic Iterative Methods for Computed Tomography

Algebraic Iterative Methods for Computed Tomography Per Christian Hansen DTU Compute Department of Applied Mathematics and Computer Science Technical University of Denmark Per Christian Hansen Algebraic