A PARALLEL APPROACH FOR ALIGNMENT OF MULTI- MODAL GRID-BASED DATA

Transcription

1 ISBN: IADIS A PARALLEL APPROACH FOR ALIGNMENT OF MULTI- MODAL GRID-BASED DATA Egor Dranischnikow, Elmar Schömer Institute of Computer Science, Johannes Gutenberg-University, Mainz Ulrich Schwanecke Dept. of Design, Computer Science an Meia, Hochschule RheinMain, University of Applie Sciences, Wiesbaen Ruesselsheim Geisenheim Ralf Schulze, Dan Brüllmann Dept. of Oral Surgery (an Oral Raiology), University Meical Center of the Johannes Gutenberg-University, Mainz ABSTRACT Multi-moal registration is still a big challenge in image processing. In this article we present a variation of the wellknown fast Fourier transform (fft) accelerate methos for fining the alignment between two atasets, i.e. the rigi transformation consisting of a rotation as well as a translation mapping all regions in both atasets belonging together in an optimal manner. Our metho can be applie to such multi-moal registration problems as computer tomography (CT)/positron emission tomography (PET) matching, or matching CT ata with ata obtaine by magnetic resonance imaging (MRI). We reformulate the alignment problem into an optimization problem concerning a metric measure. The particular form of the propose objective function can be exploite to fft-accelerate the translational part of the alignment-problem. Thus the reuce problem can be solve for the three remaining egrees of freeom of the rotatory part using stanar optimizers, such as ownhill-simplex or Powell s metho. A further avantage of our approach is the straight forwar parallelization of the objective function s computation. Our implementation on a graphic processing unit (GPU) yiele a speeup factor between 5 an 5 epening on the size of the ata. The results show, that the application of a GPU can be highly rewaring for all fft accelerate algorithms. KEYWORDS Alignment, cone-beam ata, multi-moal registration, parallel implementation, GPGPU. 1. INTRODUCTION The most common approach in multi-moal registration is base on the concept of mutual information (MI). Albeit MI base metric measure has been shown to be robust an generally applicable (see Maes, F. et al (1997), Mattes, D. et al (003), an Viola an Wells (1997)), a series of rawbacks are known. In particular the following two facts are problematic: The istributions of the ranom variables neee for computation of mutual information are unknown an their estimation can consume a lot of processor resources; furthermore, the resulting objective function possesses many local maxima (Haber an Moersitzki (005)). If both atasets have been recore with the same technology, it is possible to use the meansquare error as a metric. Its main avantage is the possibility to fft-accelerate the optimization process in some egrees of freeom. A fast metho for the solution of the pure translational alignment problem was presente in Cowtan (001), an a combination of translational an rotatory egrees of freeom was introuce in Kovacs et al (003). Fourier methos, in particular the phase correlation metho, are pretty well known an play a prominent role in D image processing an pattern analysis (see Castro an Moranim (1987), Pan et al (009), Rey an Chatterji (1996)). Trying to match CT an PET atasets, we face the following ilemma: the MI base general approach is too slow, because it cannot be fft-accelerate, an the methos which can be accelerate are not applicable to 330

2 IADIS International Conference Applie Computing 009 multi-moal ata. In this article we present an approach that is not as universal as the conventional MI metho but then can be fft-accelerate. Our alignment metho is base onmetrics of the form ϕ ( f, g ) = w ( x ) f ( x ) g ( x ), with Ω - a omain in 3 N, w - a weight function an f, g - arbitrary scalar functions. The weight function is the most important ifference to the cases consiere in Rey an Chatterji (1996) an Castro an Moranim (1987). Both papers assume that one function of the two to be aligne is a rotate an translate replica of the other one, but for the multi-moal ata this is almost never true. The presence of issimilar parts has a negative impact on the reliability of the phase correlation metho. Therefore we are going to use an iea similar to that propose in Cowtan (001), with a weight function as a fragment mask. The remainer of this paper is structure as follows. In the next section we introuce relevant efinitions an mathematics as well as the core algorithm of our metho. Section 3 covers the parallelization of the algorithm on a GPU, whereby we are also going to iscuss the performance of the parallelize version. Experiments with synthetic ata are presente in section 4, whereas section 5 contains some wiesprea applications, for which the propose metho yiels goo results.. MATHEMATICS We consier a three-imensional ataset as a perioic function f : Ω R. The set of these functions will be enote by Abb( Ω, R). To evaluate the istance between two elements f, g from Abb( Ω, R), we use the pseuo metric ( f, g) = w( f ( g(, w with w( x ) 0. w is a metric if an only if w( > 0 x Ω. Using a pseuo metric for istance estimation enables us to neglect parts of the ata, which are for sure issimilar in the multi-moal atasets. A rigi boy transformation T ( ζ ; ) between elements of Ω, which epens on three rotatory egrees of freeomα an three translational egrees of freeom ζ, generates the operator Tˆ efine by ˆ ~ T ζ ( f ) : = f T 1 ( ζ ; ). [ ] ( ) Thereby ~ 3 f is the continuation of the ata recor f from Ω to continuous space R which can be compute by means of trilinear, nearest neighbor or any other interpolation scheme. In our experiments both methos yiele similar results. With the help of w we can efine the objective function ϕ( ζ ) : = w( f ', g) with ˆ f ': = T [ ζ ]( f ) beeing the transforme ataset. ϕ measures the quality of a coorinate transform or rather the quality of a given ata matching. Now we can consier the alignment problem as the following optimization problem. Problem 1 (General alignment-problem) Given f, g from Abb( Ω, R), fin * * ( α, ζ ) : = arg minϕ( ζ ). 3, R α ζ Initially we consier only the translational part of the alignment problem, i.e. the problem of fining the best translational movement for a fixe rotation. Therefore we obtain the following reuction of the general alignment problem: Problem (Translational alignment-problem) Given f, g from Abb( Ω, R), evaluate 331

3 ISBN: IADIS ϕ ( α) : = minϕ( ζ ) min ϕ( ζ ). 3 R ζ R From now on we only consier the integer translational movements, establishing the " " in the equation ζ Ω above. The objective function for a fixe α given by ( ζ ) : = ϕ( ζ ) = w( f '( x ζ ) g( with f ': T [ 0 ]( f ) ϕ, ˆ = an ζ Ω, can be rewritten as: ϕ ( ζ ) = w( f '( x ζ ) w( f '( x ζ ) g( + w( g( By using the mirror operator S ˆ( f )( ) : = f ( ) an convolution for perioic functions ( f g)( y) : = f ( y g( we get ˆ ( Sw ˆ ) f ' ( Sˆ( g w) ) U0 Sϕ = f ' + with U 0 a constant originating from the thir term. The above equation hols, because of the following calculation for two perioic functions f1, f Abb( Ω, R) : f x y) f ( f ( x' y) f ( x') f (( y) x') Sf ˆ ( x') f Sf ˆ = = = ( by: ( ) ) 1( y x' = x x' Ω x' Ω It is well-known, that the convolution of two functions f 1 an f can be efficiently compute using fft f = NFˆ ( Fˆ ( f ) Fˆ ( )) 1 1 f 1 f whereby Fˆ enotes the Fourier transform an the factor N epens on the size of the omain Ω. This leas us to the following form of the objective function (see Cowtan (001) or Dranischnikow (008) for more etails): ˆ ˆ 1 ˆ ˆ ˆ ˆ ϕ = NS F F( f ' ) F( Sw) F 1 Fˆ ( f ') Fˆ ( Sˆ( w g)) + U. ( ( ) ( ) 0 By using fft we can calculate the best translation in ( n n) result in an ( n ), O log whereas the naïve implementation woul O time complexity. It is easy to transform these formulae into an efficient parallel algorithm, which we will iscuss in section 3. We split the general alignment problem 1 into the translational alignment problem just mentione an the problem of fining the best rotational movement assuming that we ha solve problem. This leas to the following subproblem. Problem 3 (Rotatory alignment-problem) Given f, g from Abb( Ω, R), evaluate min 3 ϕr( α). α R To solve problem 3 we resort to conventional optimizers such as ownhill-simplex from Neler an Mea (1965) or Powell s metho (Press et al (00)). Our experiments clearly inicate, that the ownhillsimplex metho is more suitable (see table ). In comparison to the initial problem, the search-space has been reuce to three imensions. The main avantage of this proceure is that the objective function ϕr ( α) possesses, accoring to experience, consierably less local extrema.. 33

4 IADIS International Conference Applie Computing PARALLELIZATION The evaluation of the objective function ϕr ( α) consumes the lion s share of the computation time an consists of the following steps: 1. Computation of the rotate ataset by means of trilinear interpolation. Solving the translational alignment problem: (a) Squaring the values of the rotate ataset (b) Fourier transform (c) Multiplication (point wise) an aition of vectors in the frequency omain () Inverse Fourier transform (e) Search for the minimum Steps 1, a an c o not eman any communication between the processes an are therefore easy to parallelize. It is known, that the Fourier transform (b an ) possesses a quite high potential for parallelization (Gupta an Umar (1993), Tong an Swarztrauber (1987), an Swarztrauber (1987)). Searching for a minimum (e) is a typical example for reuction (compare Zomaya (1996) or Harris (007)) an can be efficiently implemente on most parallel architectures. The architecture of our choice for the parallelization of the algorithm was a NVIDIA GPU. The CUDAframework from NVIDIA makes it possible to use a GPU without casting the problem into a graphical API. Despite the simple hanling, a careful implementation is necessary. Careless memory access can be evastating for the performance because of the absence of a cache. However, with texture an share memory the programmer has enough tools on his isposal to evae most of the bottle necks cause by the relative slow global memory. Furthermore, with CUFFT (NVIDIA Corporation (007)) an efficient implementation of the necessary fft routines is available. We are using textures for computation of the rotate ataset because otherwise it is quite challenging to optimize the global memory access. The efficient global access for steps a an c is straight forwar, because the simple ata layout allows the simultaneous memory accesses to be coalesce into a single memory transaction. The optimal size of the blocks can be foun in fftw manner (Frigo an Johnson (005)) uring a testing phase. This ensures that the best possible performance can be reache on ifferent GPUs. When using real-to-complex in-place Fourier transforms a paing is necessary (see NVIDIA Corporation (007) for more etails). Due to the SIMD architecture of the GPU it is more efficient to treat those pae elements the same way as the other "normal" elements. Table 1. Comparison of the run times between GPU with trilinear interpolation, CPU with trilinear interpolation an CPU with nearest neighbor interpolation. Because of the textures the GPU has a kin of intrinsic avantage over the CPU, therefore we also offer a comparison with a CPU implementation, that applies the less costly nearest neighbor interpolation. We use an Intel Core Duo.4 GHz with NVIDIA 8800GTX (CUDA.1) for this benchmarking test Kin of Computation Number of voxels in the ataset: N. N=18 N=19 N=0 N=1 N= N=3 N=4 N=5 CPU (time in ms) CPU Nearest Neighbor (time in ms) GPU (time in ms) Speeup CPU/GPU Table 1 shows the gains, which can be achieve through parallelization on a GPU. The biggest speeup can be obtaine for embarrassingly parallel tasks. The computation of the rotate volume profits enormously from the use of textures, which are faster than the normal global memory. The fft computation oes not yiel the same speeup because the main part of the communication must pass through the slow global memory. However the size of the CPU cache is insufficient for fft on bigger atasets, therefore the speeup factor of fft increases with the size of the ataset. 333

5 ISBN: IADIS 4. EXPERIMENTS WITH SYNTHETIC DATA Before going to the real life examples, we investigate the ability to recover the rotation with help of synthetic ata. There are two major questions to answer. The first one is which methoownhill-simplex or Powell s metho yiels better results an is more suitable an the secon one is, what is the probability of fining the right rotation. We use two kins of ata: Rotations uniformly istribute over the whole range an only rotations up to 45 but otherwise uniformly istribute. Both 18x18x18-atasets were cut out from a bigger CT volume an share an ientical region, which is precisely known. This scenario is similar to section 5.3, compare also figure. A matching was consiere successful, when the inaccuracy of the calculate rotation i not excee 1. Table : Experimental results with synthetic ata. Success rate an the number of function evaluations per case are average over 500 runs metho all rotations rotation up to π / 4 success rate # evaluations success rate # evaluations ownhill-simplex Powell The experimental results in table suggest that ownhill-simplex turns out to be a better choice. The higher success rate an the lower computation cost are two powerful arguments. The high success rate implies, that a few starting configurations are sufficient to fin the global minimum. It also turns out that an aitional Gaussian noise oes not have any notable impact on the success rate of the algorithm. 5. APPLICATIONS In this section we are going to escribe some real-worl alignment problems of multi-moal ata for which our metho can be successfully use. 5.1 PET/CT-Matching The research project FUNMIG (Funamental processes of raionuclie migration) stuies the behavior of inserte raioactive elements in iverse rocks. The results of computer simulations, which preict the migration of the raioactive substance, shoul be compare with experimental results in orer to evaluate correctness an accuracy of the unerlying moel. The inner structure of a rock has been investigate by means of CT, whereas PET was use for monitoring the iffusion of the raioactive elements. Because the two measurements were carrie out from within two ifferent coorinate systems (an as a matter of fact in two ifferent cities), there is the necessity to fin the right transformation in orer to be able to compare the ata (Ulenkampff et al (008)). The CT-recor of the rock is available as a Boolean moel. The hollow voxels are enote by 1, the massive voxels by 0. This ataset must be matche with 5 PET recors. Each recor escribes the configuration of the raioactive elements in the rock for a fixe point in time. The values of the voxels can vary between 0 (nonraioactive) an 16 (very raioactive). All PET-measurements have been carrie out from within the same coorinate system. We assume that high raioactivity (the term "high" shoul be seen in relationship to the raioactivity of the other voxels) can be reache only in hollow voxels, because only there a large accumulation of raioactive elements is possible. It is inee thinkable, that some thin fissures have not been registere by CT, but still can contain a non-negligible amount of raioactive atoms. However, the raioactivity of such voxels shoul not be very high. On the other han, the raioactive material oes not sprea over the whole crevice; therefore hollow voxels exist that are not contaminate. 334

6 IADIS International Conference Applie Computing 009 These consierations lea us to the objective function ϕ ( α, ζ ) : = p( c( 1, (1) p( > v where c enotes the transforme CT ata, p enotes the PET ata an v the threshol from which on we consier a voxel as being raioactive. The objective function punishes voxels, that are consiere to be raioactive (i.e. p ( x ) > v ) but are not hollow. Accoring to our assumption, that higher raioactivity increases the probability that a voxel is hollow, the punishment is severer for massive voxels with high raioactivity. This simple approach is sufficient to accomplish a successful matching, as the left han sie of figure 1 illustrates. Further etails an results can be foun in Dranischnikow (008). 5. MRI/CT Matching The next application of our registration metho arises in meical science. The lung of a patient has been examine with the help of two ifferent methos. Conventional CT acquires the tissue of the lung whereas inhale ³He gas mae visible by means of magnetic resonance imaging (see Kauczor et al (1997)) epicts the parts of the lung with proper function. Merging the two atasets woul help to iagnose the state of a isease more precisely an choose a more aequate treatment. A certain resemblance to the problem above is evient. Comparing this scenario to the above, the lung plays the role of the crevice an the ³He gas the role of the raioactive material. With this interpretation the objective function (1) can be reuse an again elivers successful matchings, as figure 1, right-han sie, illustrates. More etails an results are provie in Dranischnikow (008). Figure 1. On the left-han sie: The CT an PET (after 60 minutes) atasets are successfully matche. First volume shows the PET ata, where re color correspons to high raioactivity. Secon volume shows the matching with the CT ata, where the crevice is epicte brown. Thir volume shows the surface of the raioactive voxels matche with the CT ata. On the right-han sie: Four slices through the lung of a patient: gray color isplays the lung in the CT ata, the more intense the re color the higher the concentration of ³He. 5.3 CT-Fusion The last application of our metho copes with a task from the fiel of entistry. Here sometimes an object (usually the hea of a patient) is too large for an available cone beam CT with small or meium size fiel of view. Thereforeit is necessary to perform two or more measurements. Afterwars the small atasets shoul be merge into one large ataset. Figure (left-han sie) epicts, how this task can be reuce to the alignment problem of the share region. We choose f : = first ataset, g : = secon ataset an 335

7 ISBN: IADIS 1, x A w( : = 0, x A whereby A enotes the (estimate) share region of two atasets. These efinitions allow us to use our algorithm in orer to merge the small volumes into a bigger one. The results are shown in figure on the right-han sie. Figure. Left-han sie: The solution for the alignment problem of the share region enables the require merging of two atasets. Right-han sie: The result of merging two small atasets into one big recor of a jaw. Two possibilities to expan the ata are shown. The left volume complete with parts of the right volume an the other way aroun. The accuracy of the merging as well as the junctions is clearly visible. 6. CONCLUSION In this paper we presente a new metho to register multimoal ata. Our approach is base on a special metric that makes it possible to neglect subsets of the ata that are for sure issimilar. The propose metric has been successfully use for matching multi-moal atasets. Experiments showe that this approach is applicable for PET/CT- an MRI/CT-matchings. The algorithm also operates on single-moal ata an can be applie to register an object/pattern in an n-imensional space. The main avantage of our approach is the possibility to fft-accelerate the search in some egrees of freeom. In this paper we consiere the acceleration of the search for the translational movement, which resulte in fewer local minima of the new objective function. This leas to a better performance in terms of success rate of matching as well as in terms of run time. A further improvement can be achieve by using parallel harware such as a GPU. A speeup factor up to 5 coul be achieve using moern commoity graphics harware. Thus, the utilization of GPUs for computation, because of the impressive performance that GPUs offer at a very moest financial cost can be an alternative to the use of expensive harware such as combine PET/CT- or MIR/CT-scanner. The propose strategy for parallelization can also be use for other fft-accelerate algorithms such as the phase correlation metho an therefore can substantially spee up those applications. 336

8 IADIS International Conference Applie Computing 009 ACKNOWLEDGEMENT We gratefully acknowlege Frieer Enzmann (Institut fuer Geowissenschaften er Johannes Gutenberg- Universitaet Mainz) an Oliver Weinheimer (Department of Raiology Johannes Gutenberg-University) for proviing ata an insight into the problems iscusse in section 5. REFERENCES Castro e, E. an Moranim C., Registration of translate an rotate images using finite fourier transforms. IEEE Transactions on Pattern Analysis an Machine Intelligence, PAMI-9(5), pp Cowtan, K., 001. Fast fourier feature recognition. Acta Cryst., D57, pp Dranischnikow, E, 008. Parallele Algorithmen zum Alignment von Gitterbasierten Daten. Diploma Thesis, Johannes Gutenberg-Universitaet Mainz. Frigo, M. an Johnson, S, 005. The esign an implementation of fftw3. Proceeings of the IEEE, 93(3), pp. 31. Gupta, A. an Umar, V. K., The scalability of fft on parallel computers. IEEE Transactions on Parallel an Distribute Systems, 4(8), pp Haber, E. an Moersitzki, J., 007 Intensity graient base registration an fusion of multi-moal images, Methos of Information in Meicine, 46(3), pp Harris, M. 007, Parallel Prefix Sum (Scan) with CUDA. NVIDIA Corporation, 007. Kauczor, H.-U. et al, Imaging of the lungs using 3he mri: Preliminary clinical experience in 18 patients with an without lung isease. J. Magn. Res. Imag., 7, pp Kovacs, J.A., et al, 003. Fast rotational matching of rigi boies by fast fourier transform acceleration of five egrees of freeom, Acta Cryst., D, pp Maes, F. et al, Multimoality image registration by maximization of mutual information. IEEE Trans. Meical Imaging, (4), pp Mattes, D. et al, 003. Pet-ct image registration in the chest using free-form eformation. IEEE Trans. on Meical Imaging, (1), pp Neler, J. A. an Mea, R. A., A simplex metho for function minimization, Computer Journal, pp NVIDIA Corporation, 007. CUFFT Library. Pan, W. et al, 009. An aaptable-multilayer fractional fourier transform approach for image registration. IEEE Transactions on Pattern Analysis an Machine Intelligence. 31(3), pp Press W.H. et al, 00. Numerical Recipes in C++. The Art of Scientific Computing. Cambrige University Press. Rey, B.S. an Chatterji, B.N., An fft-base technique for translation, rotation, an scale-invariant image registration. IEEE Transactions on Image Processing, 5(8), pp Swarztrauber, P. N., Multiprocessor ffts. Parallel Computing. 5, pp Tong, C. an Swarztrauber, P. N., Orere fast fourier transforms on a massively parallel hypercube multiprocessor. J. Parallel an Dist. Comput., pp Ulenkampff, J. K et al, 008. Evaluation of positron-emission-tomography for visualization of migration processes in geomaterials. Phys. Chem. Earth, 33, pp Viola, P. an Wells, W. M. III, Alignment by maximization of mutual information. International Journal of Computer Vision, 4(), pp Zomaya, A.,1996. Parallel an Distribute Computing Hanbook, McGraw-Hill,