Parallelization of Mutual Information Registration

Transcription

1 Parallelization of Mutual Information Registration Krishna Sanka Simon Warfield William Wells Ron Kikinis May 5, 2000 Abstract Mutual information registration is an effective procedure that provides an accurate spatial relationship between two volumetric medical images of differing modalities. This had aided surgeons in precisely locating pathological tissues and avoiding critical structures. This paper focuses on accelerating the registration procedure, while preserving accuracy, by implementing parallelism with POSIX threads. When compared to the currently used serially executing registration process, the time needed to align two sets of volumetric imagery was sharply reduced. Although thread overhead and maintenance presented some limitations, the registration speedup increased - within a range of available CPUs - linearly with respect to the available CPUs with a slope close to one, and with accuracy preserved. Rapid alignment through parallel registration allows for shorter interventions and improved surgical navigation. This leads to a superior execution of clinical procedures, such as tumor resections, which leads to a better surgical outcome and faster recovery for the patient. Keywords: Mutual information, POSIX threads, symmetric multi-processor, medical image registration 1 Introduction Registration is the process of determining an accurate spatial relationship between sets of medical image data. Applications of registration range from image guided surgery to the analysis of functional images, and include the characterization of normal and abnormal anatomical variability (brain mapping), the detection of change in disease state over time, the visualization of multimodality data sets, and modeling anatomy in the process of segmentation. This paper presents registration in the context of providing an accurate spatial relationship between intra-operative volumetric image data and pre-operative volumetric image data. There are several advantages to obtaining intra-operative images. For example, using a high field interventional MR imager allows the quick acquisition of cross-sections of a patient s anatomy and can portray the current state of the anatomy in terms of characteristics such as deformation. Intra-operative MR images acquired during surgery serve 1

2 the purpose of guiding the surgeon s actions. It provides additional valuable information by allowing the surgeon to see beneath the surface of structures. However, this quick procurement of intra-operative images comes with a price. Compared to a conventional clinical scanner, a lower magnet field strength is used in obtaining intra-operative images, which leads to a poorer signal-to-noise ratio. In addition, there is a smaller field of view, which results in only the region of interest being scanned. Finally, while acquiring images with a larger voxel size accelerates the process with an improved signal-to-noise ratio, it also leads to larger sampling artifacts than are usually seen in conventional MR scanners [1]. However, the value of intra-operative imaging is greatly increased by registration. By registering the intra-operative data with the pre-operative data, one could compare and determine whether changes in tissue structure have occurred. For instance, this would be extremely useful in evaluating the extent of a tumor. In general, utilizing such a registration would allow a surgeon to more precisely identify and avoid critical structures and more accurately locate pathological tissues during a procedure. There are several algorithms that one can use in order to perform a registration. Here, the registration process involves the utilization of the mutual information that is shared between the sets of data. Mutual information (MI) is an information-theoretic measure of the relatedness of two sets of data that is defined in terms of various entropies of the data [2, 3]. When registration is conducted by maximizing mutual information, the spatial relationship of two data sets is adjusted to maximize their mutual information. It has been shown to be an effective mechanism for multi-modality registration of volumetric medical image data in surgical cases [4]. As shown in intra-operative imaging and image guided surgery, high performance computing is becoming increasingly important in imaging applications because radiological image analysis is shifting from a visual analysis of collections of planar images to a computerized quantitative image analysis of volumetric images [5]. This method of analysis is increasingly limited by memory and speed constraints of conventional workstations. Therefore, in order for registration (utilizing any given algorithm) to become a widely used clinical tool, techniques need to be available that can execute in a clinically compatible time frame on widely available hardware. It has been shown that the current state of performing registrations between intraoperative and pre-operative image data has room for improvement. In particular, it would be desirable to employ parallelism, through the use of threads, in order to accelerate the MI-based registration process. In essence, the registration process maximizes mutual information through many iterations. Although there can be several ways in which to implement parallelism in the registration process, it seems that the most efficient implementation would require a small change in the execution of the MI algorithm so that these iterations can be performed in parallel. This paper focuses on expediting the registration process through this method while preserving the accuracy of the coordinate transformation that relates the coordinate systems of the pre-operative and intra-operative image data. 2

3 for (i = 0; i < 20000; i++) { // sample randomly selected coordinates // perform calculations to get di/dt // T <- T + (lambda * di/dt) } Figure 1: The implementation of the current system runs iterations in order to calculate an appropriate pose. di/dt is the derivative of the mutual information. lambda is the learning rate. T is the coordinate transform (pose). Our implementation of registration by maximization of mutual information makes use of the EMMA approach to estimating derivatives of MI with respect to the coordinate transformation parameters [3, 2, 6]. EMMA is an example of the stochastic gradient descent approach to optimization, which has been described by Widrow and Hoff [7]. This method has been commonly used in the neural network community where it is referred to as the LMS rule. In the present work we will describe a method for parallelizing the stochastic gradient descent approach. In the remainder of the paper we will endeavor to answer the following questions. First, will a parallel MI-based registration provide the same accuracy as the current system? Second, will there be an appropriate speedup as the number of CPUs in the system increase? Finally, will the overhead associated with implementing threads be tolerable enough in order to see an appropriate speedup compared to the current system? Through the execution and comparison of many trials of the parallel registration and the current system, this paper will show that the above questions can be answered, within a certain scope, in the affirmative. 2 Method The current system for conducting registration consists of many iterations that maximize the mutual information between two sets of volumetric medical image data. In every iteration, a pre-determined number of coordinates are randomly sampled from one of the sets of image data. Unless said otherwise, 50 coordinates were sampled. Several calculations that maximize mutual information are then performed. This leads to obtaining an estimate of the derivative of the mutual information. This derivative is then multiplied by a learning rate parameter and used to update the coordinate transform (a pose ) to a more accurate value. After all iterations have been completed the pose has been modified sufficiently to provide a precise spatial relationship between the two sets of volumetric data. This can be seen in Figure 1. As shown in Figure 1, the new value of the pose depends on its value in the previous iteration. In order to use POSIX threads to implement parallelism in an effective manner, it was decided that a number of iterations (equal to the number of threads) would be 3

4 for (i = 0; i < 20000; i += (number of threads)) { for (j = 0; j < (number of threads); j++) { // sample randomly selected coordinates // perform calculations to get di/dt // MI[j] = di/dt} for (k = 0; k < (number of threads); k++) { // T += (lambda * MI[k]) } Figure 2: The algorithm has been changed slightly in order to implement parallelism. This is the serial implementation of the algorithm. As expected, it has the same performance characteristics when compared to the original system. In the parallel version, the for loop that is indexed by j is calculated by all the threads in parallel which allows the overall computation to proceed at a faster rate. Test Machine Architecture CPUs Ecache Operating System 1 Enterprise 450 4x300MHz 2MB SunOS Ultra 80 4x450MHz 4MB SunOS Ultra 60 2x360MHz 4MB SunOS Wildfire Architecture 28x250MHz 4MB SunOS 5.6 Table 1: The characteristics of the machines on which the registration trials were performed. Features listed are the machine architecture, number and type of CPUs, amount of Ecache, and operating system. All machines were manufactured by Sun Microsystems, Inc. and use Ultrasparc II processors. performed in parallel. The derivative of mutual information obtained from each of those iterations would then be accumulated before the pose was updated. Figure 2 shows how this changed algorithm would be executed serially. Although this amounts to a slight change in the algorithm, it will be shown in the results section to be just as accurate as the current one. In introducing parallelism into mutual-information based registration, a standard workpile decomposition [8] is implemented with POSIX threads to parallelize the calculations of the derivative of the mutual information in each iteration. One process manages the workpile by controlling access to the units of work and collating partial results, while other processes request work units (in this case, an iteration of calculations) and carry out the work independently. The expected speedup of the system is linear, with a slope close to one, with the number of threads (and therefore CPUs) available for computation. The parallel version of the MI-based registration was tested and compared to its serially executing counterpart as well as with the original serial version. These tests were performed on several machines. The characteristics of these machines are depicted in Table 1. 4

5 It should be noted that the architecture of machine #4 in Table 1 is experimental 1 [9]. On each machine, the number of threads were varied while executing the registration code. For each variation, 100 trials were performed with each version of the code. Before each trial began, one of the volumetric image data sets was randomly offset by a pose so that registration could be performed to have the two data sets converge. This random offset was determined by two factors: a rotational perturbation about a randomly selected axis by a random angle uniformly selected from a pre-determined interval, and a uniformly random translational offset selected from a pre-determined interval that was added to each translational axis (X, Y, and Z). The translational interval was 20 mm and the rotational interval was 30 degrees. At the end of each trial, the standard deviation of each translational axis for the final pose was calculated. An average rotation angle from an average rotation was also computed. The final pose was also evaluated for accuracy. It was deemed successful if all the translational components were within 3 mm of the translational components of the original pose. In addition, the duration of each trial was measured in seconds in order to evaluate the effectiveness of parallelism in MI-based registration. 3 Results Figure 3 displays the results of a successful trial. The results obtained from all trials are displayed in graphical or tabular format. Unless specified otherwise, all data is taken from trials that randomly sampled 50 coordinates during each iteration of the registration process. 4 Analysis The results show that in each case the parallel registration converged to a successful registration result. The variance of the difference between the ideal and obtained translations was small, as shown in Table 2, and the maximum error over all trials was less than 3mm which is comparable to the voxel size. It must also be kept in mind that the equivalent of one round of registration was tested. In a clinical setting, several more rounds of refined registration would be performed that would even further reduce the maximum error. The modified parallel algorithm has similar accuracy to the original serial algorithm, and provides a significant speedup on small symmetric multi-processor (SMP) machines. It can be seen in Table 8 that the time taken to perform parallel registration with one CPU available was larger than the time elapsed for the serial registration procedure. One would expect these times to be the same, because both types of registration are operating under 1 WildFire is Sun s internal codename for an advanced Server architecture that is under development. WildFire prototype systems have not been tested, optimized, or qualified for sale; they have been built for evaluative purposes only. Elements of the WildFire prototype may or may not be used in future Sun products. 5

6 Figure 3: Display of a successful trial. The 3 images on the left are obtained from an interventional MRI scan that represents intra-operative data and the 3 images on the right are conventional MRI images that represent pre-operative data Speedup Factor Number of Available CPUs Figure 4: Speedup Factors for MACHINE 1 6

7 MACHINE CODE CPUs T θ FINAL TRIALS SUCCESS XYZ σ X σ Y σ Z θ ±mm mm % 4 P P P P P P P P P P SP S P P SP S P P P P SP S P P P P SP S Table 2: MRT MRI volumetric registration results table. The MACHINE column refers to the test machines in Table 1. In the CODE column, P refers to the parallel registration program, SP refers to the serial implementation of the parallel registration program, and S refers to the currently used serially executing system. The CPUs column refer to the number of CPUs available for computation. From a known position and orientation, a random offset uniformly selected from the interval T was added to each translational axis ( T) after the reference volume had been rotated about a randomly selected axis by a random angle uniformly selected from the interval θ. The distributions of the final poses can be evaluated by comparing the standard deviations of the location of the center, computed separately in X, Y and Z. Furthermore, the average rotation angle from an average rotation is computed ( θ ). Finally, the number of trials that succeeded in converging to near the correct solution (by visual inspection) is reported. 7

8 Speedup Factor Number of Available CPUs Figure 5: Speedup Factors for MACHINE 2 8

9 2 Speedup Factor Number of Available CPUs Figure 6: Speedup Factors for MACHINE 3 9

10 30 TEST MACHINE 4-50 Samples TEST MACHINE Samples TEST MACHINE Samples Speedup Factor Number of Available CPUs Figure 7: Speedup Factors for MACHINE 4 shown for three sets of trials at different coordinate sample sizes 10

11 MACHINE CODE CPUs TRIALS AVERAGE EXECUTION TIME 3 P P SP S P P P P SP S P P P P SP S Table 3: Average execution time in seconds over 100 trials for all the different registration programs. Column headings are as in the previous tables. conditions that allow serial computation only. This difference in time, which is probably due to increased costs of thread synchronization, is described further in Section 4.2. Machine 1, 2, and 3 all showed a significant amount of speedup as the number of CPUs used was increased. Figures 4, 5, and 6 show that the speedup in registration was linear, with a slope that was close to one. When compared to the currently used serial registration system, the slope of the speedup was not as close to one as desired. Nonetheless, there was still a significant speedup. The reduction in speedup is probably due to the overhead of the POSIX threads, which is described further in Section 4.2. The performance of the serial implementation of the parallel registration system was very similar to the currently used system. This was expected because neither system utilized threads to implement parallelism. 4.1 Performance Analysis on Wildfire Architecture Machine 4, which operates on the experimental Wildfire architecture, provided some interesting results. Three sets of 100 trials were performed on Machine 4 with the parallel registration system. The only difference in the trials is that one set of trials randomly sampled 50 coordinates every time the derivative of the mutual information was found. Another set of trials sampled 100 coordinates for every instance in which the derivative of mutual information was obtained, and the last set of trials sampled 200 coordinates. As more coordinates were sampled, the trials would take longer to compute the final pose because there would be more work being done in each iteration of the registration process. 11

12 MACHINE CODE CPUs TRIALS AVERAGE EXECUTION TIME 4 P P P P P P P P P P P P P P P P P P P P P P P P P P SP S Table 4: Average execution time in seconds over 100 trials for TEST MACHINE 4. In each trial, 50 coordinates are randomly sampled. Column headings are as in the previous tables. 12

13 MACHINE CODE CPUs TRIALS AVERAGE EXECUTION TIME 4 P P P P P P P P P P P P P P P P P P P P P P P P P P Table 5: Average execution time in seconds over 100 trials for TEST MACHINE 4. In each trial, 100 coordinates are randomly sampled. Column headings are as in the previous tables. 13

14 MACHINE CODE CPUs TRIALS AVERAGE EXECUTION TIME 4 P P P P P P P P P P P P P P P P P P P P P P P P P P Table 6: Average execution time in seconds over 100 trials for TEST MACHINE 4. In each trial, 200 coordinates are randomly sampled. Column headings are as in the previous tables. SAMPLES TIME WITH 26 CPUs TIME WITH 1 CPU SPEEDUP Table 7: Speedup associated with 26 CPUs available on MACHINE 4 for sets of trials with differing sample sizes. Average times are reported in seconds. 14

15 MACHINE PARALLEL TIME SERIAL TIME ADDITIONAL TIME CPU CLOCK FOR 1 CPU FOR 1 CPU PER ITERATION (MHz) Table 8: Additional time spent per iteration by parallel code for registration. The time to execute the parallel code with 1 CPU available is reported in seconds in Column 2. The time to execute the serial code with 1 CPU available is reported in seconds in Column 3. The additional time spent per iteration by the parallel code is reported in milliseconds in Column 4. In each implementation, the registration consisted of iterations. The speed of each machine s CPUs are reported in Column 5. In all cases, the slope of speedup of the parallel computation was close to 1 up to about 6 CPUs. However, as more CPUs were being used, the nature of the speedup changed for the three sets of trials. The speedup in the set of trials that sampled 50 coordinates no longer varied and seemed to reach a plateau with very minor fluctuations. The trials that sampled 100 coordinates still had a positive speedup after 6 CPUs, but the rise was much smaller with a slope below 1. Eventually, the speedup also plateaued with minor fluctuations. The exception to this was seen when a drop in speedup occurred with available CPUs. In the trials that sampled 200 coordinates, the speedup s slope fell below 1, but kept rising without plateauing for 20 CPUs. There was a slight drop in speedup when 21 CPUs became available, but continued to rise afterwards. This can be seen in Figure 7. In the set of trials that sampled 200 coordinates, there was a greater speedup overall compared to the set of trials with 100 samples. Not surprisingly, the set of trials that sampled 100 coordinates had a greater speedup overall compared to the set of trials with 50 samples. As the data shows, using 26 CPUs with 50 samples showed a speedup of approximately When 26 CPUs were utilized with 100 coordinates being sampled, the computation time was larger, but the speedup was When 200 coordinates were sampled, there was an even greater increase in computation time. However, a greater speedup of was associated with it. This seems to imply that the threads were not being given enough work for computation, and that speedup can be dependent on the amount of work that each individual thread must accomplish. On occasion, it was also observed that some threads spent more time on computation than others, which would lead to a degradation in performance. As described below, each thread executes the same code on the same amount of data and each is expected to complete in the same amount of time, so we attribute this difference to different times spent in contention for the workpile mutual exclusion lock, condition variable signalling and LWP scheduling. In any case, the threads did not provide the desired performance when more CPUs were available for computation. 15

16 4.2 Possible Limitations on Threads Why might there not be more of a speedup when more CPUs are utilized? Ideally speaking, if a number of threads equal to the number of CPUs available are to be utilized in performing registration, the rate at which the registration was performed would rise with a slope of 1 whenever an additional CPU was made available. One reason why this ideal might not be reached is that each thread does not have enough work to compute. The work that is needed to create, manage, and destroy the job structure for the thread may be greater than the work required to perform the desired computation. If a thread were forced to perform more work, then the requirements for computation would probably take more time to accomplish than the requirements for thread maintenance. As the time required for each thread job increases the relative importance of the fixed cost of building the job structure decreases. The cost of thread synchronization also becomes relatively less significant. It must be kept in mind that the time required for each iteration is on the order of milliseconds. A good approximation of this time can be obtained by dividing the total execution time of a trial by the total number of iterations, which was in this case. As was shown with Machine 4, when each thread was given more work to do, there was an increase in speedup. The speedup is also probably limited because of increased thread synchronization costs that do not scale with the available CPUs. In particular, the ratio of time to compute one iteration to the time required to synchronize threads between iterations might not be as high as desired when the number of CPUs increase. Our measurements indicate that the increased synchronization costs can be accounted for by the fact that as the number of CPUs increase, the threads don t execute at the same time because they are spending as much time waiting for work as they are when they execute an iteration. Another reason for not achieving the ideal could also lie in Amdahl s Law, which takes into account the fraction of original execution time that is amenable to improvement. The overall speedup of the registration process may be affected due to this limitation. It was observed that the threads did not start performing computations at the same time. Ideally, if N threads are to be used for a computation that take T seconds to complete, all N threads should begin at the same time so that the total time elapsed will be T seconds. The worst case scenario is where the threads executed serially so that NxT seconds elapse. However, even in the ideal situation, threads do not begin computation at exactly the same time due to several factors, such as mutex competition. In any case, there were cases of some threads executing after other threads had finished. Although each thread might have taken the same amount of time to perform computation, this situation led to the total elapsed time being doubled. If this happened often enough, there would definitely be a drop in speedup. Again, this can be solved if each thread has more work to do because that would result in a reduced probability that a thread would begin computation after another thread has finished computation. Although this would lead to an increase in speedup, the time needed to finish all computation would increase. If one were to have each thread perform more computations in parallel registration, special 16

17 care would have to be taken to preserve the accuracy of the process so that a credible final pose could be obtained. Finally, we observed that the time taken to perform the parallel registration process using one CPU was larger than the time taken to perform the serial registration process. Theoretically, these times should be identical since the parallel registration is operating under the conditions of a serial system. However, as Table 8 shows, the time difference amounts to a fraction of a millisecond more being spent on each iteration of the parallel registration process. This fraction of a millisecond could easily be attributed to the synchronization overhead that limited the speedup. This synchronization is controlled with the POSIX condition variables. As Table 8 also shows, the magnitude of the extra time per iteration could be influenced by the speed of the processors being used by the machine. For instance, Machine 4 is made up of 250 MHz processors and has the largest amount of extra time per iteration at 0.64 ms. However, Machine 2 is comprised of 450 MHz processors and has the smallest amount of extra time per iteration at 0.31 ms. As a whole, the data shows that extra time per iteration is inversely proportional to the speed of a CPU. 5 Future Work This paper addressed the possibility of accelerating the registration process while preserving its accuracy by implementing parallel computing through the use of POSIX threads. In the future, one could attempt to further reduce the synchronization overhead for even better performance. This could be achieved through algorithmic changes and different implementations of parallelism. For instance, utilizing the master-slaves paradigm may reduce the synchronization overhead for better performance. In this scenario, each slave would compute an iteration of the registration process after the work has been prepared. Through changes of this sort, the registration process could evolve to become even more efficient and effective. If a similar success ratio can be maintained within similar translational and rotational perturbations, then it would be very beneficial to implement this change. 6 Conclusion In an effort to accelerate the process of volumetric mutual information registration, parallelism was implemented using POSIX threads. By using threads to execute a standard workpile decomposition that parallelizes the computation of the derivative of mutual information from different iterations of the registration process, the time needed for two sets of volumetric medical image data to be aligned was sharply reduced compared with the current system already in use. Trials performed on several machines showed that the speedup of the registration process was linear with the number of CPUs available for computation. The slope of this speedup was close to one for a small number of processors, but 17

18 was reduced as the number increased. In addition, the accuracy of the registration process was preserved as these changes were implemented. The successful results of these trials show that the implementation of parallelism can greatly speed up the registration of intraoperative data with pre-operative data in order to reduce the length of an intervention and increase the overall comfort of a patient. In addition, the rapid alignment achieved with parallelism would allow a better use of two-dimensional, orthogonal two-dimensional, or low resolution three-dimensional images and hence better surgical navigation. As a result, it would become more practical to acquire more images during surgery, which would lead to a better surgical outcome for the patient. References [1] Simon K. Warfield, Ferenc A. Jolesz, and Ron Kikinis. Real-Time Image Segmentation for Image-Guided Surgery. In Supercomputing 1998, page 1028, November [2] P. Viola and W. Wells. Alignment by maximization of mutual information. In Proceedings of the 5th International Conference on Computer Vision, [3] W. Wells, P. Viola, S. Nakajima, H. Atsumi, and R. Kikinis. Multi-modal volume registration by maximization of mutual information. Medical Image Analysis, 1(1):35 53, [4] J. West, J.M. Fitzpatrick, et al. Comparison and Evaluation of Retrospective Intermodality Brain Image Regsistraion Techniques. Journal of Computer Assisted Tomography, July/Aug [5] G. D. Rubin, S. Napel, and A. N. Leung. Volumetric analysis of volumetric data: Achieving a paradigm shift. Radiology, 200: , [6] P. Viola. Alignment by Maximization of Mutual Information. PhD thesis, Massachusetts Institute of Technology, [7] B. Widrow and M. Hoff. Adaptive switching circuits. IRE WESCON Convection Record, [8] S. Kleiman, D. Shah, and B. Smaalders. Programming with Threads. Prentice Hall, [9] L. Noordergraaf and R van der Pas. Performance Experiences on Sun s Wildfire Prototype. In Supercomputing 1999,