Response to Reviewers We thank the reviewers for their feedback and have modified the manuscript and expanded results accordingly. There have been several major revisions to the manuscript. First, we have removed results for dry femur bones because it was noted by a reviewer that such scenario is not clinically realistic. Questions were raised as to how the accuracy of 2D/3D registration affects the reconstruction result. We have included results describing the sensitivity of the reconstruction algorithm to registration error in the revised manuscript. Results reported in the original manuscript were based on synthesized pre / post-operative X-Ray images. Based on insights gained from those experiments, we redesigned the imaging / reconstruction pipeline and conducted a second cadaver experiment. Results we report in the second cadaver experiment are based on real X-Ray images, and the imaging pipeline has been modified so 2D/3D registration is not required. Detailed responses and changes are as follows: Reviewer: 1 General comments The present method is sound. It can be regarded as an extension of the well-known Chan-Vese model (Chan T.F. and Vese L.A., Active contours without edges, IEEE Transactions on Image Processing, 10(2):266 277, 2001) to solve a joint segmentation and reconstruction problem. As a region-based approach, the present method is able to efficiently use global image statistics inside and outside the active region and thus can drastically improve the robustness of the joint segmentation and reconstruction results. The incorporation of the prior CT information, if it is available, into the present solution, also helps to improve the results. We agree that the successful work by Chan and Vese on region-based segmentation has been extended here for cement reconstruction. In pp. 3 bottom right, we ve noted that the constant foreground / background assumption was inspired by their work. I also find that the paper is easy to follow and that the presentation is clear. Thank you! Several issues as listed below, however, need to be addressed: (1) Although comprehensive results are presented to demonstrate the efficacy of the present method and to investigate the influences of different factors on the reconstruction accuracy, the present method was evaluated only with synthetic X-ray images of phantoms and one cadaver. It is unclear how well the present method will perform when real X-ray images are used. Furthermore, it would be interesting to know how the accuracy of the intensity and geometric calibration of the DRR generator with respect to real X-ray images influences the reconstruction results. We thank the reviewer for pointing out that accurate DRR generators are very difficult to build and calibrate, and that the original manuscript left questions as to how the algorithm would perform on real data. Based on these comments, we changed the imaging procedure and conducted a second cadaver experiment that uses real X-Ray images for reconstruction. See Section 4c (pp. 6). (2) In the cadaver experiment, a prior CBCT was used. It is known from the description that the prior CBCT is registered to a post-operative CBCT using intensity-based 3D/3D registration and that the
synthetic X-ray images used for reconstruction are generated from the post-operative CBCT. However, it is unclear how accurate the intensity-based 3D/3D registration is and how the registration accuracy influences the reconstruction results. The sensitivity of the present method to the accuracy of matching the pre-operative CT to intra-operative X-ray images should be investigated if the ultimate goal of this work is to apply the present method to a clinical situation. We acknowledge that registration error will contribute to reconstruction accuracy. In response, we have included results that show the sensitivity of SxMAC to registration error, which we report in section 4b (pp. 5) and plots appear in Fig. 10 (pp. 9) and Fig. 11 (pp. 10). We thank the reviewer for pointing out that matching intensities between pre-operative CT and intra-operative X-ray images is important for making this algorithm applicable to a real clinical situation. We investigated this source of error and decided the best way to address it was to circumvent the problem by using real X-Ray images instead of pre-operative DRRs. See section 4c (pp. 6) for a description on how the reconstruction pipeline was modified. These modifications do not change the clinical workflow because the images are already acquired for image guidance. (3) As the present method tries to solve a joint segmentation and reconstruction problem, it will be also interesting to see how well the X-ray image segmentation performs. Probably this will give more intuitive insight about the performance of the present method. We agree that 2D image segmentation is important to the initialization process, and therefore should be investigated. In section 4c (pp. 6), we discuss results for initializing SxMAC with a silhouette reconstruction obtained with automatic 2D image segmentation. Specific Comments 1. Page 3, right column, line 22: M is an M x N matrix should be M_{k} is an M x N matrix (I used latex style to type the subscript.) 2. Page 7, References. Please carefully check following citations to make sure that they are all correct Ref [6]: there is no journal name for this reference. Ref [7]: are you sure that there is a journal called IEEE Trans on Medical Image Analysis? If you refer this paper to the one that authors published in IEEE Trans. On Medical Imaging, then the title is not correct. Thank you for noticing this error, it has been corrected. Reviewer: 2 Comments to the Author 1) Fonts: Please use the standard TMI fonts. At the moment, the abstract and various subsections appear in a different font than the text itself. Also, there is some weird bolding going on in the abstract, please fix. We apologize for any formatting errors. This manuscript was written in Microsoft Word 2007 for the PC. We have double checked that our fonts are consistent with the Word Template provided by the publisher (http://www.ieee.org/publications_standards/publications/authors/authors_journals.html).
However, there are known to be formatting differences between Word and Latex templates that will give the document a different appearance. We will work with the publisher to insure the document is formatted correctly for final submission. 2) References: Given that the main input to your algorithm is a perfect alignment between CT and X- rays, please cite the main papers in X-ray to CT registration. Perhaps it is best to cite: Markelji et al., A review of 3D/2D registration methods for image-guided interventions, MedIA, 2010 (perhaps it could be added to page 5, right column, beside [24]). Thank you for finding this reference. We have cited the Markelji et al. paper pp. 5 top right. 3) The fundamental hypothesis is that accurate reconstruction of the shape of the cement will assist with improving the accuracy of the FEM analysis to identify the augmented bone strength. It is not clear how accurate this shape has to be to achieve this objective. However, this point does not take away anything from the contributions of this paper. We agree with the reviewer that since we re ultimately interested in FEM analysis, the reconstruction algorithm should be analyzed in that context. This paper focuses on just the reconstruction algorithm for an image analysis audience, but subsequent work will describe how the reconstruction algorithm interacts with the FEM analysis. This has been identified as future work and described in the conclusion. See section 6 (pp. 7). 4) Please define CBCT (Cone-beam CT) on the second page, left column, second line. A definition of CBCT has been added in Section 2, pp. 2 top left. 5) Please define E_model, and E_data, as well as P_bg, P_rg, I(s,thera), and K in equations (1), (2), (4) and (6). We ve modified the manuscript to reference K, E_model and E_data the text, pp. 2, top left. P_bg and appear in P_rg Table 1. 6) I am not sure why you are using C-Arm and X-Ray throughout, usually they are written as Carm and X-ray. We ve changed C-Arm to C-arm and X-Ray to X-ray throughout. 7) DRR_k in Figure 2 is defined on page 3. Equation is now referenced in figure 2 captions (pp. 4). 8) Your figures captions in general are very short and do not contain much information. Please make them self contained. We have changed captions in Figures 4, 5, and 6 to be more descriptive, but welcome comments regarding more specific changes and clarifications. 9) Page 3, left column, first paragraph, change we re to we are. 10) Please check throughout and make sure before each respectively, there is a comma.
11) Section IV.A. It is not clear to me how the results would change for different values of lambda_k. It seems that you pick these numbers randomly. Are these shape specific? In all your experiments on phantoms and cadavers, did you use the same value of lambda? The optimal lambda is shape specific and sensitive to both the number of images and image contrast. How to choose the optimal lambda is an extended topic of discussion, but we ve added on pp. 4 top right The regularization parameter should be chosen large enough to reduce directional bias, but small enough to preserve sharp object features. We now mention in the conclusion (pp. 7 bottom right) that optimizing lambda is part of future work. 12) Figure 3 is not clear at all. Please add explanation to the caption and the text. We ve clarified the figure by adding: Reconstructions shown in (a), (b), and (c) are juxtaposed to (d) ground truth in (e), (f), and (g). 13) Page 5, right column, the steps described after the first paragraph: It is not clear at all why these specific steps are chosen. Please provide the logic in the text with more description. The imaging scenario for bone augmentation is described in Y. Otake, et al., "An image-guided femoroplasty system: development and initial cadaver studies," in SPIE, San Diego, CA, 2010, p. 76250P. To clarify that the imaging procedure is described in previous work, we ve added to the text in pp. 5 bottom left that The imaging scenario has been described in previous work [5] as follows: 14) Page 5, right column, after Table 3, please explain For experiments without soft tissue, the algorithm was initiated with active contour.. Also, you have used / many times, it is not clear if it is an OR or and AND. Please remove them and clear up the text. 15) Page 5, it is not clear how the injector s tip is tracked. We ve added to pp. 5 bottom right: Such initialization is clinically plausible because the cement injector s tip is tracked with a Polaris optical tracker as part of the bone augmentation procedure. 16) Figure 5, it is not clear what is pre-operative and what is post-operative. Please add more information. We ve changed the caption to read Synthesized (a) pre-operative X-Ray image of dry femur and (b) post-operative X-ray image with cement attenuation of 1900 HU. 17) Please fix the fonts in the reference list. We ve checked that the fonts are consistent with the Word 2007 template provided by the publisher. We will work with the publisher to make sure all fonts are correct in final formatting. 18) Figure 6, it is not clear why the error drops in Figure 6 (a) and saturates after 1900 HU. Also, why the error increases in Fig 6 (c)? Please add (a), (b), to the subfigures. We ve removed results with dry femur bones, so no discussion is necessary.
19) There is very little discussion on the results for Figure 6, 8 and 9. Please add that to the text. See comments below. 20) In Figure 8, why do you think your error saturates after 4 images? Error will saturate once error measurements approach the resolution of the grid (1 mm). The remaining error is due to regularization because the smoothing parameter (lambda_kappa) is nonzero. However, we have removed this figure since dry femurs are not a realistic scenario. 21) In Figure 9 (b), there is a monotonic reduction in error with sweep angle, and the standard deviation is very large. Why? Figure 9 b has been removed, but we ve added in the text, pp. 6 bottom left: Reconstruction accuracy improves with increasing sweep angle because there is less redundant information between views. This trend is evident in phantom experiments (Fig. 4). 22) What is significant about 60 degrees sweep that your error saturates in Figure 9 (d). Figure 9d has been removed because it deals with dry femur bones. 23) Something that is not discussed at all is the impact of initial 2D-3D registration error. Given that this registration is not going to be perfect, you need to do further analysis on the errors you are reporting in relation to this error. I would suggest further simulations that could address this. If the results get significantly impacted, would there be a real need for the proposed technique, as your FEM analysis might be incorrect at the end. We agree with the reviewer s comment and have conducted experiments with synthesized X-Ray images to analyze the sensitivity of SxMAC to registration error. The experiment is described pp. 6 top left, and results described pp. 6 bottom left. Algorithms for 2D/3D registration in this region of the body are fairly accurate and reported to have less than 2 mm translation and 1 degree rotation error. Reviewer: 3 Comments to the Author The paper describes a 3D segmentation based sparse reconstruction method for a free-form object which is based on a series of x-ray images and a prior CT scan. My evaluation is done in three categories: 1- Clinical Utility: There are a lot of assumption made in the course of the paper, these are namely, (a) Data availability, CT and a few intra-operative x-ray, (b) relatively accurate registration (2D-3D), (c) accurate DRR computation in the light of various characteristics of x-ray and CT, etc. The question then remains as to what are the advantages of the proposed method to either intra-operative CBCT, or CT. The authors should provide a discussion whether the cost saving and dose reduction justifies a presumably lesser accurate results for a sensitive procedure such as vertebroplasty. We agree with the reviewer that the argument for using this reconstruction algorithm can be strengthened. SxMAC requires only 4 images, or about 1% of the dosage of performing the same procedure with intra-operative CBCT. The clinical workflow for the bone augmentation procedure requires a pre-operative CT for finite element analysis. However, the X-ray differencing technique described in the second cadaver study does not require a pre-operative CT, making it adaptable to existing procedures like Vertebroplasty. We ve expanded our discussion of its potential impact pp.7 top right.
2- Method and novelty: The equation 6 captures the contribution of the paper. However, it falls short in several aspects. First, L2 norm for comparing the DRR and X-ray shown to be not a good measure, mainly due to various ma and also the characteristic differences in fan beam CT and x-ray detectors. Second, the Euler-Lagrange equation of 16 guarantees a local optima, where in this case since you have a 2D observation space, and 3D solution space, can be very much dependant on the initial estimates of the segmentation. In both cases authors should better explain the reasons for choosing sum of squared differences, and also do discuss the dependency to the initial estimates of the segmentation if there is any. Currently, only the silhouette based initialization is used. The method seems be rather straightforward, what I have mostly questions about is the robustness, which needs to be evaluated for example based on some empirical experiments. We recognize that simulation of realistic X-Ray images from diagnostic CT is a difficult and open problem. SxMAC assumes there is an accurate DRR simulator for which the L2 norm is a reasonable metric for comparing DRRs and real X-ray images. The problem is lessened if a CBCT obtained with the same X-ray machine is used instead of diagnostic CT. In the second cadaver experiment, we use real X-ray images instead of DRRs as a prior to circumvent issues with accurate DRR generation. The second cadaver experiment does not require a pre-operative CT, but one will be acquired and registered to the patient regardless because it is required for FEM analysis. Secondly, we recognize that SxMAC is sensitive to initialization. The second cadaver experiment uses silhouette reconstruction, which we show is a very accurate, although not robust for large numbers of images. See pp. 6 and Figure 14 (pp. 10). 3- Validation: The phantom experimentation is not realistic. The reconstructed object is the sole object in the space ("no soft tissue"), which is not matching the clinical case of where cement is injected for vertebroplasty. The author should craft a better more representative phantom experimentation, specially in light of limited cadaver experiments. The phantom experiments should also have analysis results with regards to initial estimate variation, errors due to registration, and variation in x-ray imaging characteristics, etc. We agree with the reviewer that phantom experiments with no soft tissue are not realistic and have removed these results from the manuscript. We ve added experimental results on registration error, described pp. 6 top left, and results described pp. 6 bottom left. 4 Editorial: Some of references (for example 13, 15) are missing information such as year, journal, or conference proceedings, page number, etc. Overall, the paper requires some revision prior to the publication.