Using a handheld stereo depth camera to overcome limited field-of-view in simulation imaging for radiation therapy treatment planning

Transcription

1 Using a handheld stereo depth camera to overcome limited field-of-view in simulation imaging for radiation therapy treatment planning Cesare Jenkins Departments of Radiation Oncology, Stanford University, Stanford, CA 94305, USA Departments of Mechanical Engineering, Stanford University, Stanford, CA 94305, USA Lei Xing and Amy Yu a) Departments of Radiation Oncology, Stanford University, Stanford, CA 94305, USA (Received 3 October 2016; revised 3 March 2017; accepted for publication 3 March 2017; published 17 April 2017) Purpose: A correct body contour is essential for reliable treatment planning in radiation therapy. While modern medical imaging technologies provide highly accurate patient modeling, there are times when a patient s anatomy cannot be fully captured or there is a lack of easy access to computed tomography (CT) simulation. Here, we provide a practical solution to the surface contour truncation problem by using a handheld stereo depth camera (HSDC) to obtain the missing surface anatomy and a surface surface image registration to stich the surface data into the CT dataset for treatment planning. Methods: For a subject with truncated simulation CT images, a HSDC is used to capture the surface information of the truncated anatomy. A mesh surface model is created using a software tool provided by the camera manufacturer. A surface-to-surface registration technique is used to merge the mesh model with the CT and fill in the missing surface information thereby obtaining a complete surface model of the subject. To evaluate the accuracy of the proposed approach, experiments were performed with the following steps. First, we selected three previously treated patients and fabricated a phantom mimicking each patient using the corresponding CT images and a 3D printer. Second, we removed part of the CT images of each patient to create hypothetical cases with image truncations. Next, a HSDC was used to image the 3D-printed phantoms and the HSDC-derived surface models were registered with the hypothetically truncated CT images. The contours obtained using the approach were then compared with the ground truth contours derived from the original simulation CT without image truncation. The distance between the two contours was calculated in order to evaluate the accuracy of the method. Finally, the dosimetric impact of the approach is assessed by comparing the volume within the 95% isodose line and global maximum dose (D max ) computed based on the two surface contours for the breast case that exhibited the largest contour variation in the treated breast. Results: A systematic strategy of using a 3D HSDC to compensate for missing surface information caused by the truncation of CT images was established. Our study showed that the proposed technique was able to reliably provide the full contours for treatment planning in the case of severe CT image truncation(s). The root-mean-square error for the registration between the aligned HDSC surface model and the ground truth data was found to be 2.1 mm. The average distance between the two models was mm (mean SD). Maximum deviations occurred in areas of high concavity or when the skin was close to the couch. The breast tissue covered by 95% isodose line decreased by 3% and D max increased by 0.2% with the use of the HSDC model. Conclusions: The use of HSDC for obtaining missing surface data during simulation has a number of advantages, such as, ease of use, low cost, and no additional ionizing radiation. It may provide a clinically practical solution to deal with the longstanding problem of CT image truncations in radiation therapy treatment planning American Association of Physicists in Medicine [ doi.org/ /mp.12207] Key words: camera, CT simulation, extended field-of-view, obese, treatment planning 1. INTRODUCTION Accurate imaging and subsequent modeling are crucial for radiation therapy treatment planning. The introduction of computed tomography (CT) for simulation, treatment planning, and patient setup has revolutionized the field of radiation therapy. 1,2 During treatment simulation, the patient is usually imaged by a CT scanner in order to obtain the data necessary for planning and dose optimization. In most cases, the nominal field-of-view (FOV) for a CT scanner is 50 cm. Some scanners offer an extended FOV 3,4 in which a FOV of up to 70 cm can be reconstructed. Clinically, the use of wide bore CT is often required in order to capture the entire external contour and internal anatomy of the patient with adequate 1857 Med. Phys. 44 (5), May /2017/44(5)/1857/ American Association of Physicists in Medicine 1857

2 1858 Jenkins et al.: Solution for the limited FOV of CT simulation 1858 immobilization devices in place. The extended FOV image reconstruction may not always be an option, even when a wide bore CT scanner is available (e.g., GE Revolution CT has a wide bore but no extended FOV reconstruction). Furthermore, for some specially positioned patients or obese patients, their body might extend beyond the extended FOV, which results in missing areas of the external contour. 5 In this case, in order to retain the correct dose calculation, some beam angles have to be avoided which leads to a suboptimal treatment plan. 6 The purpose of this work is to develop a novel surface contour imaging strategy that makes effective use of a handheld stereo depth camera (HSDC) and surface surface image registration techniques. HSDCs offer the ability to capture three-dimensional (3D) images of a subject with a simple hand-held device. HSDC imaging has a number of advantages, such as low cost, portability, ease of use, no ionizing radiation, and high accuracy, 7,8 making it an ideal tool for us to capture the 3D contour of the patient to compensate for truncated CT data. In this paper, we first provide a brief description of HSDC surface imaging and data processing techniques. A set of experiments using 3D-printed phantoms created from three patients CT images were then performed to demonstrate the proposed clinical workflow and to evaluate the effectiveness of the proposed strategy. Our study showed that effective use of HSDC and feature-based surface surface registration provides a reliable solution to overcome the image truncation problem in radiation therapy planning. When implemented clinically in the future, the approach should lead to improved workflow and accuracy in radiation therapy. 2. MATERIALS AND METHODS Supplemental information about a patient s body contour must meet two requirements to be useful to solve the image truncation problem in treatment planning. The first is the fidelity in capturing the detailed variations of patient topology, and the second is the accurate alignment of the supplemental information to the primary body contour obtained from simulation CT. The fidelity of surface image is primarily determined by the quality of the HSDC used, whereas the alignment accuracy depends on the performance of the image registration algorithm used to register the camera-generated model and a CT scan. The following provides details on these two aspects and our quantitative assessment of a proposed implementation. Due to the extensive use of acronyms throughout the remainder of the paper, we have included a brief description of each in Table I. 2.A. Handheld depth camera Three-dimensional scanning technologies have been available for well over a decade. Recently, these technologies have become increasingly accessible. The camera used in this study, a Realsense R200 (Intel Corp., Santa Clara CA, USA), is a development module intended to demonstrate hardware and software that will eventually be integrated into laptops, tablets, or phones. The camera consists of three independent imaging sensors and an IR laser projector. The laser projector projects a grid of dots that are viewed using two monochrome IR imaging sensors. These two imaging sensors are calibrated as a stereoscopic pair, enabling an image processing algorithm to determine the location, in three dimensions, of thousands of points in the imager field-of-view. The third image sensor, a standard color imaging sensor, provides color information that can be correlated to each point. This information can then be used to create a three-dimensional scan of a subject by combining the sets of points from different vantage points into a single model. The surface model consists of a cloud of points with x, y, and z coordinates relative to a given reference frame. These points are grouped together in triangles to form a mesh of small planar surfaces. The software provided with the camera further works to segment a human subject from the background and allows model creation to points only corresponding to that subject. TABLE I. Summary of acronyms. Acronym Name Source Where used CTSM CT-generated surface model Created from contours created by the TPS on a nontruncated patient CT Truncated to created TSM, serves as ground truth for comparison to RCSM CSM Camera-generated surface model Created by the HSDC and associated software following HSDC scan Registered to TSM to create RCSM TSM Truncated CT surface model Created by truncating the CTSM. In a real-world scenario, this surface would be created from contours generated from a patient s truncated CT scan. The CSM is registered to this surface. RCSM Registered camera-generated surface model. Created by registering the CSM to the TSM. This surface is imported to the TPS to be used to complete the patient contour and override pixels not included in the original CT scan. RMSE Root mean square error Measure of unsigned distance between two point clouds. Calculated as part of the registration between two surface models. C2M Cloud-to-mesh distance Measure of signed distance between a cloud of points and a mesh. Calculated as the error between the RCSM and CTSM.

3 1859 Jenkins et al.: Solution for the limited FOV of CT simulation B. Proposed workflow The proposed workflow for utilizing the HSDC to obtain missing surface contour information for a patient is summarized in Fig. 1. The process begins with capturing surface data of the patient at the time of CT simulation by performing a brief scan with the HSDC immediately prior to or following the CT scan while the patient is on the couch. These data are then formed into a mesh surface model using software provided by the camera manufacturer (Fig. 1: CSM). This mesh is then registered to the CT by performing a surface-to-surface registration of the HSDC mesh and the body contour extracted from in field-of-view areas of the CT scan (Fig. 1: TSM). The registered mesh data (Fig. 1: RCSM) are then imported into the TPS where it may be used by the physician or dosimetrist in completing the body contour. Areas missing internal anatomy are assigned a value of zero Hounsfield units (HU) and the resulting dataset is used for further treatment planning. 2.C. Patient-specific phantoms In order to evaluate the accuracy of the proposed approach, the following experiments were performed (please see Fig. 2 for a visual representation of the experimental workflow). First, we selected three previously treated patients whose CT images were not truncated. The patients included two breast cases and one prostate case. A phantom mimicking each patient was created by using a treatment planning system (TPS) automatic contouring module to segment the body surface from the corresponding CT dataset (Eclipse, v 11, Varian Medical Systems, Palo Alto, CA, USA). This surface was then transformed into a solid 3D model using 3D Slicer. 9 This CT-based surface model (CTSM) forms the ground truth for each patient. Each CTSM was then fabricated on a fused deposition modeling 3D printer (Makerbot Z18, Makerbot Industries, Brooklyn, NY, USA). 2.D. 3D Camera scanning and registration The patient-specific phantom for each case was scanned using the HSDC and its accompanying software application. Areas of the scan corresponding to the planar surfaces at each end of the phantom were manually removed using a mesh manipulation software application (Meshmixer, Autodesk Inc., San Rafael, CA, USA). Any background areas mistakenly included by the software were also removed. The result was a camera-generated surface model (CSM) ready to be registered to the patient CT data. Since the goal of the project was to examine the use of a 3D camera in an out of FOV scenario, each CTSM was truncated to simulate the effects of an out of FOV scan, resulting in a truncated surface model (TSM). The CSM was registered to TSM to bring it the camera-generated data into the same coordinate system as the CT data. This registered camera surface model (RCSM) was then compared with the original CTSM in order to evaluate the quality of the resulting data. The registration technique used was similar to that described previously by Kim et al. 10 and consisted of a rough Truncated CT Truncated Contour Truncated CT Surface Model (TSM) Patient CT Scan TPS Countouring Model Generation Registration Camera Contour Final Contour for Planning HSDC Scan Registration Registered Camera Surface Model (RCSM) Import to TPS Override Camera Surface Model (CSM) FIG. 1. The proposed patient workflow for an out of field-of-view patient. The patient is scanned with the CT scanner and the HSDC. The treatment planning system is used to generate a body contour for the truncated CT. These contours are used to create surface model (TSM) to which the camera-generated surface model (CSM) is registered. The registered camera surface model (RCSM) is then imported into the treatment planning system. Areas outside of the CT scan, but within the RCSM are overridden with a default value. [Color figure can be viewed at wileyonlinelibrary.com]

4 1860 Jenkins et al.: Solution for the limited FOV of CT simulation 1860 Patient CT Patient Contour CT Surface Model (CTSM) Phantom TPS Countouring Model Generation Comparison 3D Printer Artificial Truncation Comparison Measured Error HSDC Scan Truncated CT Surface Model (TSM) Registration Registered Camera Surface Model (RCSM) Registration Camera Surface Model (CSM) FIG. 2. Overview of experimental evaluation of the technique. A patient surface model (CTSM) was created from contours generated by the TPS from the patient s CT scan. This model was fabricated on a 3D printer to create a physical phantom which was scanned with the HSDC to create a camera-generated surface model (CSM). The CSM was then registered to an artificially truncated version of the CTSM (TSM). The registered CSM is then compared with the CTSM. [Color figure can be viewed at wileyonlinelibrary.com] initial alignment based on four manually selected pairs of points, followed by a fine-tuned alignment using the iterative closest point algorithm. 11 The points selected for the rough alignment pairs corresponded to easily identified landmarks, such as the nose, on each of the two surface models. Kim et al. demonstrated that the results of the fine-tuning alignment were largely insensitive to variations in the rough alignment pair selection. The iterative closest point (ICP) algorithm seeks to minimize the sum of the squared distances between each point on the CSM and the closest point on the TSM. All registrations and subsequent measurements were made using an open source mesh alignment and evaluation tool E. Evaluation The accuracy of CSM to TSM registration was evaluated by calculating the root-mean-square error (RMSE) across all points included in the alignment. Points determined to be outliers by the ICP algorithm are not included. Outliers are identified as points with the greatest alignment error after reducing the average error across all points. Outliers are removed from consideration and the alignment is re-optimized based on the remaining points. A percentage of points is specified as the minimum that must remain in the utilized set. The accuracy of each registered camera-generated surface model (RCSM) was evaluated by comparing it to the original CTSM. The distance between each point on the RCSM and the closest planar mesh element on the CTSM was calculated. These distances, referred to as cloud-to-mesh distances (C2M), were summarized both for the RCSM as a whole, and for those areas of the RCSM that corresponded to areas simulated to be outside the FOV in the TSM. Finally, in order to evaluate the dosimetric impact of the contour discrepancy between the RCSM and CTSM, a treatment plan was created for one of the breast cases included in the study. This case was selected as it contained the largest deviations between the CTSM and RCSM contours within the volume for which the dose was to be calculated. The RCSM was converted to a DICOM structure and imported into the TPS where it was selected and used as the patient contour. Areas of the CT that corresponded to the simulated missing tissue, i.e., within the RCSM contour but outside the TSM contour, were set to a Hounsfield unit (HU) of zero. A dose calculation was performed with the RCSM contour and overridden HU values. An additional calculation was performed with the CTSM contour and original HU values. The volume of tissue within the 90% and 95% isodose lines and the global maximum dose (D max ) were evaluated and compared between the two models. 3. RESULTS A 3D scan of each 3D printed model was completed in approximately 60 s. The average registration error (RMSE) for the registration of the CSM and TSM across the three

5 1861 Jenkins et al.: Solution for the limited FOV of CT simulation 1861 patients was 2.1 mm. When considering the entire RCSM model, the C2M distance between the RCSM and CTSM was mm. When considering only the areas of the RCSM that correspond to regions that were simulated to be outside of the FOV, the C2M distance between RCSM and CTSM was mm. Refer to Table II for detailed statistics for each patient. Refer to Fig. 3 for a visual representation of the differences between the RCSM and CTSM. The contours corresponding to the CTSM and RCSM were examined and found to be within 1 cm (Fig. 4). Areas of largest deviation were found in regions of highly concave anatomy or in regions close to the couch (see Section 4 for detailed discussion). A dosimetric study was performed and showed that the volume breast tissue covered by the 95% and 90% isodose lines decreased by ~3% and D max increased by 0.2% with the TABLE II. Summary statistics for registration error and measured errors between the registered camera surface model (RCSM) and CT-generated surface model (CTSM). Patient Treatment type Registration error (RMSE) for CSM to TSM (mm) Measured error (C2M) between RCSM and CTSM (mean SD, mm) Entire RCSM RCSM in simulated out of FOV region A Breast B Prostate C Breast Average (a) (b) (c) Points Points Points C2M distance (mm) C2M distance (mm) C2M distance (mm) FIG. 3. Visual representation of the camera-generated surface models color-coded for the distance between the RCSM and the CTSM (top). Corresponding histograms of C2M distances for the entire RCSM are also shown (bottom). The horizontal axis is the C2M distance for each bin and the vertical axis is the number of points in each bin. All dimensions are in mm. [Color figure can be viewed at wileyonlinelibrary.com]

6 1862 Jenkins et al.: Solution for the limited FOV of CT simulation 1862 use of the RCSM model. The isodose line for the two models is shown in Fig DISCUSSION While the introduction of extended field-of-view reconstruction has increased the accuracy of patient contouring, there are some instances where it is not available or insufficient. This may be the case in clinics that do not have access to modern equipment, who share equipment with diagnostic radiology departments, or where patient anatomy extends beyond the extended field-of-view. Some approaches have been introduced previously to deal with the image truncation problem in these instances. Notably, an image fusion method was proposed that registers overlapping regions of two or three CT scans with the center of each scan set at different positions within the patient. 13,14 However, this procedure is rather tedious, delivers additional radiation dose to the patient, and prolongs the imaging time. In addition, the shifts required to reposition the patient between scans may result in deformations in the patient or immobilization setup which would compromise the fidelity of the fused scan. Dramatically different from the existing techniques, in this work, the surface contour of a patient is obtained by using a HSDC immediately prior to or following the simulation CT scan while the patient is on the CT couch. This adds only one or two minutes to simulation time, which minimizes the level of discomfort to patient. The technique promises to provide a viable solution to mitigate the longstanding problem of image truncation in radiation therapy. The RMSE of alignment reported in column 1 of Table II summarizes the performance of the proposed method. The mean and standard deviations of C2M distances offer a global view of how accurately the CSM represented the true patient anatomy. The largest discrepancies were observed in regions of highly concave anatomy or in regions close to the couch. This is due to the fact that both monochrome sensors must be able to simultaneously view an area in order to create 3D points for that area. In highly concave areas of the body, such as where the arms are placed close to the head during a breast treatment, it is difficult or impossible for both sensors to view these regions. As the camera software combines multiple views together to form a final model, it smoothens over these missing areas by connecting the closes available points. Even if a few points are available, the model generation inherently smooths over very sharp features as these may represent outliers in the point data. This effect can be seen in the neck creases of the breast patients in Fig. 3. Fortunately, the total volume of tissue subject to this error is generally small which results in a small dosimetric impact as shown in Fig. 4. While the effect is small, it may be useful in future implementations to indicate the level of confidence the camera software has in various areas of the generated mesh model, or reduce the distances over which the smoothing algorithm is allowed to fill in mesh data. A second source of error in the scans arises from misalignment between various views as they are combined into a single model. This misalignment results in model FIG. 4. Comparison of contours for a breast patient. The slice of the simulation CT with the greatest contour deviation is shown with the contours corresponding to the CTSM, TSM, and CSM are shown in green, blue, and magenta (from the left to right where the white arrow pointing at). The largest discrepancy is less than 1 cm (white arrow). The 90% isodose line (pink, outer line) and 95% isodose line (yellow, inner line) are comparable between CTSM (bottom left) and RCSM model (bottom right). The D max increase 0.2% for CSM model. [Color figure can be viewed at wileyonlinelibrary.com]

7 1863 Jenkins et al.: Solution for the limited FOV of CT simulation 1863 errors such as the one highlighted by the white arrow in the top image of Fig. 4 where the patient anatomy is aligned on the anterior surface, but slightly offset from the posterior surface. This type of contour offset is expected to have the largest dosimetric impact on a breast case because the treatment volume is primarily determined by the body contour rather than internal anatomy. We therefore selected the patient where the greatest contour variation was observed within the dose calculation volume for the dose calculation investigation. While this type of error affects larger volumes of tissue, the error can be reduced by moving the camera more slowly around the patient, or by improving the model generation software (the software used for this study was a beta version that is expected to be improved with future updates). In a real-world scenario, truncation artifacts that obscure the anterior portion of the patient contour would be expected to reduce the accuracy of the alignment between the CSM and the TSM (internal or posterior artifacts would be expected to have little or no effect on this technique). As a means for improving the alignment between the CSM and TSM, it may be possible to employ fiducials that are clearly visible in the CT dataset and also easily identified and localized by the HSDC during a scan. The locations of these fiducials could be automatically determined in both datasets and enable fully automated, highly accurate registration. This automatically registered dataset would then be imported into the TPS to be available, in addition to the contour automatically created from the truncated CT, to the individual creating the patient treatment plan. The additional software development necessary to implement this technique is currently underway. Finally, it is known that factors such as room lighting, camera calibration, and user technique can affect HSDC accuracy. It would be prudent to perform periodic quality assurance on the camera by having common operators scan objects of known dimensions in the CT room to ensure accurate performance. The technique proposed here is quite general and may be applied for extending the FOV of MRI-based simulation in radiation therapy. Because of its superior soft tissue contrast and absence of ionizing radiation, MRI-based simulation and treatment planning is currently under intensive study. 15 In practice, being able to patch truncated MRI images would greatly facilitate the clinical workflow and improve the quality of treatment. Finally, we note that the HSDC could also be useful to provide body contours for some special treatment planning procedures, such as emergency treatment and total body irradiation (TBI) 8 where no volumetric CT imaging scan is available. In the latter case, the HSDC data can be utilized to guide the 3D printing of compensators to improve the plan quality. 5. CONCLUSIONS Conversion of a physical object into digital information can occur in a few ways. The use of HSDC is perhaps one of the most convenient approaches to accomplish this, as demonstrated in many areas of science and engineering. In this work, we have, for the first time, applied HSDC to recover the truncated imaging data for radiation therapy treatment planning. The feasibility of combining 3D camera and CT imaging data to fully capture the patient s surface contour has been demonstrated successfully. Immediately available reconstruction of complex patient anatomy fulfills an unmet clinical need and provides a useful tool for improved patient treatment. While the technique was shown to be highly accurate when scanning the 3D printed patient models, further work is required to ascertain its accuracy on the actual patients, particularly in the presence of additional immobilization apparatus. A protocol for carrying out the patient study is pending approval of IRB approval and will be reported somewhere else in the future. ACKNOWLEDGMENT We wish to thank Intel Corporation for providing the 3D camera used for the study. This work was partially supported by the National Institute of Health (NIH) (1R43TR001285, 1R01 CA133474, and 1R01 EB016777). CONFLICTS OF INTEREST The authors have no relevant conflicts of interest to disclose. a) Author to whom correspondence should be addressed. Electronic mail: amysyu@stanford.edu. REFERENCES 1. Kijewski PK, Bjarngard BE. The use of computed tomography data for radiotherapy dose calculations. Int J Radiat Oncol Biol Phys. 1978;4: Aird EG, Conway J. CT simulation for radiotherapy treatment planning. The British journal of radiology. 2002;75: Hsieh J, Chao E, Thibault J, et al. A novel reconstruction algorithm to extend the CT scan field-of-view. Med Phys. 2004;31: Zamyatin AA, Nakanishi S. Extension of the reconstruction field of view and truncation correction using sinogram decomposition. Med Phys. 2007;34: Wu V, Podgorsak MB, Tran TA, Malhotra HK, Wang IZ. Dosimetric impact of image artifact from a wide-bore CT scanner in radiotherapy treatment planning. Med Phys. 2011;38: Srivastava SP, Das IJ, Kumar A, Johnstone PA. Dosimetric comparison of manual and beam angle optimization of gantry angles in IMRT. Med Dosim. 2011;36: Khoshelham K, Elberink SO. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors (Basel). 2012;12: Lee MY, Han B, Jenkins C, Xing L, Suh TS. A depth-sensing technique on 3D-printed compensator for total body irradiation patient measurement and treatment planning. Med Phys. 2016;43: Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging. 2012;30: Kim Y, Li R, Na YH, Lee R, Xing L. Accuracy of surface registration compared to conventional volumetric registration in patient positioning

8 1864 Jenkins et al.: Solution for the limited FOV of CT simulation 1864 for head-and-neck radiotherapy: A simulation study using patient data. Med Phys. 2014;41: Besl PJ, McKay HD. A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell. 1992;14: Software C.V.G. Retrieved from Wu H, Zhao Q, Cao M, Das I. A line-profile based double partial fusion method for acquiring planning CT of oversized patients in radiation treatment. J Appl Clin Med Phys. 2012;13: Fisher CM, Fortenberry BR, Jhingran A, Eifel PJ. Novel technique for simulation and external beam treatment planning for obese patients. Pract Radiat Oncol. 2011;1: Jonsson JH, Karlsson MG, Karlsson M, Nyholm T. Treatment planning using MRI data: An analysis of the dose calculation accuracy for different treatment regions. Radiat Oncol. 2010;5:62.