Prediction of Respiration-Induced Internal 3-D Deformation Fields From Dense External 3-D Surface Motion

Transcription

1 Prediction of Respiration-Induced Internal 3-D Deformation Fields From Dense External 3-D Surface Motion Oliver Taubmann 1,2, Jakob Wasza 1, Christoph Forman 1,2, Peter Fischer 1,2, Jens Wetzl 1,2, Andreas Maier 1,2, Joachim Hornegger 1,2 June 26, 2014 CARS, Fukuoka, Japan 1 Pattern Recognition Lab, FAU Erlangen-Nuremberg, Germany 2 Erlangen Graduate School in Advanced Optical Technologies (SAOT)

2 Outline Introduction Motivation Contributions Respiratory Motion Modeling Internal Motion Model External Motion Model Joint Motion Model Experiments and Results Data Experimental Setup Results Summary and Outlook Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 2

3 Motivation Procedures in thorax/abdomen benefit from respiratory motion compensation (radiation therapy, cardiac interventions,... ) Intraprocedural tracking of dense internal motion is not feasible Approach: Employ external surrogate for internal motion prediction E.g. phase/amplitude information from spirometry or point tracking typically low-dim. surrogates Figure: Respiratory surrogate acquired with a spirometer, from [Hughes et al., 2009]. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 3

4 Motivation Procedures in thorax/abdomen benefit from respiratory motion compensation (radiation therapy, cardiac interventions,... ) Intraprocedural tracking of dense internal motion is not feasible Approach: Employ external surrogate for internal motion prediction E.g. phase/amplitude information from spirometry or point tracking typically low-dim. surrogates Figure: Respiratory surrogate acquired with a spirometer, from [Hughes et al., 2009]. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 3

5 Motivation (cont.) Challenges: Intra-cycle variation in-/exhalation ( hysteresis ) Inter-cycle variation abdominal/thoracic breathing ( reproducibility ) Inter-subject variation More sophisticated, higher-dimensional surrogates Idea: Use the whole body surface Increasing affordability of real-time range imaging (RI) Markerless and non-intrusive acquisition Better correlation [Fayad et al., 2012, Hughes et al., 2009] Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 4

6 Motivation (cont.) Challenges: Intra-cycle variation in-/exhalation ( hysteresis ) Inter-cycle variation abdominal/thoracic breathing ( reproducibility ) Inter-subject variation More sophisticated, higher-dimensional surrogates Idea: Use the whole body surface Increasing affordability of real-time range imaging (RI) Markerless and non-intrusive acquisition Better correlation [Fayad et al., 2012, Hughes et al., 2009] Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 4

7 Contributions Statistical prediction framework making use of the whole surface Dense 3-D motion fields both internally and externally Comprehensive study of several parameters (smoothness, coverage,... ) Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 5

8 TrainingNPhase ApplicationNPhase GroundNTruthNData SequenceNofN3-DNimagesNin differentnrespiratorynstates ApplicationNData Real-timeNacquisitionN ofnthenbodynsurface InternalNMotionNModel NonrigidNRegistration ExternalNMotionNModel BodyNSurfaceNExtraction DeformableNRegistration FullNDeformationNFields DenseNSurfaceNMesh DenseNInternalNMotion DenseNExternalNMotion DenseNExternalNMotion DimensionalityNReduction DimensionalityNReduction Out-of-sampleNExtension InternalNModelNIntrinsics ExternalNModelNIntrinsics InternalNMotionNFeatures ExternalNMotionNFeatures ExternalNMotionNFeatures Ext.-Int.NCorrelation External-InternalNCorrelationNEstimation InternalNMotionNFeatures JointNModelNIntrinsics Reconstruction JointNExternal-InternalNMotionNModel PredictedNInternalNMotion

9 Respiratory Motion Modeling Internal Motion Model External Motion Model Joint Motion Model

10 Nonrigid Volumetric Registration Curvature-regularized framework [Fischer and Modersitzki, 2004], deformation field D minimizes the functional J [D ] = U[R, T ; D ] + αs curv[d ]. R, T (1) Reference and template volumes Distance measure, in our case implied by Demons [Thirion, 1995] Penalizes high curvatures, α controls degree of regularization U αs curv Figure: Respiratory motion field cropped to an internal ROI. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 8

11 Model Intrinsics Perform invertible dimensionality reduction on data matrix: Y = [ d int 0,..., d int t ref 1, d int t ref +1,..., d int N ] R N 3M (2) d int t N M Mean-centered internal deformation fields (vectorized) Number of deformation fields Number of vectors per field Compute first r principal components v i, Q = [v 0, v 1,..., v r 1 ] R 3M r, (3) and projections onto them (mapped, r-dim. points) as features, Y m = YQ (4) Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 9

12 Respiratory Motion Modeling Internal Motion Model External Motion Model Joint Motion Model

13 Surface Extraction Acquisition using RI system combined with MRI is difficult (limited room inside scanner, electromagnetic interference,... ) Extract surface and its motion directly from volumetric data Ray-casting based approach to extraction Cylindrical sensor avoids shortcomings of pinhole camera model Figure: Virtual sensor geometries. Left and middle images adapted from [Wasza et al., 2013]. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 11

14 Model Intrinsics Perform extensible dimensionality reduction on data matrix: X = [ d ext 0,..., d ext t ref 1, d ext t ref +1,..., d ext N ] R N 3V (5) d ext t N V Mean-centered external deformation fields (vectorized) Number of deformation fields Number of surface vertices / motion vectors Generate features X m R N l as before with PCA, or... Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 12

15 Model Intrinsics (cont.)... alternatively, use a nonlinear method, such as: ( ) x1 x k(x 1, x 2 ) = exp σ Image source: [Saul et al., 2006] Isomap [Tenenbaum et al., 2000] Kernel PCA (Gaussian RBF) [Schölkopf et al., 1998] 2 3 Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 13

16 Respiratory Motion Modeling Internal Motion Model External Motion Model Joint Motion Model

17 External-Internal Correlation Correlate feature spaces with a simple regression, complexity should be handled by individual models We use multivariate multilinear regression (cf. [Klinder et al., 2010]): B = argmin B tr [ ( Y m X m B ) ( Y m X m B ) ], (6) where B R r l is the ordinary least squares estimator for Y m = X m B. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 15

18 Experiments and Results

19 Data Synthetic Data Volunteer Data Derived from XCAT phantom [Segars et al., 2010, Maier et al., 2014] Time-resolved cardiac MRI of 6 healthy volunteers [Forman et al., 2013] Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 17

20 Experimental Setup Test phase distant from reference, but not full ex-/inhale Exclude adjacent phases from training to reduce bias Error: Vector magnitudes of difference, { D predicted (x i, y i, z i ) D ground truth (x i, y i, z i ) 2 } i=1...m (7) Magnitudes of internal displacements in test phase: ID Magnitude [mm] ID Magnitude [mm] P ± 1.81 P ± 1.66 P ± 1.45 P ± 1.70 P ± 0.61 P ± 0.17 XCAT 3.51 ± 3.62 Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 18

21 Experimental Setup Test phase distant from reference, but not full ex-/inhale Exclude adjacent phases from training to reduce bias Error: Vector magnitudes of difference, { D predicted (x i, y i, z i ) D ground truth (x i, y i, z i ) 2 } i=1...m (7) Magnitudes of internal displacements in test phase: ID Magnitude [mm] ID Magnitude [mm] P ± 1.81 P ± 1.66 P ± 1.45 P ± 1.70 P ± 0.61 P ± 0.17 XCAT 3.51 ± 3.62 Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 18

22 Experimental Setup Test phase distant from reference, but not full ex-/inhale Exclude adjacent phases from training to reduce bias Error: Vector magnitudes of difference, { D predicted (x i, y i, z i ) D ground truth (x i, y i, z i ) 2 } i=1...m (7) Magnitudes of internal displacements in test phase: ID Magnitude [mm] ID Magnitude [mm] P ± 1.81 P ± 1.66 P ± 1.45 P ± 1.70 P ± 0.61 P ± 0.17 XCAT 3.51 ± 3.62 Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 18

23 Results Surface Coverage Prediction error using PCA for external dimensionality reduction Error [mm] Full Surface Abdominal Area Thoracic Area Random 10% 0 P1 P2 P3 P4 P5 P6 XCAT Data Set Small thoracic/abdominal areas consistently yield equal or worse results With large coverage, uniformly distributed random 10% are sufficient Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 19

24 Results Smoothness of Registration Error [mm] Prediction error using PCA for external dimensionality reduction α = 2500 α = 500 α = 50 0 P1 P2 P3 P4 P5 P6 Data Set Motivation: accuracy anatomical plausibility Weaker regularization results in more complex/noisy motion fields that are harder to predict accurately Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 20

25 Summary and Outlook Prediction framework with promising accuracy Dense internal displacements from dense external surface motion Evaluated effects of feature dim., coverage, smoothness Validated on synthetic 4-D phantom and cardiac MRI of volunteers Future work will need to focus on a clinical setting with actual RI Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 21

26 Summary and Outlook Prediction framework with promising accuracy Dense internal displacements from dense external surface motion Evaluated effects of feature dim., coverage, smoothness Validated on synthetic 4-D phantom and cardiac MRI of volunteers Future work will need to focus on a clinical setting with actual RI Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 21

27 Summary and Outlook Prediction framework with promising accuracy Dense internal displacements from dense external surface motion Evaluated effects of feature dim., coverage, smoothness Validated on synthetic 4-D phantom and cardiac MRI of volunteers Future work will need to focus on a clinical setting with actual RI Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 21

28 Summary and Outlook Prediction framework with promising accuracy Dense internal displacements from dense external surface motion Evaluated effects of feature dim., coverage, smoothness Validated on synthetic 4-D phantom and cardiac MRI of volunteers Future work will need to focus on a clinical setting with actual RI Any questions? Thanks for your attention! Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 21

29 References Bauer, S., Berkels, B., Ettl, S., Arold, O., Hornegger, J., and Rumpf, M. (2012). Marker-less Reconstruction of Dense 4-D Surface Motion Fields using Active Laser Triangulation for Respiratory Motion Management. In Ayache, N., Delingette, H., Golland, P., and Mori, K., editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2012, Lecture Notes in Computer Science, pages Springer. Bengio, Y., Paiement, J.-F., and Vincent, P. (2003). Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering. In Advances in Neural Information Processing Systems. Fayad, H., Pan, T., Pradier, O., and Visvikis, D. (2012). Patient specific respiratory motion modeling using a 3D patient s external surface. Med Phys, 39: Fischer, B. and Modersitzki, J. (2004). A unified approach to fast image registration and a new curvature based registration technique. Linear Algebra and its Applications, 380: Forman, C., Grimm, R., Hutter, J. M., Maier, A., Hornegger, J., and Zenge, M. O. (2013). Free-breathing whole-heart coronary MRA: Motion compensation integrated into 3D cartesian compressed sensing reconstruction. In Mori, K., Sakuma, I., Sato, Y., Barillot, C., and Navab, N., editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, Lecture Notes in Computer Science, pages Springer. He, T., Xue, Z., Xie, W., and Wong, S. T. (2010). Online 4-D CT estimation for patient-specific respiratory motion based on real-time breathing signals. In Jiang, T., Navab, N., Pluim, J. P., and Viergever, M. A., editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2010, Lecture Notes in Computer Science, pages Springer. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 22

30 References (cont.) Hughes, S., McClelland, J., Tarte, S., Lawrence, D., Ahmad, S., Hawkes, D., and Landau, D. (2009). Assessment of two novel ventilatory surrogates for use in the delivery of gated/tracked radiotherapy for non-small cell lung cancer. Radiotherapy and Oncology, 91: Klinder, T., Lorenz, C., and Ostermann, J. (2010). Prediction framework for statistical respiratory motion modeling. In International conference on Medical image computing and computer-assisted intervention, pages Springer. Maier, A., Taubmann, O., Wetzl, J., Wasza, J., Forman, C., Fischer, P., Hornegger, J., and Fahrig, R. (2014). Fast Interpolation of Dense Motion Fields from Synthetic Phantoms. In Deserno, T., Handels, H., Meinzer, H.-P., and Tolxdorff, T., editors, Bildverarbeitung für die Medizin 2014, pages , Berlin. Saul, L. K., Weinberger, K. Q., Ham, J. H., Sha, F., and Lee, D. D. (2006). Spectral methods for dimensionality reduction. Semisupervised Learning. MIT Press: Cambridge, MA. Schaerer, J., Fassi, A., Riboldi, M., Cerveri, P., Baroni, G., and Sarrut, D. (2012). Multi-dimensional respiratory motion tracking from markerless optical surface imaging based on deformable mesh registration. Phys Med Biol, 57: Schölkopf, B., Smola, A., and Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput, 10: Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 23

31 References (cont.) Segars, W. P., Sturgeon, G., Mendonca, S., Grimes, J., and Tsui, B. M. W. (2010). 4D XCAT phantom for multimodality imaging research. Med Phys, 37: Tenenbaum, J. B., de Silva, V., and Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290: Thirion, J.-P. (1995). Fast Non-Rigid Matching of 3D Medical Images. Technical Report RR-2547, INRIA. Wasza, J., Bauer, S., and Hornegger, J. (2012). Real-time motion compensated patient positioning and non-rigid deformation estimation using 4-D shape priors. MICCAI: International Conference on Medical Image Computing and Computer-Assisted Intervention, 15(Pt 2): Wasza, J., Bauer, S., and Hornegger, J. (2013). Real-time respiratory motion analysis using manifold ray casting of volumetrically fused multi-view range imaging. In Mori, K., Sakuma, I., Sato, Y., Barillot, C., and Navab, N., editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, Lecture Notes in Computer Science, pages Springer. Williams, C. and Seeger, M. (2001). Using the nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13, pages MIT Press. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 24

32 Related Work [Fayad et al., 2012] [Hughes et al., 2009] Heuristic partitioning of surface depth maps Simulate 1-D surrogates, e.g. the estimated lung volume [Schaerer et al., 2012], Actual surface motion tracking, [Bauer et al., 2012] but no internal prediction [Klinder et al., 2010] [He et al., 2010] Generic prediction framework, not evaluated on surface surrogates External fiducials, simulate respiratory phases from 3-D CT of unseen subject Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 25

33 Nonrigid Volumetric Registration Details Symmetric Demons forces: v = 2(R T D)( R + T D ) R + T D 2 2 (8) Cf. derivative of SSD: v = 2(R T D ) T D Curvature regularization: S curv [D] = 1 2 d ( D l ) 2 dx (9) Ω l=1 Gâteaux derivative of Eq. (9) is A curv [D] = 2 D Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 26

34 Internal Model Details For faster convergence, initialize optimization with result of nearest phase closer to reference Deformation fields are 2 downsampled and cropped to internal ROI The d tint are few, but high-dimensional: Eigendecompose Gram matrix K = YY > instead of ΣY = Y >Y Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 27

35 Surface Extraction (cont.) Find air-skin transition with heuristic thresholding Postprocessing (smoothing, closing holes) required due to fat saturation in cardiac MRI sequence Interpolate deformation fields at vertex positions to obtain motion Figure: Extracted body surface before (left) and after (middle, right with motion vectors) postprocessing. Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 28

36 Multilinear Regression Closed form solution: B = Y m X m ( X m X m ) 1 X m ( X m X m ) 1 = ( X m ) is the Moore-Penrose pseudoinverse. Tikhonov regularization / ridge regression: If numerically unstable, invert X m X m + γi instead of X m X m. (10) Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 29

37 Internal Motion Prediction Elastic registration of instantaneous to reference surface Nonrigid ICP variant [Schaerer et al., 2012] Model-based ICP with 4-D shape priors [Wasza et al., 2012] Active Laser Triangulation [Bauer et al., 2012] Out-of-sample extension of external model Trivial for PCA: subtract mean and project onto eigenvectors Approximate solution needed for kernel methods (Nyström method, [Williams and Seeger, 2001, Bengio et al., 2003]) Perform regression, reconstruct with principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 30

38 Internal Motion Prediction Elastic registration of instantaneous to reference surface Nonrigid ICP variant [Schaerer et al., 2012] Model-based ICP with 4-D shape priors [Wasza et al., 2012] Active Laser Triangulation [Bauer et al., 2012] Out-of-sample extension of external model Trivial for PCA: subtract mean and project onto eigenvectors Approximate solution needed for kernel methods (Nyström method, [Williams and Seeger, 2001, Bengio et al., 2003]) Perform regression, reconstruct with principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 30

39 Internal Motion Prediction Elastic registration of instantaneous to reference surface Nonrigid ICP variant [Schaerer et al., 2012] Model-based ICP with 4-D shape priors [Wasza et al., 2012] Active Laser Triangulation [Bauer et al., 2012] Out-of-sample extension of external model Trivial for PCA: subtract mean and project onto eigenvectors Approximate solution needed for kernel methods (Nyström method, [Williams and Seeger, 2001, Bengio et al., 2003]) Perform regression, reconstruct with principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 30

40 Data Sizes ID Phases Size [voxels] Spacing [mm] Int. ROI [voxels] Surface Vert. P ,368 P ,321 P ,628 P ,759 P ,493 P ,521 XCAT ,566 Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 31

41 Default Method Parameters l = r = 4 Kernel PCA, σ = 40 for MRI and σ = 400 for XCAT data Isomap, k = 4 (k = 7 if graph not fully connected) α = 2500 (cf. Eq. (1)) and full surface coverage Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 32

42 Cumulative Relative Eigenvalues For P1 with α = 2500: 1 Data variance explained by the n most important principal components Fraction of variance explained Target Motion Surrogate Number of principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 33

43 Cumulative Relative Eigenvalues For P1 with α = 500: 1 Data variance explained by the n most important principal components Fraction of variance explained Target Motion Surrogate Number of principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 33

44 Cumulative Relative Eigenvalues For P1 with α = 50: 1 Data variance explained by the n most important principal components 0.95 Fraction of variance explained Target Motion Surrogate Number of principal components Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 33

45 Errors Dimensionality Reduction Options Prediction error using PCA for external dimensionality reduction Error [mm] Dim = 1 Dim = 2 Dim = 3 Dim = 4 0 P1 P2 P3 P4 P5 P6 XCAT Data Set Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 34

46 Errors Dimensionality Reduction Options Prediction error using Isomap for external dimensionality reduction Error [mm] Dim = 1 Dim = 2 Dim = 3 Dim = P1 P2 P3 P4 P5 P6 XCAT Data Set Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 34

47 Errors Dimensionality Reduction Options Error [mm] Prediction error using KernelPCA for external dimensionality reduction Dim = 1 Dim = 2 Dim = 3 Dim = 4 0 P1 P2 P3 P4 P5 P6 XCAT Data Set Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 34

48 Results Dimensionality Reduction Options PCA outperforms nonlinear methods w. r. t. prediction error, improves with increasing feature space dimensionality Isomap and Kernel PCA not quite as accurate, only most significant feature dimension is meaningful Best result in MRI: max. error of 0.42 mm, given a median magnitude of 3.91 mm in the ground truth deformation field Phantom is predicted almost perfectly in any configuration Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 35

49 Results Runtimes Training phase (not time-critical): Registration (C++, using ITK): one to several hours for a whole sequence Surface extraction (C++, using RITK): few ms (+ manual param. choice) Mesh processing (ParaView, Meshlab): few minutes (automatic + manual) Model training (MATLAB, using DR toolbox): few seconds Application phase (usually time-critical): Deformable mesh registration: e.g. 40 ms on GPU with [Wasza et al., 2013] (untested as no RI data were used in our experiments) Prediction using PCA for both models (CUDA, using cublas): 5-10 ms Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 36

50 Results Runtimes Training phase (not time-critical): Registration (C++, using ITK): one to several hours for a whole sequence Surface extraction (C++, using RITK): few ms (+ manual param. choice) Mesh processing (ParaView, Meshlab): few minutes (automatic + manual) Model training (MATLAB, using DR toolbox): few seconds Application phase (usually time-critical): Deformable mesh registration: e.g. 40 ms on GPU with [Wasza et al., 2013] (untested as no RI data were used in our experiments) Prediction using PCA for both models (CUDA, using cublas): 5-10 ms Jun 15, 2014 Oliver Taubmann Pattern Recognition Lab, FAU Prediction of Internal Deformations From Surface Data 36