Automated Kidney Segmentation In 3D Ultrasound Imagery, and its Application in Computer-assisted Trauma Diagnosis

Transcription

1 Automated Kidney Segmentation In 3D Ultrasound Imagery, and its Application in Computer-assisted Trauma Diagnosis Final PhD Oral Examination - Presentation Slides September 1 st, 2016 Mahdi Marsousi Supervisor: Prof. Konstantinos N. Plataniotis

2 Slide #1 Introduction Abdominal trauma: injuries to abdomen, either blunt or penetrating, resulting in severe blood loss. Importance: massive internal bleeding quickly threatens patient s life. Cure: rapid diagnosis and surgery [1] [2] [1] [2]

3 Slide #2 Diagnosis by 3D ultrasound (3DUS) For hemodynamically unstable patients, ultrasound imagery is the preferred imaging modality, because it is portable. This is called: Focused Assessment with Sonography for Trauma FAST. 2D ultrasound is the popular diagnostic tool. 3DUS imagery is key to design computer-assisted diagnosis, because: a) detecting and localizing internal organs in 3D space is possible by 3DUS; b) 3DUS facilitates measuring the volume of internal bleeding.

4 Slide #3 FAST exam Six major abdominal views associated with FAST exam. For computer-assisted trauma diagnosis, we focus on the Morison s pouch view (2), Because it is the most sensitive view to abdominal bleeding. An abdominal bleeding usually places between the right kidney and liver [1]. Morison s pouch view shows the right kidney and a portion of liver [1]. The kidney has a unique shape among internal organs. :Focus of this presentation By segmenting the kidney shape, we only look for an abdominal bleeding around the top surface of the kidney shape. [1] Heller et al. Ultrasound use in trauma: the fast exam. Academic Emergency Medicine, 14(6): , [2] Ingeman et al. Emergency physician use of ultrasonography in blunt abdominal trauma. Academic Emergency Medicine, 3(10): , 1996.

5 Slide #4 Motivation Problem definition: Because of limited surgical resources in massive causalities, only true trauma patients should be identified and transported for rapid surgery. This importance necessitates the use of paramedics to conduct FAST exam. However, paramedics are not capable to do FAST exam with supervision. Solution: a computer-assisted tool can be used to help paramedics to conduct trauma diagnosis.

6 Contributions Slide #5

7 3DUS dataset[1] With-kidney image: taken from Morison s pouch view Without-kidney image: taken from other views. Training set: 6 with-kidney and 6 without-kidney images. Evaluation set: 15 with-kidney and 15 without-kidney from healthy volunteers, and 8 with-kidney images from abnormal patients. Images of abnormal patients represent morphological changes to the kidney shape. [1] G. Sakas, Phase I: Ultrasound Imaging Data Collection and Applications," MedCom GmbH, Tech. Rep., September Slide #6

8 Contribution 1: Kidney detection

9 Overview on kidney detection Kidney detection: searching within 3D image domain to answer: whether kidney shape exists in a 3DUS image? if yes, what is its alignment? State-of-the-art: Name Methodology Advantage Disadvantage Noll et al. [1] - applies volume enhancement - uses redial rays to find kidney shape mass-center robust against speckle & low-contrast intensity may wrongly detect other structures instead of kidney Proposed solutions: [1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc. Translational Research in Medical Imag. Springer, 2014, pp Slide #7

10 Slide #8 Kidney shape model Definition: mathematical representation of kidney shape to add shape prior in kidney detection Proposed shape modeling: complex-valued implicit shape model (CVISM) Ψ ԦX = ψ PS ԦX + ψ KC ԦX i ψ RM ԦX, Real values are used to model bright regions: Imaginary values are used to model dark region: Kidney Capsule Renal Medulla KC: Kidney Capsule PS: Pyelocalyceal System RM: Renal Medulla Pyelocalyceal System ψ ps, ψ KC, and ψ RM are real-positive functions, defining voxel s membership to Pyelocalyceal System, Kidney Capsule, and Renal Medulla, respectively. Generating regions: using ground truth data of training set of with-kidney images.

11 Slide #9 Shape-to-volume registration Objective: registering kidney CVISM on 3DUS images, to detect kidney shape and estimate its alignment. Challenges: Partial occlusion of kidney shape Non-kidney structures with similar appearance to the kidney shape Components: Global deformation, Similarity metric, optimization algorithm. Global deformation: similarity transformation (shape-preserving) ԦY = ST Ԧpst,1 ԦX + Ԧp st,2 Orientations & scaling Ԧp st,1 = θ x,θ y,θ z,s T Translations Ԧp st,2 = t x,t y,t z T CVISM domain Image domain

12 RCNCC similarity metric Proposed similarity metric: Regularized complex normalized cross-correlation (RCNCC) Formulation: Γ Ԧp st,1, Ԧp st,2 = max Ԧp st,2 Ωsub V 1 Λ Ԧp st,1, Ԧp st,2 max 0,R Σ I Ԧp st,1, Ԧp st,2 Σ R II Ԧp st,1 max 0,I Σ I Ԧp, ԦX,V Σ I II Ԧp st,1 Regularization Term Robustness against Partial kidney shape occlusion Checking similarity of image data with Kidney Capsule and Pyelocalyceal System Checking similarity of image data with Renal Medulla Considering multi-regional structure of the kidney shape, Λ Ԧp st,1, Ԧp st,2 1 increases as kidney CVISM aligns out of the ultrasound field of view. max a, b : selects maximum of a and b. max s are used to avoid multiplication of two negative values in RCNCC. R{a} and I{b} extract real part of a and imaginary part of b. Slide #10

13 Slide #11 Registration problem Objective: Finding Ԧp st,1 Optimization problem: and Ԧp st,2 which provides a maximum RCNCC for input image. Γ, Ԧp st,1, Ԧp st,2 = max Ԧp st,1 Γ Ԧp st,1, Ԧp st,2 Optimization algorithm: initialization: searching for best seed point from a set of seed points to initialize registration. iterative improvement: iteratively updating registration parameters using Gradient Descent. Figure shows RCNCC metric versus θ x and θ z. Yellow regions correspond to desirable registration solutions. The gray stars are seed point. Red star is selected seed point. Black arrows show iterations of Gradient descent.

14 Slide #12 Processing pipeline of kidney detection Γ th is threshold on similarity metric to decide whether the kidney shape exists or not. It is obtained using training set, as Γ th = 3.5.

15 Slide #13 Experimentation Objective: to evaluate shape-based and atlas-based kidney detection, compared to Noll et al. [1]. Dataset: 15 with-kidney and 15 without-kidney images of healthy volunteers 8 with-kidney images abnormal patients Comparison metrics: accuracy: Acc KD = 100% sensitivity: Sens KD = N TP N TP +N FN, specificity: Spec KD = N TN N TN +N FP. N TP +N TN N TP +N FN +N TN +N FP, N TP, N TN, N FP, and N FN are numbers of true positive, true negative, false positive, false negative detections. [1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc. Translational Research in Medical Imag. Springer, 2014, pp

16 Kidney detection accuracy Using with-kidney and without-kidney images of healthy volunteers: Method N TP N TN N FP N FN Accuracy (%) Sensitivity Specificity Noll et al Shape-based kidney detection Atlas-based kidney detection Shape-based and atlas-based provide higher accuracy, compared to Noll et al., because: 1) higher specificity: involving multi-structural regions of kidney shape, 2) higher sensitivity: regularization factor provides robustness against kidney shape s occlusion. Atlas-based shows higher accuracy than shape-based because of using texture information. Using actual ultrasound volumes of abnormal patients: Accuracy is reduced, because: different device settings from healthy volunteers, Method N TP N FN Accuracy (%) Noll et al. [1] Shape-based Atlas-based Morphological changes of RUQ view in abnormal patients. Slide #14

17 Slide #15 Examples of kidney detection Blue region: Renal Medulla Red region: Kidney Capsule/Pyelocalyceal System

18 Contribution 2: Automated kidney segmentation

19 Overview on automated kidney segmentation Problem definition: automatically segmenting the kidney shape in 3DUS images. State-of-the-art: Name Methodology Advantage Disadvantage Noll et al. [1] MRF-AC [2] Radial ray trace + fast marching + edge-based levelset 2D active contour + Markov random field (MRF) + 3D reconstruction - Robust against speckle and low-contrast intensity profile - Automated initialization - Not robust against kidney deformation - High-computational cost - Low computational cost - Manual initialization - Discontinuity along z-axis Proposed method: complex-valued regional level-set with shape prior (CVRLS-SP) Complex-valued structure is used to add multi-regional structure into segmentation. Adding multi-regional structure is important because it improves segmentation specificity. [1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc. Translational Research in Medical Imag. Springer, 2014, pp [2] Martn-Fernndez et al., An approach for contour detection of human kidneys from ultrasound images using markov random fields and active contours, Med. Image Anal., 1 23, Slide #16

20 Slide #17 CVRLS-SP representation Mathematical representation of regions in CVRLS-SP: br : X pyelocalyceal system or kidney capsule, if R φ X > 0, dr : X renal medulla, if I φ X > 0, bg : X background, otherwise. where φ is level-set function. Prior shape: (φ s ) is generated by aligning CVISM on detected kidney shape: φ s X = 2 Ψ ST Ԧpst,1 X + Ԧp st,2 > i, X Ω V where ST Ԧpst,1 X + Ԧp st,2 places kidney CVISM, Ψ, on detected kidney shape. level-set function, φ, is initialized by prior shape: φ X; t = 0 = φ s X, t is iteration.

21 Slide #18 CVRLS-SP evolution Conventional regional level-set with shape prior defines energy function as [1], F c, φ, φ s AT Ԧpaf, V = λ F SP φ, φ s AT Ԧpaf + μ F int φ + γ F ext (c, φ, V) F SP φ, φ s AT Ԧpaf : shape prior energy, F int φ : internal energy, controlling smoothness. F ext (c, φ, V): external energy, pushing level-set toward region of interest. c: average intensity level of region of interest, AT Ԧpaf : affine transformation. λ, μ, γ: Lagrange multipliers Contribution: CVRLS-SP s energy functional is defined as follows, F c br, c dr, φ, φ s AT Ԧpaf, V = λ F br SP R{φ}, R{φ s AT Ԧpaf } + λ F dr SP I{φ}, I{φ s AT Ԧpaf } +μ F int φ + γ F br ext (c br, R{φ}, V) + γ F dr ext (c dr, I{φ}, V) Multi-regional segmentation is added By dividing both F SP and F ext into br and dr terms. [1] Chan and Zhu. Level set based shape prior segmentation. IEEE CVPR, vol 2, pp ,

22 Slide #19 CVRLS-SP segmentation procedure Initialization: Creating prior shape and level-set initialization Iterative evolution: φ is updated by reducing F c br, c dr, φ, φ s AT Ԧpaf, V, using Euler-Lagrange equation. convergence criteria: ΣΔφ t = φ X, t φ X, t 1 2 dxdydz

23 Experimentation Objective: to evaluate proposed kidney segmentation compared to Noll et al. [1] and MRF-AC. [2] Dataset: evaluation set of actual ultrasound volumes Comparison metrics: Dice s similarity coefficient (DSC)= Accuracy metric (ACC)= Sensitivity measure (SENS)= TP TP+FN, Specificity measure (SPEC)= TN TN+FP. 2TP, 2TP+FN+FP TP+TN TP+TN+FP+FN 100%, Parameter setting: Name λ μ γ N itr t max ε Value [1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc. Translational Research in Medical Imaging, Springer, 2014, pp [2] Martin-Fernandez et al., An approach for contour detection of human kidneys from ultrasound images using Markov random fields and active contours, Medical Image Analysis, 1 23, Slide #20

24 Dice's Coefficient (DSC) Slide #21 Kidney segmentation accuracy μ: mean, σ:standard deviation DSC ACC SENS SPEC Method μ σ μ σ μ σ μ σ CVRLS-SP Noll et al MRF-AC Specificity of CVRLS-SP is higher because: capability of segmenting multi-regional structure, using shape prior. Box-plot: shows statistical variation of DSC of the methods: CVRLS-SP MRFAC Noll et al.

25 Examples of kidney segmentation Original image Detected kidney Segmented kidney regions Second and third columns: Blue and red regions indicate Renal Medulla and Pyelocalyceal System/Kidney Capsule, respectively. Fourth column: Yellow, red, and green colors are TP, FP, and FN regions. Comparing automated segmentation vs. ground-truth 3D view Slide #22

26 Slide #23 Conclusions & Future Work Conclusions: We discussed the importance of automated kidney segmentation in designing a computerassisted trauma diagnosis. Shape-based and atlas-based kidney detection methods were introduced. Automated kidney segmentation was introduced The accuracy of kidney detection and segmentation methods were evaluated and compared to state-of-the-art Experimental results validate the superiority of the proposed methods. Future work: Investigating the effect of motion artifact and interference using a bigger dataset Using the automated kidney segmentation to automatically detect free fluids.

27 Publications Journal Papers: [1] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, An Automated Approach for Kidney Segmentation in Three-Dimensional Ultrasound Images, IEEE Journal of Biomedical and Health Informatics, DOI: /JBHI , (Date of publication: June 13 th 2016) [2] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, Computer-Assisted 3D Ultrasound Probe Placement for Emergency Healthcare Applications, IEEE Transactions on Industrial Informatics, DOI: /TII , (Date of publication: May 18 th 2016) [3] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, Kidney Detection in 3D Ultrasound Imagery Based on Shape and Texture Priors, to be submitted at IEEE Transaction on Biomedical Engineering (T-BME) on September Conference Papers: [4] M. Marsousi, X. Lee, and K. Plataniotis, Shape-included Label-Consistent Discriminative Dictionary Learning: An Approach to Detect and Segment Multi-Class Objects in Image, IEEE international conference on Image Processing (ICIP), 2016, accepted. [5] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. Atlas-based segmentation of abdominal organs in 3D ultrasound, and its application in automated kidney segmentation., IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp , [6] M. Marsousi and K. Plataniotis, Binomial classification based on DLENE features in sparse representation: Application in kidney detection in 3D ultrasound. IEEE International Conference in Acoustics, Speech and Signal Processing (ICASSP), pp , [7] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, Shape-based kidney detection and segmentation in three-dimensional abdominal ultrasound images. IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp , [8] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. A multi-steps segmentation approach for 3D ultrasound images using the combination of 3D- Snake and Level-Set. IEEE International Conference in Digital Signal Processing (DSP), pp. 1-4, US patent: [9] S. Stergiopoulos, P. Shek, K. Plataniotis, and M. Marsousi. Computer aided diagnosis for detecting abdominal bleeding with 3D ultrasound imaging. U.S. Patent Application 14/159,744, filed January 21, 2014.