Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery. Simultaneous Localization and Mapping for Minimally Invasive Surgery

Transcription

1 Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery Simultaneous Localization and Mapping for Minimally Invasive Surgery

2 Introduction University of Bristol using particle filters to track football players from a single static camera. Imperial College -SimultanousLocalization and Mapping (SLAM) with by Guang-Zhong Yang and Andrew Davison. Siemens Corporate Research Valve modelling

3 Current Situation

4 Why is Camera Motion Important

5 Estimating Camera Motion & Building a 3D Map

6 Problem: Noisy Sensor Measurements 1 mm = Many Pixels

7 Noise in the Camera Image

8 Noisy System How to we get accurate models and estimates from noisy measurements? Structure From Motion V Simultaneous Localization And Mapping (SLAM)

9 Structure From Motion and Simultaneous Localization And Mapping Structure From Motion

10 Structure From Motion and Simultaneous Localization And Mapping SLAM Filtering Approach

11 Simultaneous Localization And Mapping Extended Kalman Filter Unscented Kalman Filter Particle Filter Information Filter FastSLAM RatSLAM Submapping Hybrid Systems Dense Sparse Large Scale Augmented Reality Robotic Navigation Wearable Computers Autonomous Vehicles/ Helicopters / Planes / Cars / boats / submarines Compute Games Games Controllers Smart Phones Space Exploration Surgery

12 EKF SLAM [1] -State Extended Kalman Filter SLAM Davison et al [1] MonoSLAM: Real-Time Single Camera SLAM Andrew Davison, Ian Reid, Nicholas Molton and Oliver Strasse. IEEE PAMI June 2007 (vol. 29 no. 6) pp

13 EKF SLAM Uncertainty

14 EKF SLAM - Prediction

15 EKF SLAM - Prediction

16 EKF SLAM - Measurement

17 EKF SLAM Active Search

18 EKF SLAM -Update

19 EKF SLAM -Update

20 EKF SLAM Adding new features

21 EKF SLAM Adding new features 1. Feature/Map initialisation 2. Predict motion 3. Feature matching 4. Update Pose and map [1]

22 SLAM for MIS Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. Peter Mountney, Danail Stoyanov, Andrew Davison, Guang-Zhong Yang. MICCAI (1) 2006: pp

23 SLAM for MIS - Validation CT SLAM A Stereoscopic Fibroscope for Camera Motion and 3D Depth Recovery During Minimally Invasive Surgery. David Noonan, Peter Mountney, Daniel Elson, Ara Darzi and Guang-Zhong Yang. In proc ICRA 2009, pp

24 Dynamic View Expansion Dynamic View Expansion for Minimally Invasive Surgery using Simultaneous Localization And Mapping. Peter Mountney and Guang- Zhong Yang. In Proc EMBC 2009: pp

25 Dense Stereo Real-time Stereo Reconstruction in Robotic Assisted Minimally Invasive Surgery. Danail Stoyanov, Marco Visentini Scarzanella, Philip Pratt and Guang-Zhong Yang. In Proc MICCAI 2010

26 SLAM Dense Mapping Dense Surface Reconstruction for Enhanced Navigation in Minimally Invasive Surgery. Johannes Totz, Peter Mountney, Danail Stoyanov, and Guang-Zhong Yang. Accepted to MICCAI 2011 z

27 Optical Biopsy Mapping Mauna Kea -

28 Optical Biopsy Mapping Optical Biopsy Mapping for Minimally Invasive Cancer Screening. Peter Mountney, Stamatia Giannarou, Daniel Elson and Guang- Zhong Yang. In proc MICCAI(1), 2009, pp

29 Optical Biopsy Mapping

30 Motion Compensated SLAM in Dynamic Environments

31 Motion Compensated SLAM in Dynamic Environments Motion Compensated SLAM for Image Guided Surgery. Peter Mountney and Guang-Zhong Yang. In Proc MICCAI 2010

32 Motion Compensated SLAM in Dynamic Environments

33 Motion Compensated SLAM in Dynamic Environments 8 x PCA

34 Motion Compensated SLAM in Dynamic Environments

35 Motion Compensated SLAM in Dynamic Environments Respiration Model EKF state EKF Prediction Model EKF Measurement Model

36 Motion Compensated SLAM in Dynamic Environments

37 Motion Compensated SLAM in Dynamic Environments

38 Motion Compensated SLAM in Dynamic Environments

39 Future Work in this Area PTAM DTAM Non rigid structure from motion Incorporation of sensors Incorporating pre operative models Parallel Approaches Biomechanical Models

40 Questions Thank you for listening