Vienna Computer Vision Meetup 27.01.2016 sponsored by Medical Computer Vision Markus Holzer markus@radiology-explorer.com www.radiology-explorer.com www.meduniwien.ac.at www.cir.meduniwien.ac.at
Overview About Me Medical Images 3 Examples: - A Standard Approach with Example on X-Ray Images Fetal MR: Brain Development CT/MR: 3D Image Search
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.hsc.stonybrook.edu/gyn-atlas/images/dscn3451.jpg
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.radiologyinfo.org/gallery-items/images/chest-xray.jpg
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.northstarradiology.com/images/lung-mip.jpg
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.brainfacts.org/~/media/brainfacts/article% 20Multimedia/About% 20Neuroscience/Technologies/MRI_blackandwhite.ashx
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: https://upload.wikimedia. org/wikipedia/commons/9/9a/default_mode_network-wrnmmc. jpg
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.oct-optovue.com/images/oct-normal.jpg
Medical Images Digital Pathology X-Ray CT - Computed Tomography MRI - Magnetic Resonance Imaging fmri - functional MRI OCT - Optical Coherence Tomography PET - Positron Emission Tomography Source: http://www.rsc.org/images/news-page10-300_tcm18-92717.jpg
Overview About Me Medical Images 3 Examples: - A Standard Approach with Example on X-Ray Images Fetal MR: Brain Development CT/MR: 3D Image Search
A Common Computer Vision approach: Input: Images Annotations Methods Model (e.g. shape of a bone) Classifier (based on extracted features) Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: Dataset: 50 images (30 for training, 20 for testing) X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: Dataset: 64 annotations per image X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: Shape Variations X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: Shape Variations (first 3) X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output 1st dim. (highest variation) Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015. 2nd dim. 3rd dim.
X-Ray Example: Bone Detection Feature Extraction: Examples Input: X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output gray values Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015. x-coordinate gradient Haar-like
X-Ray Example: Bone Detection Input: Classifier: Random Forest - training X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015. Mask Features
X-Ray Example: Bone Detection Input: Classifier: Random Forest X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output gray values gradient Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015. Haar-like #1 x-coordinate
X-Ray Example: Bone Detection Input: Classifier: Random Forest - predict X-Ray Images Annotations Methods Shape Model Feature Extraction Classifier Output Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Detection Input: X-Ray Images Annotations Methods Classifier: Random Forest - predict -> = + Shape Model Feature Extraction Classifier Output Classifier Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015. Model Result
X-Ray Example: Bone Detection Results Medizinische Bildverarbeitung UE, Vienna University of Technology, 2013-2015.
X-Ray Example: Bone Quantification Quantification G. Langs, P. Peloschek, H. Bischof and F. Kainberger. Automatic Quantification of Joint Space Narrowing and Erosions in Rheumatoid Arthritis. IEEE TMI, 28(1):151-164, Jan 2009. Example:
Overview About Me Medical Images 3 Examples: - A Standard Approach with Example on X-Ray Images Fetal MR: Brain Development CT/MR: 3D Image Search
Fetal-MR Example: Fetal Brain Development Input: Fetal-MR Volumes Methods Volume Creation Registration / Atlas Segmentation Extract Surface E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Fetal Brain Development Input: Example: Fetal-MR Volumes Methods Volume Creation Registration / Atlas Segmentation Extract Surface E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Fetal Brain Development Input: Example: Fetal-MR Volumes Methods Volume Creation Registration / Atlas Segmentation Extract Surface E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Fetal Brain Development Input: Fetal-MR Volumes Methods Volume Creation Registration / Atlas Segmentation / Surface Ext. Visualization E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Fetal Brain Development Input: Fetal-MR Volumes Methods Volume Creation Registration / Atlas Segmentation / Surface Ext. Visualization E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Fetal Brain Development Input: Fetal-MR Volumes Methods Video Link Volume Creation Registration / Atlas Segmentation / Surface Ext. Visualization E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Region Development Region Map of Adult Brain: How do they develop in the fetus? E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Fetal-MR Example: Region Development Region Map of Adult Brain: H. Lombart, L. Grady, JR. Polimeni, F. Cheriet. FOCUSR: feature oriented correspondence using spectral regularization -- a method for precise surface matching. IEEE TPAMI, 35(9):2143-60, Sep 2013.
Fetal-MR Example: Fetal Brain Development Visualization: Video Link E. Schwartz. Fetal-MR Analysis. www.cir.meduniwien.ac.at/team/schwartz
Overview About Me Medical Images 3 Examples: - A Standard Approach with Example on X-Ray Images Fetal MR: Brain Development CT/MR: 3D Image Search
CT Example: 3D Image Search - Khresmoi Input: Lung CT Volumes Methods www.khresmoi.eu Registration/Localization Feature Extraction Retrieval
CT Example: 3D Image Search - Khresmoi Input: CT Volumes Methods www.khresmoi.eu Registration/Localization Feature Extraction Retrieval
CT Example: 3D Image Search - Khresmoi Input: Over-Segmentation: CT Volumes Methods Registration/Localization Feature Extraction Retrieval M. Holzer, R. Donner. Over-Segmentation of 3D Medical Image Volumes based on Monogenic Cues. Proceedings of the 19th CVWW, pp 35-42, 2014.
CT Example: 3D Image Search - Khresmoi
CT Example:
Summary About Me Medical Images 3 Examples: - A Standard Approach with Example on X-Ray Images Fetal MR: Brain Development CT/MR: 3D Image Search
Unlocking the potential of medical image data image analysis machine learning office@radiology-explorer.eu www.radiology-explorer.eu consulting
Vienna Computer Vision Meetup 27.01.2016 sponsored by Medical Computer Vision Markus Holzer markus@radiology-explorer.com www.radiology-explorer.com www.meduniwien.ac.at www.cir.meduniwien.ac.at