Hayit Greenspan, PhD. Dept of Biomedical Engineering Faculty of Engineering Tel-Aviv University

Transcription

1 ICRI-CI 2017 Retreat, May 9,2017 Deep Learning in Medical Imaging: the Data Challenge Hayit Greenspan, PhD. Dept of Biomedical Engineering Faculty of Engineering Tel-Aviv University

2 Imaging 25 second CT scans produce up to 2000 images PET/CT requires review of up to 6000 images Breast US can create 5000 images 5 Billion studies per year worldwide, and growing

3 Pain Points Limited Time to Review Ever increasing Number of Images

4 Radiologist Workflow ER MD Yes High Priority Queue In-Patient MD STAT? PACS Delay No Out-Patient MD Low Priority Queue Key Pain Points: No time to read Missed findings

5 Can we use Deep Learning for Medical Image Analysis? Enlarged Heart? Pleural- Effusion?

6 Deep Learning in Medical Image Analysis - Today IEEE TMI Special Issue on Deep Learning, Co-editors: Greenspan, van Ginneken, Summers, May 2016 Guest Editorial: Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique, IEEE Transactions on Medical Imaging, Volume: 35, Issue: 5, , May 2016 Elsevier BOOK: Deep Learning for Medical Imaging, February 2017 Co-editors: Zhou, Greenspan, Shen Conferences: Leading interest in all medical imaging conferences Many Startups emerging

7 The Data Challenge Difficult to find & extract from archives Pathologies even more difficult Need expert labeling Long tedious process Noisy labels

8 Solving the Data Challenge Transfer Learning Data Augmentation Know your Context

9 Solving the Data Challenge Examples I. Chest Pathology Identification and Categorization X-ray images Image-level labeling Transfer Learning II. Brain lesions Detection and Segmentation MRI images Patch-based labeling Classical + Deep Learning III. Liver lesions - Detection and Segmentation CT images Fully convolutional networks

10 I. CHEST PATHOLOGY IDENTIFICATION & CATEGORIZATION Healthy Enlarged heart; Pleural Effusion Enlarged Cardiomegaly Mediastinum X-ray Data Via Transfer Learning

11 Chest Radiographs? X-ray: The most common exam in radiology with 2B procedures/year (CT: 500M) No. of Examinations (2012) 28,689 66,968 50, , ,653 Modality MR CT US CR CR CHEST Courtesy: Sheba

12 System for Chest Pathology Identification and Retrieval in Large Radiology Archives Collaboration with radiologists from Sheba medical center; data and feedback from the experts. Collection of a large number of Chest Radiographs --continuous augmentation of the dataset --labeled by medical experts --sequential data coming from real clinical setting

13 BIG DATA

14 Chest Pathology Detection and Categorization Healthy / Non-healthy Liquid in the lungs? Enlarged heart? Automated System

15 Methodology: Transfer Learning Pathology - No Pathology - Yes Improvement of learning in a new task through the transfer of knowledge from a related task that has already been learned Deep NN (learned on ImageNet)

16 Transfer Learning Utilize image representations learned with CNNs on large-scale annotated datasets in other visual recognition tasks with limited amount of training data Oquab et al., Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks, Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. IEEE, 2014.

17 Transfer Learning

18 System Overview Feature extraction SVM Classifier for Cardiomegaly Features Vector SVM Classifier for Mediastinum... Multiple pathologies SVM Classifier for Pleural Effusion Multiple labels for a case Left Effusion Cardiomegaly Mediastinum

19 Results (1000+) Enlarged Heart Cardiomegaly Enlarged Mediastinum Right Pleural Fluid Left Pleural Fluid Negative Positive AUC Spec. at ~95% Sens. Spec. at ~90% Sens. Sens. at ~90% Spec

20 Results True Positive Rate=Sensitivity GIST Deep(L6) 0.6 Deep(L5) Deep(L7) 0.55 BOVW BOVW+GIST+DEEP(L5) False Positive Rate=1-True Negative Rate=1-Specifcity True Positive Rate=Sensitivity GIST Deep(L6) 0.6 Deep(L5) Deep(L7) 0.55 BOVW BOVW+GIST+DEEP(L5) False Positive Rate=1-True Negative Rate=1-Specifcity Right Pleural Effusion Detection Cardiomegaly Detection

21 Ginneken (MIA-2015):Detection of Pleural Effusion in Chest Radiographs Segmentation; Landmarks Detection; Localization; Feature extraction and Classification Lung segmentation and chest wall contour delineation using pixel classification techniques Accurate localization of Costophrenic point(cp) Extraction of features in the region around the CP point: Angle; Intensity; Region morphology Supervised learning for PE detection using Random Forest classifier [Maduskar et al. Automatic detection of pleural effusion in chest Radiographs,, MEDIA 2015 ] Right Pleural Effusion Ginneken AUC CNN features AUC Left Pleural

22 II. LONGITUDINAL MS LESION SEGMENTATION MRI data Multi-View Convolutional Neural Networks

23 Multiple Sclerosis Lesion Segmentation MS is one of the most common neurological diseases in young adults. It affects approximately 2.5 million people worldwide The immune system attacks the central nervous system and damages the myelin, a fatty tissue which protects the nerve fibers - This leads to deficiency in sensation, movement and cognition MS lesions (scars) are formed in damaged regions, mostly in the WM 1/2 7 T 1 -w T 2 -w PD-w FLAIR 1/33

24 Multiple Sclerosis Lesion Segmentation Efficient MS treatment: reduces lesion volume Manual segmentation: time consuming and subjective Automatic segmentation algorithms are needed! Very challenging: MS lesions vary in size, location, texture and shape MS Lesion 2/33

25 3/33 Guttmann et al Lesions in Time Studies have shown: MS lesions change significantly over time ISBI2015: Longitudinal MS lesion segmentation challenge Current state-of-the-art methods: Many algorithms: Random Forests (Geremia et al. 2013), Sparse Dictionary Learning (Weiss et al. 2013), Deep Learning (Brosch et al. 2016) But No use of temporal data! T 1 -w

26 ISBI 2015: Data Set Description Training Set: o 5 Patients, 4-5 time points per patient (Total scans: 21) o Manually segmented by 2 expert raters Test Set: o 14 Patients, 4-6 time points per patient (Total scans: 61) o No publicly available manual segmentations o Evaluated online 3T MR scanner 4 Contrast images: o T 1 -w: voxel dimensions = 0.82x0.82x1.17 mm o FLAIR, T 2 -w, PD-w: voxel dimensions = 0.82x0.82x2.2 mm Follow-up time: 1 year 20/33

27 Solution: Segmentation as a voxel classification task o Total segmented lesion voxels: ~250K Solution #1: Patch based training (and classification) for segmentation CNN Solution #2: 3D Data Augmentation p Non Lesion p Lesion

28 Patch Extraction Extract 24 32x32 patches around each candidate lesion voxel: 3 views (Axial, Coronal, Sagittal); 4 images (FLAIR, T1, T2, PD); 2 time points for each view Axial Coronal Sagittal Previou s scan Current scan FLAIR T 2 -w T 1 -w PD-w FLAIR T 2 -w T 1 -w PD-w FLAIR T 2 -w T 1 -w PD-w 12/33

29 Solution: Candidate Extraction Based on two observations: o Lesions are hyper intense in FLAIR o Lesions are located in WM or on the border between WM and GM Candidate mask: M p = 1, I FLAIR p > θ FLAIR Dilate R WM p 0, otherwise > θ WM I FLAIR Dilate R WM M 8/33

30 Network Architecture (I) All 24 extracted patches are fed into a convolutional neural network, which outputs a lesion probability for each voxel CNN data fusion: Modalities Fused at first layer. Utilizes fine-level voxel intensity correlations Time Points Fused at intermediate layer. Able to detect larger scale features such as change in lesion size Views Fused by fully connected layers, rather than convolutions (since they are not connected spatially). Utilizes high level features.

31 Network Architecture (II) Axial V-Net, T i Axial V-Net, TCoronal i 1 V-Net, Coronal T i V-Net, Sagittal T i 1 V-Net, Sagittal T i V-Net, T i 1 Axial L-Net Sagittal L-Net Coronal L-Net

32 Technical Details Training infrastructure: Keras (Theano wrapper) Nonlinearity: Leaky ReLU (α = 0.3) Leave-Patient-out cross validation (4 training / 1 validation) Avoiding overfitting: o Dropout (p = 0.25) after every convolutional and fully connected layer o Weight Sharing: Shared weights for T i and T i 1 V-Nets o Data Augmentation: Rotations in 3D drawn from Gaussian distribution μ 18/33

33 Qualitative Example Lesion segmentation is a subjective task with substantial inter-rater variability (IRV) A successful algorithm yields a variability similar to expert s IRV Input Proposed Expert #1 Expert #2 Expert #1 22/33

34 Quantitative Analysis Comparing automatic and manual rater segmentation: o o S A, S R - Automatic and Rater segmentation volumes Λ A, Λ R - Automatic and Rater lesion lists Cross validation metrics: o Volume correlation: VC S A, S R = ρ S A, S R [ 1,1] o Dice S A, S R = 2 S A S R S A + S R [0,1] S A S R o PPV S A, S R = S A S R + S A S C [0,1] R Λ o LTPR S A, S R = A Λ R [0,1] Λ A Λ R + Λ C A Λ R o LFPR S A, S R = o Test Evaluation metric: Λ A Λ R C Λ A Λ R C + Λ A C Λ R C [0,1] S R S A S R S A o Sc S A, S R = 1 Dice S 8 A, S R + 1 PPV S 8 A, S R + 1 LFPR S 4 A, S R + 1 LTPR S 4 A, S R + 1 VC S 4 A, S R o Averaged across all cases and all raters o Normalized such that the lower inter-rater score is equal to 90 23/33

35 Quantitative Analysis Time Point s Images Dice (R1) Dice (R2) p-value 1 FLAIR < T 1,T 2, PD, FLAIR FLAIR T 1, T 2, PD, FLAIR Multiple images improve accuracy Multiple time points enhance segmentation even further Best model nearly reaches human level accuracy Rater # /33

36 Examples Input Proposed System Expert 32/33

37 Test Set Results Test Set Results: Top 5 groups out of 18 groups Rank Method ISBI score Dice 1 Proposed System PVG Proposed System, no postprocessing IMI VISAGES State-of-the art in challenge score and Dice Post processing improves overall score, as predicted in cross validation Challenge score higher than 90, comparable to performance of an expert 28/33

38 III. LIVER SEGMENTATION AND LESIONS DETECTION CT data Fully Convolutional Network

39 Fully Convolutional Networks (FCN) conv, pool, nonlinearity End-to-End Pixels to Pixels A more efficient computational scheme Good localization using skip connections between layers Combine coarse, high layer information with fine, low layer information Long, Shelhamer and Darrell Fully Convolutional Networks for Semantic Segmentation, CVPR (2015) upsampling pixelwise output + loss

40 FCN for Liver Lesions Detection Adjacent slices end-to-end, joint learning of semantics and location

41 Data Lesion detection network training and validation: 20 patients with 1-3 CT examinations per patient; 68 lesion segmentation masks and 43 liver segmentation masks Liver segmentation network training: 20 CT scans with entire 3D liver segmentation masks. Input CT scan Labels image

42 Example Results Liver Segmentation Lesion Detection

43 Results Liver Segmentation Lesion Detection Combined liver segmentation and lesion detection 0.86 TPR 0.6 FPC FPC (False Positives Per Case): total number of false detections divided by the number of cases TPR (True Positive Rate): total number of detected lesions divided by the total number of lesions

44 Multi-class patch-based CNN system Deep learning approach that models explicitly the variability within the non-lesion class, to support an automated lesion detection system.

45 Parallel Multi-class CNN

46 Experiments and Results The dataset includes CT images of 132 livers and 498 lesions.

47 Experiments and Results Multi-class CNN reduces false-positive detections and improves the robustness of the system.

48 Experiments and Results

49 To Conclude Major challenges in the Bio and Medical domains Solving the data challenge: Data augmentation Data representation (patches) More shallow networks Know your Context! Going Forward: GAN (Generative Adversial Networks): synthetically augment data samples Noisy Labels

50 Thank You Prof. Hayit Greenspan Department of Biomedical Engineering, Tel-Aviv University, Israel Collaborators: Prof. Goldberger, Prof. Wolf, Dr. Lieberman, Dr. Klang, Dr. Amitai Prof. Konen Students: Idit Diamant, Ariel Birenbaum, Avi Ben-Cohen, Maayan Frid, Ofer Geva, Yaniv Bar Funding: INTEL Collaborative Research Institute for Computational Intelligence (ICRI-CI) KAMIN Israel Industrial Ministry Ela Kodesz Institute for Medical Engineering and Physical Sciences Adams Super-Center for Brain Studies, Tel-Aviv University.

51 Related Publications: Y. Bar, I. Diamant, L. Wolf, S. Lieberman, E. Konen, H. Greenspan, "Chest Pathology Detection Using Deep Learning with Non-Medical Training,' IEEE International Symposium on Biomedical Imaging (ISBI), April 2015 Y. Anavi, I. Kogan, E. Gelbart, O. Geva, H. Greenspan, ''A Comparative Study for Chest Radiograph Image Retrieval using Binary, Texture and Deep Learning Classification, IEEE Engineering in Medicine and Biology Society, (EMBC), August 2015 Y. Bar, I. Diamant, L, Wolf, S, Liberman, E. Konen, H. Greenspan, Chest Pathology Identification using Deep Feature Selection with Non-Medical Training, Journal of Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, May 2016 A. Bar-Cohen, I. Diamant, E. Klang, M. Amitai, H. Greenspan, Fully Convolutional Network for Liver Segmentation and Lesions Detection," MICCAI workshop on Deep Learning in Medical Image Analysis (DLMIA`16), MICCAI conference, Athens, Greece, October A. Birenbaum, H. Greenspan, Longitudinal Multiple Sclerosis Lesion Segmentation using Multi- View Convolutional Neural Networks," MICCAI workshop on Deep Learning in Medical Image Analysis (DLMIA`16), MICCAI conference, Athens, Greece, October Y. Bar, I. Diamant, L. Wolf, S. Lieberman, E. Konen, H. Greenspan,"Chest Pathology Identification using Deep Learning," in Deep Learning for Medical Image Analysis, Editors: K. Zouh, H. Greenspan, D. Dinggshan, Elsevier/Academic Press, Dec H. Greenspan, B. Van Ginneken, R. Summers, Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique, IEEE Transactions on Medical Imaging, Volume: 35, Issue: 5, , May 2016

52 Deep Learning Tasks & Methods Tasks Detection, Segmentation, Categorization Organ level, Pathology level Methodologies Input to network: Pixels, Patches, ROIs, Full image Combine Classical with Deep vs All Deep Transfer Learning methods & Fine Tuning Unsupervised/ Weakly Supervised/Supervised Learning Greenspan Medical Image Processing Lab

53 GAN Figure by Chris Olah Generative Adversarial Networks. I Goodfellow et al. arxiv: , 2014

54 GAN Application Image generation Image-2-Image Vector space arithmetic Image-to-Image Translation with Conditional Adversarial Networks. Isola et al. arxiv: , 2016 Super resolution Unsupervised Representation Learning With Deep Convolutional Generative Adversarial Networks. Radford et al. arxiv: , 2016 Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Ledig et al. arxiv: , 2016

55 GAN Application for Medical CT/Xray-2-Segmentation Synthesis of medical data Towards Adversarial Retinal Image Synthesis. P Costa et al. arxiv: , 2017 SCAN: Structure Correcting Adversarial Network for Chest X-rays Organ Segmentation. Wei Dai et al. arxiv: , 2017

56 Example Segmentation Input Image Candidate voxels CNN Prediction Manual Segmentation

57 Results Liver Segmentation Lesion Detection Combined liver segmentation and lesion detection 0.86 TPR 0.6 FPC FPC (False Positives Per Case): total number of false detections divided by the number of cases TPR (True Positive Rate): total number of detected lesions divided by the total number of lesions

58 Module 2: Algorithms - The algorithms used in the processing of big data, specifically those applied in medicine, artificial intelligence, and computer vision. When we have Big Data we can use new Machine Learning algorithms called Deep Learning algorithms With the data examples, we can shift from rules determined by experts, to discoveries of unknown relationships in the data, which are learned automatically. We use systems that have millions of parameters, and these can be tuned from to match the examples. Need to tune them to match the training data that is given to us, but to learn the relationships that will be true, and generalizable to new data coming into the system. We need to make sure we do not overfit the training set of examples. I am focusing on Image Data In Computer vision we now have Millions of images with annotations, with thousands of Categories; that is Big Data. We can Learn from Examples. In Medical image analysis, we can extract images from PACS, and even reports and clinical parameters, but we lack true labeling of the data, and the road to get a large set of training data from the experts is a long and tedious one.

59 System overview Cropping ROI of CP angle from segmentation Cropped image of lungs bounding box Pre-trained network for ImageNet: VGG-S Features aggregati on from layers: FC5 FC6 FC7 Adding direct measurement of cardiothoracic ratio Optimized SVM per each pathology Right pleural fluid Y/N Left pleural fluid Y/N Enlarged heart Y/N Enlarged mediastinu m Y/N Return of the Devil in the Details: Delving Deep into Convolutional Networks', Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, BMVC 2014

60 Multiple Sclerosis (MS) Lesion Segmentation MS is the most common neurological disease in young adults. An auto-immune disease which causes lesions in the central nervous system visible in conventional MRI Segmentation PD T1 T2 FF Greenspan Medical Image Processing Lab Time point

61 Comparisons with Classical Approaches Medical Dataset GIST features An image BoVW features Feature Extraction (GIST, BoVW, CNN features) Frequency Image model Word number Feature Selection (optional) Feature Fusion (optional) CNN features SVM Classifier Pathology Detection/Screening

62 Xray: Methodology Technical Details Binary categorization (per pathology): Cases contain pathology - labeled as positive Cases without this pathology - labeled as negative Classification using: Support Vector Machine (SVM) classifier Intersection kernel Feature Standardization (Normalization): Normalize each feature across all CV train set images: Subtract mean, divide by standard deviation Apply mean and standard deviation on CV test set

63 MS MRI: CNN - Technical Detai Nonlinearity: Leaky ReLU with α = 0.3 Dropout with p = 0.25 after every convolutional and fully connected layer Weight Sharing: Shared weights for T i and T i 1 V-Nets Data Augmentation: Random rotations in 3D drawn from Gaussian distribution with μ = 0, σ = 5 ) Class Balancing: Equal number of positive and negative samples in each batch. Batch size = 128 Solver: AdaDelta Running Times (GPU: NVIDIA GeForce GTX 980 Ti): Training: 4 Hours Prediction: 27 seconds

64 Why Fully Convolutional? Patch-wise CNN Quite slow - redundancy due to overlapping patches Only local context FCN A more efficient computational scheme Good localization using skip connections between layers

65 TRAINING DATA FOR CNNS It is a well known fact that for training a CNN there is need of many (thousands) of training examples for each class. In the medical domain it is almost impossible to have such a huge number of labeled images.

66 II. CHEST CT LOCAL FINDINGS Local findings in specific small areas. Here, the data challenge can be solved via data augmentation. Data augmentation means synthesizing many samples out of one source sample. Data augmentation needs to be done so the generated data truly represents real findings, and without redundancy between the different samples.

67 NODULE APPLICATION In the nodule detection applications, we had 2 main challenges: Shape: Nodules are 3D objects, while deep-learning architecture was mainly developed for images (gray-scale or RGB). Data: It is very difficult to get a large set of validated nodules in order to train a CNN. We used half of the LIDC data set for training, consisting of ~1000 validated true nodules.

68 SOLVING THE SHAPE CHALLENGE: 2.5D REPRESENTATION Sagittal, Coronal and Axial views of nodule 2.5D Representation

69 SOLVING THE DATA CHALLENGE: DATA AUGMENTATION In order to successfully train a CNN, we enriched the candidates by augmenting each validated true nodule 100 times Data was augmented by running random 3D linear transformations and therefore, creating many different 2.5D representations for each nodule These representations truly represent real findings, and have very low redundancy between them Two 2.5D representations of same nodule Axial Coronal Sagittal

70 Example: Lung Nodules Output to Report: Summary of nodules detected Detailed characteristics for each nodule Representative key image with overlay marks for identified nodules Example sentences: Summary: 2 lung nodules were detected: Nodule #1, Solid, having diameter 11 mm in RUL in slice #83 Nodule #2, Solid, having diameter 9 mm in RUL in slice #70 Details for each nodule: Type: Solid Nodule Number: 1 Image Number: 83 Volume: 0.1 cm3 Diameter: 7 mm Solid Volume: 0.1 cm3 Solid Diameter: 7 mm Calcified: N Fat: N Lobar Position: RUL

71 Lung Nodules Results Train data set: 400 cases from LIDC data base. ~1000 validated lung nodules Test data set: 129 cases from the LIDC data base. Positive nodules are only those agreed by all four readers, and larger than 3mm or 2 x slice thickness (the larger of the two) Number of positive nodules: 106 Results: Sensitivity: 80/106 (75.5%) FP Rate: 1.84

72 IV. CHEST X-RAY GLOBAL FINDINGS Global appearance and hard to segment in single image. Cannot use patch-based classification solution. Data challenge very significant! Solution: use transfer learning: Taking a CNN trained in other domains on a huge data set (e.g. ImageNet). Pleural Fluid Enlarged Heart Enlarged Mediastinum

73 IMAGE-LEVEL LABELING USING TRANSFER LEARNING Right pleural fluid Y/N Pre-trained network for ImageNet: VGG-S Features aggregation from layers: FC5 FC6 FC7 Optimized SVM per each pathology Left pleural fluid Y/N Enlarged heart Y/N Enlarged mediastinu m Y/N

74 Example: Cardiomegaly App Clinical need: Indicative for chronic condition resulting from obesity or coronary artery disease An enlarged heart may not pump blood effectively, resulting in congestive heart failure Important for screening Algorithmic Tasks: Identification of typical image presenting enlarged heart Alternative: direct measurement of CTR from lungs segmentation In The Report: Detection (normal/ abnormal) Future- present CTR measurement overlaid on original image

75 Cardiomegaly App - Results Train set: 631 PA scans 140 positives 491 negative Test was performed on 383 chest PA scans. 81 positives 239 negative AUC = 0.95 Sensitivity: 0.9 Specificity: 0.85

76 Solving the Data Challenge Different Application domains & varying Solutions Main types of biomarkers (findings): Localized pixel level findings (brain lesions in MRI) Localized larger area findings (liver lesions in CT) Global findings (Cardiomegaly, pleural fluid in X-ray) For each finding type we solve the data challenge accordingly

77 I. CHEST CT GLOBAL FINDINGS Distributed findings. Classification is done per slice. A standard chest CT has slices. Many findings are scattered in the lungs, and therefore they can be seen in many slices. A positive case can have tens or even hundreds of positive slices. A negative case has hundreds of negative slices. Therefore, even with a training set as small as 100 for each class, one can have many thousands of slices for the task.

78 FREE PLEURAL AIR, OPACITIES & PLEURAL FLUID APPLICATIONS Classification is done per side for each slice, on an ROI around the lung. Each ROI is classified to: Contains / doesn t contain free pleural air Contains / doesn t contain opacities Contains / doesn t contain pleural fluid A global (per side) classification is done according to these slice-based results.

79 Example: Opacities App Output to Report: Binary output (opacity present or not). Laterality (right/ left) Representative key image Example sentences: There is evidence of consolidations or parenchymal opacities in the left lung.

80 Opacities App Results Train data set: 328 lungs Positive: 92 Negative: 236 Test data set: 321 lungs Positive: 89 Negative: 232 Results: Sensitivity: Moderate: 63/66 (95.5%) Severe: 23/23 (100%) Specificity: 231/232 (99.6%)

81 III. CHEST X-RAY SEGMENTABLE FINDINGS Located in a specific connected component in single image. Segmentation is a classification process, where the classifier differentiates between positive pixels and negative pixels. Each pixel is represented by a patch of a certain size. Huge number of positive patches per positive case.

82 Free Pleural Air Application Binary classification: free air vs. lung tissue CNN is capable of learning typical textures for lungs/ free air Transferring from hundreds of training samples to ~5M training patches Learned 1st layer texture filters

83 Example: Free Pleural Air App Clinical need: Can be caused by physical trauma or as a complication of clinical intervention Small accumulation of air might be asymptomatic Free air can be acutely accumulated and becomes life threatening Algorithmic Tasks: Identification of each lung pixel as free-air or other lung tissue Diagnosis per lung based on threshold on lung coverage by free-air pixels Report includes: Detection (yes/ no) Localization (right/ left) Size estimation (percentage of lung area) Ability to present the suspected regions in the report

84 Free Pleural Air App - Results Test was performed on 86 chest PA scans. 48 positives 60 negatives Test was performed on 86 chest PA scans. 43 positives 43 negatives AUC per lung = 0.96 Sensitivity: 0.9 Specificity: 0.86

85 FCN in the Medical World U-net architecture Outperformed best method on the IEEE_ISBI challenge for segmentation of neuronal structures in electron microscopic stacks Won the Cell Tracking Challenge at ISBI 2015 Ronneberger, Olaf, et al.. "U-net: Convolutional networks for biomedical image segmentation." MICCAI, 2015.

86 FCN in the Medical World V-Net 3D Convolutional architecture 3D Convolutions 3D Prostate segmentation in MRI volumes Dice Loss layer (for background foreground balance) Fast and accurate results Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation.", MICCAI 2016