Voxel Deconvolutional Networks for 3D Brain Image Labeling

Transcription

1 Voxel Deconvolutional Networks for 3D Brain Image Labeling Yongjun Chen, Hongyang Gao, Lei Cai, Min Shi, Dinggang Shen, Shuiwang Ji Washington State University University of North Carolina at Chapel Hill KDD 2018

2 Deep Learning Becoming Pervasive [1] [2]

3 Encoder-Decoder Network * * * * * *

4 Encoder-Decoder Network * * * * * * Deconvolutional layer

5 Checkerboard Artifact Issue

6 1D Deconvolutional Layers kernel kernel1 kernel2 w 1 w 2 w 3 w 4 w 1 w 3 + w 2 w 4! " w 1 w 2 w 3 w 4! " w 1 w 3! " w 2 w 4! # w 1 w 2 w 3 w 4!! $ w 1 w 2 w 3 w 4!! $ w 1 w 3!! # w 1 w 3! #! $ w 2 w 4 w 2 w 4! % w 1 w 2 w 3 w 4! % w 1 w 3! % w 2 w 4! & w 1 w 2 w 3 w 4! & w 1 w 3! & w 2 w 4 '( ) '( * '( + '(, '( - '( )) '(. '( / '( 0 '( 1 '( )2 '( ). '( ) '(. '( * '( / '( + '( 0 '(, '( 1 '( - '( )2 '( )) '( ). '( ) '(. '( * '( / '( + '( 0 '(, '( 1 '( - '( )2 '( )) '( ). '( '( (5) (7)

7 3D Deconvolutional Layers ØAssume X R is the input and Y R is the output of a deconvolutional layer. Y " = X k ' for i = 1,, 8 (1) Y = Y " Y 7 Y ; Y < Y = Y > Y? (2) is convolution operation, is the periodical shuffling and combination operation, Y ' is the i 9: feature map and k ' is corresponding kernel

8 3D Deconvolutional Layers ØAssume X R is the input and Y R is the output of a deconvolutional layer. Y " = X k ' for i = 1,, 8 (1) Y = Y " Y 7 Y ; Y < Y = Y > Y? (2) is convolution operation, is the periodical shuffling and combination operation, Y ' is the i 9: feature map and k ' is corresponding kernel

9 Voxel Deconvolutional Layers

10 Building Relationships Y " = X k " ; Y ' = X, Y ",, Y 'B" k ', for i = 2,, 8 (3)

11 Variances of VoxelDCLs Ø ivoxeldlc: Y " = X k " ; Y ' = X, Y ",, Y 'B" k ', for i = 2,, 8 (3) Ø ivoxeldcla: 'B" Y " = X k " ; Y ' = X + D Y E k ', for i = 2,, 8 (4) Ø VoxelDCLc: Ø VoxelDCLa: 'B" Y " = X k " ; Y ' = D Y E k ', for i = 2,, 8 EF" EF" Y " = X k " ; Y ' = Y ",, Y 'B" k ', for i = 2,, 8 (5) (6)

12 Efficient Implementation

13 Reducing Dependencies Input feature map Intermediate feature maps (a) 1,2 3,4,5 1,2,3,4,5 6,7,8 (Output) (b)

14 Voxel Deconvolutional Networks

15 Voxel Deconvolutional Networks Ø ivoxeldcnc o Replace deconvolutional layers with ivoxeldclc layers. Ø ivoxeldcna o Replace deconvolutiona layers with ivoxeldcla layers Ø VoxelDCNc o Replace deconvolutional layers with VoxelDCLc layers Ø VoxelDCNa o Replace deconvolutional layers with VoxelDCLa layers. 1@ @ @ @ @ @ @ @ @ @ @ U-Net 32@ @ @ @ @ * * * * * *

16 3D Brain Image Labeling

17 Datasets Ø ADNI o Labeling hippocampus o 2 Classes ØLONI LPBA40 o Labelingregions of interest (ROIs) o 54 Classes

18 Measurement ØDice Ratio is used to measure prediction results N DICE = 1 k D DICE A 1 ', B ' = k D 2 A ' B ' A ' + B ' 'F" k is total number of classes, A ' is prediction for the i 9: ROI and B ' is corresponding ground truth label. N 'F"

19 Results ØIncrease 1.39% dice ratio on ADNI dataset. ØIncrease 2.21% dice ratio on LONI LPBA40 dataset. ØAll proposed methods outperform baseline method U-Net. Dataset Model Dice Ratio (%) U-Net ivoxeldcnc ADNI ivoxeldcna VoxelDCNc VoxelDCNa U-Net ivoxeldcnc LONI LPBA40 ivoxeldcna VoxelDCNc VoxelDCNa

20 U-Net VoxelDCNa VoxelDCNc ivoxeldcna ivoxeldcnc L Hippocampus L Putamen L Caudata L Cingulate gyrus L Insular cortex L Fusiform gyrus L Lingual gyrus L Parahippocampal gyrus L Inferior temporal gyrus L Middle temporal gyrus L Superior temporal gyrus L Cuneus L Inferior occipital gyrus L Middle occipital gyrus L Superior occipital gyrus L Precuneus L Angular gyrus L Supramarginal gyrus L Superior parietal gyrus L Postcentral gyrus L Gyrus rectus L Lateral orbitofrontal gyrus L Middle orbitofrontal gyrus L Precentral gyrus L Inferior frontal gyrus L Middle frontal gyrus L Superior frontal gyrus Dice Ratio R Hippocampus R Putamen R Caudata R Cingulate gyrus R Insular cortex R Fusiform gyrus R Lingual gyrus R Parahippocampal gyrus R Inferior temporal gyrus R Middle temporal gyrus R Superior temporal gyrus R Cuneus R Inferior occipital gyrus R Middle occipital gyrus R Superior occipital gyrus R Precuneus R Angular gyrus R Supramarginal gyrus R Superior parietal gyrus R Postcentral gyrus R Gyrus rectus R Lateral orbitofrontal gyrus R Middle orbitofrontal gyrus R Precentral gyrus R Inferior frontal gyrus R Middle frontal gyrus R Superior frontal gyrus Dice Ratio

21 Timing Comparison ØIncrease training time, but not dramatically. ØTesting times are comparable. ØOverall, we not consider it as a major bottleneck of proposed methods. Dataset Model Training time Testing time U-Net 45 h 43 min sec VoxelDCNa 54 h 57 min sec ADNI VoxelDCNc 57 h 12 min sec ivoxeldcna 59 h 31 min sec ivoxeldcnc 59 h 54 min sec U-Net 52 h 05 min 12 min 12 sec VoxelDCNa 60 h 37 min 20 min 35 sec LONI LPBA40 VoxelDCNc 63 h 24 min 19 min 85 sec ivoxeldcna 71 h 12 min 20 min 32 sec ivoxeldcnc 75 h 40 min 20 min 05 sec

22 Conclusion

23 ØPropose the voxel deconvolutional layer (VoxelDCL) to address the checkerboard artifact of 3D deconvolutional layers. ØProvide an efficient implementation to improve the computational efficiency. ØBuild the voxel deconvolutional networks (VoxelDCN) and apply them to the volumetric brain image labeling. ØDemonstrate the effectiveness of the proposed methods.

24 Voxel Deconvolutional Networks for 3D Brain Image Labeling KDD 2018

25 Q & A