Life and Medical Biology Data Accelerator (Lambda)

Transcription

1 Life and Medical Biology Data Accelerator (Lambda) : Guangming Tan Ins>tute of Compu>ng Technology, Chinese Academy of Sciences 1

2 Biological Imaging Data Challenge Higher Resolu>on GAP: O(years)! High-Throughput Image Data Analysis is Required! Moritz Helmstaedter, Cellular resolu>on connectomics: challenges of dense neural circuit reconstruc>on, Nature Method, 10(6),

3 High Spa>otemporal Resolu>on Two- Photon Microscope Imaging System Peking University In vivo High Dimension 3

4 Event Detec>on at Cellular Level Institute of Computing Technology, Chinese Academy of Sciences Elementary Events of Calcium Signals Cheng, H Calcium Spark Science 1993 Sparks and Transients Cheng, H Cell 2008 Superoxide Flash Superoxide Flash 5µm Visualiza>on of Reac>ve oxygen species (ROS) 5 mm Animal s dynamic neural signals Dendrite Calcium Imaging 2µm (Zhuang Zhou, Xiaowei Can) 4

5 Life and Medical Biology Data Accelerator (Lambda, λ) Data PostgreSQL Bio-Format lambda Engine Domain-Specific Accelerator Auto-tuning library Pipeline Built-in modules Customizable framework 5

6 λ- Image SoYware/Hardware Stack Cardiovasology Biological Data Analysis Pipeline (cell event detection, segmentation) Mouse embryo heart image cell lineage Biological Data Analysis Algorithm Toolkit Brain deconvolution denoising stencil machine learning Mice brain cell Ca2+ spark detec>on MPI Spark CUDA OpenCL Database RDMA Accelerator High- dimension & mul>- mode biological image data system Data analysis pipeline for massive biological image Accelera>ng data- intensive algorithms for biological image analysis Endocrinology Islet forming in pancrea>c and imaging in vivo 6

7 High- throughput Image Processing Algorithm O(N*P 3 ) fmri sstem sbem LSFM Unbiased Analysis of Events Machine Learning Current Compu>ng Systems(SoYware/Hardware):O(Years) Interac>ve High accuracy High Performance Compu>ng Pla_orm: O(Minutes) 7

8 Paralleliza>on with in- memory Compu>ng Model Image L1 Raw Data Map Image L2... Image R1 Image R2... Raw Data Left Side Right Side Preprocess Preprocess Preprocess Preprocess 3D Deconvolution Intensity Normalization Subtract Background Preprocessed Data Match Powell Mutual Information Left Side Right Side Preprocessed Image L1 Registration Registered Image L1 Preprocessed Image L2 Registration Registered Image L2 Preprocessed Image R1 Registration Registered Image R1 Preprocessed Image R2 Registration Registered Image R2 Fusion Fusion Fused Image1 Fused Image2... Left Side Right Side Wavelet Decomposition Merge Image Stack Activity Measure Fused Data Fused Image Stack Fusion Decomposition Fused Data Planarity Enhancement Tensor Voting Segmetation Final Result For Visualization Process Reduce 3D Watershed Labelmap Image Data Spark 8

9 GPU Accelera>on of Algorithm Modules CPU GPU 0 DeconvoluJon Median Filter Objectness Filter IteraJve Closing 9

10 Image Processing & Analysis Pipeline GPU Spark Analysis Segmenta>on Fusion Registra>on Deconvolu>on 2D and 3D iterative deconvolution. Mutual Information is derived from Information Theory and its application to image registration has been proposed in different forms use global five-level wavelet decomposition Watershed segmentation Labelmap selection Particle analysis Mice Brain Cell Ca 2+ Spike Detection RL/Sparse Mutual Information Wavelet based Machine Learning Machine Learning (Event detection or Pattern Analysis) 10

11 Deconvolu>on of Pancreas Islet Images Preprocessing e:image p:psf for N iterations /*apple imaging model to estimate*/ E=gpu_fft(e, batch) B=gpu_multiply(E, PSF, batch) b=gpu_ifft(b, batch) Terabyte EM Images deconvolution GPU batch /*captured image divided by blurred estimate*/ r=gpu_divide(o, b, batch) /*calculate correction vector*/ R=gpu_fft(r, batch) C=gpu_multiply(R, PSF, batch) c=gpu_ifft(c, batch) /*apply correction vector*/ e=gpu_multiply(e,c, batch) end 4.7YEARS 4 GPUs (K20) name pixel(xyz) #ite rs size Fiji (JAV A) GPU speed up beta.tif 1024x2048x MB 60m 22s 163 glucose_sequen tial2.tif 512x512x MB 30m 10s DAYS 11

12 Institute of Computing Technology, Chinese Academy of Sciences Extrac>ng Cells from Mouse Embryos Images Time1 Time2 Time3 light- sheet microscopes images detecjon 200 Time points 2x500 images 2048x2048 pixels 4GB*2*200 = 1.6TB 1.5 DAYS excitajon detecjon culture ReconstrucJon Fast, two- side, 3D, duel- color imaging 12

13 Blitz:High Performance Machine Learning Toolkit NVIDIA DIGITS (Customized) Classifica>on SVM Clustering K- means Dimensionality PCA DNN CNN Algorithm Interface KNN Distributed Parallelism Data Parallelism Model Parallelism Pipelining Parallelism Communica>on Avoiding Layer Opera>on Performance Interface Operator Language (Linear Algebra / Tensor Primi>ves) Automa>c Performance Tuning Programming Hardware vectoriza>on Accelerator: mul>thread RDMA Virtual Backend Sugon Xmachine 13

14 Convolu>onal Nets 2012 (AlexNet) Conv2 Conv1 Pool1 Institute of Computing Technology, Chinese Academy of Sciences 13- layer architecture Layer Type Maps and neurons Kernel size Input 0 Input 1 map of 224*224 neurons Hardware Environment CPU: Dual Intel(R) Xeon(R) CPU E v3, 28 CPU- Memory: 128GB GPU: Tesla K20 GPU- Memory:6GB blitz caffe blitz caffe Convolu>ons Pooling 1310s 1960s 125ms 196ms Convolu>ons batch size=128 1 epoch running >me 1 batch size running >me Pooling 1 ConvoluJon 64 maps of 55*55 neurons 11*11 2 Pooling 64 maps of 27*27 neurons 3*3 3 ConvoluJon 192 maps of 27*27 neurons 5*5 4 Pooling 192 maps of 13*13 neurons 3*3 5 ConvoluJon 384 maps of 13*13 neurons 3*3 6 ConvoluJon 256 maps of 13*13 neurons 3*3 7 ConvoluJon 256 maps of 13*13 neurons 3*3 8 Pooling 256 maps of 6*6 neurons 3*3 9 Fully- connected 4096 neurons 1*1 10 Dropout 4096 neurons 1*1 11 Fully- connected 4096 neurons 1*1 12 Dropout 4096 neurons 1*1 13 Fully- connected 1000 neurons 1*1 14

15 Flash Detec>on Institute of Computing Technology, Chinese Academy of Sciences E.Coli, Jme series, 512X512X(100 frames). A B C Intensity increases rapidly Intensity declines obviously averaged intensi>es change con>nuously A nonstandard flash is not found by either expert or threshold- based method 15

16 Automated Flash Detec>on based on Blitz Input Stack of fluorescence images Preprocessing Image registration Cell segmentation Intensity average Skew elimination Data skew elimination Feature selection Data overfitting check If model accuracy is close to 100% Feature addition Test set upper bound estimation Model train If model accuracy is close to upper bound Data ratio adjustment Cross validation Model generation Model prediction Result collection F value: F=2 precision recall/(precision+recall), where precision =(no. returned flashes)/(no. returned peaks) recall =(no. returned flashes)/(no. all the flashes). Event detection Events output 9 features: 6 local, 3 global (amplitude, width, slope)*(left, right) Average intensity of trace, distance to the (last, next) peak Use cross valida>on to find parameters to train a model which can get beper accuracy and F value. Use MPI + CUDA paralleliza>on to reduce training >me 16

17 Membrane Segmenta>on based on Blitz Brain Heart Group Simple Thresholding Rand error [ 10 3 ] Warping error [ 10 6] Pixel error [ 10 3 ] Training Time NIPS Days (Four GPUs) Our approach Days (One GPUs) Deep Neural Network 17

18 Conclusion Develop a Spark- based paralleliza>on framework for high throughput image analysis pipelines Op>miza>ons on GPU Core algorithms in image processing (3x- 10x) SGEMM in deep learning ( 30%) Achieve significant speedups for image processing Years à days 18

19 Thanks! 19