DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. Andrew Liu, Ryan Shen Deep Learning Solution Architect

Transcription

1 DEFECT INSPECTION FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning Solution Architect

2 Defect Inspection and its challenges AGENDA NGC Docker images Model set up - Unet Data preparation - DAGM Production - Speed up with TensorRT 2

3 INDUSTRIAL DEFECT INSPECTION 3

4 INDUSTRIAL DEFECT INSPECTION 4

5 NGC DOCKER IMAGES 5

6 Benefits for Deep Learning Workflow High Level Benefits and Feature Set Single software stack Develop once, deploy anywhere Scale across teams of practitioners Developer, DevOp, QC

7 Defect inspection Workflow from scratch to production within container Training Inference NGC optimized docker image TENSORRT GRE docker pull nvcr.io/nvidia/tensorflow :18.02-py3 docker pull nvcr.io/nvidia/tensorrt: py2 7

8 MODEL SET UP 8

9 WAYS FOR DEFECT INSPECTION Depends on domain adaptation Image Classification Object Detection Image Segmentation Label: 0 / 1 (Defect / Non Defect) Label: Bounding-Box Label: Polygons Mask 9

10 CNN STRUCTURE LeNet Source: Design of Deep Convolutional Neural Network Architectures for Automated Feature Extraction in Industrial Inspection, D. Weimer et al,

11 convnets perform classification < 1 millisecond defect class dim vector end-to-end learning 11

12 lots of pixels, little time? ~1/10 second??? end-to-end learning 12

13 U-Net structure X3 Conv2d+ReLU 2X2 MaxPool X2 Conv2dTranspose copy and concatenate

14 KERAS IMPLEMENTATION Convolution 14

15 DECODING deconvolution 15

16 Image segmentation on medical images Same process among various use cases Data Science BOWL 2016 Data Science BOWL 2017 Data Science BOWL 2018 MRI image Left ventricle heart disease CT image Nodule Lung cancer Image Nuclei Drug discovery 16

17 DATA PREPARATION 17

18 INDUSTRIAL OPTICAL INSPECTION German Association for Pattern Recognition 18

19 INDUSTRIAL OPTICAL INSPECTION German Association for Pattern Recognition Pass NG Pass NG Pass NG Pass NG 19

20 DATA DETAILS Original images are 512 x 512 grayscale format Output is a tensor of size 512 x 512 x 1 Each pixel belongs to one of two classes Training set consist of 100 images Validation set consist of 50 images 20

21 MORE EXAMPLES 21

22 IMBALANCE DATA 22

23 Dice Metric (IOU) Metric to compare the similarity of two samples: 2A nl A n + A l Where: A n is the area of the contour predicted by the network A l is the area of the contour from the label A nl is the intersection of the two The area of the contour that is predicted correctly by the network 1.0 means perfect score. More accurately compute how well we re predicting the contour against the label We can just count pixels to give us the respective areas 23

24 24 LOSS FUNCTION WITH KERAS 24

25 25 LEARNING CURVES LOSS 25

26 APPLICATION: INDUSTRIAL INSPECTION NVIDIA 1000 defect-free, 150 defect images Challenges: Not all deviations from the texture are necessarily defects. 26

27 PRODUCTION 27

28 FINAL DECISION Plus Human Logic Size, Position, etc 28

29 INFERENCE PIPELINE Inference Post-processing and decision making DataCenter DGX Server TensorRT+GRE Camera Fetch image Detectors/Classifiers/Segment Composite Result Metadata Domain Criteria Defect Pattern Ratio Defect Level Defect region size Defect counts Determine threshold Precision/ Recall Edge Machine TensorRT+GRE P40 29

30 Development flow ONNX Called UFF, Universal Framework Format 30

31 Defect inspection Workflow from scratch to production within container Training Inference NGC optimized docker image TENSORRT GRE docker pull nvcr.io/nvidia/tensorflow :18.02-py3 docker pull nvcr.io/nvidia/tensorrt: py3 31

32 SUMMARY Challenges Delivers Training, inference environment is hard to build, maintain, share. Using NGC Docker images. Model optimizations and speed up throughput. TensorRT + GRE SDK So many deep learning model out there, how to choose the right model? If your dataset, demand requirement fit the scenario like we do. U-Net model is great choice for segmentation task. 32

33 Thank You 33