DEEP LEARNING APPROACH FOR REMOTE SENSING IMAGE ANALYSIS

Transcription

1 DEEP LEARNING APPROACH FOR REMOTE SENSING IMAGE ANALYSIS Amina Ben Hamida*,** Alexandre Benoit*, Patrick Lambert*, Chokri Ben Amar** * LISTIC, Université Savoie Mont Blanc, France {amina.ben-hamida,alexandre.benoit, patrick.lambert}@univ-smb.fr ** REGIM, ENIS, Tunisia, chokri.benamar@ieee.org

2 Presentation outline Scientific context Big Data Deep Learning (DL) Remote Sensing DL for hyperspectral Data Experimental dataset DL architectures Results Discussion & Future work

3 Scientific Context Specific Fields 350 millions photos are uploaded daily Big Data 100 hours of videos are uploaded every minute: 2 billions each year 1.4 millions of minute chats are saved every minute Medical Imaging. Remote Sensing: (RS) Use case example : Sentinel satellites which provide some thousands of terabytes of data on a scale of 10 years.

4 Scientific Context Can we adapt recent methods developed in the multimedia community for RS?

5 Deep Learning Modelling high level abstractions from multiple non linear transformations Rachel

6 Deep Learning Fully connected layer : connects all the neurons to all available inputs No spatial embedding Non linearity : Impact of convergence speed!!!

7 Deep Learning Convolutional layer : Local filtering Rich feature maps generation Pooling layer : Subsampling signals Add translation robustness

8 Hyperspectral Data

9 DL for Hyperspectral Data Classification Taking into account the spatial and spectral components Seperately Only using spectral Early combining (using SAE) information spatial and spectral dimensions Explodes Forget parameters Spatial number information more data for training? Looks good

10 Experimental dataset University of Pavia dataset Single image pixels 103 bands 9 classes

11 DL architecture Cascading 3D convolutions, 1D convolutions and final fully connected layers

12 Hyperspectral Deep Network architectures 3 layers 3D/1D 4 layers 3D/1D 6 layers 3D/1D

13 Results : accuracy vs complexity Accuracy when training on ~5% of the data Accuracy *1 4 layers * * 5*5 6 layers Spatial range impact layers Number of parameters * Hu&al, Deep convolutional neural networks for hyperspectral image classification, in Journal of Sensors,

14 Results : accuracy vs complexity Deeper models for increased performances and less parameters. Spatial information does matter but spatial range depends on the use case Deeper networks need more time to train

15 Results : 6 layers deep net, 5*5 neighbors

16 Results : 6 layers deep net, 5*5 neighbors Spectral profiles

17 Results : 6 layers deep net, 5*5 neighbors Per class accuracy mostly stable ~95% on average Classification errors explained by : similar spectral profiles boundary effects (ROI size vs neighborhood class)

18 Results : confusion vs neighborhood 3*3 1*1 5*5 Processing time (caffe, CPU mode, Dual core i7 proc). 1h 2h 5h Observation : spatial information gradually corrects spectral based errors

19 Results : Accuracy vs training dataset size Accuracy CNN challenger, 5*5 neighbors, no pretraining K. Makantasis&al Deep supervised learning for hyperspectral data classification through convolutional neural networks, IGRS2015 ~20000 parameters Accuracy on Pavia University dataset layers, 3*3 neighbors, ~4419 parameters 6 layers, 5*5 neighbors, ~6074 parameters Training samples ratio (%) SAE challenger, 7*7 neighbors, with pretraining X. Ma&al Hyperspectral image classification via contextual deep learning, EURASIP JIVP 2015 >>20000 parameters

20 Conclusion Deep Learning can do the job! Automatic adaptation to the context and good results Deeper is better... up to a limit? Main issues : Expertise required Network architecture design Training procedures design Reduce the number of parameters

21 Future Work guideline Enhance architectures Siamese Learning metrics from similarity measures Networks SqueezeNet Get lighter models! approach Adapt to new contexts The Switch to multispectral data Sentinel Use case Play with unlabelled data 35

22 What's next? Yes, DL was so far so good for simple RS application But, what gaps will it be facing when hardening the task? Questions? 32

23 Thank you for your attention

24 Results : from one dataset to another Accuracy vs dataset, deepness, neighborhood 3 layers 6 layers Pavia Univer sity Pavia Cente r 1*1 neighbors 75.9 % 90.5 % 3*3 neighbors 84.0 % 94.5 % 5*5 neighbors 93.8 % 96.4 % 7*7 neighbors 85.9 % 96.2 % 1*1 neighbors 86.5 % 3*3 neighbors 92.3 % 5*5 neighbors 93.8 % 98.5 %

25 Future Work guideline Testing the robustness level of the DL structure Facing Injecting noise into the system in order to test its ability to deal Noise with noisy images. Degrade Testing to what extent can the system face a variety of trials to performances degrade its performances. 37

26 Future Work guideline Relying on larger ground truth databases Larger The use of other dabases in order to create ground truth amount of annotaded ones. data This work can be done in collaboration with other labs. 38

27 Future Work guideline Extending the work to the sentinel databases Resorting to multispectral and hyperspectral data, with The complex challenges to rise. Sentinel Use case Facing the challenge of large unlabelled data 40

28 Conv layer hints parameters vs IO dimensions mi<=n fli<=f 24