Image Denoising AGAIN!?

Transcription

1 1

2 Image Denoising AGAIN!? 2

3 A Typical Imaging Pipeline 2

4 Sources of Noise (1) Shot Noise - Result of random photon arrival - Poisson distributed - Serious in low-light condition - Not so bad under good light (2) Electronic Noise - Instability of voltage/current - Temperature fluctuation - Analog to digital error - Gaussian distributed Shot noise Electronic noise Simplified diagram illustrating the two sources of noise 3

5 Noise! Shot noise Anscombe transform Gaussian noise My work: Gaussian Noise! 5

6 Adaptive Image Denoising for my PhD 6

7 Image Denoising Consider an additive i.i.d. Gaussian noise model: Our goal is to estimate from where Our Approach: Maximum-a-Posteriori 7

8 MAP Framework Since the noise i.i.d. is Gaussian, the conditional distribution is Therefore, the MAP is 8

9 Image Priors Markov Random Field (80s) Gradients (80s) Total Variation (90s) X-lets (wavelet, contourlet, curvelet,, 90s) Lp norm (00s) Dictionary (KSVD, 00s) Example (00s) Non-local (BM3D, nonlocal means, 2005, 2007) Shotgun! (2011) Graph Laplacian (2012) 9

10 Patch-based Priors What is a patch? A patch is a small block of pixels in an image Why patch? What is patch-based prior? 10

11 Training a Patch-based Prior Typically, we train a patch-based prior from a large collection of images EM Algorithm e.g., Gaussian mixture: 11

12 Good Training Set 12

13 How good? Example: Text Image clean image noisy image BM3D [Luo-Chan-Nguyen, 15] (single image method) (use targeted training) 13

14 Challenge: (1)Finding good examples is HARD. (2)Finding a lot of good examples is EVEN HARDER. My work: Can priors be learned adaptively? Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 14

15 Our Proposed Idea 15

16 Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 16

17 Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 17

18 Half Quadratic Splitting General Principle [Geman-Yang, T-IP, 1995] The Algorithm: 18

19 Solution to Problem (1): Example Gaussian Mixture Model [Zoran-Weiss 11] If where 19

20 Solution to Problem (2): The solution to (2) is 20

21 Question 1 : How to SOLVE this optimization problem? For Gaussian Mixture: 21

22 Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 22

23 Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 23

24 Toy Example Imagine that: (a) Original generic database (A LOT of samples) (b) Ideal targeted database (A LOT of samples) (c) In reality, samples from targeted database are FEW!!! 24

25 25

26 EM Adaptation 26

27 EM Adaptation 27

28 EM Adaptation Classical EM: EM Adaptation: 28

29 EM Adaptation 29

30 EM Adaptation Classical EM: EM Adaptation: 30

31 EM Adaptation 31

32 EM Adaptation Classical EM: EM Adaptation: 32

33 EM Adaptation in the literature J. Gauvain and C. Lee, Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains, IEEE Transactions Speech and Audio Process., vol. 2, no. 2, pp , Apr D.A. Reynolds, T.F. Quatieri, and R.B. Dunn, Speaker verification using adapted gaussian mixture models, Digital signal process., vol. 10, no. 1, pp , P.C. Woodland, Speaker adaptation for continuous density hmms: A review, in In ITRW on Adaptation Methods for Speech Recognition, pp , Aug M. Dixit, N. Rasiwasia, and N. Vasconcelos, Adapted gaussian models for image classification, in IEEE Conference Computer Vision and Pattern Recognition (CVPR 11), pp , Jun

34 Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 34

35 Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 35

36 EM Adaptation for Noisy Images i.e., denoise the image with a method you like. Assume the pre-filtered image satisfies In this case, the adaptation process becomes E-step: M-step: 36

37 Stein s Unbiased Risk Estimator (SURE) What is the difference? Clean: Pre-filtered: 37

38 38

39 39

40 40

41 41

42 42

43 43

44 44

45 45

46 46

47 47

48 EM adaptation is - a method to combine generic database and the noisy image EM adaptation swings between - Generic database - When noise is extremely high - When patches are relatively smooth - Where there are insufficient training samples - Noisy image - When there are sharp edges in a patch - When there are enough training samples 48

49 [1] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Image Denoising by Mixture Adaptation, submitted to IEEE Trans. Image Process [2] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Image Denoising by Targeted Databases, IEEE Trans. Image Process [1] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Patch-based Image Denoising by EM-adaptation, in Proceedings of IEEE Global Conference on Signal & Information Process. (GlobalSIP 15), 2015 [2] E. Luo, S.H. Chan, and T. Nguyen, "Image Denoising by Targeted External Databases," in Proceedings of IEEE Intl. Conf. on Acoustics, Speech and Signal Process. (ICASSP'14), [3] E. Luo, S.H. Chan, S. Pan, and T. Nguyen, "Adaptive Non-local Means for Multiview Image Denoising: Searching for the Right Patches via a Statistical Approach," in Proceedings of IEEE Intl. Conf. on Image Process. (ICIP'13), [4] E. Luo, S. Pan, and T. Nguyen, "Generalized Non-local Means for Iterative Denoising," in Proceedings of European Signal Process. Conf. (EUROSIP'12),

50 Image Denoising AGAIN!? 50

51 51

52 52

53 Gaussian mixture model 53

54 54

55 55

56 56

57 57

58 58