1
Image Denoising AGAIN!? 2
A Typical Imaging Pipeline 2
Sources of Noise (1) Shot Noise - Result of random photon arrival - Poisson distributed - Serious in low-light condition - Not so bad under good light (2) Electronic Noise - Instability of voltage/current - Temperature fluctuation - Analog to digital error - Gaussian distributed Shot noise Electronic noise Simplified diagram illustrating the two sources of noise 3
Noise! Shot noise Anscombe transform Gaussian noise My work: Gaussian Noise! 5
Adaptive Image Denoising for my PhD 6
Image Denoising Consider an additive i.i.d. Gaussian noise model: Our goal is to estimate from where Our Approach: Maximum-a-Posteriori 7
MAP Framework Since the noise i.i.d. is Gaussian, the conditional distribution is Therefore, the MAP is 8
Image Priors Markov Random Field (80s) Gradients (80s) Total Variation (90s) X-lets (wavelet, contourlet, curvelet,, 90s) Lp norm (00s) Dictionary (KSVD, 00s) Example (00s) Non-local (BM3D, nonlocal means, 2005, 2007) Shotgun! (2011) Graph Laplacian (2012) 9
Patch-based Priors What is a patch? A patch is a small block of pixels in an image Why patch? What is patch-based prior? 10
Training a Patch-based Prior Typically, we train a patch-based prior from a large collection of images EM Algorithm e.g., Gaussian mixture: 11
Good Training Set 12
How good? Example: Text Image clean image noisy image BM3D [Luo-Chan-Nguyen, 15] (single image method) (use targeted training) 13
Challenge: (1)Finding good examples is HARD. (2)Finding a lot of good examples is EVEN HARDER. My work: Can priors be learned adaptively? Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 14
Our Proposed Idea 15
Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 16
Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 17
Half Quadratic Splitting General Principle [Geman-Yang, T-IP, 1995] The Algorithm: 18
Solution to Problem (1): Example Gaussian Mixture Model [Zoran-Weiss 11] If where 19
Solution to Problem (2): The solution to (2) is 20
Question 1 : How to SOLVE this optimization problem? For Gaussian Mixture: 21
Question 1 : How to SOLVE this optimization problem? (If we cannot solve this problem, then there is no point of continuing.) Question 2 : How to ADAPTIVELY learn a prior? Generic prior (from an arbitrary database) Specific prior (match the image of interest) 22
Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 23
Toy Example Imagine that: (a) Original generic database (A LOT of samples) (b) Ideal targeted database (A LOT of samples) (c) In reality, samples from targeted database are FEW!!! 24
25
EM Adaptation 26
EM Adaptation 27
EM Adaptation Classical EM: EM Adaptation: 28
EM Adaptation 29
EM Adaptation Classical EM: EM Adaptation: 30
EM Adaptation 31
EM Adaptation Classical EM: EM Adaptation: 32
EM Adaptation in the literature J. Gauvain and C. Lee, Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains, IEEE Transactions Speech and Audio Process., vol. 2, no. 2, pp. 291 298, Apr. 1994. D.A. Reynolds, T.F. Quatieri, and R.B. Dunn, Speaker verification using adapted gaussian mixture models, Digital signal process., vol. 10, no. 1, pp. 19 41, 2000. P.C. Woodland, Speaker adaptation for continuous density hmms: A review, in In ITRW on Adaptation Methods for Speech Recognition, pp. 11 19, Aug. 2001. M. Dixit, N. Rasiwasia, and N. Vasconcelos, Adapted gaussian models for image classification, in IEEE Conference Computer Vision and Pattern Recognition (CVPR 11), pp. 937 943, Jun. 2011. 33
Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 34
Image of interest update Generic database [Zoran-Weiss 11] 2 million 8x8 image patches Gaussian mixture model 35
EM Adaptation for Noisy Images i.e., denoise the image with a method you like. Assume the pre-filtered image satisfies In this case, the adaptation process becomes E-step: M-step: 36
Stein s Unbiased Risk Estimator (SURE) What is the difference? Clean: Pre-filtered: 37
38
39
40
41
42
43
44
45
46
47
EM adaptation is - a method to combine generic database and the noisy image EM adaptation swings between - Generic database - When noise is extremely high - When patches are relatively smooth - Where there are insufficient training samples - Noisy image - When there are sharp edges in a patch - When there are enough training samples 48
[1] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Image Denoising by Mixture Adaptation, submitted to IEEE Trans. Image Process. 2016. [2] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Image Denoising by Targeted Databases, IEEE Trans. Image Process. 2015. [1] E. Luo, S.H. Chan, and T. Nguyen, Adaptive Patch-based Image Denoising by EM-adaptation, in Proceedings of IEEE Global Conference on Signal & Information Process. (GlobalSIP 15), 2015 [2] E. Luo, S.H. Chan, and T. Nguyen, "Image Denoising by Targeted External Databases," in Proceedings of IEEE Intl. Conf. on Acoustics, Speech and Signal Process. (ICASSP'14), 2014. [3] E. Luo, S.H. Chan, S. Pan, and T. Nguyen, "Adaptive Non-local Means for Multiview Image Denoising: Searching for the Right Patches via a Statistical Approach," in Proceedings of IEEE Intl. Conf. on Image Process. (ICIP'13), 2013. [4] E. Luo, S. Pan, and T. Nguyen, "Generalized Non-local Means for Iterative Denoising," in Proceedings of European Signal Process. Conf. (EUROSIP'12), 2012. 49
Image Denoising AGAIN!? 50
51
52
Gaussian mixture model 53
54
55
56
57
58