Thomas Abraham, PhD (tabraham1@hmc.psu.edu)
What is Deconvolution? Deconvolution, also termed as Restoration or Deblurring is an image processing technique used in a wide variety of fields from 1D spectroscopy to 3D biomedical imaging. It has been rigorously applied in the field of astronomy, because so much expense and time are spent in acquiring astronomical images. Raw Image Deconvolved
What makes an image? An image is produced by an instrument. It is not the same as the object but is some photocopy of the object. In mathematical terms, the image is the object convolved by the point spread function (PSF). Image = Object PSF Convolution The PSF describes what the instrument does to the object to produce an image.
Point Spread Function {PSF} Every lens element in the optical path alters the object (specimen) in some way. We call this the point spread function (PSF). The PSF of the system is a convolution of all of the PSF s of all the lens elements and apertures in the optical path.
Point Spread Function {PSF} For instance, an image of point source is not a point but a point spread in 3D space. Likewise, a macromolecular object can be considered as a large number of points arranged in 3D space. The image of that macromolecular object can be considered as the superposition of large number of points convolved with the PSF. Point Source Image
In mathematical terms, for each point, take the object and multiply it by the entire PSF centered at that point in the image and sum the results of the multiplication. Image(x,y,z) = Object(x,y,z )psf(xx,y-y,z-z )dx dy dz Convolution Theorem
Convolution Theorem Image = Object PSF Image(x,y,z) = Object(x,y,z )psf(xx,y-y,z-z )dx dy dz Whatever we record as the image intensity is the convoluted product of the PSF and the object intensity. Restoring the object intensity from the smear of PSF by some mathematical operations Deconvolution
The Fourier Transform From spatial domain to Frequency Domain Any image can be decomposed into a series of sine and cosine wave functions. i I(x) a i (cosk i x) ib i (sink i x) Amplitudes Phase
Fourier Transform of the elastin collagen fiber bundle in lung alveoli PSD FT Low frequency (Coarse structural details) High frequency (Fine structural details) Spatial Domain Frequency Domain Closely related to the diffraction pattern of the object
Fourier Transform, Examples Bad looking Vs Good Looking FT Dude (Big Labowski) FT Spatial Domain Frequency Domain
Fourier Transform, Examples Unidirectional Objects Vs Randomly Oriented Objects FT Frequency distribution is orthogonal FT Spatial Domain Frequency Domain
Fourier Transform, Examples Fine Vs Fuzzy Objects FFT FFT Spatial Domain Frequency Domain
Inverse Fourier Transform of the Fourier Transform Returns the original Image F -1 = A powerful mathematical tool for image processing
Deconvolution using Fourier Transformation F {Image} = F {Object PSF} It has been proved that the Fourier transform of the image is just the Fourier transform of object times the Fourier transform of the PSF in frequency domain F {Image} = F {Object} X F {PSF} So, to obtain the object, we simply divide Fourier transform of Image by the Fourier transform of the PSF F {Object} = F {Image} / F {PSF} This is known as Optical Transfer Function or OTF or CTF
By applying an inverse Fourier transform, We can restore object (specimen) Object (or sample) = F {Object} -1 Transforms back to spatial domain
Steps in Deconvolution Step 1: Acquire the image and obtain its Fourier transform Convoluted Image {Object PSF} Fourier Transform of Image
Steps in Deconvolution Step 2: Generate PSF and obtain its Fourier transform Point spread function Fourier transform of PSF
Steps in Deconvolution Step 3: Divide and inverse Fourier transform F -1 Object
Raw Image FT Deconvolved Image ( Restored Object) FT
F {Object} = F{Image} / F{PSF} OTF or CTF Object (or sample) = F.T. {Object} -1 Works perfectly if there is no noise in the image. But there is always noise
Various Sources of NOISE Photon (or shot) noise: Photon detection is a quantum mechanical event. So, it is subjected to random fluctuations (or random noise) which is governed by Poisson statistics. Electronic (thermal) noise: The light is detected electronically and all detectors have thermal noise. Bit noise: Digital images have noise due to the number of bits (~8-14) used during A/D conversion. Processing noise: The computer is operating on digital data with limited precision.
Relatively Noise Free Effect of noise in data Less Noise More Noise Each spatial frequency has an associated noise component resulting from various sources of the noise, and at high spatial frequencies (corresponds fine structural details) this can be very large.
Non-iterative Deconvolution F {Object} = F{Image} / F{PSF} OTF or CTF So simply dividing F{Image} by OTF will magnify noise components - this obscures real structural details. Alternate approach is to modify OTF (or CTF) on the basis of local signal levels Wiener filter.
Deconvolution using Wiener filter Here an extra noise dependent factor (K) is added to the OTF values, so that in noisy regions the divisor is larger, and the noisy components are not boosted. One major drawback here is that we don t actually know what is real signal and what is noise. This often results in rejecting genuinely real spatial information. Consequently the reconstructed image can be of lower resolution.
Deconvolution by Iteration 1) Take a guess at the object 2) Convolve guess (object) with the PSF. If PSF is unknown, guess the PSF too. 3) If blurred guess = image then stop iteration, otherwise refine guess. Richardson-Lucy algorithm (which includes Poisson Noise model) : This is also known as maximum likelihood approach.
xz plane A xy plane B C D E F Abraham et al. (2010) Micron
xz plane xy plane A B C Abraham et al. (2010) Micron
A B C D Abraham et al. (2010) Micron
Summary Imaging is a convolution of the object by the microscope s PSF and an image is a convoluted product of the object and PSF. Deconvolution attempts to restore the various spatial frequency components to something nearer what they were in the original object before being attenuated by the optical path.
Recommended Readings: Cannell, et al. (2006) Image Enhancement by Deconvolution, In Handbook of Biological Confocal Microscopy, Third Edition, Pawley, J. B, Ed., Springer Science + Business Media, LLC, New York Shaw, P. (1994) Deconvolution in 3-D optical microscopy, Histochemical Journal 26, 687-694. Agard, D. A and Sedat, J. W. (1983) Three-dimensional architecture of a polythene nucleus. Nature 302, 676-81. Sindelar, C.V and Grigorieff,N. (2011) An adaptation of the Wiener filter suitable for analyzing images of isolated single particles, J. Struct. Biol. 176, 60-74. Sigworth, et al., (2010) An Introduction to Maximum-Likelihood Methods in Cryo-EM, Methods in Enzymology, Volume 482, 263-294