5. Mathematical tools: overview and examples in astronomy

Transcription

1 Master ISTI / PARI / IV Introduction to Astronomical Image Processing 5. Mathematical tools: overview and examples in astronomy André Jalobeanu LSIIT / MIV / PASEO group Jan lsiit-miv.u-strasbg.fr/paseo PASEO

2 Mathematical tools: overview and examples in astronomy Direct tools (basic ops & filtering) Modeling (instrument/object description) Probability theory and statistics Introduction (pdfs, estimation, detection) Bayesian inference & graphical models Transforms and representations Projections Frequency Multiresolution Multi- shape and resolution Multidimensional data Functional optimization Mathematical morphology

3 Direct tools: basic operations & filtering Focus on operations involving single pixel or their neighbors Understand the simplest tools for image processing See how filtering can effectively be used in astronomy

4 Single pixel (elementwise) operations Image Addition Frame co-addition in deep field imaging (increase sensitivity) Image Subtraction Sky background and detector bias subtraction Registered images subtraction for comparison/detection purposes Image Division Flat-field correction Pixel value transformation thresholding functions Look-up tables for speed-up (integer values) Thresholding, contrast enhancement, dynamic range transforms Vector-valued pixels: change of basis (multispectral)

5 Multiple pixel operations: kernel filtering, interpolation Neighborhood filters Linear: Finite Impulse Response (FIR) filters/masks (3x3, 5x5...) Convolution with a kernel (image space) Sharpen, Blur, 1 st /2 nd Derivatives, discretized operators (Laplacian...) Rank filters (min, max, median) Sort pixel values within the pixel neighborhood Denoising (impulse noise) More complex: filter and decision (comparison, threshold...) Denoising, Detection... Interpolation Bilinear, Key s bicubic (1 step) Spline (2 steps: prefiltering-prediction) Synthesis function interpolant function [Unser 95] bilinear interpolation 1 y 0 x 1

6 Filtering: processing representations Keep the signal, filter the noise Work in a different representation (transform: Fourier, wavelet...) to separate signal & noise signal noise Linear filtering Multiply by a factor ak depending on the coefficient index k e.g. Fourier transform, factor = function of spatial frequency e.g. Wiener filter for image deblurring Nonlinear filtering Apply a nonlinear function f of the coefficient e.g. thresholding Stationary: function f does not depend on the coefficient index e.g. Denoising via simple wavelet coefficient thresholding Adaptive: function fk depends on the coefficient index e.g. Deblurring via adaptive wavelet coefficient thresholding

7 Modeling tools Find out how to describe astronomical objects in 3D or 2D Know the basics of forward modeling principles: instrument and sensor description

8 Tools for object description Geometry Simple objects (e.g. lines, spheres, ellipsoids) Polygonal objects (e.g. planetary surface) Parametric functions (1D, 2D, 3D) Simple functions (e.g. circular or ellipsoidal Gaussian) Complex analytic functions (e.g. radial sigmoid) Non-analytic functions (e.g. 2D integration of a 3D model)

9 Tools for image acquisition modeling y R y v R u I p p A Δ Φ Δ Δ R x x R v u sampling grid Image space Frequency space Sampling theory [Nyquist 28, Shannon 49] Regular sampling: regular grids (rectangular, hexagonal lattice) Generalized sampling on irregular grids Object-based rendering: model discretization Physics: optics, turbulence, motion, sensor Product of MTFs Analytic expressions or numerical models image spectrum Object-based rendering Hexagonal lattice Probability & statistics (noise) Airy pattern

10 Probability theory & statistics Give a short introduction to probability theory and the related statistical tools Understand the principles of Bayesian inference and estimation Get acquainted with complex modeling tools via graphical models

11 Introduction to probabilistic tools random variables and pdfs Random variables: stochastic processes probability density function (pdf) distribution theory constraints: positive, normalized discrete or continuous variables conditional marginal Joint pdf of several variables x,y random var. sets (e.g. pixels, parameters Θ) Marginal pdf: P(y) = Z x P(x,y)dx P(data) evidence P(Θ) prior Conditional pdf: Bayes theorem: P(x y) = P(x,y)/P(y) P(y x) = P(x y)p(y)/p(x) P(data Θ) likelihood P(Θ data) posterior

12 Statistical tools and function fitting Function fitting principles: Provide a parametric function (the model) Provide an error function Squared difference in the Gaussian Case Explicit -log P(datak Θ) in general Robust functions allowing for impulse noise statistical estimation: maximum likelihood arg maxθ P(data Θ) P(data Θ) =Πk P(datak Θ) indep. assumption Minimize the total error (cost function): regression General case: iterative method (see functional optimization) Special cases: closed-form solution Some examples: Smooth background extraction Model: linear or polynomial function of the coordinates Closed-form solution: low order moments Star extraction and aperture photometry Model: 2D Gaussian ( PSF); free parameters: location (x,y), intensity L Approx closed-form solution: 1 st order moments for (x,y) centroid, mean for L (the exact method is iterative) L = I i j i, j Gaussian noise least squares x = ii i j /L i, j y = i, j j I i j /L

13 Graphical models Independence properties of random variables Stochastic independence: P(x,y) = P(x) P(y) Conditional independence: P(x,y z) = P(x z) P(y z) Dependence graphs or graphical models Each node is a random variable (or a set of variables) Edges represent dependencies (stochastic independence no connexion between nodes) Directed graphs or Bayesian networks: set of converging arrows = conditional pdf (causality) Joint pdf: Undirected graphs or Markov networks: graph separation conditional independence Joint pdf: P(X) = t P(X t X π(t) ) P(X r ) parents P(X) = 1 Z e c V c (X c ) r roots clique potentials X1 X1 V12 [Jordan, MacKay] X2 X2 X4 Bayes X4 V23 X3 V34 X3

14 Markov Random Fields (MRF) Conditional dependence assumption: P(pixel all others) = P(pixel neighbors) Define a neighborhood and clique system [Besag 86, Geman 84] Define the clique potentials Vi Hammersley-Clifford theorem X MRF P(X) = e -U(X) /Z undirected graphical models (U: Gibbs energy = sum of Vi, Z: normalizing const.) first order neighborhood first order cliques A pixel only depends on its neighbors! Sampling from P(X) using Metropolis or Gibbs Use of Markov Chain Monte Carlo (MCMC) methods for inference & estimation

15 Hidden & auxiliary variables Hidden variables Underlying process explaining the complexity of a model Causality: directed graphical models (e.g. Gaussian mixture) Examples: class labels in segmentation, line process in adaptive regularization (denoising/deblurring), missing data (bad CCD pixels) Auxiliary variables Created to facilitate modeling, optimization or sampling Not necessarily causal relation Examples: edge-preserving regularization (denoising/deblurring) Joint P(X,M) Μ X 1 3 λ,δ B x X Y spatially variable model map M 2 Hidden variables: model map (optimal representations) X B x Edge processes B x B y

16 Markov trees & multiscale dependencies Encode dependencies between scales wavelet coefficients propagate through scales... Markov tree structure: parent/children dependencies Hidden variables: encode the implicit dependence the variance propagates, but the value can change Joint subband histogram P(ξ t, ξ π(t) )? Hidden Markov Tree (HMT) Multiscale, discrete Gaussian mixture model ξ π(t) ξ t wavelet transform P(s t,s π(t) ) discrete probabilities P(ξ t s t ) Gaussian pdf

17 Bayesian inference Express the joint posterior pdf Build the full joint pdf: entire model + observed data e.g. use graphical models P(entire model data) full joint pdf Marginalization: eliminate the nuisance variables elimination algorithm, integration (Laplace approx.) Result: posterior marginal P(Θ data) Usually there is no closed-form solution Monte Carlo Approx. (sample from P) Gaussian Approx. Bayesian vs. classical Mode = Maximum A Posteriori (MAP) approach: use prior P(Θ) found by optimization of -log P Uncertainties = Covariance Matrix [Σ] provided by the second derivatives of -log optimum Hypothesis testing, model assessment, model selection & mixture... Θ Θ z z Y Y

18 Bayesian estimation: when you have to make a decision Bayes Estimator: Statistics: function of the data θ=f(y) Cost function: estimation error C(θ,θ) Bayes risk: estimator error EP(θ,Y)[C(θ,θ)] Bayes estimator: arg minθ EP(θ,Y)[C(θ,θ)] Y=data cost fuction related estimator MAP Maximum A Posteriori PM Posterior Mean [Marroquin 85] In practice: Empirical Bayesian estimation (Laplace or saddlepoint approx.) [MacKay]

19 Transforms & representations Understand how changing representations can help model information in astronomical images Find out how to efficiently separate the signal from the noise Grasp the principles and properties of several multiresolution transforms

20 Projections onto various subspaces Project images, scalar or vector-valued pixels, geometric models, parameters... Simple projections: enforce constraints Strong support constraints (e.g. reduced spatial support) Strong range constraints (e.g. positivity) Projections onto convex sets Change of basis Orthonormal change of basis (e.g. Fourier, wavelets) Arbitrary change of basis (e.g. frames, biorthogonal wavelets) Overcomplete representations Arbitrary number of vectors (e.g. mixture of models)

21 Frequential representation (Fourier) Modeling Spatial convolution = single Fourier coefficient multiplication diagonalization of convolution-based operators e.g. stationary self-similar processes ( 1/f power spectrum) Filtering Stationary & independent noise: same in the freq. space Apply factor according to spatial frequency: Noise filtering, deconvolution, enhancement Reconstruction Blind deconvolution methods (unknown image and psf) Interferometry Detection Template matching via cross-correlation (convol. matched filter)

22 Multiresolution analysis Set of closed nested subspaces of Approximation a j at scale j : projection of f on V j Detail d j at scale j : proj. of f on W j such that V j-1 = V j W j Basis functions: scaling functions & wavelets [Mallat, Vetterli, Daubechies, etc.] Galaxy details at different resolutions or scales Starck, Murtagh, Bijaoui Multiresolution analysis & wavelets tutorial

23 Multiscale Vision Model & applications [Bijaoui 95] scale space Scale-space isosurface visualization First 6 scales of à trous wavelet transform Establish interscale connexions: links between regions Keeping only significant coefficients (3σ) Segmentation in significant regions

24 Wavelets & sparse representations Sparse representation: Good approximation achieved by keeping only a small number of coefficients; Information concentrated in a few, high magnitude coefficients pixels noise coef noise approx. Wavelet pyramid: few significant coefficients w.r.t. original image image space wavelet space connection with image compression Asymptote E~N -1/2 Haar Symmlet-8 Approximation error vs. number of coefficients

25 Multi shape/resolution representations One wavelet is not enough! Different wavelets, different shapes & properties Detection theory: correlation with template function to detect Use multiple templates examples of 2D wavelets examples of 1D wavelets Optimal representation Find pixons in images: circular or elliptical objects [Pina & Puetter 93] Various scales, locations and intensities Detect various objects in noisy observations...the residual should be the observation noise Find such objects in spectra (IFS) or in transform spaces: optimal representation (information theory)

26 Curse of dimensionality Problem: high dimensionality of pixels (multi or hyperspectral data) Statistical learning: number of bins = N d number of needed samples increases exponentially! typical data in integral field spectroscopy: 24x24 pixels, d=284 16x16 pixels, d= x80 pixels, d=2000 Solution: find low dimensional subspaces & project

27 Representing multidimensional data Principal Component Analysis (PCA) Image covariance matrix, eigenvectors = principal variation axes Select largest eigenvalues and the related eigenvectors Independent Component Analysis (ICA) Search for independent sources (non-gaussian!) Nonlinear manifold representation Deterministic/Probabilistic versions various modeling levels: with or without noise, single or mixture of principal components ICA nonlinear manifold

28 Functional optimization Understand why optimization is needed in most processing tasks Get to know the main optimization methods in multiple dimensions Know how to choose between various deterministic and stochastic algorithms

29 Why optimization is needed Direct, iterative methods No explicit definition of a forward model However, the solution is sought by an optimization procedure Constrained optimization of an objective or cost function; Simplest solution (e.g. max. entropy/smoothness) under data-related constraints: set-theoretic (e.g. positivity) or strong (e.g. data prediction) Most likely solution (e.g. least squares) under model-related constraints: set-theoretic (e.g. smoothness, bounds) or strong (e.g. normalization) Inverse, iterative methods Explicit forward modeling; no closed-form solution Solution given by optimizing a functional F: Probabilistic methods: F = - log posterior pdf Other methods: F = Distance(prediction, observations) Different kinds of optimization: image processing: image = arg min F parameter fitting: parameters = arg min F in some cases, functional F = processed image (e.g. CLEAN)

30 Deterministic methods Optimization without derivatives Simplex method (linear programming) Principal axis method (Brent) Gradient-based methods Steepest descent ( simple functions) Use line optimization only Newton s method (convex functions) Use the Hessian (2nd derivative) Quasi-Newton & modified Newton methods Use an approximation/correction of the Hessian Gauss-Newton methods (sum of squares) Conjugate gradient: linear/nonlinear (contours quadratic form) Linear case for quadratic form: conv. finite number of steps

31 Deterministic optimization: problems, solutions Objective function: high dimensionality & nonlinearity! Low computational efficiency Choose an appropriate optimization method! Take into account the dependence structure Choose the representation where the dependence is minimized (diagonalization) Perform single variable optimization whenever possible e.g. Iterative Conditional Modes (ICM) Multiple local optima, nonconvexity Multigrid optimization Use coarse to fine approximations of the objective function, initialize each level with the result of the previous level Graduated nonconvexity & approximations Auxiliary variables Stochastic methods... multigrid

32 Example - quasi-newton optimization Iterative optimization scheme: object & camera param. estimation! Linearize the intensity (rendering): I(S,Θ) I(S 0,Θ 0 ) + I S S S 0 ( ) + I Θ Θ Θ 0 ( ) objective function approx. by a quadratic form! optimization of S, Θ using a conjugate gradient! Result: used to initialize the next iteration! Convergence: small variation of S, Θ 1 2 Each step is simpler than the original problem (linear optimization) 3 S = geometric object model Θ = camera parameters

33 Auxiliary variable methods Half-quadratic extensions φ-functions : quadratic near 0, linear or log-like at Additive extension φ(u)=infb (b-u) 2 +ψ1(b) Multiplicative extension φ(u)=infb bu 2 +ψ2(b) Alternate optimizations w.r.t. b and u [Charbonnier 94] Missing data, augmented process: Expectation-Maximization [Baum 72, Dempster 77] Nonlinear regularization (e.g. deblurring or denoising) E M Q simpler to optimize than P(ϑ Y) complex P(ϑ Y) augmented process P(z, ϑ Y) z ϑ

34 Stochastic methods Exception: no optimization Bayesian inference: full pdf P or its properties Monte Carlo methods Sample from the distribution P e-u(x) where U=objective function Compute the mode and higher order moments: approx. the pdf P Simulated annealing [Kirkpatrick 83] Allow for wrong directions: escape from local optima Sample by slowly decreasing the temperature T in P e -U(X)/T e.g. Deblurring, blind deconvolution, optimal representations via nonlinear fitting

35 Genetic algorithms [Holland 75] Principle: Choose initial population Repeat crossover mutation - Evaluate the individual fitnesses of a certain proportion of the population - Select pairs of best-ranking individuals to reproduce - Apply crossover operator - Apply mutation operator Until terminating condition Examples: Multispectral image classification [Petremand et al. 05] Fitting galaxy rotation curves, variable star period determination [Charbonneau 95]

36 Derivatives & partial differential equations Learn how to compute the derivatives needed in deterministic optimization methods See how some iterative processing techniques amount to solving partial differential equations

37 Computing derivatives why compute the derivatives? series A deterministic optimization algorithms help compute uncertainties parallel A Example: rendering a polygonal object deriv. of a pixel intensity W w.r.t. model parameters Pj B C C A = C B B A B 1 B 2 C C A = C B 1 B 1 A + C B 2 B 2 A π i P j A π i W A 2D polygon vertex polygon area pixel intensity Basic tool: the chain rule... How does a change in the model affect the predicted image intensity?

38 Partial Differential Equations in image processing general form Image denoising Isotropic diffusion: heat equation t I = I e.g. semi-implicit scheme Related to physics: each pixel has a temperature Gaussian smoothing (convolution) of the image Anisotropic diffusion Smooth only along object edges Adaptive smoothing, edge-preserving, sharper details Connexions with nonlinear regularization noisy image isotropic anisotropic

39 Mathematical morphology Recall of the basic principles of mathematical morphology Understand how simple morphological tools can be applied to astronomical images

40 Principles [Matheron, Serra 82] Set-theoretic approach Structuring element (neighborhood system) - shape Sets of values, rank operations (e.g. min, max) Basic Functions Erosion: min {In} Dilation: max {In} Morphological tools Combination opening (erosion-dilation) / closing (dilation-erosion) / top-hat (opening-subtract.) Hit-and-miss, skeleton, reconstruction, thinning... Morphological filters Smoothing: open-close Gradient, Laplacian...

41 Application examples Star extraction and mapping Star/galaxy classification Multispectral image segmentation... top-hat transform: erosion, dilation, difference [Candéas et al, 1997] original image Abell 3698 top-hat: background & halo removal open-close: morphological smoothing

42 Further reading An interactive image processing course Mathworld (Wolfram research) IRIS tutorial on CCD image processing, C. Buil ICCV 03 course on learning & vision (Blake, Freeman, Bishop, Viola) Various wavelet resources MacKay Book on information theory, inference & learning Introduction to graphical models & bayesian networks Statistical learning, decision, graphical models (M. Jordan)