Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization

Size: px

Start display at page:

Download "Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization"

Derek Russell
6 years ago
Views:

1 Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization CS 650: Computer Vision Bryan S. Morse

2 Optimization Approaches to Vision / Image Processing Recurring theme: Cast vision problem as an optimization problem Use established optimization techniques Examples: Automated thresholding Fitting-based segmentation Edge-based segmentation using graphs Graph-cut region segmentation Snakes.

3 Bayesian Inference We want to know the real world... but we only have pictures of it The world causes the picture... but we lose information in the process and all kinds of things can go wrong So, can we infer the world from the (corrupted) image? Common approach: Bayesian Inference

4 Bayesian Reconstruction Example: Bayesian Image Reconstruction Reconstruction: Process of attempting to recreate the original signal given a corrupted one. Terms in Image Reconstruction: Scene: the real world Image: a (possibly corrupted) picture of a scene Image reconstruction attempts to recreate the scene from an image.

5 Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization Bayesian Reconstruction Likelihood Possible reconstruction Original Possible reconstruction Which possible reconstruction is more likely to have produced the original image when corrupted?

6 Bayesian Reconstruction Knowledge About the Corruption Process Knowledge about the corruption process puts limits on reconstruction. Usually thought of as fitting the data : the reconstructed image can t vary too much from the original corrupted image. Key: Can evaluate how plausible any solution is given the input

7 Bayesian Reconstruction Prior Knowledge Possible Original Possible reconstruction reconstruction Which possible reconstruction seems better to you?

8 Bayesian Reconstruction Knowledge About Properties of the Original Scene Possible general properties: Generally smooth A few scattered rapid transitions Possible specific properties: Known scene contents (subject, anatomy, etc.) Other related images/scenes (video frames, other views, etc.) Key: Can evaluate how plausible any solution is at all

9 Review: Conditional Probablities and Bayes Theorem Notation: Probabilities The probability of discrete event A occurring is P(A) The continuous random variable x has a probability density function (pdf) p(x) For vector-valued random variables x, we write this as p(x) Properly speaking, it s really the probability that some specific variable X has the value x: p(x = x) but we usually just assume we know what variable x refers to.

10 Review: Conditional Probablities and Bayes Theorem Conditional Probabilities We write the conditional probability of A given B as P(A B) This means the probability of A given B or given B, what is the probability of A? Or for random variables, or p(x A) p(x y)

11 Review: Conditional Probablities and Bayes Theorem Bayes Theorem Generally: P(A B) = P(B A) P(A) P(B)

12 Bayesian Reconstruction (revisited) Bayesian Reconstruction Goal: for all possible reconstructed scenes f, find the one that maximizes p(f g) for measured image g. Problem: your knowledge of the imaging process tells you P(g f), but how do you determine P(f g)? Really Big Problem: How big is the space of all possible scenes f?

13 Bayesian Reconstruction (revisited) Bayesian Reconstruction P(f g) = P(g f) P(f) P(g) P(g f) is the likelihood or data term P(f) is the a priori knowledge (prior) P(g) is independent of f, and can thus be ignored when choosing best f P(f g) is called the a posteriori estimate This is often called maximum a posteriori (MAP) estimation.

14 Bayesian Reconstruction (revisited) Likelihood The likelihood is all about what can go wrong with the imaging process. Uncertainty is introduced by pixel noise, measurement noise, error, etc. Example: assuming white noise with standard deviation σ, the probability of getting noisy image g from scene f is P(g f) = i e (f i g i ) 2 /σ 2

15 Bayesian Reconstruction (revisited) Priors The priors are all about what we expect about good solutions. Example - penalize unsmooth images: P(f) = i k N(i) e (f i f k ) 2 where N(i) denotes the neighborhood of i. Notice that one large discontinuity in intensity is more likely than several smaller discontinuities. Results in piecewise-constant images with infrequent but rapid discontinuities. This is an example of what is formally called a Markov Random Field.

16 Bayesian Reconstruction (revisited) Bayesian Reconstruction Likehood: P(g f) = i e (f i g i ) 2 /σ 2 Prior: P(f) = i So, we want to choose f to optimize P(g f) P(f) = i k N(i) e (f i g i ) 2 /σ 2 e (f i f k ) 2 Looks like an expensive computation, right? i k N(i) e (f i f k ) 2

17 Bayesian Reconstruction (revisited) Bayesian Reconstruction (cont d) Optimizing this is the same as optimizing its logarithm: log e (f i g i ) 2 /σ 2 e (f i f k ) 2 i i k N(i) = log e (f i g i ) 2 /σ 2 + log e (f i f k ) 2 i i k N(i) = log e (f i g i ) 2 /σ 2 + log e (f i f k ) 2 i i k N(i) = (f i g i ) 2 /σ 2 + (f i f k ) 2 i i k N(i)

18 Bayesian Reconstruction (revisited) Example (cont d) Maximizing this is the same as minimizing its negation: (f i g i ) 2 /σ 2 + (f i f k ) 2 i i k N(i) }{{}}{{} fitting data prior

19 Bayesian Reconstruction (revisited) Bayesian Reconstruction If P(g f) and P(f) are negative exponentials, the process usually boils down to minimizing some function where data(f, g) + λ prior(f) data(f, g) penalizes reconstructions f that don t agree with the original image g prior(f) penalizes reconstructions that are a priori unlikely The weight λ controls the relative importance of the two

20 Other Bayesian Methods Other Bayesian Methods Many other vision techniques have this general framework of optimizing a function of the form data(f, g) + λ prior(f) data term drives the system to solutions that maximize fitting the image data prior term drives the system to desirable solutions (smooth, known shape, other known priors, etc.) Basic approach: set up the likelihood and prior terms, then choose a suitable optimization technique.

21 Other Bayesian Methods Balancing the Data and Prior Terms data(f, g) + λ prior(f) If λ is set too low, the data term dominates and the solution may overfit the noisy data. If λ is set too high, the prior term dominates and the solution may depend too much on the prior(s).

22 Optimization Techniques Optimization Since the space of all f to search is far too large, non-exhaustive optimization techniques must be used: Gradient-descent and variants (conjugate gradient, etc.) Simulated annealing Genetic or other evolutionary algorithms Graduated non-convexity All of these give good but not always the best possible solution. Note: sometimes there exist globally optimal polynomial-time solvers for what you want if so, big win!

23 Optimization Techniques Gradient Descent The simplest form of optimization is gradient-descent minimization. Idea: find minimum of function f by iteratively taking a step downhill. For one variable: x t+1 = x t γ df dx (xt) where γ controls the size of the step at each iteration. For two variables: x t+1 = x t γ df dx (xt) y t+1 = y t γ df dy (yt) Or, more generally for any function of a vector x: x t+1 = x t γ f (x t)

24 Optimization Techniques Implementation: Gradient Descent General form: x t+1 = x t γ f (x t ) Implemention is actually quite simple: grad = CalculateGradient(f,x); while (magnitude(grad) > convergence threshold) { x -= gamma * grad; grad = CalculateGradient(f,x); } The difficult part isn t implementing the minimization, it s differentiating the function you re trying to minimize.

25 Optimization Techniques Simulated Annealing Simulated annealing tries to overcome two problems of gradient-descent: having to differentiate the criterion function getting stuck in local minima Basic approach: randomly generate a small change if better, keep it with some probability, allow a change to a worse solution in order to explore more greatly the solution space (and decrease this probability as you go)

26 Optimization Techniques Genetic Algorithms Encode possible solutions as strings. Start with a set of initial potential solutions. For each iteration (each generation ), three possible operations: reproduction test each string to see how good it is if good, keep it and produce more if bad, delete it crossover with some probability, splice pieces of different existing strings together mutation with some probability, randomly change a character Similar in spirit to simulated annealing but with the idea of occasionally merging parts of promising solutions

27 Optimization Techniques Graduated Nonconvexity Idea: smooth the function to get rid of local minima Problem: not the same function anymore! But the solution makes a good starting point for a less-smoothed form of the function!

28 Optimization Techniques Graduated Nonconvexity (cont d) Algorithm: Choose a large smoothing factor s so that the smoothed function has only one minimum. Choose a starting point and iterate until convergence: 1. use gradient descent to find the minimum of this smoothed function 2. use this minimum as the starting point for the next iteration 3. reduce s and smooth the original function again by s

29 Optimization Techniques Optimization and Iterative Algorithms All of the approximate optimization techniques we ve talked about involve iteration. Each employs a specific strategy for finding local/global minima (or hopefully global minima). Nearly every iterative vision algorithm employs similar strategies, explicitly or not look for it!

Regularization and Markov Random Fields (MRF) CS 664 Spring 2008

Regularization and Markov Random Fields (MRF) CS 664 Spring 2008 Regularization in Low Level Vision Low level vision problems concerned with estimating some quantity at each pixel Visual motion (u(x,y),v(x,y))