Computer Vision Exercise 4 Labelling Problems 13/01/2014 Computer Vision : Exercise 4 Labelling Problems
Outline 1. Energy Minimization (example segmentation) 2. Iterated Conditional Modes 3. Dynamic Programming 4. Block-wise ICM 5. MinCut 6. Equivalent transformations + α-expansion 7. Row-wise stereo (Cyclopean view) 8. Assignments: a) Segmentation b) Stereo 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 2
Energy Minimization (Segmentation) 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 3
Energy Minimization 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 4
Iterated Conditional Modes Idea: choose (locally) the best label for the fixed rest [Besag, 1986] Repeat: extremely simple, parallelizable coordinate-wise optimization, does not converge to the global optimum even for very simple energies 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 5
Dynamic Programming Suppose that the image is one pixel high a chain The goal is to compute 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 6
Dynamic Programming example 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 7
Dynamic Programming General idea propagate Bellman functions by The Bellman functions represent the quality of the best expansion onto the processed part. 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 8
Dynamic Programming (algorithm) Time complexity is the best predecessor for -th label in the -th node 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 9
Iterated Conditional Modes (again, but now 2D) Fix labels in all nodes but for a chain (e.g. an image row) Before (simple) The auxiliary task is solvable exactly and efficiently by DP The overall schema iterate over rows and columns until convergence 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 10
MinCut 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 11
MinCut for Binary Energy Minimization 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 12
Search techniques 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 13
α-expansion 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 14
α-expansion After α-expansion we have but we need in order to transform it further to MinCut. What to do? 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 15
Equivalent Transformation (re-parameterization) Two tasks and are equivalent if holds for all labelings. equivalence class (all tasks equivalent to ). Equivalent transformation: 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 16
Equivalent Transformation Equivalent transformation can be seen as vectors, that satisfy certain conditions: 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 17
Back to α-expansion Remember out goal: It can be done by equivalent transformations. 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 18
Row-wise stereo Pixel of the left image should be labelled by disparity values: Constraint: d i + 1 d i 1 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 19
Row-wise stereo (Cyclopean view) Symmetric definition transform the coordinates We are searching for a 4-connected path. Constraint: d i + 1 = d i ± 1 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 20
Assignments 1. Segmentation: a) Binary Segmentation with the Ising Model MinCut. b) Possible extensions: multi-label segmentation (with ICM, DP, α-expansion [1]), more complex appearance models [2], contrast dependent edge potentials [3]. 2. Stereo: a) Block Matching, row-wise stereo. b) Possible extensions: row-wise Iterated Conditional Mode, more complex data-terms (e.g. Normalized Cross- Correlation), global solutions by MinCut [4], approximate solutions with α-expansion [1], re-parameterization [4]. a) implemented (can be used as a template), b) assignments Deadline 07.02.2014 per e-mail an Dmytro.Shlezinger@... 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 21
Literature [1] Boykov, Veksler, Zabih: Fast Approximate Energy Minimization via Graph Cuts. 2001 [2] Rother, Kolmogorov, Blake: GrabCut Interactive Foreground Extraction using Iterated Graph Cuts. 2002 [3] Boykov, Jolly: Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images. 2001 [4] Ask me 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 22