Computer Vision : Exercise 4 Labelling Problems

Transcription

1 Computer Vision Exercise 4 Labelling Problems 13/01/2014 Computer Vision : Exercise 4 Labelling Problems

2 Outline 1. Energy Minimization (example segmentation) 2. Iterated Conditional Modes 3. Dynamic Programming 4. Block-wise ICM 5. MinCut 6. Equivalent transformations + α-expansion 7. Row-wise stereo (Cyclopean view) 8. Assignments: a) Segmentation b) Stereo 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 2

3 Energy Minimization (Segmentation) 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 3

4 Energy Minimization 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 4

5 Iterated Conditional Modes Idea: choose (locally) the best label for the fixed rest [Besag, 1986] Repeat: extremely simple, parallelizable coordinate-wise optimization, does not converge to the global optimum even for very simple energies 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 5

6 Dynamic Programming Suppose that the image is one pixel high a chain The goal is to compute 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 6

7 Dynamic Programming example 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 7

8 Dynamic Programming General idea propagate Bellman functions by The Bellman functions represent the quality of the best expansion onto the processed part. 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 8

9 Dynamic Programming (algorithm) Time complexity is the best predecessor for -th label in the -th node 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 9

10 Iterated Conditional Modes (again, but now 2D) Fix labels in all nodes but for a chain (e.g. an image row) Before (simple) The auxiliary task is solvable exactly and efficiently by DP The overall schema iterate over rows and columns until convergence 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 10

11 MinCut 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 11

12 MinCut for Binary Energy Minimization 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 12

13 Search techniques 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 13

14 α-expansion 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 14

15 α-expansion After α-expansion we have but we need in order to transform it further to MinCut. What to do? 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 15

16 Equivalent Transformation (re-parameterization) Two tasks and are equivalent if holds for all labelings. equivalence class (all tasks equivalent to ). Equivalent transformation: 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 16

17 Equivalent Transformation Equivalent transformation can be seen as vectors, that satisfy certain conditions: 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 17

18 Back to α-expansion Remember out goal: It can be done by equivalent transformations. 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 18

19 Row-wise stereo Pixel of the left image should be labelled by disparity values: Constraint: d i + 1 d i 1 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 19

20 Row-wise stereo (Cyclopean view) Symmetric definition transform the coordinates We are searching for a 4-connected path. Constraint: d i + 1 = d i ± 1 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 20

21 Assignments 1. Segmentation: a) Binary Segmentation with the Ising Model MinCut. b) Possible extensions: multi-label segmentation (with ICM, DP, α-expansion [1]), more complex appearance models [2], contrast dependent edge potentials [3]. 2. Stereo: a) Block Matching, row-wise stereo. b) Possible extensions: row-wise Iterated Conditional Mode, more complex data-terms (e.g. Normalized Cross- Correlation), global solutions by MinCut [4], approximate solutions with α-expansion [1], re-parameterization [4]. a) implemented (can be used as a template), b) assignments Deadline per an Dmytro.Shlezinger@... 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 21

22 Literature [1] Boykov, Veksler, Zabih: Fast Approximate Energy Minimization via Graph Cuts [2] Rother, Kolmogorov, Blake: GrabCut Interactive Foreground Extraction using Iterated Graph Cuts [3] Boykov, Jolly: Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images [4] Ask me 13/01/2014 Computer Vision : Exercise 4 Labelling Problems 22