GENERAL. CSE 559A: Computer Vision STEREO ROUNDUP STEREO ROUNDUP

Transcription

1 CSE 559A: Computer Vision GENERAL roblem Set 3 Due Thursday. roject roposals Deadline Extended to 11:59 M Tuesday Oct 31st. Submitted through blackboard. 2-3 aragraphs. Can be DF / Text File / ut directly in the text box. ush back SET 4/5 by two days each. SET 4 Will be osted Tuesday (will incle stuff we do next week. Will be e two weeks from then. Fall 2017: T-R: Lopata 101 Instructor: Ayan Chakrabarti (ayan@wustl.e. Staff: Abby Stylianou (abby@wustl.e, Jarett Gross (jarett@wustl.e SET 5 will be pushed back. Now e Thu a er TG break instead of Tue. New schele is online. Oct 26, 2017 Last time: "message" passing / augmented cost Consider the case when S(d, : 0 if d = 1 if d = 1 2 otherwise. Can we do this efficiently? Need to go through each line sequentially. But can go through all lines in parallel. C [x, d] = C[x, d] + min C [x 1, ] + λs(d, But what about d? Do we need to do minimization for every d independently? Note: It doesn't matter if we add / subtract constants to all d's: Why not? C[x, d] with C[x, d] + C 0 [x] C [x, d] with C [x, d] + C 0 [x] C [x, d] = C[x, d] + min C [x 1, ] + λs(d, Because the minimization will always be over d. You are never comparing C[ x 1, d 1 ] with C[, ]. x 2 d 2

2 Step 1 (Simplify: Replace [x 1, ] with The MAXIMUM value for min d [x 1, ] + S(d, is 2. Step 2: This means that for every value of d, we just need to consider four values. min d [x 1, ] + S(d, is the min of 2 [x 1, d 1] + C [x, d] = C[x, d] + min [x 1, ] + S(d, S(d, = 0 if d = 1 if d = 1 2 otherwise C [x 1, d ] = [x 1, ] [x 1, ] 1 [x 1, d + 1] + 1 C min d C d min d [x 1, ] + S(d, 2 [x 1, d 1] + 1 [x 1, d + 1] + 1 [x 1, d] C [x, d] = C[x, d] + min [x 1, ] + S(d, S(d, = is the min of Can do this in parallel with matrix operations for all d and all lines. Full algorithm in paper: Hirschmueller, Stereo rocessing by Semi-Global Matching and Mutual Information, AMI if d = 1 if d = 1 2 otherwise [x 1, d] SGM Algorithm Averages along four directions: Bur C C [n, d] = C[n, d] + min C [n [1, 0 ] T, ] + λs(d, C [n, d] = C[n, d] + min C [n + [1, 0 ] T, ] + λs(d, C [n, d] = C[n, d] + min C [n [0, 1 ] T, ] + λs(d, C [n, d] = C[n, d] + min C [n + [0, 1 ] T, ] + λs(d, d[n] = arg min C [n, d] + C [n, d] + C [n, d] + C [n, d] is still smoothing the original cost. d SGM Algorithm Averages along four directions: C [n, d] = (C[n, d] + C [n, d] + C [n, d] + C [n, d] + min C [n [1, 0 ] T, ] + λs(d, [n, d] = (C[n, d] + [n, d] + [n, d] + [n, d] + [n [1, 0, ] + λs(d, C C C C min C ] T [n, d] = (C[n, d] + [n, d] + [n, d] + [n, d] + [n [1, 0, ] + λs(d, C C C C min C ] T [n, d] = (C[n, d] + [n, d] + [n, d] + [n, d] + [n [1, 0, ] + λs(d, C C C C min C ] T Wouldn't this be better? Why not this? Because this is a circular definition.

3 Loopy Belief ropagation (one version [n, d] = (C [n, d] + min [n [1, 0 ] T, ] + λs(d, [n, d] = (C [n, d] + min [n [1, 0 ] T, ] + λs(d, [n, d] = (C [n, d] + min [n [1, 0 ] T, ] + λs(d, [n, d] = (C[n, d] + [n, d] + [n, d] + [n, d] + [n [1, 0, ] + λs(d, C t C t C t min ] T Other methods for discrete minimzation---based on "Graph Cuts". SGM / Loopy B: Generalize that there is an exact solution for a chain. Graph Cuts (with expansions / swaps: Generalize that there is an exact solution if only two values of d. D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV Do this iteratively More generally, at time step t, pass a message from node n to n, based on all messages n has at that time, except for the message from n. Read more: - Yedidia, Freeman, Weiss, "Understanding belief propagation and its generalizations," IJCAI 2001 (Distinguished aper - Tappen & Freeman, "Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters", ICCV 2003.

4

5

6

7 NEXT TIME This is only two view stereo. General 2D Motion: Optical Flow More complicated versions incle finding correspondences along multiple cameras: Multi-view Stereo Grouping ixels Then onto ML