Robert Collins CSE586 CSE 586, Spring 2009 Advanced Computer Vision

CSE 586, Spring 2009 Advanced Computer Vision Dynamic Programming

Decision Sequences Let D be a decision sequence {d1,d2,,dn}. (e.g. in a shortest path problem, d1 is the start node, d2 is the second node you visit, and so on) Assume a cost function that tells how good a sequence of decisions is (e.g. could be the length of the path) Dynamic programming is a way of efficiently finding the optimal decision sequence, that is, the one with the lowest cost. (efficient = without having to compare all possible sequences)

Robert Collins Sample Graph Redrawn from http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html Nodes. Directed edges with weights (costs). Start state (A). End state (J). Getting from A to J is an example of a sequence of decisions. At each node, you have to decide which arc to follow to get to another node. 2 2 2 1 1 3 3 3 3 3 5 6 6 A B C D E F G H I J 7

Sample Graph What are some paths from A to J in this graph? Each has an associated cost. A 2 2 B C 6 3 2 7 1 E F 6 3 3 1 H I 3 J D 5 G 3 Examples: A B E H J A C F H J A D E I J (Cost 2+7+1+3 = 13) (Cost +2+6+3 = 15) (Cost 2+++ = 1)

How Many Paths? Enumeration of paths in this example H J I E F G E F G B C D B C D B C D B C D B C D B C D A A A A A A A A A A A A A A A A A A Note, the tree structure here is only to methodically generate and visualize all paths. We are not doing tree search.

What is Shortest Path? J 3 H I 1 6 3 3 3 E F G E F G 7 3 2 1 6 5 7 3 2 1 6 5 B C D B C D B C D B C D B C D B C D 2 2 2 2 2 2 2 2 2 2 2 2 A A A A A A A A A A A A A A A A A A 13 11 10 15 15 12 1 1 13 17 15 1 13 13 10 15 15 1 We could evaluate cost for every path, then choose smallest. However, this would be intractable for larger graphs. Note: there are two solutions in this example.

Principle of Optimality If set of decisions D is optimal, then all proper subsequences of D are also optimal e.g. if A-B-C-D-E is shortest path from A to E, then B-C-D must be the shortest path from B to D Note: the converse is not true 2 B 1 C 1 A D 3 E 2

Verify Principle of Optimality To verify that the principle of optimality holds in a given problem, assume D is optimal is true, but that all subsequences are optimal is false, and try to derive a contradiction. We will use shortest path as an example: Suppose D={p1,p2,...,pn} is shortest path from p1 to pn, and that it contains a nonoptimal subpath {i,v1,,v2,j}. If {i,v1,,v2,j} is nonoptimal, then there is a shorter path from i to j, call it {i,z1,,z2,j}. But then {p1,...,i,z1,,z2,j,...,pn} would be a shorter path from p1 to pn than {p1,...,i,v1,,v2,j,...,pn} Therefore D was not the shortest path. Contradiction

Principle of Optimality The Principle of Optimality typically holds for cost functions that monotonically increase or decrease with each new decision For example, sums or products of all positive or all negative numbers.

Principle of Optimality is Source of Efficiency 3 J 1 H I E F G E F G B C D B C D B C D B C D B C D B C D A A A A A A A A A A A A A A A A A A Consider possibilities for subpath E J E H J : length E I J : length 8 principle of optimality

Principle of Optimality is Source of Efficiency 3 J H I 1 3 3 6 3 E F G E F G B C D B C D B C D B C D B C D B C D A A A A A A A A A A A A A A A A A A Savings would be more dramatic for larger problems. Can often reduce an exponential problem, e.g. O(2 n ), to a polynomial time one, e.g. O(n 2 )

Robert Collins MultiStage Graphs 2 2 2 1 1 3 3 3 3 3 5 6 6 A B C D E F G H I J Stage 1 Stage 2 Stage 3 Stage Stage 5 Has a natural 1D ordering by stage number related to concepts of partial ordering and topological sort Our sample graph has a special structure.

DP on MultiStage Graphs An obvious approach is to optimize multistage paths one stage at a time Forward approach: we can assume at stage k that we already know the optimal path from each node in stage k+1 to the goal state. Backward approach: we can assume at stage k that we already know the optimal path from the start state to each node in stage k-1.

DP on MultiStage Graphs Forward approach: assume at stage k that we know the min cost path from each node in stage k+1 to the goal state. R1 MinCost(R1,Goal) Cost(Q,R1) R2 Q Cost(Q,R2) MinCost(R2,Goal) Goal Cost(Q,Rn) Rn Stage k Stage k+1 MinCost(Rn,Goal) MinCost(Q,Goal) = Min(Cost(Q,R1)+MinCost(R1,Goal), Cost(Q,R2)+MinCost(R2,Goal),, Cost(Q,Rn)+MinCost(Rn,Goal))

DP on MultiStage Graphs Backward approach: assume at stage k that we know the min cost path from each node in stage k-1 to the start state. MinCost(Start,P1) P1 Cost(P1,Q) Start MinCost(Start,P2) P2 Cost(P2,Q) Q Cost(Pn,Q) MinCost(Start,Pn) Pn Stage k-1 Stage k MinCost(Start,Q) = Min(Cost(P1,Q)+MinCost(Start,P1), Cost(P2,Q)+MinCost(Start,P2),, Cost(Pn,Q)+MinCost(Start,Pn))

DP on MultiStage Graphs Forward approach: we can assume at stage k that we already know the optimal path from each node in stage k+1 to the goal state. However, we then solve backwards, considering optimal path from nodes in stage n-1 to goal, then from stage n-2 to goal, and so on, until we find optimal path for the single start node in stage 1. Similarly, the Backward approach involves solving in the forward direction, starting at stage 1 and proceeding in increasing stages until reaching the goal node in stage n.

Example work through on board A 2 2 B 6 3 C 7 2 1 E F 6 3 3 1 H I 3 J D 5 G 3

Topological Ordering Although we have been looking at multistage graphs, it is not stages that matter. What is important is that we impose a topological ordering on the nodes such that Vi comes before Vj for i < j. A 2 2 B C 6 3 2 1 E F 6 3 3 1 H I 3 J D within a stage, we can impose an arbitrary ordering (C < B < D) 5 G 3 F < H H < J

Imposing Order DP can be used on more general graphs if we can impose a topological ordering. Directed Acyclic Graphs (DAG) can be trivially topologically ordered. We can impose an ordering on arbitrary graphs by some traversal method such as breadth-first-search (BFS) or depth-first search (DFS). However, you have to verify that they impose an ordering that makes sense for the problem being considered. Coming up with a topological ordering is often the hardest part of dynamic programming.

Edit Distance Problem Statement: given two strings s1 and s2, edit distance is the minimum number of operations needed to change s1 into s2, where the valid operations are 1) change one letter to another 2) insert one letter 3) delete one letter Example: astrea to stereo has edit distance 3 0 astrea 1 strea ; delete letter a 2 sterea ; add letter e 3 stereo ; change a to o

Edit Distance Ordering An element of our graph is a pair (Ai,Bj) where Ai is a prefix string of s1 with length i and Bj is a prefix string of s2 with length j. Examples: (s1= astrea, s2= stereo ) (A2, B3) = ( as, ste ) (A, B1) = ( astr, s ) (A0, B0) = (, ) [null strings] etc...

Edit Distance Ordering For each set of prefix strings, there is a natural order A0 < A1 < A2 <... < An B0 < B1 < B2 <... < Bm Examples: (s1= astrea, s2= stereo ) (A2, B3) = ( as, ste ) (A, B1) = ( astr, s ) (A0, B0) = (, ) [null strings] etc... To extend to ordering of pairs, let (Ai,Bp) < (Aj,Bq) if p < q or ((p=q)&(i < j)) so ordering on B dominates ordering on A

Dynamic Time Warping Computing edit distance is an example of a more general method called Dynamic Time Warping You want to compare two elements of two sequences, each of which might have extra or missing elements. Sequence 1 A B C D Sequence 2 A B D

Dynamic Time Warping Think of shifting two pointers along each string, element by element left-to-right, and deciding whether to compare the current elements or to skip one or the other of them. Sequence 1 i 1 Sequence 2 i 2

Dynamic Time Warping The indices i1, i2 can be thought of as indices in a 2D table Sequence 1 Sequence 1 Sequence 2 i 1 i 2 Sequence 2 I 1, I 2

Edit Distance Recurrence Relation Consider two strings astrea and stereo (A6,B6) What local edits can we do? (we will work from end of string backwards, but could develop the other way, equivalently) Skip letter at end of A Skip letter at end of B Match/Change last letters of A and B cost to skip one letter cost to edit astre into stereo cost(i,j) = 1 + cost(i-1,j) cost to skip one letter cost to edit astrea into stere cost(i,j) = 1 + cost(i,j-1) cost to change letter cost to edit astre into stere cost(i,j) = 1 + cost(i-1,j-1) cost(i,j) = 0 + cost(i-1,j-1) no cost if both letters match!

Edit Distance Keep a 2D matrix C(0.. s1, 0.. s2 ) to hold partial edit distance values. C(i,j) = dist(s1(1..i),s2(1..j)). %boundary conditions C(0,0) = 0; C(i,0) = i; %cost of skipping i letters from start of s1 C(0,j) = j; %cost of skipping j letters from start of s2 %recursive relation C(i,j) = min([ C(i-1,j-1) + (s1(i)!=s2(j)), %match, cost is 0 if same, %cost is 1 if different C(i-1,j) + 1, %skip one letter from s1, cost is 1 C(i,j-1) + 1]); %skip one letter from s2, cost is 1 Now compute and memoize from smallest substrings to largest.

Edit Distance Compute and memoize from smallest substrings to largest. %recursive relation C(i,j) = min([ C(i-1,j-1) + f(i,j), %match, cost f(i,j)=0 if same, 1 if different C(i-1,j) + 1, %skip one letter from s1, cost is 1 C(i,j-1) + 1]); %skip one letter from s2, cost is 1 i-1,j-1 i,j-1 match cost = f(i,j) Skip letter from string 2 cost = 1 i-1,j Skip letter from string 1 cost = 1 i,j

note: i is col variable. j is row variable. Edit Distance Example f(i,j) Letter match scores C(i,j) matrix entries %recursive relation C(i,j) = min([ C(i-1,j-1) + f(i,j), %match, cost f(i,j)=0 if same, 1 if different C(i-1,j) + 1, %skip one letter from s1, cost is 1 C(i,j-1) + 1]); %skip one letter from s2, cost is 1

Edit Distance Example 1+1 0+1 (1) 1+1 f(i,j) Letter match scores C(i,j) matrix entries

Edit Distance Example 1+0 2+1 1+1 (1) (1) f(i,j) Letter match scores C(i,j) matrix entries

Edit Distance Example 2+1 3+1 3+1 +1 +1 5+1 5+1 6+1 1+1 2+1 3+1 +1 (1) (1) (2) (3) () (5) f(i,j) Letter match scores C(i,j) matrix entries

Edit Distance Example 1+1 2+1 1+1 1+1 1+1 1+0 2+1 2+1 3+1 2+1 +1 +1 5+1 (1) (1) (2) (3) () (5) 2+1 (2) 2+1 (2) 1+1 (1) 2+1 (2) 2+1 (3) () f(i,j) Letter match scores C(i,j) matrix entries

Edit Distance Example (1) (1) (2) (3) () (5) (2) (2) (1) (2) (3) () (3) (3) (2) (2) (2) (3) () () (3) (2) (3) (3) (5) (5) () (3) (2) (3) (6) (6) (5) () (3) (3) f(i,j) Letter match scores C(i,j) matrix entries Distance Is 3!

Edit Distance Example We now know that, upon making optimal edit decisions, our minimum edit distance is 3. However, we still don t know what the optimal sequence of decisions was. Two ways to recover the decision sequence: 1) recover what it was by reasoning backwards from the goal node. 2) annotate the decisions made as we fill in the cost table.

Edit Distance Example Alternative: Update algorithm to annotate decisions. %boundary conditions C(0,0) = 0; C(i,0) = i; decision is L(skip s1), go C(0,j) = j; decision is R(skip s2), go %recursive relation [C(i,j),argmin] = min([ C(i-1,j-1) + (s1(i)!=s2(j)), C(i-1,j) + 1, C(i,j-1) + 1]); If argmin==1 decision is M (match or change), go else if argmin == 2 decision is L (skip s1), go else if argmin == 3 decision is R (skip s2), go Endif

Edit Distance Example Memoize decisions. R R R R R R L L L L L L (1) (1) (2) (3) () (5) M M R R R R (2) (2) (1) (2) (3) () M M M R R R (3) (3) (2) (2) (2) (3) M M L M M R () () (3) (2) (3) (3) M M L M M M (5) (5) () (3) (2) (3) M M L L M R (6) (6) (5) () (3) (3) M M L L L M Letter match scores C(i,j) matrix entries

y x T x Example of DTW in Vision: Simple Stereo System z z y x P Epipolar line Left camera Right camera Same Y Coord!

Matching using Epipolar Lines Left Image Right Image For a patch in left image Compare with patches along same row in right image Match Score Values

Matching using Epipolar Lines Left Image Right Image Select patch with highest match score. Repeat for all pixels in left image. Match Score Values

Adding Inter-Scanline Consistency Problem: matching each left image patch independently along the right epipolar line can lead to errors. We would like to enforce some consistency (smoothness) among matches in the same row. We also have to take into account occlusions, which are patches seen in only the left or the right image, but not both.

Disparity Space Image First we introduce the concept of DSI. The DSI for one row represents pairwise match scores between patches along that row in the left and right image. Pixels along right scanline Pixel j Pixels along left scanline Pixel i C(i,j) = Match score for patch centered at left pixel i with patch centered at right pixel j.

Disparity Space Image (DSI) Left Image Right Image Dissimilarity Values (1-NCC) or SSD

Disparity Space Image (DSI) Left Image DSI Dissimilarity Values Enter each vector of match scores as a column in the DSI

Disparity Space Image Left scanline Right scanline

Start DSI and Scanline Consistency Assigning disparities to all pixels in left scanline now amounts to finding a connected path through the DSI End

Lowest Cost Path We would like to choose the best path. Want one with lowest cost (Lowest sum of dissimilarity scores along the path)???

Constraints on Path It is common to impose an ordering constraint on the path. Intuitively, the path is not allowed to double back on itself.

Ordering Constraint A B C D A B C A B C D D A B C D A B C D D A B C Ordering constraint and its failure

Dealing with Occlusions Left scanline Right scanline

Dealing with Occlusions Left scanline Right scanline Occluded from right scanline Match Match Match Occluded from left scanline

An Optimal Scanline Strategy We want to find best path, taking into account ordering constraint and the possibility of occlusions. Algorithm we will discuss now is from Cox, Hingorani, Rao, Maggs, A Maximum Likelihood Stereo Algorithm, Computer Vision and Image Understanding, Vol 63(3), May 1996, pp.52-567.

Cox et.al. Stereo Matching Occluded from right i-1,j-1 i-1,j Occluded from left match match Occluded from left i,j-1 Occluded Three cases: from right Matching patches. Cost = dissimilarity score Occluded from right. Cost is some constant value. Occluded from left. Cost is some constant value. i,j C(i,j)= min([ C(i-1,j-1) + dissimilarity(i,j) C(i-1,j) + occlusionconstant, C(i,j-1) + occlusionconstant]);

Start Cox et.al. Stereo Matching Recap: want to find lowest cost path from upper left to lower right of DSI image. End At each point on the path we have three choices: step left, step down, step diagonally. Each choice has a well-defined cost associated with it. This problem just screams out for Dynamic Programming! (which, indeed, is how Cox et.al. solve the problem)

Robert Collins Real Scanline Example DSI DP cost matrix (cost of optimal path from each point to END) Every pixel in left column now is marked with either a disparity value, or an occlusion label. Proceed for every scanline in left image.

Example Result of DP alg. Black pixels = occluded.

Result of DP alg Example Result without DP (independent decisions)

Other DP Solutions All shortest paths problem find shortest path between any two nodes. 0/1 knapsack problem subset selection traveling salesman problem n! choices