Image Segmentation. Luc Van Gool, ETH Zurich. Vittorio Ferrari, Un. of Edinburgh. With important contributions by

Transcription

1 Image Segmentation Perceptual and Sensory Augmented Computing Computer Vision WS 0/09 Luc Van Gool, ETH Zurich With important contributions by Vittorio Ferrari, Un. of Edinburgh Slide credits: K. Grauman, B. Leibe, S. Lazebnik, S. Seitz, Y Boykov, W. Freeman, P. Kohli

2 Topics of This Lecture Perceptual and Sensory Augmented Computing Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with path search

3 Slide credit: Kristen Grauman Grouping in Vision Perceptual and Sensory Augmented Computing Fast, bottom-up mechanisms to determine regions that belong together stepping stone between pixels/retina cell responses and scene interpretation. Psychophysics has listed features that seem to provoke perceptual grouping

4 Slide adapted from Kristen Grauman Similarity in appearance

5 Slide credit: Kristen Grauman Symmetry

6 Slide credit: Kristen Grauman Common Fate Image credit: Arthus-Bertrand (via F. Durand)

7 Slide credit: Kristen Grauman Proximity

8 The Gestalt School Grouping is key to visual perception `Belonging together is inferred from relationships The whole is greater than the sum of its parts Illusory/subjective contours Slide credit: Svetlana Lazebnik Occlusion Familiar configuration Image source: Steve Lehar

9 Gestalt Theory Gestalt: whole or group Whole is greater than sum of its parts Relationships among parts can yield new properties/features Psychologists identified series of factors that predispose set of elements to be grouped (by human visual system) Slide credit: B. Leibe I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees. Max Wertheimer ( ) Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung, Vol. 4, pp ,

10 Slide credit: B. Leibe Image source: Forsyth & Ponce Gestalt Factors These factors make intuitive sense, but are very difficult to translate into algorithms.

11 Slide credit: B. Leibe Figure-Ground Discrimination

12 The Ultimate Gestalt test Slide adapted from B. Leibe

13 Slide credit: Steve Seitz, Kristen Grauman Image Segmentation Goal: identify groups of pixels that go together

14 Slide credit: Svetlana Lazebnik The Goals of Segmentation Separate image into objects Image Human segmentation

15 Topics of This Lecture Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with path search

16 Slide credit: Kristen Grauman Image Segmentation: Toy Example These intensities define the three groups. We could label every pixel in the image according to which of these it is. 1 2 input image 3 black pixels intensity i.e. segment the image based on the intensity feature. What if the image isn t quite so simple? gray pixels white pixels

17 Input image Input image Slide credit: Kristen Grauman Pixel count Pixel count Intensity Intensity

18 Slide credit: Kristen Grauman Input image Pixel count Intensity Now how to determine the three main intensities that define our groups? We need to cluster.

19 Slide credit: Kristen Grauman Intensity 1 2 Goal: choose three centers as the representative intensities, and label every pixel according to which of these centers it is nearest to. Best cluster centers are those that minimize SSD between all points and their nearest cluster center c i : 3

20 Slide credit: Kristen Grauman Clustering With this objective, it is a chicken and egg problem: If we knew the cluster centers, we could allocate points to groups by assigning each to its closest center. If we knew the group memberships, we could get the centers by computing the mean per group.

21 Slide credit: Steve Seitz K-Means Clustering Basic idea: randomly initialize the k cluster centers, and iterate between the two steps we just saw. 1. Randomly initialize the cluster centers, c 1,..., c K 2. Given cluster centers, determine points in each cluster For each point p, find the closest c i. Put p into cluster i 3. Given points in each cluster, solve for c i Set c i to be the mean of points in cluster i 4. If c i have changed, repeat Step 2 Properties Will always converge to some solution Can be a local minimum Does not always find the global minimum of objective function:

22 Slide credit: Kristen Grauman Segmentation as Clustering K=2 K=3

23 Slide credit: Kristen Grauman Feature Space Depending on what we choose as the feature space, we can group pixels in different ways. Grouping pixels based on intensity similarity Feature space: intensity value (1D)

24 Feature Space Depending on what we choose as the feature space, we can group pixels in different ways. Grouping pixels based on color similarity B Feature space: color value (3D) Slide credit: Kristen Grauman R G R=15 G=189 B=2 R=3 G=12 B=2 R=255 G=200 B=250 R=245 G=220 B=248

25 Segmentation as Clustering Depending on what we choose as the feature space, we can group pixels in different ways. Grouping pixels based on texture similarity F 1 F 2 F 24 Feature space: filter bank responses (e.g. 24D) Slide credit: Kristen Grauman Filter bank of 24 filters

26 Slide adapted from Kristen Grauman Spatial coherence Assign a cluster label per pixel à possible discontinuities Original How can we ensure they are spatially smooth? Labeled by cluster center s intensity? 1 2 3

27 Slide adapted from Kristen Grauman Spatial coherence Depending on what we choose as the feature space, we can group pixels in different ways. Grouping pixels based on intensity+position similarity Intensity Y X Way to encode both similarity and proximity.

28 Slide credit: Kristen Grauman Summary K-Means Pros Simple, fast to compute Converges to local minimum of within-cluster squared error Cons/issues Setting k? Sensitive to initial centers Sensitive to outliers Detects spherical clusters only

29 Slide credit: Steve Seitz Probabilistic Clustering Basic questions What s the probability that a point x is in cluster m? What s the shape of each cluster? K-means doesn t answer these questions. Basic idea Instead of treating the data as a bunch of points, assume that they are all generated by sampling a continuous function. This function is called a generative model. Defined by a vector of parameters θ No hard (as with K-means) but a soft assignment to different clusters each with their probability

30 Slide adapted from Steve Seitz Mixture of Gaussians One generative model is a mixture of Gaussians (MoG) K Gaussian blobs with means µ b covariance matrices V b, dimension d Blob b defined by: Blob b is selected with probability ( ) The likelihood of observing x is a weighted mixture of Gaussians, K α b = 1 b=1

31 Slide adapted from Steve Seitz Expectation Maximization (EM) Goal Approach: Find blob parameters θ that maximize the likelihood function over all all datapoints 1. E-step: given current guess of blobs, compute probabilistic ownership of each point 2. M-step: given ownership probabilities, update blobs to maximize likelihood function 3. Repeat until convergence

32 Slide adapted from Steve Seitz EM Details E-step M-step Compute probability that point x is in blob b, given current guess of θ Compute overall probability that blob b is selected Mean of blob b Covariance of blob b (N data points)

33 Slide credit: B. Leibe Image source: Serge Belongie Segmentation with EM K = 3

34 Slide adapted from B. Leibe Summary: Mixtures of Gaussians, EM Pros Cons Probabilistic interpretation Soft assignments between data points and clusters Generative model, can predict novel data points Relatively compact storage Initialization often a good idea to start from output of k-means Local minima Need to know number of components K solutions: add a cost for model complexity Need to choose generative model (math form of a cluster?)

35 Topics of This Lecture Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with GraphCuts

36 Topics of This Lecture Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with path search

37 Slide adapted from Steve Seitz Finding Modes in a Histogram How many modes are there? Mode = local maximum of a given distribution Easy to see, hard to compute

38 Slide credit: Svetlana Lazebnik Mean-Shift Segmentation An advanced and versatile technique for clusteringbased segmentation D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature Space Analysis, PAMI 2002.

39 Slide adapted from Steve Seitz Mean-Shift Algorithm Iterative Mode Search 1. Initialize random seed center and window W 2. Calculate center of gravity (the mean ) of W: 3. Shift the search window to the mean 4. Repeat steps 2+3 until convergence

40 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

41 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

42 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

43 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

44 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

45 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass Mean Shift vector

46 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Region of interest Center of mass

47 Real Modality Analysis Tessellate the space with windows Slide by Y. Ukrainitz & B. Sarel Run the procedure in parallel

48 Slide by Y. Ukrainitz & B. Sarel Real Modality Analysis The blue data points were traversed by the windows towards the mode.

49 Slide by Y. Ukrainitz & B. Sarel Mean-Shift Clustering Cluster: all data points in the attraction basin of a mode Attraction basin: the region for which all trajectories lead to the same mode

50 Slide adapted from Svetlana Lazebnik Mean-Shift Clustering/Segmentation Choose features (color, gradients, texture, etc) Initialize windows at individual pixel locations Start mean-shift from each window until convergence Merge windows that end up near the same peak or mode

51 Mean-Shift Segmentation Results Slide credit: Svetlana Lazebnik

52 Slide adapted from Svetlana Lazebnik Summary Mean-Shift Pros General, application-independent tool Model-free, does not assume any prior shape (spherical, elliptical, etc.) on data clusters Just a single parameter (window size h) Cons h has a physical meaning (unlike k-means) == scale of clustering Finds variable number of modes given the same h Robust to outliers Output depends on window size h Window size (bandwidth) selection is not trivial Computationally rather expensive Does not scale well with dimension of feature space (sparsity problems in high-dimensional spaces )

53 Topics of This Lecture Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with path search

54 Slide adapted from Steve Seitz Images as Graphs q Fully-connected graph p w pq Node (vertex) for every pixel Edge between every pair of pixels (p,q) Affinity weight w pq for each edge w pq measures similarity Similarity is inversely proportional to difference (in color, texture, position, ) w

55 Source: Forsyth & Ponce Measuring Affinity Distance { 2 } 1 2σ aff ( x, y) = exp x y 2 d Intensity Color Texture { 2 } 1 2σ aff ( x, y) = exp I( x) I( y) 2 d { ( ) } dist c x c y 2σ aff ( x, y) = exp ( ), ( ) d (some suitable color space distance) { 2 } 1 2σ aff ( x, y) = exp f ( x) f ( y) 2 d (vectors of filter outputs)

56 Slide adapted from Steve Seitz Segmentation by Graph Cuts w A B C Break Graph into Segments Delete edges crossing between segments Easiest to break edges with low similarity (low weight) Similar pixels should be in the same segments Dissimilar pixels should be in different segments

57 Slide adapted from Steve Seitz Graph Cut (GC) GC = edges whose removal partitions a graph in two Cost of a cut Sum of weights of cut edges: A graph cut gives us a segmentation A B cut( A, B) w p, q p A, q B What is a good graph cut and how do we find one? =

58 Slide credit: Khurram Hassan-Shafique Minimum Cut We can do segmentation by finding the minimum cut in a graph Drawback: Efficient algorithms exist for doing this Weight of cut proportional to number of edges in the cut Minimum cut tends to cut off very small, isolated components Ideal Cut Cuts with lesser weight than the ideal cut

59 Normalized Cut (NCut) Min-cut has bias toward partitioning out small segments This can be fixed by normalizing for size of segments The normalized cut cost is: cut(a,b) assoc(a,v ) + cut(a,b) assoc(b,v ) assoc(a,v) = sum of weights from A to all nodes in the graph The exact solution is NP-hard but an approximation can be computed by solving a generalized eigenvalue problem. J. Shi and J. Malik. Normalized cuts and image segmentation. PAMI 2000 Slide adapted from Svetlana Lazebnik

60 Slide adapted from Steve Seitz Interpretation as a Dynamical System Treat the edges as springs and shake the system Elasticity proportional to cost Vibration modes correspond to segments Can compute these by solving a generalized eigenvector problem

61 NCuts Example NCuts segments Slide credit: B. Leibe Image source: Shi & Malik

62 Slide credit: Steve Seitz Image Source: Shi & Malik Color Image Segmentation with NCuts

63 Results with Color & Texture

64 Slide credit: Kristen Grauman Summary: Normalized Cuts Pros: Cons: Generic framework, flexible to choice of function that computes weights ( affinities ) between nodes Does not require any model of the data distribution Time and memory complexity can be high Dense, highly connected graphs many affinity computations Solving eigenvalue problem Preference for balanced partitions If a region is uniform, NCuts will find the modes of vibration of the image dimensions

65 Slide credit: William Freeman Markov Random Fields Allow rich probabilistic models for images But built in a local, modular way Learn local effects, get global effects out Observed evidence Hidden true states Neighborhood relations

66 Slide adapted from William Freeman MRF Nodes as Pixels (or Patches) Image pixels Φ( x, y ) i i Ψ( x, x ) i j Image states (e.g. foreground/background)

67 Slide adapted from William Freeman Network Joint Probability Pxy (, ) ( x, y) ( x, x) states Image = Φ Ψ i Image-state compatibility function i i i j i, j Local observations state-state compatibility function Neighboring nodes

68 Energy Formulation Joint probability Pxy (, ) ( x, y) ( x, x) = Φ Ψ i i i i j i, j Maximizing the joint probability is the same as minimizing the -log log P(x, y) = log Φ(x i, y i ) log Ψ(x i, x j ) i E(x, y) = ϕ(x i, y i ) + ψ (x i, x j ) i i, j This is similar to free-energy problems in statistical mechanics (spin glass theory). We therefore draw the analogy and call E an energy function. ϕ and ψ are called potentials. Slide credit: B. Leibe i, j

69 Energy Formulation Energy function Unary potentials Encode local information about the given pixel/patch How likely is a pixel/patch to be in a certain state? (e.g. foreground/background)? Pairwise potentials Exy (, ) = ϕ( x, y) + ψ( x, x) i i i i j i, j Unary potentials Pairwise potentials ϕ( x, y ) Encode neighborhood information How different is a pixel/patch s label from that of its neighbor? (e.g. here independent of image data, but later based on intensity/color/texture difference) Slide adapted from B. Leibe ϕ ψ i i ψ ( x, x ) i j

70 Slide credit: B. Leibe Energy Minimization Goal: Infer the optimal labeling of the MRF. Many inference algorithms are available, e.g. Gibbs sampling, simulated annealing Iterated conditional modes (ICM) Variational methods Belief propagation Graph cuts Recently, Graph Cuts have become a popular tool ϕ( x, y ) Only suitable for a certain class of energy functions But the solution can be obtained very fast for typical vision problems (~1MPixel/sec). i i ψ ( x, x ) i j

71 Graph Cuts for Optimal Boundary Detection Idea: convert MRF into source-sink graph Minimum cost cut can be computed in polynomial time (max-flow/min-cut algorithms) hard constraint n-links t s a cut hard constraint Slide adapted from Yuri Boykov [Boykov & Jolly, ICCV 01]

72 Slide credit: Yuri Boykov [Boykov & Jolly, ICCV 01] Adding Regional Properties D p (t) n-links t a cut Regional bias example I s Suppose are given expected intensities of object and background t-link w pq s t-link D p (s) t and I ( s 2 2 D ( s) exp I I / 2σ ) D p p p ( t 2 2 I I / 2 ) ( t) exp σ p

73 Slide credit: Yuri Boykov [Boykov & Jolly, ICCV 01] Adding Regional Properties More generally, regional bias can be based on any intensity models of object and background D p (s) D p (t) t s a cut D ( L ) = log Pr( I L ) Pr( Pr( I p I p t) s) p p p p given object and background intensity histograms I p I

74 Slide credit: Pushmeet Kohli How Does it Work? The s-t-mincut Problem Source v 1 v Sink 4 Graph (V, E, C) Vertices V = {v 1, v 2... v n } Edges E = {(v 1, v 2 )...} Costs C = {c (1, 2)...}

75 Slide credit: Pushmeet Kohli The s-t-mincut Problem What is an st-cut? Source v 1 v Sink = 16 An st-cut (S,T) divides the nodes between source and sink. What is the cost of a st-cut? Sum of cost of all edges going from S to T

76 Slide credit: Pushmeet Kohli The s-t-mincut Problem What is an st-cut? Source v 1 v Sink = 7 An st-cut (S,T) divides the nodes between source and sink. What is the cost of a st-cut? Sum of cost of all edges going from S to T What is the st-mincut? st-cut with the minimum cost

77 Slide credit: Pushmeet Kohli How to Compute the s-t-mincut? Source v 1 v Sink 4 Solve the dual maximum flow problem Compute the maximum flow between Source and Sink Constraints Edges: Flow < Capacity Nodes: Flow in = Flow out Min-cut/Max-flow Theorem In every network, the maximum flow equals the cost of the st-mincut

78 Maxflow Algorithms Source v 1 v 2 5 Flow = Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

79 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

80 Maxflow Algorithms Flow = Source v 1 v Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

81 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

82 Maxflow Algorithms Source v 1 v 2 3 Flow = Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

83 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 9 4 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

84 Maxflow Algorithms Flow = Source v 1 v Sink Slide credit: Pushmeet Kohli 5 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

85 Maxflow Algorithms Source v 1 v 2 3 Flow = Sink Slide credit: Pushmeet Kohli 5 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

86 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 5 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

87 Maxflow Algorithms Flow = Source v 1 v Sink Slide credit: Pushmeet Kohli 4 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

88 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 5 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

89 Maxflow Algorithms Source v 1 v Flow = Sink Slide credit: Pushmeet Kohli 5 0 Augmenting Path Based Algorithms 1. Find path from source to sink with positive capacity 2. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

90 Slide credit: B. Leibe Dealing with Non-Binary Cases For image segmentation, the limitation to binary energies is a nuisance. Binary segmentation only We would like to solve also multi-label problems. NP-hard problem with 3 or more labels There exist some approximation algorithms which extend graph cuts to the multi-label case E.g. -Expansion They are no longer guaranteed to return the globally optimal result. α α But -Expansion has a guaranteed approximation quality and converges in a few iterations.

91 Slide credit: B. Leibe Summary: Graph Cuts Segmentation Pros Powerful technique, based on probabilistic model (MRF). Applicable for a wide range of problems. Very efficient algorithms available for vision problems. Becoming a de-facto standard for many segmentation tasks. Cons/Issues Graph cuts can only solve a limited class of models Submodular energy functions Can capture only part of the expressiveness of MRFs Only approximate algorithms available for multi-label case

92 Slide credit: Kristen Grauman Segmentation: Caveats We ve looked at bottom-up ways to segment an image into regions, yet finding meaningful segments is intertwined with the recognition problem. Often want to avoid making hard decisions too soon Difficult to evaluate; when is a segmentation successful?

93 Slide credit: Svetlana Lazebnik Speeding up 1: start from `superpixels Start from an over-segmentation, similar-looking pixels have been grouped together quickly; requires object boundaries to be preserved as part of superpixel edges! superpixels X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003.

94 Speeding up 2: objectness Trying to draw bounding boxes around Focus on regions objects, that an `objectness score indicates as probably containing without knowing an object what they are e 7: Best windows proposed by our method (yellow), out of 1000, for the ground-truth annotations (blue). The examples in yellow: bb by computer / blue: by human

95 Topics of This Lecture Introduction Gestalt principles Image segmentation Segmentation as clustering k-means Feature spaces Mixture of Gaussians, EM Model-free clustering: Mean-Shift Graph theoretic segmentation: Normalized Cuts Interactive Segmentation with path search

96 Dynamic path search: principle Guided by a user-supplied cost function, expressing expectations like good edges to contain pixels with high gradients, edges to be smooth, etc. find optimal path through the image: 1. having lowest cost 2. satisfying constraints (e.g. given endpoints) Useful in interactive applications (e.g. medical), or when environment constrained

97 Dynamic path search : nomenclature A graph consists of nodes (pixels) connected by arcs (steps) Nodes connected by steps are parents and successors Identifying a node s successors is expansion of that node A tree is a graph with 1 parent for the nodes (our arcs are undirected)

98 Dynamic path search : nomenclature cont d Often the arcs are assigned a cost A sequence of nodes n 1,n 2,,n k (n i = sucessor of n i-1 ) is a path of length k Usually path cost = Σ arc costs

99 Dynamic path search : cost functions Cost function incorporates problem-specific information e.g. penalize changes in edge direction e.g. penalize the inclusion of pixels with low intensity gradient problem is one of optimization : l 1. gradient descent l 2. path array methods l 3. best-first search

100 Gradient descent Always choose the next pixel that adds the smallest cost Example :

101 Gradient descent : example angiogram image :

102 Gradient descent : example cont d Cost : 3 i 3 + d 1

103 Gradient descent : problem Gradient descent never looks back :

104 Gradient descent : remarks Gradient descent used for several purposes Fast but no guarantee that the optimal path is found

105 Path array techniques : principle Illustration of one-pass example : the angiogram? Select cheapest step to a node Remember by putting a pointer back

106 Path array techniques : principle Upon reaching the last column BACKTRACK Select cheapest node in last column

107 Path array techniques : principle Summary From left to right : determine cheapest path + cost for each pixel set pointer back Determine pixel with lowest value in right column backtrack

108 Path array techniques : example Optimal paths are found :

109 Path array techniques : remarks guaranteed to find the optimal path solves many optimisation problems simultaneously : from each point cheapest path can be backtracked suboptimal in that sense program is simple cannot handle meandering paths, so far

110 Path array techniques : principle of F* Solution to meandering problem via multipass algorithm : F* Path array is constructed iteratively until no changes F* finds optimal paths notation : P( i, j ) = cost up to point ( i, j ) C( i,j, i, j ) = cost for step from ( i, j ) to ( i, j )

111 Path array techniques : F* algorithm l 1. P initialized to, starting node = 0 l 2. Top to bottom, row by row updating of P : first, fromleft to right : P( i, j), C( i 1, j 1, i, j) + P( i 1, j 1 ), P( i, j) : = min C( i 1, j, i, j) + P( i 1, j), C( i 1, j + 1, i, j) + P( i 1, j + 1 ), ( ) ( ) C i, j 1, i, j + P i, j 1 then, from right to left : P( i, j) : = min ( P( i, j), C( i, j + 1, i, j) + P( i, j + 1) ) l 3. Alternating bottom to top and top to bottom l 4. Stops when no changes l 5. Optimal path is backtracked

112 Path array techniques : F* example Road detection in aerial image : Endpoints are given

113 Path array techniques : F* example Cost : C ( i, j ) = (255 - F( i, j )) a F ( i, j ) is scaled output of matched convolution filter :

114 Path array techniques : F* example Result for a = 1 Result for a = 2.4

115 Path array techniques : F* remarks Choice of cost function is crucial but not trivial! Note special structure of cost function in the ex.: C ( i, j, i, j ) = C ( i, j ) backtracking via cheapest neighbour instead of pointers F* yields the optimal solution Invests much effort in finding irrelevant solutions as before

116 Best - first search : principle Strike a balance between the two previous ones n 1. Uses problem-specific information to guide the process selectively n 2. Returns the optimal solution if applied properly New data structures: list OPEN: end nodes of paths list CLOSED: nodes already passed through furthermore: evaluation function f(n)

117 Best - first : the algorithm BF 1. Start node s on OPEN 2. OPEN = empty exit with failure 3. OPEN node n with f min., put it on CLOSED 4. Expand n 5. A successor = a goal node solved 6. For every successor n Calculate f (n ) If n neither on OPEN nor CLOSED : put n on OPEN, set ptr If n already on OPEN or CLOSED, replace f (n ) and redirect ptr if new f (n ) lower, if n on CLOSED, back to OPEN 7. Go to step 2. ñ ñ

118 Best - first : the algorithm BF 1. Start node s on OPEN 2. OPEN = empty exit with failure 3. OPEN node n with f min., put it on CLOSED 4. Expand n 5. A successor = a goal node solved 6. For every successor n Calculate f (n ) If n neither on OPEN nor CLOSED : put n on OPEN, set ptr If n already on OPEN or CLOSED, replace f (n ) and redirect ptr if new f (n ) lower, if n on CLOSED, back to OPEN 7. Go to step 2. ñ ñ

119 Best - first : BF* Risk of missing the optimal solution In step 5 : quits if successor is a goal node We should include the cost of the last arc

120 Best - first : BF* BF* algorithm wait until new arc is part of f (n) exit when node n is a goal node

121 Best - first : cost functions The BF and BF* algorithms advance slowly. For a typical cost function, short paths will have lower costs and need to be developed before the actual solution path can reach a goal node. We now try to do something about that.

122 Best - first : cost functions C (n) = cost for portion from n to the END of the path C (n) is recursive if for immediate successor n s C (n) = F (E (n), C (n s )) where E (n) function of local properties only If such rollback function F exists, it is possible to evaluate the cost of a path knowing the optimal remaining cost and the costs of steps taken so far Examples of rollback functions are the maximum or the sum of the arc costs.

123 Best - first : crystal balls??????!!!!!! BUT we don t know the path to the goal...

124 Best - first : crystal balls? We introduce an estimate h (n) Recursiveness of the roll-back function allows us to effectively use such estimate! this version of BF is the Z algorithm this version of BF* is the Z* algorithm

125 Best - first: Z Examples of non-recursive cost functions: 1 2 Largest - smallest node cost C ( n ) = c( n, n ) C ( n) nr + With Cnr(n ) the cost from start node to n Suppose n-1 is a start node, n+2 is a goal node ( 1) = c( n + 1, n + 2) + c( n, n + 1) + c( n 1, n) C n Cannot be retrieved from c(n-1,n) and ( n) = c( n + 1, n + 2) + c( n n + 1) nr C,

126 Best - first : Z* Z* finds the optimal path if 1. h (n) underestimates the remaining cost 2. for n successor of n : ( E( n), h ( n ) F( E( n), h ( n ) F 3. for parents n 1 and n 2 of a F E n h n F E n, h ( ( ) ( )) ( ( ) ( n) ) 1, 2 common successor n : ( E( n ) h ( n) ) F ( E( n ), h ( n) ) F 1 ( n ) h ( n ) h, 2 Sum and maximum cost are acceptable functions

127 Best - first : A* A* uses a sum : C(n) = c( n,n s ) + C(n s ) f(n) = g(n) + h(n), with g(n) sum of arc costs up to the current point

128 Best - first : the A* algorithm 1. Start node on OPEN 2. OPEN = empty exit, failure 3. n with min. f from OPEN, put it on CLOSED 5. expand n n not on OPEN or CLOSED, estimate h(n ), calculate f (n ), set ptrs. n on OPEN or CLOSED, direct ptr along path with lowest g (n ) if n obtained new ptr + on CLOSED, then put on OPEN 6. Go to step 2. ñ 4. n = goal node solved ñ

129 Best-first : consistency assumption if h ( m) h( n) c ( m, n) min then reconsidering CLOSED is unnecessary explanation : If the assumption holds, costs always increase over time. Hence returning to a node with lower cost is impossible remark : with h(n) = 0, the assumption normally applies

130 Best-search : Z*, A* remarks Increase efficiency by penalizing path incompleteness Spend less time on needlessly growing inferior paths Nevertheless, e.g. F* can be superior if the optimum is not outstanding, due to bookkeeping overhead or if there can be changes in the goal