Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem

Transcription

1 Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem Jan Lellmann, Frank Lenzen, Christoph Schnörr Image and Pattern Analysis Group Universität Heidelberg EMMCVPR 2011 St. Petersburg, July 25, 2011 J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 1

2 Motivation Problem Labeling problem: Partition image domain Ω into L regions Discrete decision at each point in continuous domain Ω Variational Approach: min l s(l(x), x)dx Ω }{{} local data fidelity + J(l) }{{} regularizer J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 2

3 Motivation Multiclass Labeling I Applications: Denoising, segmentation, 3D reconstruction, depth from stereo, inpainting, photo montage, optical flow,... I Spatially continuous formulation avoids metrication artifacts: J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 3

4 Model Multi-Class Labeling Multi-class relaxation: [Lie et al. 06, Zach et al. 08, Lellmann et al. 09, Pock et al. 09] Embed labels into R L as E := {e 1,..., e L }, relax to the unit simplex: L := {x R L x 0, x i = 1} = conv E, i min f (u), f (u) := u(x), s(x) dx + Ψ(Du) u BV(Ω, L ) Ω Ω Advantages: No explicit parametrization, rotation invariance, convex J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 4

5 Model Rounding Fractional solutions may occur: Goal: Find rounding scheme u ū : Ω {e 1,..., e L } such that for some δ 0. f ( }{{} ū ) (1 + δ)f ( ue rounded relaxed solution }{{} best integral solution ). J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 5

6 Model Rounding A posteriori approaches: [BaeYuanTai10, LellmannSchnörr10] Require primal and/or dual solution Depend strongly on specific input data Existing a priori bounds: Discretized setting [Dahlhaus et al. 94, KleinbergTardos 99, Boykov et al. 01, KomodakisTziritas 07] Continuous two-class case [Strang 83, ChanEsedogluNikolova 06, Zach et al. 09, Olsson et al. 09] This talk: Spatially continuous, multi-class, a priori bounds J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 6

7 Rounding Two-Class Case Two-class case: Find region C Ω minimizing [ChanVese01] min (I c 1 ) 2 dx + (I c 2 ) 2 dx + λh d 1 ( C) C C Ω\C Reformulate using indicator function of C and relax: min u(i c 1 ) 2 + (1 u)(i c 2 ) 2 dx + λtv (u) u BV(Ω,[0,1]) Ω J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 7

8 Rounding Two-Class Case Two-class case: Generalized coarea formula f (u) = 1 0 f (ū α )dα, ū α := { 1, u(x) > α, 0, u(x) α. Also: Choquet integral, Lovász extension, levelable function,... Consequence: Global integral minimizer for a.e. α, f (ūα) = f (u ). This talk: multi-class generalization. J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 8

9 Rounding Multi-Class Case Probabilistic interpretation: f (u) = 1 0 f (ū α )dα = E α f (ū α ). Rounding step u ū α does not increase f in the mean Multi-class generalization (approximate generalized coarea formula): (1 + δ)f (u) f (ū γ )dµ(γ) = E γ f (ū γ ) Need to define: parameter space Γ parametrized rounding method u ūγ probability measure µ on Γ bound δ independent of input Γ J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 9

10 Optimality Randomized Rounding Algorithm 1 (Randomized Rounding in BV) 1. Input: u 0 BV(Ω, L ) 2. For k = 1, 2, Sample γ k = (i k, α k ) {1,..., L} [0, 1] uniformly 4. u k e i k 1 {u k 1 i k >α k } + uk 1 1 {u k 1 i k α k } 5. Output: Limit ū of (u k ) Parameter space: sequences γ Γ := ({1,..., L} [0, 1]) N J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 10

11 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

12 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

13 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

14 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

15 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

16 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

17 Optimality Example J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 11

18 Optimality Main Result Theorem (Optimality) Let u BV(Ω, L ), s L 1 (Ω) L, s 0, Ψ : R d l R 0 positively homogeneous, convex and continuous, and λ l > 0, λ u < such that Ψ(z = (z 1,..., z L 1 L L )) λ l z i 2 z R d L, z i = 0, 2 i=1 Ψ(ν(e i e j ) ) λ u i, j {1,..., L}, ν R d, ν 2 = 1. i=1 Then Ef (ū) 2 λ u λ l f (u) and Ef (ū ) 2 λ u λ l f (u E). Provides approximate generalized coarea formula J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 12

19 Optimality Metric Labeling Specially designed Ψ: Local envelope relaxation for metric interaction potentials [Chambolle et al. 08, LellmannSchnörr 10] For a given metric d, Ef (ū ) 2 max i j d(i, j) min i j d(i, j) f (u E). Compatible with bounds for finite-dimensional multiway cut, α-expansion, LP relaxation Formulated in BV, independent of discretization J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 13

20 Summary Motivation: Convex relaxation of multiclass image labeling Solutions may be fractional! Goal: A priori bounds for rounded solution Rounding Strategy: Probabilistic approach Motivated by LP framework Bounds: A priori bound Provides approximate generalized coarea formula Compatible with finite-dimensional results Fully spatially continuous BV framework J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 14

21 Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem Jan Lellmann, Frank Lenzen, Christoph Schnörr Image and Pattern Analysis Group Universität Heidelberg EMMCVPR 2011 St. Petersburg, July 25, 2011 J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 15

22 Optimality Termination Theorem (Termination) Let u BV(Ω, L ). Then (almost surely) Alg. 1 generates a sequence that becomes stationary in some ū BV(Ω, E). Result is in BV Independent of data term J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 16

23 Experiments Iterations J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 17

24 Experiments I J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 18

25 Experiments II J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 19

26 Experiments Results problem # points # labels mean # iter a priori ε a posteriori - first-max prob. best prob. mean J. Lellmann Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 20