Parallel constraint optimization algorithms for higher-order discrete graphical models: applications in hyperspectral imaging

Transcription

1 Parallel constraint optimization algorithms for higher-order discrete graphical models: applications in hyperspectral imaging MSc Billy Braithwaite Supervisors: Prof. Pekka Neittaanmäki and Phd Ilkka Pölönen Department of Mathematical Information Technology Spectral Imaging Laboratory (SLI) University of Jyväskylä 26 April 2016

2 Introduction: Background BSc and MSc in Computer Science from University of Eastern Finland (UEF) in 2012 and 2015 respectively. Research assistant ( ): Speaker Evaluation ( ); Computer vision ( ). Machine learning, pattern recognition, computer vision, algorithms. Bachelor s and Master s thesis on metric indexing (similarity search). PhD studies at JYU, MIT, Spectral Imaging Laboratory November 2015.

3 Current PhD research proposal Designing and analyzing parallel algorithms for structured prediction using higher-order discrete probabilistic graphical models. Solving higher-order graphical models viewed as solving (valued) constraint satisfaction problems. Proximal methods used as solvers. Modeling and analysis of hyperspectral images using discrete probabilistic graphical models.

4 Primer on graphical models A graphical model is a realization of state spaces of individual vertices in a graph. Definition Let G = (V, E) be a graph, where V and E are sets of vertices and edges respectively. Also let i V be vertices, and let X = (X i ) i V be random joint variables and x = (x i ) i V X = i V X i be the realization of X. Then the graphical is the tuple (G, X). Two (main) types of graphical models: Markov Random Fields and Bayesian Belief Networks. Additional types: Factor Graphs and Coniditional Random Fields.

5 Markov Random Fields Markov Random Fields (MRFs) is an undirected graphical model, satisfying the following conditions: Local Markov property: i V, X i X j X c C, where c C a set of cliques of G. p(x) = 1 Z θ c (x c ). (1) c C Global Markov property: If V 1, V 2, V 3 V there exists a path from i V 1 to j V 2 which includes at least one k V 3, then X V1 X V2 X V3.

6 Markov Random Fields Inferece in graphical models are posed as optimization problems: solving p(x) as a maximum a posteriori (MAP) problem: x = arg max x X p(x) (2) p(x) = 1 Z exp( E(x)), Z = i V exp( E(x i )), (3) E(x) = c C θ c (x c ). (4) Equation (2) is equivalent to x = arg min x X E(x). (5)

7 Factor Graphs A Factor Graph (FG) is a more unified representation of a MRF, which uses an additional vertices coined factor vertices. The associated joint probability of a FG is a product of its factors: p(x) = 1 Z θ f (x f ). (6) f F FGs are bipartite graphs which gives a clearer (and more explicit) picture of the underlyingh factorization of the distribution of the graphical model.

8 Discrete probabilistic graphical models in vision Markovian Factor Graph x 1 x 2 x 3 x 1 x 2 x 3 x 6 x 4 x 5 x 7 f 1,2,4 f 1,3,5 x 5 x 6 x 4 x 7 x 8 x 9 f 4,6,8 f 5,7,9 x 8 x 9

9 Vision models using graphical models Pairwise models: E(x) = θ i (x i ) + θ c (x c ), C 2. (7) i V c C w i,j (1 δ(x i x j )) θ c (x c ) = min{k i,j, x i x j } min{k i,j, (x i x j ) 2 } (Potts models) (truncated absolute distance) (truncated quadratic) Higher-order models: E(x) = θ i (x i ) + θ c (x c ), C > 2 (8) i V c C

10 Graph-cuts Discrete optimization methods based on the Ford-Fulkerson max-flow/min-cut algorithm: C(S, T ) = i S,j T,{i,j} E st c(i, j). The aim is to construct a directed graph, and perform the s t-cut by partitioning the graph into two disjoint sets. Can be applied to multilabeling scenarios. Exact optimization possible when θ c ( ) is submodular: f (S) + f (T ) f (S T ) + f (S T )

11 Belief propagation First proposed for tree structures. Computes the marginal distribution of a graphical model via max-product: p(x) = 1 θ i,j (x i, x j ) θ i (x i ) (9) Z {i,j} E i V Standard Belief propagation update rule: m i,j (x j ) α θ i,j (x i, x j )θ i (x i ) m k,i (x i ) x i k N i \j b i (x i ) αθ i (x i ) k N i m k,i (x i ) (10a) (10b)

12 Dual-methods The MAP inference reformulated as an integer linear programming problem: E(θ, τ) = θ, τ = θ i (a)τ i (a) + i V a X i {i,j} E (a,b) X i X j θ i,j (a, b)τ i,j (a, b) a X i τ i (a) = 1 i inv, a X τ τ G = i τ i,j (a, b) = τ j (b) {i, j} E, b X j, τ i (a) {0, 1} i V, a X i, τ i,j (a, b) {0, 1} {i, j} E, (a, b) X i X j τ τ G is minimized over a relaxed domain ˆτ G (local marginal polytope).

13 Valued constraint satisfaction problem (VCSP) Solving E( ) in equation (8) can be viewed as an instance of constraint programming. This is known as VCSP. Linear Programming relaxation of VSCP: θ S (x)µ S (x) min, (11a) S H x D S µ S (y) = µ i (x), i S H, x D, (11b) y D S y i =x µ i (x) = 1, i V, (11c) x D µ S (x), S H, x D S, (11d) µ i (x) 0, i V, x D. (11e)

14 Challenges graphical models: higher-order cliques Higher-order interactions offers better modeling of the structures within an image. Challenges arise when C > 2: E(x) = i V θ i (x i ) + c C θ c (x c ) (12) Decision making/optimization becomes intractable = NP-Completness.

15 Order-reduction of higher-order graphical models E(x) = i V θ i (x i ) + {i,j} E θ i,j + c S P n model: Let l L = {0, 1, 2,... k} be labels. { 0, if x i = l k, i c, θ c (x c ) = θ 1 c θα, else Robust P n model: θ c (x c ) = { N i (x c ) 1 Q γ max, if N i (x c Q, γ max, else, θ c (x c ). (13) where N i (x c ) = min k ( c n k (x c )) and γ max = c θα (θ 1 + θ 2 M(c)) and M(c) a quality measure.

16 Split Bregman iteration For solving l 1 -regularized problems. The general form: min u Φ(u) + H(u) (14) The Split Bregman iteration decouples the l 1 and l 2 portions of equation (14): min u,d d + H(u), such that d = Φ(u). (15) Convert Equation (15) into an unconstrained optimization problem: min u,d d + H(u) + λ 2 d Φ(u) 2 2. (16)

17 Split Bregman iteration The Bregman distance: D J p (u, v) = J(u) J(v) < p, u v >, (17) Two step iteration for constrained optimization: { uk+1 min u (D J p k (u, u k ) + λh(u)), µ > 0, p k+1 p k H(u k+1 ).

18 Split Bregman iteration Apply the Bregman distance and the above iteration scheme to equation (16): (u k+1, d k+1 ) min u (D J (u, u p k k, d, d k ) + λ 2 d Φ(u) ), µ > 0, pk+1 u pu k λ( Φ)T (Φ(u k+1 ) d k+1 ) pk+1 d pu k λ(d k+1 Φ(u k+1 )) The Split Bregman iteration { (u k+1, d k+1 ) min u,d d + H(u) + λ 2 d Φ(u) b k 2 2 b k+1 b k + (Φ(u k+1 ) d k+1 )

19 Parallel computing approaches Both multithread and GPU approaches considered. The Split Bregman Iteration (also proximal methods in general) are well-suited for parallel computing. For example using Domain Decomposition on Equation (5), each subproblem ( slave problem) can be solved independently with the Split Bregman Iteration via saddle-point formulation of the Lagrangian multipliers. Using parallel computing for computing promising clique configurations for a graphical model.

20 Application: Dense CRFs for image segmentation Give a random field X of random variables and a graph G with a discrete label set L = {l 1, l 2,..., l k }, the CRF (I, X) is characterized by a Gibbs distribution: p(x I ) = 1 Z exp( c C θ c(x I )). The corresponding Gibb s energy: E(x) = θ u(x i ) + θ p(x i, x j ) i V i<j Pairwise potentials: θ p(x i, x j ) = µ(x i, x j ) k(f i, f j ) = w 1 exp( p i p j 2 2θ 2 α K w m k m (f i, f j ), m=1 I i I j 2 2θ 2 α ) + w 2 exp( p i p j 2 ) 2θ 2 γ

21 Application: Dense CRFs for image segmentation (a) Input image. (b) Priors. (c) Output. (a) Input image. (b) Priors. (c) Output.

22 Applications in proposed PhD research Applying optimization and machine learning methods in bio-medical related computer vision applications. For example, texture analysis and segmentation. Present research plan is to apply hyperspectral imaging techniques to bio-medical imaging applications: Analyzing histology image samples from Central Hospital of Jyäskylä Modeling and simulation(?) of cell structures and diseases from cell cultivations obtained from FICAM Skin cancer prognosis/analysis from imaging samples from Helsinki University Central Hospital (HUS)

23 Current status 60/60 ECTS acquired during December of First journal article Quicker range-and-k-nn joins in metric spaces published in Used as an auxilary data structure for computing distance oralces. Second journal article Split Bregman solvers for higher-order random fields in progress. Estimated submission: June 2016 to International Journal of Computer Vision. Third jourrnal article Non-submodular function minimization via Split Bregman iterators in preparation.