Fuzzy Sets and Fuzzy Techniques. Outline. Motivation. Fuzzy sets (recap) Fuzzy connectedness theory. FC variants and details. Applications.

Transcription

1 Sets and Sets and 1 sets Sets and Lecture 13 Department of Image Processing and Computer Graphics University of Szeged sets 2 sets 3 digital connected 4 5 Sets and Object characteristics in images Sets and Basic idea of fuzzy sets Graded composition heterogeneity of intensity in the region due to heterogeneity of material and blurring caused by the imaging device Hanging-togetherness natural grouping of voxels constituting an a human viewer readily sees in a display of the scene as a Gestalt in spite of intensity heterogeneity sets local hanging-togetherness (affinity) based on similarity in spatial location as well as in intensity(-derived features) global hanging-togetherness ()

2 Sets and set and relation Sets and Operations on fuzzy sets A fuzzy subset A of X is Intersection sets A = {(x, µ A (x)) x X } sets A B = {(x, µ A B (x)) x X } µ A B = min(µ A, µ B ) where µ A is the membership function of A in X µ A : X [0, 1] Union A B = {(x, µ A B (x)) x X } µ A B = max(µ A, µ B ) A fuzzy relation ρ in X is Complement ρ = {((x, y), µ ρ (x, y)) x, y X } Ā = {(x, µā(x)) x X } µā = 1 µ A with a membership function µ ρ : X X [0, 1] and are also called T-norm and T-conorm (S-norm). Several (corresponding pairs) of T- and S-norms exist. In the FC framework min and max are used. Sets and sets Properties of fuzzy relations ρ is reflexive if x X µ ρ (x, x) = 1 ρ is symmetric if x, y X µ ρ (x, y) = µ ρ (y, x) ρ is transitive if x, z X µ ρ (x, z) = µ ρ (x, y) µ ρ (y, z) y X ρ is similitude if it is reflexive, symmetric, and transitive Note: this corresponds to the equivalence relation in hard sets. Sets and sets digital connected digital spel adjacency is a reflexive and symmetric fuzzy relation α in Z n and assigns a value to a pair of spels (c, d) based on how close they are spatially. Example 1 if c d < a small distance µ α (c, d) = c d 0 otherwise digital (Z n, α) Scene (over a fuzzy digital ) C = (C, f ) where C Z n and f : C [L, H]

3 Sets and Sets and Paths between spels sets digital connected is a reflexive and symmetric fuzzy relation κ in Z n and assigns a value to a pair of spels (c, d) based on how close they are spatially and intensity-based-property-wise (local hanging-togetherness). Example µ κ (c, d) = h(µ α (c, d), f (c), f (d), c, d) µ κ (c, d) = µ α (c, d) (w 1 G 1 (f (c) + f (d)) + w 2 G 2 (f (c) f (d))) ( ) where G j (x) = exp 1 (x m j ) 2 2 σ 2 j sets digital connected A path p cd in C from spel c C to spel d C is any sequence c 1, c 2,..., c m of m 2 spels in C, where c 1 = c and c m = d. Let P cd denote the set of all possible p cd from c to d. Then the set of all possible in C is P C = c,d C P cd Sets and Strength of Sets and Let θ [0, 1] be a given threshold κ θ component sets digital connected The fuzzy κ-net N κ of C is a fuzzy subset of P C, where the membership (strength of ) assigned to any path p cd P cd is the smallest along p cd µ Nκ (p cd ) = min j=1,...,m 1 µ κ(c j, c j+1 ) The fuzzy κ- in C (K) is a fuzzy relation in C and assigns a value to a pair of spels (c, d) that is the maximum of the strengths of assigned to all possible from c to d (global hanging-togetherness). sets digital connected Let K θ be the following binary (equivalence) relation in C { 1 if µ κ (c, d) θ µ Kθ (c, d) = 0 otherwise Let O θ (o) be the equivalence class of K θ that contains o C Let Ω θ (o) be defined over the fuzzy κ- K as Ω θ (o) = {c C µ K (o, c) θ} µ K (c, d) = max p cd P cd µ Nκ (p cd ) Practical of FC relies on the following equivalence O θ (o) = Ω θ (o)

4 Sets and sets digital connected connected The fuzzy κ θ O θ (o) of C containing o is { η(c) if c O θ (o) µ Oθ (o)(c) = 0 otherwise that is µ Oθ (o)(c) = { η(c) if c Ω θ (o) 0 otherwise where η assigns an ness value to each spel perhaps based on f (c) and µ K (o, c). Sets and sets digital connected Spels graph nodes Spel faces graph edges as a graph search problem spel-affinity relation edge costs all-pairs shortest-path problem connected s connected components connected s are robust to the selection of seeds. Sets and Computing fuzzy Dynamic programming Sets and variants sets digital connected Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push all spels c C o such that µ κ(o, c) > 0 to Q while Q do remove a spel c from Q f val max d Co [min(f o(d), µ κ(c, d))] if f val > f o(c) then f o(c) f val push all spels e such that µ κ(c, e) > 0 f val > f o(e) f val > f o(e) and µ κ(c, e) > f o(e) endif endwhile end sets Multiple seeds per Scale-based fuzzy affinity Vectorial fuzzy affinity Absolute fuzzy Relative fuzzy Iterative

5 Sets and Components of fuzzy affinity Sets and sets spel adjacency µ α (c, d) indicates the degree of spatial adjacency of spels The homogeneity-based component µ ψ (c, d) indicates the degree of local hanging-togetherness of spels due to their similarities of intensities The -feature-based component µ φ (c, d) indicates the degree of local hanging-togetherness of spels with respect to some given feature sets µ κ (c, d) = µ α (c, d)g(µ ψ (c, d), µ φ (c, d)) Expected properties of g range within [0, 1] monotonically non-increasing in both arguments Examples µ κ = 1 2 µ α (µ ψ + µ φ ) µ κ = µ α µψ µ φ Sets and sets Homogeneity-based component Non-scale-based Homogeneity-based component µ ψ (c, d) = W ψ ( f (c) f (d) ) Sets and sets Object-feature-based component Non-scale-based Object-feature-based component { 1 if c = d µ φ (c, d) = otherwise W o(c,d) W b (c,d)+w o(c,d) Expected properties of W ψ range within [0, 1] and W ψ (0) = 1 monotonically non-increasing should also be related to overall homogeneity Examples the right-hand-side of an appropriately scaled box, trapezoid, or Gaussian function W o (c, d) = min[w o (f (c)), W o (f (d))] W b (c, d) = max[w b (f (c)), W b (f (d))] Expected properties of W o and W b range within [0, 1] monotonically non-increasing Examples an appropriately scaled and shifted box, trapezoid, or Gaussian function

6 Sets and Scale-based affinity Sets and Object scale sets Considers the following aspects spatial adjacency homogeneity (local and global) feature (expected intensity properties) scale sets Object scale in C at any spel c C is the radius r(c) of the largest hyperball centered at c which lies entirely within the same region The scale value can be simply and effectively estimated without explicit segmentation Sets and sets Input: C, c C, W ψ, τ [0, 1] Output: r(c) begin k 1 while FO k (c) τ do k k + 1 endwhile r(c) k end Computing scale Sets and sets Neighborhood selection for scale-based s For properties based on a single spel c use B r (c) = {e C e c r(c)} For properties based on a pair of spels c and d use B cd (c) = {e C e c min[r(c), r(d)]} Fraction of the ball boundary homogeneous with the center spel W ψs ( f (c) f (d) ) B cd (d) = {e C e d min[r(c), r(d)]} FO k (c) = d B k (c) B k (c) B k 1 (c)

7 Sets and Intensity variations in neighborhoods Sets and Homogeneity-based component Scale-based sets Intensity variations (inhomogeneity) surrounding c and d (µ α (c, d) > 0) intra- variation (expected to be random and to have a zero mean) inter- variation (expected to have a direction and to be larger than the intra- variation) sets Directional inhomogeneity over the regions around c and d D(c, d) = e B cd (c) e B cd (d) e c=e d Homogeneity-based component (1 W h (f (e) f (e )))ω cd ( e c ) f (e) f (e ) f (e) f (e ) µ ψs (c, d) = 1 D(c, d) ω cd ( e c ) e B cd (c) Sets and Object-feature-based component Scale-based Sets and Brain tissue segmentation FSE sets Filtered version of f (c) taking into account the neighborhood f (e)ω c ( e c ) f a (c) = e B r (c) ω c ( e c ) sets e B r (c) Object-feature-based component 1 if c = d µ φs (c, d) = W os (c, d) W bs (c, d) + W os (c, d) otherwise W os (c, d) = min[w os (f a (c)), W os (f a (d))] W bs (c, d) = max[w bs (f a (c)), W bs (f a (d))]

8 Sets and Efficiency problems with FC (1) Sets and Efficiency problems with FC (2) sets sets Problem it takes too long to compute Solution use more powerful computer scales with CPU speed but also depends on the algorithm Problem most time is spent with affinity Solution compute affinity only for spels that are used can reduce % depending on the Sets and Efficiency problems with FC (3) Sets and Effect of affinity thresholds sets sets Problem many spels are unnecessarily visited Solution use pre-determined thresholds can reduce % depending on the

9 Sets and Efficiency problems with FC (4) Sets and Computing fuzzy Dynamic programming sets Problem many spels are revisited Solution reduce the number of revisits by ordering label-correcting vs. label-setting algorithms binary, d-ary, and Fibonacci heaps LIFO and FIFO lists hash tables of various sizes with various hash functions additional pointer arrays sets Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push all spels c C o such that µ κ(o, c) > 0 to Q while Q do remove a spel c from Q f val max d Co [min(f o(d), µ κ(c, d))] if f val > f o(c) then f o(c) f val push all spels e such that µ κ(c, e) > 0 f val > f o(e) f val > f o(e) and µ κ(c, e) > f o( endif endwhile end Sets and Computing fuzzy Dijkstra s-like Sets and FC with threshold MRI sets Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a priority queue Q of spels sets begin set all elements of C o to 0 except o which is set to 1 push o to Q while Q do remove a spel c from Q for which f o(c) is maximal for each spel e such that µ κ(c, e) > 0 do f val min(f o(c), µ κ(c, e)) if f val > f o(e) then f o(e) f val update e in Q (or push if not yet in) endif endfor endwhile end

10 Sets and FC with threshold CT and MRA Sets and Vector-valued scenes Scene sets sets C = (C, f) where C Z n and f = (f 1, f 2,..., f l ) T f : C [L 1, H 1 ] [L 2, H 2 ] [L l, H l ] Object scale d B k (c) W h (f(c) f(d)) FO k (c) = B k (c) B k 1 (c) ( W h (x) = exp 1 ) 2 xt Σ 1 h x Sets and Homogeneity-based component Vectorial scale-based Sets and Object-feature-based component Vectorial scale-based sets Total unidirectional inhomogeneity over the scale regions around c and d D(c, d) = e B cd (c) e B cd (d) e c=e d (1 W h (f(e) f(e )))ω cd ( e c ) f(e) f(e ) f(e) f(e ) µ ψvs (c, d) = 1 D(c, d) ω cd ( e c ) e B cd (c) sets ( W o (f a (c)) = exp 1 ) 2 (f a(c) M o ) T Σ 1 o (f a (c) M o ) µ φvs (c, d) = { 1 if c = d min[w o (f a (c)), W o (f a (d))] otherwise

11 Sets and Relative fuzzy Sets and Relative fuzzy sets always at least two s automatic/adaptive thresholds on the boundaries s ( seeds) compete for spels and the one with stronger wins sets Let O 1, O 2,..., O m, a given set of s (m 2), S = {o 1, o 2..., o m } a set of corresponding seeds, and let b(o j ) = S \ {o j } denote the background seeds w.r.t. seed o j. 1 define affinity for each κ 1, κ 2,..., κ m 2 combine them into a single affinity κ = j κ j 3 compute fuzzy using κ K 4 determine the fuzzy connected s O ob (o) = {c C o b(o) µ K (o, c) > µ K (o, c)} { η(c) if c O ob (o) µ Oob (c) = 0 otherwise Sets and sets knn vs. VSRFC Sets and sets Image segmentation using FC MR brain tissue, tumor, MS lesion segmentation MRA vessel segmentation and artery-vein separation CT bone segmentation kinematics studies measuring bone density stress-and-strain modeling CT soft tissue segmentation cancer, cyst, polyp detection and quantification stenosis and aneurism detection and quantification Digitized mammography detecting microcalcifications Craniofacial 3D imaging visualization and surgical planning

12 Sets and Protocols for brain MRI Sets and FC segmentation of brain tissues sets sets 1 Correct for RF field inhomogeneity 2 Standardize MR image intensities 3 Compute fuzzy affinity for GM, WM, CSF 4 Specify seeds and VOI (interaction) 5 Compute relative FC for GM, WM, CSF 6 Create brain intracranial mask 7 Correct brain mask (interaction) 8 Create masks for FC s 9 Detect potential lesion sites 10 Compute relative FC for GM, WM, CSF, LS 11 Verify the segmented lesions (interaction) Sets and Segmentation in two phases Sets and Brain tissue segmentation FSE sets Phase 1: Training performed once for each task (protocol, body region, organ) a few datasets are selected and used to extract the values for the parameters mostly requires continuous user control sets Phase 2: Segmentation performed for each individual dataset most steps are automatic (parameters are fixed in Phase 1) interactive steps require mouse clicks from the user to specify points cut and add when correcting the brain mask

13 Sets and Brain tissue segmentation T1 Sets and Brain tissue segmentation SPGR sets sets Sets and MS lesion quantification FSE Sets and MS lesion quantification T1E sets sets

14 Sets and MTR analysis Sets and Brain tumor quantification sets sets Sets and Skull from CT Sets and MRA slice and MIP rendering sets sets

15 Sets and sets MRA vessel segmentation and artery/vein separation Sets and sets J. K. Udupa and S. Samarasekera. and definition: Theory, algorithms, and applications in image segmentation. Graphical Models and Image Processing, 58(3): , P. K. Saha, J. K. Udupa, and D. Odhner. Scale-based fuzzy connected image segmentation: Theory, algorithms, and validation. Computer Vision and Image Understanding, 77(2): , P. K. Saha and J. K. Udupa. connected delineation: Axiomatic path strength definition and the case of multiple seeds. Computer Vision and Image Understanding, 83(3): , P. K. Saha and J. K. Udupa. Relative fuzzy among multiple s: Theory, algorithms, and applications in image segmentation. Computer Vision and Image Understanding, 82(1):42 56, J. K. Udupa, P. K. Saha, and R. A. Lotufo. Relative fuzzy and definition: Theory, algorithms, and applications in image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(11): , L. G. Nyúl, A. X. Falcao, and J. K. Udupa. -connected 3D image segmentation at interactive speeds. Graphical Models, 64(5): , Y. Zhuge, J. K. Udupa, and P. K. Saha. Vectorial scale-based fuzzy-connected image segmentation. Computer Vision and Image Understanding, 101(3): , Sets and sets