Department of Image Processing and Computer Graphics University of Szeged. Fuzzy Techniques for Image Segmentation. Outline.

Transcription

1 László G. Nyúl systems sets image László G. Nyúl Department of Processing and Computer Graphics University of Szeged

2 systems sets image 1 systems 2 sets 3 image thresholding clustering 4

3 Dealing with imperfections systems sets image Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.

4 Dealing with imperfections systems sets image According to a researcher (sic) at Cambridge University, it doesn t matter in what order the letters in a word are, the only important thing is that the first and last letter be at the right place. The rest can be a total mess and you can still read it without problem. This is because the human mind does not read every letter by itself but the word as a whole.

5 systems sets image 1 systems 2 sets 3 image thresholding clustering 4

6 systems systems sets image systems and models are capable of representing diverse, inexact, and inaccurate information logic provides a method to formalize reasoning when dealing with vague terms. Not every decision is either true or false. logic allows for membership functions, or degrees of truthfulness and falsehoods.

7 Membership function examples systems sets image young person

8 Membership function examples systems sets image young person

9 Membership function examples systems sets image young person cold beer

10 Application area for fuzzy systems systems sets image Quality control Error diagnostics Control theory Pattern recognition

11 Object characteristics in images systems sets image Graded composition heterogeneity of intensity in the object region due to heterogeneity of object material and blurring caused by the imaging device Hanging-togetherness natural grouping of voxels constituting an object a human viewer readily sees in a display of the scene as a Gestalt in spite of intensity heterogeneity

12 systems sets image 1 systems 2 sets 3 image thresholding clustering 4

13 set systems sets image Let X be the universal set. For (sub)set A of X µ A (x) = { 1 if x A 0 if x A For crisp sets µ A is called the characteristic function of A. A fuzzy subset A of X is A = {(x, µ A (x)) x X } where µ A is the membership function of A in X µ A : X [0, 1]

14 systems sets image Probablility Probability vs. grade of membership is concerned with occurence of events represent uncertainty probability density functions Compute the probability that an ill-known variable x of the universal set U falls in the well-known set A. sets deal with graduality of concepts represent vagueness fuzzy membership functions Compute for a well-known variable x of the universal set U to what degree it is member of the ill-known set A.

15 Probability vs. grade of membership Examples systems sets image This car is between 10 and 15 years old (pure imprecision) This car is very big (imprecision & vagueness) This car was probably made in Germany (uncertainty) The image will probably become very dark (uncertainty & vagueness)

16 membership functions systems sets image triangle trapezoid gaussian singleton

17 systems sets image set properties Height height(a) = sup {µ A (x) x X } Normal fuzzy set height(a) = 1 Sub-normal fuzzy set height(a) 1 Support Core Cardinality supp(a) = {x X µ A (x) > 0} core(a) = {x X µ A (x) = 1} A = x X µ A (x)

18 systems sets image Operations on fuzzy sets Intersection A B = {(x, µ A B (x)) x X } µ A B = min(µ A, µ B ) Union A B = {(x, µ A B (x)) x X } µ A B = max(µ A, µ B ) Complement Ā = {(x, µā(x)) x X } µā = 1 µ A Note: For crisp sets A Ā =. The same is often NOT true for fuzzy sets.

19 relation systems sets image A fuzzy relation ρ in X is with a membership function ρ = {((x, y), µ ρ (x, y)) x, y X } µ ρ : X X [0, 1]

20 Properties of fuzzy relations systems sets image ρ is reflexive if x X µ ρ (x, x) = 1 ρ is symmetric if x, y X µ ρ (x, y) = µ ρ (y, x) ρ is transitive if x, z X µ ρ (x, z) = µ ρ (x, y) µ ρ (y, z) y X ρ is similitude if it is reflexive, symmetric, and transitive Note: this corresponds to the equivalence relation in hard sets.

21 systems sets image thresholding clustering 1 systems 2 sets 3 image thresholding clustering 4

22 systems sets image thresholding clustering image image is the collection of all approaches that understand, represent and process the images, their segments and features as fuzzy sets. The representation and depend on the selected fuzzy technique and on the problem to be solved. (From: Tizhoosh, Processing, Springer, 1997)... a pictorial object is a fuzzy set which is specified by some membership function defined on all picture points. From this point of view, each image point participates in many memberships. Some of this uncertainty is due to degradation, but some of it is inherent... In fuzzy set terminology, making figure/ground distinctions is equivalent to transforming from membership functions to characteristic functions. (1970, J.M.B. Prewitt)

23 image systems sets image thresholding clustering

24 systems sets image thresholding clustering thresholding 0 if f (x) < T 1 µ g(x) if T 1 f (x) < T 2 g(x) = 1 if T 2 f (x) < T 3 µ g(x) if T 3 f (x) < T 4 0 if T 4 f (x)

25 thresholding La szlo G. Nyu l Example systems sets image thresholding clustering original CT slice volume rendered image

26 Fuzziness and threshold selection systems sets image thresholding clustering original image

27 Fuzziness and threshold selection systems sets image thresholding clustering original image Otsu

28 Fuzziness and threshold selection systems sets image thresholding clustering original image Otsu fuzziness

29 k-nearest neighbors (knn) systems sets image thresholding clustering Training: Identify (label) two sets of voxels X O in object region and X NO in background Labeling: For each voxel v in input scenes... Find its location P in feature space Find k voxels closest to P from sets X O and X NO If a majority of those are from X O, then label v as object, otherwise as background Fuzzification: If m of the k nearest neighbor of v belongs to object, then assign µ(v) = m k to v as membership

30 k-means clustering systems sets image thresholding clustering The k-means algorithm iteratively optimizes an objective function in order to detect its minima by starting from a reasonable initialization. The objective function is J = k j=1 i=1 n x (j) 2 i c j

31 systems sets image thresholding clustering k-means clustering 1 Consider a set of n data points (feature vectors) to be clustered. 2 Assume the number of clusters, or classes, k, is known. 2 k < n. 3 Randomly select k initial cluster center locations. 4 All data points are assigned to a partition, defined by the nearest cluster center. 5 The cluster centers are moved to the geometric centroid (center of mass) of the data points in their respective partitions. 6 Repeat from (4) until the objective function is smaller than a given tolerance, or the centers do not move to a new point.

32 k-means clustering Issues systems sets image thresholding clustering How to initialize? What objective function to use? What distance to use? Robustness? What if k is not known?

33 c-means clustering systems sets image thresholding clustering A partition of the observed set is represented by a c n matrix U = [u ik ], where u ik corresponds to the membership value of the k th element (of n), to the i th cluster (of c clusters). Each element may belong to more than one cluster but its overall membership equals one. The objective function includes a parameter m controlling the degree of fuzziness. The objective function is J = c n j=1 i=1 (u ij ) m x (j) i c j 2

34 systems sets image thresholding clustering c-means clustering 1 Consider a set of n data points to be clustered, x i. 2 Assume the number of clusters (classes) c, is known. 2 c < n. 3 Choose an appropriate level of cluster fuzziness, m R >1. 4 Initialize the (n c) sized membership matrix U to random values such that u ij [0, 1] and c j=1 u ij = 1. n i=1 5 Calculate the cluster centers c j using c j = (u ij) m x i n i=1 (u ij) m, for j = 1... c. 6 Calculate the distance measures d ij = x (j), i c j for all clusters j = 1... c and data points i = 1... n. 7 Update the fuzzy membership matrix U according to d ij. If [ c ( ) 2 ] 1 dij m 1 d ij > 0 then u ij = k=1 d ik. If d ij = 0 then the data point x j coincides with the cluster center c j, and so full membership can be set u ij = 1. 8 Repeat from (5) until the change in U is less than a given tolerance.

35 c-means clustering Issues systems sets image thresholding clustering Computationally expensive Highly dependent on the initial choice of U If data-specific experimental values are not available, m = 2 is the usual choice Extensions exist that simultaneously estimate the intensity inhomogeneity bias field while producing the fuzzy partitioning

36 systems sets image 1 systems 2 sets 3 image thresholding clustering 4

37 Basic idea of fuzzy systems sets image local hanging-togetherness (affinity) based on similarity in spatial location as well as in intensity(-derived features) global hanging-togetherness ()

38 systems sets image digital space spel adjacency is a reflexive and symmetric fuzzy relation α in Z n and assigns a value to a pair of spels (c, d) based on how close they are spatially. Example 1 if c d < a small distance µ α (c, d) = c d 0 otherwise digital space (Z n, α) Scene (over a fuzzy digital space) C = (C, f ) where C Z n and f : C [L, H]

39 systems sets image spel affinity spel affinity is a reflexive and symmetric fuzzy relation κ in Z n and assigns a value to a pair of spels (c, d) based on how close they are spatially and intensity-based-property-wise (local hanging-togetherness). Example µ κ (c, d) = h(µ α (c, d), f (c), f (d), c, d) µ κ (c, d) = µ α (c, d) (w 1 G 1 (f (c) + f (d)) + w 2 G 2 (f (c) f (d))) ( ) where G j (x) = exp 1 (x m j ) 2 2 σ 2 j

40 Paths between spels systems sets image A path p cd in C from spel c C to spel d C is any sequence c 1, c 2,..., c m of m 2 spels in C, where c 1 = c and c m = d. Let P cd denote the set of all possible paths p cd from c to d. Then the set of all possible paths in C is P C = c,d C P cd

41 Strength of systems sets image The fuzzy κ-net N κ of C is a fuzzy subset of P C, where the membership (strength of ) assigned to any path p cd P cd is the smallest spel affinity along p cd µ Nκ (p cd ) = min j=1,...,m 1 µ κ(c j, c j+1 ) The fuzzy κ- in C (K) is a fuzzy relation in C and assigns a value to a pair of spels (c, d) that is the maximum of the strengths of assigned to all possible paths from c to d (global hanging-togetherness). µ K (c, d) = max p cd P cd µ Nκ (p cd )

42 systems sets image Let θ [0, 1] be a given threshold κ θ component Let K θ be the following binary (equivalence) relation in C { 1 if µ κ (c, d) θ µ Kθ (c, d) = 0 otherwise Let O θ (o) be the equivalence class of K θ that contains o C Let Ω θ (o) be defined over the fuzzy κ- K as Ω θ (o) = {c C µ K (o, c) θ}

43 systems sets image Let θ [0, 1] be a given threshold κ θ component Let K θ be the following binary (equivalence) relation in C { 1 if µ κ (c, d) θ µ Kθ (c, d) = 0 otherwise Let O θ (o) be the equivalence class of K θ that contains o C Let Ω θ (o) be defined over the fuzzy κ- K as Ω θ (o) = {c C µ K (o, c) θ} Practical computation of FC relies on the following equivalence O θ (o) = Ω θ (o)

44 connected object systems sets image The fuzzy κ θ object O θ (o) of C containing o is { η(c) if c O θ (o) µ Oθ (o)(c) = 0 otherwise that is µ Oθ (o)(c) = { η(c) if c Ω θ (o) 0 otherwise where η assigns an objectness value to each spel perhaps based on f (c) and µ K (o, c).

45 connected object systems sets image The fuzzy κ θ object O θ (o) of C containing o is { η(c) if c O θ (o) µ Oθ (o)(c) = 0 otherwise that is µ Oθ (o)(c) = { η(c) if c Ω θ (o) 0 otherwise where η assigns an objectness value to each spel perhaps based on f (c) and µ K (o, c). connected objects are robust to the selection of seeds.

46 as a graph search problem systems sets image Spels graph nodes Spel faces graph edges spel-affinity relation edge costs all-pairs shortest-path problem connected objects connected components

47 systems sets image Computing fuzzy Dynamic programming Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push all spels c C o such that µ κ(o, c) > 0 to Q while Q do remove a spel c from Q f val max d Co [min(f o(d), µ κ(c, d))] if f val > f o(c) then f o(c) f val push all spels e such that µ κ(c, e) > 0 endif endwhile end

48 systems sets image Computing fuzzy Dynamic programming Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push all spels c C o such that µ κ(o, c) > 0 to Q while Q do remove a spel c from Q f val max d Co [min(f o(d), µ κ(c, d))] if f val > f o(c) then f o(c) f val push all spels e such that f val > f o(e) endif endwhile end

49 systems sets image Computing fuzzy Dynamic programming Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push all spels c C o such that µ κ(o, c) > 0 to Q while Q do remove a spel c from Q f val max d Co [min(f o(d), µ κ(c, d))] if f val > f o(c) then f o(c) f val push all spels e such that f val > f o(e) and µ κ(c, e) > f o(e) endif endwhile end

50 systems sets image Computing fuzzy Dijkstra s-like Input: C, o C, κ Output: A K-connectivity scene C o = (C o, f o) of C Auxiliary data: a priority queue Q of spels begin set all elements of C o to 0 except o which is set to 1 push o to Q while Q do remove a spel c from Q for which f o(c) is maximal for each spel e such that µ κ(c, e) > 0 do f val min(f o(c), µ κ(c, e)) if f val > f o(e) then f o(e) f val update e in Q (or push if not yet in) endif endfor endwhile end

51 Brain tissue segmentation FSE systems sets image

52 FC with threshold MRI systems sets image

53 FC with threshold CT and MRA systems sets image

54 variants systems sets image Multiple seeds per object Scale-based fuzzy affinity Vectorial fuzzy affinity Absolute fuzzy Relative fuzzy Iterative relative fuzzy

55 Scale-based affinity systems sets image Considers the following aspects spatial adjacency homogeneity (local and global) object feature (expected intensity properties) object scale

56 Scale-based affinity systems sets image Considers the following aspects spatial adjacency homogeneity (local and global) object feature (expected intensity properties) object scale

57 systems sets Object scale Object scale in C at any spel c C is the radius r(c) of the largest hyperball centered at c which lies entirely within the same object region image The scale value can be simply and effectively estimated without explicit object segmentation

58 systems sets image Input: C, c C, W ψ, τ [0, 1] Output: r(c) begin k 1 while FO k (c) τ do k k + 1 endwhile r(c) k end Computing object scale Fraction of the ball boundary homogeneous with the center spel W ψs ( f (c) f (d) ) FO k (c) = d B k (c) B k (c) B k 1 (c)

59 Relative fuzzy systems sets image always at least two objects automatic/adaptive thresholds on the object boundaries objects (object seeds) compete for spels and the one with stronger wins

60 Relative fuzzy systems sets image Let O 1, O 2,..., O m, a given set of objects (m 2), S = {o 1, o 2..., o m } a set of corresponding seeds, and let b(o j ) = S \ {o j } denote the background seeds w.r.t. seed o j. 1 define affinity for each object κ 1, κ 2,..., κ m 2 combine them into a single affinity κ = j κ j 3 compute fuzzy using κ K 4 determine the fuzzy connected objects O ob (o) = {c C o b(o) µ K (o, c) > µ K (o, c)} { η(c) if c O ob (o) µ Oob (c) = 0 otherwise

61 knn vs. VSRFC systems sets image

62 systems sets image segmentation using FC MR brain tissue, tumor, MS lesion segmentation MRA vessel segmentation and artery-vein separation CT bone segmentation kinematics studies measuring bone density stress-and-strain modeling CT soft tissue segmentation cancer, cyst, polyp detection and quantification stenosis and aneurism detection and quantification Digitized mammography detecting microcalcifications Craniofacial 3D imaging visualization and surgical planning

63 Protocols for brain MRI systems sets image

64 systems sets image FC segmentation of brain tissues 1 Correct for RF field inhomogeneity 2 Standardize MR image intensities 3 Compute fuzzy affinity for GM, WM, CSF 4 Specify seeds and VOI 5 Compute relative FC for GM, WM, CSF 6 Create brain intracranial mask 7 Correct brain mask 8 Create masks for FC objects 9 Detect potential lesion sites 10 Compute relative FC for GM, WM, CSF, LS 11 Verify the segmented lesions

65 systems sets image FC segmentation of brain tissues 1 Correct for RF field inhomogeneity 2 Standardize MR image intensities 3 Compute fuzzy affinity for GM, WM, CSF 4 Specify seeds and VOI (interaction) 5 Compute relative FC for GM, WM, CSF 6 Create brain intracranial mask 7 Correct brain mask (interaction) 8 Create masks for FC objects 9 Detect potential lesion sites 10 Compute relative FC for GM, WM, CSF, LS 11 Verify the segmented lesions (interaction)

66 Brain tissue segmentation SPGR systems sets image

67 MS lesion quantification FSE systems sets image

68 Brain tumor quantification systems sets image

69 Skull object from CT systems sets image

70 MRA slice and MIP rendering systems sets image

71 MRA vessel segmentation and artery/vein separation systems sets image

72 systems sets image