Mathematical morphology in polar(-logarithmic) coordinates for the analysis of round-objects. Shape analysis and segmentation.

Transcription

1 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 1 29ème journée ISS France Mathematical morphology in polar(-logarithmic) coordinates for the analysis of round-objects. Shape analysis and segmentation. Jesús Angulo angulo@cmm.ensmp.fr ; angulo Centre de Morphologie Mathématique - Ecole des Mines de Paris February 2nd, 2006

2 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 2 Motivation Application of mathematical morphology to round-objects (i.e. which contain a some kind of radial symmetry, or in general, which have a center ) Difficulty to take advantage of radial/angular properties in Cartesian coordinates (definition of neighborhood, adapted structuring elements, etc.) Examples (from biomedical microscopy):

3 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 3 Aim To change the geometric representation of the image: The polar-logarithmic representation (or a general polar coordinates system) presents many advantages for these objects To adapt classical morphological operators to this representation

4 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 4 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

5 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 5 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

6 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 6 (Log-)Pol coordinates (1/7) Definition The logarithmic polar transformation converts the cartesian image function f(x, y) : E T (T Z, E Z 2 ) into another log-polar image function f (ρ log, θ) : E ρ log,θ T, where the angular coordinates are placed on the vertical axis and the radial coordinates are placed on the vertical one. More precisely, with respect to a central point (x c, y c ): ρ = (x x c ) 2 + (y y c ) 2 ρ log = log(ρ) 0 ρ log R; ( ) θ = arctan y yc x x c ; 0 θ < 2π The support is the space E ρ log,θ, (ρ log, θ) (Z Z p ).

7 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 7 (Log-)Pol coordinates (2/7) Definition A relation is established where the points at the top of the image (θ = 0) are neighbors to the ones an the bottom (θ = p 1, period p equivalent to 2π): the image can be seen as a strip where the superior and inferior borders are joined. Example:

8 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 8 (Log-)Pol coordinates (3/7) Implementation log and arctan are continuous functions to fit in a digital grid. The transformations are adjusted to the boundary points (size of support space). For each point, a bi-linear interpolation is used to avoid the effect of discretisation.

9 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 9 (Log-)Pol coordinates (4/7) Properties Rotation: Rotations in the original Cartesian image become vertical cyclic shifts in the transformed log-pol.

10 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 10 (Log-)Pol coordinates (5/7) Properties Scaling: The changes of size in the original image become horizontal shifts in the log-pol transformed image.

11 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 11 (Log-)Pol coordinates (6/7) Properties The polar transformation preserves the property of rotation but the changes of scale involve a combined scaling/horizontal shift effect.

12 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 12 (Log-)Pol coordinates (7/7) Choice of a center The choice of the center (x c, y c ) is crucial in the transformation. We propose to use the maxima of the distance function (ultimate erosion) to compute the center of binary objects. For gray level images, it is proposed to compute the maximum of a gray-weighted generalised distance function to the border of the image.

13 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 13 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

14 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 14 Cyclic morphology (1/10) Motivation Introduce the angular periodicity: The aim is to preserve the invariance with respect to the rotation in the Cartesian space:

15 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 15 Cyclic morphology (2/10) Implementation In order to implement the new neighborhood relation and to be able to use morphological operators, two possibilities can be considered: Modify the neighborhood relation and the code of the basic operators (erosion, dilation, etc.) by adding the operator module of the size of the image in the direction of the periodic coordinate Extend the image along its angular direction by adding the top part of the image on the bottom and the bottom part on the top. The size of the vertical component from each part should be bigger than the size from the vertical component of the structuring element in order to avoid a possible edge effect.

16 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 16 Cyclic morphology (3/10) Implementation After having cycled the image, morphological operators can be applied as usual and only the image corresponding to the initial mask is kept. Using this approach, all the existing code is recyclable!

17 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 17 Cyclic morphology (4/10) Meaning of structuring elements The use of classical structuring elements in the log-pol image is equivalent to the use of radial - angular structuring elements in the original image, e.g. a vertical structuring element corresponds to an arc in the original image or a square corresponds to a circular sector. Example of horizontal structuring element:

18 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 18 Cyclic morphology (5/10) Meaning of structuring elements Other examples:

19 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 19 Cyclic morphology (6/10) Tools Circular filtering: Method for extracting inclusions or extrusions from the contour of a relatively rounded shape with simple openings or closings. The proportion of the vertical size from the structuring element with respect to the whole vertical size represents the angle affected in the original Cartesian image. With respect to a classical extraction in Cartesian coordinates, the choice of size is not as critical, making this a very advantageous point. It is due to the large zone plate in the openings/closings spectrum that is always found after a determined value (until the complete elimination of the object).

20 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 20 Cyclic morphology (7/10) Tools Circular filtering Example of opening with a vertical structuring element of size 20% of the whole image (i.e. 72 ).

21 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 21 Cyclic morphology (8/10) Tools Radial skeleton: Interesting for objects without holes. Radial inner skeleton is the skeleton obtained by an homotopic thinning from the log-pol transformation of an objet. The invert transformation to Cartesian coordinates from the branches of the radial inner skeleton has radial sense and tend to converge to the center (ρ = 0). Radial outer skeleton is obtained by an homotopic thinning from the negative image of the log-pol transformation of an object. The invert branches tend to diverge to an hypothetical circumference in the infinity (ρ ).

22 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 22 Cyclic morphology (9/10) Tools Radial skeleton: Illustrative example,

23 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 23 Cyclic morphology (10/10) Tools Radial skeleton: More examples,

24 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 24 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

25 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 25 Application: Erythrocyte shape analysis (1/4) Aim: In hematology, the morphological analysis of erythrocytes (size, shape, color, center,...) is fundamental: anomalies and variations from the typical red blood cell are associated to anemia or other pathologies. The analysis of shape categories is particularly interesting: Normal Mushroom Spicule Echinocyte Bitten

26 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 26 Application: Erythrocyte shape analysis (2/4) Spicule and Echinocyte erytrocytes: Extraction of extrusions using circular filtering and selection/classification by radial skeleton.

27 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 27 Application: Erythrocyte shape analysis (3/4) Mushroom erytrocytes

28 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 28 Application: Erythrocyte shape analysis (4/4) Bitten erytrocytes

29 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 29 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

30 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 30 Generalized dist. function and minimal paths (1/7) Motivation Inner (spot center) marker and outer (bounding box) marker watershed-based: problems of segmentation for low intensity spots or for spots on strong noisy background; and on the other hand, difficulty to define a right segmentation/quantification for doughnut and egg-like spots Original Filtered Segmented

31 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 31 Generalized dist. function and minimal paths (2/7) Motivation Spot segmentation can be approached in a more flexible and understandable way when working in polar coordinates, but the same weaknesses of the watershed on the low or noisy gradients are still underlying. Typical Doughnut-like Extract interesting morphological features from projections of the spot in polar coordinates.

32 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 32 Generalized dist. function and minimal paths (3/) Watershed transformation segmenting left/right sides: Extracting a continuous track (= crest-line ) going from the top to the bottom of the image by means of a constrained watershed using as markers the right side and the left side of the image,

33 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 33 Analysis the watershed line (by Vincent, 98): It fails when SNR is low = sensitivity of watershed line to noise; The watershed between two markers A and B depends on the position of the saddle points (for all the paths joining A to B with minimal elevation, the highest pixels along those paths are the saddle points) between the markers, and their location is one of the main factors determining the location of the line; The criteria used to build the watershed are based on grey levels, and the length of watershed lines is irrelevant. Length constraints can be introduced by using global minimal paths algorithms. This approach is also useful to detect disconnected crest-line between two markers.

34 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 34 Generalized dist. function and minimal paths (4/7) Generalized distance function, GDF: Modification of the classic two-pass sequential distance function algorithm so that: (1) edge cost is taken into account; (2) raster and anti-raster scans are iterated until stability. Let us to denote by N + (p) (resp., N (p)) the neighbors of pixel p scanned before p (resp., after p) in a raster scan, for a 8-connected grid (neighborhood graph): In this graph, each edge between two neighboring pixels p and q of an image f has associated the cost value C f (p, q) = f(p) + f(q) (or any other increasing function, such as max(f(p), f(q)) or min(f(p), f(q))).

35 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 35 More specifically, the algorithm proceeds as follows, Algorithm: GDF to set X in image f Initialise result image d: d(p) = 0 if p X and d(p) = + otherwise; Iterate until stability: Scan image in raster order For each pixel p, do: d(p) min{d(p), min{d(q) + C f (p, q), q N + (p)}} Scan image in anti-raster order For each pixel p, do: d(p) min{d(p), min{d(q) + C f (p, q), q N (p)}} The algorithm typically converges in two or three iterations (relatively efficient).

36 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 36 Generalized dist. function and minimal paths (5/7) Global minimal paths, GMP: Each path P in the 8-connect graph has associated a cost C f (P ), equal to the sum of the cost of its successive edges. We can now define the distance d f (p, q) between two pixels p and q in the image f as: d f (p, q) = min{c f (P ), P path between p and q}. For the simple problem of finding a path of minimal cost (or global minimal path) going from the top row U to the bottom row D of the image, we use the following result: a pixel p belongs to such minimal path if and only if d f (p, U) + d f (p, D) = d f (U, D).

37 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 37 To extract such path, we can therefore proceed as follows: Algorithm: Up/Down GMP in image f Compute GDF to set U in image f: for each pixel p, compute d f (p, U); Compute GDF to set D in image f: d f (p, D); Sum these two distance functions, d f (U, D)(p) = d f (p, U) + d f (p, D); Find u min, the minimal value of d f (U, D) and threshold the result in order to keep only the pixels whose values in d f (U, D) is equal to u min From an algoritmic point of view, the problem is reduced to computing two gray-weighted generalised distance transforms.

38 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 38 Generalized dist. function and minimal paths (6/7) Global minimal paths, GMP Illustrative example: f dist U (f) dist D (f) dist UD (f) Iso(dist UD (f)) min(dist UD (f))

39 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 39 Generalized dist. function and minimal paths (7/7) Examples of up/down global minimal path: The main limitation of the approach lies on the degree of horizontal curvature (see example (d)) but it is very robust against to the noise (see example (c)).

40 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 40 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

41 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 41 Application: Model-based spot segmentation (1/6) Flow chart of algorithm: Starting from the spot in polar coordinates, the aim is to segment its contour using the GMP technique. The segmentation algorithm must be able to yield a spot segmentation in several regions according to the spot structure

42 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 42 Application: Model-based spot segmentation (2/6) Filtering in polar coordinates: Anisotropic effect in polar coordinates by applying two separable directional filtering (unidimensional filtering). Usually for the polar image of spots the vertical (according to the angular coordinate) filtering has a size n ρ which is notably higher than the size n θ horizontal filtering (radial direction).

43 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 43 Application: Model-based spot segmentation (3/6) Circular minimal path to close contour: In polar coordinates in the Up/Down GMP the initial radial value ρ up (for θ = 2π) and the final one ρ down (for θ = 0) are equal. Typical problem when the center of spot is shifted with respect to (x c, y c ).

44 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 44 Application: Model-based spot segmentation (4/6) Circular minimal path to close contour: To apply the Up/Down GMP to the cycled image. In fact, even if ρ up ρ down, but ρ up ρ down ρ ( ρ being a small value), the contour can be closed by dilation.

45 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 45 Application: Model-based spot segmentation (5/6) Example of spot segmentation: Original (x10) Ref. Segment.

46 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 46 Application: Model-based spot segmentation (6/6) Example of spot segmentation: Original (x10) Minimal Paths

47 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 47 Plan 1. (Log-)Pol coordinates 2. Cyclic morphology 3. Application: Erythrocyte shape analysis 4. Generalised distance function and global minimal path algorithms 5. Application: Model-based spot segmentation by minimal paths 6. Conclusions and perspectives

48 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 48 Conclusions and perspectives Key issue: to obtain operators that are adapted to the nature of the objects to be analyzed, not by deforming them, but by transforming the image itself. The conversion into polar-logarithmic coordinates as well as the derived cyclic morphology appears to be a field that may provide satisfying results in image analysis applied to round objects or spheroid-shaped 3D-object models. We have developed a methodology to describe in detail and classify the shape of cells. We have developed a model-based evolved methodology for segmenting the spots in fluorescence-marked microarray images, allowing an automatic adaptation to all the situations.

49 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 49 References M.A. Luengo-Oroz, J. Angulo, G.Flandrin, J. Klossa. Mathematical morphology in polar-logarithmic coordinates. In Proc. of the 2nd Iberian Conference on Pattern Recognition and Images Analysis (IbPRIA 04), Estoril, Portugal, Springer LNCS 3523, p J. Angulo. Automated spot classification in cdna images using mathematical morphology. Internal Note N-19/05/MM, Ecole des Mines de Paris, January 2005, 28 p. J. Angulo, F. Meyer. Spot segmentation in cdna images by computing minimal paths in polar coordinates. Internal Note N-20/05/MM, Ecole des Mines de Paris, June 2005, 73 p. A. Rosenfeld and J. Pfaltz. Distance functions on digital pictures. Pattern Recognition, 1:33 61, 1968.

50 Mathematical morphology in (log-)polar coordinates: Shape analysis and segmentation 50 L. Vincent. Minimal Path Algorithms for the Robust Detection of Linear Features in Gray Images. In Proc. of International Symposium on Mathematical Morphology (ISMM 98), Amsterdam, Kluwer, pp , June C. Sun, S. Pallottino. Circular shortest paths in images. Pattern Recognition, 36, , B. Appleton, C. Sun. Circular shortest paths by branch and bound. Pattern Recognition, 36, , B. Appleton, H. Talbot. Globally Optimal Geodesic Active Contours. Journal of Mathematical Imaging and Vision, 23, 67 86, 2005.