Edge Tracking of subjective contours in Biomedical Imaging

Edge Tracking of subjective contours in Biomedical Imaging Giovanni Garibotto(*), Valentina Garibotto (**) * Elsag spa, Genova, Italy ** Un. La Bicocca, Milano, Italy giovanni.garibotto@elsagdatamat.com Abstract The paper describes a method for computer-assisted edge tracking and following by local profile matching. It is used primarily to integrate and support manual selection when complex contours have to be found and identified. Such a requirement is quite common in biomedical applications, including a wide spectrum of image modalities, from histological samples to MRI images. The referred example takes part in the analysis of in vitro receptor auto-radiographic data, for the architectonic characterization of transmitter receptors within a defined cortical region. The practical validation we provide of the proposed tracking model reveals that it is a very promising approach, enabling accurate and fast processing of multiple images. 1. Introduction Edge detection and tracking represents one of the fundamental early processing steps in most imaging and computer vision applications. This is particularly important for tissue segmentation and classification in biomedical imaging. The goal of most imaging applications in the biomedical field is the quantification of different parameters that allow differential characterization of biological tissues and regions. The preliminary step for all such measurements consists in the definition of a region of interest and/or segmentation of the image in its constituting elements. Strategies to address this problem are various tissue classification approaches, whereby single pixel elements are assigned to a tissue class according to their intensities. However, there are no standard tools available yet for 2D images having a high spatial resolution and a relatively poor contrast. These images are commonly found when referring to many different biomedical fields, including histological samples and autoradiographies, electron-microscopic sections, and also in the emerging field of high resolution magnetic resonance imaging (MRI). In these application, the contour definition is currently performed manually, as described in Eickhoff et al., 2007, [1], Sätzler et al, 2002 [2], Walters et al., 2007, [3], and this necessarily increases the analysis time and work burden. In particular, the processing system described in this paper arises from a reference application as described in [1] for the analysis of the concentration of transmitter receptors in the human cerebral cortex. This method is based on the analysis of in vitro receptor auto-radiographic data, aimed to define regional cortical receptor distribution patterns. The cortical contours show a large variability in terms of contrast and are sometime difficult to be detected and identified also by a human operator. Quite often the edge contours must be extrapolated by exploiting visual continuity constraints (subjective contours). On the other hand a complete manual operation is very heavy to be performed, since there are extended areas where edge position can be reasonably predicted and tracked. As such, the best solution seems to be a semi-automatic edge feature tracking approach with human supervision (man in the loop). By the way, this requirement is quite common in many other context of computer vision where edge line features must be detected and fitted with continuous curves. This is the case of most biomedical imaging applications with high resolution images of radiographic records (ribs and other bone features) as well in the identification of relevant areas in the brain images. The technique described in this paper is limited to grey-level images, although it may be easily extended to deal with color pictures too. Main requirements of this edge contour representation are an accurate and regular description of the subjective profile, as well as a mono-pixel representation of the connected edge chain with 4-connectivity constraints, to allow an Proceedings of the 14 th International Conference on Image Analysis & Processing (ICIAP 2007), 10-13 Sep., Modena, Italy

effective implementation of the following steps in the processing chain (region filling with Bresenham line interpolation). 2. Overview of the proposed approach Edge contours are described as a sequence of points that exhibit a similar orthogonal grey level profile. The first step of the process consists in the evaluation of a sequence of interpolation points the density of which is variable and proportional to the degree of curvature of the contour itself. This is a very general model of an edge profile to fit almost all possible situations from high-contrast edges up to highly smoothed transitions. HMI selection of 2 starting points Edge-profile model update HMI correction of last edge position & new edge point selection End of the process Edge-point prediction / tracking by correlation Stop criterion low-correlation Fig.1. flow diagram of the edge tracking process. HMI: Human Machine Interaction The proposed approach is a supervised edge detection and tracking where human intervention is required when the system is unable to decide with a sufficient level of confidence the appropriate position of the next interpolation edge point in the chain. The processing sequence is a combination of local edge chain fitting and tracking, whose length and extension depends on the continuity and regularity of the edge itself. The confidence criterion is based on the matching of the current edge profile with an edge model that is continuously updated along the contour chain. When the accuracy level falls below a suitable threshold, the model function is reset according to the new human selection. The resulting interpolated edge point sequence may be affected by some irregularities as possible intersections, loops, adjacency, that may prevent the following steps of the biomedical application, i.e. region filling and analysis. As such a further step of chain post-processing is required, to achieve a mono-pixel representation of the resulting edge chain sequence. 3. Edge following by local profile matching The starting point of the edge chain must be detected by the human operator in the image. A second manual point is also required, to define the starting local direction v e of the edge contour. Both edge points are used to build a model of the local edge profile. A local smoothing is applied by weighting the edge profile on a local neighborhood across the edge, to reduce noise effects and local irregularities. This regularization is particularly important when dealing with poor contrast images with a low level of signal to noise ratio. Then the automatic process starts, by predicting the next candidate position along the contour direction v e (with a step that is proportional to the local curvature of the edge chain). The new predicted position is corrected, across the edge profile, in the gradient direction v g orthogonal to the edge direction v e to get the maximum of correlation with respect to the current model. The process is repeated by computing the new edge direction v e between the last corrected point and the previous one. This method is summarized in the flow diagram of fig.1. An example of the edge profile matching process is shown in fig.2 where the new edge position s E along the gradient direction s, is shifted of the amount d to get the maximum value of the normalized cross covariance function: C(d) = [P(s-d)-P m ] [M(s)-M m ] ( [P(s-d)-P m ] 2 ) 1/2 ( [M(s)-M m ] 2 ) 1/2 (1) where P(s) is the current edge profile with P m as the average value, and M(s) is the current profile model with M m as the average value. The shift value d corresponds to the best estimation of the new edge position. The process is repeated until the covariance function C(d) has reached a value below a minimum level of acceptance (index of uncertainty). Hence the automatic search process stops and the human intervention is required to confirm the current position (or make a correction).

The edge profile model updating is a critical process with a risk of divergence when dealing with ambiguous contours due to the very local analysis of the automated process. 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 Fig.2 Matching between the edge profile P(s) ( ) with the current model M(s) ( ). re-projected available contour points and the red line is the results of editing and final edge chain construction as described in the next section. The editing tool provides a series of functions to the human operator, starting from the simple translation and rotation of the full edge chain to correct the main large differences of alignment and registration. Unfortunately a rigid roto-translation is not sufficient since most differences are due to local deformations. The proposed solution is an elastic matching process on the edge points before and after the one that has been corrected by the human operator. He/she may select any part of the contour that is considered too far from ideal position and shift it to the right place. The nearby contour points are modified in such a way to achieve the best fit to the ideal edge profile automatically, by weighting the amount of the edgeshift with the distance along the edge sequence. This is another reason to support the semiautomatic approach and the human supervision to correct any possible error of the system in such variable unconstrained situations. The result of this processing phase is a sequence of edge points with a variable displacement. 4. Contour editing by elastic matching Due to the complexity of the contour shape and variable contrast, it is possible to get a result that needs some additional corrections. In fact some of the detected points may be not properly located at the edge position and the overall shape of the contour sequence may be not fully satisfactory. An editing process is then implemented, to support the human operator in inserting new points, replacing wrong points, and improving the shape of the contour when necessary. Another motivation for the implementation of an interactive editing process is the possibility to transfer some of the already detected contour sequences to other similar images. Such a situation is quite common when dealing with temporal sequences (like heart video recordings) as well with 2D high resolution sectioning of 3D volumes. This is also the case in our reference applications where neighbouring sections are quite similar, considering the individual thickness of 20 µm of each section. Hence, the already available contours, from previous sections, may fit some part of the brain cortical shape, and it is more effective to re-shape the edge chain rather than rebuild it from scratch. An example of such full contour prediction is referred in fig.3. The light blue dots correspond to the Fig.3. Example of re-projection of a sequence of edge points (light-blue) onto neighbouring images and the matching results (red line). The cost function to be minimized is based on the usual cross covariance (1) of the edge profile computed for all neighbouring edge points, with continuity constraints on the first derivative of the interpolated edge contour chain (local curvature constraint). The first order approximation of the edge shift is further optimized for each point (in a local

neighbourhood of the corrected edge position) in the direction orthogonal to the edge. 5. Continuous edge chain representation The final step of the process consists in the reconstruction of the full edge chain sequence, starting from the list of the edge tracking result. The continuous chain sequence is obtained by an interpolation function that may be selected (parametric) by the user according to the desired level of curve smoothing. Unfortunately the resulting edge chain sequence may exhibit some local irregularities as multiple touching pixels especially in correspondence to curvature discontinuities or even some local loops. In our application [1] it is required to recover a mono-pixel chain code representation to simplify the following steps of region filling. To achieve such result it was necessary to develop a chain-parsing algorithm based on a local analysis of the 3x3 neighbourhood around the current edge point. The problem can be stated as follows: for any edge point of the chain P 0 =P[ind], only two edge connections are allowed in the local 3x3 neighbourhood as shown below P + P - P 0 P - P 0 P + Case 2. The triplet (P 0,P 1,P 2 ) of the last 3 consecutive points must be considered. With the assumption that the previous connection (P 1,P 2 ) is correct, the only irregular situation may occur when P 0 is found in the same position of P 2 corresponding to an opposite direction along the edge chain. In this case both P 0 and P 1 must be removed from the chain. Case 3: The current edge point P 0 should have only one busy connection in the local neighbourhood (corresponding to the previous edge point in the chain P 1 ) in both 8-connectivity and 4-connectivity models. Otherwise, if there is another edge point P k connected it is necessary to keep P 0, and remove all the intermediate points (from P 1 to P k ) since they would correspond to a local loop. It is worth to remind that these are the only possible situations that may occur and are recursively corrected on a local buffer of edge chain points to satisfy the 1- point connectivity constraint. The length of the buffer determines the length of the loops that can be removed to prevent any edge line crossing effect. 6. Interactive Human Interface A window application has been developed to improve human operation through a user-friendly interface. Fig 5 shows a screen-shot of the HMI (Human Machine Interaction) environment. It provides an overall view of the full image on the left, and the corresponding zoom view on the right, where human corrections and interactions are performed, through mouse interaction. Fig.4 Example of a mono-pixel chain test: 8-connectivity (left) and 4-connectivity (right) The removal of spurious edge points is quite simple, but extremely expensive if performed on the image map. A much more effective solution consists in exploiting the causal sequential representation of the edge chain. The proposed solution is a recursive process that takes into account all possible combination of position of the last edge points of the chain, namely points P 0 =P[ind], P 1 =P[ind-1], P 2 =P[ind-2] P k =P[indk]. This approach can be used both in the case of 4- connectivity and 8-connectivity constraints. A buffer of NP points is fed from the last position P 0 at any new interpolation point. The following situations must be checked, as follows: Case 1: the two last consecutive points (P 0, P 1 ) must satisfy regular continuity. This is achieved by construction and is always valid. Fig.5 Screen-shot of the HMI interactive environment On the right side of the screen the HMI interface contains the list of control keys to perform: Input-output image management

Selection of manual / automatic tracking of the edge points The editing menu of previously collected edge chain sequences. Start/stop of the tracking process On-line updating of the process parameters 7. Results The proposed approach has been used for the processing of a large number of images of variable contrast. The reference application has been described in [1] for the analysis of the concentration of transmitter receptors in the human cerebral cortex. cryostat microtome at 20 C. Alternating sections were histological stained, for cell bodies or incubated with sixteen different receptors of all classical transmitter systems. Due to the complexity of the task, the standard method that has been used so far consists in a fully manual segmentation of the contour profiles that are approximated by a series of segments of variable length. As a consequence, the accuracy of the contour representation is manually controlled by the human operator and the time required for each sample image is quite relevant (depending on the complexity of the scene). It is quite clear the great potential advantage of a computer assisted approach, considering the large size of the image samples (approximately in the range of 3500 x 5500 pixels). Such large size images require always a multi-scale control by the human operator, to manage the wide-field view of the overall edge contour shape (lower-resolution) together with a more precise analysis of the edge details (high-resolution requirement). Moreover the database of images to be processed is very huge (more than 700 images per hemisphere are currently available) with a proportional cost of the processing time. Fig.6 Image sample of the cortical shape This method is based on the analysis of in vitro receptor auto-radiographic data, aimed to define regional cortical receptor distribution patterns. Briefly, human hemispheres were obtained at autopsy from subjects with no record of neurological or psychiatric diseases and used for quantitative in vitro receptor auto-radiography. All subjects had given written consent before death and had been included in the body donor program of the Department of Anatomy at the University of Düsseldorf, Germany. Serial coronal sections (thickness 20 µm) were cut using a large-scale Fig.7. Contour tracking result on a small piece of the biomedical image sample The proposed system has been used for the edge reconstruction of the full sequence of brain slices. The obtained results have been found very satisfactory and have been judged comparable to the best quality achieved by full manual operations. In fact the human operator is often constrained to reduce the number of edge points due to time constraints as well as for a poor

perception of the local similarity of the edge profiles. The availability of a quantitative matching process is really the most relevant achievement of the proposed edge-tracking process, in terms of accuracy of edge chain reconstruction, more stable area filling within edge tracked contours, and better interpolation of smoothed edge curvatures. The improved accuracy of the automatic process has proved quite relevant for the success of the specific biomedical application. Another great advantage can be measured by the processing time that has been reduced by more than 60% by the use of the semi-automatic process, as compared to the traditional manual data acquisition scheme. 8. Conclusions The paper describes a method for computer-assisted edge tracking and following by local profile matching. It is a very specific solution used primarily to support human operation when complex contours have to be found and identified. It has been motivated by an application of reconstruction and representation of auto-radiographic data on brain sections. Anyway it may be successfully used also in a large variety of other applications including image processing and computer vision and many other biomedical imaging applications, including histological sections, digital radiographies and high resolution MRI imaging. The proposed solution is based on quite standard image processing and pattern recognition technology. Edge tracking is performed with an adaptive control of the local direction and a continuous update of the gradient profile model. Finally a quite significant contribution comes from the chain parsing algorithm that has been developed to achieve an error-free mono-pixel reconstruction of the connected edge chain sequence, an essential requirement for any further processing step of region filling and analysis. Acknowledgment The authors would like to thank the Institute of Medicine of the Jülich Forschungzentrum, and in particular Dr. S.B.Eickhoff, Dr. N. Palomero-Gallagher and Prof. K. Zilles, for the kind support in providing the sample data for the current research. Many thanks are also due to Luisa DeVena (OnAir srl) for the support in the implementation of the Human Interaction application. 9. References [1] Eickhoff S.B., et al. «Analysis of neurotransmitter receptor distribution patterns in the cerebral cortex», Neuroimage. 34(4):1317-30 (2007). [2] Sätzler K, et al. Three-dimensional reconstruction of a calyx of held and its postsynaptic principal neuron in the medial nucleus of the trapezoid body. J. Neuroscience 22, 10567-10579. (2002) [3] Walters N.B., et al Observer-Independent Analysis of High-Resolution MR Images of the Human Cerebral Cortex: In Vivo Delineation of Cortical Areas. Human Brain Mapping 28:1 8(2007) [4] Takashi Fujita et al. Semi-automatic tracking of the diaphragm contour in X-ray image sequences: preliminary results, Proceedings of the Digital Imaging Computing: Techniques and Applications (DICTA 2005). [5] Tae-Yong Kim, Jihun Park, Seong-Whan Lee, Object Boundary Edge Selection using normal direction derivatives of a Contour in a Complex Scene, Proc. Of the 17 th Int. Conf. on Pattern Recognition (ICPR 04), 2004.