An Efficient Processing and Analysis Algorithm for Images Obtained from Immunoenzymatic Visualization of Secretory Activity

Transcription

1 1 An Efficient Processing and Analysis Algorithm for Images Obtained from Immunoenzymatic Visualization of Secretory Activity Wojciech Bieniecki *, Szymon Grabowski *, Dominik Sankowski *, Katarzyna Koscielska-Kasprzak **, Barbara Bernat ** and Marian Klinger ** Abstract - The paper presents some aspects of automatic processing and analysis of biomedical images obtained from immunoenzymatic visualization of secretory activity of single lymphocytes with the use of ELISPOT procedure. The ELISPOT research needs evaluation of specific morphological parameters of the objects appearing as round spots. The quantitative analysis of images is possible with the use of commercial systems, but the time and cost of such examination are unacceptable for scientific research. We have developed an efficient subsystem for unattended image segmentation that is based on local threshold supported by a region based method. Our results were successfully compared to those obtained with the commercial software. Additionally, as our method is more sensitive, it enables detection of small, low contrasted spots skipped by a commercial procedure. We take into account some morphological parameters of the segmented objects for a more accurate identification. We approach to evaluate the strength of the secretory activity by new measure based on segmented image intensity analysis, rather than on simple area measurement. Keywords ELISPOT procedure, image segmentation, quantitative image analysis. I. INTRODUCTION Enzyme linked immunospot assay (ELISPOT [1], [2] is a powerful technique used for detection and quantification of antigen specific immunological responses at the single cell level. The ELISPOT idea is based on the detection of single cells secreting a cytokine, which is the mediator of immunological response. Particularly, ELISPOT is suited for determination of alloreactivity of transplant recipient lymphocytes against donor antigens. It has been reported [3], [4], [5] that monitoring of cellular immunological responses after renal transplantation could have a prognostic value to diagnose the long-term graft outcome and to evaluate the level of immunosupression therapy needed. The research project performed at the Department of Nephrology and Transplantation Medicine of the Medical University in Wroclaw is focused on the optimization of the method based on the ELISPOT approach for determination of alloreactivity of renal transplant recipient in clinical practice. The goal of the project is to obtain a non-invasive diagnostic tool for prediction of long term renal allograft function and early detection of markers of chronic graft rejection process. *Computer Engineering Dept., Technical University of Lodz, Poland, wbieniec@kis.p.lodz.pl; **Dept. of Nephrology and Transplantation Medicine, Wroclaw Medical University, Poland, katarzyna_kk@o2.pl II. THE CONSIDERED IMAGES Our current research has focused on the enumeration of the peripheral blood recipient that are able to produce interferon gamma in response to stimulation with renal allograft donor antigens. The ELISPOT (Fig. 1 experiment is performed in membrane bottomed 96-well plates with each well bottom (6 mm diameter coated with antibodies specific against interferon γ. The known number of the analyzed recipient lymphocytes is incubated with or without donor lymphocytes to observe the donor specific immunological responses. Fig. 1 The diagram of the ELISPOT approach The membrane bound antibodies trap interferon gamma around the cytokine-secreting cells. When the cells are removed the antibody bound cytokine is detected with immunoenzymatic reaction biotin-streptavidine-alkaline phosphatase. The enzyme mediated conversion of the BCIP/NBT chromogen to insoluble blue product leads to spots visualization at the sites of former presence of the

2 2 cytokine secreting cells (Fig. 2. The image is microscopically analyzed or captured with digital camera at the resolution of at least pixels for quantification. Fig. 2. ELISPOT microscopic image Each spot obtained on the image is the shade of a single cell, and the spot radius and the color intensity depend on the amount of the cytokine secreted. The subject of the analysis is the spot number and their morphological parameters. The spot morphology is the result of actual cytokine productivity, lateral diffusion and dissociation. Theoretically, the spots are almost circular and the color intensity diminishes with the distance from the spot center (Fig. 3. It poses some problems for segmentation algorithms as spot boundaries are diffused and not well defined. Fig. 3. Exemplary spot dimensions and color intensity distribution III. IMAGE PROCESSING ISSUES All vision systems including our own, SpotView have similar data processing flow. Image acquisition and archiving. Inspected images are taken by a digital camera with macro function, that enables 20x magnification. The images are stored as color JPEG or TIFF files. We decided not to introduce any additional interface for camera control or any photography database system. Image preprocessing: This step performs the following tasks: extraction of ROI (region of interest, scale evaluation and preliminary filtering for segmentation. Segmentation is usually defined as a procedure that splits the whole image into connected, non-overlapping components which have strong correlation with real objects. In SpotView this goal is achieved in two steps: binarization with the use of adaptive thresholding, and region oriented segmentation. Once the segmentation parameters are set, such an approach enables unattended processing for all images. Object identification all segmented particles are examined by means of some morphological parameters to remove the shapes that are not expected for the spots. For the current research we do not require identification of more than one class of spots, in the future the two-color staining and analysis will be introduced. Morphology measurement apart from standard shape descriptors, as area, perimeter, roundness, compactness and aspect ratio, we use our own measurement that evaluates the amount of material in a spot. Our software was not intended for any statistical data processing. The results may be exported to Microsoft Excel workbook individually or in a form of histograms. IV. IMAGE PREPROCESSING PHASE An initial step of image processing in SpotView is an extraction of the region of interest. The ELISPOT reaction happens in round wells, and the camera acquires a rectangle area (Fig. 1. So, formally, the region of interest for the image: is a subset ROI = {( x, y : ( x, y [ 0, M ] [ 0 N ]} I =, {( x, y : ( x, y I ( x x + ( y y R } 0 ROI should enclose only a diaphragm with the spots. After opening the image file the program automatically draws a circle corresponding to our ROI. No sophisticated algorithm is used for this routine; simply it inscribes the circle to best fit the rectangle. In most cases such an approach is sufficient. After segmentation and object identification any shapes that touch the border of ROI and do not satisfy all morphological constraints are removed. For any case ROI may be adjusted manually (with a mouse. All further steps are performed for the ROI only. One of the frequently occurring image defects is nonuniform lighting. We mitigate this problem via background equalization. This step is performed with a convolution filter, which subtracts the background from the objects seen on the image. Formally, I Fil = I I M, where I input image, M set of pixels from a square mask (its dimension is chosen experimentally, and I Fil is the output image. We assume that, for our application, the mask size is 0.1 times the image diameter. The described filter works in RGB color model, which makes possible to equalize not only lightness but also color saturation, and consequently improves the accuracy of the next phases: desaturation (conversion to grayscale and gray histogram equalization. Those two operations are standard filters and do not require extra explanation. 0 (1 (2 (3

3 3 Finally, the image is smoothed by a median filter. All the filtering operations are presented in the scheme (Fig. 4. smoothing ROI drawing grayscale conversion background Fig. 4. Image preprocessing scheme equalization V. SEGMENTATION AND IDENTIFICATION The goal of the segmentation is to extract all the spots visible within ROI. While real ROI diameter is set to 6 mm, spot diameter varies from 0.05 mm to less than about 0.5 mm. As different image resolutions are used (from about 0.5 to 3 MegaPixels, the segmentation algorithm must capture all the objects regardless their dimensions. Additionally, the contrast of objects compared to surrounding background is poor, especially for the smallest spots. Preprocessing algorithms presented in Section IV enhance the image in a sufficient extent to make the segmentation possible. Image binarization by gray level thresholding is still one of the most efficient segmentation algorithms, and may be applied for ELISPOT images. Having in mind that even after image preprocessing the objects have different contrast and different dimensions, we decided to use the local threshold based algorithm. We chose a variant introduced by Bernsen [6], because it is known to be fast as it does not need to compute a histogram for a subimage. Each pixel ( i, j is considered with surrounding (usually square window W as well as in spatial filtering algorithms. ( (4 W ij = { x, y I : y i < b x j < b As a threshold we take the mean of the maximum and the minimum pixel intensity in W. T ( i, j = 0.5[max w ( I ( i + m, j + n + min w ( I ( i + m, j + n] = (5 = 0.5[ I high ( i, j + I low ( i, j] Additionally, the contrast for the search window for pixel i, is defined: ( j C( i, j = I ( i, j I ( i, j high If the contrast for pixel ( i, j is not sufficient (the threshold set experimentally; usually for images of global contrast 255 it is set as 15 the threshold value from eq. (5 is replaced by a global value τ 0. The value of τ 0 is obtained by another algorithm of adaptive threshold selection. We chose SIS (Simple Image Statistics [7]. Let us assume that the perfect image presents objects with intensity a over the background with intensity b, and because of some light and material non-uniformity the intensity values are distorted to some extent the noise is introduced to image pixels. Despite of this, the best threshold that discriminates the objects from background is τ = ( a + b 2. Let us call each image point p ( i, j, its intensity l ( i, j, and define its gradient module: e ( i, j = p = (7 = max l( i 1, j l( i + 1, j, l( i, j 1 l( i, j + 1 low { } Then we may state optimal threshold value as m n l( i, j e( i, j i= 1 i= 1 τ 0 =. m n e i= 1 i= 1 ( i, j The size of window W must be set experimentally before applying the Bernsen method. We show in [8] that the value must match the size of the objects. When W is too small, large plateaus (regions of small contrast are inadequately classified. With a too large window the method works slowly and tends to misclassify small spots. This observation led us to introduce a multi-pass algorithm which enables proper segmentation of spots from a given range of diameters. The algorithm classifies pixels in several passes each time doubling the size of the window W. Only pixels not classified in pass i are processed in pass i+1. Surprisingly, the algorithm works faster than one-pass variant, for the same final size of W. In [8] we discuss different variants of this algorithm and test its speed and accuracy. In application to ELISPOT images the range of W has been set identically as expected range of spot diameter, i.e., [0.05 mm, 0.5 mm]. Actually the size is changed to pixel (6 (8

4 4 scale, and the top value of range is rounded to nearest power of 2 multiplied by the bottom value. A process of segmentation by different thresholding techniques is presented in Fig. 5. Grayscale image SIS thresholding Fig. 6. Particles separation with aid of watershed lines Bernsen thresholding Multi-pass Bernsen thresholding Fig. 5. Results of thresholding binarization variants Image binarization is not sufficient, as neighboring spots after binarization may produce a single object. A deagglomerating algorithm is needed to separate touching components. A solution, successfully used in our previous research [9], is watershed algorithm. Watershed transform is applied for the grayscale, smoothed image. The grid of watershed pixels is multiplied by binarized image. Such an operation makes all pixels belong to watershed grid background pixels (Fig. 6. We tested different watershed algorithms for segmentation accuracy and finally chose our own variant [9]. Finally, pixels that form connected components are labeled. VI. OBJECT IDENTIFICATION For all found connected components, the following measures are computed: area a value proportional to number of pixels in one component is computed with use of the scale info; perimeter the value is evaluated by contour extraction and is expressed only in pixel scale. This value helps to compute the shape descriptors, which are: roundness, compactness and aspect ratio. In this step the segmented image is filtered to rule out the components that do not seem to be spots: delete all components that touch the border of ROI; delete all components smaller than 0.03 mm; delete all components, for which the compactness is less than 50%. In fact, the spots are not exactly round and the stain distribution within the spot is not exactly as given in Fig. 3. VII. SPOT AREA DISTRIBUTION ANALYSIS We carried out a statistical comparison of the results obtained by our segmentation algorithm and the one used in the system ImmunoSpot [10], courtesy of Cellular Technology Ltd, USA. Table 1 contains number of spots (N extracted from 10 images obtained by these two applications. Our algorithm seems to be more sensitive for small hardly visible objects which are omitted by a commercial software. It results in large difference between N for some images. All analyzed images have been manually inspected for any segmentation errors: skipped spots or image noise/dirt classified as spots. Both programs did not recognize false spots, but some spots were misdetected (Fig. 7. It can be seen that SpotView perceives tighter contours of the spots.

5 5 Sample no. C8 D6 D12 E8 G12 N Immunospot N SpotView goodness of fit σ 2 95% Yes Yes Yes Yes Yes goodness of fit x 95% Yes Yes No Yes Yes IMMUNOSPOT SPOTVIEW Fig. 7. A fragment of segmented image In Immunospot output data are available as histograms of spot area rescaling the values to log mm 2. SpotView, on the other hand, outputs the area values individually for each spot, but for compatibility we decided to build the histograms in the same way. For each histogram we computed the mean and the variance. Variances were statistically compared with F- Snedecor variance test and for successfully compared pairs mean values have been tested with t-student test. We can observe that despite some differences between the number of detected spots, the distributions (which are in fact the most important data in 7 cases are fitted. TABLE 1 THE COUNTS N, MEAN VALUE x AND VARIANCE σ 2 FOR SELECTED SAMPLES Sample no. B4 B5 B6 C4 C7 N Immunospot N SpotView goodness of fit σ 2 95% Yes Yes Yes Yes No goodness of fit x 95% Yes Yes Yes Yes N/A Sample no. C8 D6 D12 E8 G12 N Immunospot N SpotView goodness of fit σ 2 95% Yes Yes No Yes No goodness of fit x 95% Yes Yes N/A Yes N/A Table 2 presents the comparison of spot area distribution if the smallest objects (diagonal less than 0.12 mm were filtered out. The experiment was based on the assumption that Immunospot does not detect the smallest spots. The results show that for medium and big spots both programs return the same area distributions. TABLE 2 THE COUNTS N, MEAN VALUE x AND VARIANCE σ 2 FOR THE FILTERED SAMPLES Sample no. B4 B5 B6 C4 C7 N Immunospot N SpotView goodness of fit σ 2 95% Yes Yes Yes Yes Yes goodness of fit x 95% No Yes Yes Yes Yes VIII. AREA VS. WEIGHTED AREA As stated before, the aim of the image analysis is to estimate the amount of cytokine secreted by each lymphocyte. Due to the compound nature of color distribution (actual secretion level, lateral diffusion and dissociation, it is clear that the amount of cytokine is not directly proportional to the spot area. To perform a more accurate measurement we introduced another descriptor called weighted area. The measure adds the intensity values of each pixel of original image but within individual connected component. The value in fact needs some scaling. Let { S 1, S2,..., S k } be a set of all connected components segmented from image I; Lmin = min { l( x, y } the minimum pixel intensity of original ( x, y U S i image within all components, and Lmax = max { l( x, y } the ( x, y U S i maximum pixel intensity of original image within all components. We define a weighted area of the component as follows: W k L ( max (, max l x, y nk L l x y = = : ( x y L L, S max min k L max L We performed the tests for this measure and compared the results (Table 3 of area distribution for 10 images. TABLE 3 MEAN VALUE x AND VARIANCE σ 2 FOR LOG10 OF THE AREA Sample no. B4 B5 B6 C4 C7 N SpotView Weight σ Area σ Weight x Area x Sample no. C8 D6 D12 E8 G12 N SpotView Weight σ Area σ Weight x Area x Additionally, the results were presented in a form of the histograms (Figs 8 and 9. min (9

6 6 cooperation of Technical University of Lodz and Medical University in Wroclaw, may become a valuable tool for further research. Fig. 8. Spot area vs. weighted area distributions for samples B4, B5 Fig. 9. Spot area vs. weighted area distributions for samples B6, C4 IX. CONCLUSION Our segmentation algorithm is comparable to the one incorporated in the commercial Immunospot system. Our application, however, is capable of detecting smaller spots. We have introduced a new measure, weighted_area, which, intuitively, better approximates the degree of immunological reaction. We showed that the area and weighted area distributions vary significantly, which suggests that the plain area is not an appropriate indicator of lymphocyte secretory activity. The comparison of the results obtained with SpotView against the results offered by Immunospot, a respected system for ELISPOT image processing and analysis, allows to claim that our image processing algorithms are appropriate for this application. Working with Immunospot is costly and time-consuming, therefore our software, a fruit of the ACKNOWLEDGMENTS This work has been supported partially by Polish State Committee for scientific research (grant no. 3T11E Katarzyna Koscielska-Kasprzak is KKK is a laureate of the Scholarship for Young Scientists of the Foundation for Polish Science. REFERENCES [1] Versteegen J. M., Logtenberg T., Ballieux R. E., Enumeration of IFN-gamma-producing human lymphocytes by spot-elisa. A method to detect lymphokineproducing lymphocytes at the single-cell level. J. Immunol. Methods. vol. 111, pp , [2] Tary-Lehmann M., Hricik D. E., Justice A. C., Potter N. S., Heeger P. S., Enzyme-linked immunosorbent assay spot detection of interferon-gamma and interleukin 5- producing cells as a predictive marker for renal allograft failure. Transplantation, vol. 66, pp , [3] Gebauer B. S., Hricik D.E., Atallah A., Bryan K., Riley J., Tary-Lehmann M., Greenspan N. S., Dejelo C., Boehm B. O., Hering B. J., Heeger P. S., Evolution of the enzyme-linked immunosorbent spot assay for posttransplant alloreactivity as a potentially useful immune monitoring tool. Am. J. Transplant. vol. 2, pp , [4] Hricik D. E., Rodriguez V., Riley J., Bryan K., Tary- Lehmann M., Greenspan N., Dejelo C., Schulak J. A., Heeger P. S., Enzyme linked immunosorbent spot (ELISPOT assay for interferon-gamma independently predicts renal function in kidney transplant recipients. Am. J. Transplant. vol. 3, pp , [5] Heeger P. S., T-cell allorecognition and transplant rejection: a summary and update. Am. J. Transplant. vol. 3, pp , [6] Bernsen J., Dynamic thresholding of grey-level images, Proceedings 8th International Conference on Pattern Recognition, Paris, pp , [7] Sahoo P. K., Soltani S., Wong A. C. and Chen Y. C., A survey of thresholding techniques, Computer Vision, Graphics and Image Processing, vol. 41, pp , [8] Bieniecki W, Grabowski Sz. Multi-pass approach to adaptive thresholding based image segmentation, this conference. [9] Bieniecki W., Oversegmentation avoidance in watershed-based algorithms for color images, Proceedings of TCSET 2004, Lviv, 2004, pp [10] Karulin, A. Y., Hesse M. D., Tary-Lehmann M., Lehmann P. V., Single-cytokine-producing CD4 memory

7 7 cells prevail in vivo, in type 1/type 2 immunity. J. Immunol. vol. 164, pp , 2000.