Robust object characterization from lensless microscopy videos

Transcription

1 Robut object characterization from lenle microcopy video Olivier Flaeur, Loïc Deni, Corinne Fournier, and Éric Thiébaut Univ Lyon, UJM-Saint-Etienne, CNRS, Intitut d Optique Graduate School, Laboratoire Hubert Curien UMR 5516, F-42023, Saint-Etienne, France Univ Lyon, Univ Lyon1, ENS de Lyon, CNRS, Centre de Recherche Atrophyique de Lyon UMR5574, F-69230, Saint-Geni-Laval, France Abtract Lenle microcopy, alo known a in-line digital holography, i a 3D quantitative imaging method ued in variou field including microfluidic and biomedical imaging. To etimate the ize and 3D location of microcopic object in hologram, maximum likelihood method have been hown to outperform traditional approache baed on 3D image recontruction followed by 3D image analyi. However, the preence of object other than the object of interet may bia maximum likelihood etimate. Uing experimental video of hologram, we how that replacing the maximum likelihood with a robut etimation procedure reduce thi bia. We propoe a criterion baed on the interection of confidence interval in order to automatically et the level that ditinguihe between inlier and outlier. We how that thi criterion achieve a bia / variance trade-off. We alo how that joint analyi of a equence of hologram uing the robut procedure i hown to further improve etimation accuracy. I. INTRODUCTION Thank to the maive development of low-cot imaging enor and the imultaneou increae in computational capabilitie, lenle microcopy i emerging a a method of choice for time-reolved analyi of microcopic object. It i already ued in everal field where the accurate etimation of 3D location and ize over time i crucial, uch a in the tudy of fluid flow or biomedical imaging [1]. Analyi of digital hologram i traditionally performed by firt recontructing the 3D volume by light back-propagation, then analyzing thi 3D volume to egment, locate and ize object [2]. However, thi approach uffer from artifact due to the imperfect recontruction tep, in particular to patial ditortion cloe to the image border and puriou diffraction pattern known a twin image in holography [2]. Location and ize etimate can be ignificantly improved by uing a maximum likelihood etimator, i.e., by fitting a model of the diffraction pattern generated by the object of interet directly on the hologram rather than by recontructing the image volume. Thi method i very ucceful for the tudy of iolated object [3], [4], but ome bia appear when unwanted object in the field of view alter the hologram. Thee object may have complex or unknown hape and their influence on the hologram may be difficult to explicitly model and account for. We ugget replacing the maximum likelihood etimator with a robut etimator o that any notable dicrepancie between the model of the pattern of interet and the data due to unwanted object have le influence on the final etimate. Section II provide a brief preentation of the lenle microcopy principle and the model of the diffraction pattern generated by pherical object behaving a opaque object uch a droplet, cell or cocci bacteria. Section III then decribe the propoed robut etimation procedure to characterize microcopic object. Special attention i paid on the etting of the level to ditinguih between inlier and outlier. An automatic criterion baed on the interection of confidence interval i propoed to achieve a bia / variance tradeoff. Section IV illutrate the performance of the propoed method on holographic video. II. CHARACTERIZATION OF MICROSCOPIC OBJECTS WITH LENSLESS IMAGING Lenle microcope are baed on the in-line holographic etup originally propoed by Denni Gabor [5]. The object are illuminated by an incident monochromatic plane wave of wavelength λ. After free-pace propagation over ditance z p, a hologram d i recorded by a monochromatic image enor, ee Fig. 1. Baed on diffraction theory, it i poible to model the diffraction pattern created on the hologram plane by an object of interet of given 3D hape, aborption and optical index. The cae of opaque pherical object i of particular practical interet ince it cover opaque particle introduced within the flow in fluid mechanic, and colored pherical object uch a cocci bacteria or cell in flow cytometry in biology. When the ditance z p i uch that z p 4π.r2 p λ, the diffraction pattern of an opaque phere located at (x p, y p, z p ), with radiu r p, i [6]: m p (x, y) = π ( ) ( ) r2 p 2π rp ρ p π ρp J 1c in, (1) λ z p λ z p λ z p where ρ p = (x x p ) 2 + (y y p ) 2 i the tranveral ditance to the center of the phere and J 1c (u) = J 1 (u)/u i the cardinal Beel function of firt order. More generally, in the following m (a, b) denote the intenity of the diffraction pattern generated at the pixel location (a, b) on the enor by an object of interet. Thi object of interet i fully characterized by a vector of parameter (e.g., for an opaque phere = {x p, y p, z p, r p, }). ISBN EURASIP

2 Fig. 1. Lenle microcopy etup and object fitting. Maximum likelihood etimation of the parameter of the object of interet, under the hypothei of additive white Gauian noie, amount to olving a nonlinear leat-quare problem [3]: ˆ (ML) = arg min w(a, b) [m (a, b) d(a, b)] 2, (2) where d(a, b) repreent the recorded data, w(a, b) 1/ɛ(a, b) 2 i a weight that i et to zero for defective pixel and that can account for a non-tationary noie variance ɛ(a, b) 2, and A and B are the height and width of the enor, in pixel. III. PROPOSED ROBUST ESTIMATION SCHEME A. Robut etimation by reweighted leat quare To reduce the impact of outlier on the etimation, Huber introduced the o-called M-etimator [7] that replace the leat quare by another objective function ρ in order to reduce the penalization of the larget deviation between the model and the data. Minimization problem (2) i then replaced by: ˆ (M) = arg min ( m (a, b) d(a, b) w(a, b) ρ ), (3) where ρ i a non-negative, continuou and ymmetric function with a minimum equal to 0 at 0, and i a parameter that cale the reidual by etting the level that ditinguihe between inlier and outlier. Depending on the choice of the function ρ, the minimization problem (3) can be difficult to olve. A imple algorithmic trategy leading to a local minimizer conit in olving a equence of leat-quare problem of the form (2): the Iteratively Reweighted Leat Square (IRLS) [8]. In thi cheme, the etimate ˆ k+1 at iteration k +1 i obtained by reweighting the quare with weight w k (a, b) derived from the reidual of iteration k: ˆ k+1 := arg min w k (a, b) [m (a, b) d(a, b)] 2, (4) where the weight w k (a, b) are computed from the reidual r k (a, b) = m k (a, b) d(a, b) at iteration k according to an update rule that depend on the objective function ρ: w k (a, b) = w(a, b) r k (a, b) ρ(u). (5) u u=rk (a,b)/ Many different objective function ρ and their aociated weight update rule have been tudied and claified in the literature [9]. So-called hard-redecender are function ρ that are contant above a given threhold, thereby leading to aign zero weight to data point that diplay a mifit larger than the choen threhold. So-called oft-redecender are function ρ whoe limit at infinity i finite, thu aymptotically producing zero weight for large reidual value. For lenle application, we elected a oft-redecender to achieve a good compromie between bia reduction and lo of efficiency of the etimator. In the following, we ue the Cauchy objective function: ρ(u) = 1 2 log(1 + u2 ), (6) which give the following weight update: w k (a, b) = w(a, b) [1 + (r k (a, b)/) 2] 1. (7) B. Automatic tuning of parameter : ditinguihing inlier and outlier Adequate tuning of the parameter i eential in robut etimation cheme [10]. On the one hand, if i too low, mot reidual r k (a, b) will be (much) larger than and the weight of the correponding data point will decreae in the following iteration, thereby dicarding mot of the ignal of interet. In thi cae, the reult will be an increae in the variance of the etimation. On the other hand, if i too large (much larger than the typical dicrepancy of outlier), the M-etimator tend to the leat quare etimator and loe it robutne propertie. In thi cae, reduction in the bia caued by outlier i no longer obtained. A common method to etimate the cale parameter in the cae of i.i.d. noie i baed on the median abolute deviation (MAD) etimator, which provide a robut etimate of the reidual tandard-deviation [7]: ŝ (MAD) = 1.48 median [ r median(r) ], (8) where r denote the collection of all reidual value r(a, b) for a [1, A] and b [1, B]. MM-etimator have been propoed to achieve high robutne to the preence of outlier while conerving high efficiency in the abence of outlier[11]. Thee etimator conit in two tep: (i) etimation of the cale of the reidual uing an M-etimator defined by a hard-redecender ρ; and (ii) actual etimation of the parameter of interet with a more efficient penalty function ρ (uch a a oft-redecender). One drawback of thee method of etting parameter i that they do not account for the actual impact of the outlier on the etimation. When conidering unwanted object within the field of view, the location of the object with repect to the object of interet may or may not lead to biae. In other word, while ŝ (MAD) i uitable in cae where the outlier are uniformly ditributed in the meaured ignal, a more appropriate choice of could be made in cae of nonuniform ditribution of outlier. We propoe to etimate parameter uing a method that account for the actual impact of the outlier on the etimation. Starting from the value ŝ (MAD), which may be overconervative, we apply the interection of confidence interval (ICI) rule to decide up to what point parameter can be ISBN EURASIP

3 Fig. 2. Etimation of with the ICI rule. ˆ (M) increaed to reduce etimation variance before being ubject to bia due to outlier. The ICI rule ha been uccefully applied in image denoiing to locally elect the larget neighborhood on which to perform denoiing by local polynomial approximation [12]. It application only require knowledge of the etimate along with it tandard deviation. The vector of parameter i etimated uing the robut IRLS etimation cheme (4) by progreively increaing parameter until the new etimate obtained with parameter lead to a confidence interval [ˆ (M) (M) δ, ˆ + δ ] with an empty interection with the preceding confidence interval: { ŝ (ICI) = max [ ŝ (MAD), ), 0 [ ŝ (MAD), ], [ˆ(M) 0 δ 0, ˆ (M) 0 + δ 0 ] }. (9) Thi principle i illutrated in Fig. 2 conidering a ingle parameter. To build the confidence interval, we ued the Cramér-Rao Lower Bound (CRLB) [13]. The CRLB δ(i) repreent the minimum tandard-deviation for any unbiaed etimator of the parameter (i). Under additive white Gauian noie, the leat quare etimator (2) aymptotically reache the CRLB given by: δ 2 (i) = [ I() 1] i,i, (10) where Fiher information matrix I i defined by: [I()] i,j = w(a, b) m (a, b) m (a, b), (11) (i) (j) In the preence of outlier, we approximate the confidence interval by computing the CRLB with the weight obtained by the IRLS procedure. To reduce the computational cot of etimating parameter according to the ICI rule (9), we conider a non-uniform dicretization of the et [ ŝ (MAD), ) uch that two ucceive value lead to a given decreae in the CRLB. In practice, each value i obtained by biection to achieve the precribed CRLB reduction. We now illutrate the tuning of parameter baed on the ICI rule (9) on a imple 1D problem: etimating the x p Fig. 3. Etimation of the poition of a pattern by a robut approach uing the MAD and the propoed ICI rule. Evolution of ŝ and MSE are repectively in dah and olid line. location of a Gauian-haped pattern m. We imulate 100 noiy realization d by adding to the model a Gauian white noie (with a peak ignal-to-noie ratio of 10) and a contant outlier formed by a rectangular function with 3 ample in width (with a peak ignal-to-noie ratio of 1), ee Fig. 3(a). The ICI rule lead to value of that are about 5 time larger than ŝ (MAD) when the outlier i too far to introduce a ignificant bia, and to value of cloe to that of ŝ (MAD) when the outlier affect the etimation, ee Fig. 3(b). When ŝ (ICI) i higher than ŝ (MAD) (reulting in a weaker penalization of the reidual), the mean quare error (MSE) on the location of the pattern i reduced by a factor between 2 and 4. In practice, the empirical tandard-deviation reache the CLRB when the outlier ha little impact on the etimate. A difference of a few ten of percent i oberved when the contant outlier ha an impact on the etimate ince the etimator i no longer aymptotically unbiaed. Determination of with an ICI rule i till poible. In concluion, chooing the parameter with the ICI rule improve the mean quare error compared to a MAD etimator. IV. APPLICATION TO LENSLESS MICROSCOPY VIDEOS In thi ection, the performance of the robut etimate cheme applied in lenle microcopy i evaluated on two lenle imaging application. The goal i to accurately etimate the radiu r p of pherical object. We alo evaluate the performance of a robut etimation performed jointly over everal conecutive frame (uper-reolution). In thee example, the image formation model m i non-linear with repect to the parameter, a given by equation (1). A a reult, the minimization problem (2) and each IRLS iteration in (4) i olved uing a few iteration of Levenberg-Marquardt algorithm. The original video and the reult of the robut etimation are available online at fr/deniloic/robut lenle video.html. A. Holographic video of an object of contant ize We conider a 200 frame video of a lightly moving particle located near an unwanted object. The object of interet ISBN EURASIP

4 Fig. 4. Calibrated particle near an outlier example of a frame. i a circular chromium depoit on a gla plate, with a calibrated radiu of 50 µm (Optimak TM, diameter ± 1 µm, roundne error ± 0.25 µm). The unwanted object i a fixed human hair [14]. Since the outlier i fixed throughout the video, it i poible to roughly remove outlier pixel by applying a tatic binary weighting mak w(a, b) = 0 if pixel (a, b) i in the maked region, w(a, b) = 1 otherwie. Figure 4 how two frame from the video and the manual removal of the outlier by making. Due to the mak w, part of the ignal due to the particle of interet (diffraction ring) i not accounted for. Since the radiu r p of the particle remain contant during acquiition, it i poible to jointly etimate thi parameter in everal conecutive frame. Thi uper-reolution approach ha already been uccefully applied in lenle microcopy conidering the non-robut minimization problem (2) [14]. We propoe to combine thi uper-reolution approach with a robut etimation cheme (minimization problem (4)) conidering 40 tack of 5 ucceive frame. In each frame the vector of parameter i etimated by leat quare fitting, Eq. (2), with contant weight (hereafter denoted ingle without mak) or with the binary mak (denoted ingle with mak), and uing the robut M- etimator, Eq. (4) and ŝ (MAD) (denoted ingle robut). In the cae of a joint etimation, the term ingle i replaced by the term joint in the denomination of the method teted. Figure 4 i an example of the weight ued for the lat IRLS iteration. The weight are cloe to a egmentation of the pixel impacted by the unwanted object. The diffraction ring characterizing the particle of interet are well preerved (weight cloe to 1). Figure 5 how the evolution of the etimated radiu during the IRLS iteration for the firt 5 frame of the video. Thi how that the algorithm converge in a few iteration and that the joint etimate of the radiu carried out by uper-reolution i very cloe to the ground truth equal to 50 µm. Table I lit the mean etimated radiu ˆr p (M), the correponding tandard-deviation σ and the CRLB δ obtained with ŝ (MAD) for each method teted. The manual removal of the outlier appear to reduce the etimation bia. The bia i further reduced when a robut weighting of reidual by IRLS i applied ince it preferentially eliminate the outlier while retaining the ignal of interet. A a reult, the etimation uing a robut approach lead to a very mall increae in CLRB Fig. 5. Etimated radiu a a function of the number of IRLS iteration. TABLE I MEAN ESTIMATED RADIUS, STANDARD-DEVIATION σ AND CRLB δ FOR THE DIFFERENT METHODS TESTED Method: ˆr (M) p (µm) σ (µm) δ (µm) ingle without mak ingle with mak ingle robut joint without mak / 5 joint with mak / 5 joint robut / 5 which i negligible compared with the reduction of the bia. The combination of a robut etimation method with a uper-reolution cheme (joint robut) exploiting the information redundancy between different frame ignificantly reduce the etimation tandard-deviation. In agreement with tatitical reult [15], we oberved a decreae in the tandarddeviation by a factor at leat equal to 5. Thi joint robut etimation approach enable accurate (i.e. low bia and low tandard-deviation) etimate of the radiu of the holographic object and thu improve the overall performance of a conventional leat quare etimation proce. B. Holographic video of evaporating droplet We ued a 160 frame video of an evaporating diethyl ether droplet dipered in a homogeneou and nearly iotropic turbulence that i tracked over time. A the particle evaporate, it produce a plume of vapor that diffue around it. The vapor plume create outlier ince it ignal mixe with the ignal produced by the particle of interet. Other evaporating particle paing cloe to the particle of interet alo act a outlier. The evaporation rate of thee particle wa recently tudied uing the leat quare approach (2) [16]. Figure 6(b) how two frame from the video and the 2D trajectory of the particle over time (uperimpoed color line). Figure 6(a) how the evolution of the recale factor of the reidual obtained with a MAD etimator and with the propoed ICI rule a well a the IRLS weight for two different frame. The ICI rule lead to a larger value of than MAD but without notably modifying the etimation of the radiu r p. Thi i the cae when outlier do not diturb the etimate like in frame 60 (left column) ince the particle of interet i iolated and the plume of vapor i rather mall. In contrat, ŝ (MAD) and ŝ (ICI) are very cloe when the particle of interet i trongly diturbed by outlier. Thi i illutrated by frame 97 (right ISBN EURASIP

5 Fig. 7. Etimated radiu over time with the different method teted V. C ONCLUSION We have hown that the ue of a robut approach to etimate the 3D poition and ize of object in lenle microcopy video reduce the etimation bia caued by unwanted object. We propoed a imple method to etimate the cale parameter that ditinguihe between inlier and outlier in the robut etimation cheme. Thi method i baed on an interval confidence interection rule and achieve a better bia / variance trade-off than the uual MAD rule. Finally, we have hown that the combination of the propoed robut approach with a joint etimation cheme alo reduce the tandard-deviation of the etimation and conequently improve the overall performance of the method. Work upported by CNRS grant DETECTION (DEFI IMAG In 2015) Fig. 6. Robut detection and tracking of an evaporating droplet. (a) evolution of (MAD) and (ICI). (b), (c) and (d) repectively give the hologram, confidence interval diagram, and robut weighting map for two video frame. (holographic video credit: Fluid Mechanic and Acoutic Laboratory LMFA, CNRS Univerite de Lyon, France) column) where another evaporating particle i cloe to the particle of interet and diturb it ignal. Thee obervation underline the fact that the MAD etimator only account for the preence of outlier trongly deviating from the model but doe not account for their patial localization and hence, their actual impact on the etimate. Figure 6(c) illutrate the evolution of confidence interval for increaing value of. Figure 6(d) how the weight obtained at the end of the IRLS procedure for (MAD) (firt row) and (ICI) (econd row). On frame 60, (ICI) i ignificantly larger than (MAD), leading to larger weight, while on frame 97 (MAD) = (ICI) and the weight are identical. Figure 7 how the evolution of the etimated radiu over time with the three method of etimation teted. The robut etimation cheme lead to an average increae of 0.4 µm in the etimated radiu. We alo oberved that it reduce the etimation tandard-deviation. Thee experimental reult are explained by the fact that the moving outlier (vapor plume plu other particle) caue a non-tationary diturbance of the ignal of interet over time. The robut approach conidering a MAD or ICI rule to et the parameter lead to cloe reult in thi application. Thi i due to the fact that the weighting matrix obtained with (MAD) already guarantee a high level of efficiency (around 90% of the maximum efficiency obtained without reweighting). ISBN EURASIP 2017 R EFERENCES [1] A. Greenbaum et al., Imaging without lene: achievement and remaining challenge of wide-field on-chip microcopy, Nature method, vol. 9, no. 9, [2] T. Krei, Handbook of holographic interferometry: optical and digital method. John Wiley & Son, [3] F. Soulez et al., Invere-problem approach for particle digital holography: accurate location baed on local optimization, JOSA A, vol. 24, no. 4, [4] C. Wang et al., Holographic characterization of protein aggregate, J. of pharmaceutical cience, vol. 105, no. 3, [5] D. Gabor, A new microcope principle, Nature, no. 1161, [6] G. A. Tyler and B. J. Thompon, Fraunhofer holography applied to particle ize analyi a reaement, J. of Modern Optic, vol. 23, no. 9, [7] P. J. Huber, Robut tatitic. Springer, [8] P. W. Holland and R. E. Welch, Robut regreion uing iteratively reweighted leat-quare, Communication in Statitic-theory and Method, vol. 6, no. 9, [9] J. E. Denni Jr and R. E. Welch, Technique for nonlinear leat quare and robut regreion, Communication in Statitic-Simulation and Computation, vol. 7, no. 4, [10] P. J. Roueeuw and A. M. Leroy, Robut regreion and outlier detection. John Wiley & on, 2005, vol [11] V. J. Yohai, High breakdown-point and high efficiency robut etimate for regreion, The Annal of Statitic, [12] V. Katkovnik, K. Egiazarian, and J. Atola, Adaptive window ize image de-noiing baed on interection of confidence interval (ici) rule, J. of Mathematical Imaging and Viion, vol. 16, no. 3, [13] A. Stuart and J. Odd, Kendall Advanced Theory of Statitic : Ditribution Theory. London, Arnold, [14] N. Verrier and C. Fournier, Digital holography uper-reolution for accurate three-dimenional recontruction of particle hologram, Optic letter, vol. 40, no. 2, [15] D. Robinon and P. Milanfar, Statitical performance analyi of uperreolution, IEEE Tran. on Image Proceing, vol. 15, no. 6, [16] J. L. Marie et al., Digital holographic meaurement of the lagrangian evaporation rate of droplet dipering in a homogeneou iotropic turbulence, Experiment in Fluid, vol. 58, no. 2,