Why use video imaging? Estimation and validation for imaging-based measurement of particle size distribution

Transcription

1 Why use video imaging? Estimation and validation for imaging-ased measurement of particle size distriution Paul A. Larsen and James B. Rawlings High value-added products in the chemical industry are ecoming increasingly complicated in structure. Pharmaceutical compounds are complex: multiple crystal haits and multiple crystal structures. Department of Chemical and Biological Engineering University of Wisconsin Madison ACT 1 Octoer 26 Needles α-glycine γ-glycine Larsen, Rawlings (Wisconsin) PSD measurement validation 1 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 2 / 5 Why in-situ imaging is used only qualitatively 1 Ojective and constraints Segmentation: Separating ojects of interest from the ackground Ojective Demonstrate roust and efficient image segmentation for in-situ images of highly non-spherical particles. Constraints Acquire images without sampling. Illuminate using reflected light. Keep vessel well-mixed. Image Analysis System Imaging Window Video Camera Stroe Light TT Controller 1 1 Hot Stream TT Cold Stream 1 Braatz, R.D., Annual Reviews in Control, 22. Larsen, Rawlings (Wisconsin) PSD measurement validation 3 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 4 / 5

2 Challenges for automatic image segmentation Model-ased oject recognition for shape measurement Challenges for thresholding and edge detection-ased methods: 2 Non-uniform color/intensity. Poorly-defined outline. Challenges for template-matching: Non-uniform size and shape. Random orientation in 3-D space. Other challenges: Motion lur, out-of-focus lur. Agglomeration, attrition, reakage. 2 Calderon De Anda, Wang and Roerts, ChE Sci, 25 Larsen, Rawlings (Wisconsin) PSD measurement validation 5 / 5 Advantages 1 Parallel, distriuted algorithms. 2 Roust to noise or missing data. 3 Generalizale to many shapes. Basic approach 3 1 Find linear features in the image. 2 Find groups of linear features that appear significant on the asis of viewpoint-independent relationships (e.g. parallelism, proximity of endpoints). 3 Fit a 2D or 3D model to each group of linear features. 3 Lowe, D.G., Artificial Intelligence, Larsen, Rawlings (Wisconsin) PSD measurement validation 6 / 5 SHARC: 2-D model-ased image analysis for needles M-SHARC: 3-D models for more complex shapes Parameterized, wireframe model. Viewpoint-invariant groups used as cues for location and size of crystals in image. y Original image Linear feature detection Collinearity identification t h x Junction Parallel pair z t w Parallelism identification Cluster properties w Symmetric pair Arrow Larsen, Rawlings (Wisconsin) PSD measurement validation 7 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 8 / 5

3 M-SHARC example: α-glycine crystal M-SHARC validation (a) Original image () Linear features (c) Salient line group Experimental Unseeded, cooling crystallization of α-glycine. Acquire 3 sets of video images (3 frames/second). Analyze images from each set using M-SHARC. Evaluate performance visually. Low solids Med. solids High solids (d) Model initialization (e) Further matches (f) Optimized Fit Larsen, Rawlings (Wisconsin) PSD measurement validation 9 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 1 / 5 Low solids conc. (13 min. after appearance of crystals) Low solids conc. (13 min. after appearance of crystals) Larsen, Rawlings (Wisconsin) PSD measurement validation 11 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 12 / 5

4 Medium solids conc. (24 min. after appearance of crystals) Medium solids conc. (24 min. after appearance of crystals) Larsen, Rawlings (Wisconsin) PSD measurement validation 13 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 14 / 5 High solids conc. (43 min. after appearance of crystals) High solids conc. (43 min. after appearance of crystals) Larsen, Rawlings (Wisconsin) PSD measurement validation 15 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 16 / 5

5 M-SHARC validation Human measurements using LaelMe Otaining gold standard measurements Analyze 1 images from each set using M-SHARC. Analyze same images manually using human operator (with MIT s we-ased image annotation tool, LaelMe). Larsen, Rawlings (Wisconsin) PSD measurement validation 17 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 18 / 5 M-SHARC versus human measurement Human results using LaelME M-SHARC versus human measurement Automatic results using M-SHARC Larsen, Rawlings (Wisconsin) PSD measurement validation 19 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 2 / 5

6 M-SHARC versus human measurement M-SHARC versus human measurement Hit: red/lue; Miss: lack; False positive: white Low Med. High Solids Solids Solids Hits (N H ) Misses (N M ) False Positives (N FP ) Hit numer fraction (N H /(N H + N M )) False Pos. numer fraction (N FP /(N H + N FP )) Hit area fraction (A H /(A H + A M )) False Pos. area fraction (A FP /(A H + A FP )) Larsen, Rawlings (Wisconsin) PSD measurement validation 21 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 22 / 5 Oservations and questions Population sampling y imaging We have assessed the accuracy of our particle size and shape measurements only with respect to human measurements. How do we overcome the iases inherent in imaging-ased measurement to estimate the true state of the particle population? The measurement quality depends on the properties of the particle population. How do we characterize the time-varying reliaility of the measurement? a d Larsen, Rawlings (Wisconsin) PSD measurement validation 23 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 24 / 5

7 PSD estimation from a finite field of view Sampling ias y edge effects Clean tile and the Buffon-Laplace needle prolem a a a Single Intersection clean ρ = 63 particles/9 units = 7 Non-order particles: ˆρ = 4 Midpoints inside image: ˆρ = 8 Border and non-order: ˆρ = 1 Doule Intersection Solomon,H., Geometric Proaility, SIAM 1978 Larsen, Rawlings (Wisconsin) PSD measurement validation 25 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 26 / 5 Sampling ias y edge effects Result of ignoring edge effects As particle length increases, the proaility of a clean landing decreases. 1 PSD measured y counting non-order particles: uniform distriution proaility of clean landing a a2 + 2 relative PSD needle length particle length Larsen, Rawlings (Wisconsin) PSD measurement validation 27 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 28 / 5

8 Sampling ias y other effects Correcting edge effects: the stochastic geometry approach 4 Occlusion effects Miles-Lantuejoul estimation a Approach: Delete order particles. M j d F,v Bin particle j ased on its length L j weighted with 1/M j. a d F,h d F,h Orientation effects Projected particle lengths are less than true lengths unless particles are oriented in the plane perpendicular to camera s optical axis. Normalize. Gives asymptotically uniased estimate of relative PSD. d F,v L j 4 Miles, R.E., in Stochastic Geometry, Wiley 1974 Larsen, Rawlings (Wisconsin) PSD measurement validation 29 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 3 / 5 Maximum likelihood estimation of asolute PSD Maximum likelihood estimation (cont.) Definitions: X = (X 1,..., X T ): random vector in which X i gives the numer of non-order particles of size class i oserved in a single image. Y = (Y 1,..., Y T ): random vector in which Y i gives the numer of order particles with oserved lengths in size class i that are oserved in a single image. p XY : joint proaility density for X and Y. x and y: realizations of the random vectors X and Y. q = (q 1,..., q T ): relative PSD in which q i is the fraction of particle population in size class i. ρ = (ρ 1,..., ρ T ): asolute PSD in which ρ i is the numer of particles of size class i per unit volume of crystallizer. Maximum likelihood estimator of ρ: ˆρ = arg max p XY (x 1, y 1, x 2, y 2,..., x T, y T ρ) ρ Assuming X 1, Y 1,... X T, Y T independent (occlusion effects negligile), the joint density is given y p XY = T p Xi (x i ρ)p Yi (y i ρ) i=1 The maximum likelihood estimate is therefore given y ˆρ = arg min ρ T log p Xi (x i ρ) log p Yi (y i ρ) i=1 Larsen, Rawlings (Wisconsin) PSD measurement validation 31 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 32 / 5

9 Test case 1: monodisperse particles of length.5a Comparison etween estimators for asolute PSD asolute PSD 1 4 True value Estimated value w/ orders Size class -2e-5 2e-5 error, ρ i ˆρ i True vs Estimated PSD Error distriution 1 simulations 1, simulations Larsen, Rawlings (Wisconsin) PSD measurement validation 33 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 34 / 5 Test case 2: uniform distriution on [.1a.9a] Estimated vs true relative PSD.25.2 True value Estimated value relative PSD Size class Larsen, Rawlings (Wisconsin) PSD measurement validation 35 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 36 / 5

10 Comparison etween estimators for relative PSD Comparison etween estimators for relative PSD Distriution of errors for 1, simulations, 1 images/simulation: Distriution of errors for 1, simulations, 5 images/simulation: w/ orders Miles w/ orders Miles w/ orders Miles w/ orders Miles error, q i ˆq i error, q i ˆq i error, q i ˆq i error, q i ˆq i Smallest size class (.1a) Largest size class (.9a) Smallest size class (.1a) Largest size class (.9a) Larsen, Rawlings (Wisconsin) PSD measurement validation 37 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 38 / 5 Test case 3: uniform distriution on [.1a 2.a] Comparison etween estimators for asolute PSD Distriution of errors for 1, simulations, 2 images/simulation: w/ orders -2e-5-1e w/ orders -2e-5-1e-5 error, ρ i ˆρ i error, ρ i ˆρ i Smallest size class (.1a) Largest size class (1.a) Larsen, Rawlings (Wisconsin) PSD measurement validation 39 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 4 / 5

11 Estimated vs true asolute PSD Characterizing measurement reliaility Measurement reliaility depends on a variety of factors: 2e-5 True value Mean estimated value Imaging conditions: Camera resolution, R Process conditions: Solids concentration (w/v), S w asolute PSD a 2a Field of view, a, Depth of field, d Particle length, L Particle width, w These factors can e lumped into a single factor denoting the numer of crystals appearing in the image: Size class N c = S w ad ρ c w 2 L Larsen, Rawlings (Wisconsin) PSD measurement validation 41 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 42 / 5 Calculating the proaility of overlap Comparison of images at constant solids concentration and constant D The proaility that two crystals placed randomly in the image will e overlapping: p ovp = 2 ( L 2 + w 2 + Lw(2 + π) ) πa As an indicator of the degree of difficulty of an image, we define the parameter D as A ovp a w L D = (N c 1)p ovp Constant solids Constant D Larsen, Rawlings (Wisconsin) PSD measurement validation 43 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 44 / 5

12 Case study: image analysis at various D. Examples of synthetic images at various D Larsen, Rawlings (Wisconsin) PSD measurement validation 45 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 46 / 5 Results for image analysis with perfect algorithm Results for image analysis with SHARC D=. D=.1 D=.3 D=.5 D=.9 D=1.4 D=2.3 D= Length relative PSD D=. D=.1 D=.3 D=.5 D=.9 D=1.4 D=2.3 D= Length D=. D=.1 D=.3 D=.5 D=.9 D=1.4 D=2.3 D= Length relative PSD D=. D=.1 D=.3 D=.5 D=.9 D=1.4 D=2.3 D= Length Non-corrected histogram Corrected histogram Non-corrected histogram Corrected histogram Larsen, Rawlings (Wisconsin) PSD measurement validation 47 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 48 / 5

13 Conclusions Acknowledgment Model-ased vision is an effective framework for automating the measurement of crystal size and shape distriutions in noisy, in situ images. The algorithms are fast and likely suitale for real-time measurement of crystal size and shape distriutions. Maximum likelihood estimation is advantageous over alternative methods for estimating the asolute PSD ut offers little advantage for estimating the relative PSD. Measurement reliaility can e characterized in terms of a single parameter lumping process and imaging conditions. National Science Foundation, Grant No Professor Nicola Ferrier, Mech. Eng. Dept. Image analysis consulting. Professor Lian Yu, School of Pharmacy Polymorphism expertise. XRPD and Raman analysis for initial glycine studies. Dr. Philip C. Dell Orco, GlaxoSmithKline Imaging equipment. Larsen, Rawlings (Wisconsin) PSD measurement validation 49 / 5 Larsen, Rawlings (Wisconsin) PSD measurement validation 5 / 5