ESTIMATION OF VISUAL QUALITY OF INJECTION-MOLDED POLYMER PANELS

Transcription

1 ESTIMATION OF VISUAL QUALITY OF INJECTION-MOLDED POLYMER PANELS J. Jay Liu an John F. MacGregor McMaster Avance Control Consortium, Department of Chemical Engineering, McMaster University. Hamilton, ON L8S 4L7 Canaa Abstract A new machine vision approach for estimating manufacture prouct appearance is illustrate. This new approach consists of: (1) extraction of textural information from prouct images, (2) estimation of measures of the visual quality of the prouct from the textural information extracte. This metho is specifically aime at treating the stochastic nature in the visual appearance of many manufacture proucts. This non-eterministic nature of prouct appearance has been a main obstacle for the success of machine vision in the process inustries. This approach is successfully applie to an inustrial process for estimation of the visual appearance of injection-mole plastic panels. Keywors: Injection moling, Machine vision, Principal Component Analysis, Visual appearance, Wavelet texture analysis 1 Introuction The efinition of machine vision or computer vision can be given as interpretation of an image of an object or scene through the use of optical non-contact sensing mechanisms for the purpose of obtaining information an/or controlling machines or processes [1]. The main purpose of machine vision is to allow a computer to unerstan aspects of its environment using information provie by visual sensors [1] an thus it requires a combination of low-level image processing to enhance the image quality an higher level pattern recognition an image analysis to recognize features present in the image [2]. Machine vision has been stuie for about 40 years but the stuy in last 15 years has shown rapi progress ue to the great avances in imaging an computing technologies. The subject of machine vision now embraces innumerable topics an applications: automatic assembly an inspection, automatic vehicle guiance, automatic ocument interpretation, verification of signatures, checking of fingerprints an recognizing faces, an analysis of remotely sense images, to list just a few [1,3]. Automatic inspection an assembly is one of the areas where machine vision has been most successfully applie an it is still showing substantial growth. The necessity of improvements in quality, safety, an cost saving is the riving force for this growth. However, most successful techniques an their applications in this area have been confine to a specific type of environment where certain assumptions can be mae about the scene [1]. In typical manufacturing inustries such as microelectronics fabrication, for example, an image provies a scene of objects with pre-etermine shape, structure, orientation, an so on, unless the position of a camera changes. In other wors, images from such inustries are essentially eterministic. The primary goal of the inspection in such manufacturing inustries is to check whether there are missing objects in pre-specifie regions in the image or whether objects in the image are in esire orientations or of esire size, an the necessary analysis is mainly one irectly on the image itself; i.e., in image space.

2 On the other han, in the process inustries there is another class of problems where the major concern is some ill-efine visual appearance of proucts or processes such as the aesthetics of manufacture countertops, the health of mineral flotation froth [4], or visible patterns on injection-mole polymer panels. In this case, simple assumptions about the scene cannot be mae any longer ue to the ominantly stochastic nature of the visual scene. Therefore, machine vision has selom been applie to those processes an has ha little success when it has. In woo inspection for example, sample-to-sample variation in the grains is too large because natural materials have almost arbitrary shape an orientation. Therefore the state or the quality of these processes or proucts is almost always juge by traine human operators an any control ecisions are left to their iscretion at the present time. Inconsistency in human jugment still remains a critical issue in the process inustries for this reason. The contribution of this paper is to propose a new machine vision approach for the assessment of the visual quality of manufacture proucts. In this approach, machine vision will inclue new application areas an new tasks that have selom been trie in contemporary machine vision research. New application areas inclue all process inustries where stochastic visual appearance of proucts or processes is the major concern. New tasks inclue estimation, moeling, an optimization of visual quality of the process or the prouct. Visual quality stuie in this paper means textural appearance of processes an proucts. However, it can also inclue spectral (i.e., color) appearance of proucts. The rest of this paper is organize as follows: In Section 2, we propose a new machine vision approach by presenting the relate theories such as wavelet texture analysis, estimation of visual quality from textural information, an causal moeling of visual quality. This new approach is then illustrate via an application to the visual appearance of injection-mole plastic panels in Section 3. Summary an conclusion are given in Section 4. 2 A New Machine Vision Approach 2.1 Extracting Textural Information Using The Two-Dimensional Wavelet Transform The appearance or aesthetics of a prouct usually epen on a combination of color an textural properties of its surface. In the injection-moling application in Section 3 of this paper, the images are grayscale images an appearance is relate only to textural properties. Therefore, we will focus on the extraction of textural properties via two-imensional wavelet transforms. However, there are many works on the extraction of spectral information from color images [5,6] an some recent work on combining both spectral an textural analysis [4,7] Wavelet Transform In wavelet analysis, a continuous signal f(x) is ecompose in terms of a family of orthonormal bases ψ m,n (x) obtaine through translation an ilation of a mother wavelet ψ(x), i.e., m 2 m ψ ( x) = 2 ψ (2 x ) (1) m, n n where m, n are integers. Due to the orthonormal property, the wavelet coefficients then can be efine as the convolution of the signal with these wavelet bases:

3 c = f x) ψ ( x) x = ψ ( x), f ( ). (2) R m, n ( m, n m, n x The mother wavelet ψ(x) is relate to the scaling function φ(x) with some suitable sequence h[k] [8-10] ; ψ ( x ) = 2 h1[ k] φ(2x k), (3) k where φ( x ) = 2 h0 [ k] φ(2x k) an h 1 [k] = ( 1) k h 0 [1 k]. Using the following relations, the k iscrete wavelet transform (DWT) at ecomposition level j can be performe without requiring the explicit forms of ψ(x) an φ(x); j / 2 j φ j, l[ k] = 2 h0[ k 2 l], (4) j / 2 j ψ k] = 2 h [ k 2 ]. (5) j, l[ 1 l The DWT coefficients of a signal f(x) are now compute as a l] = f [ k], [ ] an [ l] = f [ k], [ ], (6) [ ( j) φ j, l k ( j) ψ j, l k where the a (j) [l] s are expansion coefficients of the scaling function or approximation coefficients an the (j) [l] s are the wavelet coefficients or etail coefficients. In achieving a two-imensional (2-D) iscrete wavelet transform (DWT), there are two ifferent solutions epening on the type of filters an the type of own-sampling lattices [10]. A separable solution (Figure 1a) gives rectangular ivisions of frequency spectrum (Figure 1b) an strongly oriente coefficients (often calle subimages because the wavelet coefficients for 2-D signals are also 2-D) in the horizontal an vertical irections (see Figure 1a). Visible wave patterns in the plastic panels stuie in this paper have istinct (or a combination of) orientations an thus a 2-D DWT implemente by the separable solution is suitable for extracting the visible patterns (see images in Section 3) Wavelet Texture Analysis Among other texture analysis methos, a 2-D DWT-base metho, which is often calle wavelet texture analysis (WTA) seems best not only because it has shown better performance than other methos in many cases [11,12] but also there is strong psychophysical evience that the human visual system oes multi-channel, space-frequency analysis [13]. WTA has been successfully applie to characterization of steel surface an flotation froth monitoring in previous stuies [4,11]. A basic assumption for WTA is that a texture has its unique istribution (i.e., energy or entropy istribution) in the 3-D scale space consisting of the two spatial axes an an aitional scale axis. Therefore, if the scale axis of the scale space of a texture image is iscretize appropriately, ifferent textures will have ifferent features at the iscretize scales. When a k wavelet subimage (i.e., a (J) an ( j ), j {1, 2,, J} an k {h, v, } for a separable 2-D DWT, Figure 1a) is treate as a matrix, then the power or energy of the subimage can be efine

4 using the Frobenius norm, as F k. For example, the energy of the etail subimage ( j ) is given k ( j) 2 E kj =. (7) ( ) F Often this is ivie by the number of pixels, yieling average power or normalize energy. A feature vector compose of energies of all subimages is often calle wavelet energy signature, one of the most popular wavelet textural features. Other popular features inclue entropy an average l 1 -norm of subimages. Since the normalize energy of each subimage is equal to variance of a corresponing channel (after mean-centering for the approximation subimage) the wavelet energy signature also represents contrast information of subimages. Entropy signatures are equivalent to high-orer moments of subimages. ver H a ( j) f vertical hor H π a( j 1) ver H h ( j ) horizontal π/2 h 1 1 hor H ver H v ( j ) vertical a 1 v 1 ver H ( j ) iagonal π/2 π f horizontal (a) Figure 1. A separable solution for 2-D DWT. (a) A separable two-imensional filter bank at the j-th ecomposition stage. It consists of horizontal an vertical filtering of 2-D signals using low-pass an high-pass 1-D wavelet filters H0 an H1, an separable horizontal (2 1) an vertical (1 2) own-sampling by 2. (b) Division of spatial frequency spectrum by a separable solution for one-level ecomposition. The iea of WTA base on the 2-D DWT can be extene to 2-D wavelet packets (WP) with an arbitrary tree structure [14,15]. When an image is ecompose own to the J-th level, the size of a feature vector for an image (when incluing an approximation subimage) is 3J+1 an 4 J for the 2-D DWT an the 2-D full-tree WP, respectively. The approximation subimage is sometimes exclue because the variations inuce by lighting or illumination are usually capture in the approximation subimage. 2.2 Estimation of Visual Quality in Latent Variable Spaces Although a single image can provie an enormous amount of information about the scene epicte, the human visual system can selectively extract the information that is relevant only to specific tasks. Furthermore, the human brain can reuce the imension of the extracte information an analyze it. For example, operators evaluate the visual quality of steel surfaces as goo, meium, an ba, or evaluate the aesthetic quality of manufacture (b)

5 countertops as on-specification an off-specification by looking at an image or a scene that can be easily of several megabytes an hunres of thousans of pixels. After extracting wavelet textural features (usually much less than 100 features per image) from images, further imensional reuction can be one using a metho such as Principal Component Analysis (PCA) [16-18]. Fisher s Discriminant Analysis (FDA) [19] can be use when class labels are available, an Inepenent Component Analysis (ICA) [20] an Projection Pursuit (PP) [21] can also be use on the PCA scores (calle pre-whitening by PCA). All these linear projection methos fin an operator (a matrix) that can map high-imensional feature space to a low-imensional (usually 2 ~ 4 imension) latent variable subspace an they are perfect caniates in estimating visual quality. Let f be a (K 1) feature vector after 2- D DWT of an image an followe by a nonlinear transform (e.g., ) an let t be a (A 1) latent vector after imensionality reuction. Then the following equality hols via a (A K) mapping matrix W; t = Wf (8) The matrix W is calle a loaing matrix in PCA an a separating or unmixing matrix in ICA. In any linear projection metho, the rows of the mapping matrix represent contributions of features f to each of the latent variables in t because each latent variable is simply a linear combination of features with elements in each row of the mapping matrix as coefficients. Therefore if features have certain psychophysical meanings then we can also give a psychophysical meaning to each latent variable. This is crucial when we numerically estimate visual quality an this is the one reason why we choose projection methos. Another reason is that the visual quality of proucts an/or processes of interest is not a iscrete or isjoint quality as in typical classification tasks, rather it is a continuous quality [4, 11] ; For example, the quality of a steel surface may graually eteriorate from goo to meium an from meium to ba, an the state of a mineral flotation process may graually change epening upon the amount of chemical reagents ae, the mineral content of the ore, an the state of previous flotation cells. Therefore using projection methos an working with continuous latent variables are more suitable in this circumstance than using classification methos an working with iscrete class labels. 3 Application to Injection-mole Plastic Panels 3.1 Description of Data The image ata set use consists of 50 grayscale images of injection-mole polymer panels. This ata set was obtaine via a esigne experiment with three operating variables. The variables are polymer formulation, injection spee, an plaque position an the number of levels were five, two, an five respectively. Four samples of the images are shown in Figure 2. Depening on operation conitions, plastic panels can contain visible patterns such as stripes, swirls, an ripples with varying characteristics in their shapes, irections, an intensities as shown in the figure. Plastic panels with the esire visual quality have no such visible patterns, i.e., they are visually flat as I074 in the figure. The ultimate goal of this stuy is to fin operating conitions that can prouce plastic panels with the visual quality specifie by users. The spatial frequency components of the four images in Figure 2 are very ifferent from each F

6 other. I074 has almost zero frequency (DC) components while I243 has very high frequency components (i.e., fine gray ots). I183 an I283 have istinct visible patterns with very low frequency components (e.g., stripes an ripples) as well as some visible patterns with high frequency components. The main ifficulty arises from the fact that there is no istinct class of patterns since ifferent visible patterns can merge together to form more complicate patterns. For example, the four ripples in I183 in Figure 2 are not ientical to each other; from top to bottom, the ripple becomes w-shape (the 2 n the an 3 r ripples) an loses its istinct shape at the bottom. Five arcs in I283 in Figure 2 also show varying characteristics of patterns. There can be an infinite number of patterns epening on the number an the types of basic patterns, their varying characteristics, an how they are merge. In general, there is a wie range of stochastic pattern ifferences in the images. I074 I183 I243 I283 Figure 2. Four samples of pre-processe original images before converte to a grayscale complement. A plastic panel with esire visual quality (I074). Three images with unwante visual quality (I83, I243, an I283). 3.2 Estimation of Visual Quality An image was croppe from each of the original 50 images, ownsample by 3 to prouce , an finally converte to obtain the complement of the grayscale image. After these pre-processing steps, a 4-level 2-D DWT was applie to each image using orer-2 Daubechies filters. These choices were mae by trial an error but one can follow some guielines [15, 23]. Level-one etail subimages of all 50 images showe nothing but irrelevant noise an thus they were exclue from the extracte wavelet feature vectors. All other etail subimages capture horizontal, vertical an iagonal features of the original images. Level 3 an 4 wavelet subimages of I128 are shown in Figure 3 (Figure 3 will help unerstaning the PCA score space later). The square root of the normalize energy was calculate for each subimage an use as textural features. Therefore each image is h v represente by a (10 1) feature vector (energies corresponing to a 4 an i, i, i (i = 2, 3, 4)).

7 (a) h 4 Figure 3. (a) image I128 an (b) some of its wavelet subimages. v 4 4 (b) h 3 v 3 3 Table 1. Summary of Principal Component Analysis results Dimension of Latent Space R 2 Q After auto-scaling, PCA was applie to the (50 10) X matrix where each row consiste of a feature vector for one image. Four statistically significant principal components were foun base on Bootstrap risk estimate [24]. R 2 (the fraction of variance of feature vectors explaine by the PCA moel) an Q 2 (the fraction of variance preicte for images not use in the PCA moel) are summarize in Table 1 an a resiual plot (istance to the moel, DmoX [25] ) is given in Figure 4. Clearly, the variability of images in wavelet feature space is well moele by this PCA moel. In orer to see whether the four score values of the feature vectors of the images reveal similarity an/or issimilarity between the visual qualities of the images, a simple nearest-neighbor clustering base on the Mahalanobis istance [26] of score values is applie to all 50 images to fin the nearest neighbors of the images in Figure 2. The two nearest neighbors of each of the four images an the corresponing Mahalanobis istances are shown in Figure 5. To give a reference regaring the similarity/issimilarity between visual quality an the corresponing Mahalanobis istance, the istances between the four images in Figure 2 are liste in Table 2. It is clear from the Figures 2 an 5, an Table 2 that the nearest neighbor images in Figure 5 an corresponing images in Figure 2 are very close in terms of visual quality an in terms of the Mahalanobis istance. As expecte from the Figure 2, I283 is the farthest neighbor of I074 (the Mahalanobis istance is 3.614) an the istances between I074 an I243 an between I183 an I283 are an 1.831, respectively. By comparing the magnitue of the Mahalanobis istances in Table 2 with those in Figure 5 an comparing the visual similarity/issimilarity between images in Figures 2 an 5, the principal components o reveal similarity an/or issimilarity between the visual qualities of images.

8 2.5 2 I44 I144 Control limit(0.05)= SPE Image Number Figure 4. A resiual plot for the PCA moel Table 2. Mahalanobis istances between the four images in Figure 2 I074 I183 I243 I283 I I I I Loaing plots an score plots from the PCA moel are shown in Figures 6, 7 an 8. Since the textural signature use in the analysis is equivalent to the stanar eviation of wavelet subimages, it represents the presence of visible patterns as well as multiresolution contrast information of the pre-processe original images. The p 1 loaing plot in Figure 6 shows that images in the positive t 1 irection have more contrast in all subimages. In other wors, visible patterns in images, whether they are is stripes, swirls, ripples, or fine gray ots, become more an more noticeable when moving towar the positive t 1 irection. To verify this, three images I263, I223, I054 that are at each en an close to the center of the t 1 axis in the t 1 -t 2 score plot of Figure 7 are shown in Figure 9. As expecte, I263 at the negative en of t 1 axis in the score plot has almost no noticeable visible patterns while I223 at the opposite en of t 1 axis has visible patterns that are strongly noticeable. The visual quality of I054 lies between that of I263 an I223 in both the image space an the t 1 score value. On the other han, the p 2 loaing plot in Figure 6 tells that images in the positive t 2 irection have highlystructure visible patterns such as stripes, swirls, an ripples because they have low frequency signals in horizontal, vertical an/or iagonal irections as represente by the big positive p 2 h v loaings on 4, 4, an 4. Figure 3(b) illustrates how the wavelet subimages capture these low frequency wave patterns. Images in the negative t 2 irection have less structure visible patterns such as fine gray ots. Images I267 (negative t 2 ) an I133 (positive t 2 ) in Figure 9

9 show this behavior. Again the visual quality of I054 is in the mile of I267 an I133 in both image space an the t2 score value (a) (b) (c) () Figure 5 Two nearest neighbors of the four images shown in Figure 2 an corresponing Mahalanobis istances to the images. (a) I263 an I268 for I074 (b) I198 an I188 for I183 (c) I267 an I278 for I243 () I293 an I288 for I283 The Psychophysical meaning of the latent variable t3 can be interprete in a similar way. It is clear from the p3 plot that images in the positive t3 irection will have strongly noticeable vertical patterns (see big positive weights of 4v an 3v in Figure 6) while images in the negative t3 irection will have strongly noticeable horizontal patterns (see negative weights of 4h an 3h in Figure 6). Two images, I123 (negative t3 in Figure 8) an I228 (positive t3 in Figure 8), are shown in Figure 10 to verify this behavior. The psychophysical meaning of t4 is less clear because the score vectors are force to be orthogonal to previous scores an the most obvious visual characteristics have been explaine in t1~t3. But from the p4 plot, images in

10 the positive t 4 irection shoul just have arker low frequency backgroun as shown in Figure 6 by high p 4 loaing on the approximation a 4. The two images I089 (a negative t 4 score) an I059 (a positive t 4 score) in Figure 10 verify this behavior. 1 p p 2 p p a 4 4 h 4 v 4 3 h 3 v 3 2 h 2 v 2 Figure 6 Loaing plots for the 4 principal components 2 I I138 I128 I123 1 I118 t 2 t I263 I074 I268 I113 I064 I039 I079 I203 I044I208 I213 I218 I163 I173 I253 I178 I258 I248 I278 I267 I168 I054 I089 I243 I273 I109 I049 I154 I144 I148 I099 I158 I193 I069 I198 I188 I059 I238 I084 I183 I228 I233 I283 I298 I288 I293 I t 1 t 1 Figure 7 A t 1 -t 2 score plot

11 0.6 I059 t 4 t I123 I158 I193 I183 I198 I054 I233 I188 I263 I074 I163 I268 I144 I113 I248 I283 I I I148 I178 I208 I213 I203 I099 I253 I128I044 I258 I218 I267 I243 I I298 I069 I173 I168 I223 I238 I228 I154 I I133 I273 I118 I I084 I049 I t 3 t 3 Figure 8 A t 3 -t 4 score plot I263 I223 I267 I133 I054 Figure 9 Five selecte images in the t 1 -t 2 score plot. 4 Conclusion A new paraigm for machine vision is presente for hanling stochastic textural appearance in the process inustries. The new machine vision can also be extene easily to inclue spectral appearance as well by using proper methos [7]. It is applie in this paper for the estimation of the visual quality of injection-mole plastic panels but the new paraigm also opens many opportunities for many systems engineering tasks such as moeling, monitoring, control, an optimization of the visual quality of proucts or processes. It is also

12 being applie in a ifferent context to the monitoring an control of the visual appearance of flotation froths in mineral processing [4]. Acknowlegement Authors woul like to thank Dr. Nancy Jestel at GE Avance Materials for proviing the ata for this stuy. I123 I228 I089 I059 Figure 10 Four selecte images in the t 3 -t 4 score plot.

13 Reference (1) Marshall, A.D; an Martin R.R. Computer Vision, Moels an Inspection, Worl Scientific Publishing: Singapore, (2) Hyper Dictionary. (3) Davies, E.R. Machine Vision: Theory, Algorithms, Practicalities, 2n e., Acaemic Press: San Diego, (4) Liu, J.; MacGregor, J.F.; Duchesne, C., an Bartolacci, G. Monitoring of Flotation Processes using Multiresolutional Multivariate Image Analysis, Minerals Engineering 2004, in press. (5) Honglu, Y.; MacGregor, J.F.; Haarsma, G.; Bourg, W. Digital Imaging for On-line Monitoring an Control of Inustrial Snack Foo Processes, Inustrial & Engineering Chemistry Research, 2003, 42, (6) Honglu, Y.; MacGregor, J.F. Monitoring Flames in an Inustrial Boiler Using Multivariate Image Analysis, AIChE Journal 2004, 50, (7) Liu, J.; MacGregor, J.F. On the Extraction of Spectral an Spatial Information for Image Analysis, submitte for publication, (8) Mallat, S.G. A theory for multiresolution signal ecomposition: The wavelet representation, IEEE Transactions on Pattern Analysis an Machine Intelligence, 1989, 11, 674. (9) Daubechies, I. Ten Lectures on Wavelets, CBMS-NSF Reg. Conf. Series in Applie Math. no. 61, SIAM, Philaelphia, (10) Vetterli, M.; Kovaevi, J. Wavelets an Subban Coing. Prentice Hall: Englewoo Cliffs, NJ, (11) Bharati, M; Liu, J.; MacGregor, J.F. Image Texture Analysis: Methos an Comparisons, Chemometrics an Intelligent Laboratory Systems, 2004, 72, 57. (12) Ranen, T.; Husoy, J.H. Filtering for texture classification: a comparative stuy, IEEE Transactions on Pattern Analysis an Machine Intelligence, 1999, 21(4), 291. (13) Tuceryan, M.; Jain, A.K. Texture Analysis, Hanbook of Pattern Recognition an Computer Vision (ch. 21), 2n e., C.H. Chen, et al. es., Worl Scientific Publishing Co.: NJ, (14) Etma, K.; Chellappa, R. Separability-base multiscale basis selection an feature extraction for signal an image classification, IEEE Transactions on Image Processing, 1998, 7, (15) Chang, T.; Kuo, C.C.J. Texture Analysis an Classification with Tree-Structure Wavelet Transform, IEEE Transactions on Image Processing, 1993, 2, 429. (16) Hotelling, H. Analysis of a complex of statistical variables into principal components, Journal of Eucational Psychology, 1933, 24, 417. (17) Karhunen, K. Uber lineare methoen in er Wahrsccheilichkeitsrechnung, Annales Acaemiae Scientiarum Fennicae, Serie A1: Mathematica-Physica, 1947, 37, 3. (18) Loéve, M. Probability Theory, Van Nostran: New York, (19) Fisher, R.A. The use of multiple measurements in taxonomic problems, Annals of Eugenics, 1936, 7, Part II, 179. (20) Bell, A.J.; Sejnoski, T.J. An information-maximization approach to blin separation an blin econvolution, Neural Computation, 1995, 7, (21) Frieman, J.H.; Tukey, J.W. A projection pursuit algorithm for exploratory ata analysis, IEEE Transactions on Computers, 1974, 23, 881.

14 (22) Moolman, D.W.; Eksteen, J.J.; Alrich, C.; Van Deventer, J.S.J. The significance of flotation froth appearance for machine vision control, International Journal of Mineral Processing, 1996, 48, 135. (23) Mojsilovi, A.; Popovi, M.V.; Rackkov, D.M. On the selection of an optimal wavelet basis for texture characterization, IEEE Transactions on Image Processing, 2001, 19, (24) Besse, P.; e Falguerolles, A. Application of resampling methos to the choice of imension in principal component analysis, Computer intensive methos in statistics, W. Härle an L. Simar, es., Physica-Verlag: Heielberg, Germany, pp , (25) Jackson, J.E. A User s Guie to Principal Components, Wiley-Interscience: New York, (26) Dua, R.O.; Hart, P.E.; Stork, D.G. Pattern Classification, Wiley-Interscience: New York, 2000.