Failure Detection and Isolation in Ceramic Tile Edges Based on Contour Descriptor Analysis

Transcription

1 Control & Automation, July 27-29, 27, Athens - Greece T15-5 Failure Detection and Isolation in Ceramic Tile Edges Based on Contour Descriptor Analysis Ž. Hocenski, T. Keser University J.J. Strossmayer, Faculty of Electrical Engineering, Osijek, Croatia Abstract Today s industry is based on the principle of automatics. In order to increase production quality and improve production yield that reduces production costs, there is tendency to automate every phase in the production line and if possible to replace the human resource entirely. Ceramic tile industry is not an exception. In the ceramic tile production line almost every phase is more or less automated. The only part which is least automated is visual inspection and classification of a ceramic tile. In this paper we present a method for visual inspection of ceramic tile edges, their failure detection and isolation using a machine vision system. For edges inspection, detection and failure isolation we use a method based on contour tracing and description of tile edges. The method developed generates a contour reference descriptor based on al contour tracing, determines tile orientation for referenced preprocessing and localizes and isolates the failure. The method is tested and results are given for several types and sizes of ceramic tiles. I. INTRODUCTION In modern ceramic tile industry almost every production phase is automated. In most cases the last phase of visual ceramic tile quality inspection is not automated. The tendency in every industry is to introduce automation in every stage of production. The reasons lie in the fact that today customers are very demanding as to product quality. To increase production quality it is necessary to increase production costs if traditional non automated production methods are used. Reduction of production costs and keeping good quality of final products could be achieved by automating as many phases in the production line as possible. By automation better production efficiency and an increase in the total amount of products in time which directly decreases total production costs and final product price could also be achieved. Ceramic tile production is a very complex process. The production process is an assembly of chemical, mechanical, thermo-dynamical and other processes which must be finely tuned for good and successful production. In the production chain there are numerous ways for things to go wrong. Every failure directly reflects on the final ceramic tile. Thus, a failure could be at the beginning of the process being reflected on the ceramic tile structure as irregularity in geometrical composition of the tile (broken tile, broken edges and/or corners, surface scratches, planar concavity and convexity, bumps, pits, etc.). In the middle and at the end of the process (glazing, texture printing and biscuit baking) a failure could appear as small pin holes, surface cracks, glazing coverage irregularity, texture printing misalignment or misprint and wrong color and texture composition. During the whole production process there is a constant possibility of mechanical damage of ceramic tiles while transporting them from one production phase to another. The final production phase is visual inspection and classification of ceramic tiles. In most cases the last phase is based on human perception capability. Human resources as controllers in this phase are very unreliable. During hard working conditions in plant and human visual perception limitations, a man as a controller chain could be a source of failures and decrease production yield and product quality which increases total production costs. To avoid or minimize human related failure causes as much as possible, this paper presents one of the ways how to introduce automatics as the main worker for this stage in the ceramic tile production. II. CERAMIC TILE VISUAL INSPECTION SETUP Ceramic tile visual inspection for the purpose of detecting failures on tiles is performed by the machine vision system and image processing algorithm subsets. In this case a machine vision system fully replaces human resources and in synergy with appropriate algorithms it could be even more powerful than men as a visual inspection resource. In some areas of visual perception, men are still irreplaceable as visual preceptors, especially in areas where texture recognition is important. The main reason for introduction of machine vision instead of man is to increase reliability of a visual inspection system, production quality and yield as well as to reduce production costs. A machine vision system consists of one, two or more digital acquisition cameras and one or more processing computers, depending on process complexity. Today s cameras are developed enough to be able to register even the smallest detail on the tile surface. Acquisition cameras are often line scan cameras (area scan cameras are used less because of their high cost and other problems related to acquisition processes). These are mono or color cameras with a relatively high resolution of acquisition. Monochromatic cameras are grayscale cameras used for acquisition of a high detailed image of a ceramic tile where image color composition information is not required for analysis. In contrast to them, color cameras are used to acquire images needed for color composition based analysis and they often have resolution lower than mono cameras. Processing computers are ordinary personal computers adapted to the usage in industrial environment with

2 Control & Automation, July 27-29, 27, Athens - Greece T15-5 camera interfaces and processing algorithms incorporated. Also, processing computers have interfaces for ceramic tile automated classification machines. A typical setup of the machine vision system for ceramic tile quality inspection is shown in Fig. 1. Processing Computer Classification Machine Cameras Conveyor Belt Tile Classification Actuator Figure 1. Principle setup of the machine vision system for ceramic tile quality inspection and classification. A visual inspection system with machine vision is supplemented by the system for automated ceramic tile classification. The tile classification machine is a logical extension of the machine vision system for the purpose of full automation of the whole stage of visual inspection and classification in the production path. III. EDGE INSPECTION The complete process of visual inspection in ceramic tile quality control and classification consists of many process analysis subsets as in [1]. Each subset analysis covers one or more features and parameters of the image which is significant for a specific kind of a ceramic tile. Image Acquisition Adjust Image for Analysis Feature Extraction Edges... Texture... Feature Interpreter and Failures Detection Classification Rules Classification and Actuation Figure 2. Principle block diagram of visual inspection in ceramic tile quality control and classification. Edge feature inspection is also just one of many parameters observed during the visual inspection process. Edge irregularity is one of the first parameters which should be inspected. If a tile has significant irregularities in edges with respect to the reference tile, that tile goes into recycling and there is no need for further analysis. If irregularities are relatively small and acceptable, then the tile goes further into the analysis process but could be treated immediately as the lowest class tile. Many more or less successful approaches and methods as to edge and surface inspection have been developed so far [2,3,4,5,6]. Basically, all edges inspection methods are derived from edge finding methods in conjunction with Adjustable Processing Parameters geometrical analysis as in [7] or shape recognition and reconstruction [8,9,1]. One of the frequently used methods is the one based on the well known Hough transformation. Hough transformation finds strait lines in image shapes and gives the number of lines, their angles and position as information. With Hough transformation it is possible to find accurately boundary lines of shape edges only if the shape is rectangular. Other shapes with curvatures incorporated into the shape figure are not suitable. For Hough transformation it is necessary to previously find edges of shapes in the analyzed image. Methods based on al gradient analysis are most frequently used methods with good results. There are numerous methods to find edges but commonly used are gradient methods with Canny, Prewitt or Sobel based 1 st order derivative kernel appliance [11]. The result of the edge finding method is an image with a binary result where logical zero means no edge and logical one means significant gradient in image intensity which is the edge of the shape. Upon application of an edge filter, Hough transformation finds strait lines on the image with extracted edges. Generated strait lines describe the analyzed shape and edge irregularity can be determined by further geometrical analysis. A disadvantage of Hough transformation is relatively slow for strait lines and not being suitable for a high resolution image where the amount of data to process in real time processing environment is significant. In [12] Hough transformation is successfully presented and used as an efficient way to find edges irregularities in small amounts. A. Edge Contour Description Edge contour description is based on well known edge contour tracing methods. As in [13], there are three typical most frequently used conventional contour tracing algorithms. They are pixel center, pixel corner and edge point tracing method. These algorithms exist in many modified forms. Basically, they trace shape edges on a binary image which is result of an edge applying filter and generates a shape descriptor as a representative of shape edges. The method used in this paper is based on a modified pixel center tracing method with al. First, it is necessary to prepare acquired image I(x,y) for the first analysis step, i.e. edge detection of the ceramic tile. For better edge detection we emphasize the image of the tile against its background using simple intensity thresholding method as described in [12] and using (1), ( x, y) T( I ) ( x, y) > T( I ) if I B( x, y) =, (1) 255 if I where B(x,y) is the emphasized image and T(I) is a thresholding level. The used thresholding method consists of a histogram analysis of the portion of image background, I B, and portion of tile image, I T, as shown in Fig. 3a. After determining histograms, H 1 (I T ) and H 2 (I B ), just simply subtract these two histograms to determine a new histogram H(I), (2). () I H ( I ) H ( ) H 1 T 2 I B = (2)

3 Control & Automation, July 27-29, 27, Athens - Greece T15-5 To determine a threshold level T(I), from new histogram H(I) we calculate its mean value, m(i), and standard deviation, s(i) and by means of (3) we get a threshold level value, T () I = m() I k s() I, (3) where k usually ranges from.8 to 1.2 for the best result. The emphasizing result, image B(x,y), is shown in Fig. 3b, where it can be seen that a tile portion of the image has maximum contrast against the background thus producing a better result by edge detection than the ordinary image without emphasizing. where D(k) is a matrix with calculated preferred s and is given by (5). cos ) sin ) + π 4) sin + π 4) π 4) sin π 4) + π 2) sin + π 2) π 2) sin ϕ( k 1) π 2 cos D ( k) = cos (5) ( ) cos cos Preferred s are incorporated into matrix D(k). Most preferred, as shown in Table I, is the first row of a D(k) matrix, then the second, third, fourth and fifth, which is checked at the end of the process. TABLE I. SEARCHING DIRECTION WEIGHT AND ORDER a) Acquired image I(x,y) with analyzed portions. b) Emphasized image, B(x,y), of tile against background. Figure 3. Emphasizing image of ceramic tile using histogram subtraction techniques. After emphasizing image intensity, the next step is to analyze ceramic tile edges. For edge detection a commonly used gradient technique with Canny kernel is used. A Canny edge detector gives a contour of a ceramic tile image as shown in Fig. 4b. The result is a new matrix of data, new image E(x,y), whose values are represented binary, logical zeros and ones. Zeroes in the matrix represent (black) pixels which are not edge pixels and ones represent edge (white) pixels. a) Acquired image, I(x,y). b) Detected edges image, E(x,y). Figure 4. Image edge detection using a Canny edge detector. A al contour tracing method is based on for edge pixels of the tile shape with additional information about their along the path. The method consists of two parts; i.e. for any edge contour pixel, e(x,y), along a predefined angle, ϕ, in image E(x,y), and tracing edge contour pixels and registering their against the tracing path. For al and tracing we need two parameters. One is angle of the previous contour pixel (step k-1), ϕ(k- 1) and the other is window with coordinates of current pixel, S(k), for the next one. Searching window, S(k), is a matrix with five new calculated coordinates of pixels in the k-th step against the previous located edge contour pixel, e(x,y) in the k-1-th step. Matrix S(k) is given in (4), ( k) S( k ) D( k) S = 1 +, (4) Direction Weight Searching Order º High Priority First 45 º 1 Second -45 º 2 Third 9 º 3 Fourth -9º 4 Least Priority Fifth When S(k) is calculated, then, as shown in Fig.5a, the algorithm inspects those five new pixels neighborhood for edge contour presence and marks those ones being edge pixels. After locating an edge pixel, the algorithm accepts new ϕ(k) as ϕ(k-1) and proceeds further to find the next one. At the beginning the algorithm marks the first located edge pixel as the first pixel of contour ϕ(k) 45 ϕ(k) ϕ(k-1) ϕ(k-1) ϕ(k-2) a) Search in k-th step. b) Searching in k+1-th step. Figure 5. Directional contour tracing method example for two steps. The algorithm stops when it reaches the first marked contour pixel or if there are no more pixels in the inspection neighborhood. There is a potential problem if contour of the shape has holes. To avoid this problem, it would be good to previously check contour continuity and if there are holes, fill them to close contour. In each tracing step the al contour tracing method gives to each located pixel their against the previously located. As a result, we get a vector ϕ(n) of n elements in length where each element has an absolute angle of contour shape against the referent coordinate system of image E(x,y). Length of vector, n, matches the shape contour length in pixels and it is variable depending on the shape size and image resolution. In Fig.6 results of this tracing method are given for several types of shapes. Each analyzed shape has a very different result than expected with this method.

4 square.bmp Proceedings of the 15th Mediterranean Conference on Control & Automation, July 27-29, 27, Athens - Greece T15-5 CCW a) Square b) Absolute angle change for square shape. CW c) Circle d) Absolute angle change for circle shape. CCW e) Ellipse f) Absolute angle change for ellipse shape. CCW g) Defected sq. h) Absolute angle change for defected sq. shape. CW i) Flower j) Absolute angle change for flower shape. Figure 6. Absolute angle change during shape contour tracing for several types of shapes using al contour tracing method. A tracing result, the vector with angle change ϕ, uniquely describes contour of the shape always in the angle interval from to 2π, (6), and represents the analyzed shape contour descriptor. max ( ϕ) min( ϕ) = 2π (6) The slope of the result depends on entry point tracing. For examples given in Fig.6 and π/4 entry from upper left corner, a positive slope (Fig. 6d and 6j) means a clockwise tracing and counter-clockwise for negative slope (Fig. 6b, 6f and 6h). B. Edge Failure Detection and Isolation Edge failure detection is based on contour descriptor analysis of reference and the analyzed shape of the tile. First, it is necessary to calculate a contour descriptor of the reference tile shape which has no edge failures, ϕ ref. Because the analyzed tile could be of the same shape but different size overall there is a need also to know contour length of the reference tile shape, L ref. After calculating ϕ ref and L ref a currently analyzed shape using the mentioned tracing method produces new sets ϕ and L. The easiest way to detect a failure in the analyzed shape is to compare reference and analyzed contour descriptor, ϕ ref and ϕ. Since contour lengths L ref and L can be different, there must be ϕ ref and ϕ normalized to the same length (7). Usually normalize ϕ to size of ϕ ref, ϕ( k) ( F ) + [ ϕ( C) ϕ( F )]( P F ), L = Lref ϕ '( k) = (7) ϕ, L Lref where k is in the interval k ( 1...L ref ), R is a normalizing factor, F and C is flooring and ceiling function, respectively, and P is a calculated sampling point (8). R = L L ref, P = 1+ R( k 1) ( P), C = ceil( P) F = floor Entry points and can also be different. So, the reference tile shape could be expanded at least over 4π angles (two description cycles) to successfully match two normalized descriptors and find descriptor difference with (9). [ ] ref L ( k ) = ϕref '( k ) ϕ' k= 1 Finding minimum of descriptor difference, δ ( k ) δ (9), gives a matched entry point and matched reference and shape. Consider shapes shown in Fig. 6a and 6g. There are two very similar shapes but with a slight difference, the shape in Fig. 6g has edge and corner defects. Defects presence can be directly seen from their contour descriptors comparison, Fig. 6b and 6h. Descriptor of shape in Fig. 6g has strong irregularity in one strait edge and one relatively weak on the upper right corner. Irregularities of these shapes are marked and shown in Fig. 7a and 7b (8) a) Descriptor comparison. b) Descriptor difference. Figure 7. Failure detection and isolation on defected square shape.

5 Control & Automation, July 27-29, 27, Athens - Greece T15-5 From Fig.7b it can be seen that descriptor difference has comparison spikes. Only bipolar shaped spikes represent defects on tile and those spikes should be searched for. Unipolar spikes in Fig. 7b represent no ideally matched two shape descriptors as normalization of the analyzed shape unintentionally increases length of the shape contour. Position of bipolar shaped spikes against the entry point of analysis gives the exact position where defects occur. With those features we achieve good localization of defects and prepare further analysis for isolation of those failures. From Fig.7a it can be seen that there are two defects (entry point is near the upper left edge), one occurs on third strait tile edge and the other on the third corner of the tile along the contour tracing path. IV. EXPERIMENTAL RESULTS The presented method was applied on ceramic tiles for edge defects detection purposes. Results are shown in Fig.8. As mentioned in the previous chapter, with emphasizing intensity of ceramic tile portion of image and applying the Canny edge detector we get edges contour of a ceramic tile. Tile Contour Descriptor Contour Difference Properties 1 1 a) (168x13) -2 Tile size: 25x2cm t B=.4 s t E=4.1 s t M=.5 s T=4.64 s L=4292 Px L/Lref=1. No defects 1 1 b) (168x13) -25 Tile size: 25x2cm t B=.4 s t E=4.15 s t M=.49 s T=4.68 s L=4223 Px L/Lref=.985 Corner defect 1 2 c) (168x13) -25 Tile size: 25x2cm t B=.4 s t E=4.1 s t M=.47 s T=4.61 s L=426 Px L/Lref=.992 Corner defect 1 3 d) (168x13) -25 Tile size: 25x2cm t B=.4 s t E=4.1 s t M=.5 s T=4.64 s L=4292 Px L/Lref=1. Edge defect e) (168x13) -25 Tile size: 25x2cm t B=.4 s t E=4.5 s t M=.47 s T=4.56 s L=492 Px L/Lref=.953 Large corner defect Figure 8. Results of al contour tracing method for ceramic tile visual inspection, failure detection and isolation.

6 Control & Automation, July 27-29, 27, Athens - Greece T15-5 Directional tracing method applied on edges contour of the tile generates a contour descriptor which is analyzed. Ceramic tiles are rectangular shapes and they have a contour descriptor very similar to square shapes, as shown in Fig. 6b, but have two strait portions of a descriptor longer than other two which makes them different. Visual inspection setup for ceramic tiles image acquisition is capable of producing the image of tiles almost identically aligned but with enough small amounts of translational and rotational component to each other to be acceptable for analysis without additional contour difference matching. So, the analysis entry points are very similar and the tracing method produces contour descriptor slopes with same orientations. Fig.8 shows the influence of defects on the method. Tile in Fig.8a is a reference tile and tiles in Fig.8b through 8e are analyzed tiles with their results against the reference tile. There are large or small bipolar shaped changes in descriptor difference. Larger differences are caused by larger defects and vice versa. Properties for each tile are also given. Parameter t M is the time needed for applying the method, t B for emphasizing intensity of the image, t E for edge detection using the Canny edge detector and T is the total execution time which represents the sum of t M, t B and t E (based on AMD XP32@2.2GHz and Matlab 7.3). Parameter L is the length of contour in pixels, L/L ref the ratio of geometrical extension or shortening of the contour analyzed. From the given properties it can be seen that larger defects cause noticeable shortening in contour length and significantly larger contour differences. Also, shortening of contour, for large resolution images like in Fig.8, causes relatively small reduction in time needed to complete the tracing process. V. CONCLUSION In this paper we present a method successfully applied as an approach to ceramic tile edges failure detection and isolation using contour tracing analysis, descriptions and representation. The method is based on principles of contour tracing methods modified with an additional feature where the contour edge trace process additionally generates for each traced pixel their against the previous one. This feature of a contour pixel helps us describe the shape through absolute orientation of the traced contour against the reference coordinate system which is the image of the acquired ceramic tile. Such traced contour of the tile is represented with a vector of angles for each contour pixel and is called a shape descriptor. From contour descriptors their contour structures can be clearly seen, that are always the same for a specific type of shapes and their geometry. Sharp changes in the descriptor of analyzed shapes are caused by portions of contour where corners or corner like structures are rectangular or polygon shapes. Contrary to polygon shapes, circular or curved shapes have much less changes in their contour descriptors. That characteristic of shape description with this method allows us to use this method as an efficient detector for ceramic tile edges inspection purposes by simply comparing a reference contour descriptor with the analyzed one. From differences of these two descriptors we extract failures in analyzed shapes and additionally locate them if necessary for further analysis for classification purposes. The method is shown as relatively fast even if a high resolution image for processing is used during the experiment. Execution times achieved in the experiment are still not acceptable for applying real time environment processing, especially the time needed for edge detection. With further optimization during code programming and using the method for edge detection which is not so much computationally intensive, the time execution could be additionally reduced. By using processor architecture specific multimedia or image processing features execution time could also be significantly reduced, thus making the presented method more convenient for real time processing. REFERENCES [1] H. Elbehiery, A. Hefnawy and M. Elewa, Surface Defects Detection for Ceramic Tiles Using Image Processing and Morphological Techniques, Transactions on Engineering, Computing and Technology, Vol. 5, pp , April 25 [2] M.L. Smith and R.J. Stamp, Automated inspection of textured ceramic tiles, Computers in Industry, Vol. 43 (1), pp , August 2 [3] Z. Hocenski, S. Rimac-Drlje and T. Keser, Visual Diagnostics Based on Image Wavelet Transformation, 9th European Conference on Power Electronics and Applications, EPE 21, pp , Graz, Austria, August 27-29, 21. [4] C. Boukouvalas, J. Kittler, R. Marik and M. Petrou, Color Grading of Randomly Textured Ceramic Tiles Using Color Histograms, IEEE Transactions on Industrial Electronics, Vol. 46 (1), pp , February [5] S. Vasilic and Z. Hocenski, The Edge Detecting Methods in Ceramic Tiles Defect Detection, IEEE International Symposium on Industrial Electronics, ISIE26, pp , Montreal, Canada, July 26. [6] C. Boukouvalas, J. Kittler, R. Marik and M. Petrou, Automatic Color Grading of Ceramic Tiles Using Machine Vision, IEEE Transactions on Industrial Electronics, vol.44 (1), pp , February 1997 [7] F. Lopez, A study of registration methods for ceramic tile inspection purposes, Proceedings of the pattern recognition and image analysis (SNRFA'21). Vol. 1, pp , Castellón, Spain, 21. [8] H. Kauppinen, T. Seppanen and M. Pietikainen, An experimental comparison of autoregressive and fourier-based descriptors in 2d shape classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, pp , [9] F. Mokhtarian, S. Abbasi and J. Kittler, Robust and efficient shape indexing through curvature scale space, British Machine Vision Conference, [1] P. Pala and S. Santini, Image retrieval by shape and texture, Pattern Recognition, Vol. 32, pp , [11] T. Lindeberg, "Edge detection and ridge detection with automatic scale selection", International Journal of Computer Vision, 3, 2, pp , [12] J.M. Valiente, F. Acebron and F. Lopez, A ceramic tile inspection system for detecting corner defects, Proceedings of the pattern recognition and image analysis (SNRFA'21), Volume 2, pp , Castellón, Spain, 21. [13] T. Miyatake, H. Matsushima and M. Ejiri, Contour representation of binary images using run-type codes, Machine Vision and Applications, Volume 9(4), pp , February 1997.