Available online at ScienceDirect. Procedia Engineering 188 (2017 ) 72 79

Transcription

1 Available online at ScienceDirect Procedia Engineering 188 (2017 ) th Asia Pacific Workshop on Structural Health Monitoring, 6th APWSHM HweeKwon Jung a, ChangWon Lee a, Gyuhae Park a * a School of Mechanical Engineering, Chonnam National University, South Korea Abstract The crack detection during the manufacturing process is an important step for quality management of panel products. Traditional crack detection methods are subjective and expensive because they are performed by experienced human inspectors. Therefore, crack detection techniques for automated and accurate inspection are required. In this paper, a crack detection technique based on image processing is proposed, which utilizes the images of panel products captured by a regular CCTV camera system. First, the binary panel object image is extracted from various backgrounds after considering RGB color factors. Edge lines are then generated from a binary image using a percolation process. Finally, crack detection and localization are performed with a unique edge line evaluation. In order to demonstrate the capability of the proposed technique, lab-scale experiments were carried out with a thin aluminum plate and a real sample panel. In addition, the test was performed with the images acquired at an actual press line. Experimental results show that the proposed technique could effectively detect panel cracks with an improved rate and speed Published The Authors. by Elsevier Published Ltd. This by Elsevier is an open Ltd. access article under the CC BY-NC-ND license ( Peer-review under responsibility of the organizing committee of the 6th APWSHM. Peer-review under responsibility of the organizing committee of the 6th APWSHM Keywords: Pressed panel product, Image processing, Percolation, Crack detection, Edgeline analysis 1. Introduction Various mechanical components are produced by sheet metals with several manufacturing processes, including press-working. During these processes, including punching, blanking, embossing, materials undergo large deformations in high speed, which may result in manufacturing defects such as cracks, necking and marking lines [1, 2]. In order to detect such defects, crack detection is usually implemented by human inspectors. The detection rate of * Corresponding author. Tel.: address: gpark@jnu.ac.kr Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license ( Peer-review under responsibility of the organizing committee of the 6th APWSHM doi: /j.proeng

2 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) the method is affected by the skill and the experience of human inspectors. Additionally this method is less reliable and unstable in many cases. For these reason, the development and implementation of an automated and accurate defect detection technique is important during the press-working process. One type of defect inspection technique is to monitor the vibrations of pressure signals of a press line [3, 4]. In these methods, the distortion and/or shape change of pressure signals indicate abnormal conditions of the press line. However, this method has a limitation that it is not appropriate for individual panel inspection. Compared to the methods, image based defect detection methods could provide several advantages over existing methods in the press line because they are non-invasive, accurate, and could be easily implemented in the manufacturing. Thus various studies on image processing for crack detection have been conducted. Image processing based methods for crack detection enables quick inspection of large area of structural surface. Moreover, stable and accurate inspection is possible with an automated process. Currently, most studies on image processing focused on detection of cracks on large structures such as bridge, building and pavement. Niemueller et al. [5] introduced several methods to detect crack and corrosion on the surface of pipes. Jian and Bunke. [6] developed an edge detection algorithm based on the scan line approximation technique. Abjel-Qader et al. [7] compared and evaluated the performances of the four crack detection methods including Fast Haar Transformation (FHT), Fast Fourier Transformation (FFT), Sobel and Canny filter. The study confirmed that FHT was the most reliable method in crack detection. Fujita and Hamamoto. [8] proposed a robust and automated detection method, targeting crack detection on the surface of concrete structures. In their study, a median filter was implemented to refine the collected images, and cracks were detected based on the image difference method. Meanwhile, Lyer and Sinha. [9] developed a pipe inspection system based on a pattern recognition process. Defect detection based on image processing is usually applied to inspect large detection area. Thus automated detection and analysis are essential. Also the acceleration of processing speed is critical for real-world applications. Valenca et al. [10] developed an image processing technique to detect cracks in concrete surfaces to monitor the crack initiation and growth in concrete specimen. Yang et al. [11] achieved three dimensional images using two stereo cameras in a real bridge. The deflection of the bridge posts was estimated by tracking grid s movement. Crack region was finally visualized based on the deflection analysis. Zou et al. [12] conducted three image processing steps in order to automatically detect cracks on pavements. In their study, cracks were detected by applying a tree representation and a pruning algorithm to the generated map. To improve the processing speed, Yamaguchi et al. [13] performed the study on the acceleration of the percolation model, which is necessary for crack detection. Compared to the many studies for large structure inspection, only a few image processing technique were applied to products in manufacturing lines [14, 15]. There are following requirements that image processing for production line must have, It could detect manufacturing defects with high accuracy. False positive error should be minimized. Inspection should be completed in real-time. This paper aims to develop an image processing technique which is capable of identifying surface defects for pressed panel products. The proposed technique consists of the following steps; i) object extraction from backgrounds, ii) object s shape and edge line extraction, and iii) edge line analysis for crack detection. Several labscale experiments are conducted to demonstrate the performance of the proposed technique. Additionally, the proposed technique is applied to a manufacturing press line to detect crack on real panels. 2. Image processing technique for crack detection Crack detection procedure based on image processing consists of four steps as shown in Fig. 1. First, a new panel images, during the manufacturing stage, is captured using a camera system installed in a press line. In order to extract the target image of interest from various backgrounds, every pixel value is calculated by considering Red, Green, Blue (RGB) color factors and brightness factors. A binary image is then generated with the pre-defined threshold value, as the second step. In the third step, the percolation method is applied to extract the edge line information of the object. As the final step, the extracted edge lines are analyzed for defect detection. Almost all of

3 74 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) edge lines of panels contain smooth variances of angle in the edges for pressed panel products. When a crack occurs in panel products, there is a sudden variance of the edge line angle, which could be used as an indication of the presence of a crack. Therefore, the relative angle variances of each line are evaluated for detection and localization of cracks in this study. Fig. 1. Overall image based processes for crack detection 2.1. Object image extraction Once a panel image is captured, the target object is extracted from backgrounds. In this study, the concept of visual perception was introduced, as the same way that a human interprets target objects from surrounding environments by processing the information of color and brightness. In each image, the color and the brightness of pixels are presented with three color factors (Red, Green and Blue). Based on the concepts, a ratio of the light factor is used to distinguish the color and amplitude and to assess the brightness. For the first step, the values of the light factor in a pixel of target object are selected and the ratios of the values are derived using Eq. 1. (After the ratios are derived, they are defined as a reference ratio value.) Then the same procedure is applied to all pixels in the image. In order to distinguish the pixels based on the color factors, the differences between ratios of the light factors are obtained using Eq. 2. Meanwhile, the brightness is also considered to distinguish the pixels. Because cracks and holes are darker than the other parts, the values of the light factors are relatively low in those pixels. By using the predefined threshold values, cracks and holes could be identified. (1) (2) After the image processing for color ratio difference, the preprocessed image is converted into a binary image.

4 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) Fig. 2. Object extraction process considering color and brightness factors 2.2. Percolation model based shape recognition and edge line extraction After the binary image generation, the percolation model is implemented to extract the object shape and edge line. The percolation model is based on natural phenomenon of liquid permeation, which could be used to identify various shapes. In our study, scalable local processing based on the percolation model for shape recognition and edge line extraction is employed. Before the percolation process, an initial seed point is defined. From left side of the binary image, we conduct line scanning to the right direction. Because a pixel with zero value represents the target object, the initial seed point is defined as a first search pixel. From the initial seed point, the percolation process is initialized. The process evaluates eight neighboring pixels of a pixel in center, which is described below. The values of the eight neighboring pixels are identified from the center pixels. For the first step, the center pixel is defined as a reference pixel where percolation model starts from. If the neighboring pixels have zero value from the predefined reference pixels, they are defined as next reference pixels. Until a neighboring pixel with zero value is not found, the process repeats the step 2. If there is no pixel with the zero value, the percolation process is terminated. During the processes, the percolation model is used for two purposes. First, the shape of the object is extracted. Based on the extracted shape information, the edge line is generated by searching pixels whose neighboring pixel contains background pixels. With the procedure, the generated edge line has single pixel of width. After then, the percolation model is used again in order to extract the edge lines.

5 76 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) Fig. 3. Flow chart of percolation model 2.3. Crack detection using edge line analysis After the edge lines are extracted based on the percolation model, cracks are detected using a unique edge line analysis. In this study, the edge line analysis is performed by tracking the edge lines. For example, two edge lines of panels with and without crack are compared as depicted in Fig. 4. In case that a panel product contains no defect, the angle variation of each edge is not over 90 degree while tracking the entire edge lines. However, an acute relative angle variation occurs where a crack is formed in a panel product. We defined the acute angle variation which is higher than 140 degree as an indication of a crack. Fig. 4. Edge line analysis

6 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) Experimental results Several experiments were performed to verify the proposed technique. A lab scale experiment was performed with simple objects. After then, the technique was applied to a real press line. MATLAB and intel 4 th i5 (3.2GHz) with OS window 7 and 8GB RAM was used as signal processor. While taking images, we utilized a cell phone camera as an image acquisition device Crack detection results with simple objects During the first experiment, an aluminum thin plate was used. In this experiment, every image had 4128 x 2322 pixels. For efficient image processing, resolution of image was decreased by four times compared to that of the original image. In the objects shown in Fig. 5, a grey thin aluminum plate was a target object and the others were background objects. Fig. 5 (a) shows panel without any defects. A crack was simulated by tearing the panel. Circular and rectangular shaped holes were also introduced to the objects for false-positive indication. The proposed crack detection technique was applied to the both panels. In case of the panel without a crack, the algorithm reports no crack in the structure. For the defected panels, no false positive error was reported with the circular and rectangular holes due to the fact that there relative angles are within the pre-defined threshold limit. Fig. 5. (a) Crack detection results Fig. 5. (b) Crack detection results (an aluminum thin plate without crack) (an aluminum thin plate with crack) 3.2. Crack detection results with real panel images Second experiment was carried out at a real press line. As shown in Fig. 6, the cell phone camera captured the image of panel (washing machine) in a real press line. The camera was placed 1-m above the press line. During the process, each panel was produced in every 10 second. A total of 78 images for 13 minutes are collected with the image acquisition area of 70 x 45 cm. The extracted image contained 750 x 400 pixels. During the experiment, no defective product was identified. Therefore, manual cracks, with the sizes in the range of 1 x 7cm, 0.5 x 4cm, 0.5 x 2cm and 0.2 x 2cm, were simulated at various locations in the images. In order to improve the processing time, the initial image resolution was lowered by two to four times. As the results shown in

7 78 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) Fig. 7, it was demonstrated that the crack could be detected with a very high accuracy. It should be emphasized that, even the real panel products contain sharply created holes which look similar to cracks, there was no false-positive error reported. Fig. 6. Image acquisition setup in a real press line and extracted panel image Fig. 7. Results of crack detection on a real panel product 4. Conclusion This paper proposed an image processing technique for rapid and automated crack detection for pressed panel products. The target object is extracted from backgrounds by considering color and brightness factors. Then the edge line of the object is extracted using a percolation model. Because the percolation process is computationally intensive, the acceleration concept was introduced to reduce the processing time for percolation. Finally, cracks are detected and localized with the unique edge line analysis developed in this study. For the validation of the proposed technique, several experimental investigations are performed. Through experiments, it was shown that the proposed image processing technique was able to detect surface cracks with the reasonable accuracy and the speed. The proposed image processing technique could be efficiently used for crack detection in the pressed panel with the advantages of cost reduction, fast inspection and high accuracy. Acknowledgements This research was supported by Basic Science Research program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education ( ). This work (C ) was also supported by business for Cooperative R&D between Industry, Academy, and Research Institute funded Korea Small and Medium Business Administration in 20.

8 HweeKwon Jung et al. / Procedia Engineering 188 ( 2017 ) References [1] S. Kalpakjian and S. R. Schmid, Manufacturing process for engineering materials, 5/e in SI units, pp [2] Taylan Altan, Dissecting defects part 1: Examining process variables to fined stamped part quality flaws, <FMA-the Fabricators & Manufacturers Association>, October 9, [3] H. Du and B. E. Klamecki, Force sensors embedded in surfaces for manufacturing and other tribological process monitoring, Journal of Manufacturing Science and Engineering, Vol. 121, Issue. 4, pp , [4] N. Mahayotsanun, J. Cao, M. Peshkin, S. Sah, R. Gao, C.T. Wang, Intergrated sensing system for stamping monitoring control, IEEE SENSORS 2007 Conference, Vol. 5, pp , [5] T. Niemueller, Automatic detection and segmentation of cracks in underground pipeline images, Seminar: Medical Image Processing Summer Semester 2006, No , [6] X. Jiang and H. Bunke, Edge detection in range images based on scan line approximation, Computer Vision and Image Understanding, Vol. 73, No. 2, pp , [7] I. Abdel-Qader, O. Abudayyeh, M. E. Kelly, Analysis of edge-detection techniques for crack identification in bridges, Journal of Comuting in Civil Engineering, Vol. 17, Issue. 4, pp , [8] Y.Fujita and Y. Hamamoto, A robust automatic crack detection method from noisy concrete surfaces, Machine Vision and Applications, Vol. 22, pp , [9] S. Iyer and S. K. Sinha, Segmentation of pipe images for crack detection in buried sewers, Computer-Aided Civil and Infrastructure Engineering, Vol. 21, pp , [10] J. Valenca, D. Dias-da-Costa, E.N.B.S. Julio, Characterisation of concrete cracking during laboratorial tests using image processing, Construction and Building Materials, Vol. 28, pp , [11] Y. S. Yang, C. M. Yang, C. W. Huang, Thin crack observation in a reinforced concrete bridge pier test using image processing and analysis, Advances in Engineering Software, Vol. 53. pp , [12] Q. Zou, Y. Cao, Q. Li, Q. Mao, S. Wang, CrackTree: Automatic crack detection from pavement images, Pattern Recognition Letters, Vol. 33, pp , [13] T. Yamaguchi, S. Hashimoto, Fast crack detection method for large-size concrete surface images using percolation-based image processing, Vol. 21, pp , [14] H. Elbehiery, A. Hefnawy, M. Elewa, Surface defects detection for ceramic tiles using image processing and morphological techniques, International Journal of Computer, Electrical, Automation, Control and Information Engineering, Vol. 1, No. 5, pp , [15] C. H. Kim, S. H. Choi, W. J. Joo, G. B. Kim Classification of surface defect on steel strip by KNN classifier, Advances in Engineering Software, Vol. 83, pp , 2015.