A Nondestructive Bump Inspection in Flip Chip Component using Fuzzy Filtering and Image Processing 103 A Nondestructive Bump Inspection in Flip Chip Component using Fuzzy Filtering and Image Processing Surachet Karnpracha 1, Anakkapon Seanton 2, and Somyot Kaitwanidvilai 3, Non-members ABSTRACT In this paper, the fuzzy filtering and conventional image processing are adopted for inspecting the bump in Flip-Chip component, an important part of hard disk drive. Bump is the soldering point of either circuit line or component in flip-chip. To inspect the bump, high resolution X-ray camera is applied. Human inspection is mostly used to detect bump properties such as the bump ratio, bump s connection, bump s size etc from the x-ray image. However, this process results in time consuming and human error. To overcome this problem, in this paper, automatic visual inspection which mainly composes of the image processing algorithms is applied. In addition, to enhance the capability of noise reduction in image processing algorithms, fuzzy filtering is investigated. Fuzzy techniques are powerful tools for knowledge representation and processing. This technique can manage the vagueness and ambiguity efficiently. The conventional image filters are also performed for comparison. In this paper, mean filter, median filter, adaptive Wiener filter and fuzzy filter are performed for performance comparison. Experimental results show that the fuzzy filter has more efficient than other techniques when apply to filter the Gaussian noise. The automatic visual inspection is successfully achieved by the proposed algorithms. Keywords: Automatic Visual Machine Inspection, Fuzzy Image Processing, Fuzzy Filtering, Image Processing 1. INTRODUCTION In the hard disk drive company, component inspection is an important process for this manufacturing. Physical inspection is the one of inspection process which usually applied for inspect the HDD component. In most cases, the high resolution camera is applied for this task. In addition, for non-destructive inspection, X-ray camera and ultrasonic sensors may Manuscript received on March 31, 2007 ; revised on May 15, 2007. 1 The authors are with the Electrical and Computer Engineering, Faculty of Engineering, Naresuan University, Pitsanulok, Thailand, Emails: surachetka@nu.ac.th, u45360559@nu.ac.th, somyotk@nu.ac.th be applied. This paper studied the implementation of image processing for inspection the Flip-Chip at Belton Thailand Co., Ltd. Flip-Chip is an important part which has many specifications to be inspected. Interconnection and bump shape are the most important specifications which failures in these specifications will cause rejection and waste. At present, the interconnection and bump shape (in this company) are inspected by human. The major problems in this process are human error, time consuming, inaccurate and non-repeatable inspection result. By this reason, this research work is proposed to study and solve the problem by using the development of automatic visual inspection for Flip-Chip. The main objective of this research is to develop a real time, fast and accurate algorithm for this inspection process. Based on the size of this component, high precision in micrometer-scale X-ray camera is required for this purpose. In this paper, an accurate 2D image processing algorithm using fuzzy filtering and conventional image feature extraction are developed. In many image processing applications, the expert knowledge is frequently incorporated to overcome the difficulties (e.g. object recognition, scene analysis). FIP is the powerful tools to represent and process human knowledge in form of fuzzy if-then rules. On the other side, many difficulties in image processing, which is not always due to the randomness but to the ambiguity and vagueness, can be managed by FIP. Examples of FIP are shown in [1-3]. Ali, M.A. [1] proposed a new shape-based algorithm, called fuzzy image segmentation using shape information (FISS) by incorporating general shape information. The new FISS algorithm compared to other well-established shape-based fuzzy clustering algorithms in their paper. Marino, P. et. al.[2] applied the FIP to a visual inspection for a quality control process. They showed that the fuzzy image processing is able to the inspection of each can end efficiently. Van De Ville, D. [3] introduced a new fuzzy filter which has two stages. The first stage computes a fuzzy derivative and second stage uses these fuzzy derivatives to perform fuzzy smoothing. They also implemented and compared this algorithm to the other filters which FIP gained some advantages in noise reduction over the conventional filter. In this paper, the fuzzy fil-
104 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2 August 2007 tering and conventional filtering are investigated for Flip-Chip visual inspection machine. The remainder of this paper is organized as follows. Section 2 addresses the concept of conventional filtering and Fuzzy filtering. Section 3 describes the experimental setup and results. Section 4 concludes the paper. 2. FUZZY FILTERING AND CONVEN- TIONAL FILTERING METHODS In this paper, the X-ray image from X-ray camera is used as a raw image for our visual inspection. Fig.1. shows the diagram of experimental setup in this research work. The image from X-ray camera is send to the image processing unit which developed on PC. In this paper, the image processing for inspecting the bump shape can be divided as 2 parts. First is image filtering and other is image feature extraction. This section describes the theory and concept of all image processing used in this paper. used for convolution is usually much smaller is called as the kernel. Fig.2: An example small image (left) and kernel (right) for illustrating convolution.[6] O 57 = I 57 K 11 + I 58 K 12 + I 59 K 13 + I 67 K 21 + I 68 K 22 + I 69 K 23 (1) For the mean filter, in this paper, a 3 3 square kernel as shown in Fig. 3 is applied. Fig.3: 3 3 kernel of mean filter. 2. 1...2 Median Filtering Fig.1: The experimental setup for Automatic Visual Inspection Machine. 2. 1 Image Filtering Image filtering is a process by which we can enhance images. There are two types of filtering, low pass and high pass filter. To remove the noise, low pass image filters are usually applied. Following filtering techniques are the low pass image processing which attempted for removing the image noise. 2. 1...1 Mean Filtering Mean filtering is a simple, intuitive and easy to implement method of smoothing images, i.e. reducing the amount of intensity variation between one pixel and the next. Convolution provides a way of multiplying together two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality [6]. To illustrate this operation, see Fig. 2 for the example of convolution operation. In image processing, one of the frequently used input arrays is normally just a grey level image. The second array Median filter calculate the output image from the median value of a pixel neighbourhood. See [8] for more details. 2. 1...3 Adaptive Wiener Filtering Wiener filter is a filter which uses the average and variance for determining the output image value. Following equations are used to illustrate this filter. σ 2 = 1 NM u = 1 NM n,m η n,m around η I(n, m) I 2 (n, m) u 2 (2) Where σ 2 is variance, η is the neighborhood pixel Output image pixel value can be determined by o new (x, y) = u + σ2 + v 2 σ 2 (I(x, y) u) (3) Where v 2 is the average value of variance
A Nondestructive Bump Inspection in Flip Chip Component using Fuzzy Filtering and Image Processing 105 Fig.4: Neighbourhood of pixel position (x,y) and pixel values indicated in gray are used to compute the Fuzzy derivative for the direction NW. 2. 1...4 Fuzzy Filtering (Fuzzy Noise Reduction) [7] In this paper, the fuzzy image processing in [7] is adopted. This section illustrates the concept of fuzzy filtering for noise reduction. Figure 4 shows the neighborhood of the central pixel (x,y) defined in this paper. The position of each direction is described by table 1. The fuzzy derivative of each direction can be computed using some of the neighborhood pixel. For example, the pixel values in green shaded box in figure 4 are used for the fuzzy derivative calculation of direction NW. Table 1: Position in each direction. Direction NW W SW S SE E NE N position (x 1,y 1) (x 1,y) (x 1,y+1) (x,y+1) (x+1,y+1) (x+1,y) (x+1,y 1) (x,y 1) The fuzzy noise reduction steps described in [7] can be shown in following: A.Fuzzification Before applying the fuzzy image for noise reduction, the derivation denoted by D (x, y) is defined. Example of derivations in direction N and NW are shown as follows: N (x, y) = I(x, y 1) I(x, y) Fig.5: Membership function for the property small. B.Fuzzy Smoothing The membership function for the property small is defined as m K (u) = 1 u K, 0 K = 0, u > K (6) See Figure 5 for illustration of membership function. The value of fuzzy derivative for the pixel (x, y), F D (x, y), in every direction can be calculated by applying the fuzzy rule. For example in the NW direction; direction: If( NW (x, y) is small) and NW (x 1, y + 1) is small) or ( NW (x, y)is small) and ( NW (x + 1, y 1) is small) or ( NW (x 1, y + 1)) is small) and ( NW (x + 1, y 1) is small) Then ( F NW (x, y) is small) To compute the correction term for any pixels, the pair of fuzzy rules for each direction is applied [7]. Example for the direction in NW is shown in following. λ + NW : if F NW (x, y) is small and NW (x, y) is positive then c is positive λ + NW : if F NW (x, y) is small and NW (x, y) is negative then c is negative Where properties positive and negative are defined the membership function in Fig. 7 and, respectively. The value L is the number of gray level of the images. In this paper, L = 255. The correction term for this fuzzy noise reduction technique for the direction D can be computed by [7] = L 8 (λ + D λ D ) (7) Dɛdir N (x, y) = I(x, y 1) I(x, y) (4) To compute the degree in which the fuzzy derivative in a certain direction is small, the membership function as shown in Fig. 6 is defined. K is the adaptive parameter [7] defined as follows K = σα (5) where α is final amplification factor that can be selected, σ is the noise variance [7]. Fig.6: Membership function for the properties positive negative.
106 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2 August 2007 2. 2 Feature Extraction Features are used to represent the shape or character of interesting pattern in the object recognition and mostly used to classification process. In this paper, the image features such as area, eccentricity, angles are used to identify the completeness of bump in Flip-Chip. For example, area is used the search and counting for each interesting object. The eccentricity is the ratio of the distance between the foci of the ellipse and its major axis length. The value is between 0 and 1. (0 and 1 are degenerate cases; an ellipse whose eccentricity is 0 is actually a circle, while an ellipse whose eccentricity is 1 is a line segment.)[8]. For example of eccentricity ratio, see Figure 7. In this figure, the image is converted to binary image and the processing will count the pixel for determining the area and also the major and minor axis length (pixel connected in the direction of major and minor axis). For the area value, the image processing will only count the pixel connected in the same object. See [8] for more details. (c) (d) Fig.8: Original image from X-Ray camera Image with additive Gaussian white noise variance σ = 0.01 (c) Detail Image of Fig.2 (d) Detail Image of Fig.2 Figure 9 shows the experimental results for applying the fuzzy filtering. A number of iterations of applying the fuzzy filtering to the image with noise were performed to evaluate the optimal iteration. To evaluate this performance, MSE in following equation was adopted. Fig.7: Example of image feature extraction: eccentricity value. 3. EXPERIMENTAL RESULTS To reduce the image noise, image filtering is firstly performed. The image feature extraction algorithms are next applied for inspecting the bump shape. To evaluate the performance of fuzzy filtering and conventional methods, all of filtering methods discussed in this paper are implemented for filtering the image with noise. In this paper, the Gaussian noise is added to the original image for making the situation of poor image which contains noise. Mean square error (MSE) between the original and filtered image is used as the performance index. Figure 8 shows the original image and image with additive Gaussian noise level (noise variance σ = 0.01). (c) (f) (d) (g) Fig.9: Original image from X-Ray camera Image with additive Gaussian white noise σ = 0.01 (c) 1 st Iteration of applying Fuzzy filter (d) 5 th iteration (e) 10 th iteration and (f) 20 th iteration. In Fig. 9, it is clearly shown that a large number of iterations of applying fuzzy filter to the image appear to be smoother than the lower one. However, the performance index in this paper is not defined by the smoothness. To compare the effective of using the
A Nondestructive Bump Inspection in Flip Chip Component using Fuzzy Filtering and Image Processing 107 fuzzy filter, the MSE versus iterations in various are shown in Figure 10. By this figure, the α which makes a lowest MSE is 2. By this figure, the number of iteration which is optimal is 2. MSE can be written as Conventional image feature extraction is applied for finding the interesting feature in interesting region. As shown in Fig. 11, for instance, eccentricity and area are extracted using this method. These values can be used for inspecting the completeness of bump in inspection process. Table 2: MSE value of overall filters in this paper. Fig.10: MSE versus Iterations of applying fuzzy filtering in various α. The conventional filters are adopted for comparison. The comparison of MSE between overall filters is shown in Table 2. As shown in this table, the fuzzy filtering gains the lowest M SE compared to overall filters. It is meaning that the output image when applying this filter is closer to the original image than others. (c) Fig.11: Conventional image feature extraction Original image region of Interest and (c) Results of area and eccentricity measurement. Type of Image MSE Image with Noise 285.3613 Mean filter 102.1824 Median filter [3 3] 100.5341 Adaptive Wiener filter [3 3] 81.6742 Fuzzy Filter [3 3], α = 1 83.9128 Fuzzy Filter [3 3], α = 2 72.5506 Fuzzy Filter [3 3], α = 3 74.1844 4. CONCLUSIONS Conventional image processing can be applied for Flip-Chip interconnection and bump shape inspection. The image feature such as bump size, bump s eccentricity can be used for inspecting the bump shape. The fuzzy image processing used in this paper is fuzzy filtering which can reduce the noise as shown by experimental. The comparisons between fuzzy filtering and other filters are presented. As shown by results, in this case, fuzzy filter has a lowest MSE for filtering process. The proposed algorithms are implemented in visual inspection in HDD industrial. The experimental results prove the ability of the proposed technique in practical work. 5. ACKNOWLEDGEMENT The authors would like to express our gratitude to NECTEC, HDD Cluster for the support of this research work. References [1] Ali, M.A. Karmakar, G.C. Dooley, L.S., Fuzzy image segmentation using shape information, ICME 2005. IEEE International Conference on Multimedia and Expo, 6-8 July 2005. [2] Marino, P. Pastoriza, V. Santamarfa, M. Martinez, E., Fuzzy image processing in quality control application, 2005. Sixth International Conference on Computational Intelligence and Multimedia Applications, 16-18 August 2005 [3] Van De Ville, D. et. al., Noise reduction by fuzzy image filtering, IEEE Transactions on Fuzzy Systems, Vol. 11 : 4, pp.: 429-436, August 2003. [4] Zadeh, L. A., Fuzzy sets, Information and Control, Vol. 8, pp. 338-353. (1965). [5] http://pami.uwaterloo.ca/tizhoosh/edge.htm [6] http://www.cee.hw.ac.uk/hipr/html/mean.html
108 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2 August 2007 [7] Dimitri Van De vile, Mike Nachtegral, et al. Noise Reduction Technique by Fuzzy Image filtering, IEEE trans on fuzzy system, Vol. 11, No. 4, August 2003. [8] www.mathworks.com/products/image/ Surachet Kanprachar received B.Eng. (First class honors) in Electrical Engineering from Chulalongkorn University, Bangkok, Thailand in 1996 and received M.Sc. and Ph.D. in Electrical Engineering from Virginia Tech, Virginia, USA, in 1999 and 2003, respectively. He is currently an Assistant Professor at the Department of Electrical and Computer Engineering, Faculty of Engineering, Naresuan University, Phitsanulok, Thailand. His research interests are optical communications, wireless communications, and coding theory applications. Anakapol Santhon received B. Eng. in Computer Engineering from Naresuan University. He is currently a master student at the Department of Electrical and Computer Engineering, Faculty of Engineering, Naresuan University, Phitsanulok, Thailand. His research interests are image processing and robotics. Somyot Kaitwanidvilai received B. Eng.(honors) and M.Eng. in Electrical Engineering from King Mongkut Institute of Technologies Lardkrabang, Bangkok, Thailand in 2000 and 2004, respectively. He is currently a lecturer at the Department of Electrical and Computer Engineering, Faculty of Engineering, Naresuan University, Phitsanulok, Thailand. His research interests are image processing, mechatronics, power electronics and EMC.