Artificial Intelligence and Image Processing Approaches in Damage Assessment and Material Evaluation S.S. Kumar 1, F. Taheri 2, and M.R. Islam 2 1 Graduate Student, 2 Professor, Respectively; Dept of Civil Engg, Dalhousie University, Halifax, NS- B3J 1Z1, Canada E-mail: {sskumar, Farid.Taheri, Rafiqul.Islam}@dal.ca Abstract The Ultrasonic is an inspection technique (UT), which employs high frequency acoustic waves to probe the sample being inspected. As the acoustic wave penetrates the sample, the wave is attenuated and/or reflected as a result of variation in the density (sound velocity) of the material. By observing and post processing the returned signal, be it the reflected signal or the signal emanating from the opposite side of the sample, one can effectively evaluate the material s characteristics such as material microstructures, as well as flaws existing in the material. This paper describes different Artificial Intelligence (AI) and Image Processing methods, which could be utilized to investigate various defects in metals as well as composites. The proposed system is highly robust and effective in situations where a large number of similar samples are to be investigated. The proposed methods utilizes Artificial Neural Networks (ANN), Fuzzy Logic and Image Analysis to recognize various types of defects in a given specimen. Image processing and wavelets techniques are used to determine the details of the damage geometry. The above system is an integral part of a robust damage analysis software under the development. An Adaptive Neuro Fuzzy Inference System is also being developed for composites, suggestive repair mechanicsm. MATLAB language is used in developing a real time automated damage assessment and evaluation prototype system. 1. Introduction Ultrasonic technique [1] has been most widely applied to detect cracks, delamination, debonding and defects hidden in solids and material evaluation. Selection of proper transducers, water scan or air scan, pulse echo or through transmission, longitudinal waves or shear waves or plate waves, spike pulse or tone burst signal and reference standards are all key parameters. Appropriate Image Pre processing is a very important step to make images suitable for various purposes. It sharpens the image feature, adjusts contrast, converts RGB image to binary and so forth. In practical situations, noisy input data are inevitable. The Wavelet technique could play an important role in de-noising and compressing of images. The Artificial Neural Networks (ANN) have the capability of constructing an arbitrary nonlinear mapping from multiple input data to multiple output data within the network through learning sample input versus output relations, and estimating appropriate output data, even for unlearned input output relations. Either Perceptron neural networks or Probabilistic neural networks (PNN) can be used for our classification problem [2], where one needs to classify the sample as good or defective based on the pixel values of the C-scan image. The Genetic Algorithm could be used for automatic configuration of neural networks, as well as for weight optimization. Fuzzy c-means (FCM) is a data clustering technique wherein each data point belongs to a cluster to some degree that is specified by a membership grade. FCM groups pixel data points of a C-scan image into a specific number of different clusters as good or defect. Image Analysis of a C-scan image analysis the distribution of intensities in an indexed image. A binary image histogram plot could be drawn by making 2 equally spaced bins ( good or defect ), each representing a range of data values. It then calculates the number of pixels within each range. The Image Viewer provides information about the size of the image, the display range of pixel values, and the value of the pixel in the location of the mouse pointer.
In an attempt to mimic the expertise of a human by a computer, this paper also describes an Adaptive Neuro-Fuzzy Expert System (ANFIS) developed for composite repair mechanism, designed to mimic the human decision process. An expert system allows for easy encoding of expert knowledge as a set of rules. The Fuzziness of an expert system allows better treatment of the uncertainties of the problem, and simplifying the expert system itself [3]. A Graphical User Interface based (GUI), Integrated Software Package is currently being developed using the MATLAB language for automated investigation of flaws in materials and for evaluation of material properties based on ultrasonic testing. This package will include various modules such as a geographic information system, database management system, risk assessment, expert system, digital image processing, medical image processing, neural networks and fuzzy logic for damage assessment to mention a few. The proposed system would be a generalized one, such that it would be capable of treating any type of images obtained by various tools such as optical scanner, MRI scan, CT scan, gyroscopic technology, SSET and other methods. 2. Automated Damage Evaluation System The development of the automated damage evaluation system [4] is divided into the following four steps, as schematically shown in Figure 1. collected and displayed in a number of different formats. The C-scan presentation [1] provides a plantype view of the location and size of test specimen features. The relative signal amplitude or the time-offlight is displayed as a shade of gray or a color for each of the positions where data was recorded. The C-scan presentation (Figure 2) provides an image of the intensity of the reflected and scattered sound within a graphite-epoxy test specimen. Figure 2. C-scan image of the sample 2.2. Step 2 - Image Pre-Processing Image Pre-processing is an important step to make images suitable for various purposes. It sharpens the image feature, adjusts contrast, performs RGB to GRAY scale conversions; also, performs resizing, splitting and de-noising of images [5]. De-noising is one of the most important applications of wavelets. Two-Dimensional wavelet analysis [6] is performed repeatedly on the noisy image until we are able to get satisfactory de-noised image. The Haar and sym6 wavelets can be used in succession to remove blocking and ringing effect in an image (see Figures 3 and 4). Figure 3. Noisy c-scan image Figure 4. De-noised c-scan image 2.3. Step 3 Damage detection methodologies Figure 1. Damage evaluation system 2.1. Step 1 - C-scan image acquisition A typical ultrasonic inspection system consists of several functional units, such as the pulser/receiver transducer, and display devices. Ultrasonic data can be Any of the following four methodologies could be used to determine defect in a given sample; one could also visualize the intensity of the defect. Each method has its own advantage as well as disadvantages. 2.3.1. Step 3(a) - Perceptron Neural Networks Damage Assessment. Perceptron is one of the simplest single-layer networks whose weights and biases could be trained to produce a correct target vector when presented with the corresponding input vector. The training technique used is called the
perceptron learning rule. The perceptron has the ability to generalize from its training vectors and learn from its initially randomly distributed connections. Perceptrons are especially suited for simple problems in pattern classification. A perceptron neuron, which uses the MATLAB s hard-limit transfer function hardlim, is shown in Figure 5. Within this procedure, each external input is weighted with an appropriate weight w 1j, and the sum of the weighted inputs is sent to the hard-limit transfer function, which also has an input of 1 transmitted to it through the bias. The hardlimit transfer function, which returns a 0 or a 1 is shown in Figure 6. right of the line L cause the neuron to output 0. The dividing line can be oriented and moved anywhere to classify the input space as desired by picking the weight and bias values. Hard-limit neurons without a bias will always have a classification line going through the origin. Adding a bias allows the neuron to solve problems where the two sets of input vectors are not located on different sides of the origin. The bias allows the decision boundary to be shifted away from the origin as shown in the above plot. The perceptron network consists of a single layer of S perceptron neurons connected to R inputs through a set of weights w i,j as shown in Figure 8 in two forms [7]. As before, the network indices i and j indicate that w i,j is the strength of the connection from the j th input to the i th neuron. Figure 5. perceptron neuron Figure 6. Transfer function The perceptron neuron produces a 1, if the net input into the transfer function is equal to or greater than 0; otherwise, it produces a 0. The hard-limit transfer function gives a perceptron the ability to classify input vectors by dividing the input space into two regions. Specifically, outputs will be 0 if the net input n is less than 0, or 1 otherwise. The input space of a two-input hard limit neuron with the weights w 1,1 = -1, w 1,2 = 1 and a bias b =1, is shown in Figure 7. Figure 7. Input space of hard limit neuron In reference to Figure 7, two classification regions [7] are formed by the decision boundary line L at Wp + b = 0. This line is perpendicular to the weight matrix W and is shifted according to the bias b. Input vectors above and to the left of the line L will result in a net input greater than 0; and therefore, cause the hard-limit neuron to output a 1. Input vectors below and to the Figure 8. Perceptron neural network architecture In supervised learning, the learning rule is provided with a set of examples (the training set) of proper network behavior: {p 1, t 1 }, {p 2, t 2 },..., {p q, t q }, where p q is an input to the network, and t q is the corresponding correct (target) output. As the inputs are applied to the network, the network outputs are compared to the targets. The learning rule is then used to adjust the weights and biases of the network in order to move the network outputs closer to the targets. The perceptron learning rule falls in this supervised learning category. Perceptron neural networks can be used for our classification problem [2], where one needs to classify the sample as good or defective based on the pixel values of the C-scan image. Figure 9 shows the two-way classification of our defective test specimen.
Figure 9. Perceptron neural networks damage assessment and classification 2.3.2. Step 3(b) - Probabilistic Neural Networks Damage Assessment. Probabilistic neural networks (PNN) can be used for our classification problem [2] where one needs to classify the sample as good or defective based on the pixel values of the C-scan image. When an input is presented, the first layer computes distances from the input vector to the training input vectors, and produces a vector whose elements indicate how close the input is to a training input. The second layer sums these contributions, for each class of input, to produce a vector of probabilities as its net output. Finally, MATLAB s compete transfer function on the output of the second layer picks the maximum of these probabilities, and associates a 1 for that class and a 0 for the other classes. A PNN is guaranteed to converge to a Bayesian classifier, provided it is given adequate training data. These networks generalize well, but are slower to operate because they use more computation intensive than the other kinds of networks. The architecture for this system [7] is shown in Figure 10. Figure 10. Probabilistic neural network architecture The PNN is trained and tested using the reference samples as the target. Our defective sample (Figure 2) shows a specimen with severe defect shown in blue color, mild defect is identified in green and best portion is identified by a red color. After performing the necessary pre-processing of the C-scan image of the samples, the images are fed into the PNN for damage assessment. At this juncture, the PNN is ready to detect the damage by comparing the pixel color values of the samples with the pixel values of the reference healthy speciemn, and thus performing a three-way classification as (i) severe defect, (ii) mild defect and (iii) best, based on pixel intensity values. Figure 11 shows the three-way classification of our defective sample. Figure 11. Probabilistic neural networks damage assessment 2.3.3. Step 3(c) -Fuzzy C-Means Clustering Damage Assessment. Clustering of numerical data forms the basis of many classification and system modeling algorithms. The purpose of clustering is to identify natural groupings of data from a large data set to produce a concise representation of a system's behavior. It can be used to classify the sample as good or defective based on the pixel values of the C-scan image. You can use the cluster information to generate a Sugeno-type fuzzy inference system that best models the data behavior, using a minimum number of rules. The rules partition themselves according to the fuzzy qualities associated with each of the data clusters. Fuzzy c-means (FCM) is a data clustering technique [8], wherein each data point belongs to a cluster to some degree that is specified by a membership grade. It provides a method that shows how to group data points that populate some multidimensional space into a specific number of different clusters. MATLAB s command line function fcm starts with an initial guess for the cluster centers, intended to mark the mean location of each cluster. The initial
guess for these cluster centers, however, is most likely incorrect. Additionally, the fcm assigns every data point of a cluster to a membership grade. By iteratively updating the cluster centers and the membership grades for each data point, the fcm iteratively moves the cluster centers to the right location within a data set. This iteration is based on minimizing an objective function that represents the distance from any given data point to a cluster center weighted by that data point's membership grade. The fcm is a command line function whose output is a list of cluster centers and several membership grades for each data point. You can use the information returned by the fcm to help you build a fuzzy inference system by creating membership functions to represent the fuzzy qualities of each cluster. one of only two discrete values. Essentially, these two values correspond to on (or good), and off (or defective). A binary image is stored as a logical array of 0's (off pixels) and 1's (on pixels). An image histogram is a chart [9] that shows the distribution of the intensities of an indexed, binary or intensity image. The MATLAB image histogram function [5] creates this plot by making n equally spaced bins, each representing a range of data values. It then calculates the number of the pixels within each range. The Figures 13 (a) and (b) displays a binary image of our defective test sample s C-scan image with and without colormap, and Figure 13(c) shows a histogram based on the two binaries. a) Binary image of our defective sample with colormap b) Binary image of our defective sample Figure 12. A typical plot based on the fuzzy c-mean clustering damage assessment Since would not have a clear idea as to the number of the clusters that would be in a given set of data, the subtractive clustering, is a fast one-pass algorithm available in the MATLAB for estimating the number of clusters and the clusters centers in a set of data. The cluster estimates obtained through this function [8] could be used to initialize the iterative optimizationbased clustering methods (fcm) and the model identification methods (such as the anfis). The subclust function finds the clusters by using the subtractive clustering method. Fuzzy C-Means Clustering can be used for our classification problem, where one would need to classify the test sample as good or defective, based on the pixel values of the C-scan image. Figure 12 shows the two-way classification of our defective sample. 2.3.4. Step 3(d) Image Analysis for Damage Assessment. In a binary image, each pixel assumes (c) Histogram of binary image of our sample Figure 13. Outcome of binary image analysis Using MATLAB s image viewer and pixel region tool [5] one could obtain information about specific pixels in an image. The pixel region rectangle defines the region of the image that one would desire to examine. The pixel region tool displays a grid of cells where each cell represents a pixel in the region specified by the rectangle. Each cell contains the numeric value of the pixel. For RGB images, each cell contains three numeric values, one for each band of the image. For indexed images, the cell contains the index value and the associated RGB value. The color of the cell represents the color of the pixel. If a defect is present, one could then assess the intensity of the defect by performing pixel by pixel analysis of the C- scan image. Figure 14 shows the pixel by pixel analysis of the defective sample.
Figure 14. Pixel by pixel image analysis 2.4. Step 4 Neuro-Fuzzy Expert System The basic structure of this type of fuzzy inference system is a model that maps the input characteristics to input membership functions, the input membership function to rules, rules to a set of output characteristics, the output characteristics to output membership functions, and the output membership function to a single-valued output or a decision associated with the output. The membership functions are usually fixed, and somewhat arbitrarily selected. Also, one should apply the fuzzy inference to modeling systems whose rule structure is essentially predetermined by the user's interpretation of the characteristics of the variables in the model. membership functions to the input/output data (Figure 15), in order to account for these types of variations in the data values. This is where the so-called neuroadaptive learning techniques [10] incorporated using ANFIS in our software package. The basic idea behind these neuro-adaptive learning techniques is very simple. These techniques provide a method for the fuzzy modeling procedure to learn information about a data set, in order to compute the membership function parameters that best allows the associated fuzzy inference system to track the given input/output data. This learning method works similarly to that of neural networks. A network-type structure similar to that of a neural network (Figure 16), which maps inputs through input membership functions and associated parameters, and then through output membership functions and the associated parameters to outputs, can be used to interpret the input/output map. Figure 16. ANFIS Model Structure Figure 15. Input/Output map There will be some modeling situations (as in our case), in which one cannot simply look at the data and discern what the membership functions should look like. Rather than choosing the parameters associated with a given membership function arbitrarily, these parameters could be chosen so as to tailor the The following six parameters are taken as the input [11] in the development of the proposed Expert System for Composites Repair Mechanism modules of the system: (i) Surface or deep damage; (ii) thin or thick composite; (iii) temporary or permanent repair; (iv) lightly or heavily loaded; (v) partial or full strength; and (vi) rough or smooth flush finish, which could also be changed as per our requirements. Based on the six input parameters, the ANFIS would suggest an output of repair mechanism from a series of defined repair mechanisms like cosmetic, resin injection, semi-structural plug/patch, structural mechanically fastened doubler, structural bonded external doubler, and structural flush repair. A sample output of our Adaptive Neuro- Fuzzy Inference System is shown in Figure 17.
4. Acknowledgement We acknowledge the support of the Atlantic Innovation Fund, the Canada Foundation for Innovation, and other partners which fund the Facilities for Materials Characterization, managed by the Institute for Research in Materials. 5. References [1] NDT Resource Center. http://www.ndt-ed.org / index_flash.htm. [2] Hagan, M.A., H.B. Demuth, M.H. Beale. (2003): Neural Network Design, Brooks Cole, ISBN: 0-9717321-0-8. [3] L. Zadeh. (1987): Fuzzy Sets and Applications: Selected Papers by L.A. Zadeh, ed. R.R. Yager et al, John Wiley, New York. Figure 17. Output of the adaptive neuro-fuzzy inference system 3. Conclusions In this paper, the details of several new methodologies organized for damage assessment, using the Artificial Neural Networks (ANN), Fuzzy Logic and Image Analysis and the associated suggestive repair mechanism Expert System based on Adaptive Neuro-Fuzzy Expert System (ANFIS), were discussed. A real time automated prototype system using the MATLAB language for the damage assessment and the repair mechanism was developed. The proposed system is a generalized one that not only could it be used to decipher ultrasonic C-scan images, but also would be capable of treating any type of images obtained by various tools such as an optical scanner, gyroscopic technology, MRI scan, CT scan, SSET and other similar methods. The proposed composite repair mechanism could also be easily modified to accommodate other materials. Our future endeavors will include further development of the system, such that it could be used in real time through the Internet, as an online Web-based structural health monitoring system. [4] Kumar, S. and F. Taheri, F. (2004): Neuro-Fuzzy Approaches for FRP Oil and Gas Pipeline Condition Assessment, American Society of Mechanical Engineers, Pressure Vessels and Piping Division (publication), V490, Storage Tank Integrity and Materials Evaluation, p271-275. [5] Image processing toolbox user s guide. (2005) The Math Works, Natick, Massachusetts, USA. [6] Wavelet toolbox user s guide. (2005) The Math Works, Natick, Massachusetts, USA. [7] Neural Network toolbox user s guide. (2005) The Math Works, Natick, Massachusetts, USA. [8] Fuzzy Logic toolbox user s guide. (2005) The Math Works, Natick, Massachusetts, USA. [9] Gonzalez R, R.E. Woods and S.L. Eddins. (2004): Digital Image Processing Using MATLAB, Pearson Prentice Hall, ISBN 0-13-008519-7. [10] Jang J. S. R, (1992). ``Neuro-Fuzzy Modeling: Architectures, Analyses, and Applications.'' Ph.D. Dissertation, EECS Department, Univ. of California at Berkeley. [11] Armstrong K. and R. Barrett. (1998): Care and Repair of Advanced Composites, SAE International, ISBN: 0768000475.