Image Processing Techniques Applied to Problems of Industrial Automation

Transcription

1 Image Processing Techniques Applied to Problems of Industrial Automation Sérgio Oliveira Instituto Superior Técnico Abstract This work focuses on the development of applications of image processing for a smart camera, within the automation industry scope. Since the smart camera was acquired recently, it was necessary to install the whole software, to get familiarized with the software routines of the provided package and to the camera operating system, and its connection to programmable logic units. The applications fall into the area of recognition of objects from their contours, obtained from captured images. This involves the need to extract features form objects and develop systems for automatic decision. Decision systems based on fuzzy sets were used in conjunction with Fourier descriptors. The implemented system was able to recognize all the objects when subjected to validation. Keywords: Smart cameras, Fourier descriptors, fuzzy classification systems, recognition of objects 1 Introduction The recognition of objects is a task performed daily by humans and is related with the need to interact with the surrounding environment. Humans tend to view images as being composed of individual objects that can be identified by their shape. When there is lack of human experience or the job puts their health at risk, the automated vision systems are their natural substitutes. The automated recognition of objects may be an important aid in the manufacture and quality control and operate continuously with a constant and consistent performance. These systems have to be reasonably fast and robust to classify an object, to be more effective than a human. Using a human for recognition of objects has the disadvantage of him getting distracted or annoyed if the task is very monotonous and if he is not fast enough if the pace of production so requires. A smart camera is an integrated vision system that includes a processor that extracts and executes procedures without the need of an external processing. It has interfaces to provide the results to others equipment, e. g., RS232 serial interface, outputs for a PLC, actuators, etc. Currently in industry, smart cameras are mainly used to distinguish the good parts from those which show defects, i.e., carry out quality control, but can also perform other tasks, such as measurements without the need for contact, identification and separation of parts, reading and verification of codes, among others. Recognition of forms has many applications including the recognition of handwriting [CKZ1994] [Cohen1994], recognition of faces [KP2004] [FC1995], etc. There are many forms of 1

2 recognition, systems based on statistical classifiers, neural networks [MWA1995], or using the Euclidean distance [RM2002]. This work is an integrated classification system based on fuzzy sets associated with Fourier descriptors for a smart camera. This paper it is organized in the following way: In Section 2 is presented the Fourier descriptors, fuzzy systems in section 3 and section 4 presents the smart camera used in this work. Finally, results are presented in section 5 and conclusions in section 6. 2 Fourier Descriptors In general the representation of a contour of an object is a one dimensional function that describes areas or two-dimensional contours. Different types of representations have been used for the calculation of descriptors. Complex coordinates, curvature function, angular cumulative function and distance to the centroid are the most used forms of contour representation [ZL2003]. The Fourier descriptors are widely used to describe the form of an existing object in an image. These descriptors are formed by the coefficients of the discrete Fourier Transform (DFT). This operation transforms a complex or a real function in other complex function. The new function is the representation in the frequency domain of the first function and describes the frequencies present in the function. In Fourier descriptors, the general characteristics of the shape of an object are captured by low frequencies, while high frequencies capture the finest details. These descriptors appear to be less sensitive to noise and are easy to normalize and safeguard the information. Before applying the Fourier transform a sample of fixed points belonging to the contour of the object is collected. Let it be two functions b n = x n + iy n e b n = x n + iy n of the contour from the same object. A typical measure of similarity is the Euclidean distance, which corresponds to the mean square error and is directly related to the cross-correlation [RM2002]. N 1 D 2 b, b = b b 2 n=0 This relationship becomes ambiguous if the vectors with the coordinates of the contours have different sizes. To avoid this problem a fixed number of points within the contour is used to obtain the Fourier descriptors. This process has the effect of smoothing the contour by eliminating noise and finest details. To facilitate the use of the calculating algorithm for the Fourier transform FFT (Fast Fourier Transform), the number of points to consider will be a power of two. There are generally three methods of contour normalization (i) equal points sampling; (ii) equal angle sampling; and (iii) equal arc length sampling. [ZL2002]. The equal angle sampling selects candidate points spaced at equal angle θ = 2π/K. The equal points sampling method selects candidate points spaced at equal number of points along the shape boundary. The equal arc length sampling method selects candidate points spaced at equal arc length along the shape boundary. Among the three sampling methods, the equal arc length sampling method apparently achieves the best equal space effect, because achieves the unit 2

3 speed of motion along the shape boundary [Otterloo1991]. The discrete Fourier transform (DFT) requires that the function being transformed is discrete and that their values different from zero are finite. The DFT evaluates only enough data to reconstruct the finite segment that is being analyzed. The DFT is given by. Where N 1 i2πkn N Z k = z n e n=0, k = 0, 1, 2, 3,, N 1 z n = x n + iy n, n = 0, 1, 2, 3,, N 1 And x n and y n are the coordinates of the points in Cartesian space. The inverse transform is given by z n = 1 N N 1 k=0 i2πkn N Z k e, n = 0, 1, 2, 3,, N 1 In forms analysis we only are interested in the boundary of the shape. The position, size, and rotation are not important. In order to compare the descriptors of the object they must be invariant to rotation, scale, translation and to the starting point. The translation will affect only the first element of the discrete Fourier transform, which can then be obtained by eliminating the element Z 0. The invariance to rotation and to the starting point is achieved by discarding the phase angle, i.e. by only considering the magnitude. To make invariant to scale, magnitudes are divided by the magnitude of the element Z 1. 3 Classification In evaluating the similarity between objects we use attributes that are distinguishable by nature. The shape of objects is one of those attributes and it differs from object to object with greater or lesser degree of similarity. For a numerical representation of the shape of an object we use shape descriptors such as the Fourier descriptors. If the choice of prototypes of a particular subject is appropriate it is expected that a sample of other similar objects to the prototype corresponds to a set of points in the vicinity of that prototype, in a characteristics space of n dimensions. Also, if the features were well chosen it is expected that the groups of points of each class are reasonably separated allowing a better distinction between classes. A classification system is used to interpret a set of characteristics of a given object and assign a class to the object. A fuzzy classifier is a classification system based on the fuzzy sets rules and consists of three main components: database, rule base and a method of reasoning. The database describes the semantics of fuzzy sets associated with linguistic labels. The rule base has the if-then rules where each rule specifies a subspace in the universe of discourse. The method of reasoning provides the mechanism to classify a pattern using the information from the database and rule base. To classify a given input pattern described by a vector X, it is then calculated its degree of compatibility with each rule of the system. An if-then rule has the following form: 3

4 If x is A then y is B Where x is A the antecedent and y is B the consequent. A and B are fuzzy sets defined in the universe of discourse X and Y respectively. Usually only one rule is not enough for the system to be efficient. So it is necessary to use two or more rules that have, for each, an output. These fuzzy sets are then aggregated to produce a single output. To join the rules is used a fuzzy inference system (FIS) [JSM1997]. There are two main models of FIS, the Mandan and Takagi - Sugeno (TS). These models differ in how they calculate the outputs. Mamdini outputs a fuzzy set that need to be defuzzufied to obtain a crisp solution and TS outputs an already crisp solution. This study used the TS model. A rule of the TS model is given by IF x 1 is A 1i and x 2 is A 2i and and x n is A ni then y i = a i T x j + b i The output of the model given by Y = N i=1 ω i y i ω i (3. 1) The aggregation of the antecedents (μ 1 (x 1 ), μ 2 (x 2 ),, μ n (x n )) can be made through the Cartesian product of the outcome of the membership functions of a particular rule: ω i = μ i1 x 1 μ i2 x 2 μ in x n Or through the minimum value of the membership functions belonging to a given rule. ω i = min μ i1 x 1, μ in x 2,, μ in x n The TS model uses the idea of linearization of a fuzzy region defined in space. The fuzzy regions are parameterized and each region is associated with a linear subsystem. Due to antecedents fuzzily defined, the non-linear systems are a collection of multiple linear models [AF2004]. The identification of the TS model is divided into two parts 1. Learning the antecedents, this consists in determining the parameters of the membership functions. 2. Determination of parameters of the consequents a 1, a 2,, a n Learning the parameters of the antecedents can be solved by using heuristic approaches such as methods of fuzzy clustering. With the antecedent parameters fixed, the estimation of the consequent parameters, that in the TS model are linear equations, can be transformed into a problem of least square error. In the design of a classifier, data is usually separated into a set for training and a set for test. Test a classifier means to estimate the probability of the classifier to not classify correctly. The normal method for testing a classifier is to submit the test set and count the errors of classification. This yields the apparent error rate. [BKKP2005] There are others representations for the error. if the test data is the same as training data then the error is called resubstitution 4

5 error rate. A third error rate that is sometimes used is called the validation error. This idea springs from the increasingly frequent practice of using test sets to decide when the classifier is "well trained", by repeatedly computing the error while varying the parameters of the classifier and/or the training set [BKKP2005]. 4 Smart Cameras A smart camera is an integrated vision system that, in addition to the capture of images, includes a processor, which extracts information from images and performs procedures without need of an external processing unit, and uses interfaces to provide the results to other equipment. The camera used in this work was a VCM50 manufactured by Vision Components. This camera has the ability to do processing and communicate with a PLC. It is a lightweight and compact camera. This camera was designed for industrial applications and is insensitive to vibrations and shocks. This camera contains an operating system, VC/RT, which controls all basic functions, providing a command line for easy access to resources. This command line is accessible only via a PC. The camera VCM50 has no direct output of video. To show an image this one it is converted into JPEG format and then transferred to a device capable of displaying JPEG files. Communication between a PC and the camera is done via the supplied software, using the serial interface, such as the development of applications. The camera has a progressive scan CCD sensor, in gray scale, with 640x480 pixels. It is housed in a 30 mm in diameter a tube with integrated lenses, LEDs for illumination and six indication LEDs. It has 8 MB of dynamic memory and 2 MB of flash memory for storing programs and data. 4.1 Process The process implemented in the camera is outlined in Figure 4.1. The binarization is done using the functions available in the provided libraries. In the normalization of the border were used 128 points. Image Aquisation Binarization Contour Extraction Fourier Tranform Contour Normalization Classification The Classification algorithm is as follows Comunicate Result Figure 4.1 Process 1. Normalization of descriptors: The descriptors are normalized to be between zero and one 2. Calculation of the membership function associated to each input 5

6 3. Aggregation of the antecedents using the minimum value obtained in the calculation of the membership function 4. Calculation of the Takagi-Sugeno affine function 5. Repeat the procedures 2, 3 and 4 for each rule 6. Obtaining the output using the equation (3.1). 5 Results To obtain the experimental results, objects in figure 5.1 were photographed in twenty different positions. Then those images were binarized, and the contour was extracted and normalized. The Fourier descriptors were calculated and made invariant to rotation, translation, scale and to the starting point. The development of the TS fuzzy model consists of three parts 1. Selection of input-output variables. 2. Selection of the structure and estimation of parameters. 3. Validation of the model obtained. The obtained model has the descriptors obtained by Fourier transform for inputs and outputs the class to which a given object belongs. The biggest concern in choosing the input vector was how many of those descriptors would be needed for the TS model obtained to be able to distinguish objects. Class 1 Class 2 Class 3 Class 4 Class 5 Figure 5.1 Objects The identification of the model and implementation of algorithms for clustering was performed using the fuzzy logic toolbox of Matlab. The models were trained and validated using the contours of twenty images of each object in different positions. Of these twenty contours 128 points in each were taken to calculate the Fourier transform. Of the 126 descriptors obtained after the normalization of the Fourier transform only two were needed to obtain a model capable 6

7 Variançia Variance of identifying objects. With this model built it was implemented in C to be incorporated in the camera VCM50. For the selection of points to consider in the normalization of the contour described in section 2, the variances for two descriptors were calculated. For this comparison the coefficients -1 and -2 were used because, for four points, the coefficients 0 and 1 are equal to 0 and 1 respectively, due to obtaining the invariance explained in section 2. Figure 5.2 and Figure 5.3 show the results. In Figure 5.3, the variance of the descriptor -1 only begins to be approximately constant from 64 points but only for 128 points it "stabilizes". Thus the number of points to use in this work is ,12 Variance of Coeficient -2 0,1 0,08 0,06 0,04 0, Classe1 Classe 2 Classe 3 Classe 4 Classe 5 Points Figure 5.2 Variance of coeficient -2 Variance of Coeficient Classe1 3 Classe 2 2 Classe 3 1 Classe 4 0 Classe Pontos Figure 5.3 Variance of coefficient -1 For the selection of coefficients to use in the training of the intelligent system were used those that showed greater variance, because they have a higher probability that their midpoints are distant from each other. In Table 5.3 we can see that the most variant coefficients are -1 and 2. 7

8 Coefficient 2 Although the variance does not always ensure that items are properly separated was used as a starting point. Coefficients Class , , ,2838 2, , , , , , , , , , , , , , , , , , , , , , , , , , , Variance 0, , , , , , Table 5.1 Variance of the average of the coefficients for 128 points of contour In Figure 5.4 is shown the distribution coefficients of -1 and 2 of the Fourier transform. The groups for each class are clearly separated, by using only two coefficients. So the classification system only need two coefficients, even if when applying the inverse transform the original form is not rebuilt. 3 2,5 2 1,5 1 0,5 0 2D representation of the coefficients (128 Points) Coefficient -1 Classe 1 Classe 2 Classe 3 Classe 4 Classe 5 Figure 5.4 2D representation of the coefficients The data was divided into training data (75%) and validation data (25%). One hundred procedures were performed to obtain a classifier with different training and validation sets. The results of the evaluation for the classification systems for different numbers of contour points are shown in Table 5.2. We measured the maximum error and minimum error obtained in the sample of one hundred classifiers. Was also counted the overall average error obtained. In the last column is the number of classifiers that achieved the minimum error. 8

9 Number of Points Minimum error (%) Maximum error (%) Average Error % Minimum error ,33 0, All Table 5.2 Result of fuzzy classification for different points, using the coefficients -1 and 2 For the Classification was only necessary five clusters, the same number as the existing classes, because the data are well separated, indicating that the choice of descriptors to use was the right one. Comparing with Table 5.3 that are the results of a feed-forward back propagation neural network with two layers with four neurons each, the fuzzy system has a better performance. Number of Points Minimum error (%) Maximum error(%) Average Error % Minimum error , , , , , , ,4 83 All ,56 86 Table 5.3 Result of Neural Network classification for different points, using the coefficients -1 and 2 In tests made with the camera, the execution time of the implemented system varied between two seconds and five seconds, depending on the size of the contour. This time may be reduced by applying an algorithm for the extraction of contours that is optimized for this camera. The performance of the classification system was quite satisfactory because the objects were correctly identified in various positions and at all times they were subjected to the test. 6 Conclusion In this work it was demonstrated the VCM50 camera is easy to program and operate. Without loss of performance it was possible to implement procedures for classification based on intelligent systems and in objects contour described by Fourier descriptors. The biggest loss in performance came from image processing, which varies with the size of the contour of the object. Note that the processing of image was not optimized to reduce the time of calculation. The intelligent system based on fuzzy sets has proven highly effective with only two Fourier descriptors, even when these were calculated using few points of the object s contour. Probably this performance is due to the fact that the objects are sufficiently different and so, easy to obtain properly separated clusters. These intelligent systems have been quick to train and are simple to implement. However the situation can get complicated when the number of rules increases, because it is necessary to introduce all the necessary parameters. In general the solution presented for an object recognition system using a smart camera had a good performance and can be used in industrial environments without much loss of performance. 9

10 As future developments it is proposed to optimize the process of image processing to reduce the time spent by the camera to classify an object, the study of other neural network architectures, and other intelligent systems, to provide a comparison with the work here presented and a study on the influence that each coefficient of the Fourier transform will have in decision-making. References [AF2004] Angelov, P. P. e Filev, P. D. (2004), An Approach to Online Identification of Takagi- Sugeno Fuzzy Models, IEEE Transactions on Systems, Man and Cybernetics Part B: Cybernetics, VOL. 34, NO. 1, pp [Bishop1995] Bishop, C. M. (1995), Neural Networks for Pattern Recognition, Oxford University Press. Oxford. [BKKP2005] Bezdek, J. C., Keller, J., Krinapuram, R. e Pal, N. R. (2005) Fuzzy Models and Algoritms for Patern Recognition and Image Processing, Springer, USA. [CKZ1994] M. Chen, A. Kundu, e J. Zhou. (1994), Off-line handwritten word recognition using a hidden Markov model type stochastic network. IEEE Trans. Pattern Anal. Mach. Intel., vol. 16, pp [Cohen1994] Cohen, E. (1994), Computational theory for interpreting handwritten text in constrained domains, Artif. Intell., vol. 67, pp [FC1995] Fadzil, M.H.A. e Choon, L. C. (1995), Fourier descriptors and neural networks far shape classification, International Conference on Acoustics, Speech, and Signal Processing, ICASSP-95. vol.5, pp [JSM1997] Jang, J. S., Sun, C. T. e Mizutani, E. (1997), Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. Prentice Hall, New Jersey, [KP2004] Kwak, K-C e Pedrycz, W. (2004), Face recognition using fuzzy Integral and wavelet decomposition method, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 34-4, pp [MWA1995] McElroy, T., Wilson, E. e Anspach, G. (1995), Fourier Descriptors and Neural Networks for Shape Classification, International Conference on Acoustics, Speech, and Signal Processing. ICASSP-95. [Otterloo1991] Otterloo, Peter J. van. (1991), A contour-oriented Approach to Shape Analysis. Prentice Hall International (UK) Ltd. [ZL2002] Zhang, D. e Lu, G.(2002), A Comparative Study on Shape Retrieval Using Fourier Descriptors with Different Shape Signatures, In Proceedings of the Sixth Digital Image Computing, Techniques and Applications (DICTA02), Melbourne, Australia. Pp [ZL2003] Zhang D e Lu G. (2003), A comparative study of curvature scale space and Fourier descriptors for shape-based image retrieval, Journal of Visual Communication & Image Representation 14: