Vehicular shape-based objects classification using Fourier descriptor technique

Transcription

1 Journal of Scientific & Industrial Research 484 Vol. 68, June 2009, pp J SCI IND RES VOL 68 JUNE 2009 Vehicular shape-based objects classification using Fourier descriptor technique Raj Bahadur Yadav 1, Naveen K Nishchal 2, Arun K Gupta 3 * and Vinod K Rastogi 4 1 Department of Information & Communication Engineering, The University of Electro-communications, Tokyo , Japan 2 Department of physics, IIT Guwahati, Guwahati , India 3 Photonics Division, Instrument Research and Development Establishment, Dehradun , India 4 Department of Physics, C C S University, Meerut , India Received 22 January 2008; revised 10 July 2008; accepted 24 February 2009 This paper reports classification of vehicular shape-based objects using Fourier descriptor (FD) technique. Retrieval and classification of different types of shapes was done for closest match, comparing set of feature vectors of query object to each object of every class. Centroid distance based shape signatures were used for derivation of feature vectors. Euclidean distance was calculated as a similarity measure parameter for shape classification. Classification of noisy objects was carried out using FD technique. FD performs better than wavelet descriptor (WD) technique. Keywords: Euclidean distance, Fourier descriptors, Image retrieval, Shape representation, Wavelet descriptors Introduction Techniques 1-20 exist for shape descriptions in image processing and pattern recognition. Most common boundary based shape descriptors are chain codes 1, moments 2, wavelet descriptor 3 (WD), curvature scale space 4, and Fourier descriptor 5-7 (FD). Technique FD finds applications in contour coding 8, invariant 2-D shape recognition 9, classification of chromosomes 9, aircraft identification 10, and scene analysis etc. A method 11 using FD for analysis and synthesis of plane closed curves has been developed. FD technique is reported 12,13 to give best results. An improved technique, generic FD 14, overcomes drawbacks of existing shape representation techniques. Mathematical techniques have also been applied in different applications as template matching, character recognition, product cross correlation and Kalman filtering in navigation, and missile guidance 15. Belongie et al 16 discussed another approach for measuring similarity between shapes and used it for silhouettes, trademarks, handwritten digits, and COIL data set. Lowe 17 presented a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. Sebastian et al 18 reported a *Author for correspondence Tel: ; Fax: akgupta@irde.res.in framework for recognition of objects based on their silhouettes. Wavelet transform has been widely used in image processing and pattern classification but only few applications in shape description. Yang et al 19 reported that obtained WD are not rotation invariant and also matching scheme is complicated and time consuming than that of FD. This paper extends study of Wimmer et al 6, who proposed FD technique for distorted tank shape scale, translation and rotation, considering a larger database of different types of military vehicles and using distance shape signature to find out feature vectors and Euclidean distance (ED) for similarity measurement. Classification results using FD technique were also compared with WD technique. Materials and Methods Six types of vehicular shapes (aircrafts, helicopters, missiles, tanks, military trucks and cars) have been used for study. FD s were obtained by applying Fourier transform to the boundary of shape of objects. After approximating shapes of objects by boundaries, a set of features was computed for each object in the form of a set of FD s. Only 20 FD s, all invariants against scaling, translation and rotation for shape retrieval, were used. Image Preprocessing and Shape Retrieval Shapes were downloaded from Internet and converted into gray level images for further processing 7 using

2 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION 485 (i) (i) (ii) (ii) (iii) (iii) (iv) (iv) (v) (vi) (i) (ii) Fig. 1 a) Image preprocessing i) Original object, ii) Binary gradient mask, iii) Dilated gradient mask, iv) Cleared border image, v) Segmented image, and vi) Boundary edge of original object; b) Shape retrieval i) Binarised shape, and ii) Retrieved shape Photoshop software. Images were preprocessed to extract boundary information or coordinates of boundary from the shape. Using a proper threshold, input grayscale image converted into a binary image, which has values of 0 (black) for all pixels in input image with luminance less than threshold level and 1 (white) for all other pixels (Fig. 1). Fig. 1a(i) shows shape of aircraft image converting color into grey level. After binarizing shapes, edge takes an intensity image as its input, and returns a binary image of same size as I, with 1 s where the function finds edges and 0 s elsewhere. The sobel method was applied to find edges of binarised shapes. It returns edges at those points where gradient of intensity of image is maximum

3 486 J SCI IND RES VOL 68 JUNE 2009 [Fig. 1a (ii)]. The imdilate function accepts two primary arguments: i) Input image to be processed is a structuring element object, returned by strel function; and ii) Depending on shape, strel can take additional parameters as rectangle, square, periodic line etc, which create morphological structuring element or a binary matrix defining neighborhood of a structuring element [Fig. 1a (iii)]. Filling holes within inside region of shape clear out the edge of object [Fig. 1a(iv)] and segmented shape [Fig. 1a (v)]. After that, MATLAB inbuilt zero crossing edge function was applied to find boundary function [Fig.1a (vi)]. Binarised shape of aircraft [Fig. 1b (i)] was used and Fourier transform was applied on boundary function. Only 20 normalized FD s were selected to retrieve the shape of object [Fig. 1b (ii)]. Shape Signatures Shape signature is a 1-D function representing 2-D boundaries of any object. Generally, boundaries are in complex form. Therefore, to represent the position function, two shape signatures complex coordinates and centroid distance are considered for test and comparison. These shape signatures are used in FD implementation for classification of given database. Complex Coordinates A complex coordinate function is simply the complex number generated from boundary coordinates (BCs). Assume (x(t), y(t)), t = 0,1,, L -1) be shape BCs of the object, complex coordinate z(t) of shape boundary thus becomes (1) For removing effect of the bias, shifted coordinate function was used as z(t) = [x(t) -- x c ] + j[y(t) - y c ] (2) where (x c, y c ) is centroid of shape, which is average of BCs. x 1 c = 1/L Σ 1 (t = 0) t (L 1), x(t) y 1 c = 1/L Σ 1 (t = 0) t (L 1), y(t) (3) This introduced shift makes the shape representation invariant to translation. Centroid Distance Centroid distance [ r(t)] function is expressed by the distance of boundary points from centroid (x c, y c ) of shape contour as r(t = {([x(t) x c ] 2 + [y(t) y c ] 2 )]} ½ (4) r(t) also represents position of shape from BCs and becomes invariant to translation. Fourier Descriptor (FD) One-dimensional FD has simple derivation and normalization, simple to do matching, and robust to noise etc 14. Consider N contour points of an image component as a discrete function (5) Using this function, a complex function z(t) is defined as (6) can be transformed into frequency domain using discrete Fourier transformation (DFT). Without any loss, result can be transformed back into spatial domain with inverse DFT (IDFT). The DFT and IDFT of defined as where where are (7) (8) Coefficients Z(k), called FD, represent discrete contour of a shape in Fourier domain. Geometric transformations of z(t) are related to simple operations in Fourier domain, e.g., translation by z 0 C (C is an integer) only affects first FD Z(0), while other FD s remain the same. Scaling of contour with a factor α leads to scaling of FD by α. Rotating contour by an angle θ 0 yields a constant phase shift of θ 0 in FD. A change of starting point has the effect on phase information of descriptors. Therefore, omitting phase information of FD s results in starting point

4 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION 487 Fig. 2 Shape reconstruction (5, 10, 30 coefficients) independent description of original shape. A rotation by an angle φ of object s outline in spatial domain has effect on phase information of FD. To achieve rotation invariance, only magnitude of Z(k) are considered for recognition process. Scaling of outline causes a multiplication of FD Z(k) by a constant factor c. Descriptors are normalized by descriptors Z(2),, Z(N-1) by magnitude of Z(1). Translation of contour by dx and dy results in change of FD s Z(0). All other descriptors are not influenced by translation in spatial domain. Therefore, translation invariance can be achieved by omitting descriptors Z(0). Shape translation can be expressed as (9) (10) Thus, FD s are invariant to translation except first coefficient, which only reflects the position or average scale of shape and is not useful in shape description. Key Descriptors Generally, FD s of shape contour represents the object in frequency domain. There are two (low and high) types of frequency descriptors (coefficients). Low frequency descriptors have details of general features while high frequency descriptors contain finer details of the object. When Fourier transform is applied on an image, a large numbers of frequency components (FCs) are obtained. Out of several FCs, some are significant for image retrieval, which are also known as contour representative. To acquire knowledge of the number of FCs necessary for shape retrieval, 5, 10 and 30 FD s respectively were used to retrieve object shape, and infer that 20 FD s are sufficient for shape retrieval and classification. Finally, selected 20 normalized descriptors form a feature vector, which are used for shape indexing. Three pictures (Fig. 2) show the same shape, but it is approximated by an increasing number of frequencies. Fourier Descriptors for Noisy Shapes To study the effect of noise on shapes to be classified, additive and multiplicative noise was applied to current database. Consider shape function composed of complex function z(t) and noise function. Shape function can be written as...(11) where is zero mean, uncorrelated random error with a finite variance σ 2. Expression for FCs of noisy shape can be written as

5 488 J SCI IND RES VOL 68 JUNE 2009 (12) Image Reconstruction from Noisy Data Shape was computed and retrieved with FCs from noisy data similar to reconstruction of shape without noise. For retrieval of noisy shapes, 20 normalized FD s were used. Error between original shape and its reconstructed image was measured using FD technique 20 as (13) Wavelet Descriptor (WD) Wavelet-based methods for pattern matching and recognition are generally based on feature extraction in different scales and subbands 8. This technique reduces an original image into sub-images at a low multiresolution level and also transform into both spatial and frequency information. In this paper, wavelet decomposition (Meyer wavelet) approach was used for pattern matching. Wavelet transform of an image z(x,y) is defined as (14) Meyer wavelet and its representation is defined as In spatial domain, In frequency domain,...(15) (16) Classification Experiment Classification results were obtained by comparing FD as feature vector using ED. For analysis, 6 types of image datasets containing images of vehicular objects were used. Similarity between features of two images was computed by ED, which is defined as where n is FD number, I I I I and F = f, f,..., f ) ( 1 2 n (17) Q Q Q Q and F = ( f1, f 2,..., f n ) are FD for a database image and query image, respectively. Randomly, one image was selected from each class in present database and used as a query image. ED has been calculated for query object in each class with respect to whole database. Basically, content-based image retrieval utilizes lowlevel image features such as color, texture and shape. Shape is one of the primary low-level image features to human perception and an essential classifying feature. Six types of target images (30 images per set) used for testing were: aircrafts (Fig. 3a); helicopters (Fig. 3b); missiles (Fig. 3c); tank targets (Fig. 3d); military trucks (Fig. 3e); and cars (Fig. 3f). Each class has standard set of FD s. Within same class, there is no difference in size of the images in whole database. Using 6 databases, FD could be used in shape-based classification of several different kinds of images. Results and Discussion Application of FD and WD on 6 Different classes The created dataset contains 180 shapes of 6 different classes (1-30 aircrafts, helicopters, missiles, tanks, trucks, and cars). Interclass discrimination is the ability of shape descriptors to differentiate shapes belonging to different classes. Either an average shape representing each class is first derived or shape descriptors of each shape of every class is derived and stored in the database. This is then used for finding out maximum shape variation within each class (intra-class variation). Maximum of these values is chosen as marginal threshold for decision-making. First Set of Testing Datasets (Aircrafts) Of 30 aircraft images (Fig. 3a), 26 have similar shapes. Feature vectors of 1 st object of this class were taken as query object and compared with all objects of each class (Fig. 4a). With computed values of ED (0.055), all shapes belong to aircraft class. Hence, this value was taken as threshold. Objects with ED greater than this do not classify. There are very small variations in ED values within class. WD was applied to this dataset and ED was computed. It was found that below the value of 0.008, all shapes belong to aircraft class. Objects with ED values greater than chosen threshold do not classify. Variation of ED in comparison to WD is less and not enough for shape classification [Fig. 4]. There is very small variation in ED values within class.

6 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION (d) 2 23 (e) (c) (f) Fig. 3 Image database: a) Aircrafts; b) Helicopters; c) Missiles; d) Tanks; e) Trucks; and f) Cars Second Set of Testing Datasets (Helicopters) Of 30 helicopter images (Fig. 3b), 25 have similar shapes. Feature vectors of 2 nd object of this class were chosen as query object and compared with all objects in this class (Fig. 4b). With computed values of ED (0.07), all shapes belong to helicopter class. Hence, this value was chosen as threshold. Objects with ED greater than this threshold do not classify. There are very small variations in ED values within class. After applying WD to this dataset, ED was computed and it was found that below the value of all shapes belong to helicopter class. Objects with ED values greater than chosen threshold do not classify. Variation of ED in comparison to WD is less, which is not enough for shape classification. Other Set of Testing Datasets Similarly as in case of aircrafts and helicopters, ED values were computed employing FD and WD techniques for missiles (Fig. 4c), tanks (Fig. 4d), trucks (Fig. 4e) and cars (Fig. 4f). It was noted that with one typical computed value of ED, all shapes belonged to one class. Hence, that particular value was taken as threshold, which was found different for different class of shapes. Also, obtained values of ED were different for different techniques. Objects with ED values greater than threshold did not classify. For different objects, ED values using FD and WD techniques respectively, were found as follows: aircrafts, 0.055, 0.008; helicopters, 0.07, 0.004; missiles, 0.03, 0.005; tanks, 0.02, 0.004; trucks, 0.032, 0.003; and cars, 0.022, It has been observed that there are very small variations in ED within class. Very high values of ED are observed in case of FD technique, which help shapes classify better. In all datasets, very small variations in ED were observed within class. In the database, query object was chosen arbitrarily. Two objects, one from helicopter (Fig. 5a) and another one from car (Fig. 5c) were found not classified. Helicopter is not matching from any class, may be due to having two wings instead of one (Fig. 5b). Car confused between car and truck, because it has some shape features like a car and some features like truck (Fig. 5d). Very small variation was observed in values of ED within class and large variation in between class for FD than WD. Effect of Noise To study effect of noise on shape retrieval and classification, additive and multiplicative noise of various magnitudes were applied in the database. Imnoise command of MATLAB was used to add Gaussian white noise of mean M and variance V to all the images. Fig. 6a shows images with additive Gaussian noise of mean 0 and variance 0.6 applied to the aircraft database. After removing salt and pepper noise (Fig. 6a), obtained

7 490 J SCI IND RES VOL 68 JUNE 2009 (c) Fig. 4 Classification of whole database with respect to query object: a) Aircrafts; b) Helicopters; c) Missiles

8 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION 491 (d) (e) (f) Fig. 4a Classification of whole database with respect to query object: d) Tanks; e) Trucks; and f) Cars

9 492 J SCI IND RES VOL 68 JUNE 2009 (c) (d) Fig. 5 Misclassification of shapes: a) Image of a helicopter; b) Plot of values of ED of whole database with respect to a); c) Image of a car; and (d) Plot of values of ED of whole database with respect to c) images are shown in Fig. 6b. Thereafter, boundary of the shape was obtained and Fourier transform to boundary function of noisy shape was applied. Then, 20 Fourier coefficients were selected and normalized. Using these coefficients, shape of object was retrieved. Experiment was repeated using constant value of mean 0 in each case and varied values of variance ( ). After iterating experiment with different values of variance, retrieval of shape was possible with variance 0.6. Using higher values of variance, shape of object was not retrieved and hence shapes were not classified, may be due to some shapes lose more boundary information than other shapes. Similarly, some other types of noise were applied on shapes of different categories. Fig. 7a shows addition of

10 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION 493 Fig. 6 Noise affected shapes and their removal (additive noise): a) Noise; and b) Removal ) ) Fig. 7 Noise affected shapes and their removal (multiplicative noise): a) Noise; and b) Removal ) Fig. 8 Classification of noise affected shapes with respect to: a) Tank (Gaussian noise); and b) missile (speckle noise)

11 494 J SCI IND RES VOL 68 JUNE 2009 Fig. 9 Performance of FD technique Magnitude of error Number of class Fig. 10 Error chart of FD & WD technique multiplicative noise (speckle noise) of constant value of mean 0 and different values of variance up to 0.3 applied to aircraft database. Fig. 7b shows removal of salt and pepper noise from Fig. 7a. Thereafter, boundary of shape was obtained and Fourier transform to boundary function of the noisy shape was applied and normalized 20 FD was selected for shape retrieval and classification. After iterating the experiment with different values of variance ( ), retrieval of shape was found possible with variance 0.3. When higher values of variance were used, shape of object could not be retrieved and hence shapes were not classified. To study the effect of noise (Gaussian and speckle) on the dataset of military vehicles using FD and WD techniques, experiment was repeated using constant value of mean 0 in each case and different values of variance. After iterating the experiment with different values of variance, it was observed that retrieval of shape is possible with variance 0.6. Using higher values of variance than 0.06, shape of object was not retrieved and hence shapes were not classified. Results for Gaussian noise (variance = 0.6) with respect to tank class (Fig. 8a) showed that there are very small variations in ED intra-class and large variation in inter-class. Similarly, results for the classification of speckle noise affected dataset with respect to missile images have been shown in Fig. 8b. With variance 0.4, FD technique showed small variations in intra-class and large variations in inter-class than WD. Using higher variance values than 0.4, shape of object was not retrieved

12 YADAV et al: VEHICULAR SHAPE-BASED OBJECTS CLASSIFICATION 495 and hence shapes were not classified. Hence, FD performs better than WD for noisy dataset of military vehicles. Performance Measurements To measure classification performance, precision and recall were calculated as evaluation measure of query results 20. Precision P is the ratio of number of retrieved relevant shapes r to the total number of retrieved shapes n, P = r/n. Recall R is the ratio of number of retrieved relevant shapes r to the total number m of relevant shapes in whole database R = r/m. Physical significance of precision is measurement of accuracy and recall is measurement of robustness of retrieval. Performance as precision and recall has been shown when additive and multiplicative noises are added to the database using FD technique (Fig. 9). Classification performances were >90% in all cases (aircrafts, helicopters, missiles, tanks, military trucks, and cars). In helicopter class, slight degradation of performance is attributed to inside variation in orientation and double wings. High performance obtained in each class validates FD s technique for classification of military vehicles used in present database. Precision and recall were also computed using FD and WD techniques. Classification error (Fig. 10) as an important performance measure for FD has been observed lower in tank class, while higher in car class. Conclusions FD has been found an effective tool for classifying 6 classes of different vehicular objects (aircraft, helicopter, missile, tank and military truck and car). One object from each class, chosen as query shape, was matched with all objects from whole database. When object similarity was above 90%, then class of the object was well recognized. ED was used for similarity measure. A little variation was observed in degree of match within class. FD method retrieved and classified Gaussian noise affected shape up to variance 0.6, while speckle noise case could be up to 0.3. Performance of FD technique is >90% in every class. FD technique is less computationally intensive and gives satisfactory performance. Variation of ED in WD was observed very less, which is not significant for shape classification. FD s are found better performing than WD s for classification of database of vehicular objects. The influence is also applicable to noisy database. Acknowledgments Authors thank Director, IRDE, Dehradun, for encouragement and permission to publish this study. Authors also thank referees and editors for suggestions to improve the manuscript. References 1 Freeman H, Shape description via the use of critical points, Pat Recogn, 10 (1978) Liao S X & Pawlak M, On image analysis by moments, IEEE Trans Pat Anal Mach Intel, 18 (1996) Chang G C-H & Kuo C-C J, Wavelet descriptor of planer curves: theory and applications, IEEE Trans Image Process, 5 (1996) Mokhtarian F & Mackworth A K, A theory of multiscale, curvature based shape representation for planer curves, IEEE Trans Pat Anal Mach Intel, 14 (1992) Dubois S R & Glanz F H, An autoregressive model approach to two-dimensional shape classification, IEEE Trans Pat Anal Mach Intel, 8 (1986) Wimmer A, Ruppert G S, Sidla O, Konard H & Gretzmacher F, FFT-descriptors for shape recognition of military vehicles, Proc SPIE, 40 (2000) Costa L F & Cesar R M, Shape Analysis and Classification, Theory and Practice (RC Press, Boca Raton, Florida) Granlund G H, Fourier preprocessing for hand printed character recognition, IEEE Trans Comput, 21 (1972) Persoon E & Fu K, Shape discrimination using Fourier descriptors, IEEE Trans Syst Man Cyber, 7 (1977) Dudani S A, Breeding K J & McGhee R B, Aircraft identification by moments invariants, IEEE Trans Comput, 26 (1977) Zahn T & Roskies R Z, Fourier descriptors for plane closed curves, IEEE Trans Comput, 21 (1972) Kauppinen H, Seppänen T & Pietikäinen M, An experimental comparison of autoregressive and Fourier-based descriptors in 2D shape classification, IEEE Trans Pat Anal Mach Intel, 17 (1995) Mehtre B M, Kankanhalli M S & Lee W F, Shape measures for content-based image retrieval: a comparison, in, Proc Manage, 33 (1997) Zhang D & Lu G, Shape-based image retrieval using generic Fourier descriptor, Signal Process Image Commun, 17 (2002) Golden J P, Terrain contour matching (TECOM): A cruise missile guidance aid, Proc SPIE, 238 (1980) Belongie S, Malik J & Puzicha J, Shape matching and object recognition using shape contexts, IEEE Trans Pat Anal Mach Intel, 24 (2002) Lowe D G, Distinctive image features from scale-invariant keypoints, Int J Comput Vision, 60 (2004) Sebastian T B, Klein P N & Kimia B B, Recognition of shapes by editing their shock graphs, IEEE Trans Pat Anal Mach Intel, 26 (2004) Yang H S, Lee S U & Lee, K M Recognition of 2-D object contours using starting-point independent wavelet coefficient matching, Vis Comm Image R, 9 (1998) Yadav R B, Nishchal N K, Gupta A K & Rastogi V K, Retrieval and classification of shape-based objects using Fourier, generic Fourier, and wavelet-fourier descriptors technique; A comparative study, Opt Laser Engg, 45 (2007)