Particle Swarm Optimization with Fuzzy Adaptive Acceleration for Human Object Detection

Transcription

1 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 0 Particle Swarm Optimization with Fuzzy Adaptive Acceleration for Human Object Detection Abstract In this paper a new approach to dynamically adapt the acceleration coefficient of particle swarm optimization (PSO) is discussed. The proposed method uses fuzzy inference system to lead the particles movement in exploring and exploiting the search area; therefore it increases the accuracy and reduces the detection time of human object detection system. The performance of the proposed method was tested on real images artificial images as well as real-time videos; the result is then compared to that of conventional method. Experiment on testing data using the proposed method improves the accuracy rate 9% better and almost twice faster than the standard window scanning method. The proposed PSO with fuzzy adaptive acceleration gives a promising contribution to solve real-world problems where computational time is critical. Index Term Fuzzy inference system human object detection particle swarm optimization. I. INTRODUCTION Human object in images sometimes need to be detected and localized for some purposes i.e. in surveillance application [ 2] automatic driver assistance [3] and human interaction for mobile robotics [2]. The problem in human detection comes in two folds firstly to determine the human object in image that may consist of many objects i.e. human object and other objects. Secondly A. Hidayat is with the University of Brawijaya Indonesia Jl. Veteran Malang 6545; arifhidayat@ ub.ac.id C. Wilson is with the Monash University Australia 900 Dandenong Road Caulfield East Victoria 345Australia Telp Fax: ; campbell.wilson@monash.edu D.Y. Liliana is with the Department of Computer Science Faculty of Mathematics and Natural Sciences University of Brawijaya Malang East Java Indonesia ( dewi.liliana@ub.ac.id) M.R. Widyanto is with the Faculty of Computer Science University of Indonesia Depok West Java Indonesia ( widyanto@cs.ui.ac.id ) Dewi Yanti Liliana M. Rahmat Widyanto rapid object detection is needed to improve the performance especially on a real-time system. The first problem is solved by training the system using supervised learning methods such as SVM [ ] and Boosting [8 9]. The second problem in the existing researches is conventionally solved by using window scanning techniques [ ] this is somehow timeconsumed. The human object detection using window scanning techniques is done in an image by scanning entire pixels starting from left-top pixel of image shifted to the right and so forth until reaching the right-bottom pixel of image. In each pixel position the window is evaluated to check whether it contains human object or not. This is called a brute-force search [] and it consumes more computational time to detect the existence of human object all over image area. To shorten the computational time the detection process can be optimized using particle swarm optimization (PSO) [0]. PSO uses a group of particles which spread on the image and do the detection by optimizing the objective function. As an optimization algorithm PSO has been successfully applied to fasten human object detection in images []. The weakness reported on the implemented PSO for human object detection is the low accuracy due to the fix-adjusted acceleration coefficient value of PSO. This fix-adjusted method makes the entire particles have the same acceleration coefficient values instead of adaptive values for each particle; therefore it cannot adaptively lead particles to better exploit and explore the search area. To improve the accuracy of human object detection fuzzy inference system (FIS) is proposed to estimate the acceleration coefficient of PSO adaptively. This adaptive adjustment of acceleration coefficient leads the particle movement and balances the ability of particles to exploit and explore the search area therefore increasing the accuracy of the human object detection. The rest of the paper is organized as follows. Section 2 will discuss the human object detection system section 3 will discuss PSO for human object detection section 4 will

2 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 discuss the weakness of PSO for human object detection section 5 will discuss PSO with FIS for human object detection and section 6 will discuss the experiment and analysis of system performance. II. HUMAN OBJECT DETECTION SYSTEM Human object detection system architecture developed in this study is depicted in Figure. The system is divided into two phases training phase and detection phase. Training Phase Training images Detection Phase Testing image 2.. HoG Feature Extraction 2.2. Adaboost 2.. HoG Feature 2.3. Human Object Detection Fig.. Architecture of human object detection system In training phase the system is trained using positive and negative training data which are images that contain human object and non human object respectively. All images either in the training or detection phase are extracted to get the image features to be the input of the next process. Previous human object detection systems used image feature extraction method i.e. Haar wavelet feature [6 8] and Histogram of Oriented Gradient (HoG) feature [7 8 2]. This study uses HoG Feature extraction method [8]. HoG feature extraction result is used for training the classifier to classify either human or non-human object. For the classifier previous human object detection systems used the SVM [ ]; and Boosting [8 9]. This study employs adaboost classifier [9] which is a discrete version of boosting which simply resulting two classes of output either positive or negative. In the detection phase the system is ready to detect and localize human object on the testing image using fixsized window which moves all over image area. The window is checked at any position using the information obtained from the adaboost classifier which is able to classify between human or non-human object. Below is the explanation of training phase and detection phase including HoG feature extraction adaboost classification and human object detection. 2.. HoG Feature Extraction Histogram of oriented gradient (HoG) is feature descriptor that well-characterized the local object appearance and geometric shape by using local intensity gradient. HoG is firstly described in [2] to solve the pedestrian detection problem in static images. In that study HoG feature performs relative better compared to the other linear features for human object detection including wavelet. HoG feature extraction is done by computing the orientation gradient in the localized region on the image. The computation is done in the densely grid using overlapped local contrast normalization to increase the performance. For image region r [ x] HoG γ at each point ( x y) r is computed using equations below L ( ) ( ) arctan x x y x y Ly( x y) ( x y ) / 2 L I * ( e ) 2 x y 2 where Lx Lyare Gaussian derivatives defined on image I [ x] and is a scale parameter. To preserve gradient orientation within the region r the region r is subdivided into four bins m N as illustrated in Figure 2. () Fig. 2. Types of compound histogram features As seen in Figure 2 HoG is computed separately for each bin m= {..4} of the same region r. HoG of each bin is accumulated and resulting HoG feature vector which characterizes the distribution of local intensity gradient or edge orientation on the image region r.

3 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: Adaboost Classification HoG feature vectors obtained from the extraction method are used in the classification process using boosting. Boosting [3] is an effective machine learning technique that produces an accurate prediction rule by combining several weak rules. Adaptive Boosting (adaboost) also known as discrete adaboost is one of the robust boosting methods. AdaBoost uses sequence of simple weighted classifiers each forced to learn a different aspect of the data to generate a final comprehensive classifier with high probability outperforms in terms of misclassification error rate of individual classifier. The adaboost defines a strong binary classifier H T H( z) sgn( tht ( z)) (2) t where H ( z) [ ] T (... T T N) is the number of weak learner h t and t is weight. At each new round t adaboost selects a new weak learner h t that best classifies training samples with high classification error in the previous rounds. Each weak learner may explore any feature f of the data z if g( f ( z)) h( z) otherwise (3) where is some threshold value. In the context of human object detection f is defined in terms of HoG feature computed in rectangular image z and then use adaboost classifier to classify it as positive or negative feature vector [Laptev 2006]. The determination of human object is on the h(z) value. If h(z) = it contains human object otherwise if h(z) = - it does not contain human object. The output of the training phase is a file of human object classifier which used as a source of information to detect human object in the detection phase 2.3. Human object detection In the detection phase object in an image is detected using standard window scanning technique. This technique is applied in almost all object detection system [ ]. The window scanning method is illustrated in Figure 3. Fig. 3. Object Detection with Window Scanning Method As seen on Figure 3 the circled number indicates a sequence of window movement. It uses sliding detection window which moves initially from left-top of image and shifted pixel by pixel to the right (marked with number ) go back to the initial position and shifted down one pixel (marked with number 2) and moves to the right again (marked with number 3) go back to the left and so forth until it reaches the right-bottom of image (marked with number 4). Each window region is extracted to get the HoG feature vector as an input to the classifier. In each window position adaboost checks whether the window contains human object or not. The results are two classes positive if it contains human object and negative if otherwise. If the result is positive class then the system marks with rectangle the window position of human object. Fig. 4. Illustration of Particles Movement to Detect Human Object III. DISCUSSION ON PSO FOR HUMAN OBJECT DETECTION PSO is population-based searching algorithm that simulates social behavior of flocking birds [4]. PSO consists of group of particles called swarm each particle represents potential solution. Particle swarm flies in a multidimensional search space and dynamically changes it position in search space. Particle has velocity to move based on personal best position (cognitive component) and group best position (social component). These two components reflect exploitation and exploration of search space [5]. The basic equation of PSO is written bellow

4 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 3 xi ( t ) xi ( t) vi ( t ) (3) vi ( t ) vi ( t) cr ( t)[ pbesti ( t) xi ( t)] c2r2 ( t)[ gbest j( t) xi ( t)] where x j (t) and v i (t) are position vector and velocity vector at iteration t (... N) respectively and N is maximum iteration. The pbest i (t) and gbest i (t) is the best solution of i th particle and the best solution of entire swarm respectively found so far until iteration t. The cc 2 are both acceleration parameter where c is refer to cognitive component that controls the influence of pbest to the particle position and c 2 is refer to social component that controls the influence of gbest to the particle position. Random function r r 2 [0] generates random number which is uniformly normal distribution. The detection process started with particle swarm P(t) initialization. The initial particles are spread randomly on the image area. Each particle P i (t) where i= the number of particles has an x and y coordinates on the input image. Each particle will become the starting point of detection window. This window will be localized if it contains human object. PSO has been successfully applied to detect human object in arbitrary images [] this method has been proven to shorten the detection process two times faster than the conventional method. The basic idea of PSO for human object detection is to find human object in the input image using a group of particles which spreads on the image search area. Particles evaluate a specific objective function which represents object classification. Particle swarm move towards and dynamically update it position on the search space. Figure 4 illustrates particles movement to detect human object. Starting from the initial position t = (most left in Figure 4) in each iteration t = 2 5 particles move step by step to the target and at the last iteration t = 5 (most right in Figure 4) particles concentrate on the target position and successfully localize human object. The PSO detection process is explained by the following algorithm. Algorithm PSO Detection Begin t = 0; initialize P(t); While(not termination condition)do evaluate P(t) position; find pbest and gbest; find velocity v(t) ; update P(t+) position from P(t)+v(t) ; t = t + ; end while End Particles are evaluated to obtain particles value from objective function computation. The objective function used in this study is the modified function of adaboost strong classifier in eq. 2 by removing the sign mark T f ( x) t ht ( z) t (4) where f (x). The modified adaboost function is chosen to assign particle value because it informs the existence of human object in a detection window based on real discriminant value f(x) it has. If f (x) threshold then the window contains human object otherwise if f(x) < threshold then it doesn t contain human object. The greater the f(x) value the more represents human object and vice versa. IV. THE WEAKNESS OF PSO FOR HUMAN OBJECT DETECTION PSO for human object detection [] has an underlined weakness reported that PSO is unable to detect human object accurately due to the particle movement which can not reach the desired object. This study analyses that it happens since the particles are not well-guided to reach the target. It is the acceleration coefficient that supposed to guide the particles movement. Acceleration coefficient is an important parameter because c and c 2 together with random vector r and r 2 control the stochastic influence of cognitive and social components of velocity [5]. In PSO for human object detection the acceleration coefficient value is fix-adjusted and found empirically so it can not adaptively leads the particles to better exploit and explore the image area. There are several conditions of acceleration coefficient values with c = c 2 = 0 particles keep flying at their current speed until they hit a boundary of the search space. If c > 0 and c 2 = 0 all particles are independent hillclimbers and perform a local search. On the other hand if c 2 > 0 and c = 0 the entire swarm is attracted to a single point gbest. If c = c 2 particles are attracted towards the average of pbest and gbest. [5]. Figure 5 illustrates the two conditions of acceleration coefficient and its effect to the particles movement. In Figure 5.a. the particles are more likely attracted to its pbest position while in Figure 5.b. particles attracted to the swarm gbest position.

5 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 4 And the graphical figure is shown bellow. (a) c > 0 and c 2 = 0 Fig. 6. Accuracy rate of fix-adjusted acceleration coefficient of PSO (b) c 2 > 0 dan c = 0 Fig. 6. Illustration of the Acceleration coefficient effects to particle Movement Particles draw their strength from their cooperative nature and are most effective when c and c 2 are adaptive to the particle value to facilitate exploitation and exploration of the search area. Wrong initialization of c and c 2 may result in divergent or cyclic behavior [5]. The accuracy rate of fixed-adjusted acceleration coefficient of PSO for human object detection is low. It can be seen in table I For five times testing the accuracy rate are not greater than 50%. T ABLE I T HE ACCURACY RATE OF FIX-ADJUSTED ACCELERATION COEFFICIENT OF PSO Fix-adjusted Acceleration Coeff. PSO Experiment c c 2 Accuracy rate (%) % % % % % It can be seen on Figure 6 that the accuracy rate of fixadjusted acceleration of PSO for five times testing using different acceleration coefficient values are equal or less than 50%. It means that the human object detection accuracy using fix-adjusted acceleration coefficient is low. The accuracy can be improved using the proposed method to adjust the acceleration coefficient using fuzzy inference system. V. T HE PROPOSED FUZZY ADAPTIVE ACCELERATION OF PSO FOR HUMAN OBJECT DETECTION This study discusses the new proposed method of PSO for human object detection using fuzzy inference system (FIS) or so called fuzzy PSO. FIS is able to estimate the parameter based on certain variables. In PSO FIS is used to adaptively adjust one of the most important PSO parameter that directly involved in the velocity computation called acceleration coefficient. The acceleration coefficient c and c 2 controls the effect of pbest and gbest to the particle movement in exploring and exploiting the search area. If c > c 2 then particle is attracted to pbest position which causes local search otherwise if c 2 > c then particle is attracted to gbest position which causes global exploration [Engelbrecht 2007]. In order to keep the exploitation and exploration in a good balance the acceleration coefficient value must be adaptively adjusted to it pbest and gbest values. FIS for adjusting the acceleration coefficient of PSO works in individual level so that each particle or individu has a different coefficient value one another based on it pbest and gbest. This distinguishes fuzzy PSO with standard PSO where in standard PSO the acceleration coefficient is fix for entire swarm. The adaptive adjustment using FIS leads the searching process and gives the accurate detection result. The proposed FIS is designed for estimating the acceleration coefficient of PSO. Two variables are selected

6 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 5 as inputs to the fuzzy system; pbest and gbest which are directly related to the acceleration coefficient and two output variables are the expected acceleration coefficient c and c 2 respectively. The input variable pbest represents the best value the particle ever had while gbest represents the best particle value among all particles in the swarm. The range of all input values are thresholded by the value that positively contains human object in the adaboost classification process. The two input variables are defined to have three fuzzy sets namely Low Medium and High with associated membership functions as left trapezoid triangle and right trapezoid respectively. The definition of these three membership functions are: x x x ( ) 2 x f Low X x x x2 x2 x 0 x x2 0 x x ( x x ) /( x2 x) x x x2 f Medium ( X ) ( x3 x) /( x3 x2) x2 x x3 0 x x3 0 x x x x f ( ) High X x x x2 x2 x x x2 where x x 2 and x 3 are critical parameters which determine the shape of the functions. For input variable pbest the value of x x 2 and x 3 are for fuzzy set Low; for fuzzy set Medium; and for fuzzy set High. While for input variable gbest the value of x x 2 and x 3 are for fuzzy set Low for fuzzy set Medium; and for fuzzy set High. Fuzzy membership function for input variables pbest and gbest can be seen on Figure7 and Figure 8. Fig. 8. Fuzzy membership functions of input variable gbest The two output variables c and c 2 are defined to have three fuzzy sets namely Low Medium and High with triangle membership functions. The range value for c and c 2 is between 0 and. The definition of these three membership functions are: x x x ( ) 2 x f Low X x x x2 x2 x 0 x x2 0 x x ( x x ) /( x2 x) x x x2 f Medium ( X ) ( x3 x) /( x3 x2) x2 x x3 0 x x3 0 x x x x f ( ) 2 High X x x x2 x2 x x x2 where the value of x x 2 and x 3 are the same for both c and c 2 ; for fuzzy set Low; for fuzzy set Medium; and 0.7 for fuzzy set High. Fuzzy membership function for output variables c and c 2 can be seen on Figure 0. Fig. 0. Fuzzy membership functions of output c and c 2 Fig. 7. Fuzzy membership functions of input variable pbest Based on the observation to the samples nine fuzzy rules are defined as: [R] if (pbest is Low) and (gbest is Low) then (c is Medium) and (c 2 is Low) [R2] if (pbest is Low) and (gbest is Medium) then (c is Low) and (c 2 is Medium)

7 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 6 [R3] if (pbest is Low) and (gbest is High) then (c is Low) and (c 2 is High) [R4] if (pbest is Medium) and (gbest is Low) then (c is High) and (c 2 is Low) [R5] if (pbest is Medium) and (gbest is Medium) then (c is High) and (c 2 is High) [R6] if (pbest is Medium) and (gbest is High) then (c is Medium) and (c 2 is Medium) [R7] if (pbest is High) and (gbest is Low) then (c is High) and (c 2 is Low) [R8] if (pbest is High) and (gbest is Medium) then (c is High) and (c 2 is Medium) [R9] if (pbest is High) and (gbest is High) then (c is Medium) and (c 2 is High) T ABLE II T HE PERFORMANCE MEASUREMENT BETWEEN SCANNING METHOD AND FUZZY PSO Scanning Method Fuzzy PSO # Object Precision Recall Accurac y Precision Recall Accurac y Average T ABLE III DETECTION TIME COMPARISON FOR ALL TESTING DATA BETWEEN SCANNING METHOD AND FUZZY PSO (IN SEC.) Real Data Artificial Data Video Data Fuzzy Fuzzy Fuzzy # Object Scanning PSO Scanning PSO Scanning PSO Avg Det. Time VI. EXPERIMENTAL RESULTS AND DISCUSSION For comparison the proposed fuzzy adaptive acceleration of PSO was tested with the conventional scanning method for the same testing data in order to test the performance of each method. The data used in this experiment was visual data which contains human object. This data was divided into three categories; real data consists of 50 arbitrary images taken from database V0C Challenges 2005 [6] artificial data consists of 60 images which have the same pixel size and is set to have certain number of human object in it (one up to three human object) and four real-time video data with one human object in each data. Three testing scenarios are done to measure the detection performance detection time and performance of fuzzy PSO vs fix-adjusted PSO respectively. The first scenario tests the performance indicated by precision recall and accuracy of fuzzy PSO and scanning method. The result of the first scenario tested on 60 artificial images data is shown in table 2. The accuracy rate for the two methods decreases as the number of human object increases. The average accuracy rate of fuzzy PSO (93%) outperforms the scanning method (88.3%). The next scenario compares the detection time for all testing data using scanning method and fuzzy PSO. Each type of data has different number of human object and it results a different detection time. The comparison of detection time can bee seen in table 3. The average detection time for all testing data using fuzzy PSO is.62 second while using scanning method is 3.35 second. The detection time of fuzzy PSO is almost two times faster than that of scanning method. The last scenario tests the performance of fuzzy PSO versus fix-adjusted PSO for five times testing using real data. By using the proposed fuzzy adaptive acceleration of PSO leads the particles movement to exploit and explore the search area and detects the human object accurately. The comparison of the accuracy rate between fixed-adjusted and fuzzy PSO is shown in table 4. It can be seen that the accuracy rate of fuzzy PSO outperforms the fixed-adjusted acceleration coefficient of PSO. The Fuzzy PSO detection is more efficient to be applied in a real world application rather than the scanning method.

8 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 7 T ABLE IV T HE ACCURACY RATE COMPARISON BETWEEN FIXED-ADJUSTED ACCELERATION PSO AND FUZZY PSO Fix-adjusted Accuracy rate (%) Experiment c c 2 adjusted Fix- PSO Fuzzy PSO % 76.67% % 80.00% % 76.67% % 77.42% % 76.67% From the overall experiment fuzzy adaptive acceleration of PSO for human object detection has the greater accuracy rate (90.67%) than that of scanning method (8.66%). Fuzzy adaptive acceleration of PSO for human object detection improves the accuracy rate of standard window scanning by 9%. The percentage of accuracy rate for all testing data is shown in table V. T ABLE V T HE PERCENTAGE OF ACCURACY RATE FOR ALL T ESTING DATA Data Scanning Fuzzy PSO Real 78.86% 83.05% Artificial 88.30% 93.00% Video 77.82% 95.97% Average 8.66% 90.67% The accuracy rate of the scanning method is lower than that of in Fuzzy PSO. This is due to the number of group detection windows that determines the confidence level of a human object which does not exceed the confidence threshold value. This occurs on the scanning method because every detection window at the same scale can only be evaluated once. While in Fuzzy PSO the greater the discriminant value of the human object the more particles attracted to this position. It means that the corresponding position is localized by many detection windows resulting in a great human confidence level. Hence Fuzzy PSO is greater in term of the accuracy rate. The weakness of Fuzzy PSO detection is when the number of object increases (as in video artificial and real data the last type has the more objects) the accuracy rate decreases. This can be explained because PSO uses optimization principles so that particles tend to be attracted to the position with the greatest discriminant value while the other positions under the greatest value are less evaluated. But this is already overcome by using fuzzy inference system to balance between the global and local search. VII. CONCLUSION In this paper a fuzzy inference system is implemented to adaptively adjust the acceleration coefficient to improve the performance of PSO. Three scenarios have been set for testing the performance of PSO. For comparison the experiment is conducted for three methods namely fuzzy adaptive acceleration of PSO conventional scanning method and fix-adjusted acceleration of PSO. The experiment results show that PSO with a fuzzy inference system tuning its acceleration coefficient can improve its performance in terms of accuracy rate and detection time. From the overall experiment it can be said that the performance of PSO for human object detection can be improved by adaptively adjusting the acceleration coefficient of PSO using fuzzy inference system. The fuzzy inference system design can be further applied in the real-world problems that use PSO to optimize the computational time and gain the best performance of the system. REFERENCES [] Papageorgiou C. Poggio T. Trainable Pedestrian Detection Center for Biological and Computational Learning Artificial Intelligence Laboratory MIT 999. [2] Moore D. A Real-world System for Human Motion Detection and Tracking Thesis California Institute of Technology June [3] Liang Zhao Stereo and Neural Network Based Pedestrian Detection IEEE Transaction on Intelligent Transportation Systems Vol. No. 3 September [4] Mohan A. Object Detection in Images by Components MIT AI Memo 664 (CBCL Memo 78) June 999. [5] Papageorgiou C. Oren M. and Poggio T. A General Framework for Object Detection Proc. International Conf. Computer Vision Jan [6] Papageorgiou C. Poggio T. A Trainable System for Object Detection International Journal of Computer Vision 38() 5 33 Kluwer Academic Publishers. Manufactured in The Netherlands [7] Levi K. Weiss Y. Learning Object Detection from a Small Number of Examples: The Importance of Good Features Proceeding of CVPR(2): [8] Viola P. Jones M. Robust Real-time Object Detection Second International Workshop on Statistical and Computational Theories of Vision Modeling Learning Computing and Sampling Vancouver Canada July [9] Laptev I. Improvements of Object Detection Using Boosted Histograms IRISA / INRIA Rennes [0] Kennedy J. Eberhart R. Particle Swarm Optimization IEEE Inter. Conference on Neural Networks 995. [] Liliana D.Y. Widyanto M.R. Human Object Localization Using Particle Swarm Optimization The 2nd International Conference of Science and Technology Applications in Industry and Education (ICSTIE) Universiti Teknologi MARA Pulau Pinang Malaysia 2 3 Dec [2] Dalal N. Triggs B. Histograms of oriented gradients for human detection Proceeding of CVPR:

9 International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: No: 0 8 [3] Schapire R.E. The Boosting Approach to Machine Learning an Overview MSRI Workshop on Nonlinear Estimation and Classification [4] Engelbrecht A.P. Computational Intelligence an Introduction 2 nd Edition John Wiley & Son 2007 page [5] Eberhart R. Shi and Y. Computational intelligence: concepts to implementations Morgan Kauffman 2007 page [6] Pascal VOC Challenges Database OC/voc2005/index.html