A Hybrid Neuro-Genetic Approach to Short-Term Traffic Volume Prediction

Transcription

1 A Hybrid Neuro-Geneti Approah to Short-Term Traffi Volume Predition 1. Introdution Shahriar Afandizadeh 1,*, Jalil Kianfar 2 Reeived: January 2003, Revised: July 2008, Aepted: January 2009 Abstrat: This paper presents a hybrid approah to developing a short-term traffi flow predition model. In this approah a primary model is synthesized based on Neural Networks and then the model struture is optimized through Geneti Algorithm. The proposed approah is applied to a rural highway, Ghazvin-Rasht Road in Iran. The obtained results are aeptable and indiate that the proposed approah an improve model auray while reduing model struture omplexity. Minimum ahieved predition r2 is 0.73 and number of onnetion links at least redued 20% as a result of optimization. Keywords: Model; Traffi Predition; Neural Network; Geneti Algorithm Reent advanes in the field of traffi data olletion have provided transportation engineers with new features for traffi network management. These emerging tehnologies are utilized in TMC ; these enters an be designed for urban or rural highways. Traveler information and traffi management are primary funtional areas of a TMC. In this paper, we fous on developing a model whih overs both funtional area of TMC. Traffi detetors ollet traffi parameters and transmit them through ommuniation bakbone to TMC. This data is utilized in order to improve traffi management strategies. Short-term Traffi Predition Model is one of the TMC features, whih puts traffi network data into pratial use. The model reeives real-time traffi ondition on eah link as input data and predits link traffi ondition for a few minutes ahead. Traffi ondition an be in the form of volume, speed and travel time. TMC an take the advantage of this information for proative traffi ontrol. Proative ontrol targets near-term antiipated onditions, and the traffi network would operate (in theory, at least) under ontrol strategies that are more relevant to the prevailing onditions. On the other hand, reative ontrol, as the name implies, reats to already-observed onditions of the traffi stream. * Corresponding Author: zargari@iust.a.ir 1 Assoiate Professor, Iran University of Siene and Tehnology 2 M.S. Transportation Engineering, Tehran, IRAN, , It is worth mentioning that short-term traffi predition model is quite distint from onventional traffi predition models. Conventional models predit traffi volume for the oming years or predit traffi volume in different senarios while short-term traffi predition models estimate traffi volume for only a few minutes ahead. Conventional models are based on a four-step method or diret models while short-term models are usually based on heuristi methods. Another differene is related to sope and usage of models. Conventional models are aimed at planning issues; on the other hand, short-term models insist on operation and management issues. Neural networks and geneti algorithm are parts of Artifiial Intelligene whih have been applied in different areas suessfully?[1],?[2]. The method proposed in this paper is a bi-level approah based on Neural Network and Geneti Algorithm. First, a primary model is developed using Network and developed Network is optimized by applying Geneti Algorithm. 2. Bakground Suessful implementation of ITS systems is highly dependent on the quality of information provided by systems[3]. Thus, ontinuing efforts have been onduted by various researhers in order to improve short-term traffi predition models auray. During the last two deades, many different models have been presented based on dynami traffi assignment, statistial methods and neural networks. To model and foreast short-term traffi flow, statistial time series analyses, in the form of the International Journal of Civil Engineerng. Vol. 7, No. 1, Marh

2 ARIMA family of models have been the most widely used approahes [4],[5]. But, although the temporal variability of traffi flow appears to be a ruial harateristi in short-term foreasting, the traditional time series methodologies frequently seem unable to apture the rapid variations of flow in urban areas[6]. Davis and Nihan attempted to replae the time series approah by a non-parametri regression approah, but they onluded that the performane of their k-nearest neighbor approah performed omparably to, but not definitely better than, the time series approah [7]. Several attempts have been made using Neural Networks as a substitute for the more traditional regression and time-series approahes [8], [9]. Common to all is a onlusion of potential superiority of Neural Networks and a reommendation for further in-depth investigation under different senarios and using larger, real databases. A reent approah in short-term traffi flow predition is based on ombination of Neural Networks and Geneti Algorithm, whih is limited only to two ases. In 2002, Abdulhai et al. developed a model based on Time-Delay Neural Networks and GA. In this researh, GA was used to determine general struture of TDNN, inluding number of inputs (look-bak time), number of hidden neurons and learning rate. Steepest deent gradient method was seleted as learning method [10]. In 2005, Karlaftis and others applied a neurogeneti method for short-term traffi predition. Their model was based on Multi-Layered Feedforward Neural Networks. GA was employed in two levels in their researh. First, the proper learning rate and momentum for bakpropagation learning method were determined. Then proper number of hidden neurons was hosen using GA. Steepest deent gradient method was also seleted as learning method in this researh [11]. The approah presented in this paper is based on ombination of neural networks and geneti algorithm, yet it differs from previous works in two different areas. In the proposed method, GA is used to optimize links whih onnet input ells to hidden ells thereby the final network would be partially onneted. Furthermore, networks are trained using Levenberg-Marquardt method whih is a seond order training method, while other researhers used steepest gradient method for training network. In addition, the proposed approah is implemented in a rural highway for 68 day period, onsidering that rural trips pattern is more disorganized than urban trip patterns. The remainder of this paper is organized as follows. The next setion is a brief overview of neural networks and the fourth setion briefly disusses geneti algorithm. The fifth setion desribes the proposed approah and sixth setion presents the empirial results from a ase study.finally, the seventh setion summarizes and disusses the findings of the paper. 3. Neural Networks Neural Networks are part of Artifiial Intelligene and inspired by human brain funtion. Up to now, several types of neural networks have been introdued and MLF neural network is one of these networks. A typial twolayer MLF (onsists of a hidden layer and an output layer) an approximate any ontinuous nonlinear funtion to an arbitrary preision [12],[13]. Considering this harateristi, MLF neural networks are seleted as ore arhiteture for short-term predition model. 3.1 Neural Network Priniples Neural networks onsist of neurons or ells. Cells are grouped into layers. Eah ell is onneted to next layer ells through links or onnetion hannels. Links transfer ells output to the next layer ells. Links have weights by whih ell output is multiplied. The value of weighs is determined during training proess. In fat, the knowledge of a neural network is stored in its weights. 3.2 Neural Network Training A neural network an predit true outputs for orresponding inputs only when network parameters are seleted properly. Link weight is 42 International Journal of Civil Engineerng. Vol. 7, No. 1, Marh 2009

3 one of the network parameters. A proess in whih proper values for weights are seleted is alled network training. Bakpropagation is a neural network training method. In this method, inputs are presented to neural network and then network output is alulated; the differene between desired outputs (target outputs) and network outputs (real outputs) shows network error. At the next step, value of weights is revised aording to network error. Eah interation of revising weights values is alled an epoh. Neural Network training an be onsidered as a minimization problem whih seeks for weights that minimize network error index?[14]. Network error index is defined as the sum of squared error between target values and real values. ( ) e T F w = e (1) where F(w) is network error index, w = [ w1 w2 L w N ] is a vetor ontaining all weights and e is a vetor ontaining network error for training pairs. Levenberg-Marquardt is a training method for MLF neural Networks?[15]. In this method orretion value for eah weight ( w ) is alulated using (2): T 1 T [ J J + I] J e w = µ (2) where J is Jaobean matrix and µ is learning rate. Learning rate is seleted in eah epoh based on results of orretion in links weights. If weights orretion brings an inrease in F(w), µ will be multiplied by deay rate β ( 0 < β < 1), and if F(w) dereased, µ will be divided by β. Levenberg-Marquardt an be illustrated in the following steps: Step 1: Selet an initial value for weights and µ (usually µ =0.1). Step 2: Calulate F(w). Step 3: Compute w using (2). Step 4: w = w + w and alulate F(w`). If F( w ) > F( w ) then w = w, µ = µ β( β = 0.1), go to step 2. Else µ = µ β, go to step 4. This algorithm stops when the desired auray ahieved or if the number of iterations exeeds defined number of iterations. We assume 100 for maximum number of iterations in our study. Neural networks should be able to alulate true outputs for inputs they have not enountered before. This ability of neural networks is alled generalization. In order to assess generalization ability of a neural network, available data is divided into three parts, training set, validation set and test set. Training set data is used to develop model. Validation set data is presented to all models developed, a model whih produes best results will be seleted as best model. Test set data are used to ensure generalization ability of best model?[16]. 4. Geneti Algorithm Geneti Algorithm is an evolutionary method whih is inspired from evolution of reatures in nature. In this method hromosomes and genes are utilized to solve problems. First, solutions are oded into hromosomes. Then an initial generation of hromosomes is produed. In the next steps, new generations of hromosomes are produed using geneti operators and positive harateristis of eah generation are transferred to new generations?[17]. Ultimately the last generation determines the answer to the problem. Reprodution, ross-over and mutation are most ommon geneti operators. Reprodution operator diretly transfers some hromosomes to new generation. Cross-over operator selets two hromosomes as parents and then produes two offspring by ombining the parents. Mutation operator randomly performs some hanges in hromosomes in order to prevent searh area from restriting to speial part of spae. Chromosomes are ompared based on their fitness. Chromosomes fitness is omputed by fitness funtion, and then hromosomes are seleted using seletion mehanism. Ranking method and roulette-wheel are most ommon seletion methods?[18]. 5. Proposed Method Figure 1 depits proposed arhiteture for short-term traffi flow predition model. The Shahriar Afandizadeh, Jalil Kianfar 43

4 model reeives reent temporal traffi profile in the form of i input variable, eah variable represents traffi volume in a t-minute period. It is worth mentioning that in the presented approah, in addition to number of inputs and hidden ells, onnetion layout of hidden layer is also synthesized,. This means that some links will be omitted (dash lines in Figure 1). The Proposed model is a MLF neural network inluding a hidden layer and an output layer. The output layer is only onsisted of one ell; output of this ell represents the traffi flow for a t- minute period. The mathematial representation of the model is as follows: i s j = w jkvk j= 0 z j = σ ( s j ) p Vt = wq zq q= 0 k = 1,2,..., i j = 1, 2,..., p (3) where V k refers to traffi volume in k th t- minute interval, w jk is weight of the link whih onnets k th input to j th hidden ell, s j is net input of j th hidden ell, σ is sigmoid funtion, w q is weight of the link whih onnets q th hidden ell to output ell and V t is the traffi volume in t- minute period ahead. The proposed approah for model development onsisted of two major stages; first a primary model is developed using MLF neural networks and in the seond stage, primary model is optimized using geneti algorithm. Figure 2 depits these stages: 5.1 Setting up primary model In the first step, a neural network is designed Fig. 1 Model Struture Model Development Proess 1- Defining number of Inputs 2- Defining number of Hidden Cells 2- Model Optimization 1- Optimizing Link Connetions 2- Model Validation Fig. 2 Stages of model development as a primary model. of input variables and hidden ells are determined in this step. Seletion of neural networks inputs and hidden ells is a trail and error proess. Trial and error proess is started with a network ontaining minimum inputs and hidden ells, and then the number of inputs and hidden ells will be inreased gradually. This proess is finished when further addition of inputs and hidden ells do not have positive impat on model performane. The trial and error proess is as follows: Step 1: Selet p as number of input variables. Step 2: Selet q as number of hidden ells. Step 3: Design a network with p inputs and q hidden ells and evaluate its performane. Step 4: onsider network performane If it reahed the desirable performane then go to step 7. Else go to step 5. Step 5: Considering last two networks, is network performane improved? If yes, then go to step 2. Else: p=p+1 and go to step 2. Step 6: q=q+1 and go to step 3. Step 7: end Model mean squared error for validation data is a measure of network performane. When trail and error proess finishes, the best network is seleted and its parameters will be transferred to next step. In ommon neural networks, all input variables are onneted to hidden ells. These networks are alled full onneted networks. But full onnetion sometimes affets model 44 International Journal of Civil Engineerng. Vol. 7, No. 1, Marh 2009

5 generalization ability. During training, the value of some link weights may beome far greater than other weights. In this ase, values arried by other links have no effet on net input of ells. This means that other links are omitted unintentionally. 5.2 Model Optimization In order to derease model omplexity and also improve model generalization ability, the optimum layout of hidden layer onnetion links is designed using geneti algorithm. During optimization proess, some links will be omitted. This ould hange model to a partially onneted neural network. Figure 3 depits the optimization proess Chromosomes Enoding and First Population Before applying geneti algorithm, answers to the problem should be introdued to algorithm. The aim of this step in developing short-term traffi predition model is optimization of onnetion links; so onnetion layout of neural networks should be enoded in order to be understood by geneti algorithm. Binary enoding method is seleted to enode onnetion links of model. Matrix C represents a typial enoding of onnetion links in a neural network: C = M p M p2 L L O L 1n 2n M pn (4) Aording to this enoding, if there exists a onnetion link from input variable j to hidden ell i, then ij would be equal to 1 and if else, ij would be equal to 0. Considering the number of inputs and hidden ells whih were determined in previous step, initial population should be generated. In this regard, several C matries are generated randomly. Eah matrix will be assigned to a neural network. These neural networks are geneti algorithm hromosomes whih ompromise initial population. Proper population size will be determined during ase study. Figure 3: Optimization proess flowhart Fig. 3 Optimization proess flowhart Produing New Generations Geneti algorithm is based on evolution of hromosomes in suessive prodution of new generations. Geneti algorithm transfer superior harateristi of urrent generations to a new generation. Fitness funtion is used to evaluate hromosomes harateristis. While geneti algorithm is used to optimize link onnetions; yet the main purpose is to attain an aeptable model. So, the defined fitness funtion is representative of model error. Equation 7 shows fitness funtion: 1 Fitness( net (5) k ) = 1 + mse where mse is model mean squared error for validation data. Geneti algorithm applies seletion mehanism to hoose hromosomes whih are going to undergo geneti operators. Roulettewheel is a seletion mehanism in whih hromosomes with high fitness are more likely to be seleted. Shahriar Afandizadeh, Jalil Kianfar 45

6 p r ( net ) k = n Fitness( net ) i= 1 (6) where p r ( net k ) is probability of network k to be seleted, Fitness( netk ) is fitness value of network k and n is population size. In this method, a random number is generated and the network orresponding to this number will be seleted. Geneti algorithm produes new generations using generi operators but an important question is the share of operators in prodution proess. Proper share of geneti operators will be determined in the ase study based on sensitivity analysis. 6. Case Study Short-term traffi volume predition models need a ontinuous temporal profile of traffi as their input. This data is usually obtained from a traffi sensor. The traffi data used in this researh is obtained from an indutive loop detetor in a rural highway, Ghazvin-Rasht Road in Iran [19]. In following setions, we try to use presented methodology to develop models for this highway. 6.1 Data Colletion Fitness( net ) k i As mentioned in previous setion, data belongs to an Indutive Loop Detetor. This data is reorded in PVR format whih means there is a separate reord for eah vehile passage in database. The original database ontains 385,831 reords; this data was reorded in winter 2005 form Jan 10 until Marh Data belongs to a 68 day period in winter and it is aptured ontinuously 24 hour a day. A omputer program in VBA was prepared for data aumulation and then MATLAB was seleted as the main workplae for model development. In order to assess generalization ability of a neural network, available data is divided into three parts, training set, validation set and test set. 50%, 25% and 25% is the share of sets in this study. parameters are population size, number of generations and operators shares. In order to determine these parameters, an initial model with 5 minute predition horizon was designed based on setion 5-1 guidelines. The model is omposed of 6 input variables and 4 hidden ells. Then the proper share of operators was assumed to be reprodution 30%, ross over 50% and mutation 20%. Several optimizations were performed starting with different population sizes. Figure 5 depits mean squared error of optimization results obtained from different population sizes. The population size of 25 provides the best mean squared error. An investigation into the effets of number of generation on model performane shows that maximum model performane is ahieved in 10th generation and keeping on produing further generations has no effet on model performane. Operators share also affets the proess outomes. Various operator ombinations were examined to find out most suitable operators ombination. Figure 5 depits the result obtained from some tested ombinations. The best result ours when the following ombination is applied; reprodution 16%, ross-over 36% and mutation 48%. 6.3 Model Development Following determination of geneti algorithm parameters in previous step, models with 5, 10 and 15 minutes predition horizons are built aording to the proess depited in Figure 3. Table 1 illustrates primary models for given horizons. After designing primary models, optimum 6.2 Population Size and Operators Share Geneti algorithm parameters should be determined prior to model optimization. The Fig. 4 Error vs Population size 46 International Journal of Civil Engineerng. Vol. 7, No. 1, Marh 2009

7 mse Mutation Perent Reprodution Perent Predition Interval (Minute) Table. 2 Final Models Speifiation of Inputs of Hidden Cells of Links 2 r Table. 3 Final Models Performane (for whole data set) Fig. 5 Effet of operators share on model performane onnetion arrangements are synthesized using geneti algorithm. Table 2 presents results obtained from optimization. These models are in fat final models. Table 3 presents models performane for whole data set. Mean squared error, average relative error, orrelation oeffiient between atual outputs and desired outputs, and r-squared are performane indies. Figure 6 depits 15-minute model predited flow and observed flow in terms of passenger ar unit for eah data set. Model generalization ability is impressive and model has aquired traffi pattern well. The results learly show that although some of onnetion links are omitted, model performane has improved. 7- Conlusion Predition Interval (Minute) Table. 1 Primary Models Speifiation of Inputs of Hidden Cells of Links 2 r Predition Interval (Minute) Mean Squared Error Average Relative Error Correlation Coeffiient This paper has presented a short-term traffi flow predition approah and produed a system based on an advaned Multi-Layer Feed forward Neural Network (MLF) model synthesized using Geneti Algorithms (GA).The model predits flow values based on their reent temporal profile at a given highway site during the past few minutes. The model performane was validated using real traffi flow data obtained from the field. Results obtained from training data, validation data and test data addue that model generalization ability is satisfatory. For the ase of 15-minute predition horizon for instane, predition r 2 indies are 0.85, 0.84 and 0.83 respetively whih evidently shows model generalization ability. Moreover, it was found that the longer the extent of predition, the more the predited values tend toward the mean of the atual for a given data resolution. In the ase of optimization effetiveness, proposed approah performed well on reduing model omplexity, meanwhile improving model generalization ability. As an 2 r Fig. 6 Predited flow vs observed flow for 15-minute horizon Shahriar Afandizadeh, Jalil Kianfar 47

8 example, number of links has been redued 37% for 5-minute predition model; furthermore, model r2 index inreased from 0.72 to The model performed aeptably using field data, bearing in mind that the real-data used is from a rural highway in winter, that is, the pattern of traffi is totally different from an urban highway. Referenes [1] Wang, Lipo, Fu Xiuju, Data Mining with Computational Intelligene, Germany, Springer, [2] Kasabov, Nikola K., Foundations of Neural Networks, Fuzzy Systems and Knowledge Engineering, England, MIT Press, [3] Kan Chen; John C. Miles, ITS Handbook 2000: Reommendations from the World Road Assoiation (PIARC), Arteh House, Otober, [4] Kirby, H., Dougherty, M., Watson, S., Should we use neural networks or statistial models for short term motorway foreasting, International Journal of Foreasting 13, [5] Williams, B.M. Multivariate Vehiular Traffi Flow Predition: An Evaluation of ARIMAX Modeling, Transportation Researh Reord 1776, [6] Abdulhai, B., Porwal, H., Reker, W., Shortterm freeway traffi flow predition using genetially-optimized time-delay-based neural networks, Transportation Researh Board, [7] Davis, G. and Nihan, N., Nonparametri Regression and Short-Term Freeway Traffi Foreasting, Journal of Transportation Engineering, 117, , [8] Smith, B. and Demetsky, M., Short-Term Traffi Flow Predition: Neural Network Approah, Transportation Researh Reord 1453, [9] Dougherty,M. and Lehevallier,Y., Short-Term Road Traffi Foreasting Using Neural Network, Reherhe Transports Seurite, English Edition 11, [10] Abdulhai, Baher; Himanshu, Porwal; Will, Reker, Short-Term Traffi Flow Predition Using Neuro-Geneti Algorithms, ITS Journal, Vol. 7, pp.3 41, 2002 [11] Karlaftis, Matthew, G.; Eleni, I. Vlahogianni; John C. Golias, Optimized and meta-optimized neural networks for short-term traffi flow predition: A geneti approah, Transportation Researh, Part C, Vol. 13, pp , [12] Prinipe, J.C., Euliano, N.R., Lefebvre, C.W., Neural And Adaptive Systems: Fundamentals Through Simulations, John Wiley and Sons, New York, [13] Ham F. M., I. Kostani, Priniples of Neuroomputing for Siene and Engineering, MGraw Hill, New York, [14] Haykin, S., Neural Networks: A Comprehensive Foundation, MMillan, New York, [15] Hagan M. T., M. B. Menhaj, Training feed forward network with the Marquardt algorithm, IEEE Trans. on Neural Net., Vol. 5, No. 6, pp , [16] Gupta, Madan, M. Liang Jin, Noriyasu Homma, Stati and Dynami Neural Networks: From Fundamentals to Advaned Theory, John Wiley & Sons, [17] Haupt, Randy L., Sue Ellen Haupt, Pratial Geneti Algorithms, New Jersey, Wiley Intersiene, [18] Melanie Mithell, An Introdution to Geneti Algorithms, MIT Press, Massahusetts, [19] Iran Road Maintenane and Transportation Organization (RMTO), 48 International Journal of Civil Engineerng. Vol. 7, No. 1, Marh 2009