Probabilistic Fuzzy Time Series Method Based on Artificial Neural Network

Transcription

1 America Joural of Itelliget Systems 206, 6(2): DOI: /j.ajis Probabilistic Fuzzy Time Series Method Based o Artificial Neural Network Erol Egrioglu,*, Ere Bas, Cagdas Haka Aladag 2, Ufuk Yolcu 3 Departmet of Statistics, Giresu Uiversity, Giresu, Turkey 2 Departmet of Statistics, Hacettepe Uiversity, Akara, Turkey 3 Departmet of Statistics, Akara Uiversity, Akara, Turkey Abstract May of forecastig methods have bee proposed i the literature. There are various classificatios of forecastig methods. No-probabilistic forecastig methods such as artificial eural etwork, fuzzy iferece systems ad fuzzy time series methods have bee commoly used i recet years. As a cosequet of this, forecastig methods ca be classified ito two groups as probabilistic ad o-probabilistic. Fuzzy time series methods are o-probabilistic forecastig methods. I the literature, may of fuzzy time series methods have bee proposed but their distributios could ot be obtaied for forecasts i these methods. I this study, a ew probabilistic fuzzy time series method is proposed firstly. The proposed method is based o movig block bootstrap method. It is possible to obtai distributios of forecasts by usig the proposed method proposed i this study. The proposed method was applied to three real world time series data ad also the performace of the proposed method was examied ad compared with other forecastig methods. Keywords Fuzzy time series, Probabilistic methods, Movig block bootstrap method, Particle swarm optimizatio. Itroductio Fuzzy time series methods are kow as importat forecastig methods i the literature. I recet years, may of ew fuzzy time series methods have bee proposed. Fuzzy time series methods are based o fuzzy set theory. There are two kid of fuzzy time series approaches as time variat ad time ivariat methods. I the literature, the umber of papers about time ivariat approaches is more tha the umber of papers about time variat approaches. There are some other classificatios of fuzzy time series methods. These are uivariate ad multivariate fuzzy time series methods, fuzzy time series methods based o first order ad high order forecastig methods. [] preseted a good review of fuzzy time series approaches. Artificial eural etworks have bee employed to defie fuzzy relatios i the literature. [2-9] used artificial eural etworks to defie fuzzy relatios. Multilayer perceptro artificial eural etworks are geerally preferred i fuzzy time series methods except a study. I the study of [9], multiplicative euro model eural etwork was preferred. Because, multiplicative euro model eural etwork has oly oe euro, it is simpler tha multilayer perceptro ad it leads successful forecastig * Correspodig author: erole977@yahoo.com (Erol Egrioglu) Published olie at Copyright 206 Scietific & Academic Publishig. All Rights Reserved results i fuzzy time series methods. Because of huma s icapability, the radomess is a good remedy for ucertaity. Probabilistic methods employ radom variables ad the values of estimators are radomly chaged sample by sample. Because of this, o-probabilistic methods cotai radomess like probabilistic methods. However, o-probabilistic methods do ot use estimators as radom variables. I fuzzy methods, fuzziess is employed as a remedy for ucertaity but these methods do ot use radomess. Like radomess, fuzziess ca be used for all data sets because the variables ca be clustered by usig fuzzy clusterig methods. It is well kow that fuzzy clusterig ca be preferred to other clusterig methods for real world data. The radomess ad fuzziess ca be used i a forecastig method, together. I this study, a probabilistic fuzzy time series approach was itroduced. I the proposed method, fuzzificatio ad determiig of fuzzy relatios were performed by usig fuzzy c-meas ad multiplicative euro model artificial eural etworks, respectively. The movig block bootstrap method was used to determie the distributios of forecasts. I the secod sectio, the techiques used i the proposed method are itroduced. I the third sectio the prosed method is itroduced. Applicatio results obtaied from our proposed method ad some other methods proposed i the literature are give i sectio four. Fially, i sectio five discussios ad coclusios are preseted.

2 America Joural of Itelliget Systems 206, 6(2): Techiques which use i Probabilistic Fuzzy Time Series Methods I the presece of multicolliearity, there are several remedies recommeded for avoidig from its udesirable effects o the probabilistic fuzzy time series method uses fuzzy c-meas, multiplicative euro model artificial eural etwork, particle swarm optimizatio ad movig lock bootstrap techiques. I followig sectios, these methods are give briefly. 2.. Fuzzy C-Meas This techique is a fuzzy clusterig techique. It was proposed by [0]. I this method, the data is partitioed ito fuzzy sets by miimizig sum of square error for groups. Let u, v i ad represet the membership value, cluster ceter ad the umber of variables, respectively. Thus, the forms of the objective fuctio to be miimized give by Equatio. c β 2 Jβ ( XVU,, ) u d ( xj, vi) = () i= j= I Equatio, β is the weightig expoet (β > ) ad d(x j,v i ) is the distace measure betwee the observatio ad the cluster ceter. J β is tried to be miimized uder the costraits give by Equatio 2. 0 u, i, j 0 < u, i (2) j = c u = i =, j Miimizatio process i FCM is performed by usig a iterative algorithm. I each iteratio, the values of u ad v i are updated by usig the formulas give by Equatios 3 ad 4. vi = β u xj j= β u j= u = 2 c d( x, ) ( ) j vi β d( x, ) k j v = k (3) (4) 2.2. Multiplicative Neuro Model Artificial Neural Network Multiplicative euro model eural etworks were proposed by []. This eural etwork does ot have a hidde layer ad it has oly oe euro. The multiplicative euro model artificial eural etwork ca be preferred to the other artificial eural etworks because of its simple architecture. This etwork uses multiplicative aggregatio fuctio istead of additive fuctio. Multiplicatios of iputs provide a advatage to obtai good traiig results i multiplicative euro model artificial eural etwork Particle Swarm Optimizatio Particle swarm optimizatio (PSO) is a stochastic ad artificial itelligece optimizatio techique. PSO techique was firstly proposed by [2]. There are may of modificatios of PSO i the literature. It is well kow that PSO with time varyig social, cogitive coefficiets ad iertia weight ca produce better optimizatio results tha stadard PSO. The most importat feature of PSO algorithm is to reach the optimum poit from several differet poits simultaeously i the search space. It ca avoid from local optimum traps because of its feature. Each particle i the search space has a positio ad velocity represetig the results for the optimizatio problem ad search directio. Ad also, the velocities ad the positios of the particles are updated by usig some equatios. Those who wat more iformatio ca look the study of [2] Movig Block Bootstrap Bootstrap techiques ca be used to obtai distributios of parameters i parametric or oparametric statistical models. I the literature, movig block bootstrap method was developed for time series data or time-depedet data. Movig block bootstrap ca be applied to time series data by choosig proper block legth. Determiig of block legth ca be effective of the results. I this study, radom movig block bootstrap (RMBB) techique proposed by [3] is employed. I RMBB method, block legth ad startig poit of a block are radomly chose by usig some distributios. 3. The Proposed Method Fuzzy time series methods are based o fuzzy set theory. I fuzzy time series methods, liguistics was employed by usig fuzzificatio techiques. Fuzziess is a importat ucertaity type ad it ca be hadled by usig fuzzy techiques. Radomess is a importat ucertaity type but it is ot employed i fuzzy time series methods. Fuzzy time series methods work with data samples. The results will be chaged whe the data sample is chaged. The data sample ca be obtaied radomly from populatio so the results will be chaged radomly. As a result of this fact, forecasts of

3 44 Erol Egrioglu et al.: Probabilistic Fuzzy Time Series Method Based o Artificial Neural Network fuzzy time series methods will be radomly chaged. I the statistical theory, ifereces ca be easily obtaied for liear models. The statistical iferece for distributio of parameters ad forecasts i fuzzy methods are ot easy because of high degree of oliear model structures. Bootstrap methods ca be employed to obtai distributio from all statistical methods. I this study, a ew probabilistic fuzzy time series method is proposed. The distributios of forecasts are obtaied by usig RMBB approach. The algorithm of the proposed method is give below: Algorithm : Mai algorithm for the proposed method Step Number of fuzzy sets (fs), legth of the test set ad model order (p) are determied. Step 2 FCM method is applied for traiig data ad the fuzzy time series are obtaied. Fuzzy time series is a series ad its elemets are idex umbers of fuzzy sets. After applyig FCM method, the highest membership value is determied for each observatio tha fuzzy time series are cosisted by usig idex umber of fuzzy set which has highest membership value for the observatio. Step 3 Block legth ad startig poit are selected radomly. The bootstrap sample is take from the traiig data of fuzzy time series accordig to determied block legth ad startig poit. Step 4 The multiplicative euro model eural etwork is traied by usig traiig data ad PSO method. The iputs of the etwork are lagged variables of fuzzy time series (FF(tt ), FF(tt 2),, FF(tt pp)), output of the etwork is fuzzy time series (F(t)). Step 5 The forecasts are obtaied for test data ad they are kept. Step 6 Steps 3-5 are repeated util it is reached to possible bootstrap sample umber. Step 7 The distributios of the forecasts ad arithmetic meas of bootstrap samples of forecasts are obtaied. The results of forecasts of the method are arithmetic meas of bootstrap samples of forecasts. Algorithm 2: Traiig algorithm for the multiplicative euro model artificial eural etwork Step Iitial velocity ad positio vectors of the particles are geerated radomly. All iitial positios of the particles are geerated radomly from (0, ) iterval. O the other had, velocities are geerated radomly from ( vvvvvvvvvv, vvvvvvvvvv) iterval accordig to the pre-determied vvvvvvvvvv limit value. Step 2 Fitess fuctio value for each particle is calculated. By usig outputs calculated from the etwork for learig samples as fitess fuctio, root-mea squared error (RMSE) value is obtaied by usig Equatio 5. RMSE = ( F ˆ ) 2 t F (5) t t = Where FF tt is the output of the etwork, FF tt is the target value ad is the umber of traiig examples. For the calculatio of RMSE value for each particle, outputs of the etwork FF tt, tt =,2,, are calculated. Step 3 PPPPPPPPPP ad GGGGGGGGGG are updated. If the fitess value of GGGGGGGGGG is uder a certai value of εε, or it is reached to the maximum umber of iteratios, the process is stopped. Otherwise, moves to Step 4. Step 4 Velocity values of the positios ad positios are updated ad retur to Step 2. Formulas give i Equatios (6) ad (7) are used. v + = w v + c r ( pbest x ) + c r ( gbest x ) (6) k k 2 2 j Implemetatio k+ k k+ x = x + v (7) Series Figure. Time series graph of Series Series 3 Figure 2. Time series graph of Series 2 Series 3 Figure 3. Time series graph of Series 3 For the evaluatio of the forecastig performace of the proposed method, three real world time series data were aalyzed. These time series iclude daily data 200, 20

4 America Joural of Itelliget Systems 206, 6(2): ad 202 Istabul Stock Exchage Market BIST00 idex. Series : BIST00 idex values with 04 observatios observed betwee ad (Figure ) Series 2: BIST00 idex values with 06 observatios observed betwee ad (Figure 2) Series 3: BIST00 idex values with 06 observatios observed betwee ad (Figure 3) I the aalysis of the time series, the last 7 ad 5 values of each series were take as test set. Alterative forecastig methods used i the aalysis ad implemetatio details of the methods are listed below. ARIMA: Autoregressive Itegrated Movig Average Model, The best model was determied Box-Jekis Procedure. ES: Expoetial Smoothig, Simple, Holt ad witers expoetial smoothig methods were applied ad the best model was selected. MLP-ANN: Multi-Layer Perceptro Artificial Neural Network, The umber of iputs ad hidde layer euros were chaged from to 5 ad the best architecture was selected by trial&error method. Leveberg Marquardt traiig algorithm was used as learig algorithm. SC: Sog ad Chissom time ivariat fuzzy time series method ([4]) the umbers of fuzzy sets were chaged from 5 to 5 ad the best umber of fuzzy sets were selected. FF: Fuzzy fuctio approach ([5]). The model order ad the umber of fuzz sets were chaged from to 5 ad from 5 to 5, respectively. Besides these methods, the proposed method was also compared with Fuzzy Time Series Network (FTS-N) proposed by [6]. For the compariso of the methods, values of RMSE ad Mea Absolute Percetage Error (MAPE) criteria calculated for the test set were used. MAPE criteria is calculated by usig Equatio (4) MMMMMMMM = XX tt XX tt= (4) XX tt I the implemetatio of the proposed method, the model order (pp) was varied betwee ad 5; the umber of fuzzy sets i the proposed method was varied betwee 5 ad 5 ad bootstrap iteratio umber was take as 00. Best-case results were obtaied i all possible situatios metioed above. I the implemetatio of PSO, ww = 0.9, cc = cc 2 = 2, pppp = 30, vvvv = 0. ad the maximum umber of iteratios was take as 200. Tables ad 2 summarize the results obtaied from test set for Series i terms of RMSE ad MAPE criteria whe the legth of test set () is 7 ad 5, respectively. I Table, the best result of the proposed method was obtaied from third-order model whe the umber of fuzzy sets was 3. I Table 2, the best forecastig performace was obtaied whe the model order is 3 ad the umber of fuzzy set is 4. Aalysis of Tables ad 2 reveals that the proposed method exhibits more successful ad superior forecastig performace whe compared with other methods i terms of MAPE ad RMSE performace measures. The graphs of actual values of the test set ad forecasts of the proposed method were give i Figures 4 ad 5, respectively whe the legth of the test set is 7 ad 5 for Series. Table. Compariso of Series forecasts for test set i terms of RMSE ad MAPE criteria whe test=7 ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Table 2. Compariso of Series forecasts for test set i terms of RMSE ad MAPE criteria whe test=5 ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Figure 4. Time series graph of test data (test=7) ad forecasts obtaied from proposed method for Series Figure 5. Time series graph of test data (test=5 ad forecasts obtaied from proposed method for Series

5 46 Erol Egrioglu et al.: Probabilistic Fuzzy Time Series Method Based o Artificial Neural Network Similarly, obtaied results for Series 2 were summarized i Tables 3 ad 4, respectively. Table 3. Compariso of Series 2 forecasts for test set i terms of RMSE ad MAPE criteria whe test=7 ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Table 4. Compariso of Series 2 forecasts for test set i terms of RMSE ad MAPE criteria whe test= ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Figure 6. Time series graph of test data (test=7) ad forecasts obtaied from proposed method for Series Figure 7. Time series graph of test data (test=5 ad forecasts obtaied from proposed method for Series 2 I Table 3, the best result of the proposed method was obtaied from fourth-order model whe the umber of fuzzy sets is ie. I Table 4, the best forecastig performace was obtaied whe the model order is ad the umber of fuzzy set is 5. Aalysis of Tables 3 4 reveals that the proposed method exhibits more successful ad superior forecastig performace compared to other methods i terms of MAPE ad RMSE performace measures. The graphs of actual values of the test set ad forecasts of the proposed method were give i Figures 5 ad 6, respectively whe the legth of the test set is 7 ad 5 for Series 2. Similarly, obtaied results for Series 3 were summarized i Tables 5 ad 6, respectively. Table 5. Compariso of Series 3 forecasts for test set i terms of RMSE ad MAPE criteria whe test=7 ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Table 6. Compariso of Series 3 forecasts for test set i terms of RMSE ad MAPE criteria whe test=5 ARIMA ES MLP-ANN SC FF FTS-N Proposed Method Accordig to all tables, the best results are obtaied from our proposed method. Moreover, proposed method gives the empirical probability distributios of forecasts. For example, the probability distributio for the first forecast of Series whe legth of test set is 5 is give i Table 7. It is oted that the forecast is the mea of this distributio. Table 7. The probability distributio for the first forecast of Series whe test=5 Value Probability Mea

6 America Joural of Itelliget Systems 206, 6(2): Discussios ad Coclusios I this paper, a probabilistic fuzzy time series method was itroduced. The proposed method was applied for Istabul stock exchage BIST00 data by usig three differet time series selected from the etire data. Accordig to fidigs, the ew method ca produce better forecasts tha other methods. The empirical probability distributios ca be estimated for forecasts i the proposed method. The proposed fuzzy time series method is the first method which produces probabilities for the forecasts i the literature. It ca be said that there is a weakess of the method about computatio time. Nevertheless, the proposed method ca complete to forecast a time series i a suitable time. Besides, oe of the most importat properties of the proposed method is to produce probabilistic results. REFERENCES [] Sigh, P., 205, A brief review of modellig approaches based o fuzzy time series, It. J. Mach. Lear. & Cyber., Doi: 0.007/s y. [2] Huarg, K., ad Yu, H.K., 2006, The applicatio of eural etworks to forecast fuzzy time series, Physica A, 363, [3] Aladag, C.H., Basara, M.A., Egrioglu, E., Yolcu, U., ad Uslu, V.R., 2009, Forecastig i high order fuzzy time series by usig eural etworks to defie fuzzy relatios, Expert Systems with Applicatios, 36, [4] Egrioglu, E., Aladag, C.H., Yolcu, U., Uslu, V.R., ad Basara, M.A., 2009a, A ew approach based o artificial eural etworks for high order multivariate fuzzy time series, Expert Systems with Applicatios, 36, [5] Aladag, C.H., Egrioglu, E., ad Yolcu, U., 200, Forecast combiatio usig artificial eural etworks, Neural Processig Letters, 32, [6] Yu, T.H.K., ad Huarg K.H., 200, A eural etwork-based fuzzy time series model to improve forecastig, Expert Systems with Applicatios, 37, [7] Huarg K.H., ad Yu T.H.K., 202, Modellig fuzzy time series with multiple observatios, Iteratioal Joural of Iovative Computig, Iformatio ad Cotrol, 8(0), [8] Yolcu, U., Aladag, C.H., ad Egrioglu, E., 203, A ew liear & oliear artificial eural etwork model for time series forecastig, Decisio Support System Jourals, 54, [9] Aladag, C.H., 203, Usig multiplicative euro model to establish fuzzy logic relatioships, Expert Systems with Applicatios, 40 (3), [0] Bezdek, J.C., Patter recogitio with fuzzy objective fuctio algorithms, Pleum Press, NY, 98. [] Yadav, R.N., Kalra, P.K., ad Joh, J., 2007, Time series predictio with sigle multiplicative euro model, Applied Soft Computig, 7, [2] Keedy, J., ad Eberhart, R.C., I: Particle Swarm Optimizatio, IEEE Iteratioal Coferece o Neural Network, , 995. [3] Yolcu, U., Yolcu, O.C., Aladag,C.H., ad Egrioglu, E., 204, A ehaced fuzzy time series forecastig method based o artificial bee coloy algorithm, Joural of Itelliget ad Fuzzy Systems, 26(6), [4] Sog Q., ad Chissom B.S., 993b, Forecastig erollmets with fuzzy time series - Part I., Fuzzy Sets ad Systems 54, -0. [5] Turkse, B., 2008, Fuzzy fuctio with LSE, Applied Soft Computig, 8, [6] Bas, E., Egrioglu, E., Aladag, C.H., ad Yolcu, U., 205, A fuzzy time series etwork to forecast oliear time series, Applied Itelligece, 43 (2),