A Naïve Support Vector Regression Benchmark for the NN3 Forecasting Competition

Transcription

1 A Naïve Support Vector Regresson Benchmark for the NN3 Forecastng Competton Sven F. Crone and Swantje Petsch Abstract Support Vector Regresson s one of the promsng contenders n predctng the 111 tme seres of the NN3 Neural Forecastng Competton. As they offer substantal degrees of freedom n the modelng process, n selectng the kernel functon and ts parameters, cost and epslon parameters, ssues of model parameterzaton and model selecton arse. In lack of an establshed methodology or comprehensve emprcal evdence on ther modelng, a number of heurstcs and ad-hoc rules have emerged, that result n selectng dfferent models, whch show dfferent performance. In order to determne a lower bound for Support Vector Regresson accuracy n the NN3 competton, ths paper seeks to compute benchmark results usng a naïve methodology wth a fxed parameter grdsearch and exponentally ncreasng step szes for radal bass functon kernels, estmatng 43,725 canddate models for each of the 111 tme seres. The naïve approach attempts to mmc many of the common mstakes n model buldng, provdng error as a lower bound to support vector regresson accuracy. The n-sample results parameters are evaluated to estmate the mpact of potental shortcomngs n the grd search heurstc and the nteracton effects of the parameters. T I. INTRODUCTION me seres forecastng wth computatonal ntellgence has receved ncreasng attenton n theory and practce. However, n order to prove ther effcacy n forecastng ther accuracy must be evaluated aganst establshed statstcal forecastng methods on emprcal datasets [1, 2]. The 2007 NN3 Forecastng Competton for computatonal ntellgence methods provdes ths opportunty on a dataset of 111 emprcal tme seres and a reduced subset of 11 tme seres n order to establsh the forecastng accuracy of computatonal ntellgence methods on busness data. Recently, the method of Support Vector Regresson (SVR) has shown promsng performance n varous scentfc forecastng domans [3-7], offerng a nonparametrc, data-drven and self-adaptve method that learns lnear or nonlnear functonal relatonshps drectly from tranng examples [2, 4]. However, recent experments have demonstrated that despte the promse of automatcally estmatng optmal predctors usng statstcal learnng theory, SVR offer substantal degrees of freedom n forecastng, requrng a data dependent selecton of the kernel functon and ts parameters from a set of potental functons, and two metrc scaled parameters to control the Sven F. Crone s wth the Department of Management Scence, Lancaster Unversty Management School, Lancaster LA1 4YX, Unted Kngdom ( , e-mal: sven.f.crone@crone.de). Swantje Petsch s wth the Insttute of Informaton Systems, Unversty of Hamburg, Hamburg, Germany; (e-mal: malng@swantjepetsch.de). cost and epslon-nsenstve margn. Hence, support vector regresson share common and well known problems wth competng forecastng methods such as artfcal neural networks, offerng near endless degrees of freedom n the choce of archtecture parameters to be selected based upon ther performance on short and nosy tme seres. Due to the lack of an establshed methodology, a number of modelng heurstcs and ad-hoc rules-of-thumb have emerged n order to gude archtecture decsons n SVR modelng. As dfferent heurstcs may lead to dstnct models and varyng performance, they requre a systematc evaluaton. To further nvestgate the capablty of SVR n tme seres forecastng, SVR s appled to forecast the 111 tme seres of the NN3 competton. In order to determne a lower bound for SVR predctons n the competton, ths study seeks to compute benchmark results usng a naïve methodology of a fxed parameter grd-search wth exponentally ncreasng step szes. The heurstc s lmted to a radal bass functon kernel, usng a smple preprocessng of nput-data, and followng a smple pck-the-best approach n model selecton of the canddate wth the lowest cross-valdaton mean squared error (MSE). Consderng the naïve heurstcs employed, the methodology actvely neglects relevant modelng gudelnes n SVR modelng, such as regardng data dependent kernel selecton, canddate model selecton usng k-fold cross-valdaton for performance predcton, adequate scalng and preprocessng of data etc. Hence we develop a naïve benchmark utlzng avalable computatonal power as a lower bound to SVR performance. The naïve grd search estmates 43,725 canddate models for each of the 111 tme seres, computng a total of 4,853,475 models for the NN3 datasets. In addton to provdng benchmark results, we analyse the nteracton of dfferent parameter choces and nterpret the results. The paper s organsed as follows. Frst we provde a bref ntroducton to ε-svr and the relevant model parameters n forecastng, followed by a short dscusson on alternatve methodologes for modellng SVR. Secton 3 outlnes the expermental setup of data pre-processng, SVR parameter selecton and the SVR canddate selecton, followed by the parameter results and an analyss of ther nteractons. II. SUPPORT VECTOR REGRESSION A. Method Background The method of Support Vector Regresson (SVR) s based on statstcal learnng theory by Vapnk [8] and estmates a functon f ( x ) that mnmzes the forecastng error on a tranng data set (( x, y ),...,( x, y )) ( X Y) whle keepng 1 1

2 the functonal form as flat as possble [4, 9, 10]. SVR s formulated as convex optmzaton problem wth slack varables ξ,ξ to allow for model errors [11, 12] 1 mnmze w 2 subject to 2 + C = y ( w x ) ( w x ) + b ξ, ξ 1 y ( ξ + ξ ) b = ε + ξ = ε + ξ 0 whch controls the trade-off between overfttng and model complexty through a regularzaton parameter C>0 [8]. We employ ε-svr, usng an ε-ntensve loss functon, that assgns an error only to those observatons ξ,ξ 0 outsde the ε-nsenstve tube [13, 14], named support vectors [15], usng the loss functon 0 f ξ ε ξ ε : = (2) ξ ε else To handle non-lnear functonal relatonshps n forecastng problems, the data are mapped usng a kernel functon φ nto a hgh dmensonal feature space F, where they may be solved lnear regresson, whch corresponds to nonlnear regresson n a lower dmensonal nput space [5].In ths study the radal bass kernel functon (RBF) s appled, as t s commonly used n ε-svr usng just one parameter γ that to determned the RBF wdth a pror [3, 4, 16]. For the RBF, the number of centres, locaton of the centres, the weghts and the thresholds are all determned whlst tranng [17]. Varous basc tutorals exst to provde an ntroducton on the background and the mathematcal propertes of support vector machnes [17] and SVR n partcular [4]. In forecastng wth SVR, the nput vector contans the lag structure of the tme seres, whch results n dot products after combnng them wth the support vectors n the kernel functon. The quadratc optmzaton problem s solved to determne Lagrangan multplers α, α that are used to determne the weghts υ = α α [4]. The dot products are then weghted by υ = α α to calculate the one-step-ahead predcton value together wth the threshold b [4]. The forecastng process can be vsualsed as n fgure 2. B. Methodologes n specfyng SVR parameters SVR s a data-drven and self-adaptve methods, whch s capable of approxmatng lnear and nonlnear functonal relatonshps from data, unlke tradtonal model-based statstcal methods [2, 4]. For applyng ε-svr n forecastng, a number of parameters must be determned a pror by determnng the Costs C, the wdth of the epslon-nsenstve loss functon ε, the kernel functon and ts kernel parameters γ [18] and the number and lags of ndependent varables to specfy the nput vector as a rollng wndow of fxed sze over a tme seres. Recent experments have shown that the forecastng performance of the ε-svr s sgnfcantly mpacted by an adequate a pror selecton of ther parameters [19], they are consdered to be sem-parametrc (1) Fg. 2. Tme seres predcton wth SVR rather then non-parametrc methods, smlar to neural networks. Hence they requre an expert to determne a pror model parameters that mpact on the forecastng capabltes. In order to determne the ε-svr parameters varous modellng heurstcs exst. Gao, Gunn, Harrs and Brown determne SVR parameters usng a Bayesan framework for Gaussan SVR [20]. Chu, Keerth and Ong follow another Bayesan approach, that combnes the merts of SVR wth the advantages of Bayesan methods for model adaptaton [21]. Chang and Ln derved leave-one-out bounds for SVR parameters [22]. Heurstcs can also lead to parameter combnatons wth nferor performance the even a smple parameter grd search [23], as n comparng a grd search wth a Bayesan approach n Ln and Weng [24]. Momma and Bennett perform model selecton by pattern search to reduce the number of parameter combnatons that need to be tested [18, 25]. Kwok and Tsang [26] as well as Smola, Murata, Schölkopf and Müller [27] determne the parameter ε as a lnear dependency on the nose of the tranng data, whch requres a pror knowledge of the nose level [22]. In contrast, Cherkassky and Ma analyse the parameter nteracton to lmt the number of relevant parameters. The suggest that for a gven ε, the value of C has only neglgble effects on the generalzaton performance as long as C s larger than a certan threshold that can be determned from the tranng data [18]. A smple method to determne sutable ε-svr parameters for each tme seres follows a systematc grd search over the parameter space [23, 28]. Instead of evaluatng every possble parameter combnaton, whch would be ntractable for parameters of nterval scale, a grd usng equdstant steps n the parameter space lmts the computatonal effort. However, dfferent grds are applcable, usng lnear step szes, exponental ncreasng sequences as n Hsu [23] or Luxburg [29] or logarthmc sequences as n Chang and Ln [22]. In addton, stepwse refnements of the grd sze n parameter space are feasble, leadng to an analytcally

3 smple yet computatonally expensve parameter selecton approach. In ths study we seek to explore the smple grdbased approach, usng a brute-force, exhaustve enumeraton of a representatve parameter space. growng sequences that cover a vast range of value combnatons. Fgure 2 shows an example grd wth exponentally growng sequences. III. EXPERIMENTAL DESIGN OF NAÏVE SVR A. Specfyng SVR nput vectors The forecastng accuracy of any method depends largely on provdng adequate nput nformaton to learn from. In tme seres forecastng ths takes the form of specfyng sgnfcant tmed lags of the dependent varable y t-n and excludng rrelevant ones, hence determnng the length of the nput vector. Multple methods exst n specfyng nput vectors, based on smple heurstc rules, statstcal autocorrelaton analyss to determne the order of autoregressve (AR), ntegrated (I) and movng average (MA) processes or mxed ARIMA-processes of lagged realsatons of the dependent varable [2] or usng spectral analyss to detect multple overlyng seasonal patterns. To pursue a naïve modellng approach we select a smple heurstc decson rule based on the observaton nterval of the tme seres, usng a constant lag structure of the past 12 monthly observatons n a year to account for possble seasonalty of the months or quarters. The same lag structure was used for all 111 tme seres, despte the possblty of dfferent lag structures for dfferent tme seres, the necessty to nclude 13 lags for seasonal ntegrated autoregressve processes SARIMA(p,d,0)(P,D,0) s or the approxmaton of MA processes of SARIMA(0,0,q)(0,0,Q) s. by extendng the nput vector to multples of the yearly seasonalty. B. Data Pre-Processng The dataset contans 111 monthly tme seres from the complete dataset of the NN3 competton. Of these seres 11 monthly tme seres form a reduced data subset of the competton. The NN3 tme seres are heterogeneous, show varous seasonal and non-seasonal patterns and nose levels, and vary n length between 49 and 126 observatons. The predcton objectve s to forecast the next 18 observatons n a multple step ahead forecast as accurately as possble. Data s scaled before tranng n order to speed up the computaton process and to avod numercal dffcultes [23]. Each tme seres observaton x t s lnearly scaled as z t nto the nterval of [-0.5; 0.5], usng the scalng functon of z t ( xt xt mn ) ( x x ) = 0,5 +, (3) t max t mn on the mnmum x t mn and maxmum value x t max of x t on the tranng and valdaton set and by applyng headroom of 50% to avod possble saturaton effects of the nodes on nstatonary tme seres patterns n the unseen test data. C. SVR tranng and parameters We employ a smple grd search of costs C, epslon ε and the parameter of a Gaussan kernel γ wth exponentally Fg. 2. A grd wth exponentally growng sequences However, employng a grd search methodology requres the settng of vald and relable lower and upper parameter bounds that defne the search space of the grd. The regularsaton parameter C determnes the trade-off between the model capacty, reflected n the flatness of the approxmated functon, and the amount to whch devatons larger then the ε-nsenstve tube are tolerated [13]. A larger value for C reduces the error contrbuton but yelds a more complex forecastng functon that s more lkely to overft on the tranng data [18]. Hence t appears reasonable to evaluate parameters of C between a very small lower bound to create SVR-models wth smple, flat functons to handle strong nose and a large upper bound to also consder SVRmodels that descrbe more complex tme seres structures. In other experments, Chang and Ln used parameter bounds between e -8 C e 8 [23] whle Hsu and Ln used bounds between 2-2 C 2 12 [30]. To follow an exhaustve approach, we set the lower bound to C l =2-10 and the upper bound to C l =2 16, exceedng the parameter range of prevous experments n order to provde the capablty for a suffcent trade off for the dfferent tme seres patterns. Ths mplements an exponental grd wth 36 steps of to evaluate the parameter values of C=[2-10,2-9,5,,2 16 ]. The ε-parameter controls the sze of the ε-nsenstve tube and hence the number of support vectors and the error contrbutons of observatons lyng outsde t [9, 15]. As ε corresponds to the level of nose n a tme seres, large values of ε allow an approxmaton of the structure of the underlyng functonal relatonshp of a tme seres wth hgh nose as opposed to overfttng to the nose. Chang and Ln [22] use margn values of e -8 ε e -1, wth Ln and Weng usng smlar margns [24]. We extend these search spaces and use a lower margn of 2-8 wth an upper margn of 2 0. We use exponental grd steps of , evaluatng 32 parameter values of ε=[2-8,2-7,75,,2 0 ] for dfferent nose. The kernel parameter γ defnes the wdth of the kernel to reflect the range of the tranng data n feature space and therefore the ablty of an SVR to adapt to the data [17, 18, 31]. Chang and Ln [22] used parameter bounds of e -8 γ e 8

4 and Ln and Weng [24] used bounds of 2-8 γ 2 1. We select an exponental grd wth steps of 2 0.5, evaluatng 30 parameter values of γ=[2-12,2-11,5,,2 0 ] to provde feasble kernel parameters for the scaled tme seres data. In total we evaluate 43,725 parameter combnatons for each tme seres. As a grd search of ths magntude s a tme ntensve approach of parameter selecton, we reduce the tranng tme by applyng a shrnkng technque to speed up the decomposton used to solve the SVR optmzaton problem. It teratvely removes bounded components, so that reduced problems are solved [28, 30]. See Fan Chen and Ln for detals [32]. To summarse, a total of 4,853,475 ε-svr canddate models are computed on the 111 tme seres usng YALE [33] and the LIBSVM lbrares [28] D. Selecton of SVR canddate models Dependng on the combnaton of model parameters, a SVR s capable of approxmatng the underlyng data generatng process of a tme seres to dfferent degrees of accuracy, permttng overfttng to the tranng data though the combnaton of sub-optmal parameters and therefore lmtng ts ablty to generalze on unseen data [4], see fgure 3. Fg.3. The dfference between overfttng and generalzaton Hence the selecton of a robust SVR canddate for each tme seres requres partcular attenton. To select the best SVR canddate model from the dfferent parameter setups, whch are 43,725 per tme seres n ths study, each tme seres s splt nto two subsets of 65% tranng data and 35% valdaton data [34]. Consderng the length of the seres the valdaton set s selected to roughly match the undsclosed test set n length, servng as a frst estmate of a quas-outof-sample accuracy. Each canddate ε-svr s traned exclusvely on the tranng data and s selected on ts valdaton dataset. As only a short valdaton dataset s used for selectng the best canddate model for that tme seres, overfttng on the valdaton set frequently occurs f the valdaton subset does not adequately represent the true data generatng process, whch cannot be expected from small data sub-samples. Multple approaches are feasble to avod overfttng to the valdaton data set n model selecton and to derve an unbased estmator on unseen data. Comprehensve methods for data sub-samplng may be consdered, ncludng k-fold cross valdaton usng dfferent numbers of data folds or leave-one-out cross valdaton [35] To adhere to the naïve approach, whle avodng the grave mstake of selecton of the best canddate model on the tranng data tself, we compute only sngle cross-valdaton errors and select the best model on the prefxed valdaton set. So all SVR canddates are parametersed exclusvely on the tranng set, whle the forecastng capablty of the models s evaluated on the valdaton set and the canddate model wth the lowest valdaton error s selected [36, 37]. Emprcal smulaton experments have proven that error measures play an mportant role n calbratng and refnng, model selecton and ex post evaluaton of competng forecastng models n order to determne the compettve accuracy and rank canddate models [36, 38]. Although they should be selected wth care, we apply the quadratc error measure of the root mean squared error (RMSE), weghtng each error devaton by the quadratc dstance usng: 2 1 RMSE = ( ) e t (4) n t = 1 Usng quadratc error measures emphasses the nfluence of large forecast errors over small ones, e.g. from outlers, and should normally be avoded n the evaluaton of model performance, but accordng to Armstrong and Collopy [38] practtoners and academcans used the RMSE frequently to draw conclusons about forecastng methods. Furthermore, they are frequently used due to ther relaton wth conventonal least-squares-estmators and ther mathematcal smplcty. Ths s also noteworthy, as the selecton crtera dverges from the fnal forecastng error metrc n the NN3 competton, whch s usng a symmetrc mean absolute percent error (SMAPE) [38, 39]: n 1 y ˆ t yt SMAPE = ( 100) n (5) y + y ˆ /2 ( ) t= 1 t t The model wth the lowest RMSE on forecastng 18 t+1 step-ahead forecasts on the valdaton set s selected and appled to predct the next 18 data ponts as multple-stepahead forecasts t+1, t+2,, t+18 on the NN3 competton data sets. It s apparent, that ths gves rse to another msmatch, as a method may show adequate accuracy on forecastng one step nto the future, yet another set of parameters may perform better n forecastng multple steps ahead. As ths s commonly not algned n prevous studes, we comply wth ths malpractce n the naïve methodology, ntroducng further potental for msspecfcaton errors. No true observatons for the fnal 18 data ponts of the NN3 competton are avalable, so no evaluaton of out-ofsample accuracy may be conducted. More thorough evaluatons of the naïve methodology would be feasble by splttng the avalable data nto tranng, valdaton and test set, but these are not conducted due to the obvous suboptmalty of the naïve approach. Ths lmts the followng nvestgaton to an analyss of correlaton between tranng and valdaton data and the ranges of optmal parameters.

5 IV. EXPERIMENTAL RESULTS To derve nsght about the potental qualty of the estmated SVR models, we nvestgate the one-step-ahead predctons for all tme seres on tranng and valdaton data usng the SMAPE. Table 1 shows the SMAPE for the 111 NN3 tme seres ncludng the 11 tme seres from the reduced dataset from NN100 to NN101. An analyss of the errors on tranng and valdaton sets ndcates varous problems of hgh errors, overfttng n the tranng process as ndcated by sgnfcantly smaller errors on the tranng data then on the valdaton data set, and overfttng to the valdaton set, ndcated by sgnfcantly smaller errors on the valdaton set then on the tranng set. The numercal analyss therefore confrms that varous msspecfcatons may have occurred, whch however cannot be attrbuted to a partcular model choce or the uncontrollable structure dataset. To support the numercal evdence of the expected low valdty and relablty of the naïve benchmark approach, we conduct a vsual nspecton of the hghlghted tme seres to valdate the assumptons. The vsual nspecton confrms that the SVR models for tme seres NN009, NN010, NN019 and NN059 overft on the tranng data, showng lmted generalsaton through sgnfcantly lower accuracy on the valdaton set. Tme seres NN40 to NN49 show extreme nose and no apparent generalsaton of the underlyng structure. For most of these tme seres the SVR model s ether very flat or sgnfcantly overftted to the nose. Tme seres NN022, NN037, NN043, NN054, NN063 and NN069 show a sgnfcant error ncrease on the valdaton set, whch can be explaned by vsual apparent outlers n the valdaton set. Tme seres NN093 and NN009 show a structural break, explanng the hgh forecastng error. The SVR models for tme seres NN027, NN029, NN043 and NN045 are very flat and hence explan the lmted ft to the nosy tme seres. V. CONCLUSIONS We compute a naïve heurstc, makng use of most frequent mstakes n SVR modelng for tme seres predcton n order to establsh a lower bound for support vector accuracy n the NN3 forecastng competton. The naïve heurstc evaluates an extensve grd search of 43,725 combnatons of cost parameters, epslon-parameters and kernel parameters for each tme seres, calculatng a total of 4,853,475 ε-svr canddate models. In amng for a lower bound, we neglect the necessty to dentfy a sgnfcant nput vector per tme seres, evaluate dfferent scalng schemes, evaluate dfferent kernel functons, control for overfttng n model selecton form the valdaton data usng k-fold cross-valdaton, conductng model selecton and evaluaton on a representatve error metrc for the ex post evaluaton of the performance or the true cost of the decson, and computng and evaluatng one step ahead predctors nstead of multple-step-ahead predctors as requred n the fnal test evaluaton. The naïve heurstc dentfes a set of parameters for each of the 111 tme seres, whch are subsequently used to forecast the next 18 steps nto the future for unseen data. Whle we hope to demonstrate the general ablty of SVR to forecast lnear and nonlnear tme seres wth seasonal and non-seasonal patterns, the dscusson of the naïve heurstc methodology also ams at drawng attenton to the most common mstakes n SVR as well as neural network model buldng. Further research, systematc evaluatons of forecastng accuracy on multple emprcal tme seres are requred to establsh a vald, relable and robust methodology for automatc SVR model buldng. Untl then, the naïve grd search heurstc may serve as a warnng benchmark whch ptfalls to avod n SVR model buldng. TABLE 1 SMAPE PREDICTION ERROR ON TRAINING AND VALIDATION SET OR THE 111 SVR MODELS ON THE NN3 COMPETITION DATASET Seres Tran Vald Seres Tran Vald Seres Tran Vald Seres Tran Vald Seres Tran Vald Seres Tran Vald NN NN NN NN NN NN ,49 NN NN NN NN NN NN ,50 NN NN NN NN NN NN ,46 NN NN NN NN NN NN ,40 NN NN NN NN NN NN ,70 NN NN NN NN NN NN ,38 NN NN NN NN NN NN ,21 NN NN NN NN NN NN ,35 NN NN NN NN NN NN ,54 NN NN NN NN NN NN ,70 NN NN NN NN NN NN ,82 NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN Hgh error e, wth e vald, e tran >x; Overfttng on the tranng set, wth e vald >e tran ; Overfttng on the Valdaton set for model selecton, wth e vald <e tran

6 ACKNOWLEDGMENT To ensure objectve and true ex ante results, all computatons were conducted by the second author, wthout access to the undsclosed NN3 test set data at any tme. None the less, the paper s exempt from offcally partcpatng n the award of the NN3 competton. REFERENCES [1] K.-P. Lao; and R. Fldes, "The Accuracy of a Procedural Approach to Specfyng Feedforward Neural Networks for Forecastng," Computers & Operatons Research, vol. 32, pp , [2] G. Zhang, B. E. Patuwo, and M. Y. Hu, "Forecastng wth Artfcal Neural Networks: The State of the Art," Internatonal Journal of Forecastng, vol. 14, pp , [3] C.-H. Wu, C.-C. We, D.-C. Su, M.-H. Chang, and J.-M. Ho, "Travel Tme Predcton wth Support Vector Regresson," presented at IEEE Intellgent Transportaton Systems Conference, [4] A. J. Smola; and B. Schölkopf, "A Tutoral on Support Vector Regresson," Statstcs and Computng, vol. 14, pp , [5] K.-R. Müller, A. J. Smola, G. Rätsch, B. Schölkopf, J. Kohlmorgen, and V. Vapnk, "Predctng Tme Seres wth Support Vector Machnes," n Advances n Kernel Methods Support Vector Learnng, B. Schölkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambrdge: MIT Press, 1999, pp [6] H. Yang, K. Huang, L. W. Chan, K. C. I. Kng, and R. M. Lyu, "Outlers Treatment n Support Vector Regresson for Fnancal Tme SeresPredcton"." Computer Scence, vol. 3316, pp , [7] S. F. Crone, S. Lessmann, and S. Petsch, "Forecastng wth Computatonal Intellgence - An Evaluaton of Support Vector Regresson and Artfcal Neural Networks for Tme Seres Predcton," presented at 2006 IEEE WCCI, Vancouver (Canada), [8] V. N. Vapnk, "An Overvew of Statstcal Learnng Theory," IEEE Transactons on Neural Networks, vol. 10, pp , [9] N. Crstann; and J. Shawe-Taylor, An Introducton to Support Vector Machnes and other kernel-based Learnng Methods. Cambrdge (Unted Kngdom): Cambrdge Unversty Press, [10] M. Anthony; and N. Bggs, Computatonal Learnng Theory. Cambrdge (Unted Kngdom): Cambrdge Unversty Press, [11] M. Wellng, "Support Vector Regresson," Department of Computer Scence, Unversty of Toronto, Toronto (Kanada) [12] J. B; and K. P. Bennett, "A Geometrc Approach to Support Vector Regresson," Neurocomputng, vol. 55, pp , [13] A. Smola, "Regresson Estmaton wth Support Vector Learnng Machnes," Technsche Unverstät München, [14] S. R. Gunn, "Support Vector Machnes for Classfcaton and Regresson," Unversty of Southampton, Techncal Report. Image, Speech and Intellgent Systems Group January [15] B. Schölkopf, "Support Vector Learnng." Berln: Technsche Unverstät, [16] H. Yang, I. Kng, and L. Chan, "Non-Fxed and Asymmetrcal Margn Approach to Stock Market Predcton Usng Support Vector Regresson," presented at 9th Conference on Neural Informaton Processng, Sngapore (Malaysa), [17] C. J. C. Burges, "A Tutoral on Support Vector Machnes for Pattern Recognton," n Data Mnng and Knowledge Dscovery, vol. 2, U. Fayyad, Ed. Boston (U.S.A.): Kluwer Academc, 1998, pp [18] V. Cherkassky; and Y. Ma, "Practcal Selecton of SVM Parameters and Nose Estmaton for SVM Regresson," Neural Networks, vol. 17, pp , [19] S. F. Crone, S. Lessmann, and S. Petsch, "Parameter Senstvty of Support Vector Regresson and Neural Networks for Forecastng," presented at Internatonal Conference on Data Mnng, Las Vegas (U.S.A.), [20] J. B. Gao, S. R. Gunn, C. J. Harrs, and M. Brown, "A probablstc framework for SVM regresson and error bar estmaton," Machne Learnng, vol. 46, pp , [21] W. Chu, S. S. Keerth, and C. J. Ong, "Bayesan Support Vector Regresson Usng a Unfed Loss functon," IEEE Transactons on Neural Networks, vol. 15, pp , [22] M.-W. Chang; and C.-J. Ln, "Leave-One-Out Bounds for Support Vector Regresson Model Selecton," Neural Computaton, vol. 17, pp , [23] C.-W. Hsu, C.-C. Chang, and C.-J. Ln, "A Practcal Gude to Support Vector Classfcaton," Natonal Tawan Unversty, Tape, [24] C.-J. Ln; and R. C. Weng, "Smple probablstc predctons for support vector regresson," Natonal Tawan Unversty, Tape [25] M. Momma; and K. P. Bennett, "A pattern search method for model selecton of support vector regresson," presented at Second SIAM Internatonal Conference of Data Mnng, [26] J. T. Kwok; and I. W. Tsang, "Lnear Dependency Between epslon and the Input Nose n epslon -Support Vector Regresson," IEEE Transactons on Neural Networks, ICANN 2001, pp , [27] A. Smola, N. Murata, B. Schölkopf, and K.-R. Müller, "Asymptotcally optmal choce of epslon-loss for support vector machnes," presented at Proceedng of the Internatonal Conference on Artfcal Neural Networks, [28] C.-C. Chang; and C.-J. Ln, "LIBSVM: a Lbrary for Support Vector Machnes," Natonal Scence Councl of Tawan, Tape (Tawan) 17. Aprl [29] U. v. Luxburg, O. Bousquet, and B. Schölkopf, "A Compresson Approach to Support Vector Model Selecton," Journal of Machne Learnng Research, vol. 5, pp , [30] C.-W. Hsu; and C.-J. Ln, "A Comparson of Methods for Multclass Support Vector Machnes," IEEE Transactons on Neural Networks, vol. 13, pp , [31] C.-C. Hsu, C.-H. Wu, S.-C. Chen, and K.-L. Peng, "Dynamcally Optmzng Parameters n Support Vector Regresson: An Applcaton of Electrcty Load Forecastng," presented at 39th Internatonal Conference on System Scences, [32] R.-E. Fan, P.-H. Chen, and C.-J. Ln, "Workng Set Selecton Usng Second Order Informaton for Tranng Support Vector Machnes," Journal of Machne Learnng Research, vol. 6, pp , [33] I. Merswa, M. Wurst, R. Klngenberg, M. Scholz, and T. Euler, "YALE: Rapd Prototypng for Complex Data Mnng Tasks," presented at Proceedng of the 12th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, [34] P. Cunnngham, "Overfttng and Dversty n Classfcaton Ensembles based on Feature Selecton," Trnty College Dubln, Dubln (Ireland), Computer Scence Techncal Report: TCD-CS , [35] D. Optz; and R. Macln, "Popular Ensemble Methods: An Emprcal Study," Journal of Artfcal Intellgence Research, vol. 11, pp , [36] S. Makrdaks, S. C. Wheelwrght, and R. J. Hyndman, Forecastng Methods and Applcatons, 3 ed. New York: John Wley & Sons, [37] A. Foka, "Tme Seres Predcton Usng Evolvng Polynomal Neural Networks," n Insttute of Scence and Technology. Manchester (Unted Kngdom): Unversty of Manchester, [38] J. S. Armstrong; and F. Collopy, "Error Measures for Generalzng About Forecastng Methods: Emprcal Comparsons," Internatonal Journal of Forecastng, vol. 8, pp , [39] R. J. Hyndman; and A. B. Koehler, "Another Look at Measures of Forecast Accuracy," Monash Unversty, Workng Paper 13/05, 20 Ma 2005.