Problem efntons and Evaluaton Crtera for Computatonal Expensve Optmzaton B. Lu 1, Q. Chen and Q. Zhang 3, J. J. Lang 4, P. N. Suganthan, B. Y. Qu 6 1 epartment of Computng, Glyndwr Unversty, UK Faclty esgn and Instrument Insttute, Chna Aerodynamc Research and evelopment Center, Chna 3 epartment of Computer Scence, Cty Unversty of Hong Kong, Hong Kong & School of Computer Scence and Electronc Engneerng, Unversty of Essex, UK. 4 School of Electrcal Engneerng, Zhengzhou Unversty, Zhengzhou, Chna School of EEE, Nanyang Technologcal Unversty, Sngapore 6 School of Electrc and Informaton Engneerng, Zhongyuan Unversty of Technology, Zhengzhou, Chna b.lu@glyndwr.ac.uk, qngfu.zhang@ctyu.edu.hk, chenqn198@gmal.com langjng@zzu.edu.cn, epnsugan@ntu.edu.sg, qby1984@hotmal.com Many real-world optmzaton problems requre computatonally expensve computer or physcal smulatons for evaluatng ther canddate solutons. Often, canoncal evolutonary algorthms (EA) cannot drectly solve them snce a large number of functon evaluatons are unaffordable. In recent years, varous knds of novel methods for computatonally expensve optmzaton problems have been proposed and surrogate model asssted evolutonary algorthm (SAEA) s attractng more and more attenton. To promote research on expensve optmzaton, we propose to organze a competton focusng on small- to medum-scale (from 1 decson varables to 3 decson varables) real parameter bound constraned sngle-objectve computatonally expensve optmzaton. We encourage all partcpants to test ther algorthms on the CEC 14 expensve optmzaton test sute whch ncludes 4 black-box benchmark functons (8 popular test problems wth 1, and 3 dmensons). The partcpants are requred to send the fnal results n the format gven n the techncal report to the organzers. The organzers wll conduct an overall analyss and comparson. Specal attenton wll be pad to whch algorthm has advantages on whch knd of problems. The C and Matlab codes for CEC 14 test sute can be downloaded from the webste gven below: http://www.ntu.edu.sg/home/epnsugan/ndex_fles/cec14 1. Introducton to the 4 CEC 14 expensve optmzaton test problems 1.1 Summary of CEC 14 expensve optmzaton test problems Eght popular test functons are used. The test sutes nclude unmodal / mult-modal, contnuous / dscrete and separable / non-separable functons. All test functons are scalable and 1 decson varables, decson varables and 3 decson varables are used. Most functons are shfted and / or rotated. For a problem wth dmensons, the global optmum s shfted by o [ o, o, o ], and o 1 s randomly dstrbuted n [ 1,1]. The shfted data are
defned n shft_data_x.txt. The rotaton matrxes M are defned n M_x_.txt, where x s the number of basc functons. The test problems are summarzed n Table I. Table I. Summary of the CEC 14 expensve optmzaton test problems No. Functons mensonalty Search ranges * * f f ( x ) 1-3 Shfted Sphere functon 1,, 3 [-,] 4-6 Shfted Ellpsod functon 1,, 3 [-,] 7-9 Shfted and Rotated Ellpsod 1,, 3 [-,] functon 1-1 Shfted Step functon 1,, 3 [-,] 13-1 Shfted Ackley s functon 1,, 3 [-3,3] 16-18 Shfted Grewank s functon 1,, 3 [-6,6] 19-1 Shfted Rotated Rosenbrock s functon 1,, 3 [-,] -4 Shfted Rotated Rastrgn s 1,, 3 [-,] functon Please notce: These problems should be treated as black-box optmzaton problems and wthout any pror knowledge. Nether the analytcal equatons nor the problem landscape characters extracted from analytcal equatons are allowed to be used, except the contnuous / nteger decson varables. However, the dmensonalty and the number of avalable functon evaluatons can be consdered as known values and can be used. 1. efntons of CEC 14 expensve optmzaton test problems 1) Shfted Sphere functon f1( x ) x 1 F( x) f ( x: 1 1 1 1,1d F ( x) f ( x: 1 1,d F ( x) f ( x: 3 3 1 1,3d 1 1 Global Optmzer : (6.9,8.1) 14 1 1 1 1 14 1 1-8 6 8 6-1 -1 - - -1 1 4 - - -1 1 4 Fgure 1. 3- map for - Shfted Sphere functon
Propertes: Unmodal ) Shfted Ellpsod functon f( x) x 1 F ( x) f ( x: 1 4,1d F ( x) f ( x:,d F ( x) f ( x: 3 6,3d Global Optmzer : (-6.76,.89) 1 1 18 16 14 1 1 18 16 14 1 1 1 1-8 8-1 -1 - - -1 1 6 4 - - -1 1 6 4 Fgure. 3- map for - Shfted Ellpsod functon Propertes: Unmodal 3) Shfted and Rotated Ellpsod functon F ( x) f (M ( x): 1 7 1,1d 3,1d F ( x) f (M ( x): 8 1,d 3,d F ( x) f (M ( x): 3 9 1,3d 3,3d
Global Optmzer : (-6.76,.89) 18 1 1 - -1-1 - - -1 1 16 14 1 1 8 6 4 1 1 - - -1 1 16 14 1 1 8 6 4 Fgure 3. 3- map for - Shfted and Rotated Ellpsod functon Propertes: Unmodal 4) Shfted Step functon f3( x) ( x. ) 1 F ( x) f ( x: 1 1 3 4,1d F ( x) f ( x: 11 3 4,d F ( x) f ( x: 3 1 3 4,3d Global Optmzer : (-8.81,3.64) 14 1 1-1 1 8 6 1 1 1 1 8 6-1 -1 - - -1 1 4 - - -1 1 4 Fgure 4. 3- map for - Shfted Step functon Propertes: Unmodal scontnuous ) Shfted Ackley s functon
1 1 ) exp( x cos( x )) e 1 1 F13 ( x ) f 4 ( x o,1 d ) : 1 f 4 ( x ) exp(. F14 ( x ) f 4 ( x o, d ) : F1 ( x ) f 4 ( x o,3 d ) : 3 Global Optmzer : (-1.4,-8.1) 3 1 18 16 16 14 1 14 1 18 1 1 1 1-1 8 8 6-6 4 4-3 - -1 1-3 4-3 - -4 Fgure. 3- map for - Shfted Ackley s functon Propertes: Mult-modal 6) Shfted Grewank s functon x x cos( ) 1 1 4 1 F16 ( x ) f ( x o6,1 d ) : 1 f ( x ) F17 ( x ) f ( x o6, d ) : F18 ( x ) f ( x o6,3 d ) : 3 Global Optmzer : (1.64,.81) 3 3 4 3. 4. 3 1 1 1 - -3 1. -4 - - 1. 1. -1 - - Fgure 6. 3- map for Shfted - Grewank s functon Propertes:.
Mult-modal 7) Shfted and Rotated Rosenbrock s functon 1 6 x 1 1 f ( ) (1( x x ) ( x 1) ) F x f M.48( x F x f M.48( x F x f M.48( x 7,1d 19( ) 6(,1d ( ) 1): 1 7,d ( ) 6(,d ( ) 1): 7,3d 1( ) 6(,3d ( ) 1): 3 Global Optmzer : (7.1,1.1) 14 14 1 1-1 1 8 6 1 1 1 1 8 6-1 -1 - - -1 1 4 - - -1 1 4 Fgure 7. 3- map for - Shfted and Rotated Rosenbrock s functon Propertes: Mult-modal Non-separable Havng a very narrow valley from local optmum to global optmum 8) Shfted and Rotated Rastrgn s functon 7 ( x) ( 1cos( ) 1) 1 f x x F f M.1( x F f M.1( x F f M.1( x 8,1d ( x) 7( 3,1d ( )): 1 8,d 3( x) 7( 3,d ( )): 8,3d 4( x) 7( 3,3d ( )): 3
1 1 Global Optmzer : (1.18,7.8) 1 9 8 1 1 9 8 7 6 1 7 6-4 4-1 -1 - - -1 1 3 1 - - -1 1 3 1 Fgure 8. 3- map for - Shfted and Rotated Rastrgn s functon Propertes: Mult-modal. Evaluaton crtera.1 Expermental settng: Number of ndependent runs: Maxmum number of exact functon evaluatons: o 1-dmensonal problems: o -dmensonal problems: 1, o 3-dmensonal problems: 1, Intalzaton: Any problem-ndependent ntalzaton method s allowed. Global optmum: All problems have the global optmum wthn the gven bounds and there s no need to perform search outsde of the gven bounds for these problems. Termnaton: Termnate when reachng the maxmum number of exact functon evaluatons or * * 8 the error value ( f f ( x )) s smaller than 1.. Results to record: (1) Current best functon values: Record current best functon values usng.1 MaxFES,. MaxFES,, MaxFES for each run. Sort the obtaned best functon values after the maxmum number of exact functon evaluatons from the smallest (best) to the largest (worst) and present the best, worst, mean, medan and standard 8 devaton values for the runs. Error values smaller than 1 are taken as zero. () Algorthm complexty: For expensve optmzaton, the crteron to judge the effcency s the obtaned best result vs. number of exact functon evaluatons. But the computatonal overhead on surrogate modelng and search s also consdered as a secondary evaluaton crteron. Consderng that for dfferent data sets, the computatonal overhead for a surrogate modelng method can be qute dfferent, the computatonal overhead of each problem s necessary to be reported. Often, compared to the computatonal cost on surrogate modelng, the cost on, 1 and 1 functon evaluatons can almost be gnored. Hence, the followng method s used:
a) Run the test program below: for =1:1 x=. + (double) ; x=x + x; x=x/; x=x*x; x=sqrt(x); x=log(x); x=exp(x); x=x/(x+); end Computng tme for the above=t; b) The average complete computng tme for the algorthm = T 1. The complete computng tme refers to the computng tme usng MaxFEs except that the global optmum s reached wth less than MaxFEs evaluatons. The complexty of the algorthm s measured by: T1/ T. (3) Parameters: Partcpants are requested not to search for the best dstnct set of parameters for each problem/dmenson/etc. Please provde detals on the followng whenever applcable: a) All parameters to be adjusted b) Correspondng dynamc ranges c) Gudelnes on how to adjust the parameters d) Estmated cost of parameter tunng n terms of number of FEs e) Actual parameter values used. (4) Encodng If the algorthm requres encodng, then the encodng scheme should be ndependent of the specfc problems and governed by generc factors such as the search ranges, dmensonalty of the problems, etc. () Results format The partcpants are requred to send the fnal results as the followng format to the organzers and the organzers wll present an overall analyss and comparson based on these results. Create one txt document wth the name AlgorthmName_FunctonNo. expensve.txt for each test functon and for each dmenson. For example, PSO results for test functon and =3, the fle name should be PSO 3_expensve.txt. The txt document should contan the mean and medan values of current best functon values when.1 MaxFES,. MaxFES,, MaxFES are used of all the runs. The partcpant can save the results n the matrx shown n Table II and extracts the mean and medan values.
Table II Informaton matrx for functon X Run 1 Run Run.1 MaxFES. MaxFES MaxFES Notce: All partcpants are allowed to mprove ther algorthms further after submttng the ntal verson of ther papers to CEC14. They are requred to submt ther results n the ntroduced format to the organzers after submttng the fnal verson of paper as soon as possble. Consderng the surrogate modelng for 3 dmensonal functons s often tme consumng, especally for MATLAB users, results usng 1 runs are requested for ntal submsson..3 Results template Language: Matlab 8a Algorthm: Surrogate model asssted evolutonary algorthm A Results Notce: Consderng the length lmt of the paper, only Error Values Acheved wth MaxFES are need to be lsted. Table III. Results for 1 Problem No. Best Worst Medan Mean Std F1 F4 F7 F1 F13 F16 F19 F Table IV. Results for Table V. Results for 3
Algorthm Complexty Table VI. Computatonal Complexty Problem No. T1/ T F1 F F3 F4 Parameters a) All parameters to be adjusted b) Correspondng dynamc ranges c) Gudelnes on how to adjust the parameters d) Estmated cost of parameter tunng n terms of number of FES e) Actual parameter values used.4 Sortng method The mean and medan values at the maxmum allowed number of evaluatons wll be used. For each problem, the algorthm wth the best result scores 9, the second best scores 6, the thrd best scores 3 and all the others score. Total score = 4 score (usng mean value) + 1 4 1 score (usng medan value) The top three wnners wll be announced. Specal attenton wll be pad to whch algorthm has advantages on whch knd of problems, consderng dmensonalty and problem characterstcs.