Smoothing Spline ANOVA for variable screening

Transcription

1 Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco

2 Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory Possble rankngs Test Cases Interestng Applcaton

3 SS-Anova Smoothng Splne ANOVA (SS-ANOVA) s a statstcal modelng algorthm based on a functon decomposton smlar to the classcal analyss of varance (ANOVA) decomposton and the assocated notons of man effects and nteractons. Each term man effects and nteractons exhbts the measure of ts contrbuton to the global varance (so ts relatve sgnfcance). SS-ANOVA s a sutable screenng technque for detectng mportant varables n a gven dataset

4 Unvarate case: Cubc Smoothng Splne In the smple unvarate case, the SS-ANOVA model f(x) s the soluton of ths mnmzaton problem: mn 1 n n '' 2 f f ( x ) f ( x) dx 0 The left term guarantees a good ft to the data. The rght term represents a penalty on the roughness of the model. The soluton s called Cubc Smoothng Splne, and t corresponds to the usual natural cubc splne.

5 General multvarate case In the general multvarate case, the model s the soluton of ths penalzed least square problem: mn 1 n n 1 2 f f ( x ) J( f ) The ANOVA decomposton can be bult nto the above formulaton through the proper constructon of the roughness functonal J(f). The theory behnd ths formulaton s based on the so-called reproducng kernel Hlbert space.

6 Screenng wth SS-ANOVA The functon ANOVA decomposton of the traned model, evaluated at samplng ponts, can be wrtten as: f * The relatve sgnfcance of the dfferent terms can be assessed by means of the contrbuton ndces k p k 1 * ( f k f Gven the aforementoned formulas, we obtan a sort of decomposton of unty: p k 1 k,f * f * k 2 1 * )

7 Metamodelng A RSM use the nformaton gven by a tranng database to predct the response of the system at unsampled ponts.

8 Metamodelng PROs: Tranng and evaluatng a RSM s usually less costly than runnng a real smulaton, both consderng computatonal resources and tme. CONs: The curse of dmensonalty s the man restrcton. For example the number of monomals composng a full polynomal of degree deg usng nvar varables s equal to: (deg nvar )! deg! nvar!

9 SS-Anova and Metamodelng SS-Anova s able to scan a complex and possbly large database. Once the most relevant nput varables are detected, t s possble to restrct the tranng mprovng tme and accuracy performances.

10 An Example (mmckng a true database) The ntroducton of SS-Anova n our software s lnked to a customer request havng a very pecular database. We cannot show ther data, but we bult a toy model wth smlar propertes, whch helped us n the development phase. We consder a polynomal of degree 3 n 6 varables, but we set to zero all the coeffcents of the terms nvolvng the last 3 varables.

11 An Example (mmckng a true database) We buld randomly a tranng (140 ponts) and a valdaton (60 ponts) database and we compare the performances of dfferent RSM before and after the use of SS-Anova. Interpolaton methods performed better on ths toy problem, whle Krgng model (ncludng nose) was the best n the orgnal test. Mean absolute error comparson Full tranng Krgng RBF 3.0E-5 1.0E-7 SVD 2.0E E-16 Usng SS-ANOVA The orgnal test case results n a mprovement of about 3 orders of magntude

12 Mult Objectve Optmzaton Mathematcal formulaton: When m>1 and the functons are n contrast, we speak about multobjectve optmzaton. The goal of MOO s to fnd the Pareto front composed by nondomnated ponts.

13 Game Theory and MOO

14 Game Theory and MOO SS - Anova

15 Rankng Varables We desgned two possble rankng and selecton strateges. 1) Determnstc: Normalze SS-Anova coeffcents Look for the hghest coeffcent and record ts varable-objectve assocaton. Remove all the coeffcents related to the assgned varable and terate 2) Stochastc: Normalze SS-Anova coeffcents For each varable perform roulette wheel selecton usng the dfferent coeffcents as weghts n order to assgn the varable to an objectve. SS-Anova coeffcent for varable x In both cases we check for objectves wthout assgned varables. Obj A Obj B Obj C

16 Test Cases: full separaton of varables Ths problem has a clear and unque answer to the varable-objectve partton problem, but t s a dffcult optmzaton task, snce the objectve functons exhbt several local optma. f f 1 2 k 1 2k k 1 ( x ( x 20) 0.3(sn(2 ( x 30) 0.3(sn(2 ( x 20))) 30))) k=2 # eval = 300 mn(f1+f2) Old MOGT New MOGT MOGA NSGA k=5 # eval = 1600 mn(f1+f2) Old MOGT New MOGT MOGA NSGA k=10 # eval = 6000 mn(f1+f2) Old MOGT New MOGT MOGA NSGA

17 Test Cases: shared varables KUR100 problem: f f exp x sn x 3 x 2 x 2 1 As hghlghted n the orgnal paper, ths algorthm s able to reach almost-optmal confguratons n few teratons. The stochastc varableobjectve couplng s used for ths problem.

18 Applcaton: boomerang optmzaton

19 Applcaton: boomerang optmzaton The optmzaton follows a b-level scheme, the outer loop tres to optmze the shape of the boomerang, the nner one ensures a satsfactory trajectory. Wth the new MOGT algorthm we are tryng to go further and to fully optmze the launch: maxmze range, mnmze force, loop constrant.

20