Mam, Florda, U.S.A., Aprl 6-8, 7 A COMPARISON OF TWO METHODS FOR FITTING HIGH DIMENSIONAL RESPONSE SURFACES Marcelo J. Colaço Department of Mechancal and Materals Eng. Mltary Insttute of Engneerng Ro de Janero, RJ, Brazl colaco@asme.org George S. Dulkravch Department of Mechancal and Materals Eng. Florda Internatonal Unversty Mam, Florda, USA dulkrav@fu.edu Debass Sahoo Infnte Cloud, LLC 48 Wall St. New York, NY dsahoo@nfntecloud.com ABSTRACT In ths paper we compare two methodologes to nterpolate lnear as well as hghly non-lnear functons n multdmensonal spaces havng up to 5 dmensons. Two methodologes are compared: the Radal Bass Functon (RBF) and the Wavelet based Neural Network (WNN). The accuracy, robustness, effcency, transparency and conceptual smplcty are dscussed. Based on the extensve testng performed on 3 test functons, the RBF approach seems easy to mplement computatonally and results are better nterpolaton for hgher dmensonal spaces than the WNN, requrng lesser computng tme. INTRODUCTION The use of RBFs followed by collocaton, a technque frst proposed by Kansa [Kansa, 99], after the work of Hardy [Hardy, 97] on multvarate approxmaton, s now becomng an establshed approach. Varous applcatons to problems n mechancs have been made n recent years see, for example Letão [Letão, ; Letão, 4]. Kansa's method (or asymmetrc collocaton) starts by buldng an approxmaton to the feld of nterest (normally dsplacement components) from the superposton of RBFs (globally or compactly supported) convenently placed at ponts n the doman and/or at the boundary. The unknowns (whch are the coeffcents of each RBF) are obtaned from the approxmate enforcement of the boundary condtons as well as the governng equatons by means of collocaton. Usually, ths approxmaton only consders regular RBFs, such as the globally supported multquadrcs or the compactly supported Wendland functons [Wendland, 998]. RBF may be classfed nto two man groups:. The globally supported ones namely the x x + c, where c j s a multquadrcs (MQ, ( ) shape parameter), the nverse multquadrcs, thn plate splnes, Gaussans, etc;. The compactly supported ones such as the Wendland [998] famly (for example, ( r) n + + p( r) where p(r) s a polynomal and ( r ) n + s for r greater than the support). Let us suppose that we have a functon of L varables x, =,,L. The RBF model used n ths work has the followng form N M L () s x = f x = α φ x x + β p x + β ( ) ( ) ( ) ( ) j j j, k k j= k= = j where x={x,,x,,x L ) and f(x) s known for a seres of ponts x. Here, p k (x ) s one of the M terms of a gven bass of polynomals [Buhmann, 3]. Ths approxmaton s solved for the α j and β,k unknowns from the system of N lnear equatons, subject to the condtons (for the sake of unqueness) N j= N j= N j= α p ( x ) j k α p ( x ) j k L α = j = = (.a) (.b) In ths work, the polynomal part of Eq. () was taken as k pk ( x ) = x (3) and the multquadrcs radal functon ( x x ) ( x x ) c j j j φ = + (4)
Mam, Florda, U.S.A., Aprl 6-8, 7 was used wth the shape parameter c j whch s kept constant as /N. Thus, a polynomal of order M s added to the radal bass functon. M was lmted to an upper value of. After nspectng Eqs. ()-(4), one can easly check that the fnal lnear system has [(N+M*L)+] equatons. The second method used for fttng hgh dmensonal functons was the Wavelets based Neural Network (WNN) model presented by Sahoo and Dulkravch [6] wth 5 neural subnets. Tranng the WNN for response surface generaton was done usng a random sequence dataset of Sobol and Levtan [976]. Typcally, the mother wavelet used n the WNN s Mexcan Hat wavelet gven by ψ x / (5) ( x) = π 4 ( x ) exp 3 Gaussan wavelets were also used along wth ths mother wavelet to construct the WNN. For each node of the WNN, genetc algorthm was used to search the best Mexcan Hat wavelet and the best Gaussan wavelet. The one havng a lower norm of resdue after performng multple lnear regresson was selected and used n the WNN archtecture. TEST FUNCTIONS In order to test the accuracy of the RBF model proposed, 3 test cases were used, representng lnear and non-lnear problems wth up to 6 varables. These test cases, defned as problems to 3 were selected by Jn et al. [] n a comparatve study among dfferent knds of metamodels. Such problems were selected from a collecton of 395 problems (actually 96 test cases), proposed by Hock and Schttkowsk [98] and Schttkowsk [987]. The frst problems do not have random errors added to the orgnal functon, whle the problem no. 3 has a nose added wth the followng form ε ( x, x ) = σ r (6) where σ s the standard devaton and r s a random number wth Gaussan dstrbuton and zero mean. In accordance wth havng multple metamodelng crtera, the performance of each metamodelng technque s measured from the followng aspects [Jn et al., ]. Accuracy the capablty of predctng the system response over the desgn space of nterest. Robustness the capablty of achevng good accuracy for dfferent problem types and sample szes. Effcency the computatonal effort requred for constructng the metamodel and for predctng the response for a set of new ponts by metamodels. Transparency the capablty of llustratng explct relatonshps between nput varables and responses. Conceptual Smplcty ease of mplementaton. Smple methods should requre mnmum user nput and be easly adapted to each problem. For accuracy, the goodness of ft obtaned from tranng data s not suffcent to assess the accuracy of newly predcted ponts. For ths reason, addtonal confrmaton samples are used to verfy the accuracy of the metamodels. To provde a more complete pcture of metamodel accuracy, three dfferent metrcs are used: R Square, Relatve Average Absolute Error (RAAE), and Relatve Maxmum Absolute Error (RMAE) [Jn et al., ]. a) (R) n ( y yˆ ) = MSE R = = n varance ( y y) = (7) where ŷ s the correspondng predcted value for the observed value y ; y s the mean of the observed values. Whle MSE (Mean Square Error) represents the departure of the metamodel from the real smulaton model, the varance captures how rregular the problem s. The larger the value of R, the more accurate the metamodel. b) Relatve Average Absolute Error (RAAE) RAAE = n = y yˆ n STD (8) where STD stands for standard devaton. The smaller the value of RAAE, the more accurate the metamodel. c) Relatve Maxmum Absolute Error (RMAE) max( y yˆ y yˆ,,..., yn yˆn ) RMAE = (9) STD Large RMAE ndcates large error n one regon of the desgn space even though the overall
Mam, Florda, U.S.A., Aprl 6-8, 7 accuracy ndcated by R and RAAE can be very good. Therefore, a small RMAE s preferred. However, snce ths metrc cannot show the overall performance n the desgn space, t s not as mportant as R and RAAE. RESULTS The RBF model presented here was compared WNN method for the 3 selected test cases. Table gves the number of tranng ponts, testng ponts, mnmum and maxmum value of each test functon, as well as the standard devaton and average value of each test functon. Table Parameters for the 3 test functons Number of tranng and testng ponts Standard Non Mnmum Maxmum Average Tranng devaton Testng Lnearty value of f value of f value of f PB # # Vars Scarce Small Large of f PB 3 98 Hgh -. 9.5 8.9 8.6 PB 3 98 Low -77.43-539.36 87.4-46.8 PB3 3 98 Hgh -374.6 -.4 456.73-34.84 PB4 3 98 Low 58.78 4779..7 85.5 PB5 6 48 6 459 Low 736.99 9568.8 794.7 696.73 PB6 9 Hgh 98.94 87.53 7.75 7.73 PB7 9 Hgh -.6.6.5. PB8 9 Low -.5 8.9.8.66 PB9 3 7 5 Hgh.5 46369.38 57.3 84. N/A PB 3 7 5 Hgh 3.4 3.84 7.7 9.3 PB 3 7 5 Low -489.97-4749.4.6-479.4 PB 9 Low -.7 3.75 5.74.5 PB3 9 Low -54.77 77.8 3.94.6 Three methodologes were used to solve the lnear algebrac system resultng from Eqs. ()- (4): LU decomposton, SVD and the Generalzed Mnmum Resdual (GMRES) teratve solver. When the number of equatons was small (less than 4), the LU solver was used. However, when the number of varables ncreased over 4, the resultng matrx becomes too ll-condtoned and the SVD solver had to be used. For more than 8 varables, the GMRES teratve method wth the Jacob precondtoner was used. In order to check the accuracy of the metamodel when dfferent samples were employed, three dfferent sets of tranng ponts were used, as suggested by Jn et al. []. Intally, the accuracy of the RBF expanson was tested for a large set of tranng ponts as defned n Table. Fgure shows the results for dfferent orders of the polynomal part of the RBF expanson presented n Eqs. ()-(3). From ths fgure, one can see that by ncreasng the order of the polynomal the results become much better, except for the problem no. 9, where the R metrc decreases when M ncreases. In fact, for all problems, except no., the RBF expanson acheves the R metrc over.9, showng a very good global accuracy. For a large set of tranng ponts, the man concluson s that a hgh polynomal order should be used n order to acheve hgh accuracy..8.6.4. PB PB PB3 PB4 PB5 PB6 PB7 PB8 PB9 PB PB PB PB3 M= M= M=3 M=4 M=5 M=6 M=7 M=8 M=9 M= Fgure Influence of the polynomal order on the R metrc for a large set of tranng ponts Next, the same comparson was made for a small number of tranng ponts, as defned n Table. Fgure shows the results of the R metrc for ths comparson. It can be observed that problem no. s almost nsenstve to the order of the polynomal, except for M = 9. Problem no. s also nsenstve except for M greater than 8, where the performance drops rapdly. Problems no. and 4 are nsenstve to the value of M, whle problem number 7 shows a small decrease of the R value for hgh order polynomals. Problem no. 3 shows an ncrease n the R values
Mam, Florda, U.S.A., Aprl 6-8, 7 when M ncreases, just as n the case wth a large set of tranng ponts. However, problems no. 5, 6, 9, and 3 show a drop n the R value when the polynomal order s ncreased so that some metrcs were way below zero. From ths fgure, n order to keep the method robust, we conclude that, f the number of tranng ponts s small, the order of the polynomal should be kept small..8.6.4. PB PB PB3 PB4 PB5 PB6 PB7 PB8 PB9 PB PB PB PB3 M= M= M=3 M=4 M=5 M=6 M=7 M=8 M=9 M= Fgure Influence of the polynomal order on the R metrc for a small set of tranng ponts Fgure 3 shows the comparson of the R metrcs for several polynomal orders for a scarce set of tranng ponts. Only problems no. to 5 were tested for a scarce set of tranng ponts, as suggested by Jn et al. []. Problems no. and 3 have the same behavor as for a small set of tranng ponts. However, problem no. rapdly drops ts value of R for a hgh polynomal order. Agan, the mnmum value of the scale was lmted to zero, because some of the R values were way below zero. The performance of problem no. 5 oscllates when M s vared. Agan, we concluded that for a scarce number of tranng ponts, a lower polynomal order should be used n order to keep the method more robust..8.6.4. PB PB PB3 PB4 PB5 M= M= M=3 M=4 M=5 M=6 M=7 M=8 M=9 M= Fgure 3 Influence of the polynomal order on the R metrc for a scarce set of tranng ponts Next, the results obtaned wth a RBF polynomal of order usng a large number of tranng ponts and the results obtaned wth a polynomal of order for small and scarce sets of tranng ponts were compared wth the results obtaned by usng WNN method [Sahoo and Dulkravch, 6]. Agan, only problems no. to 5 were tested for a scarce set of tranng ponts, as suggested by Jn et al. []. The results for the R metrc are presented n Fg. 4, where one can see that the RBF formulaton performed better than the WNN for a scarce number of tranng ponts for problems no. and. For problem no. 4 the value of R s a lttle small for the RBF when compared to WNN. For problems no. 3 and 5 the RBF performed qute poorly, whle the WNN was able to obtan some results. For a small set of tranng ponts, the RBF was better than the WNN for problems no., 7, 8, 9, and 3, whle the WNN performed better for problems no. 3, 5, and. The performance was practcally the same for problems no., 4, 6 and. For a large number of tranng ponts, the WNN performed better for problems no. 9 and, whle the RBF had a better performance for problems no., 3 and 3. For problems no., 4, 5, 6, 7, 8, and the accuracy of both methods was almost the same. Fgure 5 shows the comparsons of RAAE for all 3 test functons, both for RBF and WNN. Recall that for ths metrc, lower values are better than hgh values. For a scarce set of tranng ponts, the RBF performed better for problem no., whle the WNN was better for problems no., 3 and 4. For problem no., RAAE values for both of the methods were comparable.
Mam, Florda, U.S.A., Aprl 6-8, 7.8.6.4. PB PB PB3 PB4 PB5 PB6 PB7 PB8 PB9 PB PB PB PB WNN (Scarce) RBF (Scarce) WNN (Small) RBF (Small) WNN (Large) RBF (Large) Fgure 4 R metrc for WNN and RBF It s nterestng to note that the R value for problems no. 3 and 4 (see Fg. 4) were bad when the RBFs were used. However, the RAAE metrcs for these two problems are better when compared to the WNN. Actually, the RAAE metrc for problem no. 3 s approxmately 5% greater for RBF than for WNN, whle the R metrcs for ths problem was under zero for RBF. For small set of tranng ponts, the RBF performed better for problems no.,, 6, 7, 8, 9, and 3, whle the WNN performed better for problems no. 3, 4, 5 and. The values of RAAE for both methods were almost equal for problem no.. For a large set of tranng ponts, the RBF performed better than the WNN, except for problems no. 5 and. Thus, as t was observed n the R metrc, for large and small sets of tranng ponts, the RBF was better than the WNN, whle for a scarce number of tranng ponts, the WNN performed better. RAEE.8.6.4. PB PB PB3 PB4 PB5 PB6 PB7 PB8 PB9 PB PB PB PB WNN (Scarce) RBF (Scarce) WNN (Small) RBF (Small) WNN (Large) RBF (Large) Fgure 5 RAAE metrc for WNN and RBF Fnally, Fg. 6 shows the results for the relatve maxmum absolute error (RMAE) for all 3 test functons. A hgh value of the RMAE means a bad local estmate. For a scarce set of tranng ponts, the general trend s very close to the prevous one, related to the RAAE metrc, showng a better performance for RBF n the problem no., whle the WNN performed better for problems no. 3, 4 and 5. However, for problem no., the RBF performed much better than WNN, ndcatng that the WNN had some naccurate local estmates. For a small set of tranng ponts, the RBF performed better for problems no., 7, 8 and, whle the WNN performed better for problems no. 3, 5, 6, and 3. For problems no., 4, 9 and the values of RMAE were close to each other. For a large set of tranng ponts, the RBF was better for problems no.,, 3, 4, 7, 8, 9, and 3, whle the WNN was better for the problems no. 5,. For problems no. 6 and the performance was practcally the same for both of them. RMAE 7 6 5 4 3 PB PB PB3 PB4 PB5 PB6 PB7 PB8 PB9 PB PB PB PB WNN (Scarce) RBF (Scarce) WNN (Small) RBF (Small) WNN (Large) RBF (Large) Fgure 6 RMAE metrc for WNN and RBF
Mam, Florda, U.S.A., Aprl 6-8, 7 In order to check the robustness of the two models when nose s added, test problem no. 3 was used wth several values of σ. For ths test, tranng ponts and testng ponts were used. Fgure 7 shows how the R metrc decreases when the added nose n the orgnal functon ncreases. It s worth to note that the RBF performed better than the WNN. In fact, even for a hgh level of nose, the RBF stll shows a value of.8 for the R metrc when the order of the polynomal s equal to M =. It s qute nterestng that when no nose s added, the R metrc decreases when the order of the polynomal ncreases, whch s exactly the opposte trend to the one presented n Fg. for the functon number 3 wthout nose..8.6.4. 4 6 8 4 6 8 Nose RBF (M=) RBF (M=) WNN Fgure 7 Influence of nose on the R metrc Fnally, a test case wth progressvely large number of varables was proposed. For ths test case, test functon no. was chosen wth 5, and 4 tranng ponts and testng ponts for varous problem dmensons. Fgure 8 shows the results for the R metrcs when the RBF was used wth a polynomal of order M =. It s surprsng that the RBF s able to mantan a hgh level of R, even for a problem wth 5 varables. It s worth to note that when the number of tranng ponts decreases, the value of R also decreases, but not sgnfcantly..8.6.4. 3 4 5 6 7 8 9 3 4 5 # of varables RSQ (4 tranng) RSQ ( tranng) RSQ (5 tranng) Fgure 8 R results for a large number of varables (RBF wth M=) Fgure 9 shows the results for the same test case presented n Fg. 8, but now for a polynomal of order M =. Snce t was observed n Fgs. -3, the problem no. s nsenstve to the order of the polynomal for scarce, small and large set of tranng ponts. It s nterestng to note the smlar results when a hgh order polynomal was used (see Fg. 8). In fact, the results are even better for 5 varables...8.6.4. 3 4 5 6 7 8 9 3 4 5 # of varables RSQ (4 tranng) RSQ ( tranng) RSQ (5 tranng) Fgure 9 R results for a large number of varables (RBF wth M=)
Mam, Florda, U.S.A., Aprl 6-8, 7 CPU Tme (s)..... RSQ (4 tranng) RSQ ( tranng) RSQ (5 tranng) 3 4 5 6 7 8 9 3 4 5 # of varables Fgure CPU tme for a large number of varables (RBF wth M=) The reason for ths s that the lnear system resultng from the RBF approxmaton has [(N+M*L)+] equatons, where N s the number of tranng ponts, L s the number of varables and M s the order of the polynomal. Thus, when a hgh order polynomal s used, the matrx becomes too large and mght become more llcondtoned. Fnally, Fg. shows the computatonal tme requred to run ths test case. It s qute surprsng that the computatonal tme was lower than seconds for all test cases, meanng that the RBF approxmaton s very fast. The code was wrtten n Fortran 9 and the CPU was an Intel T3.66Ghz (Centrno Duo) wth Gb RAM. Fgure shows the results for test problem no. for a large number of varables, usng the WNN. One can see that the accuracy, gven by the R metrc, decreases rapdly when usng tranng ponts. Also, for 4 tranng ponts, the R goes to a negatve value for more than varables, whle the RBF (see Fgs. 8 and 9) mantans good accuracy even when there are 5 varables. Fgure shows the computatonal tme requred by the WNN where one can notce the hgh computatonal cost. In fact, for varables, the tme requred was about 4 hours, whle for 3 varables t was more than hours. The code for the WNN was wrtten n Matlab 7..4 and the CPU was an Intel T3.66Ghz (Centrno Duo) wth Gb RAM. Some mprovement n the performance can be obtaned by convertng ths code to Fortran9 or C++ and ths should be nvestgated. However, the computatonal cost for the WNN for a problem wth 3 varables and 4 tranng ponts, even wth dfferent languages (Matlab and Fortran9) was approxmately 6 tmes greater than for the RBF. Fnally, the same problem was run agan usng WNN wth only one sub-net and 4 tranng ponts, usng the Mexcan Hat functon. The results are presented n Fg. 3..8.6.4. WNN (4 tranng) WNN ( tranng) 3 4 5 6 7 8 9 3 # of varables Fgure R results wth WNN for a large number of varables (WNN wth fve subnets) CPU Tme (s)...... WNN (4 tranng) WNN ( tranng) 3 4 5 6 7 8 9 3 # of varables Fgure CPU tme for a large number of varables (WNN wth fve subnets)
Mam, Florda, U.S.A., Aprl 6-8, 7.8.6.4. RSQ CPU Tme 3 4 5 6 7 8 9 3 4 5 # of varables...... CPU tme (s) Fgure 3 R results and CPU tme for a large number of varables (WNN wth one subnet) One can see that the computatonal tmes decrease when compared wth the ones presented n Fg. by a factor of fve, but the R metrcs also decreases. It s nterestng to note that, agan, after varables, the R metrcs goes to a negatve value when the WNN s used. Thus, t s not recommended to reduce the number of neural sub-nets n order to speed-up the tranng process, because the accuracy goes down. In concluson, at least for the problem no., the RBF s more robust than the WNN when a very large number of varables s used. CONCLUSIONS In ths work we performed a comparson of two nterpolaton technques for hghly non-lnear functons where large number of varables were nvolved. The RBF technque seems to be qute powerfull regardng ts accuracy and reduced CPU tme. Even when the number of varables were as large as 5, the RBF approxmaton was very fast and robust. Ths s a promsng technque for real tme nterpolatons such as target trackng or mage recognton. When only a scarce sets of data are avalable, WNN method typcally offers hgher accuracy. ACKNOWLEDGEMENTS Ths work was partally funded by CNPq (a Brazlan councl for scentfc development). The frst author s very gratefull to the fnancal support from FIU as well as the hosptally of the Dulkravch s famly durng hs stayng n Mam from September to November 6. REFERENCES Buhmann, M.D., 3, Radal Bass Functons on Grds and Beyond, Internatonal Workshop on Meshfree Methods, Lsbon. Hardy, R.L., 97, Multquadrc Equatons of Topography and Other Irregular Surfaces, Journal of Geophyscs Res., Vol. 76, pp. 95-95. Hock, W. and Schttkowsk, K., 98, Test Examples for Nonlnear Programmng Codes, Lecture Notes n Economcs and Mathematcal Systems, Vol. 87, Sprnger. Jn, R., Chen, W. and Smpson, T. W.,, Comparatve Studes of Metamodelng Technques under Multple Modelng Crtera, Proceedngs of the 8th AIAA / USAF / NASA / ISSMO Multdscplnary Analyss & Optmzaton Symposum, AIAA -48, Long Beach, CA, 6-8 Sept. Kansa, E.J., 99, Multquadrcs A Scattered Data Approxmaton Scheme wth Applcatons to Computatonal Flud Dynamcs II: Solutons to Parabolc, Hyperbolc and Ellptc Partal Dfferental Equatons, Comput. Math. Applc., Vol. 9, pp. 49-6. Letão, V.M.A.,, A Mesheless Method for Krchhoff Plate Bendng Problems, Internatonal Journal of Numercal Methods n Engneerng, Vol. 5, pp. 7-3. Letão, V.M.A., 4, RBF-Based Meshless Methods for D Elastostatc Problems, Engneerng Analyss wth Boundary Elements, Vol. 8, pp. 7-8. Sahoo, D. and Dulkravch, G. S., 6, Evolutonary Wavelet Neural Network for Large Scale Functon Estmaton n Optmzaton, th AIAA/ISSMO Multdscplnary Analyss and Optmzaton Conference, AIAA Paper AIAA- 6-6955, Portsmouth, VA, Sept. 6-8. Schttkowsk, K., 987, More Test Examples for Nonlnear Programmng, Lecture Notes n Economcs and Mathematcal Systems, Vol. 8, Sprnger. Sobol, I. and Levtan, 976, The Producton of Ponts Unformly Dstrbuted n a Multdmensonal Cube, Preprnt IPM Akad. Nauk SSSR, Number 4, Moscow, Russa. Wendland, H., 998, Error Estmates for Interpolaton by Compactly Supported Radal Bass Functons of Mnmal Degree, Journal of Approxmaton Theory, Vol. 93, pp. 58-7.