2 General Regression Neural Network (GRNN)

Size: px

Start display at page:

Download "2 General Regression Neural Network (GRNN)"

Trevor Lawson
5 years ago
Views:

1 4 Geeral Regresso Neural Network (GRNN) GRNN, as proposed b oald F. Specht [Specht 9] falls to the categor of probablstc eural etworks as dscussed Chapter oe. Ths eural etwork lke other probablstc eural etworks eeds ol a fracto of the trag samples a backpropagato eural etwork would eed [Specht 9]. The data avalable from measuremets of a operatg sstem s geerall ever eough for a backpropagato eural etwork [Specht 90]. Therefore the use of a probablstc eural etwork s especall advatageous due to ts ablt to coverge to the uderlg fucto of the data wth ol few trag samples avalable. The addtoal kowledge eeded to get the ft a satsfg wa s relatvel small ad ca be doe wthout addtoal put b the user. Ths makes GRNN a ver useful tool to perform predctos ad comparsos of sstem performace practce.

2 43. Algorthm The probablt dest fucto used GRNN s the Normal strbuto. Each trag sample, X, s used as the mea of a Normal strbuto. ( ) Y X - Ł T ( ) ( ) X - X X - X Y - Ł ł ł Eq..- The dstace,, betwee the trag sample ad the pot of predcto, s used as a measure of how well the each trag sample ca represet the posto of predcto, X. If the stace,, betwee the trag sample ad the pot of predcto s small, (- / ), becomes bg. For 0, (- / ) becomes oe ad the pot of evaluato s represeted best b ths trag sample. The dstace to all the other trag samples s bgger. A bgger dstace,, causes the term (- / ) to become smaller ad therefore the cotrbuto of the other trag samples to the predcto s relatvel small. The term Y *(- / ) for the th trag sample s the bggest oe ad cotrbutes ver much to the predcto. The stadard devato or the smoothess parameter,, as t s amed [Specht 9], s subect to a search. For a bgger smoothess parameter, the possble represetato of the pot of evaluato b the trag sample s possble for a wdeer rage of X. For a small

3 44 value of the smoothess parameter the represetato s lmted to a arrow rage of X, respectvel. Wth (Eq..-) t s possble to predct behavor of sstems based o few trag samples predct smooth mult-dmesoal curves terpolate betwee trag samples. I (Fg..-) a predcto performed b GRNN s show. The crcles represet the data pots or trag samples used to predct the sold le gog through most of these samples. The bell shaped curves are the dvdual terms of (Eq..-). Each of these curves s oe term, Y *(- / )/Σ (- / ) of the whole equato (Eq..-) used GRNN for the predcto. These terms are ormalzed ormal dstrbutos. Summg up the values of the dvdual terms at each posto elds the value of the predcto, the sold le gog through most of the data pots. The smoothess parameter was arbtrarl chose to 0..

4 45 Trag sample Predcto wth GRNN, Summato of Curves below Idvdual terms of Sum for GRNN x Fg..- GRNN wth dvdual terms cotrbutg to predcto, 0. I Chapter oe t was dscussed how Neural Networks weght the dvdual sgals of each euro dfferetl. I (Fg..-) GRNN s show a famlar represetato, a Backpropagato Neural Network was show before (Chapter ). The calculatos performed each patter euro of GRNN are (- / ), the ormal dstrbuto cetered at each trag sample. The sgals of the patter euro, gog to the eomator euro are weghted wth the correspodg values of the trag samples, Y. The weghts o the sgals gog to the Numerator Neuro are oe. Each sample from the trag data flueces ever pot that s beg predcted b GRNN.

5 46 X X X q- X q Iput Neuros Patter Neuros wth actvato fucto: (- /s ) Numerator Y Y Y Y p eomator Summato Neuros Output Neuro Y(X) Fg..- GRNN bult up a wa such that t ca be used as a parallel Neural Network

6 47. How to choose A predcto tool ca potetall be used for feed forward cotrol. Cotrollers usg a dervatve algorthm eed values ther algorthm that also clude a dervatve. Ths meas that the predcto tools have to submt a predcto that does ot ol represet the realt the precso of the value of the predcto but as well the slope of the predcto. The slope of the predcto ca eve be more mportat tha the actual value of the predcto, depedg o the algorthm a cotroller uses. epedg o the use of the predcto tool the emphass has to be put o oe of the two, precso or smoothess, or eve both. Fault detecto ad dagoss ca clude several methods, the ca clude as well algorthms usg a slope ther approach. But aga, depedg o the use of the predcto tool, ma aspects are mportat. The smoothess parameter s the ol parameter of the procedure. The search for the smoothess parameter has to take several aspects to accout depedg o the applcato the predcted output s used for. A bad characterstc that GRNN shows s that t ca have wggles (Fg..-). A wggle s defed as a flecto pot at a posto where o flecto should be. Wggles ca be as severe that such sudde chages the values of the predcto happe that these chages almost appear to be steps. I (Fg..-) aga the dvdual terms of the predcto usg GRNN are show. The dvdual terms have a ver arrow rage of fluece compared

7 48 to (Fg..-). The dvdual terms almost solel fluece the predcto the close vct of the trag sample. 0. Predcted Curve wth GRNN Trag Sample Idvdual Terms of Sum GRNN 0. x Fg..- GRNN Predcto wth extreme wggles, cludg the dvdual sgals, 7 The appearace of wggles s solel coected to the value of the smoothess parameter. For a predcto that s close to oe of the trag samples ad a suffcetl small smoothess parameter the fluece of the eghborg trag samples s mor. The cotrbuto to the predcto, (- k / ), s a lot smaller tha the cotrbuto of the trag sample that the predcto s close to, (- / ). The fluece of the trag samples that are further awa from the pot of predcto ca be eglected. ue to the ormalzato the predcto therefore elds the value of the trag sample the vct of

8 49 the each trag sample (Eq..-). For a bgger smoothes parameter the fluece of the eghborg trag samples caot be eglected. The predcto the s flueced b more pots ad the predcto s gettg smoother lm lm lm lm lm lm Y X X Y Y Y close to 0 lm Y Eq..- Wth eve larger the predcted curve wll get flatter more smooth as well. I some cases ths s desrable. For example whe the avalable data clude a lot of ose, the the predcto has to terpolate the data whereas f the data are correct, GRNN has to ft the data more precsel ad has to follow each lttle tred the data makes. If approaches ft the predcted value s smpl the average of all the sample pots (Eq..-). The effect of a bg

9 50 smoothess parameter ca be see ver earl.starts ver earl. I (Fg..-) a example for a smoothess parameter of oe s show. Trag samples Predcto wth GRNN for a large x Fg..- Predcto wth GRNN for a large smoothess parameter, lm Y Y Y lm lm Eq..- ue to the fact that data are ot geerall wthout measuremet errors ad that the crcumstaces chage from applcato to applcato, there caot be a rght or a wrog wa

10 5 to chose. The applcato ad the requred features of the predcto hghl determe whch should be fall chose. A tradeoff betwee smoothess ad mmal error has to be made depedg o the data ad the later use of the predcto... The Holdout Method Specht suggests [Specht 9] the use of the holdout method to select a good value of. I the holdout method, oe sample of the etre set s removed ad for a fxed GRNN s used aga to predct ths sample wth the reduced set of trag samples. The squared dfferece betwee the predcted value of the removed trag sample ad the trag sample tself s the calculated ad stored. The removg of samples ad predcto of them aga for ths chose s repeated for each sample-vector. After fshg ths process the mea of the squared dffereces s calculated for each ru. The the process of reducg the set of trag samples ad predctg the value for these samples s repeated for ma dfferet values of. The for whch the sum of the mea squared dfferece s the mmum of all the mea squared dffereces s the that should be used for the predctos usg ths set of trag samples. Accordg to Specht there are o restrctos to ths process, but ufortuatel t tured out that for certa codtos ths process does ot show the desred results. The holdout method works wth smoothess parameters that are ver small. The evaluato of the oetal fucto therefore ofte causes umercal problems; eve for 64 bt data storage. Gve that there are o umercal problems, aother problem s that the mmum of the Sum of Squares, Specht descrbes alread as wde s smpl so wde (Fg.

11 5.-3) that the fal choosg of s ver mprecse. chose the rage of the mmum ca show oe of the above metoed problems of ether extreme: Wggles or a uacceptable terpolato, but usuall wggles were observed. I (Fg..-3) a example of the result of the holdout method s show. The Sum of Sqaures s plotted versus the smoothess parameter. The holdout method suggests to use a smoothess parameter of. Wth the aked ee the mmum, the holdout method suggests ads up to a value of the smoothess parameter of Sum of Squares sgma Fg..-3 Result from Holdout Method I (Fg..-4) the results for these prevousl chose smoothess parameters are plotted. Both of the plots eld a curve that cludes wggles. The wggles are step lke for a

12 53 smoothess parameter of, the wggles are more getle for a smoothess parameter of 4. The precso of the predcto s ver good at the trag pots but the chages the slope of ths curve are uacceptable for several applcatos. As a comparso a predcto wth a smoothess parameter of 7 s cluded too. Ths predcto does ot clude wggles but s therefore less precse at the trag samples. A decso has to be made what curve s better. Ths decso depeds o the applcato the predcto s fall used for. predcto wth sgma ust outsde the mmum of holdout method, sgma4 predcto wth sgma7 predcto wth sgma from holdout method, sgma 0. trag sample 0. x Fg..-4 Usg as suggested b the Holdout Method ad other to compare.. The Wggle-Method All possble procedures for the choosg of are emprcal procedures ad caot eld a exact or correct value for. The holdout method ca work ver well for some examples but the results of the holdout method caot be flueced such that the

13 54 accommodate for the dfferet possble desred characterstcs of the predcto. Some effort was udertake to fd a dfferet, more flexble wa of choosg. The result s the Wggle- Method. The wggle-method s a purel emprcal method that turs out to be ver eas to work wth. The results usg the wggle-method ca be flueced ver easl. It leaves room to adust the algorthm to pa closer atteto to smoothess or to precso the predcto. The wggle-method (Fg..-5) works for a umber of dmesos. Usg a two dmesoal example the wggle-method wll be laed. The wggle-method eeds to have the allowable umber of flecto pots specfed. Ths formato s usuall kow or ca easl be assessed. The udgmet that s eeded though s to allow the rght umber of addtoal flectos to cout for uequall spaced data or measuremet errors, problems that wll be dscussed later. For the wggle-method, GRNN predcts the curve over the etre rage of the data. Frst s chose ver small, too small to predct the curve wthout wggles. The value for s creased b a specfable amout costatl utl the umber of flecto pots s equal to the allowable umber of flecto pots. Icreasg wll sooer or later eld a curve that s too smooth, that has ot eough flectos, the has to be decreased aga. Ths terato wll go o utl a stop crtera s satsfed. The stop crtera ca be a maxmum umber of teratos aroud the maxmal allowable umber of wggles, or t ca be a maxmum tolerable chage or a combato of these two. The umber of wggles s calculated wth a umercal approxmato. Ma predctos for equall spaced pots are calculated. The spacg of the predcted pots eeds to be

14 55 suffcetl more dese tha the spacg of the trag samples, otherwse the method would ot otce the flecto pots. Usg about fve tmes or more pots of predctos tha trag samples s suggested. The secod dervatve usg three cosecutve pots s the calculated umercall (Eq..-3) over the etre rage. Iflecto pots show a sg chage the secod dervatve. The umber of sg chages over the rage of the calculated predctos s equal to the umber of flecto pots. sec ( + ) + + x Eq..-3

15 56 The predcto has 0 wggles, crease x x The predcto has 0 wggles, crease x x The predcto has 0 wggles, crease xx The predcto has 4 wggles, crease xx 0. Ol two wggles left close to the edges. Now some more teratos ca follow to get closer to the border to three wggles. 0. xx Fg..-5 Wggle-method

16 57 The precso of last curve (Fg..-5) s ot as good as t could possbl be but t s ver a ver smooth curve. Ths curve has the desred characterstcs for ma applcatos, t s smooth ad t s precse. The umber of flectos rema costat over a rage of. But the precso at the trag samples chages cotuousl wth. The precso decles for bgger. should be chose at the lower boud of the rage for the desred umber sce the qualt of the predcto of the trag samples creases as gets smaller (Fg..-6). 0 Number of Iflectos 4 Choose at ths ed of the rage o ot choose at ths ed of the rage Fg..-6 Selecto of The qualt of the predcto s ot gog to be bad, supposg that the umber of flectos gve s close to the true umber of flectos that the sstem reall has. It was prove b Parze [Parze] that the probablstc approach wll coverge to the uderlg fucto as more trag samples are avalbale. Coutg the flecto pots suggested b the user s a measure of how close the predcto s to the uderlg fucto. Ths s smpl a

17 58 measure of qualt as the sum of squares s as well a measure of qualt. The wggle-method also allows hadlg of measuremet errors ad hadlg of the fluece of uequall spaced data. These problems wll be dscussed later. Usg the wggle-method the curve wll ot be flueced b the scatter of the data such that extreme devatos occur as wth Sple fuctos. The curve wll have the desred shape ad the proof of covergece b Parze make sure that the curve wll terpolate the data. A procedure ca be cluded that shows the developmet of the mea squared error over the etre process. Adustmets to the search method ca the be made. These adustmets are othg else tha creasg or decreasg the tolerable umber of wggles for the predcto order to get a more precse or smooth curve. Later (Secto 3.6) a wa to crease the precso wthout loosg the smoothess of the curve wll be dscussed. The wggle-method wll coverge to a value of the smoothess parameter. For a ver small the predcto cludes too ma wggles. For a that s amg towards ft, the value of the predcto wll be the average of the values of the trag samples wthout a wggle. Accordg to [Specht 9] there s a value of for whch the predcto coverges to the uderlg fucto of the samples. The wggle-method wll fd a value that les betwee these two boudares, ma wggles or average value ad o wggles. As metoed before a trade-off has to be made betwee precso ad smoothess. I (Fg..-7) the same example that was prevousl used s show aga. Prevousl the predcto dd ot eld ver precse values. The predcto s ver smooth but the predcto

18 59 does ot represet the trag samples ver precsel. I (Fg..-8) the predcto represets the trag samples ver well but the predcto has ma wggles. epedg o the use of the results oe or the other predcto ca be better. I (Fg..-7) ad (Fg..-8) the predcto tself ad the slope of the predcto s show for the examples used (Fg..- 7) ad (Fg..-8). The slope of the predcto for usg a 0. s ver smooth too. A cotroller usg the two sgals, value ad slope of the predcto wll perform ver dfferet for the predcto usg a 0. tha the same cotroller usg the predcto ad the slope for a Slope of Predcto slope 0. Predcto - 0. x -.0 Fg..-7 Predcto ad Slope of Predcto for a ver smooth curve, 0.

19 Slope of Predcto slope 0. Predcto x -4.0 Fg..-8 Predcto ad Slope of Predcto for a curve wth ma wggles, 7 It caot be assumed that the wggle-method s the best wa of choosg, but the results tured out to be ver satsfg. The holdout method could work wthout addtoal formato provded b the user; the wggle-method eeds addtoal formato. Ths makes the method vulerable to errors but as well more flexble.

Bezier curves. 1. Defining a Bezier curve. A closed Bezier curve can simply be generated by closing its characteristic polygon

Bezier curves. 1. Defining a Bezier curve. A closed Bezier curve can simply be generated by closing its characteristic polygon Curve represetato Copyrght@, YZU Optmal Desg Laboratory. All rghts reserved. Last updated: Yeh-Lag Hsu (--). Note: Ths s the course materal for ME55 Geometrc modelg ad computer graphcs, Yua Ze Uversty.