Fuzzy Linear Regression Analysis

Transcription

1 12th IFAC Coferece o Programmable Devices ad Embedded Systems The Iteratioal Federatio of Automatic Cotrol September 25-27, Fuzzy Liear Regressio Aalysis Jaa Nowaková Miroslav Pokorý VŠB-Techical Uiversity of Ostrava, Faculty of Electrical Egieerig ad Computer Sciece, Departmet of Cyberetics ad Biomedical Egieerig, 17. listopadu 15/2172, Ostrava Poruba, Czech Republic ( aa.owakova@vsb.cz). VŠB-Techical Uiversity of Ostrava, Faculty of Electrical Egieerig ad Computer Sciece, Departmet of Cyberetics ad Biomedical Egieerig, 17. listopadu 15/2172, Ostrava Poruba, Czech Republic ( miroslav.pokory@vsb.cz). Abstract: The theoretical backgroud for abstract formalizatio of vague pheomeo of the complex systems is fuzzy set theory. I the paper vague data as specialized fuzzy sets - fuzzy umbers are defied ad it is described a fuzzy liear regressio model as a fuzzy fuctio with fuzzy umbers as vague parameters. Iterval ad fuzzy regressio techologies are discussed, the liear fuzzy regressio model is proposed. To idetify fuzzy regressio coefficiets of model geetic algorithm is applied. The umerical example is preseted ad the possibility area of vague model is illustrated. Keywords: Regressio model, o-specificity, iterval model, vagueess, fuzzy model, fuzzy umber, geetic algorithms, possibility area. 1. INTRODUCTION Regressio models are used i egieerig practice wherever there is a eed to reflect more idepedet variables together with the effects of other umeasured disturbaces ad iflueces Bardossy (1990), Shapiro (2006). I classical regressio, we assume that the relatioship betwee depedet variables ad idepedet variables of the model is well-defied ad sharp. I the real world, however, hampered by the fact that this relatioship is more or less o-specific ad vague. This is particularly true whe modellig complex systems which are difficult to defie, difficult to measure or i cases where it is icorporated ito the huma elemet Shapiro (2006). The theoretical backgroud for abstract formalizatio of vague pheomeo of complex systems is fuzzy set theory Novák (1990). I the paper vague data as specialized fuzzy sets - fuzzy umbers are defied ad a fuzzy liear regressio model as a fuzzy fuctio with fuzzy umbers as vague parameters is described. 2. FUZZY REGRESION ANALYSIS Liear regressio model of ivestigated system Shapiro (2006) is give by a liear combiatio of values of its iput variables Y (x ) = A 0 + A 1 x A x. (1) This work has bee supported by Proect SP2013/168, Methods of Acquisitio ad Trasmissio of Data i Distributed Systems, of the Studet Grat System, VŠB - Techical Uiversity of Ostrava. Covetioal regressio model is based o the assumptio that the system characteristic is defied by sharp, precise ad deviatios betwee observed ad estimated values of the depedet variables are the result of errors of observatio. However, the statistical regressio models based o priciples of probability theory are correct oly if a umber of precoditios is met Pokorý (1993), Shapiro (2006). The most commo practical problems are a small umber of observatios, the sample is too small, we ca ot guaratee a ormal distributio of error, difficult to defie the relatioship (vagueess) betwee the iput ad output variables. These problems do ot occur whe the creatio of regressios utilize possibility theory ad regressio depedece is idetify as a fuzzy fuctio. The origi of the deviatio betwee the observed ad estimated values of the depedet variables may ot be sigificat extet caused by poor local variables of system structure. These variatios ca be caused by i ot very sharp ature of the system parameters. Such fuzzy pheomeo must also be reflected i fuzziess of the correspodig parameters of the model. If we cosider the fuzzificatio of regressio model, we ca cosider two cases (but which are ot mutually exclusive). First of all, we cosider that the iput data are crisp ad ucertai is i the defiitio of the model. I this case, the vagueess is reflected by fuzzy ature of the regressio coefficiets as model parameters. I the secod case, we ca cosider the system as a well-defied ad /2013 IFAC / CZ

2 The idetermiate ature of fuzzy regressio model is represeted by the estimated fuzzy output values Ỹ (x ) ad the fuzzy regressio coefficiets à i the form of specialized fuzzy sets - fuzzy umbers. Shape of fuzzy liear regressio model Buckley (2008), Heshmaty (1985), Taaka (1982) is give by Ỹ (x ) = Ã0 + Ã1x Ãx = Ã.x (3) where x is a trasposed colum vector x = (x 1, x 2,..., x ) ad à is a parameter vector whose elemets are fuzzy umbers. I the fuzzy regressio fuctio à is the multidimesioal fuzzy set (fuzzy relatio) as the Cartesia product of fuzzy sets of fuzzy parameters à = Ã0 Ã1... à (4) with membership fuctio i the form Fig. 1. Oe-dimesioal liear iterval regressio model fuzzy character have the measured data. The the carrier of ucertaity of the model are vague iput data. The paper aalyzes the situatio where both cases are applied. 2.1 Iterval Liear Regressio Model The first step i creatig a blurred regressio model is the work Buckley (1990), who developed the techology of iterval regressio models... Y (x ) = A A... 1 x A... x. (2) Where Y... (x ) is the estimated value of the output variable as a closed umerical iterval represetig the ucertaity of the o-specific system ad... A are the regressio coefficiets of the model agai i the form of vague closed umerical itervals. To idetify the itervals of regressio coefficiets the method of liear programmig used i Kacprzyk (1992), for algebraic calculatios with iterval umbers simple iterval arithmetic is developed Moore (1979). Example of a oe-dimesioal liear iterval regressio model is show i Figure (1). 2.2 Fuzzy Liear Regressio Model The ext step i the developmet of idetermiate regressio model is the developmet of models of vague, usig the formalizatio of ucertaity rather tha umerical itervals usig the fuzzy itervals. Regressio models reflect the vagueess of the modelled systems are called fuzzy regressio models Kacprzyk (1992), Poleshchuk (2012), Shapiro (2006). The idetermiate ature of fuzzy regressio model is represeted by the estimated fuzzy output values Ỹ (x ) ad the fuzzy regressio coefficiets à i the form of specialized fuzzy sets - fuzzy umbers. Shape of fuzzy liear regressio model Buckley (2008), Heshmaty (1985), Taaka (1982) is give by The ext step i the developmet of idetermiate regressio model is the developmet of models of vague, usig the formalizatio of ucertaity rather tha umerical itervals usig the fuzzy itervals. Regressio models reflectig the vagueess of the modelled systems are called fuzzy regressio models Kacprzyk (1992), Poleshchuk (2012), Shapiro (2006). µã (a) = {µãi (a)}, a = (a 1, a 2,..., a ). (5) The shape of the membership fuctio of fuzzy umbers output value of fuzzy liear regressio model (1) is calculated by Zadeh s extesioal priciple Novák (1990) i the form µ γ (y) = µã (a) ; {a t = y} = 0, a at =y (6) 0 ; elsewhere. Membership fuctio µãi (a i ) is approximated i the form of triagular fuzzy umbers Ghorsray (1997), Novák (1990) µãi (a i ) = 1 α i a i c i ; α i c i a i α i + c i, 0 ; elsewhere, where α i is the mea value (core) of fuzzy umber Ãi ad c i is half of the width of the carrier bearig Ãi = {α i, c i }. The term of membership fuctios for the output fuzzy sets (3) ca be writte i the form Kacprzyk (1992) µ γ (y) = 1 y α.x c i x i ; α.x c i x i y α.x + + c i x i, 0 ; elsewhere, 2.3 Idetificatio of Fuzzy Liear Regressio Model Fitess of liear regressio fuzzy model to the give data is measured through the Bass-Kwakeraakss idex H see Figure 2. I the procedure of model idetificatio the optimizatio procedure miimizes the vagueess of global fuzzy fuctio through the miimizatio of sum of fuzzy regressio coefficiets vagueess uder coditio (7) (8) mi J = mi c i (9) h H, = 1, 2,..., m. (10) 246

3 Fig. 2. Adequacy of liear regressio fuzzy model The fitess of estimated value to sampled value is doe usig α cut ad α level set at the fitess h = H (see Fig. 2) Y 0,H = Y 0,H, Y 0,H, =, Y 0,H. (11) We assume the good estimatio of output value uder the coditio is fulfilled max y {µ γ 0 (y) µ γ (y)} = Cos (Ỹ 0, Ỹ ) H. (12) The relatio (12) is satisfied uder the coditio (Fig. 2) Boudary of itervals Y Y 0, = 1, 2,..., m, Y 0 Y, = 1, 2,..., m. (13), = 1, 2,..., m we ca express = (1 H) c i x i + α T x, = (1 H) c i x i + α T x. (14) Next we ca set the optimizatio problem for usig of geetic algorithm (1) miimizatio of fuzzy model vagueess mi J = mi c i, i = 1, 2,...,, = 1, 2,..., m, (2) subect to α T x + (1 H) c i x i y 0 + (1 H) y 0, α T x + (1 H) c i x i y 0 + (1 H) y 0. c i 0. (15) (16) Quatificatio of models vagueess is formalized by calculatig fuzzy itervals of fuzzy umbers estimated output values Ỹ (x). Width of fuzzy umbers carriers are the iterval i which the values of the output variables may lie with a defied grade of membership. O Figure 3 is graphically illustrated the course of a oe-dimesioal fuzzy liear regressio fuctio together with the appropriate possibility area of estimated fuzzy output Ỹ. Fig. 3. Oe-dimesioal fuzzy liear regressio model 3. USAGE OF GENETIC ALGORITHM As it was metioed the classical used method of liear programmig for idetificatio of fuzzy regressio coefficiets was substituted by usig of geetic algorithm (GA). The idetificatio of fuzzy regressio coefficiets à 0, Ã1,..., Ã, where Ãi = {α i, c i }, was divided ito two tasks (1) the idetificatio of the mea value (core) α i of fuzzy umber Ãi ad (2) the idetificatio of c i as a half of the width of the carrier bearig Ãi = {α i, c i }. The tasks are solved by usig geetic algorithm i series. First the idetificatio of α i ad the the idetificatio of c i are doe. The sharp observed values y 0 are fuzzificated y 0 = ay 0, (17) where a 0.02; 0.1 or aother value, but the value of a is defied by the expert. The the fuzzy observed value is defied as Ỹ 0 = {y 0, y 0 }, (18) ad the estimated fuzzy value Ỹ aalogously Ỹ = {y, y }. (19) 3.1 Idetificatio of the Mea Value (Core) α i For the idetificatio of the mea value (core) α i of fuzzy umber Ãi the miimizatio of fitess fuctio mi J 1 = mi 1 J J =1 by geetic algorithm is used. [y 0 (x ) y (x )] 2, (20) 3.2 Idetificatio of the Half of the Width of the Carrier Bearig c i For the idetificatio of c i as a half of the width of the carrier bearig Ãi the miimizatio of fitess fuctio (9) 247

4 Fig. 4. Course of GA covergece mi J 2 = mi c i (21) by geetic algorithm with three costraits (16) is used. 4. CASE STUDY For provig of efficiecy of proposed method, the two dimesioal liear fuctio i form Y 0 = x x 2 (22) was chose. The set of Y 0 with te members usig (22) was created. For creatig the set of Y 0 the values of x 1 ad x 2 were chose radomly from the stadard uiform distributio o the ope iterval (0, 1) but multiplied by radom iteger. For fuzzificatio of observed value a = 0.1 was used. The the miimizatio of fitess fuctio J 1 (20) by embedded fuctio of geetic algorithm i Optimtool i Matlab eviromet was used. The parameters of GA were elected as populatio type - double vector populatio size scalig fuctio - rak selectio - stochastic uiform mutatio fuctio - costrait depedet crossover fuctio - scattered migratio - forward stop criterio - o chages i fitess fuctio The shape of covergece of values of miimizatio of fitess fuctio J 1 is depicted i Figure (4). The outputs of the miimizatio by described GA are the estimated values of the mea values (cores) α 0, α 1 ad α 2 of Ã0, Ã1 ad Ã2. The ext step was to determie the c 0, c 1 ad c 2 of Ã0, Ã1 ad Ã2. For this task the miimizatio of fitess fuctio J 2 (21) by GA was used with the same parameters as i task of determiig of α i. As we ow have the complete iformatio to assemble the estimated fuzzy umbers Ã0, Ã1 ad Ã2 we ca defie Fig. 5. Possibility area of two-dimesioal fuzzy regressio model Y (y, y ) = Ã0 (α 0, c 0 )+Ã1 (α 1, c 1 ) x 1 +Ã2 (α 2, c 2 ) x 2, y = α 0 +α 1 x 1 +α 2 x 2, y = c 0 +c 1 x 1 +c 2 x 2. (23) With kowledge of (23) we are able to create the surfaces, which are defied as the upper ad lower boudary Y = y + y, Y = y y (24). The area betwee the created lower ad upper surface boudary could be called possibility area. For chose liear regressio fuctio (22) the determied possibility area is show i Figure (5). 5. CONCLUSION Abstract mathematical models of complex systems are ofte ot very adequate because they do ot accurately reflect the atural ucertaity ad vagueess of the real world. The suitable theoretical backgroud for abstract formalizatio of vague pheomeo of complex systems is fuzzy set theory. I the paper vague data as specialized fuzzy sets - fuzzy umbers are defied ad it is described a fuzzy liear regressio model as a fuzzy fuctio with fuzzy umbers as vague parameters. Iterval ad fuzzy regressio techology are discussed, the liear fuzzy regressio model is proposed. To idetify fuzzy regressio coefficiets of model istead of commoly used liear programmig method C etitav (2013) the effective geetic algorithm is applied Goldberg (1989). The two-dimesioal umerical example is preseted ad the possibility area of vague model is graphically illustrated. Next research will be focused o developmet of fuzzy o-liear regressio model with fuzzy output value Pokorý (1993) to have possibility to ivestigate ad model vague o-liear systems. ACKNOWLEDGEMENTS This work has bee supported by Proect SP2013/168, Methods of Acquisitio ad Trasmissio of Data i Distributed Systems, of the Studet Grat System, VŠB - Techical Uiversity of Ostrava. 248

5 REFERENCES A. Bardossy. Note o fuzzy regressio. Fuzzy Sets ad Systems, volume 37, pages 65-75, J.J. Buckley, L.J. Jowers. Fuzzy Liear Regressio I. Studies i Fuzziess ad Soft Computig, volume 22, ISBN Spriger, B. C etitav, F. Özdemir. LP Methods for Fuzzy Regressio ad a New Approach. E.Krause, Ed. I Syergies of Soft Computig ad Statistics for Itelliget Data Aalysis, volume 22, ISBN Spriger, S. Ghorsray. Fuzzy liear regressio aalysis by symmetric fuzzy umbxer coefficiets. IEEE Iteratioal Coferece o Egieerig Systems INES97, 1997, ISBN D. Goldberg. Geetic Algorithms i Search, Optimizatio ad Machie Learig. ISBN Readig, MA: Addiso-Wesley Professioal, B. Heshmaty, A. Kadel. Fuzzy Liear Regressio ad Its Applicatio to Forecastig i Ucertai Eviromet. Fuzzy Sets ad Systems, volume 15, pages H. Ishibushi, H. Taaka. Idetificatios of Fuzzy Parameters by Iterval Regressio Model. Electroics ad Commuicatios i Japa, 73:volume 12, pages J. Kacprzyk, M. Fedrizzi (Ed.). Fuzzy Regressio Aalysis. Studies i Fuzziess ad Soft Computig, ISBN-13: Publisher: Physica-Verlag HD, R.E. Moore. Methods ad Applicatios of Iterval Aalysis. SIAM (Society for Idustrial ad Applied Mathematics) Philadelphia, V. Novák. Fuzzy možiy a eich aplikace (i Czech). Studies i Fuzziess ad Soft Computig, ISBN SNTL Praha, O. Poleshchuk,E. Komarov. A fuzzy liear regressio model for iterval type-2 fuzzy sets. NAFIPS 2012, ISBN Fuzzy Iformatio Processig Society, M. Pokorý. Fuzzy elieárí regresí aalýza. Doctoral Thesis (i Czech) VUT Bro, FEL, Bro, A.F. Shapiro. Fuzzy regressio models. I research-clearig-house/2006/auary/arch06v401- ii.pdf, ( ). H. Taaka, S. Ueima, K. Asai. Liear regressio aalysis with fuzzy model. IEEE Trasactios ad Systems, Ma ad Cyberetics, 12:6,