Exploration of applying fuzzy logic for official statistics

Size: px

Start display at page:

Download "Exploration of applying fuzzy logic for official statistics"

Bertha McDonald
6 years ago
Views:

1 Exploraton of applyng fuzzy logc for offcal statstcs Mroslav Hudec Insttute of Informatcs and Statstcs (INFOSTAT), Slovaka, Abstract People are famlar wth lngustc terms e.g. hgh response burden, low mgraton level, medum mgraton level whch descrbe partcular objects e.g. companes, terrtoral unts. In research that ncludes these classes t s not easy to unambguously create ther boundares. The fuzzy logc offers calculatons wth lngustc terms and approxmate reasonng n order to solve data selecton, dssemnaton and classfcaton problems n a way that more resembles human logc. Selecton of relevant enttes from data sets can be more flexble, allowng examnaton of records that almost meet the gven crtera, as well as those that clearly meet the crtera. Fuzzy classfcaton drectly employs expert knowledge by means of approxmate reasonng and lngustc terms usng fuzzy f-then rules n order to classfy data nto several classes, e.g. key objects, sgnfcant and nsgnfcant objects.webstes wth fuzzy logc could become more user frendly orented n processes of data searchng and selecton. The basc equaton, fuzzy greneralzed logcal condton (GLC) has been created and t could be extended n several ways to catch partcular needs. Offcal statstcs s a promsng area to develop and mplement the concept based on the fuzzy logc. Data sets have stored large amounts of quanttatve and qualtatve data that contan potentally valuable nformaton. Illustratve example has been created n order to present applcablty of the fuzzy logc and especally the created GLC on data from offcal statstcs. Keywords: fuzzy logc, data retreval, data classfcaton 1. Introducton Databases n offcal statstcs contan large amount of data and metadata. Hence, there s a need for new ways of computatonal technques to assst humans n extractng and examnng relevant data and nformaton from the rapdly growng volumes of data n a more human orented way. Based on prevous researches and prelmnary results obtaned n the frame of the BLUE-ETS project we have examned development of technque for flexble data examnaton that could be used for offcal statstcs and as reduced verson for broad audence (webste) n order to provde hgh qualty data and nformaton. User knowledge about stored enttes n a database and preferences what should be retreved from databases s often mperfect and mprecse. Here we treat the queston how to select only relevant data accordng to user preferences defned by lngustc terms for further use.

2 The flexble classfcaton drectly employs expert knowledge by means of approxmate reasonng and lngustc terms n order to classfy data nto several classes, e.g. key objects, sgnfcant and nsgnfcant objects. One of goals of statstcal webstes s to target the broad audence. Provdng a query by lngustc expressons gves a natural way for the database queres creaton and webstes could become more user frendly orented n processes of data searchng and selecton. In above mentoned cases the uncertanty s not based on randomness, t cannot be presented as a crsp value. Ths type of uncertanty s called fuzzness (Zmmermann, 2001). Does exst way whch allows users to descrbe problems by lngustc terms and to nclude the ambguty and uncertanty drectly nto the model? In all these felds, the fuzzy logc approach s a ratonal opton whch may offer the soluton. Fuzzy logc allows us to brng the operaton of nformaton systems closer to the workng methods of humans (Galndo et al, 2006). Users frequently deal wth vague terms such as hgh unemployment rate, hgh response burden, the majorty of, etc. These lngustc terms nclude a certan vagueness or uncertanty that nformaton systems based on the two-valued logc do not understand and therefore cannot use. In ths paper we examne ways how can be all above mentoned areas covered by the fuzzy logc approach and the generalsed logcal condton (GLC) created n (Hudec, 2009). 2. Fuzzy sets and fuzzy logc n bref The concept of fuzzy sets was ntally ntroduced n (Zadeh, 1965) where was observed that precsely defned crtera of belongng to a set often could not be defned. The fuzzy logc s an approach to computng based on degrees of truth rather than the usual true or false logc. In many real applcatons the two valued logc ( black and whte ) s not adequate and levels of gray should be ncluded. The core of both classcal and fuzzy logc s the dea of a set. In the classcal set theory an element belongs or does not belong to a set. Let consder a set called hgh unemployment (HU) defned as follows: HU={x unemployment(x)>=10%} where x s a regon. It means that regon wth 9.95% unemployment does not belong to the HU but regon wth 10% belongs. These constrants are drawback when the boundares between values of some attrbutes are contnuous. The fuzzy logc theory brngs a paradgm n work wth the graduaton, uncertanty and ambguty descrbed by lngustc terms. Ths gradaton s descrbed by a membershp functon µ valued n the nterval [0, 1], where µa(x) = 0 ndcates that x n no way belongs to fuzzy set A, µa(x) = 1 says that x wthout any doubt belongs to fuzzy set A. The HU example can be presented by fuzzy set HU shown n fgure 1. User could defne

3 that the unemployment equal and hgher than 10% s HU, the unemployment smaller than 8% defntely s not hgh and unemployment between 8% and 10% partally belongs to the HU concept and as unemployment s closer to 10% t stronger belongs to HU concept. The fuzzy approach uses knowledge that does not have clearly defned boundares. Many of the phenomena from the real world fall nto ths class. Fgure 1: Fuzzy sets for medum unemployment concept It s mportant to emphasze that shape and parameters of fuzzy sets are content dependent. It means that the fuzzy set hgh number of cloudless days has dfferent meanng for toursts and astronomers. Although mathematcs based on fuzzy sets has greater expressve power than classcal mathematcs based on crsp sets, the usefulness depends crtcally on our capablty to construct approprate membershp functons for each partcular task. Fuzzy sets operatons are aggregatons of classcal operatons of ntersecton, unon and negaton. These operatons are used n selecton and classfcaton. In classcal case there exsts one logcal functon for ntersecton, one for unon and one for negaton operators because the elementary term s satsfed (value of 1) or not (value of 0). In fuzzy logc there exst many functons descrbng ntersecton (and) operator (these functons are called t-norms) and unon (or) operator (these functons are called t-conorms) (Klr and Yuan, 1995) because each of elementary terms can be fully or partally satsfed. Let assume the conjuncton of elementary terms ar polluton s low and alttude level s more or less 200 meters. For example the terrtoral unt satsfes the low ar polluton term wth 0.8 and the second elementary term wth 0.5. Both terms are partally satsfed so the {0, 1} logc s not useful. The followng t-norm functons are often used for these purposes: mnmum: QCI = mn( µ (a )), = 1,...,n (1) product: QCI n = = 1 bounded dfference: QCI = max(0, ( µ (a )), (2) n = 1 µ ( a ) n + 1) (3) where µ (a ) denotes the membershp degree of the attrbute a to the -th fuzzy set.

4 In case of the mnmum t-norm the result of ntersecton s the value of the lowest membershp degree (0.5 n the prevous example). The product t-norm takes nto account all membershp degrees and balances the query truth membershp value across each of elementary terms nsde the where clause (0.4 n the prevous example). The thrd t-norm, the bounded dfference takes nto account only objects that sgnfcantly satsfy elementary terms and s more restrctve than other two mentoned t-norms (0.3 n the prevous example). 3. Realsaton of flexble data selecton and dssemnaton The records selecton by means of fuzzy logc s more flexble, allowng examnaton of records that almost meet the gven condton, as well as those that clearly meet the condton. The generalzed logcal condton (GLC) that extends the where clause of the Structured Query language (SQL) by lngustc terms, degrees of truth and converts fuzzy queres nto SQL was created and examned n detals n (Hudec, 2009). In ths paper, flexble queryng by the GLC has been realsed n a smplfed way for broad audence and n an advanced way for professonal users. The frst one s sutable for webstes dealng wth dssemnaton of data. A bref overvew of crsp queres, ther lmtatons and ways of mprovng them by fuzzy forms can be found n (Cox, 2005). In our research the goal was to create easy to use queryng technque and to access to data by SQL queres so no modfcaton of databases has to be undertaken. The queryng process conssts of the two steps. In the frst step lower and/or upper boundares of lngustc expressons (fuzzy sets) are used as parameters for database queres. It means that all records that partally or fully meet query condton (QC) are selected. In ths step the GLC s employed. The GLC has the followng structure: n where ( a o L ) (4) = 1 x where n denotes number of fuzzy constrants n a where clause of a query, and = or where and and or are fuzzy logcal operators, and a > Ld, a s Hgh a o Lx = a < Lg, a s Small a > Ld and a < Lg, a s About. where a s a database attrbute, L d s the lower bound and L g s upper bound of a lngustc expresson descrbed by fuzzy set shown n fgure 2. The GLC s a core for all extensons examned bellow.

5 Fgure 2: Fuzzy sets In the second step the chosen analytcal form of the fuzzy set (fgure 2) s used to calculate the membershp degree of each selected record to approprate fuzzy set. Fnally, approprate t-norms (1), (2) or (3) or t-conorms are used to calculate QC values for all retreved records. 3.1 Basc realsaton The basc realsaton s flexble queryng by the GLC and mnmum t-norm (1) for conjuncton and maxmum t-norm for dsjuncton of elementary terms. The technque s sutable for data dssemnaton on webstes n order to provde data selecton accordng to user preferences. Provdng a query by lngustc expressons gves a natural way for the database queres creaton and webstes could become more user frendly orented n processes of data searchng and selecton. An example of flexble query could be muncpaltes wth hgh number of sunny days and low dstance to the nearest ralway staton and alttude more or less 200 meters. We have created the database structure and common presentaton layer for all censuses n Slovaka for the purpose of presentaton on the webste. The presentaton emphaszes demographc data from populaton and housng censuses. The webste contans selected demographc nformaton from censuses conducted n Slovaka. Presentatons of 1991 and 2001 censuses were realsed and we expect to contnue wth other censuses. Ths webste s an excellent canddate for applyng the flexble data selecton and dssemnaton approach for broad audence. 3.3 Advanced concept of data selecton In advanced concept the addtonal functonalty has been examned. Frst of all, f overall query condton contans fuzzy as well as classcal elementary condtons, classcal

6 condtons could be easy added to the where clause by extendng the GLC (4) n the followng way: n where ( a o L ) [and/or][attrbute_m LIKE "*Strng"][and/or] (5) = 1 x where lke s the SQL comparson operator and Strng s an arbtrary strng varable. Further, all above mentoned t-norm functons (1) (3) are appled. Threshold value of relevant data s also appled to provde selecton of enttes whch sgnfcantly meet the query condton. Moreover, ths concept allows fndng enttes n databases wth the same or smlar values of selected ndcators usng the About fuzzy sets n fgure 2c. Generally, smlarty can be realsed n two ways: s(a, c), where a s the name of an attrbute and c s a constant (fuzzy or not); s(a, a k ), where a k s also an attrbute name. Apparently s s a symmetrc functon, that s s(a, a k )= s(a k, a ). The frst way of smlarty calculaton s shown n the below case study. In the further research, above mentoned approach could be broadened by preferences between elementary condtons n order to dstngush elementary query condtons wth respect to ther mportance nsde the overall query condton. In order to acheve the mentoned dstncton, the weghts w [0,1] wll be ncluded. An example of such a query for busness statstcs s as follows: lst of busness companes whch have hgh turnover and hgh export rate and medum number of employees where hgh export rate s the most mportant ndcator and other two have the same lower mportance. 3.4 Illustratve example For the prelmnary research, the data from the Urban and Muncpal Statstcal database are used n the llustratve example to test the developed approach. Ths database s n offcal use at the Statstcal Offce of the Slovak Republc. The potentalty of flexble queres n ths feld s very hgh because there are more than 2800 muncpaltes and more than 800 ndcators for each muncpalty. The proposed nterface has been adjusted to ft selectons from ths database. All management of the queryng process s stuated nsde one form shown n fgure 3. The result of a query n a tabular form s presented n the lower left part of the form. In the llustratve example, we want to obtan muncpaltes wth number of nhabtants more or less 4000 and alttude more or less 125 meters above the sea level. Muncpaltes wth the smlar values of these two ndcators are sought. Selecton condtons for both ndcators are represented by fuzzy set About (the shape as from Fgure 2c) wth these parameters L d =3500, L p =3800, L q = 4200, L g = 4500 and L d =100, L p =120, L q = 130 and L g = 150 respectvely.

7 The result of fuzzy query s shown n fgure 3. The mnmum t-norm (1) s used for the calculaton of the QC. The fgure 4 shows two muncpaltes fully satsfyng the query; two muncpaltes are extremely close to satsfy the query and another seven muncpaltes are close to the query crteron. If SQL were used, ths addtonal valuable nformaton would reman hdden. Retreved enttes are ranked accordng to satsfacton of overall query condton defned by lngustc terms. Fgure 3: Interface proposal for flexble queres The lower rght part of the nterface could be used for other types of presentng retreved data and nformaton e.g. graphs or n case of terrtoral unts thematc maps. Database queryng processes based on fuzzy sets and fuzzy logc demand addtonal calculatons n comparson wth SQL counterpart due to a hgh amount of metadata to be processed (lngustc terms), the calculaton of query satsfacton, etc. Ths addtonal amount of calculaton s balanced wth addtonal valuable nformaton obtaned from the database n a way that more resembles human logc. 4. Classfcaton by fuzzy logc Each entty n the database s characterzed by ts own vector of attrbutes. Accordng to these attrbutes, enttes should be classfed and ranked for dfferent purposes. The man advantage of a fuzzy classfcaton compared to a crsp one s that an element s not lmted to a sngle class but can be assgned to several classes. Many examples from real world can be found where a fuzzy classfcaton would be useful. Lngustc expressons lke: hgh response burden, medum company, rural area,

8 etc. are often used and t s useful to capture them and drectly use n data classfcaton problems e.g. classfcaton of busness respondents n order to know more about them. The fuzzy classfcaton drectly employs expert knowledge by means of approxmate reasonng and lngustc terms n order to classfy data nto several classes, e.g. key objects, sgnfcant and nsgnfcant. Relevant concepts of fuzzy systems can be found n (Cox, 2005). The fuzzy approach gves two man ways for solvng classfcaton tasks: fuzzy systems and generatng fuzzy queres from prevously created fuzzy rules. Fuzzy systems and ther applcablty are examned n detals n (Sler and Buckley, 2005). The fuzzy nference system (FIS) from the MatLab software was used to create and solve muncpaltes classfcaton model (Hudec and Vujoševć, 2010). The powerful software products for the fuzzy reasonng are hghly parametrc and t mples complcated work for end users. Ths approach has hgh potental and applcablty but f doman expert who s solvng classfcaton problem s not famlar wth software he/she needs assstance from operatonal research expert n preparng nput data nto proper matrx format and convertng output vector nto a useful form, e.g. tables or maps. Although, creaton of classfcaton model s qute ntutve the doman expert also may need assstance to set approprate functons for aggregaton, mplcaton, accumulaton and defuzzfcaton n FIS. The FIS tools usually offer varety of these functons. Fuzzy classfcaton conssts of fuzzfcaton, creaton of rule base and processng of rules. The fuzzfcaton s the frst step n the fuzzy classfcaton process. In ths step, the focus s on the constructon of membershp functons, precsely dvdng the whole doman of examned tem nto the partcular fuzzy sets. For example the length of roads ndcator can be classfed nto the followng fve sets: very small, small, medum, hgh and very hgh. If doman conssts of qualtatve data, e.g. farly hgh, hgh, defntely hgh, etc t can be also fuzzfed. In the next step - the creaton of rule base, t s necessary to defne the rules that connect the nput fuzzy sets wth the output subclasses. These rules have the f-then structure. Some rules from (Hudec and Vujoševć, 2010) are as follows: If length of roads s Very Small AND number of days wth snow s Very Small, THEN mantenance s Very Small. If length of roads s Medum AND number of days wth snow s Very Small AND precptaton s Small, THEN mantenance s Small. Processng of the rules conssts of fuzzy aggregaton, mplcaton and accumulaton functons whch are part of the FIS. The dea for classfcaton usng fuzzy queres and the GLC has been found durng work on fuzzy database queres. Selecton s a specal case of classfcaton where enttes are separated nto two classes: relevant records that are selected and not relevant that are not selected. Moreover, queres are equvalent wth the f part of rules and result of the query are records that fully or partally belong to the output class representng the then part of rules. The results of all queres are objects selected nto overlappng output classes. The fnal rank for each record can be calculated from the equaton:

9 m R O = µ P (6) = 1 Oc where m s number of classes, µ Oc s the membershp degree of object O to class C and P s coeffcent descrbng class C. The prelmnary research reveals that f classfcaton problem requres larger number of rules and attrbutes where attrbutes can be quanttatve and qualtatve then the frst approach can be more sutable. Classfcaton by the fuzzy system and by the GLC has the same fuzzfcaton and creaton of the rule base steps but ways how these rules are calculated n order to obtan soluton are dfferent. 5. Concluson In all cases when the user cannot unambguously select relevant data from the not relevant ones or when the user cannot classfy enttes nto crsp classes, the fuzzy logc approach s a ratonal opton whch may offer the soluton. Fuzzy logc supports selecton and classfcaton by lngustc terms and approxmate reasonng. Fuzzy logc approach requred addtonal amount of calculaton but t provdes addtonal valuable nformaton n a way that more resembles human logc Foundaton equatons are created and they can be extended n several ways to catch partcular needs. These equatons are dscussed n the llustratve example. Ths concept s gong to have been extended and realsed for busness statstcs by the end of the BLUE-ETS project. The man advantages of the suggested approach for data selecton are as follows: access to relatonal databases does not have to be modfed; users do not need to learn a new query language; queres are created by concepts rather than numbers; presentng of retreved data s smlar as n case of crsp queres (SQL) but wth addtonal valuable nformaton; t can be extended n order to meet partcular requrements. The man advantages of the suggested approach for data classfcaton are as follows: enablng the creaton of logcal nference system based on human mnd ncludng uncertantes and lngustc terms n fuzzy rules; understandable knowledge base for further use and modfcaton. The functonalty of the GLC and fuzzy queres could be extended n the future by hedges and modfers, so users would be able to create more sophstcated queres and classfcaton rules. The role of hedges s to catch and process composte condtons lke alttude s farly hgh.

10 References Cox E. (2005) Fuzzy modelng and genetc algorthms for data mnng and exploraton, Morgan Kaufman, San Francsco. Galndo J., Urruta A., Pattn M. (2006) Fuzzy databases: Modelng, Desgn and Implementaton, Idea Group Publshng Inc, Hershey. Hudec M. (2009) An approach to fuzzy database queryng, analyss and realsaton, Computer Scence and Informaton Systems, 6(2), Hudec M., Vujoševć M. (2010) A fuzzy system for muncpaltes classfcaton, Central European Journal of Operatons Research, 18, Klr G., Yuan B. (1995) Fuzzy sets and fuzzy logc, theory and applcatons, Prentce Hall, New Jersey. Sler W., Buckley J.J. (2005) Fuzzy expert systems and fuzzy reasonong, John Wley & Sons, Inc. Zmmermann H.J. (2001) Fuzzy Set Theory: And Its Applcatons, Kluwer Academc Publshers, London. Zadeh L. A. (1965) Fuzzy sets, Informaton and Control, 8,

Type-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data

Type-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data Malaysan Journal of Mathematcal Scences 11(S) Aprl : 35 46 (2017) Specal Issue: The 2nd Internatonal Conference and Workshop on Mathematcal Analyss (ICWOMA 2016) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES