Visualizing High Dimensional Fuzzy Rules

Size: px

Start display at page:

Download "Visualizing High Dimensional Fuzzy Rules"

Roland Paul
5 years ago
Views:

1 21 Visualizing High Dimensinal Fuzzy Rules R. Hlve, M. R. Berthld, Berkeley/USA Abstract. In this paper we present an apprach t visualize a ptentially high-dimensinal and large number f (fuzzy) rules in tw dimensins. This visualizatin presents the entire set f rules t the user as ne cherent picture. We use a gradient descent based algrithm t generate a 2D-view f the rule set which minimizes the errr n the pair-wise fuzzy distances between all rules. This apprach is superir t a simple prjectin and als mst nn-linear transfrmatins in that it cncentrates n the imprtant feature, that is the inter-pint distances. In rder t make use f the uncertain nature f the underlying fuzzy rules, a new fuzzy distance-measure was develped. The visualizatins f a rule set fr the well-knwn IRIS dataset as well as fuzzy mdels fr ther benchmark.~ata sets are illustrated and discussed. 1 Intrductin Fr the analysis f large industrial data sets, the autmatic extractin f rules has raised increasing interest in the past years [2,4]: Tw main drawbacks have inhibited the applicatin f such algrithms t real prblems. Nt nly des the number f extracted rules explde fr many real wrld scenaris but in additin des the interpretability f such rules suffer tremendusly in high-dimensinal feature spaces, due t the high number f cnstraints in the rules' antecedents. In this paper we present an apprach which visualizes the entire set f fuzzy rules in tw dimensins. The riginal fuzzy rules can be defined ver a feature space f arbitrarily large dimensin. Based n a new fuzzy distance between tw fuzzy pints in this feature space, we derive a mesh f pints cupled by pairwise fuzzy distances. This matrix f fuzzy distances ill then transfrmed int a tw dimensinal mdel, where the pairwise distances are maintained as clsely as pssible. This is achieve thrugh a randm initializatin and a gradient descent based strategy which updates psitins f all pints subsequently. M. Berthld was supprted by DFG grant Be1740/7-1.

2 22 This wrk is a fuzzy variant f multi dimensinal scaling [5] and spring embedding [8] and is similar t KOAN [7], a tl that transfrms crisp distance matrices int tw (r mre) dimensins. 2 Fuzzy Multidimensinal Scaling The prpsed methd is based n a set f fuzzy rules R = {R j I 1 :s; j :s; r} which are defined thrugh a membership functin J.l'R; : IR n -+ [0,1] in an n-dimensinal feature space. The n-dimensinal set f rules is then transfrmed int a set f pints in tw dimensins: Pj = (p.,py) (1 :s; j :s; r), where each pint Pj reprellents ne rule R j. The gal f the prcedure is t find a setup which minimizes sme errr functin between the pairwise difference f pints Pi, Pj and the crrespnding difference between the tw fuzzy rules Ri and R j. Such an algrithm exists fr pints in a high dimensinal feature space. The typical errr functin knwn ~rm Multi Dimensinal Scaling then simply uses crisp distances: ~( i j fr crisp pints Pi, P j E IR n and an Euclidean distance functin d( ). (1) In the scenari discussed here n crisp pints are available but instead each fuzzy rule can be described thrugh an imprecise regin, r fuzzy pint. In rder t use equatin 1 accrdingly a fuzzy distance functin is needed. We prpse t use a fuzzy distance functin d(r i, R j ) which generates a fuzzy interval describing the distance between tw fuzzy rules Ri and R j. This fuzzy distance is cmpletely defined thrugh the fllwing membership functin: J.lJ.(n"n;) : IR -+ [0,1] Vd E IR: J.lJ.('R' 'R.)(d) = max {min{j.l'r,(al),j.l'r;(y)}: dist(al,y) = d} 11 J m,liern (2) using sme crisp distance functin dist( ) in IR n which will usually be the Euclidean distance. It can be shwn that this fuzzy distance des nt vilate the triangular inequality. Using the fuzzy distance in equatin 2 we can again generate pairwise distances fr all fuzzy rules in the given set f rules R and cmpute an errr functin: (3)

3 23...iI. 0 -' '. Figure 1: Visualizatins f tw rule sets: the well knwn Iris data (left) and the Breast data set (right).. which results in a fuzzy interval as well. Similar t classical multidimensinal scaling the crresnding pints in the tw-dimensinal representatin are initially distributed randmly. The algrithm then subsequently updates the psitins f all pints Pi until a certain errr minimum fr the sum f the squares f all pairwise errrs is reached. The update is dne in small steps, using a predefined step-rate (3: IRI \:Ij : LlPi = (3. L. (E(i,j)' (Pi - Pi)) (4), :;;1 which is still a fuzzy interval. In rder t cnvert this t a crisp update value, the center f gravity f the fuzzy errrs is used in equatin 4: \:Ij : (5) 3 Results Results fr tw well knwn datasets are shwn in Figure 1. On the left the Iris data [3] (4 dimensinal feature space) is depicted, the right shws ne benchmark f the Statlg cllectin [6], the Breast data set (9 features). Rules f different class (Iris: 3, Breast: 2) are shwn in different grey scales. The size f each rectangle illustrates the imprtance f a rule, which in this case is determined by the number f cvered training examples. It is interesting t see hw fr the Iris data ne class (dark rectangle, Iris Setsa) is clearly separable frm the Ufer tw. But als fr the ther tw classes, nly tw rules f lw imprtance (indicated by the smaller size) are clse t each ther. This is cherent with the generalizatin capability f the shwn rule set, which achieves clse t 95% crrect

4 24.." II SO Figure 2: Visualizatin f the rule set fr the Australian Credit data.. classificatin n unseen test data. The rule set fr the Breast data als shws nice separability fr the mst. imprtant rules f the tw classes, and nly a small number f rules with lwer relevance lie in an area f verlap. Als,this rule set achieves a generalizatin capability f rughly 90%. Figure 2 shws a secnd data set frm the StatLg archive, the Australia Credit data. Here tw classes need t be distinguished and the fuzzy learning algrithm generates 132 rules, 70 fr class 1 and 62 fr class 2. It is interesting t see hw the rules are gruped int tw main cluster (tp left, bttm center) with three rules in between these cluster. The larger rules are easily separable frm rules f cnflicting class but a large number f smaller rules a;~ mixed with rules f the ther class, indicating a larger area f pssible cnfusin and hence prbably an verall larger generalizatin errr. Tests n unseen data validate this assumptin, the classificatin accuracy n the unseen data set lies under 85%. In the cntext f explrative data analysis it wuld be particularly interesting t fcus n the tw clusters f rules individually and als investigate sme utliers, such as the three rules in between the tw cluster and the ne rule f class 2 which is far away frm any ther rules (bttm right). Our current implementatin des nt (yet) supprt such interactin, hwever. 4 Cnclusins I~. this paper a methddlgy was presented t visualize an entire set f fuzzy rules frm a high dimensinal feature space. The visualizatin maintains the pair-wise distances between the fuzzy rules as much as pssible and gives therefre insights int the rganizatin f the rule set in the riginal feature space. The presented methdlgy nt nly prvides a way t visualize the entire mdel at nce but als prmises interesting pprtunities ",.

5 25 fr user interactin with the entire mdel and hence ffers an interesting additi.8,ii. t intelligent data analysis [IJ in the area f mdel explratin. 5 Acknwledgments M. Berthld was supprted by stipend BeI740/7-1 f the "Deutsche Frschungsgemeinschaft" (DFG). The authrs thank Prf. Ltfi A. Zadeh and his Berkeley Initiative in Sft Cmputing (BISC) fr prviding ffice space and the pprtunity fr stimulating discussins. Literature [IJ M. Berthld and D. J. Hand, editrs. Intelligent Data Analysis: An Intrductin. Springer Verlag, [2J M. R. Berthld and K.-P. Huber. Cnstructing fuzzy graphs frm examples. Intelligent Data Analysis, 3(1):37-54, ( [3J R. A. Fisher. The use f multiple measurements in taxnmic pl:;blems. In Annual Eugenics, II, 7, pages Jhn Wiley, NY; [4J R. Hlve. Investigatin f autmatic rule generatin fr hierarchical fuzzy systems., ~ In Prceedings f FUZZ IEEE at 1998 IEEE Wrld Cngress On Cmputatinal Intelligence, pages , Anchrage, Alaska, May IEEE Press. [5J J. Meulman. A distance apprach t nnlinear multivariate analysis. DSWO Press, Leiden, The Netherlands, [6] D., Michie, D. J. Spiegelhaltei', and C. C. Taylr, editrs. Machine Learning, Neural and Statistical Classificatin. Ellis Hrwd Limited, [7] L. Pfefferer. Objektzentrierte Visualisierung mehrdimensinaler D,aten als Erweiterung knventineller Datenbankmdelle (Phd Thesis). Herbert Utz Verlag, Munich, [8J N. R. Quinn and M. A. Breuer. A frce directed cmpnent placement prcedure fr printed circuit bards. IEEE Transactins n Circuits and Systems, CAS-26(6): , 1979.

Scatter Search And Bionomic Algorithms For The Aircraft Landing Problem

Scatter Search And Bionomic Algorithms For The Aircraft Landing Problem Scatter Search And Binmic Algrithms Fr The Aircraft Landing Prblem J. E. Beasley Mathematical Sciences Brunel University Uxbridge UB8 3PH United Kingdm http://peple.brunel.ac.uk/~mastjjb/jeb/jeb.html Abstract: