1682 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER Backward Fuzzy Rule Interpolation

Transcription

1 1682 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 Bacward Fuzzy Rue Interpoation Shangzhu Jin, Ren Diao, Chai Que, Senior Member, IEEE, and Qiang Shen Abstract Fuzzy rue interpoation offers a usefu means to enhancing the robustness of fuzzy modes by maing inference possibe in sparse rue-based systems. However, in rea-word appications of interconnected rue bases, situations may arise when certain crucia antecedents are absent from given observations. If such missing antecedents were invoved in the subsequent interpoation process, the fina concusion woud not be deducibe using conventiona means. To address this important issue, a new approach named bacward fuzzy rue interpoation and extrapoation (BFRIE) is proposed in this paper, aowing the observations, which directy reate to the concusion to be inferred or interpoated from the nown antecedents and concusion. This approach supports both bacward interpoation and extrapoation which invove mutipe fuzzy rues, with each having mutipe antecedents. As such, it significanty extends the existing fuzzy rue interpoation techniques. In particuar, considering that there may be more than one antecedent vaue missing in an appication probem, two methods are proposed in an attempt to perform bacward interpoation with mutipe missing antecedent vaues. Agorithms are given to impement the approaches via the use of the scae and move transformation-based fuzzy interpoation. Experimenta studies that are based on a rea-word scenario are provided to demonstrate the potentia and efficacy of the proposed wor. Index Terms Bacward interpoation, fuzzy rue interpoation (FRI), missing antecedents, transformation-based interpoation. I. INTRODUCTION FUZZY rue interpoation (FRI) was originay proposed in [28] and [29]. It is of particuar significance for reasoning in the presence of insufficient nowedge or sparse rue bases. When a given observation has no overap with antecedent vaues, no rue can be invoed in cassica fuzzy inference, and therefore, no consequence can be derived. A number of important interpoation approaches have been proposed in the iterature, incuding [1], [7], [10], [1], [19] [21], [27], [0], [4], [50] [5], most of which can be categorized into two casses with severa exceptions (e.g., type-ii fuzzy interpoation [8], [9], [4]). The first category of approaches directy interpoates rues whose antecedents match the given observation. The consequence of the interpoated rue is thus the ogica outcome. Typica approaches in this group [28], [29], [44] are based on Manuscript received August 9, 201; revised November, 201; accepted December 25, 201. Date of pubication January 29, 2014; date of current version November 25, S. Jin, R. Diao, and Q. Shen are with the Department of Computer Science, Institute of Mathematics, Physics and Computer Science, Aberystwyth University, Aberystwyth, SY2 DB, U.K. (e-mai: shj1@aber.ac.u; rrd09@aber.ac.u; qqs@aber.ac.u). C. Que is with the Schoo of Computer Engineering, Nanyang Technoogica University, Singapore (e-mai: ashcque@ntu.edu.sg). Coor versions of one or more of the figures in this paper are avaiabe onine at Digita Object Identifier /TFUZZ Fig. 1. System structure that may benefit from BFRIE. the use of α-cuts (α (0, 1]). The α-cut of the interpoated consequent fuzzy set is cacuated from the α-cuts of the observed antecedent fuzzy sets, and those of a the fuzzy sets that are invoved in the rues used for interpoation. Having found the consequent α-cuts for a α (0, 1], the consequent fuzzy set is then assembed by appying the Resoution principe. The second category is based on the anaogica reasoning mechanism [6]. Such approaches first interpoate an artificiay created intermediate rue so that the antecedents of the intermediate rue are simiar to the given observation [1]. Then, a concusion can be deduced by firing this intermediate rue through anaogica reasoning. The shape distinguishabiity between the resuting fuzzy set and the consequence of the intermediate rue is then anaogous to the shape distinguishabiity between the observation and the antecedent of the created intermediate rue. In particuar, the scae and move transformation-based approach (T-FIR) [20], [21] offers a fexibe means to hande both interpoation and extrapoation invoving mutipe mutiantecedent rues. Despite the numerous approaches deveoped, FRI techniques are reativey rarey appied in practice [2]. One of the main reasons for this is that many appications invove mutipe-input and mutipe-output probems. The rues are typicay irreguar in nature (i.e., not aways addressing the same antecedents). In particuar, rues may be arranged in an interconnected mesh, where observations and concusions in between different subsets of rues coud be overapped, and yet not directy reated throughout the entire rue base. For such compex systems, any missing vaues in a given set of observations may ead to faiure in interpoation. In Fig. 1, R i,i=1,...,n form the rue base, incuding interpoated rues, and x p,x q,p,q =1,...,m are the variabes covering antecedents and consequence. A i q (q = 1,...,m,i=1,...,n) is the fuzzy set on the qth dimension, which is incuded in the ith rue. The fina concusion B n of rue R n cannot be interpoated straightforwardy, because the IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See standards/pubications/rights/index.htm for more information.

2 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 168 Fig. 2. Hierarchica fuzzy reasoning structure for terrorist bombing threat. three missing observations A n p, A n r, and A n m cannot be deduced by conventiona means. For instance, consider a practica scenario in detecting terrorist bombing threats. The Exposion ieihood may be directy reated to the Crowdedness of a pace and the Safety precautions. The number of peope in an area may be affected by the Popuarity of the pace, the eve of Trave convenience, and the amount of Safety precautions. A hierarchica structure for this scenario is shown in Fig. 2. For traditiona forward interpoative reasoning, in order to interpoate Exposion ieihood, the observed vaues for Crowdedness and Safety precautions must be both provided. The variabe Safety precautions is particuary important, as without it, no matter what other information is avaiabe, forward interpoation woud sti fai. Therefore, the interpoation of such crucia missing vaues may become necessary, in order to aow required inferences to be performed. To address such probems, this paper proposes a nove approach termed bacward fuzzy rue interpoation (BFRIE) by substantiay expanding and refining the initia preiminary wor of [2] and [24]. This approach enabes unnown antecedent vaues to be interpoated, given other antecedents and the concusion. Using the earier exampe of Fig. 1, the unnown antecedents A n p and A n r can be bacward interpoated according to rues R j and R i, where the concusions B j, B i, and the other terms are nown. The ast missing antecedent vaue A n m can then be interpoated using R 1, and subsequenty, B n can aso be computed, as now a required antecedents are nown to perform forward interpoation. As such, the proposed techniques support fexibe interpoation when certain antecedents are missing from the observation, where traditiona FRI methods fai. In addition, BFRIE aso enabes indirect interpoative reasoning, which invoves severa fuzzy rues, each with mutipe antecedents. Therefore, it offers a means to broaden the appication of FRI and fuzzy inference. Genera BFRIE (with mutipe missing antecedent vaues) is common in practica probems such as medica diagnosis [16], networ intrusion detection [42], oi exporation [49], and inteigence data anaysis [4]. To address this chaenging issue, two methods are deveoped. The first directy extends the singe missing antecedent case, by computing and searching for good quaity parameter combinations for the T-FIR process. The second approach wors more cosey with conventiona FRI procedures by estimating the possibe missing antecedent vaues and, subsequenty, verifying the interpoative outcome against the observation. The remainder of this paper is organized as foows. Section II reviews the genera concepts of T-FIR, which is adopted to impement the subsequent deveopments. Section III introduces the basic form of BFRIE that deas with one singe missing antecedent vaue, aong with wored exampes. Section IV presents two possibe extensions that support the scenarios where mutipe antecedent vaues are missing. Section V describes a rea-word appication to demonstrate the efficacy of the proposed approach. Systematic randomized experiments are aso conducted in order to better compare and verify the accuracies of the proposed methods. Section VI concudes the paper and suggests future enhancements. II. BACKGROUND OF TRANSFORMATION-BASED INTERPOLATIVE REASONING This section introduces the interpoation procedures invoved in T-FIR [21], and defines its underying ey concepts. T-FIR offers a fexibe means to handing both interpoation and extrapoation invoving mutipe mutiantecedent fuzzy rues. It guarantees the uniqueness, normaity, and convexity of the resuting fuzzy sets. It is aso abe to hande various fuzzy set representations, incuding poygona, Gaussian, and be-shaped fuzzy membership functions. However, trianguar and trapezoida membership functions are the most frequenty used fuzzy set representations in fuzzy systems. Therefore, they are adopted in the agorithm description beow. A ey concept used in T-FIR is the representative vaue Rep(A) of a given fuzzy set A. When trapezoida representation is used, Rep(A) is defined as the center of gravity of its four points (a 0,a 1,a 2,a ) Rep(A) = a 0 + a 1 +a a where a 0,a represent the eft and right extremities (with membership vaues 0), and a 1,a 2 denote the norma points (with membership vaue 1), as shown in Fig. (a). As a specific case of trapezoids, where a 1 and a 2 are coapsed into a singe vaue a 1, the fuzzy set becomes a trianguar set (a 0,a 1,a 2 ). In particuar, the corresponding Rep(A) degenerates to the average vaue of the tripe, as given beow and shown in Fig. (b) (1) Rep(A) = a 0 + a 1 + a 2. (2) In the foowing, the T-FIR method is outined using trapezoida fuzzy sets uness otherwise stated.

3 1684 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 The intermediate fuzzy terms A thataretobeusedtobuidthe required intermediate rue are constructed from the antecedents of the N cosest rues. These are then shifted to A such that they have the same representative vaues as those of A A = A + δ A range A, N A = ω A i A i (5) i=1 (a) where the coordinates of the new fuzzy set A are cacuated on a point-by-point basis, and δ A is the bias between A and A on the th variabe domain δ A = d(a,a ). (6) Simiar to (5), the shifted intermediate consequence B can be computed, with the parameters ω B i and δ B being aggregated from the corresponding vaues of A, such that Fig.. (b) Representative vaues of fuzzy sets. (a) Trapezoida. (b) Trianguar. A. Determination of the Cosest Rues In this paper, given a rue base U, a fuzzy rue R U with M antecedents A, =1, 2,...,M, and an observation O are expressed in the foowing format: R:IFx 1 is A 1,..., and x is A,..., and x M is A M,THEN y is B O: A 1,..., A,..., A M. The distance d between a rue and an observation is determined by computing the aggregated distance of a the antecedent variabes d = M d(a,a )2,d(A,A )= d(rep(a ), Rep(A )) range =1 A () where range A = max A min A is the domain range of the variabe x. d(a,a ) [0, 1] is the normaized resut of the otherwise absoute distance measure so that distances are compatibe with each other over different variabe domains. The N (N 2) rues which have the east distance measurements with regard to the observed vaues A, and the concusion B, are then chosen to be used in the ater steps. To hep expain, assume that the observation O and a certain set of cosest rues R i,i=1,...,n,r i U that are returned by this step are represented as foows: O: A 1,..., A,..., A M R i :IFx 1 is A i 1,..., and x is A i,..., and x M is A i M,THEN y is B i. B. Construction of the Intermediate Rue Let the normaized dispacement factor ω A i weight of the th antecedent of R i ω A i = ω A i N i=1 ω A i, ω A i denote the =1/d(A i,a ). (4) B = ω B i N ω B i B i + δ B range B i=1 = 1 M ω A i, =1 δ B = 1 M δ A. (7) =1 C. Scae Transformation For each antecedent variabe of the N chosen rues, the scae transformation wors by cacuating two scae rates s A and s A. The support (a 0,a ) of the corresponding shifted fuzzy set A is transformed into a new support (a 0,a ), and the core (a 1,a 2) is transformed into another (a 1,a 2), such that s A = a a 0 a (8) a 0 and s A = a 2 a 1 a 2. (9) a 1 This eads to a scaed fuzzy set A =(a 0,a 1,a 2,a ). The corresponding parameters s B and s B of fuzzy set B can be cacuated as foows: s B = 1 M =1 s A s B = 1 M s A. (10) =1 To maintain the convexity of a scaed fuzzy set, it is necessary to ensure that the scaed support is wider than the core. For this, the foowing scae ratio S is appied, which represents the actua increase of the ratios between the core and the support S = a 2 a 1 a a 0 a 2 a 1 a a 0 1 a 2 a 1 a a 0 a 2 a 1 a a 0 a 2 a 1 a a 0 a 2 a 1 a a 0, if s s 0, S [0, 1], if s s 0, S [ 1, 0]. (11)

4 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1685 Then, the s B of consequence B is reevant to scae ratio S s B S s s B = s B S + s B, if s B s B 0 B (12) s B S, if s B s B 0. Note that for trianguar fuzzy sets, the support (a 0,a 2) of the shifted fuzzy set A is transformed into a new support (a 0,a 2), such that the scae rate s A is cacuated as foows: s A 2 a 0 a 2. (1) a 0 = a D. Move Transformation In genera, for mutipe antecedent rues, each variabe dimension has its own move rate m A, in order to move each of the scaed fuzzy sets A to new ocations that coincide with those of the originay observed vaues. This aows the initiay constructed intermediate fuzzy terms to competey transform. The fina transformed fuzzy sets then match the exact shapes of the observed vaues A. Without osing generaity, for a given scaed intermediate fuzzy term A =(a 0,a 1,a 2,a ), its current support (a 0,a ), and core (a 1,a 2) can be moved to (a 0,a ) and (a 1,a 2 ), using a move rate m A that is cacuated as foows: m A = (a 0 a 0) a 1, a 0 a a 0 0 m A = (a 0 a (14) 0), otherwise. a a 2 Simiar to the scae transformation, the move rate m B for the consequent dimension can be cacuated by obtaining the arithmetic average of those of the antecedent variabes, such that m B = 1 M m A. (15) =1 The fina interpoated resut B can now be computed by appying the scae and move transformation to B, using the resuting parameters s B, s B, and m B. Note that for trianguar fuzzy sets, obviousy, the right and center points a 2 and a 1 are used (instead of a and a 2), when computing the move ratio according to (14), in the case of a 0 a 0. III. BACKWARD FUZZY RULE INTERPOLATION AND EXTRAPOLATION WITH SINGLE MISSING ANTECEDENT VALUE BFRIE with singe missing antecedent vaue (S-BFRIE) is proposed for interpoation invoving situations where the consequent vaue is nown and the vaues of a but one antecedent variabe are aso given. The tas is to estimate the vaue of that singe unnown antecedent. Without osing generaity, suppose that a conventiona FRI is represented as foows: B = f FRIE ((A 1,...,A,...,A M ), (R i,i=1,...,n)) (16) where f FRIE denotes the interpoation/extrapoation process from M observed vaues, using a set of seected rues R i,i= 1,...,N, that are cosest to {A =1, 2,...,M}, and B is the interpoated concusion. S-BFRIE can then be defined in the foowing form: A = f S-BFRIE ((B,A 1,...,A 1,A +1,...,A M ) (R i,i=1,...,n)) (17) where f S-BFRIE denotes the entire process of obtaining A,the unnown (or required) observation, which is to be bacward interpoated. It uses the N cosest rues, with regard to the observed (or predicted) vaues from the (M 1) antecedents and the concusion B. A. Proposed Approach A cose examination of the T-FIR agorithm reveas that, in order to successfuy bacward interpoate the missing vaue, a number of cosest rues need to be identified first. A of the parameters that are invoved in T-FIR (for trapezoida fuzzy sets) ω, δ, s, s, S, and m aso need to be computed for the nown antecedent variabes, and now observed consequent variabe. The acquisition of these essentia parameters aows a possibe transformation process to be derived, which then heps to restore the missing antecedent vaue. The proposed S-BFRIE agorithm that refects this intuition is summarized beow. 1) Determination of the Cosest Rues: In reference to the earier definition of the S-BFRIE process in (17), when B, (A 1,...,A 1,A +1,...,A M ) are given, in order to interpoate/extrapoate the unnown antecedent A, the discovery of the cosest rues R i,i=1,...,n, are required. Instead of using the distance measure that is introduced in (), a modified scheme is proposed in order to refect the biased consideration toward the consequent variabe (as per the intuition indicated previousy) d = wb d 2 B + (w A d 2 A ). (18) =1, In impementing S-BFRIE, without sufficient expert nowedge on the reative eve of significance of different antecedents, a antecedents are treated equay w B = w A =1,w A 1 = w A = w A M = 1 M. (19) =1 Note that in choosing the cosest rues, the square root used in the origina distance measure becomes unnecessary, as ony the ordering information is needed. Therefore, the distance cacuation can be simpified to ˆd = d B M =1, d A 2. (20) 2) Construction of the Intermediate Fuzzy Terms: To hep expain, assume a certain set of cosest rues R i,i= 1,...,N,R i U that are returned by the previous distance cacuation. Foowing the origina T-FIR agorithm, in order to create the intermediate (shifted) fuzzy terms for the nown antecedent variabes: A,=1,...,M,, the foowing parameters w A i,i=1,...,n, and δ A need to be computed

5 1686 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 first according to (4) (6). The parameter vaues for the intermediate (shifted) consequent fuzzy term B : w B i,i=1,...,n, and δ B can be computed using exacty the same formuae as those of A, since its vaue B is aso directy observed. The formuae given in (7), athough no onger needed in this scenario, revea that both w B i and δ B are agebraic averages of the parameter vaues from individua antecedent terms. For instance, if A were not missing, ω A i woud become part of the sum: ω B i = 1 M M =1 ω A i in (4). Thus, it has an intuitive appea to assume that, when bacward interpoating a certain parameter vaue for A,sayω A i, the parameter vaue that is associated with the consequent variabe ω B i shoud be treated with a biased weight, which is the sum of a antecedent weights. The parameter vaues for the missing antecedent such as ω A i are then cacuated by subtracting those of the nown antecedents from that of the consequent as foows: ω A i = Mω B i =1, Foowing the same ogic, ω A can be obtained δ A = Mδ B =1, ω A i. (21) δ A. (22) The acquisition of these parameter vaues aow the construction of the intermediate (shifted) fuzzy term A for the missing antecedent dimension, simiar to (5) and (7) N A = A + δ A range A, A = ω A i A i. (2) Note that according to the characteristics of the T-FIR agorithm, this shifted fuzzy term A aso determines the representative vaue of the fina interpoation output A, since the ater transformations wi not ater Rep(A ). ) Scae and Move Transformation: Having obtained the intermediate (shifted) fuzzy terms, the essentia parameters s A, s A (or a singe scae rate s A for trianguar representation), and m A that are invoved in the transformation process can be derived. Foowing the same intuition and computationa steps as those for w A i,i=1,...,n, and δ A, by reversing the forward transformation procedure that is introduced in (10) and (15), the required vaues can be found as foows: s A s A m A = Ms B = Ms B = Mm B =1, =1, =1, i=1 s A (24) s A (25) m A (26) where s B, s B, and m B are immediatey obtainabe by resoving (8), (9), and (14). Note that to guarantee the transformed fuzzy sets to be convex, s A shoud be fixed in terms of the scae ratio S A : S A = MS B =1, S A (27) where S A is the fixed scae ratio of A, S B is the scae ratio of consequent dimension B, and S A is the scae ratio of A, = 1, 2,...,M, s A S A s s A = s A S A + s A, if s A s A 0 A s A S A, if s A s A 0. (28) Finay with a parameters acquired, the transformation on A can be performed, resuting in the (bacward) interpoated vaue A T (A,A )={s A, s A, S A,m A }. (29) B. Wored Exampes This section provides three wored exampes of the proposed BFRIE approach. For each of these, the vaue of the consequent variabe is obtained by utiizing the T-FIR method (foowing the forward FRI procedure of [21]), using randomy chosen vaues for the antecedent variabes. The missing vaue is then (purposefuy) removed from the observation, aowing the appication of BFRIE. The aim of running these exampes is twofoded: 1) to demonstrate the correctness of the BFRIE method, i.e., the proposed procedure can indeed restore the originay observed vaue (with an acceptabe degree of error), and 2) to show that the proposed distance measure is effective in identifying reevant rues in order to perform interpoation (noting that the rues that are invoved in the initia generation process may, or may not be seected). Exampe.1: S-BFRIE With Trapezoida Fuzzy Sets This exampe iustrates S-BFRIE invoving mutipe mutiantecedent rues, where the variabe vaues are represented by trapezoida membership functions. The observation and the four cosest rues are given in Tabe I and Fig. 4 (whie the subprocess of seecting the coset rues is omitted because it is a straightforward appication of (18) to the sparse rue base). Here, A is the missing antecedent, which is to be inferred. 1) Construction of the Intermediate Fuzzy Terms: As expained in Section III-A2, the normaized weights of the antecedents and observed concusion are derived according to (4), their vaues are shown in Tabe II. The parameters for the missing observation ω A i,i=1, 2,, 4, can then be cacuated using (21), resuting in: ω A 1 = 0.04, ω A 2 =0.70, ω A =0.06, ω A 4 =0.19. Fromthis, the intermediate fuzzy set A =(.9, 4.4, 5.14, 5.60) can be obtained according to (22). Then, the bias δ A between A and A is cacuated using (22), which has a vaue very cose to 0 for this particuar case, indicating that no further shifting is necessary. Therefore, the vaue of the shifted fuzzy term A =(4.19, 5.21, 5.90, 6.49) can be obtained from (5), which has the same representative vaue as A.

6 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1687 TABLE I FOUR CLOSEST RULES FOR OBSERVATION TABLE III TWO CLOSEST RULES FOR OBSERVATION Fig. 4. Exampe of B-FRI with mutipe antecedents. TABLE II NORMALIZED WEIGHTS OF GIVEN ANTECEDENTS Fig. 5. Exampe of S-BFRIE with trianguar and singeton fuzzy sets. 2) Scae and Move Transformation from A to A : The individua scae and move parameters are cacuated according to (8) (15), resuting in s B =1.4, s B =0.71, m B = 0.2. The scae ratio S B =0.47 is obtained using a formua simiar to (27). Simiary, the reevant parameters s A, s A,m A of antecedents A 1,A 2,A 4 can be obtained. Foowing this and using (24) (27), it can be cacuated that s A =1.08, s A =0.76, m A = 0.28, and S A =0.70. The scaed fuzzy term A is then computed to be (4.07, 5.2, 5.84, 6.57). Finay, the transformed A =(4.01, 5.46, 5.98, 6.50) can be obtained, which is the estimated missing vaue for x. ) Verification: The resut of BFRIE can be verified by performing the conventiona T-FIR, using the reconstructed observation invoving A. Appying forward interpoation resuts in the concusion, B =(5.46, 6.51, 6.85, 8.71), Rep(B )=6.95. This is consistent with the given observed concusion (5.50, 6.50, 7.00, 8.70), which has a representative vaue of Exampe.2: S-BFRIE With Trianguar Fuzzy Sets and Singeton vaues To further demonstrate the generaity of the proposed approach, this exampe iustrates S-BFRIE that invoves mutipe antecedent variabes with trianguar membership functions and singeton vaues. The two adjacent rues, which invove singeton fuzzy sets, are given in Tabe III and Fig. 5, with the observation being A 1 =(4, 5, 6),A 2 =(5, 6, 7),B = (10, 11, 1).

7 1688 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 TABLE IV THREE CLOSEST RULES FOR OBSERVATION 1) Construction of the Intermediate Fuzzy Terms: The parameters for the consequent dimension ω B i are cacuated according to (4); ω B 1 =0.57, ω B 2 =0.4. The parameters for the missing observation can then be cacuated using (21); ω A 1 =0.65, ω A 2 =0.5, and the intermediate fuzzy set A =(5.75, 6.10, 6.45) can be obtained with respect to (22). From this, using (5) and (22), the foowing can be obtained: δ A =0.0, and the shifted fuzzy set A =(5.75, 6.10, 6.45). 2) Scae and Move Transformation From A to A : The individua scae and move parameters can be cacuated according to (1) and (14), resuting in s B =1.7, m B = 0.. From (24) and (26), it is computed that s A =1.71 and m A =0.75. The scaed fuzzy term A is therefore (5.50, 6.10, 6.70). Finay, according to (29), the transformed A =(5.65, 5.80, 6.85) can be obtained. Again, the resut can be vaidated by performing the conventiona T-FIR using the obtained A, resuting in the concusion being (9.99, 11.00, 12.99), which is consistent with the given observed concusion (10, 11, 1). Exampe.: Bacward Fuzzy Rue Extrapoation Extrapoation is a specia case of interpoation, when a of the cosest rues chosen ie on one side of the hyperpane in which the given observation is a certain point. Determining the cosest rues and constructing the intermediate rue are carried out in the same way as those for interpoation. The exampe beow outines the ey steps in the process of bacward fuzzy rue extrapoation. Suppose that the observation and the three cosest rues as given in Tabe IV and Fig. 6 are used for extrapoation, where a rues ie on the right side of the observation. In this exampe, A 2 is the missing antecedent that is to be extrapoated. 1) Construction of the Intermediate Fuzzy Terms: The normaized weights associated with the observed antecedents and concusion are isted in Tabe V. The parameters for the missing observation ω A i 2, i =1, 2, can then be cacuated using (21) such that ω A 1 2 =0.56, ω A 2 2 = 0.27, ω A 2 =0.17, and the intermediate fuzzy set A 2 = (5.82, 6.82, 7.82, 8.82) can be obtained according to (22). Then, the bias δ A 2 between A 2 and A 2 is cacuated by (22), δ A 2 = The shifted fuzzy term A 2 which has the same representative vaue as A 2, can be obtained from (5): A 2 =(2.7,.7, 4.7, 5.7). 2) Scae and Move Transformation From A to A : The individua scae and move parameters are cacuated with respect to (8) (15), resuting in s B =0.64, s B = 0.59, m B = The scae ratio S B =0.09 is obtained according to (27). Simiary, the reevant parameters s A, s A,m A of antecedents A 1,A can be ob- Fig. 6. Exampe of bacward fuzzy rue extrapoation with mutipe mutiantecedent rues. TABLE V NORMALIZED WEIGHTS OF THE KNOWN ANTECEDENTS tained. In particuar, using equations simiar to (24) (27), it foows that s A 2 =0.40, s A 2 =0.50, m A 2 = 0.91, and S A 2 = The scaed fuzzy term A 2 can then be computed: (.27,.94, 4.52, 5.19). Finay, the required A 2 =(.14, 4.21, 4.79, 5.05) is obtained by performing the transformation. Note that as with the previous iustrative cases for interpoation, the above-extrapoated resut can be verified to match we with the observation. IV. BACKWARD FUZZY RULE INTERPOLATION AND EXTRAPOLATION WITH MULTIPLE MISSING ANTECEDENTS For practica appications, there are often more than one antecedent with missing vaues. Therefore, the question about how to perform BFRIE with mutipe missing vaues is raised. This section presents two approaches that attempt to address this issue: 1) the parametric approach (see Section IV-A), which

8 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1689 directy extends the S-BFRIE method but invoves a higher computationa compexity, and 2) the feedbac approach (see Section IV-B), which is a more generaized method that wors more cosey with conventiona FRI procedures. A. Parametric Approach 1) Probem Anaysis: The ey to soving genera BFRIE probems, foowing the principes of the S-BFRIE method, ies with the cacuation of the best T-FIR parameter combination, which eads to the cosest resembance of the origina (missing) vaues. In particuar, to create the intermediate fuzzy terms that are based on the N cosest rues, the foowing set of parameters: {(ω A i,i=1, 2...,N),δ A,s A, s A,m A } (0) of cardinaity N +4is required to bacward interpoate each missing antecedent A, given trapezoida representation. Here, L, L {1,...,M}, denote the indices of the missing antecedents A. Taing parameter for bias, δ A as an exampe, the foowing constraint needs to be satisfied: = Mδ B δ A (1) δ A L =1,/ L which is an extended form of (22) that is used in S-BFRIE. Simiar formuae for the remaining parameters may aso be derived in the exact same fashion, atogether forming mutipe simutaneous equations to be resoved. Note that apart from those determined by the aforementioned equations, by definition, these parameters aso tae vaues from their underying ranges: ω [0, 1], δ [ 1, 1], s A [0, ), s A [0, ), and m [ 1, 1]. Thus, these ranges need to be discretized in order to generate the required parameter combinations. In assessing the performance of the estimation, the conventiona T-FIR procedure can then be invoed to verify the correctness and accuracy. This is done by comparing the output (using the estimated parameters) with the actua observed consequent vaue so that the most suitabe setting may be identified. After a of the parameters are acquired, in theory, the missing antecedents may be expected to be individuay derived using the previousy described S-BFRIE steps. Unfortunatey, the set of simutaneous equations cannot be resoved in a straightforward manner, due to the ac of sufficient given vaues. This is because different observations may potentiay ead to the same (or very simiar) consequent, for any system of a fair compexity. As the number of missing antecedent vaues increases, the possibe scenarios may become extremey wide-reaching or even countess. From a theoretica point of view, the compexity of this approach mainy comes from the high number of possibe parameter combinations (ω, δ, s, s, and m), a nonindependent. The weight ω, in particuar, is cacuated with regard to a L missing antecedents and a N cosest rues, thus having a consideraby high compexity O(υ N L ). Here, υ N +,υ >1, signifies the number of discretized intervas that are used to generate the possibe parameter combinations. Higher υ produces finer intervas and aows coser estimations to the actua Fig. 7. Fowchart of the parametric approach. vaues. The discretizations of the other four parameters: δ, s, s, and m a have the same computationa compexity of O(υ L ). Therefore, the overa computationa compexity of generating these combinations is O(υ (N +4) L ), which is prohibitive for arge υ, N, and L. The situation worsens if the verification of every possibe parameter combination is required in order to vaidate that the estimated transformations indeed produce reasonabe resuts, and to enabe the seection of better (coser) outputs. The resutant P-BFRIE process wi have an overa cost of O(υ (N +4) L ) O(FRI), where O(FRI) stands for the compexity of the FRIprocess itsef. 2) Simpified P-BFRIE for Two Missing Antecedent Vaues: Having undertaen the aforementioned anaysis of the computationa compexity that theoretica T-FIR invoves, practicay simper methods are necessary. For this, a simpified process that supports the interpoation of two missing antecedent vaues is proposed here, as outined in Fig. 7. The cost of discretizing ω is reduced to O(υ N ), with an overa time compexity of O(υ N +4 ) O(FRI). This simpification taes advantage of the codependence of the two weights ω A i and ω A i 1, associated 2 with each rue R i ω A i 1 + ω A i 2 = Mω B i =1,/ L ω A i (2)

9 1690 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 TABLE VI TWO CLOSEST RULES FOR OBSERVATION TABLE VII PARAMETERS FOR ESTIMATING THE MISSING ANTECEDENTS where one vaue uniquey determines the other during the discretization process. The outcomes of the now more manageabe υ N +4 parameter combinations can then be verified through FRI, where a simpe measurement of percentage error ɛ j % maybeused ɛ j % = 100 d(bj,b ). () Here, the two estimated missing antecedent vaues A j 1 and A j 2, bacward interpoated through the use of the jth parameter combination, are empoyed to obtain a certain B j. The distance between this estimated consequent and the actua observed consequent B determines the accuracy of the transformations that have taen pace. Finay, the set of A j, which corresponds to the minima resuting error, is chosen as the desired output, since it is the best approximation possibe given the imited information, and the amount of discretized intervas empoyed. Exampe 4.1: Tabe VI ists the observation and the cosest rues chosen according to (20), where x and x 5 are assumed to have two missing antecedent vaues. For this particuar exampe, if υ =12is used to discretize each parameter, the tota number of parameter combinations ω, δ, s, and m is 12 (2+4) (with N =2), resuting in the same number of pairs of possibe A j and A j 5. The parameters corresponding to the vaidated consequent with the smaest error ɛ j % =0.22% [according to ()] are isted in Tabe VII, where B j = (12.70, 1.0, 1.6, 14.66). The fina bacward interpoative vaues for the two missing antecedents are A =(2.02, 2.18, 2.40, 2.77) and A 5 = (0.52, 2.17,.0,.55). B. Feedbac Approach This section describes an aternative and more intuitive approach to BFRIE, termed the feedbac approach (F-BFRIE). It significanty reduces the time-compexity for parameter estimation. This is shown in the fowchart of Fig. 8 and iustrated in Agorithm 1. It wors by directy estimating the possibe initia vaues of the missing antecedents, then vaidating the resutant consequent through conventiona FRI, in order to identify the most suitabe vaue combination(s) that ead to the observed consequent vaue. For consistency and ease of expanation, as Fig. 8. Fowchart of the feedbac approach. with P-BFRIE, mechanisms such as T-FIR, discretization of variabe ranges, and the percentage error-based vaidation are again used in the impementation. In order to obtain the initia estimation, the domain ranges of the missing antecedents themseves (rather than those of the parameters previousy used for P-BFRIE) are discretized into υ intervas. The resuting crisp points are then used to approximate a tota of υ L possibe vaue combinations for the missing antecedent variabes {A j }, L, j =1,...,υ L. Assume a given crisp point c j for the th antecedent variabe, (4) (6) detai this approximation procedure, which is denoted by approx(c j ) in Agorithm 1

10 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1691 TABLE VIII CLOSEST RULES FOR TWO EXAMPLE RECONSTRUCTED OBSERVATIONS O p AND O q,p,q 1,...,υ L a 0A j a 1A j a 2A j a A j N = c j i=1 (Rep(Ai ) a 0 A i ) Δ N N = c j i=1 (Rep(Ai ) a 1 A i ) Δ N N = c j i=1 + (a 2 A i Rep(A i )) Δ N N = c j i=1 + (a A i Rep(A i )) Δ N (4) TABLE IX ERRORS ɛ % BETWEEN B p,b q, AND B where Δ = Δ= N ( M a A a 0 A =1,/ L N ( i =1 a A i a 0 A i ( M N =1,/ L ) + M L +1 a 2 A a 1 A N ( i =1 a 2 A i a 1 A i ) + M L +1 a B a 0 B N i =1(a B i a 0 B i ) ) (5) ) a 2 B a 1 B N i =1(a 2 B i a 1 B i ) (6) where M is the tota number of antecedent variabes and N is the number of the cosest rues. To obtain cose approximations of the missing vaues, it is usefu to remember that the estimated fuzzy sets A j are infuenced by both the seected rues: R i,i=1,...,n, and the observed vaues A, / L and B. In particuar, for a trapezoida fuzzy set A j that may be returned as the estimated outcome, the positions of its four points are defined reative to the averaged (over the N cosest rues) dispacements between their corresponding points and the representative vaues of the fuzzy antecedents A i,i=1,...,n. The points a 0A j and a A j are then scaed with respect to the ratio Δ, which is cacuated from the averaged (over the N cosest rues and a nown antecedent/consequent dimensions) ratios between the supports of the observed vaues, and those of the existing rues. Simiary, a 1A j and a A j are adjusted with respect to Δ. The υ L possibe combinations of the fuzzy sets being estimated are used to obtain their respective consequent vaues B j,j =1,...,υ L, through the conventiona T-FIR procedure. Note that the cosest rues chosen for each of the combinations may be different, since the distance cacuation is purey based on the vaues of the currenty estimated observations. The percentage error ɛ j % is then cacuated using (), and the estimated missing antecedent vaues corresponding to the smaest ɛ j % are returned as the fina resut. The computationa compexity of this approach is principay due to the generation process of the initia fuzzy sets O(υ L ), which is much more scaabe than that of P-BFRIE: O(υ (N +4) L ). The run-time cost of F-BFRIE is aso independent of the number of cosest rues N, and of course, the overhead incurred by estimating the T-FIR parameters as required in P-BFRIE is aso eiminated. Exampe 4.2: Consider the observation givens in Tabe VIII, where the vaues of x 1,x,x 5, and x 7 areassumedtobemissing. According to (4), a tota of 20 4 (υ =20, L =4) possibe vaue combinations are used to generate the same number of potentia consequent vaues. O p and O q, which are two of such combinations obtained in the process and the different cosest rues chosen using (20) are shown in Tabe VIII. After forward interpoation with these approximated fuzzy

11 1692 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 Fig. 9. Causa networ mode. sets, the ɛ j % obtained during this exampe evauation is given in Tabe IX. The smaest ɛ % is 0.74%, and the corresponding consequent B q =(9.74, 10.16, 10.9, 11.08). The resutant estimated outcome of the four missing antecedents A 1 = (1.7, 2.70,.57,.97), A =(2.,.5, 4.51, 5.58), A 5 = (4.7, 5.71, 6.61, 7.44), and A 7 =(.87, 4.65, 5.2, 5.79) is therefore the fina BFRIE resut. V. EXPERIMENTATION AND DISCUSSION In this section, a practica probem concerning the prediction of terrorist bombing threats is empoyed to demonstrate the potentia of the proposed study. It shows how the impemented techniques hep to interpoate the fina concusion, when the system is presented with partia observations, incuding the refection of interesting characteristics of the proposed study. In addition, a comparative study of the parametric and feedbac methods is aso incuded, which is systematicay conducted using a randomized numerica probem, in terms of their approximation accuracy and run-time efficiency. A. Prediction of Terrorist Bombing Threats 1) Probem Specification and Mode Construction: To sove the puzze of a serious crime incuding terrorist attacs from a set of given evidence, investigators aim to reconstruct the possibe scenarios that may have taen pace. The bottenec of accompishing this tas is the fact that humans are reativey inefficient at hypothetica reasoning, especiay when a hierarchicay structured procedure is invoved. A fuzzy decision-support system may assist investigators in generating pausibe scenarios and anayzing them objectivey [15], [41]. In this paper, a rea-word scenario that invoves the prediction of a terrorist threat is considered, which is based on a recent study on suicide bombings [40]. The causa networ mode used in this experiment is iustrated in Fig. 9. There are 11 variabes in this probem, denoted as x p,p=1, 2,...,11, invoved in four subsets of rues. 1) The first subset concerns the Popuarity of a pace. Main shopping (x 1 ) and Iconic (x 2 ) indicate whether the ocation is a principa shopping area, or a site of symboic vaue, respectivey. Main street (x ) describes how busy the area may become. The variabe Easy access (x 4 ) refers to the convenience of transportation. These factors jointy determine the Popuarity (x 8 ) of a given ocation. Fig. 10. Definition of the inguistic terms for domain variabes. 2) The second subset of rues deas with the Warning eve (x 9 ), or the amount of ris to the attacers, which is reated to the number of security Guards (x 5 ) in the area, the Aertness (x 6 ) of peope, and the numbers of Repetitive attacs (x 7 ) in the past. ) The third focuses on the prediction of Crowdedness (x 10 ). The number of peope in an area is directy reated to the Popuarity of the pace, and can aso be affected by the eve of Easy access. In addition, the Crowdedness may change in reation to the Warning eve, since cautious individuas may shy away from paces that are considered dangerous. 4) The fourth and fina rue subset is about the Exposion ieihood (x 11 ), which is in indirect reation to the number of peope in the area. In addition, the amount of pubic warning signs dispayed in the area may discourage potentia attacs, as peope are more aert to the surroundings, and suspicious individuas or items may be prompty reported. Moreover, terrorists typicay target certain paces that repetitivey draw their attention, instead of any ocations at random. A seection of the origina rues contained in the rue base are given in Tabe X. In this mode, fuzziness is naturay obtained from the presence of the inguistic terms that describe the rea-vaued domain variabes. For simpicity, trianguar fuzzy membership functions are appied in this scenario. Note that different variabes are defined on their own underying domains. To simpify nowedge representation, these domains are normaized in this experiment with a range of 0 to 1. The fuzzy sets that represent the normaized inguistic terms are given in Fig. 10. It is important to note that the origina rue base consists of substantia gaps, which maes interpoation essentia. 2) Wor Fow of BFRIE in Action: The set of observations used in this experiment is given in Tabe XI, where the vaues

12 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 169 TABLE X EXAMPLE RULES VL: Very Low, L: Low, ML: Moderate Low, M: Moderate, MH: Moderate High, H: High, VH: Very High. TABLE XI OBSERVATIONS Fig. 11. Exampe structure for bombing attac prediction. of the antecedent variabes Easy access (x 4 ), Guards (x 5 ), and Warning eve (x 9 ) are a missing, as we as that of the fina consequent x 11 to be interpoated. The antecedent variabe Warning eve (x 9 ) is of particuar importance, since it is invoved in two subsets of rues (for Crowdedness and Exposion ieihood). Without x 9, no matter what other information is avaiabe, even with the Repetitive attac (x 7 ) and Crowdedness (x 10 ) nown, forward interpoation wi sti fai. It is not uncommon for a hierarchica reasoning framewor to have more than one path of inference/interpoation [17], [6]. For this particuar set of observations, it is possibe to obtain the vaue of x 9 through two different paths, as shown in Fig ) Dotted Path via S-BFRIE: a) Cacuate the vaue of Easy access (x 4 ) according to the given consequence vaue x 8 and the antecedent vaues x 1,x 2, and x, using the subrue base Popuarity via S-BFRIE. Foowing the steps detaied previousy, the bacward interpoated vaue is x 4 =(0.42, 0.52, 0.62) (M). b) Interpoate the vaue of x 9 using the subrue base Crowdedness. The vaues of x 8 and x 10 are directy observed, and that of x 4 is obtained from the previous step. The three cosest rues are then seected using the consequence-biased distance measure as per (20). The resuting cosest rues with respect to the observation, and the bacward interpoated vaue x 9 =(0.52, 0.62, 0.72) (MH), are shown in Fig. 12. c) Use the interpoated vaue of x 9, and the other given vaues for x 7 and x 10 to forward interpoate the fina consequent variabe Exposion ieihood

13 1694 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 TABLE XII OBSERVATION USED FOR THE INVESTIGATION OF MULTIPLE POSSIBLE OUTCOMES Fig. 12. Cacuate the Warning eve using S-BFRIE. (x 11 ). This eads to the required fina resut x 11 = (0.61, 0.71, 0.81) (MH). 2) Dashed Path via P-BFRIE and F-BFRIE a) Cacuate x 4 and x 9 simutaneousy, according to the given vaues x 8 and x 10, and the subrue base Crowdedness. By empoying the P-BFRIE approach, the resuts obtained are x 4 =(0.46, 0.56, 0.66) (M) and x 9 =(0.51, 0.61, 0.71) (MH). Aternativey, the resuts obtained using F-BFRIE are x 4 =(0.4, 0.5, 0.6) (M) and x 9 =(0.59, 0.69, 0.79) (MH). Both agree with the previous resuts via the use of S-BFRIE: x 4 = (0.42, 0.52, 0.62) (M) and x 9 =(0.52, 0.62, 0.72) (MH). b) Derive, in a manner simiar to the dotted path above, the vaue of the fina consequent variabe x 11 =(0.58, 0.68, 0.78) (MH) (according to P-BFRIE: x 9 =(0.51, 0.61, 0.71)) and x 11 = (0.56, 0.66, 0.76) (MH) (according to F-BFRIE: x 9 =(0.59, 0.69, 0.79)). The errors ɛ % between the resut of the dotted path (x 11 =(0.61, 0.71, 0.81)) and these are.0% and 5.0%, respectivey. This demonstrates that both processing paths are feasibe for deaing with this probem. ) Note that after the aforementioned interpoation process, the ast remaining missing vaue for Guards (x 5 ), athough no use in this particuar prediction appication, can aso be bacward interpoated. In particuar, by using the observed vaues of x 6 and x 7, and the interpoated vaue of x 9,the resut of x 5 =(0.28, 0.8, 0.48) (ML) can be obtained via the use of subrue base Warning eve. ) Practica Significance of BFRIE: In rea appications, it is often difficut to predict and adjust the Warning eve unti (suicide) bombing attacs have actuay occurred or prevented. However, with BFRIE, the Warning eve may now be estimated from the other reated factors. This may significanty increase the effectiveness of the predication and prevention of bombing attacs. However, Easy access and Guards are controabe eements which may be adjusted in order to minimize the Exposion ieihood. In order to reduce the Exposion ieihood (x 11 ) from (0.61, 0.71, 0.81) (MH) say, to(0.0, 0.40, 0.50) (ML), according to the proposed P-BFRIE method, the vaue of Easy access (x 4 ) needs to be changed from the current (0.42, 0.52, 0.62) (M) to (0.2, 0., 0.4) (ML), and simiary, Warning eve (x 9 ) from the current (0.52, 0.62, 0.72) (MH) to(0.74, 0.8, 0.94) (H), and Guards (x 5 ) from the current (0.28, 0.8, 0.48) (ML) to (0.54, 0.64, 0.74) (MH). Thus, with the use of the proposed reverse reasoning technique to interpoate the crucia variabes, the ris of a certain area concerned may be significanty reduced (or future repetitive attacs prevented). 4) Use of Aternative Distance Metrics: If the unbiased distance measure () is used to bacward interpoate the missing vaue x 9, the same cosest rues wi no onger be seected. Instead a different outcome of Warning eve x 9 =(0.18, 0.28, 0.8), and Exposion ieihood x 11 =(0.29, 0.9, 0.49) (ML) wi be returned. Looing bac at the origina observation, given the two antecedent vaues such that Easy access (x 4 )ism and Popuarity (x 8 )ish, the intuitive deduction of Crowdedness (x 10 ) shoud be quite high, as the pace is both moderatey high in popuarity and reasonaby convenient to reach. The ony reason why the observed Crowdedness (x 10 ) has a moderate vaue may we be because of a reasonabe Warning eve. Therefore, the outcome x 9 =(0.52, 0.62, 0.72) from the use of the biased distance measure is more agreeabe than x 9 =(0.18, 0.28, 0.8) resuting from the use of the pain distance measure. Experiments show that, if x 9 =(0.18, 0.28, 0.8) and the antecedent vaues of x 4 and x 8 are M and H respectivey, T-FIR method wi resut in an interpoated x 10 =(0.14, 0.24, 0.4) (L), which wi be much further than the origina observation: Crowdedness (x 10 ) is M. This ceary demonstrates the significance in utiizing the biased distance metric proposed in this paper. 5) Mutipe Equay Probabe Interpoative Outcomes: The invovement of mutipe missing antecedents naturay impies that aternative equay probabe combinations of observations may be present, which may a ead to the same consequent observed. Assume that the observation shown in Tabe XII is

14 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1695 Fig. 1. Reationship between the vaue of υ, approximation error ɛ %, and execution time. TABLE XIII RELATIONSHIP BETWEEN Main Shopping AND Iconic TABLE XIV ERRORS OF P-BFRIE AND F-BFRIE OVER 500 TEST SAMPLES given, where the vaues of Main shopping and Iconic are both missing. The P-BFRIE is adopted to cacuate these two missing antecedent vaues and the different resuts found with simiar errors (ɛ % < 1.50%) are summarized in Tabe XIII. These resuts revea that the same vaue of Popuarity can be obtained when either Main shopping or Iconic is High or Very High.Thisaso agrees with the intuition, since either of these variabes may cause a given ocation to be attractive, and both may be equay effective to infuence the fina outcome. Theoreticay speaing, this particuar subset of rues contains redundant variabes, and may be further pruned for higher efficiency. The presence of such redundancy may prompt the use of dimensionaity reducing techniques such as feature seection [12], [22]. B. Comparative Studies To systematicay compare the two proposed methods: P-BFRIE and F-BFRIE, a numerica function shown in (7) is used. The rue base empoyed in the experimenta evauation is generated using the foowing steps: 1) a random set of crisp vaues are seected for the function variabes and the outcome is cacuated according to (7); 2) these crisp vaues are then fuzzified into trapezoida fuzzy sets; and ) the rue base is then popuated using these randomy generated rues, whie checing (and where appropriate, removing) rues to ensure the underying domain is reasonaby covered, whie there sti exist sufficient gaps between rues in order to utiize interpoation y =x 1.x x +0.5x x 5. (7) This experimenta setup enabes an initia sparse rue base to be generated that is an approximation of the underying nowedge, simuating those obtainabe by subject experts. An observation is obtained in a simiar manner, where the missing vaues are then purposefuy removed to faciitate bacward reasoning. Since the underying function, i.e., ground truth, is avaiabe. The consistency, accuracy, and robustness of the interpoative procedure can then be verified by comparing the outcome of the interpoation to the actua vaue computed using (7). This test, therefore, refects an underying principe simiar to that behind cross vaidation and statistica evauation [], []. Atogether, 500 simuated sampes are randomy drawn from the domain U =[0, 10] 5. Without osing generaity, the vaues of x and x 5 are assumed to be missing. The errors for the consequent and the missing antecedents over these testing records are summarized in Tabe XIV. For the consequent variabe B, the errors are obtained by cacuating the distance between the estimated consequent B j and the actua vaue B. The errors of the two antecedent variabes with missing vaues A and A 5 are derived from the distances between the interpoative outcomes (i.e., vaues corresponding to the smaest consequent error) and the actua vaues of A and A 5 (the ground truths). It can be seen that the parametric approach demonstrates a higher accuracy than F-BFRIE. This is iey because the parametric

15 1696 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 approach has precise contro of the parameter vaues. In addition, the shapes of the initia fuzzy sets used in F-BFRIE are approximated, which may have affected its performance. Nevertheess, both methods seem to have an acceptabe eve of errors. The vaue of υ (the number of discretized intervas per variabe) is an important factor for both approaches. Fig. 1 presents the reationships between the approximation errors and the execution times with respect to various vaues of intervas υ, for the two proposed methods. The resuts show that the parametric approach produces a higher accuracy when a arger number of intervas is used. However, it is aso ess scaabe. This experimentay demonstrates that the run-time compexity and memory requirement of F-BFRIE are more reaxed, which aso agrees with the theoretica anaysis regarding their compexities in Sections IV-A and IV-B. VI. CONCLUSION This paper has presented BFRIE, a nove approach that compements traditiona FRI by supporting bacward inference, aowing fexibe interpoation when certain antecedents are missing from the observation. This study is based upon the mechanisms of T-FIR, in order to hande mutipe mutiantecedent rues and to ensure the maintenance of convexity and normaity of interpoated outcomes. Specific agorithms have been deveoped to tace probem scenarios with singe and mutipe missing vaues, with wored exampes provided to iustrate their operations. Its practica significance and potentia have been demonstrated with a rea-word probem scenario: the prediction of terrorist bombing attacs. Systematic evauation resuts aso show that P-BFRIE is more accurate, despite its imited scaabiity for arger probems. The F-BFRIE can successfuy hande more compex scenarios, and significanty reduces the need of parameter cacuation, but its interpoative accuracy is reativey ower. The current techniques are impemented using exhaustive search-based methods, which may be better formuated with advanced soution techniques (e.g., Watz agorithm [47]) for fexibe constraint satisfaction [7]. It may aso be further improved via the use of heuristic optimization agorithms [12], [5], [48] that do not require domain discretization. This may hep to obtain even better resuts, whie reducing the search cost (for P-BFRIE in particuar). This study wi aso benefit from a mechanism for automatic identification and seection of better reasoning paths in handing hierarchica rue modes so that the interpoation process may dynamicay proceed [8] according to the current states of the system. The underying concept that supports the proposed BFRIE seems to bear cose reation to that of fuzzy inversion [2], [45], [46]. A systematic comparison between these two approaches is, however, beyond the scope of this paper and remains active research. It is aso worth extending the BFRIE approach to support other types of interpoation methods (e.g., GM [1], FIVE [1], IMUL [49]). Whie in principe, the idea of BFRIE (or bacward reasoning in genera) appears to be appicabe to both Mamdani and TSK fuzzy systems [14], [18], [25], [26], for the present impementation, the technique described reies on the scae and move transformation-based procedures and is therefore, ony appicabe to Mamdani modes. It is of natura appea to deveop the proposed technique for TSK fuzzy modes. Intuitivey, it may aso be interesting to appy the technique to different probem domains, such as networ intrusion detection [42] and oi exporation [49]. As indicated previousy, the underying probem that BFRIE addresses is that of many to one, where a number of different vaue combinations, for the antecedent variabes, may ead to very simiar observed vaues for the consequent variabe. Athough this issue has been party anayzed from an experimenta viewpoint, much remains to be done. In particuar, probem may exacerbate if the number of missing vaues becomes arger. This maes it very chaenging to restore the true origina observation. Fortunatey, different observations obtained via the BFRIE process wi generate the same or very simiar outcomes. Thus, they may be regarded as equay possibe given the imited amount of nowedge with regard to the appication probem. Nevertheess, it is important to be abe to improve the proposed approach in an effort to better hande the many to one probem. This wi be of practica significance for BFRIE to be empoyed by accuracy-critica appications (e.g., medica diagnosis [16]). The proposed methods may be further extended and combined with the adaptive fuzzy interpoation technique which ensures inference consistency [52]. Furthermore, an inteigent antecedent and/or rue seection procedure may be deveoped by identifying the most reevant information [5], [11], [12], [9] so that the appropriate terms or rues can be determined to minimize the overa system compexity. REFERENCES [1] P. Baranyi, L. T. Kóczy, and T. D. Gedeon, A generaized concept for fuzzy rue interpoation, IEEE Trans. Fuzzy Syst.,vo.12,no.6,pp , Dec [2] P. Baranyi, P. Korondi, H. Hashimoto, and M. Wada, Fuzzy inversion and rue base reduction, in Proc. Int. Conf. Inte. Eng. Syst., 1997, pp [] G. Bontempi, H. Bersini, and M. Birattari, The oca paradigm for modeing and contro: From neuro-fuzzy to azy earning, Fuzzy Sets Syst., vo. 121, no. 1, pp , [4] T. Boongoen, Q. Shen, and C. Price, Discosing fase identity through hybrid in anaysis, Artif. Inte. Law, vo. 18, no. 1, pp , [5] T. Boongoen and Q. Shen, Nearest-neighbor guided evauation of data reiabiity and its appications, IEEE Trans. Syst., Man, Cybern., vo.40, no. 6, pp , Dec [6] B. Bouchon-Meunier, R. Mesiar, C. Marsaa, and M. Rifqi, Compositiona rue of inference as an anaogica scheme, Fuzzy Sets Syst.,vo.18, no. 1, pp. 5 65, 200. [7] Y. Chang, S. Chen, and C. Liau, Fuzzy interpoative reasoning for sparse fuzzy rue-based systems based on the areas of fuzzy sets, IEEE Trans. Fuzzy Syst., vo. 16, no. 5, pp , Oct [8] S. Chen and Y. Chang, Fuzzy rue interpoation based on the ratio of fuzziness of interva type-2 fuzzy sets, Expert Syst. App.,vo.8,no.10, pp , [9] S. Chen, Y. Chang, and J. Pan, Fuzzy rues interpoation for sparse fuzzy rue-based systems based on interva type-2 Gaussian fuzzy sets and genetic agorithms, IEEE Trans. Fuzzy Syst.,vo.21,no.,pp , Jun [10] S. Chen and Y. Ko, Fuzzy interpoative reasoning for sparse fuzzy ruebased systems based on α-cuts and transformations techniques, IEEE Trans. Fuzzy Syst., vo. 16, no. 6, pp , Dec

16 JIN et a.: BACKWARD FUZZY RULE INTERPOLATION 1697 [11] R. Diao, F. Chao, T. Peng, N. Snooe, and Q. Shen, Feature seection inspired cassifier ensembe reduction, IEEE Trans. Cybern., 201, to be pubished. DOI: /TCYB [12] R. Diao and Q. Shen, Feature seection with harmony search, IEEE Trans. Syst., Man, Cybern. B, vo. 42, no. 6, pp , Dec [1] D. Dubois and H. Prade, On fuzzy interpoation*, Int. J. Genera Syst., vo. 28, no. 2-, pp , [14] G. Feng, A survey on anaysis and design of mode-based fuzzy contro systems, IEEE Trans. Fuzzy Syst., vo. 14, no. 5, pp , Oct [15] X. Fu and Q. Shen, Fuzzy compositiona modeing, IEEE Trans. Fuzzy Syst., vo. 18, no. 4, pp , Aug [16] I. Gadaras and L. Mihaiov, An interpretabe fuzzy rue-based cassification methodoogy for medica diagnosis, Artif. Inte. Med., vo. 47, no. 1, pp , [17] H. A. Hagras, A hierarchica type-2 fuzzy ogic contro architecture for autonomous mobie robots, IEEE Trans. Fuzzy Syst., vo. 12, no. 4, pp , Aug [18] F. Hoffmann, D. Schauten, and S. Hoemann, Incrementa evoutionary design of ts fuzzy controers, IEEE Trans. Fuzzy Syst., vo. 15, no. 4, pp , Aug [19] W. Hsiao, S. Chen, and C. Lee, A new interpoative reasoning method in sparse rue-based systems, Fuzzy Sets Syst., vo. 9, no. 1, pp , [20] Z. Huang and Q. Shen, Fuzzy interpoative reasoning via scae and move transformations, IEEE Trans. Fuzzy Syst., vo. 14, no. 2, pp , Apr [21] Z. Huang and Q. Shen, Fuzzy interpoation and extrapoation: A practica approach, IEEE Trans. Fuzzy Syst., vo. 16, no. 1, pp. 1 28, Feb [22] R. Jensen and Q. Shen, New approaches to fuzzy-rough feature seection, IEEE Trans. Fuzzy Syst., vo. 17, no. 4, pp , Aug [2] S. Jin, R. Diao, C. Que, and Q. Shen, Bacward fuzzy rue interpoation with mutipe missing vaues, in Proc. Int. Conf. Fuzzy Syst., 201,pp.1 8. [24] S. Jin, R. Diao, and Q. Shen, Bacward fuzzy interpoation and extrapoation with mutipe muti-antecedent rues, in Proc. Int. Conf. Fuzzy Syst., 2012, pp [25] Y. Jin, Fuzzy modeing of high-dimensiona systems: compexity reduction and interpretabiity improvement, IEEE Trans. Fuzzy Syst., vo. 8, no. 2, pp , Apr [26] T. A. Johansen, R. Shorten, and R. Murray-Smith, On the interpretation and identification of dynamic Taagi-Sugeno fuzzy modes, IEEE Trans. Fuzzy Syst., vo. 8, no., pp , Jun [27] L. T. Kóczy, K. Hirota, and L. Muresan, Interpoation in hierarchica fuzzy rue bases, in Proc. Int. Conf. Fuzzy Syst., 2000, pp [28] L. Koczy and K. Hirota, Approximate reasoning by inear rue interpoation and genera approximation, Int. J. Approx. Reason., vo. 9, no., pp , 199. [29] L. Koczy and K. Hirota, Interpoative reasoning with insufficient evidence in sparse fuzzy rue bases, Inf. Sci., vo. 71, no. 1 2, pp , 199. [0] L. Koczy and K. Hirota, Size reduction by interpoation in fuzzy rue bases, IEEE Trans. Syst., Man, Cybern. B,vo.27,no.1,pp.14 25,Feb [1] S. Kovács, Extending the fuzzy rue interpoation FIVE by fuzzy observation, Comput. Inte., Theory App., vo. 8, pp , [2] S. Kovács, Specia issue on fuzzy rue interpoation, J. Adv. Comput. Inte. Inte. Informat., p. 25, [] L. Kuncheva, Fuzzy versus nonfuzzy in combining cassifiers designed by boosting, IEEE Trans. Fuzzy Syst., vo. 11, no. 6, pp , Dec [4] L. Lee and S. Chen, Fuzzy interpoative reasoning using interva type-2 fuzzy sets, New Front. App. Artif. Inte., vo. 5027, pp , [5] M. Lee, H. Chung, and F. Yu, Modeing of hierarchica fuzzy systems, Fuzzy Sets Syst., vo. 18, no. 2, pp. 4 61, 200. [6] Z.-Q. Liu and R. Satur, Contextua fuzzy cognitive map for decision support in geographic information systems, IEEE Trans. Fuzzy Syst., vo. 7, no. 5, pp , Oct [7] I. Migue and Q. Shen, Fuzzy rrdfcsp and panning, Artif. Inte., vo. 148, no. 1, pp , 200. [8] N. Nai, R. Diao, C. Que, and Q. Shen, Towards dynamic fuzzy rue interpoation, in Proc. Int. Conf. Fuzzy Syst., 201, pp [9] N. M. Parthaain and R. Jensen, Simutaneous feature and instance seection using fuzzy-rough bireducts, in Proc. Int. Conf. Fuzzy Syst., 201, pp [40] W. L. Perry, C. Berrebi, R. A. Brown, J. Hoywood, A. Jaycocs, P. Roshan, T. Suivan, and L. Miyashiro, Predicting Suicide Attacs: Integrating Spatia, Tempora, and Socia Features of Terrorist Attac Targets. Santa Monica, CA, USA: RAND, 201. [41] Q. Shen, J. Keppens, C. Aiten, B. Schafer, and M. Lee, A scenario-driven decision support system for serious crime investigation, Law, Probabi. Ris, vo. 5, no. 2, pp , [42] A. Tajbahsh, M. Rahmati, and A. Mirzaei, Intrusion detection using fuzzy association rues, App. Soft Comput., vo. 9, no. 2, pp , [4] D. Ti and P. Baranyi, Comprehensive anaysis of a new fuzzy rue interpoation method, IEEE Trans. Fuzzy Syst., vo. 8, no., pp , Jun [44] D. Ti, I. Joó, L. Kóczy,P.Várai, B. Moser, and T. Gedeon, Stabiity of interpoative fuzzy KH controers, Fuzzy Sets Syst., vo. 125, no. 1, pp , [45] A. R. Váronyi-Kóczy, A. Amos, andt. Kovácsházy, Genetic agorithms in fuzzy mode inversion, in Proc. Int. Conf. Fuzzy Syst., 1999, vo., pp [46] A. R. Váronyi-Kóczy,G.Pécei, T. P. Dobrowieci, and T. Kovácsházy, Iterative fuzzy mode inversion, in Proc. Int. Conf. Fuzzy Syst., 1998, vo. 1, pp [47] D. Watz, Understanding ine drawings of scenes with shadows, in The Psychoogy of Computer Vision. New Yor, NY, USA: McGraw-Hi, 1975, pp [48] D. Wang, X. Zeng, and J. Keane, Intermediate variabe normaization for gradient descent earning for hierarchica fuzzy system, IEEE Trans. Fuzzy Syst., vo. 17, no. 2, pp , Apr [49] K. W. Wong, D. Ti, T. D. Gedeon, and L. T. Kóczy, Fuzzy rue interpoation for mutidimensiona input spaces with appications: A case study, IEEE Trans. Fuzzy Syst., vo. 1, no. 6, pp , Dec [50] Y. Yam and L. Kóczy, Representing membership functions as points in high-dimensiona spaces for fuzzy interpoation and extrapoation, IEEE Trans. Fuzzy Syst., vo. 8, no. 6, pp , Dec [51] Y. Yam, M. Wong, and P. Baranyi, Interpoation with function space representation of membership functions, IEEE Trans. Fuzzy Syst.,vo.14, no., pp , Jun [52] L. Yang and Q. Shen, Adaptive fuzzy interpoation, IEEE Trans. Fuzzy Syst., vo. 19, no. 6, pp , Dec [5] L. Yang and Q. Shen, Cosed form fuzzy interpoation, Fuzzy Sets Syst., vo. 225, pp. 1 22, 201. Shangzhu Jin received the B.Sc. degree in computer science from Beijing Technoogy and Business University, Beijing, China, and the M.Sc. degree in contro theory and contro engineering from Yanshan University, Qinhuangdao, China. He is currenty woring toward the Ph.D. degree in computer science with the Department of Computer Science, Institute of Mathematics, Physics and Computer Science, Aberystwyth University, Aberystwyth, U.K. His research interests incude fuzzy systems, approximative reasoning, and networ security. Ren Diao received the M.A. and M.Sc. degrees from the University of Cambridge, U.K., and the University of Birmingham, U.K., respectivey, and the Ph.D. degree from Aberystwyth University, Aberystwyth, U.K. He is currenty a Research Feow with the Department of Computer Science, Institute of Mathematics, Physics and Computer Science, Aberystwyth University. His research interests incude fuzzy set theory, nature inspired heuristics, and machine earning. He has pubished more than 20 peer-reviewed papers in eading internationa journas and conferences.

17 1698 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 22, NO. 6, DECEMBER 2014 Chai Que (SM 10) received the B.Sc. degree in eectrica and eectronics engineering and the Ph.D. degree in inteigent contro from Heriot-Watt University, Edinburgh, U.K., in 1986 and 1990, respectivey. He is currenty the Assistant Chair with the Schoo of Computer Engineering, Nanyang Technoogica University (NTU), Singapore, and a Research Member with the Centre for Computationa Inteigence, NTU. His research interests incude inteigent contro, inteigent architectures, computationa finance, neura networs, fuzzy neura systems, neurocognitive informatics, and genetic agorithms. Dr. Que is a member of the IEEE Technica Committee on Computationa Finance. Qiang Shen received the Ph.D. degree from Heriot-Watt University, Edinburgh, U.K., and the D.Sc. degree from Aberystwyth University, Aberystwyth, U.K. He hods the estabished Chair in computer science and is the Director with the Institute of Mathematics, Physics and Computer Science, Aberystwyth University. His research interests incude computationa inteigence, reasoning under uncertainty, pattern recognition, data mining, and rea-word appications of such techniques for inteigent decision support (e.g., crime detection, consumer profiing, systems monitoring, and medica diagnosis). He has authored two research monographs and more than 10 peer-reviewed papers. Dr. Shen is a ong-serving Associate Editor of the IEEE TRANSACTIONS ON CYBERNETICS and the IEEE TRANSACTIONS ON FUZZY SYSTEMS and an editoria board member of severa other eading internationa journas. He received an Outstanding Transactions Paper Award from IEEE. He is a Feow of the Learned Society of Waes.