CLUSTERED BASED TAKAGI-SUGENO NEURO-FUZZY MODELING OF A MULTIVARIABLE NONLINEAR DYNAMIC SYSTEM

Asian Jounal of Contol, Vol. 7, No., pp. 63-76, June 5 63 -Bief Pape- CLUSTERED BASED TAKAGI-SUGENO NEURO-FUZZY MODELING OF A MULTIVARIABLE NONLINEAR DYNAMIC SYSTEM E. A. Al-Gallaf ABSTRACT This eseach fame wok investigates the application of a clusteed based Neuo-fuzzy system to nonlinea dynamic system modeling fom a set of input-output taining pattens. It is concentated on the modeling via Takagi-Sugeno (T-S) modeling technique and the employment of fuzzy clusteing to geneate suitable initial membeship functions. Hence, such ceated initial membeships ae then employed to constuct suitable T-S sub-models. Futhemoe, the T-S fuzzy models have been validated and checked though the use of some standad model validation techniques (like the coelation functions). Compaed to othe well-known appoximation techniques such as atificial neual netwoks, fuzzy systems povide a moe tanspaent epesentation of the system unde study, which is mainly due to the possible linguistic intepetation in the fom of ules. Such intelligent modeling scheme is vey useful once making complicated systems linguistically tanspaent in tems of fuzzy if-then ules. The developed T-S Fuzzy modeling system has been then applied to model a nonlinea antenna dynamic system with two coupled inputs and outputs. Validation esults have esulted in a vey close antenna sub-models of the oiginal nonlinea antenna system. The suggested technique is vey useful fo development tanspaent linea contol systems even fo highly nonlinea dynamic systems. KeyWods: Neuo-fuzzy systems, fuzzy clusteing, Takagi-Sugeno modeling, nonlinea systems.. T-S Modeling I. INTRODUCTION Developing mathematical models of eal systems is a cental topic in many disciplines of engineeing and science. Models can be used fo simulations, analysis of the system s behavio, bette undestanding of the undelying mechanisms in the system, design of new pocesses, o design of contolles. Takagi-Sugeno T-S modeling plays Manuscipt eceived July, 3; evised Febuay, 4; accepted June 7, 4. The autho is with Dept. of Electical and Electonics Engineeing, College of Engineeing, Univesity of Bahain, P.O. Box 384, Kingdom of Bahain (e-mail: ebgallaf@eng. uob.bh). an essential ole in deiving local linea models of the nonlinea dynamic system unde concen. Though the use of the heuistic ules inheent in the fuzzy systems, T-S fuzzy models, then makes it possible to have a tanspaent like system which is govened by the fuzzy infeence system and ules. T-S fuzzy models have indeed eceived a focused attention in tems of thei utilization in most advanced contol paadigms. In a simila way fuzzy clusteing has been utilized as well in classifying the data-diven fuzzy modeling, since it daws a methodology fo assigning label like to simila data. Such assignment does gives quantitative diections fo shaping the fuzzy membeship functions. T-S fuzzy models can then be utilized to design advanced fuzzy contolles such as H obust contolles fo nonlinea systems. Such fuzzy H obust contolles can then be selected

64 Asian Jounal of Contol, Vol. 7, No., June 5 based on the opeating condition of the system unde contol. Once T-S fuzzy linea models ae obtained, thei state space models ae computed, since most advanced contol methodologies depend on the state space system model. Model validation and veification is also an impotant task in the modeling paadigm. This is due to the choice of the ight model fom a numbe of models that might pesent simila chaacteistics. Statistically validated models in addition to pobabilistic validation ae also used sometimes to make the suitable choice of a system model. Fuzzy modeling concens the methods of descibing the chaacteistics of a system using fuzzy infeence ules. Fuzzy modeling methods have a distinguishing featue in that they can expess complex nonlinea systems linguistically. Takagi-Sugeno (T-S) modeling plays an essential ole in deiving local linea models of the nonlinea dynamic system unde concen [,]. Though the use of the heuistic ules inheent in the fuzzy systems, T-S fuzzy models, then makes it possible to have a tanspaent like system which is govened by the fuzzy infeence system and ules. T-S fuzzy models have indeed eceived a focused attention in tems of thei utilization in most advanced contol paadigms. In a simila way, fuzzy clusteing has been utilized as well in classifying data-diven fuzzy modeling, since it daws a methodology fo assigning label like to simila data. Such assignment does gives quantitative diections fo shaping the fuzzy membeship functions. Model validation and veification is also an impotant task in the modeling paadigm. This is due to the choice of the ight model fom a numbe of models that might pesent simila chaacteistics. Statistically validated models in addition to pobabilistic validation ae used sometimes to make the suitable choice of a system model. In geneal, fuzzy contol systems can be classified as linguistic Takagi-Sugeno T-S type [] and Mamdani type [3]. The linguistic type fuzzy contol system is well ecognized and eceived by the contol society. The T-S type fuzzy system, which will be used in this aticle mainly focuses on the modeling aspect. It has been epoted that a T-S fuzzy system can exactly model any nonlinea system [4]. On the othe hand thee is a main dawback of the linguistic model compaed with the T-S model in that thee is a difficulty in dealing with a multidimensional system since a lage numbe of fuzzy ules have to be used. Gozalczany et al. [5] has biefly pesented and compaed fou neuo-fuzzy systems used fo ule- based modeling of dynamic pocesses (chaotic Mackey- Glass time seies). The following systems have been consideed: NFMOD the poposed system, the well-known Anfis and Nfident systems, and an altenative neuo-fuzzy system aleady epoted in liteatue. The main citeion of compaison of all systems is thei pefomance (modeling accuacy) vesus intepetability (the tanspaency and the ability to explain geneated decisions; it also includes an analysis and puning of obtained fuzzy-ule bases). On the othe hand, Zhang and Knoll [6] has poposed an appoach fo solving multivaiate modeling poblems with neuofuzzy systems. Instead of using selected input vaiables, statistical indices ae extacted to feed a fuzzy contolle. The oiginal input space was tansfomed into an eigen-space. If a sequence of taining data ae sampled in a local context, a small numbe of eigenvectos which possess lage eigenvalues povide a good summay of all the oiginal vaiables. Fuzzy contolles can be tained fo mapping the input pojection in the eigen-space to the outputs. Implementations with the pediction of time seies was used to validate the concept. The aticle of Ikonen et al. [7] concened a pocess modeling using fuzzy neual netwoks. In Distibuted Logic Pocessos (DLP) the ule base is paameteized. The DLP deivatives equied by gadient-based taining methods ae given, and the ecusive pediction eo method is used to adjust the used model paametes. The powe of the appoach is illustated with a modeling example whee NO x emission data fom a full-scale fluidized-bed combustion distict heating plant ae used. The method pesented in thee pape was geneal, and can be applied to othe complex pocesses. Bologna [8], has pesented a new neuofuzzy model denoted as Fuzzy Discetized Intepetable Multi-Laye Pecepton (FDIMLP). Fuzzy ules wee extacted in polynomial time with espect to the size of the poblem and the size of the netwok. He applied thei model to thee classification poblems of the public domain. It tuned out that FDIMLP netwoks compaed favoably with espect to EFUNN and ANFIS neuo-fuzzy systems. Fo NING et al. in [9], a fuzzy satisfactoy clusteing algoithm was pesented in thei pape. It stated with two cluste centes and inceases new cente if necessay. A system data set was quickly divided into seveal satisfactoy fuzzy clustes by the algoithm. A Takagi-Sugeno (T-S) type fuzzy model wee then identified. In [] Chen and Linkens intoduced a thee-layeed RBF netwok to implement a fuzzy model. Diffeing fom existing clusteing-based methods, in thei appoach the stuctue identification of the fuzzy model, including input selecting and patition validating, was implemented on the basis of a class of sub-clustes ceated by a self- oganizing netwok instead of on aw data. The impotant input vaiables which independently and significantly influence the system output can be extacted by a fuzzy neual netwok. On the othe hand, the optimal numbe of fuzzy ules can be detemined sepaately via the fuzzy c-means clusteing algoithm with a modified fuzzy entopy measue as the citeion of cluste validation. Akkizidis and Robets in [] poposed an algoithmic methodology fo identifying and modeling non- linea contol stategies. The methodology pesented was based on choices of diffeent fuzzy clusteing algoithms, pojection of clustes and meging techniques. The best featues of well-known clusteing methods such as the Gustafson- Kessel and mountain method wee combined. The latte was used to detemine and define the numbe and the appoximate positions of the cluste pototypes, wheeas the fome was used to define the shapes of the clustes accoding to the data distibution. The pojection of the pototypes and vaiables of clustes was a ecognized appoach to extacting the infomation included in the data clustes

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 65 into fuzzy sets. Meging these fuzzy sets, based on poposed guidelines, can minimize the numbe of ules and make the identifying contol stategy moe tanspaent. Some impovements to the esulting fuzzy system can be achieved by using optimization methods such as the gadient method. The poposed methodology was based on making the ight choice of the ight tools and can be descibed as a univesal appoximation in tems of identifying and modeling non-linea contol stategies.. Aticle contibution Within this aticle we shall pesent a novel eseach fame wok which has been conducted to accomplish an intelligent based modeling of a highly nonlinea antenna system via employment of clusteing and T-S modeling technique. Consequently, models wee validated. The system we ae investigating unde study is a typical of the type used fo occeanay satellite communication system and has a high nonlinea coupling among its two outputs. Hence, it is equied to have tanspaent sub-models. Fuzzy sets in the antecedent of the ules ae obtained fom the patition matix by pojection onto cetain antecedent vaiables. The obtained point-wise fuzzy sets ae then appoximated by some suitable paametic functions. The tanspaency of the antenna model obtained using the above appoach could be hindeed by the edundancy pesent in the fom of many ovelapping (well-matched) membeship functions. Cetain similaity measues wee used in ode to assess the compatibility (pai-wise similaity) of fuzzy sets in the ule base, in ode to detect sets that can be meged. Fuzzy sets estimated fom antenna taining data can also be simila to the univesal set, thus adding no infomation to the model. Sets of such natue wee emoved fom the antecedent of the ules, thus educing the numbe of the fuzzy ules..3 Aticle oganization The aticle has eight sections. Beginning with an oveview of intelligent modeling teminology in section. Section 3 intoduces neuo-fuzzy modeling, the stuctual and paametic tuning in typical neuo-fuzzy systems. The concept of modeling based on fuzzy clusteing is pesented in section 4 and the concept of extaction of a state space model fom T-S sub-models is explained in section 5. Section 6 discusses validation of the esulted fuzzy models. In section 7 T-S modeling is applied to a nonlinea antenna as a case study showing its stength and chaacteistics, and finally section 8 pesents few points fo conclusions. II. DYNAMIC SYSTEMS MODELING. NARX modeling of dynamic systems A common dawback of most standad modeling appoaches is that they cannot make effective use of exta infomation, such as the knowledge and expeience of enginees and opeatos, which is often impecise and qualitative in its natue. The fact that humans ae often able to manage complex tasks unde significant uncetainty has stimulated the seach fo altenative modeling and contol paadigms. In the case of (intelligent) modeling and contol methodologies, which employ techniques motivated by biological systems and human intelligence to develop models and contolles fo dynamic systems, have been intoduced. These techniques exploe altenative epesentation schemes using, fo instance, natual language, ules, semantic netwoks o qualitative models, and possess fomal methods to incopoate exta elevant infomation. Pecise numeical computation with conventional mathematical models only makes sense when the paametes and input data ae accuately known. As this is often not the case, a modeling famewok is needed which can adequately pocess not only the given data, howeve, also the associated uncetainty. Stochastic appoach is a taditional way of dealing with uncetainty, whee it has been ecognized that not all types of uncetainty can be dealt within the stochastic famewok.. Intelligent modeling A common dawback of most standad modeling appoaches is that they cannot make effective use of exta infomation, such as the knowledge and expeience of enginees and opeatos, which is often impecise and qualitative in its natue. The fact that humans ae often able to manage complex tasks unde significant uncetainty has stimulated the seach fo altenative modeling and contol paadigms. In the case of intelligent modeling and contol methodologies, which employ techniques motivated by biological systems and human intelligence to develop models and contolles fo dynamic systems, have been intoduced. These techniques exploe altenative epesentation schemes, using, fo instance, natual language, ules, semantic netwoks o qualitative models, and possess fomal methods to incopoate exta elevant infomation. Fuzzy modeling and contol ae typical examples of techniques that make use of human knowledge and deductive pocesses. Vaious altenative appoaches have been poposed, Fuzzy Logic and Set Theoy being one of them. Atificial neual netwoks and fuzzy models belong to the most popula model stuctues used. Fom the input-output view, fuzzy systems ae flexible mathematical functions, which can appoximate othe functions o just data measuements with a desied accuacy. Compaed to well-known appoximation techniques such as Neual Netwoks, fuzzy systems povide a moe tanspaent epesentation of the system unde study, which is mainly due to the possible linguistic intepetation in the fom of ules. The logical stuctue of the ules facilitates the undestanding and analysis of the model in a semi-qualitative manne, close to the way human eason about the eal wold.

66 Asian Jounal of Contol, Vol. 7, No., June 5 Given the state of a system with a given input, the next state x(k + ) can be detemined. In the sense of discetetime setting, it can be witten as in Eq. (): xk ( + ) = f( xk ( ), uk ( )) () whee x(k) and u(k) ae the state and the input at time k, espectively, and f is a static function. Fuzzy models of diffeent types can be used to appoximate the state-tansition function. As the state of a system is often not measued, input-output modeling is usually applied. The most common is the NARX (Nonlinea Auto-Regessive with Exogenous input) model, as defined by Eq. (): y(k + ) = f (y(k), y(k ),, y(k n y + ), u(k), u(k ),, u(k n u +)) () whee y(k),, y(k n y + ), and u(k),, u(k n y +) denote the past model outputs and inputs espectively and n y, n u ae integes elated to the model ode. Fo instance in Eq. (3), a linguistic fuzzy model of a dynamic system may consist of ules of the following fom: Ri : if y( k) is Ai and y( k ) is Ai and,, yk ( n+ ) is Ain and uk ( ) is Bi and uk ( ) is Bi and,, uk ( m+ ) is Bim then yk ( + ) is Ci (3) In Eq. (3), the input dynamic filte is a simple geneato of the lagged inputs and outputs, and no output filte is used. Since the fuzzy models can appoximate any smooth function to any degee of accuacy, models of the type in Eq. (3) can appoximate any obsevable and contollable modes of a lage class of discete-time nonlinea systems..3 Building fuzzy models Two common souces of infomation fo building fuzzy models ae the pio knowledge and data (pocess measuements). Pio knowledge can be of a athe appoximate natue (qualitative knowledge, heuistics), which usually oiginates fom expets, i.e., system opeatos. Building fuzzy models fom data involves methods based on fuzzy logic and appoximate easoning, in addition to ideas oiginating fom the field of neual netwoks, data analysis and conventional systems identification. The design of fuzzy models equies two basic items; the stuctue and the paametes of the model. Stuctue detemines the flexibility of the model in appoximation unknown mappings, wheeas the paametes ae then tuned (estimated) to fit the data at hand. In fuzzy models, stuctue selection involves the following choices: Input and output vaiables. Stuctue of the ules. Numbe and type of membeship functions fo each vaiable. Type of the infeence mechanism, connective opeatos, defuzzification method. These choices ae esticted by the type of fuzzy model (Mamdani, T-S). Within these estictions, howeve, some feedom emains, e.g., as to the choice of the conjunction opeatos. To facilitate data-diven optimization of fuzzy models (leaning), diffeentiable opeatos (poduct, sum) ae often pefeed to the standad min and max opeatos. Once the stuctue is fixed, the pefomance of a fuzzy model can be fine-tuned by adjusting its paametes. Tunable paametes of linguistic models ae the paametes of antecedent and consequent membeship functions (detemine thei shape and position) and the ules (detemine the mapping between the antecedent and consequent fuzzy egions)..4 Knowledge-based models To design a (linguistic) fuzzy model based on available expet knowledge, the following steps ae needed: Select input and output vaiables, stuctue of the ules and infeence, and defuzzification methods. Decide on the numbe of linguistic tems fo each vaiable and define the coesponding membeship functions. Fomulate the available knowledge in tems of fuzzy if-then ules, hence validating the designed model. It is assumed that a set of N input-output data pais {(x i, y i ) i =,,, N} is available. Recalling that x i R F inp ae input vectos and y i ae output scalas. Denote X R N F inp a matix having the vectos x in its ows, and y R N a vecto containing the outputs y k : x x x N = [ ] T (4) y y y N = [ ] T (5) III. NEURO-FUZZY MODEL SYSTEM In ode to optimize the paametes, which ae elated to the output in a nonlinea manne, taining algoithms known fom the aea of neual netwoks can be employed. At the computational level, a fuzzy model can be seen as a layeed stuctue (netwok), simila to atificial neual netwoks. Hence, this appoach is usually efeed to as neuofuzzy modeling. Figue shows a typically five layes of a neuo-fuzzy system that can be employed to accomplish a ule netwok. Typically, such ules ae: if x is A and x is A then y = b (6) if x is A and x is A then y = b (7) Nodes in the fist laye compute the membeship degee of the inputs in the antecedent fuzzy sets. The poduct nodes T k

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 67 Laye No. 5 Laye No. 4 Laye No. 3 Laye No. Laye No. Fig.. A five laye neuofuzzy netwok achitectue. Π in the second laye epesent the antecedent conjunction opeato. The nomalization node N and the summation node Σ ealizes the fuzzy-mean opeato. Using smooth antecedent membeship functions, such as a Gaussian function, as given below in Eq. (8): xj cij μaij( xj, cij, τ ij ) = exp, (8) τij in which c ij and τ ij paametes ae adjusted by gadientdescent leaning algoithms, such as back-popagation. This allows fo a fine-tuning of the fuzzy model to the available data in ode to optimize its pediction accuacy. 3. Stuctual and paametic leaning In a neuo-fuzzy system, two types of tuning ae equied, namely stuctual and paametic tuning. Stuctual tuning aims to find a suitable numbe of ules and a pope patition of the input space. Once available a satisfactoy stuctue, the paametic tuning seaches fo the optimal membeship functions togethe with the optimal paametes of the consequent models. Thee may be a lot of stuctue/paamete combinations which make the fuzzy model behave in a satisfactoy way. The poblem can be fomulated as that of finding the stuctue complexity which will give the best pefomance in genealization. In ou appoach we choose the numbe of ules as the measue of complexity to be popely tuned on the basis of available data. We adopt an incemental appoach whee diffeent achitectues having diffeent complexity (i.e. numbe of ules) ae fist assessed in coss-validation and then compaed in ode to select the best one. The initialization of the achitectue is povided by a hype-ellipsoid fuzzy clusteing pocedue inspied by Babuska and Vebuggen []. This pocedue clusteing the data in the input-output domain obtaining a set of hype-ellipsoids which ae a peliminay ough epesentation of the input/output mapping. Methods fo initializing the paametes of a fuzzy infeence system fom the outcome of the fuzzy clusteing pocedue. Hee we utilize the axes of the ellipsoids (eigenvectos of the scatte matix) to initialize paametes of the consequent functions, we poject the cluste on the input domain to initialize the centes of the antecedents and we adopt the scatte matix to compute the width of the membeship functions. Once the initialization is done, the leaning pocedue begins. Two optimization loops ae nested: the paametic and the stuctual one. The paametic loop (the inne one) seaches fo the best set of paametes by minimizing a sum-of- squaes cost function which depends exclusively on the taining set. The stuctual identification loop (the oute one) seaches fo the best stuctue, in tems of optimal numbe of ules, by inceasing gadually the numbe of local models.

68 Asian Jounal of Contol, Vol. 7, No., June 5 IV. FUZZY CLUSTERING TECHNIQUE 4. Fuzzy clusteing Identification methods based on fuzzy clusteing oiginate fom data analysis and patten ecognition, whee the concept of gaded membeship is employed to epesent the degee to which a given object, epesented as a vecto of featues, is simila to some pototypical object. Based on that similaity, featue vectos can be clusteed such that vectos within a cluste ae as simila as possible, and vectos fom diffeent clustes ae as dissimila as possible. This thought of fuzzy clusteing is depicted in Fig.. Data is clusteed into two goups with pototypes v and v, using the Euclidean distance measue. The patitioning of the data is expessed in the fuzzy patition matix whose elements μ ij ae degees of membeship of each data points (x i, y i ) in a fuzzy cluste with pototypes v j. Fuzzy if-then ules can be extacted by pojecting the clustes onto the axes. Figue shows a data set with two appaent clustes and two associated fuzzy ules. The concept of similaity of data to a given pototype leaves enough space fo the choice of an appopiate distance measue and of the chaacte of the pototype itself. Fo example, pototypes can be defined as linea subspaces, o the clustes can be ellipsoids with adaptively detemined shape []. Fom these clustes, the antecedent membeship functions and the consequent paametes of the T-S model can be extacted as follows []: if x is A then y = ax + b if x is A then y = ax + b (9) Each obtained cluste is epesented by one ule in the T-S model. Membeship functions fo fuzzy sets A and A ae geneated by point-wise pojection of the patition matix onto the antecedent vaiables. Such point-wise defined fuzzy sets ae then appoximated by a suitable paametic function. The consequent paametes fo each ule ae obtained as least squaes estimates. 4. Fuzzy clusteing algoithm Conside a finite set of elements X = {x, x,, x n } as being elements of the F inp dimensional Euclidean space R F inp, that is, x j R F inp, j =,,, n. The issue is to pefom a patition of such collection of elements into C fuzzy sets with espect to a given citeion. Heeby C is a given numbe of clustes. The citeion is usually to optimize an objective function that acts as a pefomance index of clusteing. The end esult of fuzzy clusteing can be expessed by a patition matix U such that: U = [ u ] i =,..., C j =,..., n () ij i In Eq. (), u ij is a numeical value in [,] and expesses the degee to which an element x j belongs to the i th cluste. Howeve, thee ae two additional constaints on value of u ij. Fist, a total membeship of the element x j X in all classes is equal to unity, that is: c uij = fo all j =,,, n () i= Second, evey constucted cluste is a nonempty and diffeent fom the entie set; that is, y Cuves of equidistance B B v v Clustes cente Local linea model μ(y) Data μ(x) Pojected clustes x If x is A then y is B If x is A then y is B A A data Fig.. Hype ellipsoidal fuzzy clustes. x

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 69 n < uij < n fo all i =,,, C () j= The geneal fom of the objective function used in fuzzy clusteing is: c n c Ju (, v) = gwx [ ( ), u] dx,v ( ) (3) ij k i ij j k i= j= k= whee w(x i ) is a pio weight fo each x i and d(x j, v k ) is the degee of dissimilaity between the data x i and the supplemental element v k, which can be consideed the cental vecto of the k th cluste. Degee of dissimilaity is defined as a measue that satisfies two assumptions given by: d( x,v ), (4) j k d( x,v ) = d( v x ) (5) j k k j Based on the above backgound, fuzzy clusteing can be pecisely fomulated as an optimization poblem: Minimize Ju,v ( ) c n c gwx [ ( ),u] dx,v ( ), = ij k i ij j k i= j= k= ik, =,,..., c; j=,,..., n (6) Subject to c uij = fo all j =,,, n and i= n < uij < n fo all i =,,, C j= One of the widely employed clusteing methods based on Eq. (6) is the Fuzzy C-Means (FCM) algoithm. The objective function of the FCM algoithm is expessed in the fom of: c n m ij k = ij j i > (7) i= j= Ju,v ( ) u x v, m whee m is called exponential weight that influences the degee of fuzziness of the membeship (patition) matix. To solve this minimization poblem, the objective function J(u ij, v k ) is diffeentiated in Eq. (7) with espect to v k (fo fixed u ij, i =,, c, j =,, n) and to u ij (fo fixed v k, i =,, C) and apply the conditions of Eq. (), obtaining: n m vi = j ( ij) j,,..., n m = u x, i = c (8) ( u ) u j= ij / m (/ xj vi ) ij = c / m k= (/ xj vk ), i =,,..., C; j =,,..., n (9) The system descibed by the Eqs. (8) and (9) cannot be solved analytically. Howeve, the FCM algoithm povides an iteative appoach to appoximating the minimum of the objective function stating fom a given position. V. LINEAR STATE SPACE MODELS EXTRACTION 5. T-S fuzzy space model At each sample time k, given an opeating point condition (fo example u(k ) and y(k )), a local linea fuzzy state-space model can be constucted via calculating the degee of fulfillment μ i (x(k)) of the antecedents, using poduct as the fuzzy logic AND opeato. The infeence of the entie stuctue (hieachy) due to ule i esults on a sub-model () which can be expessed as: y ( k + ) = l i= μ ( x ( k)) y ( k+ ) ( ( )) li l li i= μli xl k () y ( k + ) = ( ζ y( k) +η u( k) +θ ) () li li li li Defining ζ l, η l and θ l as follows: ζ = l η = l θ = l i= μli ( xl ( k)) ζli i= μli ( xl ( k)) i= μli ( xl ( k)) ηli i= μli ( xl ( k)) i= μli ( xl ( k)) θli i= μli ( xl ( k)) () (3) (4) Defining x(k), u(k) and y(k) fo the state-space desciption as: x( k)... x( k ny)... xn ( k) x ( k) =... xn ( k n ) yn u ( k )... un ( k n ) i d nu T (5) u ( k) = [ u ( k) u ( k)... u ( k)] T (6) ni y ( k) = [ x ( k) x ( k)... x ( k)] T (7) ni In ode to employ Quadatic Pogamming fo systems which depend on cuent as well as on the pevious inputs, it is necessay to constuct a state-space epesentation, such that the state vecto x(k) to accommodate not only the state vaiables, appeaing in y(k), but also the pevious inputs and the offset as last element. This esults in a system with only cuent inputs, but leads to a moe complex A-matix. The latte contains also n, coesponding to the pevious inputs. If the maximal delay in the input i, i =,, n i is u i, d max, then the numbe of the additional columns is n i i= max ( u id, max,). In the last column of A ae stoed the offsets θ. The columns with n coesponds to the pevious inputs, stoed in the state vecto; these columns ae

7 Asian Jounal of Contol, Vol. 7, No., June 5 not included in B. The ones in A coespond to the delayed values of a cetain vaiable. The local linea system matices ae deived as follows: A is α α squae matix, whee n i i= u id, max α=α + max (, ) +, n i j= n yj α = matix: max ( ), B is an α n i and C is a n α ζ ζ ζ ζ η η θ ζ ζ ζ η η θ A = ζ ζ ζ η η θ,,,3, α, i, j,,,, α, i n, j, n, n, n, α n, i n, j n, α η, η, η, ni B = η, η, η, ni η n, ηn, ηn, ni C = (8) (9) (3) The ones in C ae positioned such that y l (k) = x l (k). At any time index k, initially the contol signal u(k ) is used. Howeve, afte the optimization, u(k) is available and could be used in next iteations. VI. T-S FUZZY MODEL VALIDATION 6. Fuzzy models validations (coelation tests) The impotance of a model stuctue is clea and in this section, methods which can be used to assess the pefomance of diffeent models ae intoduced. Pactical expeience has shown that model selection citeia, descibed above, can potentially select incoect models, hence futhe validation is equied. The simplest fom of model validation is by expet inspection. If the model is simple and/o can be descibed qualitatively, an expet can veify the model by inspection o inteogation. Howeve, taditionally moe igoous statistical validation tests ae employed in which models esiduals ae examined and if found to be sufficiently coelated with a function of the data then the model is inadequate. This is achieved by defining a matix Z containing the time lag of the system taining inputs signals u(t), outputs y(t) in which Z(x t ) is: t Z( x) = [ mt ( ), mt ( ),..., mt ( t d )] (3) in which x t is the obsevational vecto of inputs, outputs and eos seen up to time step t and m(t) epesents the degee of dependency of the two taining signals y(t), u(t) and the e(t) is the eo between the fuzzy model and actual system output i.e.: t t t t T x = [ u, y, e ] (3) and m(t ) is a monomial of the vecto x t given by: mt () = yt ( ) u( t ) (33) The following two hypothesis ae defined: H : e(t) is uncoelated with Z(x t ), E (e(t) Z( )) = H : e(t) is coelated with Z(x t ), E (e(t) Z( )) T and whee the pupose of validation is to use the data to decide if H holds. Two diffeent test statistics have been poposed in the liteatue, the most common being the standad sample coelation measue ρ(k), is defined as, [3]: ρ ( k) = N K+ t = ( mt ( k+ ) et ( )) N ( mt ( k ) mt ( k )) ( et ( ) et ( )) (34) N K+ N K+ t= + + t= in which k =,, t d, and ρ(k) [, +]. If H holds this statistic asymptotically appoaches a nomal distibution, and within 95% confidence limits H is accepted if ρ( k) [.96 / N,.96 / N ]. An altenative statistic is given by: N td N td t= t= N td N td t= Z tzt t= d = N[ ( e ( t))] [ ( Z( t) e( t))] [ ( () ())] [ ( Ztet () ())] (35) which is H hold is asymptotically a X (s) distibution whee s is the numbe of delays t d. Fo a given acceptance level (typically 95%), a citical point is found, and if elements of d ae outside this acceptance egion H is ejected. VII. T-S MODELING OF A NONLINEAR SYSTEM 7. Antenna system (input/output taining patten) To test these poposed neuo-fuzzy methodologies

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 7 futhe, they ae applied to model a ealistic nonlinea dynamical system. The system consideed is a nonlinea (MIMO) dynamics of an antenna system with two coupled inputs and outputs. A data set containing 5 samples of taining pattens wee poduced by applying andom toques to the diffeent channels, with suitable sampling ate and an amplitude dawn fom unifom andom distibution in the ange (.5, +.5) N/m. Antenna system A coupled two degee of feedom satellite dish, typical of the type used fo oceanay satellite communication systems, is pesented. The behavio of the antenna is descibed by the following nonlinea idealized time invaiant state space equations, [3]: and z = [ ϕ ϕ ψ ψ ] T (36) ϕ Tϕ bϕϕ ϕψ ( I I)sin( ψ) I sin ( ψ ) + Icos ( ψ) z = ψ Tψ bψψ ( I I) ψ sin( ψ) I y eϕ () t = z + eψ () t (37) whee ϕ is the azimuth angle, ψ is the elevation angle, b ϕ and b ψ ae the associated fiction coefficients, T ϕ and T ψ ae the toques applied to the axes. The antenna nominal values ae given in Table. To poduce a moe ealistic simulation, the outputs ae coupted by additive Gaussian noise, [e ϕ (t) e ψ (t) T ], epesenting a cude appoximation to measuement noise. Table. Defined antenna paametes (non-isotopic). b ϕ b ψ I I Isotopic.3375.3375. Nms. Nms Non-isotopic.3375.3375. Nms. Nms The azimuth is pemitted to tun though a complete evolution, while end stops estict the elevation to the inteval [, π]. In this antenna thee ae essentially two souces of nonlineaity; that poduced fom the end stops on the elevation and the othe as a esults of the non-isotopic moment of inetia tenso i.e. I I. Indeed when isotopy is pesent the state space equations (above) ae linea. The stength of this non-lineaity depends on the degee of anti-isotopy and the angula velocities of the antenna. These toques ae chosen to emulate typical opeating conditions. Such block diagam used to poduce the identification data was simulated though Simulink/Matlab, using a set of nonlinea diffeential equations that descibes the antenna system. Half of the taining patten was used in the modeling of the dynamic system wheeas the othe half was used to validate the fuzzy models esulted fom the modeling. Fo a typical sequence of taining data, such esponses of the antenna inputs-outputs is shown in Fig. 3. ) Input m azimuth (N / toque e (Nm) T oqu ) Input m elevation (N / toque e (Nm) T oqu d ) output (a azimuth u (ad) m A zi d ) output (a elevation n o (ad) e va t i E l - 3 4 5 6-5 3 4 5 6-5 5 3 4 5 6-5 3 4 5 6 Time (s) Fig. 3. Input-output data taining patten.

7 Asian Jounal of Contol, Vol. 7, No., June 5 7. Clusteing and taining patten As discussed in section 3, fuzzy modeling of any dynamical system could be achieved though clusteing the taining data. Fo this simulation example, clusteing has been applied to the antenna taining patten. In Fig. 4 it is shown the taining patten following applying the clusteing algoithm, whee it illustates clealy the clustes and thei thee associated centes. Fo instant, Fig. 4(a) shows the taining patten which has been clusteed into thee clustes, wheeas Fig. 4(b) shows the clusteed antenna azimuth output fo seven clustes. Howeve, to educe the fuzzy ules while peseving the model accuacy, the numbe of clustes wee chosen to be thee clustes. In Eq. (8), to adjust the fuzzy clusteing, the fuzziness paamete (m) was kept at. with a temination citeion fo choosing cluste centes of.. The esult of the clusteing algoithm is the fuzzy patition matix and the cluste centes matix, which will be used to constuct the antenna system fuzzy models. ad 4 3 - - cente cente 3 cente -3 -.5 - -.5.5.5 Toque (N/m) Fig. 4(a). Extacted clustes functions associated azimuth angle. 7.3 Neuofuzzy modeling Neuofuzzy modeling is applied to the poblem of identifying a discete model of the antenna. A fuzzy model can be constucted fom data by using the output of the clusteing algoithm and by constucting egessos to fom inputs to the neuo-fuzzy netwok. Hence a conventional linea diffeence model with egessos is constucted containing pevious inputs and outputs, i.e.: ϕ ( k) = [ Tϕ( k ), Tϕ( k ), Tψ( k ), ϕ( k ), ϕ( k ), ψ( k ), ψ( k ), Δϕ( k )] T Azimuth output (ad) Clusteed Centes ψ ( k) = [ Tϕ( k ), Tψ( k ), Tψ( k ), ϕ( k ), ϕ( k ), ψ( k ), ψ( k ), Δψ( k )] T Fuzzy if-then ules can be extacted by pojecting the clustes onto the axes and the membeship functions of the fuzzy sets geneated by point-wise pojection of the patition matix onto the antecedent vaiables. Then consequent paametes fo each ule ae obtained as least squaes estimates. When an initial stuctue is obtained though clusteing, then the membeship functions and the consequent paametes ae tuned to satisfy cetain cost function though the leaning pocedue of the neuo-fuzzy. 7.4 Membeship functions and associated fuzzy ules As a esult, the membeship functions of all the inputs (egessos) and outputs ae shown in Fig. 5 fo azimuth angle. The antenna system has six inputs (in tems of fuzzy model inputs) and two outputs, hence two goups of seven sets of MFs ae shown. Each univese of discouse (set) has thee MFs epesenting the assigned thee clustes. Such membeships ae epesenting the ange of the input limits. Time (sec.) Fig. 4(b). A seven clusteed antenna azimuth output. 7.5 Fuzzy sub-models The pesented T-S fuzzy model has been used to identify the nonlinea antenna system. As was mentioned befoe that the numbe of ules in the T-S fuzzy model equals the numbe of clustes in the poduct space. The consequent of each ule is a local model that appoximate the output of the eal function fo the ange of x fo which the ule is applicable. As a esult of the modeling development, the following ules ae obtained fo azimuth and elevation angles: Rule. If Tϕ ( k ) is F and Tϕ ( k ) is F and Tψ ( k ) is F 3 and ϕ( k ) is F 4 and ϕ( k ) is F 5 and ψ( k ) is F 6 and ψ( k ) is F 7 then x () t = Ax() t + Bu() t and y() t = Cx () t.

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 73.5-3 - -.5 - -.5-5 5.5-3 - -.5 - -.5-4 - 4.5-4 - 4 Fig. 5. Extacted membeship functions of all inputs egessos associated with the elevation angle. Rule. If Tϕ ( k ) is F and Tϕ ( k ) is F and Tψ ( k ) is F 3 and ϕ( k ) is F 4 and ϕ( k ) is F 5 and ψ( k ) is F 6 and ψ( k ) is F 7 then x () t = Ax() t + Bu() t and y() t = Cx () t. Rule 3. If Tϕ ( k ) is F 3 and Tϕ ( k ) is F 3 and whee, Tψ ( k ) is F 3 3 and ϕ( k ) is F 3 4 and ϕ( k ) is F 3 5 and ψ( k ) is F 3 6 and 3 ψ( k ) is F 7 then x () t = A3x() t + B3u() t and y() t = C3x () t..95.465.3.998.6.348.73.8.335.3 A = A.9354.4667.55.475.9.373.384.844.375. =.9.499.4.57.8.94.98.7887.987. A 3 =.8.4 =, B B.8.8 B3 =.8.37 =, C = C = C 3 = D = D = D 3 = whee the C and D matix ae common fo all of the thee fuzzy sub-models, and the D matix is equal to zeo. Futhemoe, the elevation angle dynamics is of the same above stuctue. The antenna simulation system incopoating the thee models is shown in Fig. 6. Consequently, Fig. 6 shows the actual antenna output supeimposed ove the evaluated fuzzy model output. Fom the figue, it is appaent how the fuzzy model output esembles the actual system output. In this espect, the model output is able to move within π ange of the antenna displacement. 7.6 Fuzzy model validation Futhemoe, coelation tests was employed to check that suitable epessos ae uncoelated with the models esiduals. Figue 7 displays the coss-coelation function of the eo signal with the fist input signal of the antenna. The coelation in the figue is within the confidence inteval, which indicates that the two signals ae not coelated. To futhe investigate the constucted local linea sub-models of the antenna, Fig. 8 shows the attained lineaized sub-models

74 Asian Jounal of Contol, Vol. 7, No., June 5 6 4 azimuth angle (ad) - -4 3 4 5 6 Time (sec.) 4 elevation angle (ad) - -4 3 4 5 6 Time (sec.) Fig. 6. Fuzzy model esponses (azimuth and elevation angels) compaed to the antenna outputs. Coss Coelation of toque with eo - - -3 4-5 -4-3 - - 3 4 5 lag coss coelation of toque with its associated eo Coss Coelation of toque with eo 3 - -5-4 -3 - - 3 4 5 lag Fig. 7. Coss coelation of system fist input with its associate eo of the antenna system.

E.A. Al-Gallaf: Clusteed Based Takagi-Sugeno Neuo-Fuzzy Modeling of a Multivaiable Nonlinea Dynamic System 75 7 6 Azimuth Output (ad) 5 4 3 - - Linea sub-local model -3 5 5 5 3 35 4 45 5 Time (sec.) Fig. 8. Obtained lineaized antenna sub-models. ove the antenna time esponse. In tems of antenna nonlinea behavio, it is obvious that the entie opeating egion has been sub-divided into a numbe of local models which could be employed fo futhe contol synthesis. Fom the shown antenna esponse, fuzzy models ae useful fo descibing the antenna dynamics whee the undelying physical mechanisms ae not completely known and whee the antenna behavio is undestood in qualitative tems. Consequently, an impotant popety of fuzzy models is thei capability to epesent nonlinea dynamic systems. Theefoe, the obtained fuzzy sub-models can also be applied to systems that ae well undestood but due to the nonlineaities untaceable with standad linea methods. Rule-based stuctue of fuzzy models allow fo integating heuistic knowledge with infomation obtained fom antenna measuements. The global opeation of the antenna nonlinea pocess is divided into seveal local opeating egions. Within each egion R i, a educed ode linea model in ARMAX fom is used to epesent the local antenna behavio. That was not estictive, and any appopiate model foms can also be used. VIII. CONCLUSION Recently, the inteest in data-diven appoaches to the modeling of nonlinea pocesses and uncetain dynamic systems has inceased. Pefomances based on fuzzy sets and ule-based systems have poven suitable mainly because of thei potential to yield tanspaent models that ae at the same time easonably accuate. In this sense, this aticle has been concentated on the modeling of nonlinea antenna dynamic systems via the utilization of the well known fuzzy modeling paadigm, the Takagi-Sugeno (T-S) technique. T-S models depend heavily on some initial membeship centes of the univese of discouse of used fuzzy vaiables, such centes have been obtained though employing of clusteing algoithm. Once such centes ae computed and ae known, a fuzzy system can establish initial membeship centes though which they ae updated via a neual netwok leaning mechanism. One of the advantages of T-S modeling, is that, systems can be modeled by fewe numbe of ules, and consequently fewe linea sub-models. This advantage has ovecome the difficulty of the lage numbe of ules in the fuzzy modeling paadigm. Fuzzy models have also been veified and validated though some standad validation techniques, whee they have shown clealy the successful ability of T-S techniques to model nonlinea systems with an excellent degee of accuacy. REFERENCES. Lo, J. and Y. Chen, Stability Issues on Takagi-Sugeno Fuzzy Model, Paametic Appoach, IEEE Tans. Fuzzy Syst., Vol. 7, No. 5, pp. 597-67 (999).. Takagi, T. and M. Sugeno, Fuzzy Identification of Systems and Its Application to Modeling and Contol, IEEE Tans. Syst. Man Cyben., Vol. 5, No., pp. 6-3 (985). 3. Mamdani, E. and S. Assilian, An Expeiment in Linguistic Synthesis with a Fuzzy Logic Contolle, Int. J. Man-Mach. Stud., Vol. 3, No. 5, pp. -3 (975). 4. Wang, H., J. Li, D. Niemann, and K. Tanaka, T-S Fuzzy Model with Linea Rule Consequence and PDC Contolle: A Univesal Famewok fo Nonlinea

76 Asian Jounal of Contol, Vol. 7, No., June 5 Contol Systems, Poc. 9 th IEEE Int. Conf. Fuzzy Syst., U.S.A. (). 5. Gozalczany, E., B. Maian, and A. Gluszek, Neuo- Fuzzy Netwoks in Time Seies Modelling, Poc. Int. Conf. Knowl.-Based Intell. Elect. Syst., Vol., Bighton, U.K., pp. 45-453 (). 6. Zhang, J. and A. Knoll, Modelling Multivaiate Data by Neuo-Fuzzy Systems, Poc. 999 IEEE/IAE Conf. Comput. Intell. Financ. Eng. (CIFE), NY, U.S.A., pp. 67-7 (999). 7. Ikonen, E., K. Najim, and U. Kotela, Neuo-Fuzzy Modelling of Powe Plant Flue-Gas Emissions, J. Eng. Appl. Atif. Intell., Vol. 3, No. 6, pp. 75-77 (). 8. Bologna, G., FDIMLP: A New Neuo-Fuzzy Model, Poc. Int. Joint Conf. Neual Netwoks (IJCNN'), Vol., Washington-DC, U.S.A., pp. 38-333 (). 9. Li, Ning, Shaoyuan Li, and Yugeng Xi, Modeling ph Neutalization Pocesses Using Fuzzy Satisfactoy Clusteing, IEEE Int. Conf. Fuzzy Syst., Vol. l, No., Austalia, pp. 38-3 ().. Chen, M.Y. and A. Linkens, Fast Fuzzy Modeling Appoach Using Clusteing Neual Netwoks. IEEE Int. Conf. Fuzzy Syst., Vol. l, No., IEEE Wold Cong. Comput. Intell., U.S.A., pp. 88-93 (998).. Akkizidis, S. and N. Robets, Fuzzy Clusteing Methods fo Identifying and Modeling of Non-Linea Contol Stategies, Poc. Inst. Mech. Eng., Pat I: J. Syst. Cont. Eng., Vol. l, No. 5, pp. 437-45 ().. Babuska, R. and H. Vebuggen, Identification of Composite Linea Models via Fuzzy Clusteing, Poc. Euop. Cont. Conf., Italy, pp. 7- (995). 3. Bossley, K., Neuofuzzy Modeling Appoaches in System Identification, Ph.D. Dissetation, Univesity of Southampton, U.K. (997).