Neuro-fuzzy Systems K u h rshi h d i d A h A m

Transcription

1 Neuro-fuzzy Systems Khurshd Ahmad, Professor of Computer Scence, Department of Computer Scence Trnty College, Dubln-2, IRELAND 21 th November

2 Neuro-fuzzy models A fuzzy nference system can be shown to be functonally equvalent to a class of adaptve networks. The burden of specfyng the parameters of the fuzzy nference can be transferred to an algorthm that attempts to learn the value of the parameters Jang, Jyh-Shng Roger., Sun, Chuen-Tsa & Mzutan, Ej. (1997). Neuro-Fuzzy & Soft Computng: A Computatonal Approach to Learnng and Machne Intellgence. Upper Saddle Rver (NJ): Prentce Hall, Inc. (Chapters 8 and 12) 2

3 Neuro-fuzzy models For complex control systems, there s a wealth of observatonal a pror knowledge related to the behavour of nputs and output(s). The nput space may be parttoned between say normal behavour and abnormal behavour nducng stmul. The correspondng output can be noted Jang, Jyh-Shng Roger., Sun, Chuen-Tsa & Mzutan, Ej. (1997). Neuro-Fuzzy & Soft Computng: A Computatonal Approach to Learnng and Machne Intellgence. Upper Saddle Rver (NJ): Prentce Hall, Inc. (Chapters 8 and 12) 3

4 Neuro-fuzzy models Learn from the nput-output data: Data mnng; Machne Learnng; Neural Networks; } Genetc Algorthms Hybrds Neuro Fuzzy systems Soft Computng Jang, Jyh-Shng Roger., Sun, Chuen-Tsa & Mzutan, Ej. (1997). Neuro-Fuzzy & Soft Computng: A Computatonal Approach to Learnng and Machne Intellgence. Upper Saddle Rver (NJ): Prentce Hall, Inc. (Chapters 8 and 12) 4

5 Neuro-fuzzy models Learn from the nput-output data: If a soft computng system s able to compute the nput-output relatonshps, then t wll LEARN to compute the relatonshps Jang, Jyh-Shng Roger., Sun, Chuen-Tsa & Mzutan, Ej. (1997). Neuro-Fuzzy & Soft Computng: A Computatonal Approach to Learnng and Machne Intellgence. Upper Saddle Rver (NJ): Prentce Hall, Inc. (Chapters 8 and 12) 5

6 Neuro-fuzzy models Learn from the nput-output data: The key noton n learnng s that of THRESHOLD Threshold s an Old Englsh word meanng the pece of tmber or stone whch les below the bottom of a door, and has to be crossed n enterng a house; the sll of a doorway; hence, the entrance to a house or buldng. More techncally, n contexts of wages and taxaton, n whch wage or tax ncreases become due or oblgatory when some predetermned condtons are fulflled (esp. above a specfed pont on a graduated scale). [..] 6

7 Neuro-fuzzy models Learn from the nput-output data: The key noton n learnng s that of THRESHOLD Threshold n many specalst domans refers to a lower lmt. () Psychology: esp. n phrase threshold of conscousness. () In Physology and more wdely: (a) the lmt below whch a stmulus s not perceptble; (b) the magntude or ntensty that must be exceeded for a certan reacton or phenomenon to occur. 7

8 Neuro-fuzzy models Learn from the nput-output data: The key noton n learnng s that of THRESHOLD Threshold n many specalst domans refers to a lower lmt. () In Electroncs: (a) threshold devce, element, etc.: a crcut element havng one output and a number of nputs, each of whch accepts a bnary sgnal and multples t by some factor; the output s 0 or 1 dependng on whether or not the sum of the resultng quanttes s less than a certan threshold value; (b) threshold functon, a Boolean functon that can be realzed by such an element; threshold logc, swtchng (based on such elements). 8

9 Neuro-fuzzy models Learn from the nput-output data: The key noton n learnng s that of THRESHOLD Threshold n many specalst domans refers to a lower lmt. (v) In Fuzzy Logc and Fuzzy Knowledge Bases, rules are fred f the aggregaton of the antecedents membershp functons s non-zero. The threshold value here s any number greater than zero. 9

10 Neuro-fuzzy models Learn from the nput-output data: The key noton n learnng s that of THRESHOLD Threshold functons v w x + w = 1 2 y (1) Threshold Functon 1 φ(v) = 0 1 (2) Pecewse LnearFuncton φ(v) = v 0 1 (3) Sgmod Functon φ(v) = av 1+ exp f f v 0 v < 0 f f f v + v > v 1 2 >

11 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). There are two fuzzy rules: R1: IF x s A 1 and y s B 1 THEN f 1 =p 1 x+q 1 y+r 1 R2: IF x s A 2 and y s B 2 THEN f 2 =p 2 x+q 2 y+r 2 11

12 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). Layer 1 Layer 2 x A 1 TT w 1 N w 1 w 1 f 1 Layer 5 A 2 f y B 1 B 2 TT w 2 N w 2 w 2 f 2 Layer 3 Layer 4 12

13 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). 1.2 B Seres1 0.6 Seres y f 1 =p 1 x+q 1 y+r 1 w 1 f=(w 1 f 1 +w 2 f 2 )/(w 1 +w 2 ) 13

14 Neuro-fuzzy models: A case study The operaton of a fuzzy system depends on the executon of FOUR major tasks: Fuzzfcaton, Inference, Composton, (Defuzzfcaton). The dfferent layers n an adaptve network perform one or more of the tasks 14

15 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). Layer 1 Layer 2 x A 1 TT w 1 N w 1 w 1 f 1 Layer 5 A 2 f y B 1 B 2 TT w 2 N w 2 w 2 f 2 Layer 3 Layer 4 15

16 Neuro-fuzzy models: A case study One can argue that the frst layer, that receves nput from the external world, actually performs fuzzfcaton. Recall that fuzzfcaton nvolves the choce of varables, fuzzy nput and output varables and defuzzfed output varable(s), defnton of membershp functons for the nput varables and the descrpton of fuzzy rules. The membershp functons defned on the nput varables are appled to ther actual values to determne the degree of truth for each rule premse. The degree of truth for a rule's premse s sometmes referred to as ts a (alpha) value. If a rule's premse has a non-zero degree of truth, that s f the rule apples at all, then the rule s sad to fre. 16

17 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). LAYER 1: Every node n ths layer s an adaptve node wth a node functon O O 1, 1, = = µ µ A B ( x) 2 ( y) for for = 1,2 = 3,4 A x) = 2 1+ x c PREMISE PARAMETER SET:={a,b,c ) µ ( 1 a 17 b

18 Neuro-fuzzy models: A case study The operaton of the 2 nd and 3 rd layers n an adaptve network may be construed as equvalent to that of performng nference. In lectures on knowledge representaton we had defned nference as follows: The truth-value for the premse of each rule s computed and the concluson appled to each part of the rule. Ths results n one fuzzy subset assgned to each output varable for each rule. MIN and PRODUCT are two nference methods. In MIN nferencng the output membershp functon s clpped off at a heght correspondng to the computed degree of truth of a rule's premse. Ths corresponds to the tradtonal nterpretaton of the fuzzy logc's AND operaton. In PRODUCT nferencng the output membershp functon s scaled by the premse's computed degree of truth. 18

19 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). LAYER 2: Every node n ths layer s a fxed node ( ); the node outputs the product of all ncomng sgnals O2, = w = µ A ( x) * µ ( y) for = 1,2; j = B j 2 3,4 Each node n ths layer represents the frng strength of a rule; fuzzy AND operator can be used 19

20 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). LAYER 3: Every node n ths layer s a fxed node (N); the th node calculates the rato of the th rule s frng strength to the sum of all rules frng strengths O _ w = w = for = 3, w + w 1 2 1,2 Outputs of layer 3 are called NORMALIZED FIRING STRENGTHS 20

21 Neuro-fuzzy models: A case study The operaton of the (3 rd &) 4 th layer(s) nvolves composton. You may remember our defnton of composton: All the fuzzy subsets assgned to each output varable are combned together to form a sngle fuzzy subset for each output varable. MAX and SUM are two composton rules. In MAX composton, the combned fuzzy subset s constructed by takng the pontwse maxmum over all the fuzzy subsets assgned to the output varable by the nference rule. The SUM composton, the combned output fuzzy subset s constructed by takng the pontwse sum over all the fuzzy subsets assgned to output varable by ther nference rule. (Note that ths can result n truth values greater than 1). 21

22 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). LAYER 4: Every node n ths layer s an adaptve node wth a node functon: 4, w f w* ( p x q y r ) O = = + + The normalzed frng strengths w s a normalsed frng strength from layer 3; The parameter set, {p,q,r } s the so-called CONSEQUENT PARAMETERS SET 22

23 Neuro-fuzzy models: A case study And, fnally the output layer of an adaptve network performs the equvalent of defuzzfcaton. Defuzzfcaton was defned as process where the value from the composton stage needs to be converted to a sngle number or a crsp value. Two popular defuzzfcaton technques are the CENTROID and MAXIMUM technques. The use of CENTROID technque reles on usng the centre of gravty of the membershp functon to calculate the crsp value of the output varable. The MAXIMUM technques, and there are a number of them, broadly speakng, use one of the varable values at whch the fuzzy subset has ts maxmum truth value to compute the crsp value. 23

24 Neuro-fuzzy models: A case study Consder a frst-order Sugeno fuzzy model wth two nputs (x & y) and one output (z). LAYER 5: The sngle node n ths layer s a fxed node labelled, whch computes the overall output as the summaton of all ncomng sgnals O 5,1 = w f = _ w w f 24

25 Neuro-fuzzy models: A case study The network below s an adaptve network that s functonally equvalent to a Takag-Sugeno model. Layer 1 Layer 2 x A 1 TT w 1 N w 1 w 1 f 1 Layer 5 A 2 f y B 1 B 2 TT w 2 N w 2 w 2 f 2 Layer 3 Layer 4 25

26 Neuro-fuzzy models 26

27 Neuro-fuzzy models The goal of a number of statstcal nvestgatons s to predct the varaton of a dependent varable on one or more ndependent varables usng a mathematcal equaton. The dependence can be lnear or non-lnear. Consder the lnear dependence of a varable y on ndependent varable x, sometmes wth the provso that the ndependent varables can be observed wthout any (observatonal) error. The dependent varable y may have dfferent values for the SAME x. One can argue that y s essentally a random varable and ts dstrbuton depends on x; typcally, the quest s to fnd the relatonshp between the ndependent varable and the MEAN of 27 the dependent varable y the regresson curve of y on x.

28 Neuro-fuzzy models Assume that the dependence of y on x s lnear for any gven x the MEAN of the dstrbuton of y s gven as y ˆ = α + β x Statstcans remnd us that an observed y wll dffer from the mean ŷ by the value of a random varable, say, ε y = α + β x + ε 28

29 Neuro-fuzzy models Assume that the dependence of y on x s lnear for any gven x the MEAN of the dstrbuton of y s gven as y ˆ = α + β x Statstcans remnd us that an observed y wll dffer from the mean ŷ by the value of a random varable, say, ε y = α + β x + ε 29

30 Neuro-fuzzy models Assume that the dependence of y on x s lnear for any gven x the MEAN of the dstrbuton of y s gven as yˆ = α + β x Statstcans remnd us that an observed y wll dffer from the mean ŷ by the value of a random varable, say, ε the value may be related to the possble error of measurement and related to other varables that may have an nfluence on y y = α + β x + ε 30

31 Neuro-fuzzy models What we have to do now s to use an OBSERVED data set contanng the tuples {x,y } for a number of observatons, =1,N, for estmatng the values of α and β. Gven that we have assumed that the relaton between x and y s lnear, then we have to fnd a straght lne that may provde a ft. There could be many straght lnes that can be ftted to the data set and we have to chose the best one. We begn by predctng the value of y usng estmates of α and β, whch we wll refer to as a and b y ˆ = a + b x 31

32 Neuro-fuzzy models y ˆ = a + b x The error n predctng the value of y gven a correspondng value of x, wll be denoted as the error vector e : e = y yˆ 32

33 Neuro-fuzzy models e = y yˆ Typcally, nstead of computng e, a dffcult task, we tend to reduce the sum of errors assocated wth the N observatons to zero. Ths s rather unsutable, as one fnd totally unsutable lnes, one tends to mnmze the value of the sum of the squares of e 2 e e = [ y ( a + bx )] 2 33

34 Neuro-fuzzy models 2 2 )] ( [ bx a y e e + = Essentally, we equate the (partal) dervatves of the above equaton wth respect to a and b to zero and we get normal 34 equatons + = + = = + = + x b x a y x x b n a y to Leadng x bx a y bx a y 2 * 0 ) )]( ( [ 2 0 ) 1 )]( ( [ 2

35 Neuro Fuzzy Models Statstcans generally have good mathematcal backgrounds wth whch to analyse decson-makng algorthms theoretcally. [ ] However, they often pay lttle or no attenton to the applcablty of ther own theoretcal results (Raudys 2001:x). Neural network researchers advocate that one should not make assumptons concernng the multvarate denstes assumed for pattern classes. Rather, they argue that one should assume only the structure of decson makng rules and hence there s the emphass n the mnmzaton of classfcaton errors for nstance. Raudys, Šarûunas. (2001). Statstcal and Neural Classfers: An ntegrated approach to desgn. London: Sprnger-Verlag 35

36 Neuro Fuzzy Models In neural networks there are algorthms that have a theoretcal justfcaton and some have no theoretcal elucdaton. Gven that there are strengths and weaknesses of both statstcal and other soft computng algorthms (e.g. neural nets, fuzzy logc), one should ntegrate the two classfer desgn strateges (bd) Raudys, Šarûunas. (2001). Statstcal and Neural Classfers: An ntegrated approach to desgn. London: Sprnger-Verlag 36

37 Neuro Fuzzy Models The remarkable qualtes of neural networks: the dynamcs of a sngle layer perceptron progresses from the smplest algorthms to the most complex algorthms: Intal Tranng each pattern class characterzed by sample mean vector neuron behaves lke EDC ; Further Tranng neuron begns to evaluate correlatons and varances of features neuron behaves lke standard lnear Fscher classfer More tranng neuron mnmzes number of ncorrectly dentfed tranng patterns neuron behaves lke a support vector classfer. Statstcans and engneers usually desgn decsonmakng algorthms from expermental data by progressng from smple algorthms to more complex ones. Raudys, Šarûunas. (2001). Statstcal and Neural Classfers: An ntegrated approach to desgn. London: Sprnger-Verlag 37

38 Neuro-fuzzy models Adaptve Networks A network typcally comprses a set of nodes connected by drected lnks. Each node performs a statc node functon on ts ncomng sgnals to generate a sngle node output. Each lnk specfes the drecton of sgnal flow from one node to another. An adaptve network s a network structure whose overall nput-output behavour s determned by a collecton of modfable parameters. 38

39 Neuro-fuzzy models Drected graphs The nodes of a drected graph are called processng elements. The lnks of the graph are called connectons. Each connecton functons as an nstantaneous undrectonal sgnal-conducton path. Each processng element can receve any number of ncomng connectons, sometmes called nput connectons. Each processng element can have any number of outgong connectons but the sgnals n all of these must be the same. 39

40 Neuro-fuzzy models Drected graphs The nodes of a drected graph are called processng elements. The lnks of the graph are called connectons. Each connecton functons as an nstantaneous undrectonal sgnal-conducton path. Each processng element can receve any number of ncomng connectons, sometmes called nput connectons. Each processng element can have any number of outgong connectons but the sgnals n all of these must be the same. Input Connectons Processng Unt Output Connecton Fan Out 40

41 Neuro-fuzzy models Drected graphs In effect, each processng element has a sngle output connecton that can branch or fan out nto copes to form multple output connectons (sometmes called collaterals), each of whch carres the same dentcal sgnal (the processng element's output sgnal). Processng elements can have local memory. Each processng element possesses a transfer functon whch can use (and alter) local memory, can use nput sgnals, and whch produces the processng element's output sgnal. 41

42 Neuro-fuzzy models Drected graphs The only nputs allowed to the transfer functon are the values stored n the processng element's local memory and the current values of the nput sgnals n the connectons receved by the processng element. The only outputs allowed from the transfer functon are values to be stored n the processng element's local memory and the processng element's output sgnal. 42

43 Neuro-fuzzy models Drected graphs Transfer functons can operate contnuously or epsodcally. If they operate epsodcally, there must be an nput called "actvate" that causes the processng element's transfer functon to operate on the current nput sgnals and local memory values and to produce an updated output sgnal (and possbly to modfy local memory values). Contnuous processng elements are always operatng. The "actvate" nput arrves va a connecton from a schedulng processng element that s part of the network. 43

44 Neuro-fuzzy models Drected graphs Input Connectons Processng Unt Output Connecton Fan Out 44

45 Real Neuroscence Brans compute. Ths means that they process nformaton, creatng abstract representatons of physcal enttes and performng operatons on ths nformaton n order to execute tasks. One of the man goals of computatonal neuroscence s to descrbe these transformatons as a sequence of smple elementary steps organzed n an algorthmc way. The mechanstc substrate for these computatons has long been debated. Tradtonally, relatvely smple computatonal propertes have been attrbuted to the ndvdual neuron, wth the complex computatons that are the hallmark of brans beng performed by the network of these smple elements. London, Mchael and Mchael Häusser (2005). Dendrtc Computaton. 45 Annual Revew of Neuroscence. Vol. 28, pp

46 DEFINITIONS: Artfcal Neural Networks Artfcal neural networks emulate threshold behavour, smulate co-operatve phenomenon by a network of 'smple' swtches and are used n a varety of applcatons, lke bankng, currency tradng, robotcs, and expermental and anmal psychology studes. These nformaton systems, neural networks or neuro-computng systems as they are popularly known, can be smulated by solvng frst-order dfference or dfferental equatons. 46

47 What computers can do? Artfcal Neural Networks In a restrcted sense artfcal neurons are smple emulatons of bologcal neurons: the artfcal neuron can, n prncple, receve ts nput from all other artfcal neurons n the ANN; smple operatons are performed on the nput data; and, the recpent neuron can, n prncple, pass ts output onto all other neurons. Intellgent behavour can be smulated through computaton n massvely parallel networks of smple processors that store all ther long-term knowledge n the connecton strengths. 47

48 DEFINITIONS: Neurons & Appendages A neuron s a cell wth appendages; every cell has a nucleus and the one set of appendages brngs n nputs the dendrtes and another set helps to output sgnals generated by the cell The Real McCoy DENDRITES NUCLEUS CELL BODY AXON 48

49 DEFINITIONS: Neurons & Appendages The human bran s manly composed of neurons: specalzed cells that exst to transfer nformaton rapdly from one part of an anmal's body to another. Nucleus Dendrte Soma Axon Termnals SOURCE: 49

50 DEFINITIONS: Neurons & Appendages Ths communcaton s acheved by the transmsson (and recepton) of electrcal mpulses (and chemcals) from neurons and other cells of the anmal. Lke other cells, neurons have a cell body that contans a nucleus enshrouded n a membrane whch has double-layered ultrastructure wth numerous pores. Nucleus Dendrte Soma Axon Termnals SOURCE: 50

51 DEFINITIONS: Neurons & Appendages Neurons have a varety of appendages, referred to as 'cytoplasmc processes known as neurtes whch end n close apposton to other cells. In hgher anmals, neurtes are of two varetes: Axons are processes of generally of unform dameter and conduct mpulses away from the cell body; dendrtes are shortbranched processes and are used to conduct mpulses towards the cell body. The ends of the neurtes,.e. axons and dendrtes are called synaptc termnals, and the cell-to-cell contacts they make are known as synapses. Nucleus Dendrte Soma Axon Termnals SOURCE: 51

52 DEFINITIONS: The fan-ns and fan-outs neurons wth 10 4 connectons and an average of 10 spkes per second = 1015 adds/sec. Ths s a lower bound on the equvalent computatonal power of the bran fan-n Asynchronous frng rate, c. 200 per sec. summaton 4 10 fan-out meters per sec. 52

53 Notes on Artfcal Neural Networks Input sgnals to a neural network from outsde the network arrve va connectons that orgnate n the outsde world. Outputs from the network to the outsde world are connectons that leave the network. 53

54 Notes on Artfcal Neural Networks Artfcal Neural Networks (ANN) are computatonal systems, ether hardware or software, whch mmc anmate neural systems comprsng bologcal (real) neurons. An ANN s archtecturally smlar to a bologcal system n that the ANN also uses a number of smple, nterconnected artfcal neurons. 54

55 Notes on Artfcal Neural Networks Observed Bologcal Processes (Data) Neural Networks & Neuroscences Bologcally Plausble Mechansms for Neural Processng & Learnng (Bologcal Neural Network Models) Theory (Statstcal Learnng Theory & Informaton Theory) 55

56 Bran The Processor! 56

57 Real Neuroscence Brans compute. Ths means that they process nformaton, creatng abstract representatons of physcal enttes and performng operatons on ths nformaton n order to execute tasks. One of the man goals of computatonal neuroscence s to descrbe these transformatons as a sequence of smple elementary steps organzed n an algorthmc way. The mechanstc substrate for these computatons has long been debated. Tradtonally, relatvely smple computatonal propertes have been attrbuted to the ndvdual neuron, wth the complex computatons that are the hallmark of brans beng performed by the network of these smple elements. London, Mchael and Mchael Häusser (2005). Dendrtc Computaton. 57 Annual Revew of Neuroscence. Vol. 28, pp

58 Notes on Artfcal Neural Networks Neural Networks 'learn' by adaptng n accordance wth a tranng regmen: The network s subjected to partcular nformaton envronments on a partcular schedule to acheve the desred end-result. There are three major types of tranng regmens or learnng paradgms: SUPERVISED; UN-SUPERVISED; REINFORCEMENT or GRADED. 58

59 Notes on Artfcal Neural Networks Neurons & Appendages A neuron s a cell wth appendages; every cell has a nucleus and the one set of appendages brngs n nputs the dendrtes and another set helps to output sgnals generated by the cell DENDRITES NUCLEUS CELL BODY AXON 59

60 Notes on Artfcal Neural Networks: The fan-ns and fan-outs neurons wth 10 4 connectons and an average of 10 spkes per second = adds/sec. Ths s a lower bound on the equvalent computatonal power of the bran fan-n Asynchronous frng rate, c. 200 per sec. summaton 4 10 fan-out meters per sec. 60

61 Notes on Artfcal Neural Networks: Bologcal and Artfcal NN s Entty Bologcal Neural Networks Artfcal Neural Networks Processng Unts Neurons Network Nodes Input Dendrtes Network Arcs Output Axons Network Arcs Inter-lnkage Connectvty Synaptc Contact (Chemcal and Electrcal) Node to Node va Arcs Plastc Connectons Weghted Connectons Matrx 61

62 Notes on Artfcal Neural Networks: Bologcal and Artfcal NN s 62

63 Notes on Artfcal Neural Networks: Bologcal and Artfcal NN s 413 major areas n anmal bran Areas connected to each other Some more connected than others 63

64 Notes on Artfcal Neural Networks: An operatonal vew of Artfcal NN s x 1 Inpu ut Sgnals x 2 x 3 x 4 w k1 w k2 w k3 w k4 Neuron x k Summng Juncton Σ Actvaton Functon A schematc for an 'electronc' neuron b k 64 y k Output Sgnal

65 Notes on Artfcal Neural Networks: An operatonal vew of Artfcal NN s A neural network comprses A set of processng unts A state of actvaton An output functon for each unt A pattern of connectvty among unts A propagaton rule for propagatng patterns of actvtes through the network An actvaton rule for combnng the nputs mpngng on a unt wth the current state of that unt to produce a new level of actvaton for the unt A learnng rule whereby patterns of connectvty are modfed by experence An envronment wthn whch the system must operate65

66 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron Logc Gate: A dgtal crcut that mplements an elementary logcal operaton. It has one or more nputs but ONLY one output. The condtons appled to the nput(s) determne the voltage levels at the output. The output, typcally, has two values 0 or 1. Dgtal Crcut: A crcut that responds to dscrete values of nput (voltage) and produces dscrete values of output (voltage). Bnary Logc Crcuts: Extensvely used n computers to carry out nstructons and arthmetcal processes. Any logcal procedure maybe effected by a sutable combnatons of the gates. Bnary crcuts are typcally formed from dscrete components lke the ntegrated crcuts. 66

67 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron Logc Crcuts: Desgned to perform a partcular logcal functon based on AND, OR (ether), and NOR (nether). Those crcuts that operate between two dscrete (nput) voltage levels, hgh & low, are descrbed as bnary logc crcuts. Logc element: Small part of a logc crcut, typcally, a logc gate, that may be represented by the mathematcal operators n symbolc logc. 67

68 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron Gate Input(s) Output AND Two (or more) Hgh f and only f both (or all) nputs are hgh. NOT One Hgh f nput low and vce versa OR Two (or more) Hgh f any one (or more) nputs are hgh 68

69 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron The operaton of an AND gate Input 1 Input 2 Output AND (x,y)= mnmum_value(x,y); AND (1,0)=mnmum_value(1,0)=0; AND (1,1)=mnmum_value(1,1)=1 69

70 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. A hard-wred perceptron below performs the AND operaton. Ths s hardwred because the weghts are predetermned and not learnt x 1 x 2 w=+1 1 w=+1 2 θ = -1.5 Σ=w 1 x 1 +w 2 x 2 +θ y=1 f Σ 0; y=0 fσ< 0 70

71 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. A learnng perceptron below performs the AND operaton. An algorthm: Tran the network for a number of epochs (1) Set ntal weghts w1 and w2 and the bas θ to set of random numbers; (2) Compute the weghted sum: x 1 *w 1 +x 2 *w 2 + θ (3) Calculate the output usng a delta functon y()= delta(x 1 *w 1 +x 2 *w 2 + θ ); delta(x)=1, f x s greater than zero, delta(x)=0,f x s less than or equal to zero (4) compute the dfference between the actual output and desred output: e()= y desred - y() (5) If the errors durng a tranng epoch are all zero then stop otherwse update w j (+1)=w j ()+ α*x j *e(), j=1,2 71

72 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces: α=0.1 Θ=-0.1 Epoch X1 X2 Y desre Intal Weghts Actual Error Fnal Weghts d W1 W2 Output W1 W

73 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. Epoch X1 X2 Y desre d Intal W1 Weghts W2 Actual Output Error Fnal W1 Weghts W2 73

74 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. Epoch X1 X2 Y desre d Intal W1 Weghts W2 Actual Output Error Fnal W1 Weghts W2 74

75 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. Epoch X1 X2 Y desre d Intal W1 Weghts W2 Actual Output Error Fnal W1 Weghts W2 75

76 Notes on Artfcal Neural Networks: Rosenblatt s Perceptron A sngle layer perceptron can perform a number of logcal operatons whch are performed by a number of computatonal devces. Epoch X1 X2 Y desre d Intal W1 Weghts W2 Actual Output Error Fnal W1 Weghts W2 76

77 Neuro Fuzzy Models The remarkable qualtes of neural networks: the dynamcs of a sngle layer perceptron progresses from the smplest algorthms to the most complex algorthms: Intal Tranng each pattern class characterzed by sample mean vector neuron behaves lke EDC ; Further Tranng neuron begns to evaluate correlatons and varances of features neuron behaves lke standard lnear Fscher classfer More tranng neuron mnmzes number of ncorrectly dentfed tranng patterns neuron behaves lke a support vector classfer. Statstcans and engneers usually desgn decsonmakng algorthms from expermental data by progressng from smple algorthms to more complex ones. Raudys, Šarûunas. (2001). Statstcal and Neural Classfers: An ntegrated approach to desgn. London: Sprnger-Verlag 77