1 An Improved Neural Network Algorthm for Classfyng the Transmsson Lne Faults S. Vaslc, Student Member, IEEE, M. Kezunovc, Fellow, IEEE Abstract--Ths study ntroduces a new concept of artfcal ntellgence based algorthm for classfyng the faults n power system networks. Ths classfcaton dentfes the exact type and zone of the fault. The algorthm s based on unque type of neural network specally developed to deal wth large set of hghly dmensonal nput data. An mprovement of the algorthm s proposed by mplementng varous steps of nput sgnal preprocessng, through the selecton of parameters for analog flterng, and values for the data wndow and samplng frequency. In addton, an advanced technque for classfcaton of the test patterns s dscussed and the man advantages comparng to prevously used nearest neghbor classfer are shown. Index Terms--clusterng methods, electromagnetc transents, neural networks, pattern classfcaton, power system faults, protectve relayng, testng, tranng. I. INTRODUCTION HIS paper deals wth neural network based technque for Tprotectve relayng n power system networks. The problem of detectng and classfyng the transmsson lne faults has been known for a long tme. Tradtonal relayng prncples are based on predetermned settng and take nto account only common and easly antcpated fault condtons. Several varyng parameters: type of fault, fault locaton, fault mpedance, and fault ncdent tme, as well as many other condtons mposed by actual network confguraton, voltage levels, and varsty of other events, determne the correspondng transent current and voltage waveforms detected by the relays at lne ends. All these effects nfluence the relays tuned only to perform well durng antcpated fault condtons. The new classfcaton approach has to relably conclude, n a very short tme (around one cycle), whether, where and whch type of fault occurs under a varety of tmevaryng operatng condtons [1]. Neural networks can be used to solve power system protecton problems, partcularly those where tradtonal approaches have dffculty n achevng the desred speed, accuracy and selectvty. Neural networks are convenent for Ths study was supported by an Army/EPRI contract # WO 8333-05 between EPRI and Carnege Mellon Unversty, and has been carred out by Texas A&M Unversty under the subcontract # 542995-42590 ttled "Self- Evolvng Agents for Montorng, Control and Protecton of Large, Complex Dynamc Systems". S. Vaslc and M. Kezunovc are wth the Department of Electrcal Engneerng, Texas A&M Unversty, College Staton, TX 77843-3128 USA (e-mals respectvely: svaslc@ee.tamu.edu, kezunov@ee.tamu.edu). these tasks because they use ndvdual examples to capture general, always complex and nonlnear, relatonshps among the data. They learn from the envronment and adapt ther recognton capabltes. Varous applcatons of neural networks were used n the past to mprove recognton of the mpedance used n dstance relayng of transmsson lnes [2]. These applcatons are manly based on wdely used multlayer feed-forward networks. Whenever nput patterns wth large dmensonalty are present (as n ths partcular case), tranng of these networks s very slow, needs much larger tranng sets, and very easly converges on local mnma, usually very far from the global mnmum. On-lne tranng of ths type of network for any new tranng data requres presentng the entre set of tranng patterns agan. Instead of usng multlayer neural network, a unque type of neural network may be used for fault classfcaton [3]-[6]. The network s appled drectly to the samples of voltages and currents, and produce the fault type and zone classfcaton n real tme. Ths network s based on ISODATA clusterng algorthm [7] and belongs to a group of specal neural networks named Self-Organzng Maps [8]. The adaptve behavor of the neural network s descrbed by Adaptve Resonance Theory [9]. The man am of ths work s to provde some nsghts nto possble approaches to sgnfcantly mprove exstng neural network algorthm, through approprate condtonng of nput patterns as well as optmzng the classfcaton of new patterns that have not been presented n the tranng process. Electromagnetc transent program ATP [10] s used for smulatng a specally developed power network model. Smulaton outputs of a large number of scenaros, ncludng varous fault and normal operatng states, are used as a sgnal generator for the neural network algorthm desgn, mplemented n MATLAB [11]-[13]. The paper s organzed as follows. Secton II presents a bref descrpton of the neural network classfcaton algorthm. Secton III, dvded nto subsectons, demonstrates generaton and preprocessng of neural network nput sgnals, and defnes classfcaton task. An advanced classfcaton technque s ntroduced n secton IV. The concluson s gven at the end. II. NEURAL NETWORK CLASSIFICATION ALGORITHM Neural networks try to produce a concse representaton of system's behavor through dentfyng natural groupngs of data
2 from large data sets. The am of ths procedure, called clusterng, s to partton a gven set of nput data (patterns) nto several groups or clusters, so that each pattern s assgned to a unque cluster. Patterns that belong to the same cluster should be as smlar as possble, whle patterns that belong to dfferent clusters should be as dfferent as possble. Class label s assgned to each cluster, where class symbolzes a group of patterns wth a common characterstc. Self-organzng maps are specal type of neural networks that map nput patterns wth smlar features nto adjacent clusters after enough nput patterns have been presented [8]. The smlarty between patterns s usually measured by calculatng the Eucldean dstance between two n-dmensonal nput vectors. After tranng, self-organzed clusters represent prototypes of classes of nput patterns. ISODATA clusterng algorthm dscovers the most representatve postons of prototypes n the pattern space [7]. Adaptve Resonance Theory s characterzed by ts ablty to form clusters ncrementally, whenever a pattern, suffcently dfferent from all prevously presented patterns, appears [9]. Incremental clusterng capablty can handle an nfnte set of nput data, because ther cluster prototype unts contan mplct representaton of all the nputs prevously encountered. Usng ths technque, the on-lne tranng due to non-statonary nputs may be easly mplemented. Ths neural network s wthout hdden layers and ts selforganzed structure depends only on the presented nput data set. The neural network tranng conssts of unsupervsed and supervsed learnng phases (Fg. 1) [4]. Durng unsupervsed learnng, patterns are presented wthout ther class labels. Ths procedure tres to dentfy characterstc patterns or prototypes that can serve as cluster centers. The outcome of unsupervsed learnng s a stable famly of clusters, defned as hyperspheres n an n-dmensonal space, where n denotes the number of features,.e. the length of nput vector. Unsupervsed learnng forms stable famly of both "clean" (homogenous, havng patterns wth the same class label) and "mxed" (non-homogenous, havng patterns wth two or more class labels) clusters. It does not requre ether the ntal guess of the number of cluster, or the ntal cluster center coordnates. It conssts of two steps: ntalzaton and stablzaton. The Intalzaton establshes ntal cluster structure based on smlarty between the patterns, and by presentng each pattern only ones. Durng the stablzaton all patterns are presented agan untl a stable cluster structure occurs and there are no patterns changng ther cluster membershp durng the teratons. Stablzaton phase s repeated many tmes untl no pattern changes ts cluster membershp. In the supervsed learnng the class label s assocated wth each data pont. Supervsed learnng separates "mxed" clusters from the "clean" ones. It assgns class labels to the "clean" clusters. The tunng parameter ρ, called vglance parameter, controls the number and sze of generated clusters and s beng consecutvely decreased durng teratons. UNSUPERVISED LEARNING SUPERVISED LEARNING Start Patterns zaton Intalzaton Patterns Clusterng Stablzaton Patterns Clusterng Clean and Mxed Clusters Identfcaton Clean Clusters Class Membershp Assgnement Clean Clusters Extracton Input Set Reducton Decreasng Vglance Parameter ρ Number of Mxed Clusters s Zero or ρ < ε? Yes End Fg. 1. Neural Network Clusterng Algorthm New Iteraton Unsupervsed and supervsed learnng procedures are repeated unless only "clean" clusters exst, or current value of vglance parameter s less then specfed value. Durng the testng procedure, dstances between each test pattern and establshed clusters are calculated, and the nearest neghbors classfer [14] s effcently mplemented snce the number of optmzed prototypes s sgnfcantly smaller then the number of tranng patterns. The outcome of the testng are class labels assgned to test patterns accordng to the most common value among the K nearest prototypes. III. ALGORITHM IMPLEMENTATION FOR CLASSIFYING TRANSMISSION LINE FAULTS A. Generaton of Input Sgnals Power system used for algorthm evaluaton and testng s 345 kv power system secton, provded by Relant Energy (RE) HL&P company. The model of the gven power network s mplemented n Alternatve Transent Program (ATP) program [10]. It s used for smulatng varous fault scenaros on one of the transmsson lnes, by varyng fault parameters. The reduced network equvalent was obtaned by usng the load flow and short crcut data, and verfed usng both the steady state and transent state results. Neural network based algorthm takes voltage and current measurements from one end of the lne. It has to be traned to recognze the type and No
3 zone of the fault. The classfcaton s based only on drect use of samples wthout mpedance computng. The example of the smulaton output data, ncludng three-phase lne currents and voltages, for one specfc case, phases A to B to ground fault, s shown n Fg. 2. Current and voltage samples obtaned through smulatons are used for formng tranng and test patterns for protectve algorthm learnng and evaluaton. Fg. 3. Voltage and current samples and movng data wndow used for formng the patterns. Fg. 2. Typcal voltage and current measurements for ABG fault. Tranng patterns are generated by specfyng several values for each fault parameter, and combnng these values to cover possblty space of fault cases [14]. Prototypes formed durng tranng represent the most typcal patterns obtaned through smulatons. Test patterns for algorthm evaluaton n heurstc, prevously unseen, stuatons are generated by random settng of all fault parameters. The test patterns mght be very heterogeneous and qute dfferent from the tranng patterns snce there are many operatng states and possble events n the power network. They are classfed accordng to ther smlarty to prototypes adopted durng tranng. B. Preprocessng of Input Sgnals Neural network tranng can be made more effcent f certan preprocessng steps are performed on the network nputs. Pattern extracton from obtaned measurements depends on several algorthm tunng parameters. Those parameters sgnfcantly determne qualty of algorthm tranng. Input nto the neural network s n the form of the movng data wndow contanng samples of phase currents and voltages (Fg. 3). Selecton of sampled data for tranng n desred data wndow may nclude: three phase currents, three phase voltages, or both the three phase currents and voltages. Comparson of algorthm performances n all three cases s a challengng task. Examples of extracted patterns for dfferent values of algorthm parameters are shown n Fgs. 4 to 6 for the choce of three-phase currents. Phase current measurements are fltered by an analog flter and sampled wth desred samplng frequency. Patterns are extracted from these measurements durng the desred data wndow (after the fault occurs), normalzed, and placed together n one row to form feature vector components. Fg. 4. Example of pattern feature vectors for dfferent values of II order Butterworth analog flter crossover frequences.
4 used for tranng, and gves better nformaton about the sgnals. At the same tme t means slower tranng and testng, and may cause defcences n classfyng the fault n a real tme. Samplng frequency has smlar effect on formng the patterns as the data wndow does. Pattern feature vectors for samplng frequences 1 khz, 2 khz, and 5 khz are shown n Fg. 6. Increased samplng frequency offers mproved sgnal detecton but also may cause sgnfcant computatonal burden. In each partcular applcaton, extensve set of smulatons have to be performed to optmze all mentoned parameters for algorthm tranng. When ths condton s satsfed, algorthm tranng may be started. Fg. 5. Example of pattern feature vectors for dfferent lengths of data wndow. Fg. 6. Example of pattern feature vectors for dfferent values of samplng frequency. The appled analog flter s the second order Butterworth flter, and the effect of varous selected crossover frequences, 0.5 khz, 1 khz, and 2 khz, s shown n Fg. 4. The flter crossover frequency has to be carefully adopted, such that the nose and other hgh frequency components are fltered, whle the characterstc sgnal waveform s retaned. The effect of the selected data wndow s llustrated n Fg. 5. for the wndow lengths of 0.5, 1 and 1.5 cycles (1 cycle = 16.67 ms). Longer wndow ncreases the number of features C. Classfcaton Task Type of classfcaton mght be based on detectng the fault type (, AG, BG, CG, AB/ABG, BC/BCG, CA/CAG, ABC/ABCG), fault zone (, Zone I, Zone II), fault resstance (, Low, Hgh), or any combnaton among them. Classfcaton of testng patterns s performed by usng the cluster structure establshed durng tranng and applyng the K-nearest neghbor rule. Clusters are formed as spheres n the n-dmensonal space. Smplfed example n two dmensons s gven n Fg. 7. The cluster centers are dentfed as prototypes or characterstc patterns. Input parameter for algorthm testng s only the number K of the nearest neghbors for the rule. Test patterns are extracted and normalzed from generated patterns usng the same procedure as for tranng patterns and have equal number of features. For each test pattern, Eucldean dstances to all clusters retreved from reference set are computed and sorted n an ncreasng order. A-C -C A-C A-G -C A-C-G C-G B-G I B-C -C-G B-C I B-C-G I Fg. 7. Example of cluster structure establshed durng tranng. B-G I A-G B-G I The most frequent class label of K nearest clusters s computed and assgned to actual pattern. If the nput pattern belongs to any of the normal-state clusters then the nput data wndow s "moved" for one sample and the comparson s performed agan. If the nput pattern does not belong to the normal-state clusters then the fault s detected and executon of the fault classfcaton logc s ntated. The parameter used
5 to force the neural network to make the fnal decson s the tme. After decson tme has expred, f pattern stll does not come back to the normal state, the neural network wll classfy the fault event accordng to the fault type detected n that nstance. IV. ADVANCED K-NEAREST NEIGHBORS TECHNIQUE Standard verson of the K-nearest neghbors rule may be mproved to acheve better classfcaton of the test set of patterns that corresponds to a new set of smulated events, prevously unseen durng tranng. Between establshed clusters there s an unlabeled space makng the pattern recognton more dffcult, because n real stuaton many new upcomng patterns appear n that space. Gven a set of classfed clusters, the standard K-nearest neghbors rule [13] determnes the classfcaton of the nput pattern x based only on the class labels of the K closest clusters n the cluster structure establshed durng tranng µ x ) = f [ K, µ ( v )] (1) j ( j l where: µ j ( v l ) s membershp value whch determnes the degree of belongng of cluster l to class j; µ j ( x ) s membershp value of pattern belongng to class j; = 1,, P; where P s number of patterns; j = 1, C; where C s number of classes; l = 1, K; where K s number of neghbors; v 1, v2,, vk denotes the centers of K nearest neghbors of pattern x. µ j ( v l ) has only crsp values 0 or 1, dependng on whether or not a cluster v l belongs to class j: 1 f cluster l belongs to class j µ j ( v l ) = (2) 0 otherwse In ths rule all K nearest clusters have the equal mportance, wthout takng nto account ther rad, and dstances to the pattern that has to be classfed. The advanced K-nearest neghbors technque s a fuzzy classfcaton technque that generalzes the K-nearest neghbors rule. New patterns are classfed based on the weghted dstances ( d l ) to K nearest clusters, as well as on relatve sze ( r l ) and class labels ( c l ) of these clusters (Fg. 8). The advanced K-nearest neghbors technque calculates a vector of membershp values ( µ 1( x), µ 2 ( x), µ C ( x)) of nput pattern x n the exstng classes. The class membershp values are calculated based on the followng formula: µ x ) = f [ K, µ ( v ), d ( x )] (3) j ( j l l where now µ v ) may take any value between 0 and 1, j ( l representng the relatve sze of the actual cluster l. Each cluster belongs to one of the exstng classes, wth membershp value defned by the followng adopted relaton: rl rmax f cluster l belongs to class j µ j ( vl ) = (4) 0 otherwse The membershp degree of cluster v l belongng to class j s equal to the rato between radus ( r l ) of actual cluster l and radus ( r max ) of the largest cluster n the cluster structure. The outcome s that the larger clusters have more nfluences then the smaller ones, and the clusters wth longest radus have µ j ( v l ) = 1. r 9 r 3 c 3 r 6 c 6 c 9 d 9 c r 1 1 d 6 d 3 d 7 d 1 test pattern d 8 d 2 d 5 c r 1 2 c 5 d 4 r 5 Fg. 8. K nearest clusters (K=9 n ths example) to test pattern, wth ther class labels c j, rad r j, and dstances d j. Another mprovement toward realstc classfcaton s takng nto account dstances between pattern x and K nearest clusters. The dstance d l ( x ) may be generally selected to be a weghted Eucldean dstance between pattern x and cluster l l l c 4 m r 4 c 7 d ( x ) = x v (5) where the parameter m determnes how heavly the dstance s weghted when calculatng the class membershp. If two or more of K nearest clusters have the same class label, then they gve cumulatve membershp value to that class. When values µ j ( x ) for all K neghbors have been calculated, pattern x s classfed to the class wth the hghest membershp degree Class( x ) = { j µ ( x ) µ ( x ), j, m = 1,, C m j (6) j m Ths dea of new fuzzyfed classfcaton technque algorthm wll help classfy better a varety of test patterns, comparng to the prevously used K-nearest neghbors algorthm. The optmal values, for number of neghbors K and parameter m whch establshes weghted dstances, have to be determned and appled n each partcular mplementaton. r 7 c 8 r 8
6 V. CONCLUSION Ths study ntroduces possble drectons to mprove exstng neural network algorthm for classfyng the transmsson lne faults. Ths algorthm was used earler n the efforts amed at replacng dstance relays wth new relays not havng tradtonal settng and based on pattern recognton capabltes. The algorthm s a specfc type of clusterng algorthm. It translates nput patterns nto pattern prototypes, the structure of clusters that represents varous classes of nput data sets. The algorthm s very flexble and easly enables further modfcatons and upgradng. Condtonng of nput sgnals, as well as selecton of values for analog flterng, data wndow for takng the patterns, and samplng frequency, play sgnfcant role n the algorthm behavor durng tranng and performance durng testng. Dfferent aspects of these factors are llustrated through several examples. Furthermore, classfcaton of the test patterns s analyzed through comparson of a smple K-nearest neghbors classfer used so far, and the new fuzzyfed approach of that classfer. The advanced approach offers more realstc classfcaton of the test patterns. Ths s done by takng nto account weghted dstances between a pattern and the K nearest clusters, as well as the relatve sze of those clusters. Proposed tunngs and mprovements of the neural network algorthm may enable better classfcaton of the fault type and zone. Combnes use of neural and fuzzy technques n the same algorthm leads to complex reasonng that mproves the event classfcaton ablty for recognzng a varety of the events that could happen n power networks. VI. REFERENCES [1] Power System Relayng Commttee, Workng Group D5 of the Lne Protecton Subcommttee, Proposed statstcal performance measures for mcroprocessor-based transmsson lne protectve relays, Part I and II, IEEE Trans. Power Delvery, vol. 12, no. 1, pp. 134-156, Jan. 1997. [2] M. Kezunovc, "A Survey of Neural Net Applcatons to Protectve Relayng and Fault Analyss", Engneerng Intellgent Systems, vol. 5, no. 4, pp. 185-192, Dec. 1997. [3] Y. H. Pao, Adaptve Pattern Recognton and Neural Networks, Readng: Addson Wesley, 1989, p. 309. [4] Y. H. Pao and D. J. Sobajc, "Combned Use of Unsupervsed and Supervsed Learnng for Dynamc Securty Assessment", IEEE Trans. Power Systems, vol. 7, no 2, pp. 878-884, 1992. [5] M. Kezunovc M., I. Rkalo, and D. Sobajc, "Hgh-speed Fault Detecton and Classfcaton wth Neural Nets", Electrc Power Systems Research, vol. 34, pp. 109-116, 1995. [6] M. Kezunovc and I. Rkalo, "Detect and Classfy Faults Usng Neural Nets", IEEE Computer Applcatons n Power, vol. 9, no. 4, pp. 42-47, 1996. [7] G. H. Ball and D. J. Hall, "A Clusterng Technque for Summarzng Multvarate Data", Behavoral Scence, vol. 12, pp. 345-370, 1967. [8] T. Kohonen, Self-Organzng Maps, 1st ed., New York: Sprnger- Verlag, 1997, p. 426. [9] G. A. Carpenter and S. Grossberg, "ART2: self-organzaton of stable category recognton codes for analog nput patterns", Appled Optcs, vol. 26, no. 23, pp. 4919-4930, Dec. 1987. [10] CanAm EMTP User Group, Alternatve Transent Program (ATP) Rule Book, Portland, 1992. [11] The MathWorks, Inc., Usng MATLAB, Natck, Jan. 1999. [12] M. Kezunovc and S. Vaslc, "Advanced Software Envronment for Evaluatng Protecton Performance Durng Power System Dsturbances Usng Relay Models", CIGRE SC 34 Colloquum, Sbu, Romana, Sep. 2001. [13] M. Kezunovc and S. Vaslc, "Desgn and Evaluaton of Context- Dependent Protectve Relayng Approach", IEEE Porto Power Tech' Conference, Porto, Portugal, Sep. 2001. [14] T. M. Cover and P. E. Hart, "Nearest Neghbor Pattern Classfcaton", IEEE Trans. Informaton Theory, vol. IT-13, pp. 21-27, 1967. VII. BIOGRAPHIES Slavko Vaslc (S'00) receved hs B.S. and M.S. degrees n electrcal engneerng from Unversty of Belgrade n 1993 and 1999, respectvely, and currently s a Ph.D. canddate n electrcal engneerng at Texas A&M Unversty. Hs research nterests are neural networks, fuzzy logc, genetc algorthms, multvarable, robust and adaptve systems, and ther mplementaton n process control and pattern recognton, especally n power systems control, protecton and montorng. Mladen Kezunovc (S'77, M'80, SM'85, F'99) receved hs Dpl. Ing. degree from the Unversty of Sarajevo, the M.S. and Ph.D. degrees from the Unversty of Kansas, all n electrcal engneerng, n 1974, 1977 and 1980, respectvely. He has been wth Texas A&M Unversty snce 1987 where he s the Eugene E. Webb Professor and Drector of Electrc Power and Power Electroncs Insttute. Hs man research nterests are dgtal smulators and smulaton methods for equpment evaluaton and testng as well as applcaton of ntellgent methods to control, protecton and power qualty montorng. Dr. Kezunovc s a regstered professonal engneer n Texas, and a Fellow of IEEE.