Soochow University: Description and Analysis of the Chinese Word Sense Induction System for CLP2010

Transcription

1 Soochow Unvesty: Descpton and Analyss of the Chnese Wod Sense Inducton System fo CLP2010 Hua Xu Bng Lu Longhua Qan Guodong Zhou Natual Language Pocessng Lab School of Compute Scence and Technology Soochow Unvesty, Suzhou, Chna Emal: Abstact Recent studes on wod sense nducton (WSI) manly concentate on Euopean languages, Chnese wod sense nducton s becomng popula as t pesents a new challenge to WSI. In ths pape, we popose a featue-based appoach usng the spectal clusteng algothm to ths poblem. We also compae vaous clusteng algothms and smlaty metcs. Expemental esults show that ou system acheves pomsng pefomance n F-scoe. 1 Intoducton Wod sense nducton (WSI) s an open poblem of natual language pocessng (NLP), whch govens the pocess of automatc dscovey of the possble senses of a wod. WSI s smla to wod sense dsambguaton (WSD) both n methods employed and n poblem encounteed. In the pocedue of WSD, the senses ae assumed to be known and the task focuses on choosng the coect one fo an ambguous wod n a context. The man dffeence between them s that the task of WSD geneally eques lagescale manually annotated lexcal esouces whle WSI does not. As WSI doesn t ely on the manually annotated copus, t has become one of the most mpotant topcs n cuent NLP eseach (Pantel and Ln, 2002; Nell, 2002; Rapp, 2003). Typcally, the nput to a WSI algothm s a taget wod to be dsambguated. The task of WSI s to dstngush whch taget wods shae the same meanng when they appea n dffeent contexts. Such esult can be at the vey least used as empcally gounded suggestons fo lexcogaphes o as nput fo WSD algothm. Othe possble uses nclude automatc thesauus o ontology constucton, machne tanslaton o nfomaton eteval. Compaed wth Euopean languages, the study of WSI n Chnese s scace. Futhemoe, as Chnese has ts specal wtng style and Chnese wod senses have the own chaactestcs, the methods that wok well n Englsh may not pefom effectvely n Chnese and the usefulness of WSI n eal-wold applcatons has yet to be tested and poved. The coe dea behnd wod sense nducton s that contextual nfomaton povdes mpotant cues egadng a wod s meanng. The dea dates back to (at least) Fth (1957) ( You shall know a wod by the company t keeps ), and undeles most WSD and lexcon acquston wok to date. Fo example, when the adveb phase occung po to the ambguous wod, then the taget wod s moe lkely to be a veb and the meanng of whch s to hold somethng ; Othewse, f an adjectve phase locates n the same poston, then t pobably means confdence n Englsh. Thus, the wods suounds the taget wod ae man contbuto to sense nducton. The bake off task 4 on WSI n the fst CIPS- SIGHAN Jont Confeence on Chnese Language Pocessng (CLP2010) s ntended to pomote the exchange of deas among patcpants and mpove the pefomance of Chnese WSI systems. Geneally, ou WSI system also adopts a clusteng algothm to goup the contexts of a taget wod. Dffeently, afte geneat- Coespondng autho

2 ng featue vectos of wods, we compute a smlaty matx wth each cell denotng the smlaty between two contexts. Futhemoe, the set of smlaty values of a context wth othe contexts s vewed as anothe knd of featue vecto, whch we efe to as smlaty vecto. Both featue vectos and smlaty vectos can be sepaately used as the nput to clusteng algothms. Expemental esults show ou system acheves good pefomances on the development dataset as well as on the fnal test dataset povded by the CLP System Descpton Ths secton sequentally descbes the achtectue of ou WSI system and ts man components. 2.1 System Achtectue Fgue 1 shows the achtectue of ou WSI system. The fst step s to pepocess the aw dataset fo featue extacton. Afte that, we extact bag of wods fom the sentence contanng a taget wod (featue extacton) and tansfom them nto hgh-dmenson vectos (featue vecto geneaton). Then, smlates of evey two vectos could be computed based on the featue vectos (smlaty measuement). the smlates of an nstance can be vewed as anothe vecto smlaty vecto. Both featue vectos and smlaty vectos can be seved as the nput fo clusteng algothms. Fnally, we pefom thee clusteng algothms, namely, k- means, HAC and spectal clusteng. Dataset Pepocess Fgue 1 Featue Extacton WSI Results 2.2 Featue Engneeng Vecto Geneaton Clusteng Smlaty Measuement Smlaty As Vecto Achtectue of ou Chnese WSI system In the task of WSI, the taget wods wth the topcal context ae fst tansfomed nto multdmensonal vectos wth vaous featues, and then applyng clusteng algothm to detect the elevance of each othe. Copus Pepocessng Fo each aw fle, we fst extact each sentence embedded n the tag <nstance>, ncludng the <head> and </head> tags whch ae used to dentfy the ambguous wod. Then, we put all the sentences elated to one taget wod nto a fle, odeed by the nstance IDs. The next step s wod segmentaton, whch segments each sentence nto a sequence of Chnese wods and s unque fo Chnese WSI. Hee, we use the softwae fom Hylanda 1 snce t s eady to use and consdeed an effcent wod segmentaton tool. Fnally, snce we etan the <head> tag n the sentence, the <head> and </head> tags ae usually sepaated afte wod segmentaton, thus we have to estoe them n ode to coectly locate the taget wod dung the pocess of featue extacton. Featue Extacton Afte wod segmentaton, fo a context of a patcula wod, we extact all the wods aound t n the sentence and buld a featue vecto based on a bag-of-wods Boolean model. Bag-ofwods means that we don t consde the ode of wods. Meanwhle, n the Boolean model, each wod n the context s used to geneate a featue. Ths featue wll be set to 1 f the wod appeas n the context o 0 f t does not. Fnally, we get a numbe of featue vectos, each of them coesponds to an nstance of the taget wod. One poblem wth ths featue-based method s that, snce the sze of wod set may be huge, the dmenson s also vey hgh, whch mght lead to data spasty poblem. Smlaty measuement One commonly used metc fo smlaty measuement s cosne smlaty, whch measues the angle between two featue vectos n a hghdmensonal space. Fomally, the cosne smlaty can be computed as follows: x y cos ne smlaty < xy, >= x y whee x, y ae two vectos n the vecto space and x, y ae the lengths of x, y espectvely. 1

3 Some clusteng algothms takes featue vectos as the nput and use cosne smlaty as the smlaty measuement between two vectos. Ths may lead to pefomance degadaton due to data spasty n featue vectos. To avod ths poblem, we compute the smlates of evey two vectos and geneate an N* N smlaty matx, whee N s the numbe of all the nstances contanng the ambguous wod. Geneally, N s usually much smalle than the dmenson sze and may allevate the data spasty poblem. Moeove, we vew evey ow of ths matx (.e., an odeed set of smlates of an nstance wth othe nstances) as anothe knd of featue vecto. In othe wods, each nstance tself s egaded as a featue, and the smlaty wth ths nstance eflects the weght of the featue. We call ths vecto smlaty vecto, whch we beleve wll moe popely epesent the nstance and acheve pomsng pefomance. 2.3 Clusteng Algothm Clusteng s a vey popula technque whch ams to patton a dataset nto such subgoups that samples n the same goup shae moe smlates than those fom dffeent goups. Ou system exploes vaous cluste algothms fo Chnese WSI, ncludng K-means, heachcal agglomeatve clusteng (HAC), and spectal clusteng (SC). K-means (KM) K-means s a vey popula method fo geneal clusteng used to automatcally patton a data set nto k goups. K-means woks by assgnng multdmensonal vectos to one of K clustes, whee K s gven as a po. The am of the algothm s to mnmze the vaance of the vectos assgned to each cluste. K-means poceeds by selectng k ntal cluste centes and then teatvely efnng them as follows: (1) Choose k cluste centes to concde wth k andomly-chosen pattens o k andomly defned ponts. (2) Assgn each patten to the closest cluste cente. (3) Recompute the cluste centes usng the cuent cluste membeshps. (4) If a convegence cteon s not met, go to step 2. Heachcal Agglomeatve Clusteng (HAC) Dffeent fom K-means, heachcal clusteng ceates a heachy of clustes whch can be epesented n a tee stuctue called a dendogam. The oot of the tee conssts of a sngle cluste contanng all objects, and the leaves coespond to ndvdual object. Typcally, heachcal agglomeatve clusteng (HAC) stats at the leaves and successvely meges two clustes togethe as long as they have the shotest dstance among all the pa-wse dstances between any two clustes. Gven a specfed numbe of clustes, the key poblem s to detemne whee to cut the heachcal tee nto clustes. In ths pape, we geneate the fnal flat cluste stuctues geedly by maxmzng the equal dstbuton of nstances among dffeent clustes. Spectal Clusteng (SC) Spectal clusteng efes to a class of technques whch ely on the egen-stuctue of a smlaty matx to patton ponts nto dsjont clustes wth ponts n the same cluste havng hgh smlaty and ponts n dffeent clustes havng low smlaty. Compaed to the tadtonal algothms such as K-means o sngle lnkage, spectal clusteng has many fundamental advantages. Results obtaned by spectal clusteng often outpefom the tadtonal appoaches, spectal clusteng s vey smple to mplement and can be solved effcently by standad lnea algeba methods. 3 System Evaluaton Ths secton epots the evaluaton dataset and system pefomance fo ou featue-based Chnese WSI system. 3.1 Dataset and Evaluaton Metcs We use the CLP2010 bake off task 4 sample dataset as ou development dataset. Thee ae 2500 examples contanng 50 taget wods and each wod has 50 sentences wth dffeent meanngs. The exact meanngs of the taget wods ae blnd, only the numbe of the meanngs s povded n the data. We compute the system pe-

4 fomance wth the sample dataset because t contans the answes of each canddate meanng. The test dataset povded by the CLP2010 s smla to the sample dataset. It contans 100 taget wods and 5000 nstances n total. Howeve, t doesn t povde the answes. The F-scoe measuement s the same as Zhao and Kayps (2005). Gven a patcula class L of sze n and a patcula cluste S of sze n, suppose n n the cluste S belong to L, then the F value of ths class and cluste s defned to be 2 R( L, S) P( L, S) FL (, S) = RL (, S) + PL (, S) R( L, S ) = n / n PL (, S) = n / n whee R( L, S ) s the ecall value and PL (, S) s the pecson value. The F-scoe of class L s the maxmum F value and F-scoe value follow: F scoe( L ) = max F( L, S ) S c n F scoe = F scoe( L ) = 1 n whee c s the total numbe of classes and n s the total sze. 3.2 Expement Results Table 1 epots the F-scoe of ou featue-based Chnese WSI fo dffeent featue sets wth vaous wndow szes usng K-means clusteng. Snce thee ae dffeent esults fo each un of K-means clusteng algothm, we pefom 20 tals and compute the aveage as the fnal esults. The columns denote dffeent wndow sze n, that s, the n wods befoe and afte the taget wod ae extacted as featues. Patculaly, the sze of nfnty ( ) means that all the wods n the sentence except the taget wod ae consdeed. The ows epesent vaous combnatons of featue sets and smlaty measuements, cuently, fou of whch ae consdeed as follows: F-All: all the wods ae consdeed as featues and fom them featue vectos ae constucted. F-Stop: the top 150 most fequently occung wods n the total wod bags of the copus ae egaded as stop wods and thus emoved fom the featue set. Featue vectos ae then fomed fom these wods. S-All: the featue set and the featue vecto ae the same as those of F-All, but nstead the smlaty vecto s used fo clusteng (c.f. Secton 2.2). S-Stop: the featue set and the featue vecto ae the same as those of F-Stop, but nstead the smlaty vecto s used fo clusteng. Featue/ Smlaty F-All F-Stop S-All S-Stop Table 1 Expemental esults fo dffeent featue sets wth dffeent wndow szes usng K-means clusteng Ths table shows that S-Stop acheves the best pefomance of n F-scoe. Ths suggests that fo K-means clusteng, Chnese WSI can beneft much fom emovng stop wods and adoptng smlaty vecto. It also shows that: As the wndow sze nceases, the pefomance s almost consstently enhanced. Ths ndcates that all the wods n the sentence moe o less help dsambguate the taget wod. Removng stop wods consstently mpoves the F-scoe fo both smlaty metcs. Ths means some hgh fequent wods do not help dscmnate the meanng of the taget wods, and futhe wok on featue selecton s thus encouaged. Smlaty vecto consstently outpefoms featue vecto fo stop-emoved featues, but not so fo all-wods featues. Ths may be due to the fact that, when the wndow sze s lmted, the nfluence of fequently occung stop wods s elatvely hgh, thus the smlaty vecto msepesent the context of the taget wod. On the contay, when stop wods ae emoved o the context s wde, the smlaty vecto can bette eflect the taget wod s context, leadng to bette pefomance. In ode to ntutvely explan why the smlaty vecto s moe dscmnatve than the featue vecto, we take two sentences contanng

5 the Chnese wod (hold, gasp) as an example (Fgue 2). These two sentences have few common wods, so clusteng va featue vectos puts them nto dffeent classes. Howeve, snce the smlates of these two featue vectos wth othe featue vectos ae much smla, clusteng va smlaty vectos goup them nto the same class. <lexelt tem=" " snum="4"> <nstance d="0012"> <head> </head> </nstance> <nstance d="0015"> <head> </head> </nstance> </lexelt> Fgue 2 An example fom the dataset Accodng to the concluson of the above expements, t s bette to nclude all the wods except stop wods n the sentence as the featues n the subsequent expement. Table 2 lsts the esults usng vaous clusteng algothms wth ths same expemental settng. It shows that the spectal clusteng algothm acheves the best pefomance of n F-scoe fo Chnese WSI usng the S-All setup. Addtonally, thee ae some nteestng fndngs: Although SC pefoms best, KM wth smlaty vectos acheves compaable esults of unts n F-scoe, slghtly lowe than that of SC. HAC pefoms wost among all clusteng algothms. An obsevaton eveals that ths algothm always goups the nstances nto hghly skewed clustes,.e., one o two clustes ae extemely lage whle othes usually have only one nstance n each cluste. It s supsng that S-All slghtly outpefoms F-All by only unts n F-scoe. The tuth s that, as dscussed n the fst expement, KM usng F-All doesn t consde nstance densty whle S-All does. On the contay, SC dentfes the egn-stuctue n the nstance space and thus aleady consdes the densty nfomaton, theefoe S-All wll not sgnfcantly mpove the pefomance. Featue/ KM HAC SC Smlaty F-All S-All Table 2 Expements esults usng dffeent clusteng algothms 3.3 Fnal System Pefomance Fo the CLP2010 task 4 test dataset whch contans 100 taget wods and 5000 nstances n total, we fst extact all the wods except stop wods n a sentence contanng the taget wod, then poduce the featue vecto fo each context and geneate the smlaty matx, fnally we pefom the spectal cluste algothm. Pobably because the dstbuton of the taget wod n the test dataset s dffeent fom that n the development dataset, the F-scoe of ou system on the test dataset s , about 0.05 unts lowe than that we got on the sample dataset. 4 Conclusons and Futue Wok In ou Chnese WSI system, we extact all the wods except stop wods n the sentence, constuct featue vectos and smlaty vectos, and apply the spectal clusteng algothm to ths poblem. Expemental esults show that ou smple and effcent system acheve a pomsng esult. Moeove, we also compae vaous clusteng algothms and smlaty metcs. We fnd that although the spectal clusteng algothm outpefoms othe clusteng algothms, the K- means clusteng wth smlaty vectos can also acheve compaable esults. Fo futue wok, we wll ncopoate moe lngustc featues, such as base chunkng, pase tee featue as well as dependency nfomaton nto ou system to futhe mpove the pefomance. Acknowledgement Ths eseach s suppoted by Poject , and unde the Natonal Natual Scence Foundaton of Chna. We would also lke to thank othe contbutos n the NLP lab at Soochow Unvesty.

6 Refeences Jan A, Muty M Flynn P. Data clusteng : A Revew [J]. ACM Computng Suveys,1999,31 (3) : F. Bach and M. Jodan Leanng spectal clusteng. In Poc. of NIPS-16. MIT Pess, Samuel Body and Mella Lapata Bayesan wod sense nducton. In Poceedngs of the 12th Confeence of the Euopean Chapte of the ACL (EACL 2009), pages Nell, D. B Fully Automatc Wod Sense Inducton by Semantc Clusteng. Cambdge Unvesty, Maste s Thess, M.Phl. n Compute Speech. Age, E. and Sooa, A Semeval-2007 task 02: Evaluatng wod sense nducton and dscmnaton systems. In Poceedngs of the 4th Intenatonal Wokshop on Semantc Evaluatons:7-12 Ioanns P. Klapafts and Suesh Manandha Wod sense nducton usng gaphs of collocatons. In Poceedngs of the 18th Euopean Confeence On Atfcal Intellgence (ECAI-2008), Patas, Geece, July. IOS Pess. Kannan, R., Vempala, S and Vetta, A On clustengs: Good, bad and spectal. J. ACM, 51(3), Renhad Rapp A pactcal soluton to the poblem of automatc wod sense nducton. Poceedngs of the ACL 2004 on Inteactve poste and demonstaton sessons, p.26-es, July 21-26, 2004, Bacelona, Span Bodag, S Wod sense nducton: Tplet-based clusteng and automatc evaluaton. In Poceedngs of the 11th Confeence of the Euopean Chapte of the Assocaton fo Computatonal Lngustcs (EACL, Tento, Italy) Yng Zhao, and Geoge Kayps Heachcal Clusteng Algothms fo Document Datasets. Data Mnng and Knowledge Dscovey, 10,