A Hybrid Re-ranking Method for Entity Recognition and Linking in Search Queries

Transcription

1 A Hybrd Re-rankng Method for Entty Recognton and Lnkng n Search Queres Gongbo Tang 1,2, Yutng Guo 2, Dong Yu 1,2(), and Endong Xun 1,2 1 Insttute of Bg Data and Language Educaton, Bejng Language and Culture Unversty, Bejng , Chna {tanggongbo,yudong_blcu,edxun}@126.com 2 College of Informaton Scence, Bejng Language and Culture Unversty, Bejng , Chna guoyutng_gyt@126.com Abstract. In ths paper, we construct an entty recognton and lnkng system usng Chnese Wkpeda and knowledge base. We utlze refned flter rules n entty recognton module, and then generate canddate enttes by search engne and attrbutes n Wkpeda artcle pages. In entty lnkng module, we propose a hybrd entty re-rankng method combned wth three features: textual and semantc match-degree, the smlarty between canddate entty and entty menton, entty frequency. Fnally, we get the lnkng results by the entty s fnal score. In the task of entty recognton and lnkng n search queres at NLPCC 2015, the Average-F1 value of ths method acheved 61.1% n 3849 test dataset, whch ranks second place n fourteen teams. 1 Introducton Search engne s the most common way to access nformaton, sometmes, people have to search satsfed answer n retreval result because of nformaton exploson. To return better retreval result, we need to deal wth the entty recognton and lnkng task to understand users ntents better. Search queres are nonstandard text, contanng wrong spellngs and abbrevaton, alas names, nck names of entty. For nstance, ºˆˆÅž¾¾, obvously, º ˆˆ refers to ºð and ¾¾ s nck name of. Meanwhle, search queres are really short, the longest query may be dozens of words. Compared wth the tradtonal entty recognton and lnkng works, the context can t provde enough features to dsambguate enttes. In ths paper, we show our system for entty recognton and lnkng n search queres. There are three man stages, entty recognton, canddate enttes generaton and enttes dsambguaton. We use the rules and entty base to flter the entty menton, then we generate canddate enttes by search engne and attrbutes of Wkpeda pages. Fnally, we utlze an entty re-rankng method to score the canddate enttes, whch s combned wth three features: textual and semantc match-degree, the smlarty between canddate entty and entty menton, entty frequency. Fgure 1 shows the framework of our system. Sprnger Internatonal Publshng Swtzerland 2015 J. L et al. (Eds.): NLPCC 2015, LNAI 9362, pp , DOI: / _57

2 A Hybrd Re-rankng Method for Entty Recognton and Lnkng n Search Queres Flter Rules Synonyms Expanson Match wth Entty Base Search Engne Combned wth Wkpeda 599 A Re-rankng method based on Textual and Semantc Matchng Degree Word Embeddng Smlarty and Entty Frequency Fg. 1. The framework of our system 2 Related Work There are two methods to recognze enttes, rule based and machne learnng based. Methods based on rules always recognze entty by the spellng rules, the parts of speech of enttes and dctonary. Mkheev et al. [1] can recognze 79.3% enttes by usng a general dctonary, as some enttes are ambguous wth common nouns, the remanng 20.7% enttes are not recognzed. In addton, methods based on machne learnng always need a lot of tagged dataset to learn a model. For nstance, Asahara and Matsumoto [2] utlze a support vector machne model to recognze enttes. To generate canddate enttes, Han and Zhao [3] use Google API to get the retreval result of short text, and select the enttes n the ttle of Wkpeda pages as canddate enttes. What s more, Meng et al. [4] utlze Badu to search entty menton+ and entty menton+ and also select the enttes n the ttle of Wkpeda pages as canddate enttes. The canddate entty dsambguaton need features, Dalton and Detz [5] utlze urban dctonary to expand query. Hoffart et al. [6] use the word frequency n Wkpeda to defne popularty feature. Blanco et al. [7] consder the dstrbutonal semantcs of query words and enttes, and tran word embeddng by word2vec. Shen et al. [8] make use of a SVM model, and gve a rank to canddate entty for each entty menton wth a lnear combnaton of four features: entty popularty, semantc assocatvty, semantc smlarty and global topcal coherence between mappng enttes. 维基百科 3 维基百科 Method To reduce the system s complexty, we used some refned rules to flter and classfy entty mentons. It s smple, but effectve! To deal wth varous names, we use the synonym dctonary to expand entty mentons. And we use the search engne combned wth Wkpeda to generate canddate enttes. Fnally, we utlze a re-rankng method to get the results. Fgure 2 shows the flow of our system.

3 600 G. Tang et al. Query Flter Rules Synonyms Expand Entty Base Knowledge Base Mentons Word Embeddng Search Engne combned Wkpeda Entty Frequency Canddate Enttes Re-rankng method Lnkng Result Fg. 2. The workflow of the system 3.1 Pre-processng We use the Yebol Chnese segmentaton system as word segmentaton tool. Snce we just need to lnk enttes n the knowledge base, so we extract and process the enttes, and then buld an entty base. Table 1 shows some examples. Synonyms expanson s a smple but effectve way to deal wth menton varaton ssue. We use the same method n Meng et al. [4] to buld and expand the synonym dctonary. We get Ø as alas of Ø, and Ñ as a translaton of ONE_PIECE etc. We select the entty descrpton as feature, and expand the descrpton wth Wkpeda and Badupeda. Table 1. Examples of entty processng Orgnal Å1 Å2 Å3 Å 4 Å_(ÆÏ ) Å Å Þų kl¾ ˳ Result Å Å kl¾ë³ 3.2 Named Entty Recognton Tradtonal entty recognton methods are based on machne learnng, whch needs plenty of tagged corpus to tran a model, and the types of entty n queres are varous,

4 A Hybrd Re-rankng Method for Entty Recognton and Lnkng n Search Queres 601 so t s unsutable to learn a model for entty recognton. In addton, the task only need to lnk enttes n knowledge base, so, to smplfy the process and mprove the effcency, we use the followng methods to recognze entty. 1. Desgn refned rules by sample dataset, and then flter the entty mentons 2. Expand entty mentons by synonymy dctonary 3. Match the mentons wth enttes n entty base: f the menton s completematchng, we name t dentfed entty menton M such as Å, otherwse, we name t undentfed entty menton M u, such as. 3.3 Canddate Entty Generaton Search queres and entty mentons are nformal text wth nose, for nstance, s one of msspellng format of, and there s no matched word n synonym dctonary, so we can t lnk the entty. Fortunately, search engne provde error correcton functon and can convert to before searchng. Therefore, we adopt the method based on search engne we used last year [4], we use search engne Badu, and treat entty menton + Ë as nput query. Fgure 3 shows the workflow of entty generaton. Mentons M Search Engne One correspondng entty n KB N Y 1. Identfed page Result 2. Redrect page Result 3. Dsambguaton page Canddate Enttes M u Search Engne Has Wkpeda Result Y Canddate Enttes Fg. 3. The workflow of entty generaton If an dentfed menton has one correspondng entty n the knowledge base, and the frst retreval result s a Wkpeda page, there are three condtons: 1. An dentfed page, the entty n the ttle s our lnked entty, lke к. 2. A redrect page, the entty n the ttle s also our lnked entty. 3. A dsambguaton page, all the enttes n the page are canddate enttes, for nstance, Angelababy and ÂØ are canddate enttes of menton ÂØ. If an dentfed menton has more than one correspondng enttes n the knowledge base, all the correspondng enttes are canddate enttes. For nstance, ÞÐ, ÞÐ_(1978Æ), ÞÐ_(1978ÆÏ ) etc. are canddate enttes of menton ÞÐ. For the undentfed entty menton, f the top 10 pages n the retreval result contans Chnese Wkpeda pages, the enttes n the ttles are canddate enttes, otherwse, we remove ths menton drectly.

5 602 G. Tang et al. 3.4 Canddate Entty Re-rankng The most mportant part of entty dsambguaton s to compute the dstance between canddate enttes feature and menton s feature. We compute the textual or semantc match-degree between notonal words n queres context and canddate enttes descrpton. Mkolov et al. [9] ndcated that the word embeddng can represent the word n syntax and semantcs. So, we can compute the smlarty between entty menton and canddate enttes by word embeddng, and we use cosne smlarty. The frequency of entty can tell us the pror probablty of the appearance of a canddate entty gven the entty menton. Therefore, we propose a re-rankng method combnng wth match-degree, word embeddng smlarty and entty frequency to re-rank the canddate enttes. The Match-degree Between Menton Context and Canddate Entty Descrpton For a set of canddate enttes, f there s a canddate entty s score s 1 n method 1, then every canddate entty s score s S match1, otherwse, the score s S match2. We assume c s the th canddate entty, m s the current menton. Method 1: If the notonal word n query appears n the descrpton of a canddate entty, for nstance, query: Þк, for entty menton ÞÐ º appears n the descrpton of ÞÐ_(1986ÆÏ ) t scores 1, otherwse 0. S match 1 c m 1 notonal word n query appear snthe descrpton of c 0 else (1) Method 2: Compute the smlarty between the search query and entty s descrpton. For the entty s descrpton, we use ts notonal words mean vector v d to represent; for the search query, we use ts notonal words mean vector v q to represent, and the match-degree s the cosne dstance between v d and v q. Therefore, the fnal score s formula 3. S match v Smatch 2 c m cos vq d (2) c m, S c m S c m match 1 match 1 Smatch 2 c m else (3) The Smlarty Between Canddate Entty Vector and Entty Menton Vector We set the canddate entty vector as V c, and the entty menton vector as V m, so, the smlarty score S sm wll be represented as follows,

6 A Hybrd Re-rankng Method for Entty Recognton and Lnkng n Search Queres 603 S c m sm 0, vc nullorv m null co svc, Vm else (4) Entty Frequency We use a Chnese Wkpeda corpus to count the frequency of entty, f a canddate entty c appears n c tmes n the corpus, and entty menton m appears n m tmes n the corpus, the frequency score S freq wll be: S freq c m 1 2* n c n n c n m (5) Hence, the fnal score of a canddate entty s just lke formula 6. 1, 2, 3 S c m S c m S c m S c m n (6) fnal match sm freq And α=1, β=0.7, γ=0.2, these parameters are decded by the sample dataset. We set threshold δ to 0.005, f the dfference between hghest score and the second hghest score s greater than δ, we choose the canddate entty wth hghest scores as the fnal lnkng entty. Otherwse, the second hghest canddate entty s also chosen as lnkng entty. Process n sequence untl the adjacent enttes dfference s greater than δ, or the lnkng entty number reaches 5. 4 Experments 4.1 Dataset Wkpeda s a hgh qualty encyclopeda contanng a wde coverage of named enttes, massve knowledge about notable named enttes, so t s ft for entty lnkng work. We download the newest Chnese Wkpeda from wk dump, and get 707 MB Chnese Wkpeda corpus after processng, whch s used to tran word embeddng and get entty statstcal dataset. We use CBOW model [10], [11] n word2vec to tran word embeddng, and set the dmenson to Experment Result and Analyss Our system score s 61.1% n Average-F1, whch ranks the second place n fourteen systems. Table 2 shows experment results. We can see that our system s a lttle lower than other systems n Lnk-Recall, but our Lnk-Precson s 6.5 percent hgher than the thrd system, and 16.2 percent gap wth the frst, t shows that our system s promsng n entty lnkng.

7 604 G. Tang et al. Table 2. Part of evaluaton results system Lnk-Precson Lnk-Recall Lnk-F1 Average-F1 NO Ours NO Short context, nonstandard text and varous entty representatons n search queres make ths task dffcult. In addton, entty lnkng task s based on entty recognton, and there wll be error n entty recognton nevtably, whch may cause error accumulaton and pull precson n entty lnkng down. We concluded that there are manly three types of error: 1. We adopt a coarse-graned method to recognze entty, for example, there are, ~ and three enttes n query ~, and we mssed. 2. Some words has no word embeddng because of the data sparsty problem. For nstance, the canddate enttes of menton are and _(ÆÏ ) n query Ã, whle these two canddate enttes do not appear n tranng dataset. 3. The re-rankng method s not precse enough. The lnkng result of menton Å n query ÅÌë Ð s Å_(ÆÏ ). However, for Å1, Å2 and Å_(ÆÏ ) etc. Ther score of matchdegree, smlarty and frequency are extraordnary close, so all of them are selected as lnkng entty. 5 Concluson Ths paper ntroduces our entty recognton and lnkng system n search queres. We use a rules based method to recognze entty, then generate canddate enttes by search engne and Wkpeda page attrbutes. Fnally, we utlze an entty re-rankng method to score the canddate enttes, and get the lnkng result by the entty score. The results of the experment shows that our method s effectve. In future work, we wll optmze the word segmentaton result, recognze entty n fne-graned and mprove the entty re-rankng method n entty lnkng. Acknowledgements. The research work s partally funded by the Natural Scence Foundaton of Chna (No , ), and the Fundamental Research Funds for the Central Unverstes n BLCU (No. 15YJ03006).

8 A Hybrd Re-rankng Method for Entty Recognton and Lnkng n Search Queres 605 References 1. Mkheev, A., Moens, M., Grover, C.: Named entty recognton wthout gazetteers. In: Proceedngs of the Eacl (1999) 2. Asahara, M., Matsumoto, Y.: Japanese named entty extracton wth redundant morphologcal analyss. In: Naacl Proceedngs of the Conference of the North Amercan Chapter of the Assocaton for Co. (2003) 3. Han, X., Zhao, J.: Nlpr-kbp n tac 2009 kbp track: A two-stage method to entty lnkng. In: Proceedngs of Test Analyss Conference (2009) 4. Meng, Z., Yu, D., Xun, E.: Chnese mcroblog entty lnkng system combnng wkpeda and search engne retreval results. In: Zong, C., Ne, J.-Y., Zhao, D., Feng, Y. (eds.) NLPCC CCIS, vol. 496, pp Sprnger, Hedelberg (2014) 5. Dalton, J., Detz, L.: UMass CIIR at TAC KBP 2013 entty lnkng: query expanson usng urban dctonary. In: Text Analyss Conference (2013) 6. Hoffart, J., Yosef, M.A., Bordno, I., et al.: Robust dsambguaton of named enttes n text. In: Proceedngs of the Conference on Emprcal Methods n Natural Language Processng. Assocaton for Computatonal Lngustcs, pp (2011) 7. Blanco, R., Ottavano, G., Mej, E.: Fast and space-effcent entty lnkng for queres. In: Proceedngs of the Eghth ACM Internatonal Conference on Web Search and Data Mnng, pp ACM (2015) 8. Shen, W., Wang, J., Luo, P., et al.: LINDEN: lnkng named enttes wth knowledge base va semantc knowledge. In: Proceedngs of the 21st Internatonal conference on World Wde Web. ACM (2012) 9. Mkolov, T., Sutskever, I., Chen, K., et al.: Dstrbuted representatons of words and phrases and ther compostonalty. In: Advances n Neural Informaton Processng Systems, pp (2013) 10. Mkolov, T., Yh, W., Zweg, G.: Lngustc regulartes n contnuous space word representatons. In: HLT-NAACL (2013) 11. Mkolov, T., Chen, K., Corrado, G., Dean, J.: Effcent estmaton of word representatons n vector space. In: ICLR Workshop (2013)