Win, Yuzana; Masada, Tomonari. Citation. Issue Date Right. this work in other works.

Transcription

1 NAOSITE: Nagasaki Universiy's Ac Tile Exploring Technical Phrase Frames f Auhor(s) Ciaion Win, Yuzana; Masada, Tomonari 05 IEEE 9h Inernaional Confer Neworking and Applicaions Worksho Issue Dae URL Righ hp://hdl.handle.ne/0069/ IEEE. Personal use of his m IEEE mus be obained for all oher including reprining/republishing promoional purposes, creaing new redisribuion o servers or liss, his work in oher works. This documen is downloaded hp://naosie.lb.nagasaki-u.ac.jp

2 Exploring Technical Phrase Frames from Research Paper Tiles Yuzana Win Graduae School of Engineering Nagasaki Universiy Nagasaki, Japan Tomonari Masada Graduae School of Engineering Nagasaki Universiy Nagasaki, Japan Absrac This paper proposes a mehod for exploring echnical phrase frames by exracing word n-grams ha mach our informaion needs and ineress from research paper iles. Technical phrase frames, he oucome of our mehod, are phrases wih wildcards ha may be subsiued for any echnical erm. Our mehod, firs of all, exracs word rigrams from research paper iles and consrucs a co-occurrence graph of he rigrams. Even by simply applying PageRank algorihm o he co-occurrence graph, we obain he rigrams ha can be regarded as echnical keyphrases a he higher ranks in erms of PageRank score. In conras, our mehod assigns weighs o he edges of he cooccurrence graph based on Jaccard similariy beween rigrams and hen apply weighed PageRank algorihm. Consequenly, we obain widely differen bu more ineresing resuls. While he opranked rigrams obained by unweighed PageRank have jus a self-conained meaning, hose obained by our mehod are echnical phrase frames, i.e., a word sequence ha forms a complee echnical phrase only afer puing a echnical word (or words) before or/and afer i. We claim ha our mehod is a useful ool for discovering imporan phraseological paerns, which can expand query keywords for improving informaion rerieval performance and can also work as candidae phrasings in echnical wriing o make our research papers aracive. Keywords phrase frames; word n-grams; Jaccard similariy; PageRank; keyphrase exracion I. INTRODUCTION Nowadays, i is a worhwhile ask o exrac valuable insighs from a large scale daase. However, he massive scale of he daase prevens users from accessing he whole daase efficienly and hus from obaining informaion relevan o heir need, because search resuls based on query words may no display he documens ha do no conain he inpued query words. Therefore, users are required o have knowledge and skill in finding appropriae query words or phrases for search. Especially in research paper rerieval, which is he applicaion we mainly consider here, i can be considerably difficul for users o find appropriae query words or phrases, because users may be unfamiliar wih arge research opics. Therefore, he imporan issue we need o address is o provide users wih help in finding appropriae query words or phrases. In his paper, we propose a mehod for exracing imporan phrases as word n-grams from research papers so ha he exraced phrases provide concise and precise summarizaions of he arge research opics and hus are useful in research informaion search. However, our mehod has an ousanding feaure. Tha is, our mehod can provide phrase frames [], no phrases in he ordinary sense, as is oucome. Phrase frames are phrases wih wildcards ha may be subsiued for any word. By subsiuing he wildcards of a phrase frame wih words, we can obain a complee phrase. Imporan phrase frames can be found based on heir frequency from an arbirary corpus []. However, our problem is o find echnically imporan phrase frames. Therefore, we propose a mehod ha addresses he problem as follows. Firsly, we exrac rigrams from a large se of research paper iles. We focus on rigrams, because longer n-grams make us suffer from daa sparseness problem, and unigrams and bigrams are oo shor o compose useful echnical phrase frames. The usage of paper iles can be jusified by he fac ha he paper ile works as a good descripion of he paper. Secondly, we consruc a co-occurrence graph of he exraced rigrams, where rigram pairs co-occurring in he same paper ile are conneced by edges. Here we may simply apply PageRank [4] algorihm o he cooccurrence graph. Then we can obain he op-ranked rigrams in erms of PageRank score and regard hem as echnically imporan. However, our mehod inroduces an addiional sep. Tha is, hirdly, we compue Jaccard similariy for every cooccurring pair of rigrams and use he similariy as he weigh of he corresponding graph edge. And fourhly, we apply a weighed version of PageRank o he modified co-occurrence graph and obain a lis of op-ranked rigrams. Our mehod consiss of he above four seps. Deails of each sep will be explained laer. The op-ranked rigrams obained by weighed PageRank have a special feaure when compared wih hose obained by unweighed PageRank. The former rigrams are more phraseological han he laer ones. For example, when we use a se of he iles of he paper presened a NLP conferences, unweighed PageRank gives higher ranks o rigrams like Word Sense Disambiguaion and Saisical Machine Translaion. I can be said ha unweighed PageRank ends o provide rigrams ha have a self-conained meaning. In conras, weighed PageRank gives higher ranks o rigrams like of Linguisic and and for Saisical Machine. We can expec ha a widely differen query expansion [3] will be achieved by using such

3 rigrams. For example, he rigram for Saisical Machine exends he query words like Model and Adapaion o he righ and also exends he query words like Learning and Translaion o he lef. In shor, our mehod explores rigrams ha can provide more flexible expansions of queries. Furher, our mehod may be used as a ool o recommend candidae phrasings for making our research papers aracive, because our mehods end o give phraseological rigrams. We will presen more examples laer. The remainder of his paper is organized as follows. Secion describes he previous work. Secion 3 explains he proposed mehod. Secion 4 presens he deails of he daase used in our experimen. Secion 5 conains he resuls of he experimen. Secion 6 concludes he paper wih discussion on fuure work. II. PREVIOUS WORK PageRank algorihm is commonly used in social nework analysis, informaion rerieval, keyphrase exracion, ec. We describe he previous work relaing o keyphrase exracion, because i is relevan o our problem. There are wo ypes of keyphrase exracion, i.e., supervised and unsupervised mehods. Hasan e al. [7] analyzed which ype of unsupervised approach shows he bes performance for keyphrase exracion. They used Tf-Idf, TexRank [9], SingleRank [], ExpandRank [] and Clusering-based approach [] o gain a beer performing keyphrase exracion. KEA [8] exraced keyphrases by using Tf-Idf as feaures of naïve Bayes classificaion. This mehod is a supervised one. According o he experimenal resuls given in hese wo conribuions, i is indicaed ha Tf-Idf is useful for keyphrase exracion. In conras, our mehod focuses on co-occurrence frequency based similariy beween phrases, no on weighing of each phrase. Mihalcea e al. [9] firs proposed TexRank, a graph-based ranking model for ex processing including wo innovaive unsupervised mehods of keyword and senence exracion. They used a graph o represen he co-occurrence in a window of a fixed number of words, ranked he words in a weighed PageRank manner, and idenified imporan connecion beween adjacen words in a ex. Mihalcea e al. [0] proposed a new approach for unsupervised knowledge-based word sense disambiguaion o exrac graph srucures from documens. The researchers combined informaion drawn from a semanic nework (WordNe) wih graph-based ranking algorihms (PageRank) o implemen summarizaion and word sense disambiguaion. They buil WordNe-based conceps graph o find he meaning of words and idenified hose synses of highes imporance. Gollapalli e al. [3] proposed a mehod o exrac keyphrases from research papers using ciaion neworks. They used he CieTexRank mehod of keyphrase exracion from research papers ha are embedded in ciaion neworks. They also used PageRank algorihm on word graphs consruced from arge papers and heir local neighborhood in a ciaion nework, where muliple neighborhoods accompanied wih differen weighs are incorporaed. Our mehod also applies graph-based ranking algorihms. However, i is no similar o he above hree conribuions. Firsly, we focus on rigrams and exrac rigrams from a large se of research paper iles. Secondly, we consruc a cooccurrence graph of he exraced rigrams and use Jaccard similariy as he weigh of he corresponding graph edge o apply a weighed version of PageRank. Our combinaion of Jaccard similariy wih rigrams is no considered by any of he PageRank-ype mehods described above. A. Word Trigrams III. METHOD Our mehod, firs of all, exracs rigrams from a large se of research paper iles as shown in Table I. We focus on rigrams, because longer n-grams make us suffer from daa sparseness problem. Especially, i may be difficul o obain dense cooccurrence daa for longer n-grams. On he oher hand, unigrams and bigrams are oo shor o compose useful echnical phrase frames. We expec ha he exraced n-grams will conain a leas one echnical word, because we consider query expansion in research informaion rerieval as an applicaion of our mehod. Therefore, shorer n-grams are also no appropriae. The usage of paper iles can be jusified by he fac ha he paper ile works as a good concise descripion of he conen of he paper. We use he naural language oolki for Pyhon (NLTK) o exrac rigrams. B. Co-occurrence Graph Secondly, we consruc a co-occurrence graph of he exraced rigrams. To consruc he co-occurrence graph, exraced word rigrams are used as nodes, and co-occurrence relaions of rigrams appearing in he same paper iles are used as edges as shown in Fig., which depics a small porion of he complee co-occurrence graph. This porion is obained from he wo paper iles given in Table I. While we may apply PageRank algorihm o his co-occurrence graph, our mehod has an addiional sep for exploring echnical phrase frames. C. Jaccard Similariy Thirdly, we compue Jaccard similariy for every cooccurring pair of rigrams and use he similariy as he weigh of he corresponding graph edge. Le (, ) denoe a pair of rigrams whose similariy is o be measured. Le S ( i) denoe he se of he paper iles ha conain he rigram i. We calculae he Jaccard similariy beween and as follows:

4 sim(, S( ) S( ) ) S( S( As a resul, if wo rigrams appear in he same se of paper iles hen hey are deemed idenical, and if hey have no common se of paper iles hen hey are regarded as compleely differen. By using he similariy as he weigh of he edges of he cooccurrence graph, we can apply a weighed version of PageRank algorihm. As Choi e al. discussed in heir survey paper [4], many binary similariy measures have been proposed. There are wo reasons why we choose Jaccard similariy. The one is is simpliciy. The oher is ha Jaccard similariy is defined wih no reference o negaive maches. In our case, negaive maches correspond o he paper iles where boh rigrams are absen. For example, Russell & Rao similariy is defined wih reference o negaive maches. However, for mos pairs of rigrams, he number of negaive maches is large and comparable wih he oal number of paper iles. Therefore, he inclusion of negaive maches makes he similariy evaluaion less reliable in our case. TABLE I. EXAMPLES OF PAPER TITLES AND THEIR WORD TRIGRAMS Recogniion of Linear Conex-Free Rewriing Sysems. [( Recogni, of, Linear ), ( of, Linear, Conex-Fre ), ( Linear, Conex-Fre, Rewri ), ( Conex-Fre, Rewri, Sysem )] Opimal Head-Driven Parsing Complexiy for Linear Conex-Free Rewriing Sysems. ) ) () occurring wih i, L ( j) is he number of rigrams co-occurring wih he rigram j, and N is he oal number of exraced rigrams. The parameer d is a damping facor which can be se beween 0 and. In our experimen, we se d o 0.85 [4]. We can obain he op-ranked rigrams in erms of his PageRank score. We apply a weighed version of PageRank o he modified co-occurrence graph. Le w ji denoe he weigh assigned o he edge connecing wo nodes, i and j. We se w ji o be equal o he corresponding Jaccard similariy. Then weighed PageRank algorihm gives he PageRank score of he rigram i as follows: - d PR ( i ) d N j M ( i ) k ji M ( j w ) jk PR( where M ( i) denoe he neighbors of he node i and d is he damping facor, which is se o 0.85 as in case of unweighed PageRank. k ε M( j ) w jk is he sum of he weighs assigned o each neighbor node in M ( j). Basically, if a node has many highscored neighbors, he node will receive a high score. However, our mehod combines Jaccard similariy and weighed PageRank algorihm. Therefore, if a node has many high-scored neighbors ha are similar o he node in erms of Jaccard similariy, hen he node will receive a high score. This special feaure makes he op-ranked rigrams given by our mehod widely differen from hose obained by unweighed PageRank. w j ) (3) [( Opim, Head-Driven, Pars ), ( Head-Driven, Pars, Complex ), ( Pars, Complex, for ), ( Complex, for, Linear ), ( for, Linear, Conex-Fre ), ( Linear, Conex-Fre, Rewri ), ( Conex-Fre, Rewri, Sysem )] D. PageRank Algorihm Our mehod applies PageRank algorihm o he cooccurrence graph of he exraced rigrams. We firs explain unweighed PageRank algorihm and hen weighed PageRank algorihm. The laer is used in our proposed mehod. Le PR ( j) denoe he PageRank score, assigned o he edge connecing he wo nodes, i.e., wo rigrams, i and j. PageRank algorihm provides he score of he rigram i as a saionary probabiliy saisfying he following equaion: - d PR ( j ) PR ( i ) d N M ( ) L ( j ) where,, N are rigrams, M ( i ) is he se of rigrams co- j i () Fig.. A small porion of he complee co-occurrence graph IV. DATA We esed our mehod by choosing 37,367 research paper iles from DBLP (Digial Bibliography & Library Projec) [5]. Each DBLP record includes a lis of auhors, ile, conference name or journal name, page numbers, ec. The DBLP daa consiss of over 00,000 records, each sored in an XML file of 3

5 size around.6gb. We chose academic conferences in he wo fields: Naural Language Processing (NLP) and Daa Managemen (DM). We seleced op conferences (venue) and only use he research paper iles associaed wih he seleced conferences as shown in Table II, where we also presen some examples of paper iles. We removed duplicae research paper iles. As a resul, he oal number of paper iles conained in NLP and DM daases are 8,034 and 9,333, respecively. In his paper, we apply Porer Semmer [6] in our pre-processing o sem words o heir roo forms. The oal number of rigrams exraced from NLP and DM daase are 43,863 and 63,66, respecively. TABLE II. DATASETS Fields Examples of Paper Tiles Venue TABLE III. Trigrams () EXAMPLES OF JACCARD SIMILARITIES FOR NLP Trigrams () Jaccard Sim. ( Web, Search, Engin ) ( a, Web, Search ).000 ( Approach, o, Naur ) ( o, Naur, Languag ) ( Sais, Naur, Languag ) ( for, Sais, Naur ) ( Evalu, of, an ) ( of, an, Auoma ) ( Use, a, Gene ) ( a, Gene, Algorihm ) ( Sysem, for, Gener ) ( in, a, Mulilingu ) ( Word, Segmen, and ) ( and, Par-of-Speech, Tag ) ( Acquir, Selec, Prefer ) ( of, Japanes, Tex ) NLP Web Tex Corpus for Naural Language Processing Nonparameric Word Segmenaion for Machine Translaion A Framework of NLP Based Informaion Tracking and Relaed Knowledge Organizing wih Topic Maps ACL, EACL, COLING, CICLing, NAACL, IJCNLP, EMNLP, NLDB, TSD TABLE IV. Trigrams () EXAMPLES OF JACCARD SIMILARITIES FOR DM Trigrams () Jaccard Sim. ( Graph, Pari, and ) ( Pari, and, Daa ).000 ( in, he, Presenc ) ( he, Presenc, of ) DM Performance Comparisons of Disribued Deadlock Deecion Algorihms Using Bayesian Nework Learning Algorihm o Discover Causal Relaions in Mulivariae Time Series Efficien Reasoning abou Daa Trees via Ineger Linear Programming SIGMOD, VLDB, PODS, SIGIR, WWW, KDD, ICDE, ISWC, CIDR, ICDM, ICDT, EDBT, SDM, CIKM, ER, ICIS, SSTD, WebDB, SSDBM, CAiSE, ECIS, PAKDD ( World, Wide, Web ) ( he, World, Wide ) ( A, Gener, Approach ) ( Gener, Approach, o ) ( A, Machin, Learn ) ( Machin, Learn, Approach ) ( Languag, Base, on ) ( Queri, Languag, Base ) ( SQLServer, 000, and ) ( he, Inerne, o ) ( Objec, Orien, Daabas ) ( for, Microsof, SQL ) V. RESULTS We apply weighed PageRank and unweighed PageRank o our daases. The experimenal resuls show ha weighed PageRank can explore rigrams ha can be regarded as echnical phrase frames. In conras, unweighed PageRank only explores rigrams as phrases wih a self-conained meaning. We clarify he difference of hese wo ypes of rigrams by displaying examples. Firsly, we presen examples of similariies calculaed beween a pair of rigrams o explain wha kind of rigram pairs are esimaed as similar o each oher. For he proposed mehod, we use Jaccard similariy given in Eq. (). Tables III and IV show examples of he obained Jaccard similariies for NLP daase and DM daase, respecively. A value of 0 indicaes ha wo rigrams co-occur in no paper iles, whereas a value of indicaes ha he se of he paper iles where he one rigram appears is compleely he same wih he se of he paper iles where he oher rigram appears. In he boom cells of Tables III and IV, we give wo rigrams co-occurring in no paper iles. We, of course, consider no such pairs. TABLE V. Unweighed PageRank ( Word, Sens, Disambigu ) ( Sais, Machin, Transla ) ( Name, Enii, Recogni ) ( Naur, Languag, Process ) ( Machin, Transla, Sysem ) ( for, Sais, Machin ) ( Spoken, Dialogu, Sysem ) ( Condi, Random, Field ) PAGERANK SCORES FOR NLP Weighed PageRank ( of, Linguis, and ) ( for, Sais, Machin ) ( in, Naur, Languag ) ( of, Texual, Enail ) ( of, Verb, in ) ( for, Word, Sens ) ( of, Telegraph, Messag ) ( for, Naur, Languag )

6 TABLE VI. PAGERANK SCORES FOR DM Unweighed PageRank Weighed PageRank ( Design, and, Develop ) ( Use, and, Perform ) ( Approach, for, Inform ) ( Opim, Approach, Inegr ) ( Exend, Eniy-Relaionship, Daa ) ( Theore, Approach, o ) ( Sorag, Scheme, for ) ( a, Digi, Librari ) ( Emerg, Paern, for ) ( of, Bibliograph, Reriev ) ( Avail, in, Pari ) ( in, Collabor, Filer ) ( Web, Servic, Composi ) ( of, he, Use ) ( Theore, Framework, and ) ( in, he, Disribu ) Machine, of he Use, ec. where funcional words appear as a grammaical componen. Especially, he firs word is a preposiion in many rigrams, hough such cases are rarely observed for unweighed PageRank. I can be said ha he opranked rigrams given by our mehod are more phraseological han hose given by unweighed PageRank. Below we discuss how he phraseological naure of our mehod may work in query expansion for informaion rerieval. Secondly, we provide examples of rigrams having large PageRank scores. For unweighed PageRank algorihm, we use Eq. () and calculae saionary probabiliies, where we uilize equal weighs for all ougoing edges on every node, because we use no rigram similariies. For weighed PageRank algorihm, we use Eq. (3) and calculae saionary probabiliies, where we uilize Jaccard similariies as weighs for ougoing edges on every node. Table V and VI summarize he resuls for NLP conference paper iles and DM conference paper iles, respecively. For example, ( Word, Sens, Disambigu ) and in he op cell of he lef column of Table V mean ha he saionary probabiliy is calculaed as for he rigram ( Word, Sens, Disambigu ) by using no Jaccard similariies. Furhermore, ( of, Linguis, and ) and in he op cell of he righ column of Table V mean ha he saionary probabiliy is calculaed as for he rigram ( of, Linguis, and ) by using Jaccard similariies. While we calculae saionary probabiliies for all possible combinaions of rigrams, only he eigh rigrams op-ranked in erms of PageRank score are displayed due o space limiaion. According o he resul presened in he lef columns of Tables V and VI, we can observe ha unweighed PageRank gives higher ranks o rigrams like Word Sense Disambiguaion, Exend Eniy-Relaionship Daa, Web Service Composiion, ec. by recovering he original form from he roo form of each word. I should be noed ha less funcional words like preposiions, aricles, conjuncives, ec. are observed. Tha is, unweighed PageRank ends o provide rigrams ha make sense wihou adding words before or/and afer hem. In conras, he resul presened in he righ columns of Tables V and VI shows ha weighed PageRank gives higher ranks o rigrams like of Linguisic and, for Saisical Fig.. Example of possible expansions using he echnical phrase frames in Naur Languag in NLP daase Fig. 3. Example of possible expansions using he echnical phrase frames for Sais Machin in NLP daase Fig. 4. Example of possible expansions using he echnical phrase frames of he Use in DM daase 5

7 For example, as we presen in Fig., he rigram ( in, Naur, Languag ), whose original form is in Naural Language, can exend o he righ he following echnical phrases: Negaive Imperaives, Objec-Specific Knowledge, Corpus-Based Lexical Choice, Theoreical/Technical Issues, ec., and o he lef he following echnical words and phrases: Insrucions, Processing, Generaion, Access o Daabases, ec. These examples are acually observable in exising research paper iles. In his manner, our mehod can be uilized for expanding echnical words or phrases, which may be used as a query word or phrase for search sysem, by a echnical phrase frame including funcional words. Fig. 3 gives anoher example. The rigram for Saisical Machine can exend Reordering Model, Alignmen Models, Disorion Models, Domain Adapaion, ec. o he righ and also Translaion, Query Processing, Learning Sysem, Translaion Using Enropy, ec. o he lef. These possible expansions are also acually obained from real research paper iles. In addiion, Fig. 4 provides he hird example. The rigram of he Use may exend Empirical Evaluaion, Case Sudy, Invesigaion, ec. o he righ. However, his rigram may no be used for expanding any words or phrases o he lef, because is las word is Use, a noun mainly followed by funcional words. While we can find he paper iles in which he rigram of he Use is followed by he phrases like of Daa Models, of Knowledge Managemen Sysems, of Qualiy Merics, ec., we may no use such phrases as a query. Therefore, his rigram may be used only for he expansion o he righ. In shor, rigrams explored by our mehod can provide a query expansion more flexible in he sense ha he expansion akes ino consideraion funcional words. The concep of echnical phrase frame implemened by our mehod will lead o a new query expansion scheme more oriened oward acual user needs and ineress. A search sysem using his ype of query expansion or query suggesion may excel as an environmen for exploring and sudying echnical opics and evolving rends in academic daa reposiories. Furher, our mehod may also be used as a ool for recommending candidae phrasings when we ry o make our research papers aracive, because our mehod is likely o give phraseological rigrams. VI. CONCLUSION In his paper, we exraced rigrams from a large se of research paper iles. And hen we consruced a co-occurrence graph of he exraced rigrams and used Jaccard similariy as he weigh of he graph edge o apply a weighed version of PageRank. In paricular, we observed ha weighed PageRank could explore rigrams as echnical phrase frames. In conras, unweighed PageRank only explored rigrams as phrases wih a self-conained meaning. Therefore, i can be concluded ha our mehod may achieve query expansion in a more phraseological manner. We have a plan o evaluae rigrams exraced by our mehod from a quaniaive viewpoin. The exraced rigrams can be used as feaures of documens along wih words. Therefore, for example by using naïve Bayes classifier, we can check in wha kind of siuaion he exraced rigrams improve he classificaion accuracy. We also have a plan o implemen query expansion using he exraced rigrams and evaluae he effeciveness of rigrams by measuring he qualiy of rerieved resuls. ACKNOWLEDGMENT This work has been suppored by he Gran-in-Aid for Scienific Research (C) No of he Minisry of Educaion, Science, Spors and Culure (MEXT) from 04 o 07. We are graeful for heir suppor. REFERENCES [] T. McEnery and A. Hardie. Corpus Linguisics: Mehod, Theory and Pracice. Cambridge Universiy Press, 0. [] hp://phrasesinenglish.org/explorep.hml [3] C.D. Manning and P. Raghavan. Inroducion o Informaion Rerieval. Cambridge Universiy Press, 008. [4] S. Brin and L. Page. The anaomy of a large-scale hyperexual Web search engine. Compu. New. ISDN Sys., Vol. 30 No. -7, pp. 07-7, 998. [5] hp:// [6] hp:// [7] K.S. Hasan and V. Ng. Conundrums in unsupervised keyphrase exracion: Making Sense of he Sae-of-he-Ar. In Proceedings of he 3rd Inernaional Conference on Compuaional Linguisics, pp , 00. [8] I.H. Wien, G.W. Payner, E.Frank, C. Guwin, and C.G. Nevill-Manning. KEA: Pracical auomaic keyphrase exracion. In Proceedings of he Fourh ACM Conference on Digial Libraries, pp , 999. [9] R. Mihalcea and P. Tarau. TexRank: Bringing order ino exs. In Proc. of he 004 Conference on Empirical Mehods in Naural Language Processing, pp , 004. [0] R. Mihalcea, P. Tarau and E. Figa. PageRank on semanic neworks,wih applicaion o word sense disambiguaion. In Proceedings of he 0h Inernaional Conference on Compuaional Linguisics, Aricle No. 6, 004. [] X. Wan and J. Xiao. Single documen keyphrase exracion using neighborhood knowledge. In Proceedings of he 3rd Naional Conference on Arificial Inelligence - Volume, pp , 008. [] F. Liu, D. Pennell, F. Liu and Y. Liu. Unsupervised approaches for auomaic keyword exracion using meeing ranscrips. In Proceedings of Human Language Technologies: The 009 Annual Conference of he Norh American Chaper of he Associaion for Compuaional Linguisics, pp , 009. [3] S.D. Gollapalli and C. Caragea. Exracing keyphrases from research papers using ciaion neworks. In Proceedings of he Tweny-Eighh AAAI Conference on Arificial Inelligence, pp , 04. [4] S. Choi, S. Cha, and C.C. Tapper. A survey of binary similariy and disance measures. J. Sys. Cybern. Inf., Vol. 8, No., pp , 00. 6