Improving the Quality of Information Retrieval Using Syntactic Analysis of Search Query

Size: px

Start display at page:

Download "Improving the Quality of Information Retrieval Using Syntactic Analysis of Search Query"

Gary Newton
5 years ago
Views:

1 Improvng the Qualty of Informaton Retreval Usng Syntactc Analyss of Search Query Nadezhda Yarushkna 1[ ], Aleksey Flppov 1[ ], and Mara Grgorcheva 1[ ] 1 Ulyanovsk State Techncal Unversty, Ulyanovsk, Sev. Venec str., 32, , Russa jng@ulstu.ru, al.flppov@ulstu.ru, gms4295@mal.ru Abstract. The paper descrbes the methods for lngustc analyss of search queres to mprove the qualty of nformaton retreval. The descrpton of the parse tree n the form of a structure contanng nformaton about two or more related words wth the ndcaton of ther parts of speech and locaton n the orgnal request s used to the translaton of search query n Elastcsearch Query DSL. Elastcsearch Query DSL has several dsadvantages: the user may not know the features of Elastcsearch Query DSL, words joned by the OR operator s usng by default n nformaton retreval. The usng of the OR operator unnecessarly ncreases the recall and reduces the precson of nformaton retreval. Takng nto account the features of Elastcsearch Query DSL and the nformaton needs of the user allow to mprove the qualty of nformaton retreval. Keywords: Informaton Retreval, Syntactc Analyss, Search Queres. 1 Introducton Informaton retreval s the process of searchng n an extensve collecton of data some sem-structured (unstructured) materal (document) that satsfes the nformaton needs of the user. Sem-structured data s data that does not have a clear, semantcally notceable and easly dstngushable structure. Sem-structured data s the opposte of structured data. The canoncal example of structured data s relatonal databases. Relatonal databases are typcally used by enterprses to store product regsters, employee personal data, etc [1,2,3]. The qualty of search n nformaton retreval systems s usually characterzed by two crtera: recall and precson. The total count of found documents determnes the recall. The rato between the determnes the precson found relevant documents and the total count of documents [2,3]. The qualty of the search s affected by both the characterstcs of the nformaton retreval system tself and the qualty of the search query. An deal search query can be formed by a user who knows well the doman area. Also, to form an deal query, the user needs to know the features of current nformaton retreval system and ther

2 2 nformaton retreval query language. Otherwse, the search result wll have the low precson or low recall values [1,2,3]. 2 Man problem To nformaton retreval, the user formulates a search query. The search query s a formalzed way of expressng the nformaton needs of nformaton retreval system users. Informaton retreval query language s used for the expresson of nformaton needs. The syntax of nformaton retreval query language vares from system to system. Modern nformaton retreval systems allow enterng a query n natural language n addton to an nformaton retreval query language [1]. Informaton retreval system fnds documents contanng the specfed keywords or words that are n any way related to the keywords based on the user search query. The result of the nformaton retreval system s a lst of documents, sorted by relevance [2,3]. In ths paper wll consder the work of the proposed method on the example of the nformaton retreval system Elastcsearch [4]. Elastcsearch provdes a full Query DSL [5]. The query strng s parsed nto a seres of terms and operators. A term can be a sngle word quck or brown or a phrase, surrounded by double quotes quck brown whch searches for all the words n the phrase, n the same order. Operators allow to customze the search: 1. By default, all terms are optonal, as long as one term matches. A search for foo bar baz wll fnd any document that contans one or more of foo or bar or baz. The preferred operators are + (ths term must be present) and - (ths term must not be present). All other terms are optonal. For example, ths query: quck brown +fox news states that: fox must be present; news must not be present; quck and brown are optonal ther presence ncreases the relevance. 2. Multple terms or clauses can be grouped together wth parentheses, to form subqueres: (quck OR brown) AND fox. Therefore, the Elastcsearch search algorthm has several dsadvantages: 1. The user may not know the features of Elastcsearch Query DSL. 2. Words joned by the OR operator are used by default n nformaton retreval. The usng of the OR operator unnecessarly ncreases the recall of nformaton retreval and reduces ts precson. It s necessary to develop a method of lngustc analyss and translaton of a search query nto a search query n a format of Elastcsearch Query DSL. The new format of

3 3 search query allows to take nto account the features of Elastcsearch Query DSL and mprove the qualty ndcators (precson and recall) of nformaton retreval. 3 The method of Lngustc Analyss of Search Query for Improvng Qualty of Informaton Retreval The prmary goal of the developed method of lngustc analyss and translaton of a search query nto a search query n a format of Elastcsearch Query DSL s the mprovement of nformaton retreval qualty. The man task s to select n the search query the groups of terms, unted by some semantcs. 3.1 The method of lngustc analyss and translaton of a search query The scheme of lngustc analyss of texts does not depend on the natural language tself. Regardless of the language of the source text, ts analyss goes through the same stages [6,7]: 1. Splttng the text nto separate sentences. 2. Splttng the text nto separate words. 3. Morphologcal analyss. 4. Syntactc analyss. 5. Semantc analyss. The frst two stages are the same for most natural languages. Language-specfc dfferences usually appear n the processng of word abbrevatons, and n the processng of punctuaton marks to determne the end of a sentence. The results of the syntactc analyss are used to select n the search query the groups of terms, unted by some semantcs. To dentfy a noun phrase from a query dentfcaton of mportant query terms s necessary. It s also necessary to defne the relatonshp between the terms of the query. A noun phrase or nomnal phrase s a phrase that has a noun (or ndefnte proper noun) as ts head or performs the same grammatcal functon as such a phrase. Noun phrases are ubqutous crosslngustcally, and they may be the most frequently occurrng phrase type. SyntaxNet as an mplementaton of the syntactc analyss process s used. SyntaxNet s a TensorFlow-based syntax defnton framework that uses a neural network. Currently, 40 languages ncludng Russan are supported. The source code of the already-traned Parsey McParseface neural network model that s sutable for parsng text s publshed For TensorFlow. The man task of SyntaxNet s to make computer systems able to read and understand human language. The precson of the model traned n the SnTagRus case s estmated at 87.44% for the LAS metrc (Label Attachment Score), 91.68% for the UAS metrc (Unlabeled Attachment Score) and determnes the part of speech and the grammatcal characterstcs of words wth an accuracy of 98.27% [8,9,10,11,12,13]. It s necessary to parse the search query to obtan a parse tree on the frst step of the algorthm. To obtan data about the search query structure, dependences between words and the types of these dependences the resultng parse tree wll be used.

4 4 The parse tree can be represented as the followng set: T t 1, t 2,, t k, (1) where k s a count of nodes n the parse tree; t s a node of the parse tree, can be descrbed as: t, w, m, c, 1, k, where s an ndex of the word n search query; w W, Ww 1, w2,, wk, W s a set of words of search query; m j M, M Noun, Propernoun, Verb, Adverb, Adjectve, Conjuncto n, Preposto n, Interjecton s a set of parts of speech for natural language; c s an ndex of the word n the search query, that depends to the -th word. Thus, the search query s converted nto a parse tree on the frst step of the algorthm. For each word n the search query the part of speech, ndex of ths word n the search query, and relatons wth other words of the search query are set. The descrpton of the parse tree n the form of a structure contanng nformaton about two or more related words wth the ndcaton of ther parts of speech and locaton n the orgnal request s used on the second step of the algorthm. In the process of analyss of the nput parse tree, the nodes that reflect the semantcs of ths query are selected. Search and translaton of selected nodes nto Elastcsearch Query DSL s executed usng a set of rules. The translaton process uses a set of rules. Rules are used to add specal characters from the Elastcsearch Query DSL to the words of the search query. Also, stop words are deleted from search query durng the translaton. The result of the algorthm s a new search query that takes nto account the semantcs of nformaton needs and features of the Elastcsearch Query DSL. Ths algorthm can be represented as the followng equaton: * F Query : T, R Q j, Query The nput parameters of the functon F are the parse tree of search query T (eq. * 1) and the set of rules R, and the result s a translated query Q. R n R R, R2,, s the set of rules for searchng elements n parse tree and 1 ther translaton n Elastcsearch Query DSL. Each rule can be represented as the followng expresson: * R p, t, t,, t Q, 1 2 m j where p s a rule prorty. The prorty of the rule determnes the order n whch the rules are appled to the search query. Thus, the prorty allows applyng the more complex rules to defne complex phrases frst. If the more complex rule dd not work, the rule wth a lower prorty s appled; t s k -th element of the rule that allows to selectng the node (nodes) of the parse k tree to process; m s a count of elements n the rule;

5 5 * Q j * Q, j 1, q s an element of translated query. Each element of a translated query contans the word or words of the orgnal search query escaped by a symbol from the set of Elastcsearch Query DSL operators. 3.2 Examples of rules for lngustc analyss and translaton of a search query The formal descrpton of the rule to search the noun phrase n the search query can be represented as follows: R 4,, w, Noun, c, 1, w, Adjectve,, noun_ phrase 2, w 2, Adjectve,,, d, w, Adjectve, ' ' w1 w2 wd w '', where d s a count of adjectves n the noun phrase. Extracton of noun phrase from the parse tree of the search query fnds one or more adjectve that subordnates to the current noun tree node. The result of ths rule s a noun phrase, escaped wth + at the begnnng and at the end. Each rule conssts of the two sdes: the left sde and the rght sde. The left sde s a template of a parse tree fragment. The rght sde s a strng template. The method extracts fragments of the parse tree, matched by the pattern n the left sde of the rule, and then converts them to the strngs escaped by the symbols of Elastcsearch Query DSL, fllng the strng template. The formal descrpton of the rule to search the related nouns n the search query can be represented as follows: R 3,, w, Noun, c, 1, w, Noun,, noun_ noun 2, w 2, Noun,,, d, w ' ' w1 w2 wd w '', where d s a count of nouns that related to -th noun. Extracton of related nouns from the parse tree of the search query fnds one or more noun that subordnates to the current noun tree node. The result of ths rule s a set of nouns, escaped wth + at the begnnng and at the end. The formal descrpton of the rule to search the proper noun that subordnates to the noun n the search query can be represented as follows: R 2,, w, Noun, c, 1, w, Proper noun,, noun_ proper_ noun 2, w 2, Proper noun,,, d, w , Noun, ' ' w w1 w2 wd ', where d s a count of proper nouns that related to -th noun. Extracton of proper nouns that subordnates to the noun from the parse tree of the search query fnd one or more proper noun that subordnates to the current noun tree node. The result of ths rule s a set of nouns, escaped wth + at the begnnng and at the end. The formal descrpton of the rule to search the proper noun n the search query can be represented as follows: 2 1, Proper noun,

6 6 R proper_ noun 2, w 2 1,, w, Proper noun, c, 1, w, Proper noun,,, d, w, Proper noun,,, Proper noun, ' ' w1 w2 wd w ', where d s a count of proper nouns that related to -th proper noun. Extracton of proper nouns from the parse tree of the search query fnds one or more proper noun that subordnates to the current proper noun tree node. The result of ths rule s a set of nouns, escaped wth + at the begnnng and at the end. The formal descrpton of the rule to search the sngle noun n the search query can be represented as follows: R,, w, Noun, c w sngle_noun As the man problem of the search subsystem of the SOM, users called the low qualty of the nformaton retreval. Ths rule has a lower prorty and s executed after the rules wth a hgher prorty have been skpped only. The rule allows fndng a node of parse tree wth a part of speech noun that s not assocated wth other nodes wth a part of speech noun, adjectve, or proper noun. The formal descrpton of the rule to search the verb n the search query can be represented as follows: R,, w, Verb, c w verb. 1 Ths rule has the hghest prorty and does not overlap wth any other rule. Ths rule allows fndng a node of the parse tree wth a part of speech verb. 4 Experments To test the method of lngustc analyss of search query proposed n ths study some experments were conducted. Fgure 1 shows the prmary form of applcaton for translaton of search query n Elastcsearch Query DSL. The control Orgnal Query s used to nput orgnal search query. The parse tree for orgnal search query wll be shown n table Search query formattng elements after pressng the button Prepare query. Each lne of ths table contans the name of the trggered rule and potental semantc group. Panel Translaton rules allows the user to select necessary rules for translaton the orgnal search query to query n a format of Elastcsearch DSL. The resultng search query wll be shown n control Search query n new format after pressng the button Format. In ths paper wll consder the work of the proposed method on the example of the exstng nformaton retreval subsystem of the system for opnon mnng n socal meda (SOM). The SOM conssts of the followng subsystems: 1. Subsystem for mportng data from external sources. Ths subsystem works mass meda stes. Mass meda loader retreves data from HTML pages of mass meda stes based on rules. The creaton of own rules for each mass meda s needed. The rule should contan a set of CSS-selectors. The ontology loader allows loadng ontologes n OWL or RDF format nto the data storage subsystem. Ontology s used for a descrpton of the features of the problem area.

7 2. The data storage subsystem provdes the representaton of nformaton extracted from mass meda n a unfed structure that s convenent for further processng.

7 7 2. The data storage subsystem provdes the representaton of nformaton extracted from mass meda n a unfed structure that s convenent for further processng. The data s stored n the context data sources, versons, etc. As database management systems are used: Elastcsearch for ndexng and retrevng data; MongoDB for storng data n JSON format; Neo4j for storng graphs of socal nteracton (socal graph) and ontology. 3. The data converter converts the data mported from mass meda nto an nternal SOM unfed structures. 4. The OWL/RDF-ontology translator translates ontology nto the graph representaton [14]. 5. The semantc analyss subsystem performs preprocessng of text resources. Also, ths subsystem performs statstcal and lngustc analyss of text resources. 6. The nformaton retreval subsystem fnds objects related to a specfc search query. In ths case, the search query can be semantcally extended usng an ontology. Fg. 1. The man form of applcaton for translaton of search query n Elastcsearch Query DSL. Posts and comments downloaded from mass meda stes ulpressa.ru, ulgrad.ru and mosaca.ru/ru/ul are used as the dataset. The collecton of documents s n Russan. The proposed rules are desgned for the Russan language. As the man problem of the search subsystem of SOM, users called the low qualty of nformaton retreval.

Fgure 2 shows the parse tree for search query: Стоимость проезда на общественном транспорте в Ульяновске.

8 8 Thus, the precson ndcator of nformaton retreval s used to assess the qualty of the proposed method. The recall and F-measure values are not used because the data storage subsystem of SOM contans the large count of documents. Fgure 2 shows the parse tree for search query: Стоимость проезда на общественном транспорте в Ульяновске. For example, wll use the query Amount of fare n publc transport n Ulyanovsk n Englsh, whch s close n meanng and structure to the query n Russan. The nodes of the parse tree are the words of the search query. Each node s assgned a part of speech. Fg. 2. The parse tree for the search query Amount of fare n publc transport n Ulyanovsk. After work of the algorthm, the sgnfcant elements were found n the parse tree (fg. 3). In the resultng tree (fg. 3), the nodes labeled as a rule wth that they were found. Fg. 3. The result tree for the search query Amount of fare n publc transport n Ulyanovsk. Thus, after lngustc analyss and translaton the resultng search query for search query Amount of fare n publc transport n Ulyanovsk s + Amount fare + publc transport +Ulyanovsk. The precson value s calculated usng the followng expresson: a P, (2) b where a s a count of relevant documents n the search result; b s a total count of documents n the search result.

9 9 For search query O Q Amount of fare n publc transport n Ulyanovsk the count of relevant documents n the search result of the nformaton retreval s 8. The total O P Q of the nformaton retreval count of documents s Thus, the precson for search query Amount of fare n publc transport n Ulyanovsk s (eq. 2): O 8 P Q 0, T For search query Q + Amount fare + publc transport +Ulyanovsk translated from search query Amount of fare n publc transport n Ulyanovsk usng the proposed method the count of relevant documents n the search result of the nformaton T P Q of the retreval s 8. The total count of documents s 8. Thus, the precson nformaton retreval for search query + Amount fare + publc transport +Ulyanovsk s (eq. 2): P T 8 Q 1. 8 Thus, the usng of proposed method mprove the precson of nformaton retreval because of reducng the count of documents n the search result. 5 Concluson Elastcsearch Query DSL has several dsadvantages: 1. The user may not know the features of Elastcsearch Query DSL. 2. Words joned by the OR operator are used by default n nformaton retreval. The usng of the OR operator unnecessarly ncreases the recall and reduces the precson of nformaton retreval. A method of lngustc analyss and translaton of search query n Elastcsearch Query DSL allows mprovng the precson of nformaton retreval. The search query s converted nto a parse tree on the frst step of the algorthm. For each word n the search query the part of speech, ndex of ths word n the search query, and relatons wth other words of the search query are set. The descrpton of the parse tree n the form of a structure contanng nformaton about two or more related words wth the ndcaton of ther parts of speech and locaton n the orgnal request s used on the second step of the algorthm. The result of the algorthm s a new search query that takes nto account the semantcs of nformaton needs and features of the Elastcsearch Query DSL. Accordng to the results of 20 computatonal experments, we can conclude: the use of the proposed method allows to ncrease the precson of nformaton retreval by an average of 18 tmes. The proposed algorthm can be used wth any nformaton retreval system because t preprocessed the orgnal search query, but does not change the system parameters or logc of nformaton retreval. However, the method requres adaptaton to the features of the nformaton retreval query language of the current nformaton retreval system.

10 10 6 Acknowledgments The study was supported by: the Mnstry of Educaton and Scence of the Russan Federaton n the framework of the project No /4.6. Development of methods and means for automaton of producton and technologcalpreparaton of aggregateassembly arcraft producton n the condtons of a mult-product producton program; the Russan Foundaton for Basc Research (Grants No and ). References 1. Voorhees, E. M.: Natural language processng and nformaton retreval. In: Informaton Extracton, pp Sprnger, Berln, Hedelberg (1999) 2. Mannng C., Raghavan P., Schütze H.: Introducton to Informaton Retreval. Cambrdge Unversty Press. (2008) 3. Mwa M. The Past, Present and Future of Informaton Retreval: Toward a User-Centered Orentaton. Accessed 20 Oct Elastcsearch. Accessed 20 Oct Elastcsearch Query DSL /query-dsl-query-strng-query.html. Accessed 20 Oct SRILM - The SRI Language Modelng Toolkt. Accessed 20 Oct Mannng C., Schutze H.: Foundatons of Statstcal Language processng. In: The MIT Press. (1999) 8. Sboev A.G., Gudovskkh D.V., Ivanov I., Moloshnkov I.A., Rybka R.B., Voronna I.: Research of a Deep Learnng Neural Network Effectveness for a Morphologcal Parser of Russan Language. Accessed 20 Oct 2018 (2017) 9. Chang J., Seefred J., Taylor S., Brandner A. SyntaxNet: Google s Open-sourced Syntactc Parser. SyntaxNet.pdf. Accessed 20 Oct Petrov S. Announcng SyntaxNet: The World s Most Accurate Parser Goes Open Source. Accessed 20 Oct Wess D., Petrov S. An Upgrade to SyntaxNet, New Models and a Parsng Competton. Accessed 20 Oct Albert C., Orr D., Petrov S. Meet Parsey s Cousns: Syntax for 40 languages, plus new SyntaxNet capabltes. Accessed 20 Oct Eberle K. Natural Language Tools - Test and Comparson. Accessed 20 Oct 2018

11 14. Yarushkna N., Flppov A., Moshkn V.: Development of the Unfed Technologcal Platform for Constructng the Doman Knowledge Base Through the Context Analyss. In: Creatvty n Intellgent Technologes and Data Scence, pp (2017) 11

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task Proceedngs of NTCIR-6 Workshop Meetng, May 15-18, 2007, Tokyo, Japan Term Weghtng Classfcaton System Usng the Ch-square Statstc for the Classfcaton Subtask at NTCIR-6 Patent Retreval Task Kotaro Hashmoto