Assessment of query reweighing, by rocchio method in farsi information retrieval

Transcription

1 Univesity of Wollongong Reseach Online Univesity of Wollongong in Dubai - Papes Univesity of Wollongong in Dubai 2008 Assessment of quey eweighing, by occhio metho in fasi infomation etieval F Sabooi Bu-Ali Sina Univesity H Bashii Bu-Ali Sina Univesity Faha Ooumchian Univesity of Wollongong in Dubai, fahao@uow.eu.au Publication Details Sabooi, F., Bashii, H. & Ooumchian, F. 2008, 'Assessment of quey eweighing, by occhio metho in fasi infomation etieval', Jounal of Infomation Science an Technology, vol. 6, no. 1, pp Reseach Online is the open access institutional epositoy fo the Univesity of Wollongong. Fo futhe infomation contact the UOW Libay: eseach-pubs@uow.eu.au

2 9 F. SABOORI, M.S. / H. BASHIRI, M.S. / F. OROUMCHIAN, Ph.D. Intenational Jounal of Infomation Science an Technology ASSESSMENT OF QUERY REWEIGHING, BY ROCCHIO METHOD IN FARSI INFORMATION RETRIEVAL F. Sabooi, M.S. [1] Depatment of Compute Engineeing Bu-Ali Sina Univesity, Hamean, I. R. of Ian fazaneh_2002_c@yahoo.com H. Bashii, M.S. [2] Depatment of Compute Engineeing Bu-Ali Sina Univesity, Hamean, I. R. of Ian Coesponing Autho: bashii@gmail.com F. Ooumchian, Ph.D. Assistant Pofesso Faculty of IT Wollongong Univesity in Dubai, Dubai, UAE fooumchian@acm.og ABSTRACT - Due to the lack of uses knowlege of the collections use by seach engines an in geneal etieval systems, uses can not expess thei infomation nee appopiately in queies. In othe wos, they o not have enough expeience to fomulate thei nees to fin elate ocuments. The iea of use s quey expansion aims to help uses to impove an coect the queies. In fact, etieval system, egaing the feeback it eceives fom use at the fist stage, moves the quey in set space to moe elate ocuments. Diffeent appoaches in infomation etieval systems have been use; howeve, thee has not been any assessment of efficacy of quey expansion in Fasi infomation etieval systems. In this pape, expansion basic moel of Rocchio, assesse as the pimay moel to etieve Fasi ocuments, has been pesente. As a matte of fact, the pupose of this stuy is to etemine the effect of a stana an basic moel on quey expansion to etieve Fasi ocuments, so that the eseaches can compae thei achievements of quey expansion with the finings of this pape which showe a staightfowa an positive effect on Fasi ocument etieval. Keywos: Infomation Retieval, Quey, Quey Expansion, Pecision, Feeback, Rocchio INTRODUCTION One of the factos of low pecision an ecall of infomation etieval systems, especially in the fist etieval phase, is to pose the quey inefficiently. Inee, uses ae novice in expessing thei nees pouce ue to lack of thei enough knowlege of collections use in infomation etieval systems. Fo uses spen a lot of time fining thei favoite ocuments elying a wie set of lexicons an collection tems, infomation Januay / June, 2008 Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1

3 Assessment of Quey Reweighing, by Rocchio Metho in Fasi Infomation Retieval 10 etieval systems can help to impove an coect thei quey[3, 4]. The commonest stategy to emake quey is the feeback of elationship with use [2]. In elationship feeback cycle, use faces last etieve ocuments an afte examining intouces some of them as elate ocuments. The main iea consists of choosing effective phases elate to ocuments intouce as elevant, so that a new quey is pesente base on the efficacy of wos an phases. Likewise, new quey can be iecte towa moe elevant ocuments in collection space an get fa fom non elevant ocuments. Hee, thee appoaches can be use [1, 3]: Expansion of quey (aing new wos of elate ocuments) Reweighing the wos (changing the wo weight base on use s feeback) Combination of two expansion an eweighing metho In this stuy, we have use wo eweighing metho to change the quey an quey expansion has consiee the change of wo weight. In the next stuies, we ae to use the metho of aing elate wos. ROCCHIO APPROACH Supposing that ocument collection space is patitione into two elevant an non elevant ocuments fo each inlet quey, we will be infome that elevant ocuments vecto has a iffeent behavio with non-elevant one. Theefoe, we must coect the quey vecto in oe to appoach the elevant ocuments set. This patition is ivie into two sets of C, C n in which C is all ocuments elate to use s quey out of the whole collection. The best quey vecto to ecognize elate ocuments out of non elate ones is ientifie by Fomula 1: q opt = 1 c c 1 N C c Optimum Quey Lack of access to the ocuments of C makes the optimal quey ifficult. Rocchio moel ties to make the patition moe exact, egaing use s fee back[1]. (1) q m = q + D D D n D n (2) Moifie Quey Regaing Use's Feeback D, D n ae elevant an non-elevant ocuments set espectively which ae intouce to use in the elementay etieve. Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1 Januay / June, 2008

4 11 F. SABOORI, M.S. / H. BASHIRI, M.S. / F. OROUMCHIAN, Ph.D. DESIGNING AND IMPLEMENTING OF FARSI QUERY REWEIGHING BASED ON ROCCHIO METHOD The poceue of eseach an access to the esults befoe an afte the use s quey eweighing has been followe accoing to activity iagam in Figue 1. To weigh the quey wos, we have use Ltu metho (Fomula 3). Lnu metho has been use to weigh ocument wos in the collection une assessment, explaine in Fomula 4. The eason to use Lnu. Ltu in weighing ocument an quey wos is the esults of the stuy one by Ooumchian an Mazha [5], whee vaious weighing methos fo Fasi texts have been assesse an eventually Lnu. Ltu has been intouce as the best metho to weigh the wos. N W n iq = ( ln( tf ) + 1.0) ln ( slope NUT ) + ( 1 slope) Pivot Ltu Weighting Metho in Quey (3) Whee: w iq : the weight of quey wos t f : tem fequency N: the whole numbe of collection ocuments n: the numbe of ocuments in which the wos have been use slope: the cuve slope of collection ocument space equal to 0.25 in ou eseach NUT: the numbe of unique tems Pivot: the paamete of collection equals to fo the aveage of ocument length. Januay / June, 2008 Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1

5 Assessment of Quey Reweighing, by Rocchio Metho in Fasi Infomation Retieval 12 Figue 1: Activity iagam. W i 1+ log( tf i ) 1+ log( aveagetf ) = ( slope NUT ) + (1 slope) Pivot Ltu Weighting Metho in Collection Tems (4) To etieve ocuments, vecto space moel has been employe accoing to Fomula 5: Sim q t w w ki k i k = 1 ( qi, ) = = (5) t t qi 2 2 wki wk k = 1 k = 1 Vecto Space Moel SYSTEM ASSESSMENT - COLLECTION UNDER EXPRIMENTS The use collection in this suvey was Qavanin collection, a unique collection in Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1 Januay / June, 2008

6 13 F. SABOORI, M.S. / H. BASHIRI, M.S. / F. OROUMCHIAN, Ph.D. assessing etieval moels of Fasi texts, consisting of the ninety yeas of law in Ian with a geat vaiety of lengths. Fo example, a small law with one o moe paagaphs has been pai attention to as a ocument simila to annual buget of county with all its subivisions. To examine all etieval moels, the aticles of Qavanin collection have been ivie into passages consisting of a section o subsection of a law, making some paagaphs. Thee wee fome sections in the collection [5]. A goup of lawyes have supevise all inputs an system opeatos. Foty one questions wee use to assess this system, fo each of which the fist 20 etieve ocuments wee uge. The ange of scoes was 0-4 by a human uge, in which zeo an 4 wee consiee as non elevance an full elevance to the questions espectively. The numbe of wos was 77889, which along with the ocuments fome a geat collection to assess Fasi infomation etieval systems. Fifteen questions out of 41 wee chosen to examine the efficacy of thei eweighing in ocument etieval. - ASSESSMENT BEFORE QUERY REWEIGHING The assessment was caie out base on the pecision calculation in five sets of etieve ocuments in cut-offs 5, 10, 15, 20, 25, 30, 35 an 40. The aveage pecision was calculate in each cut off point fo all queies an use to compae etieval befoe an afte quey eweighing. The collection ocuments anke 2, 3, 4 an those anke 0, 1 wee assesse as elevant an ielevant, espectively. The iagam of pecision aveage of etieval system is shown in Figue 2. This iagam is base on vecto space moel on 15 input questions to system in mentione cut off points an shows the calculate pecision befoe the quey eweighing, with a pecision inteval of 0.7 to Pecision Vecto Space_lnu.ltu Document Cut-off Figue 2: Diagam of aveage pecision befoe quey eweighing. - ASSESSMENT AFTER QUERY REWEIGHING In this step, all the 15 queies une expeiment wee coecte base on the Fomula 2 Januay / June, 2008 Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1

7 Assessment of Quey Reweighing, by Rocchio Metho in Fasi Infomation Retieval 14 an wo weight change accoing to feeback eceive fom system about elevant an non elevant ocuments. To enhance expeiment spee an pecision, feeback elivey was pefome automatically, that is the examine ocuments in the pimay etieval wee use to specify the elevance of ocument in the collection. The iagam of aveage pecision afte the application of change in quey an epesenting to system, has been shown in Figue Pecision Document Cut-off Vecto Space_Lnu.ltu Figue 3: Diagam of aveage pecision afte quey eweighing. - COMPARISION OF AVERAGE PRECISION BEFORE AND AFTER QUERY REWEIGHING The esults of aveage pecision in each cut off point befoe an afte quey eweighing have been pesente in Figue 4 an Table 1. Figue 4: Compaison of aveage pecision befoe an afte quey eweighing. Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1 Januay / June, 2008

8 15 F. SABOORI, M.S. / H. BASHIRI, M.S. / F. OROUMCHIAN, Ph.D. Table 1: Pecision in 4 cut-offs befoe quey eweighing. Cut-off Quey No Aveage Pecision CONCLUSION Following finings wee extacte fom this stuy: The aveage pecision fo each of eight cut-off points evaluate was highe afte expansion. Although this effect was low in the fist anks, Figue 4 shows the incease of pecision aveage afte quey expansion. Table 2: Pecision in 4 cut-offs befoe quey eweighing. Cut-off Quey No Aveage Pecision The incease of aveage pecision was obseve fom cut off point of 5 onwa an appoaching the final cut off point was concuent with the incease effect of quey Januay / June, 2008 Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1

9 Assessment of Quey Reweighing, by Rocchio Metho in Fasi Infomation Retieval 16 expansion on pecision aveage. Refeing Tables 1 an 2 an examining queies 11 an 12, we unestan that pecision in each of eight cut off points is equal to 1. It seems logical that these queies must not e-ente in the system ue to thei high pecision an Rocchio is use in the state to etieve moe elevant ocuments. Table 2 shows that quey expansion in t have any effect on etieval pecision fo queies 11 an 20. Quey expansion coul help to incease the aveage pecision to , which can be obseve fom the compaison in Figue 4 an Tables 1 an 2. ENDNOTES 1. Stuent of Softwae Engineeing 2. Instucto of Compute Engineeing REFERENCES [1] Baeza-Yates, R. an Ribeio-Neto, B., Moen Infomation Retieval. ACM Pess, [2] Bai, J. et al., Quey Expansion Using TERM Relationships in Language Moels fo Infomation Retieval. Poceeings of the 14th ACM Intenational Confeence on Infomation an Knowlege Management, [3] Capineto, C. et al., An Infomation-Theoetic Appoach to Automatic Quey Expansion. ACM Tansactions on Infomation Systems, [4] Mita, M. et al., Impoving Automatic Quey Expansion. Poceeings of the 21st Annual Intenational ACM SIGIR Confeence on Reseach an Development in Infomation Retieval, [5] Ooumchian, F. an Mazha, F., An Evaluation of Retieval Pefomance Using Fasi Text. Fasi Euasia Confeence on Avances in Infomation an Communication Technology, Tehan, Ian, Intenational Jounal of Infomation Science & Technology, Volume 6, Numbe 1 Januay / June, 2008