A Software Tool to Teach the Performance of Fuzzy IR Systems based on Weighted Queries

Transcription

1 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres Enrque Herrera-Vema 1, Sergo Alonso 1, Francsco J. Cabrerzo 1, Antono G. Lopez-Herrera 2, Carlos Porcel 3 1 Dept. of Computer Scence an A.I., Faculty of Lbrary Scence, Unversty of Granaa, Granaa, Span vema, salonso, fcabrerzo@ecsa.ugr.es 2 Dept. of Computer Scence, Faculty of Computer Scence, Unversty of Jaen, Jaen, Span aglopez@uaen.es 3 Dept. of Computng an Numercal Analyss, Faculty of Computer Scence, Unversty of Coroba, Coroba, Span carlos.porcel@uco.es Ths paper escrbes a software tool that allows us to teach stuents the prncples an concepts of Fuzzy Informaton Retreval Systems base on weghte queres. Ths tool s use n the course Informaton Retreval Systems Base on Artfcal Intellgence at the Faculty of Lbrary an Informaton Scence at the Unversty of Granaa. Wth ths teachng tool stuents learn the management of the fuzzy weghte query languages whch coul be use n any conventonal Web search engne to mprove the representaton of user nformaton nees. Keywors: teachng, eucaton, fuzzy weghte queres, fuzzy connectves, fuzzy nformaton retreval. 1. INTRODUCTION Due to the growth of e-busness, the Web has become a crtcal part of many real-worl systems. Thus, t s ngreasngly mportant that nformaton technology professonals an stuents be profcent an knowlegeable n varous Web technologes lke [1] Web mnng, query processng, Informaton Retreval (IR) moels, search engnes, meta-search engnes, recommener systems, nformaton flterng, Web qualty evaluaton, etc., whch are also evolvng at a rap rate, makng t crtcal to keep up-to-ate wth them [6]. At the Faculty of Lbrary an Informaton Scence at the Unversty of Granaa there are fferent egree courses that aress these evolvng nees. In partcular, there exsts a egree course calle "Informaton Retreval Systems base on Artfcal Intellgence" whch eals wth the stuy an analyss of artfcal ntellgence tools apple n the esgn of Informaton Retreval Systems (IRSs). The key goals of ths course are to learn the founatons of fuzzy tools an genetc algorthms an ts applcaton n the esgn of IRSs. As t s known, both are mportant soft computng tools [2,26] an are beng satsfactorly apple n the evelopment of the Web access technologes [3,7,8,9,13,14,19]. Fuzzy IRSs (FIRSs) are those IRSs that use the potental of the fuzzy tools to mprove the retreval actvtes [9,14]. In our egree course we focus on fuzzy IR moels that use fuzzy weghte queres to mprove the representaton of user nformaton nees an fuzzy connectves to process such queres. We use hans-on-keyboar classroom exercses for teachng an practsng the use of fuzzy weghte queres an fuzzy connectves. However, n our teachng experence we observe that ths s not enough to show learners the searchng sklls of FIRSs. IR nstructon s an obvous applcaton for computer-supporte learnng systems. The avantage of usng computer-supporte learnng systems s that the learner gets a realstc feelng of the partcular IRS use an he/she learns typcal operatons of IRSs [11]. To o that, t s possble to use real worl search engnes lke Google, Altavsta, Lycos or to bul a-hoc tranng IRSs [5,6,10,11,18]. There are very few tranng IRSs [11] an, partcularly, a fuzzy IR tranng system oes not exst. As t s ponte out n [11], exstng tranng IR systems present several shortcomngs, e.g., they o not gve feeback about the performance or success of user queres, t s not possble to observe how a user query s evaluate, an t s not possble to compare the performance of fferent types of user queres an fferent evaluaton proceures of user queres. In ths paper we ntrouce a software tool, whch s ust beng use as frst tme. Ths software gves stuents a chance to acqure the complex sklls that prove those FIRSs base on weghte queres. Ths s a Web-base

2 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres computer-supporte learnng applcaton whose goal s to prove a envronment for emonstratng the performance of fuzzy queres an ther evaluaton usng fferent fuzzy connectves. It offers stuents the opportunty to see an compare the acheve results of fferent weghte queres. User can choose fferent semantcs (threshol, relatve mportance, eal mportance, [12,17] to formulate weghte queres, fferent fuzzy connectves to evaluate these queres (maxmum, mnmum, OWA operators, Inuce OWA operators) [23,24], an fferent expresson omans (numercal or lngustc one) [12,16] to assess weghts assocate wth queres. Furthermore, several stanar test collecton (ADI, CISI, CRANSFIELD, etc) can be use. Fnally ths tool presents vsualzaton tools to show better evaluaton processes of queres. The paper s structure as follows: n Secton 2 we revew the components of the fuzzy IR moels that we want to teach Secton 3 escrbes the structure an performance of our software tool an shows some example of ts use. In the last secton, we scuss some lessons learne from our experence an suggest some possble uses an mprovements of our computerze system. 2. CHARACTERISTICS OF FUZZY IR MODELS TO TEACH The set of fuzzy IR moels that we have mplemente n our software applcaton presents the followng components: 1. Database. We assume a atabase bult lke n an usual IRS [1,20] an therefore where the IRS-user nteracton s unnecessary because t s bult automatcally. Then, the atabase stores the fnte set of ocuments D =,, }, the fnte set of nex terms T = t,, t }, an the representaton R of each ocument { 1 m { 1 l characterze by a numerc nexng functon : D T [ 0,1] F such that R = F = (, t ) / t 1. Usng the numerc values n [0,1] F can weght nex terms accorng to ther sgnfcance n escrbng the content of a ocument n orer to mprove the retreval of ocuments. Test stanar collectons have been nexe usng a tf f scheme. 2. Query system The mplemente fuzzy IR moels prove a query system wth fuzzy weghte Boolean query languages to express user nformaton nees. Wth these languages each query s expresse as a combnaton of the weghte nex terms whch are connecte by the logcal operators AND ( ), OR ( ), an NOT ( ). The weghts can be numercal values n [0,1] or lngustc values taken from a label set S whch s efne usng the concept of fuzzy lngustc varable [25]. By assgnng weghts n queres, users specfy restrctons on the ocuments that the IRS has to satsfy n the retreval actvty. In the lterature we fn that a user can mpose four kns of restrctons on ocuments to be retreve whch are assocate to four fferent semantc nterpretatons [12,17]: a. Importance semantcs. Ths semantcs efnes query weghts as measures of the relatve mportance of each term for the query wth respect to the others n the query. By assocatng relatve mportance weghts to terms n a query, the user s askng to see all ocuments whose content represents the concept that s more assocate wth the most mportant terms than wth the less mportant ones. In practce, ths means that the user requres that the computaton of the relevance egree of a ocument be omnate by the more heavly weghte terms. b. Threshol semantcs. Ths semantcs efnes query weghts as satsfacton requrements for each term of query to be consere when matchng ocument representatons to the query. By assocatng threshol weghts wth terms n a query, the user s askng to see all the ocuments suffcently about the topcs represente by such terms. In practce, ths means that the user wll rewar a ocument whose nex term weghts F excee the establshe threshols wth a hgh relevance egree, but allowng some small partal cret for a ocument whose F values are lower than the threshols. c. Perfecton semantcs. Ths semantcs efnes query weghts as escrptons of eal or perfect ocuments esre by the user. By assocatng weghts wth terms n a query, the user s askng to see all the ocuments whose content satsfes or s more or less close to hs eal nformaton nees as represente n the weghte query. In practce, ths means that the user wll rewar a ocument whose nex term weghts are equal to or at least near to term weghts for a query wth the hghest relevance egrees.. Quanttatve semantcs. Ths semantcs efnes query weghts to express crtera that affect the quantty of the ocuments to be retreve,.e., constrants to be satsfe by the number of ocuments to be retreve. l

3 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres Formally, n [4,22] a fuzzy weghte Boolean query wth one semantcs was efne as any legtmate Boolean expresson whose atomc components are pars t, c > belongng to the set, H c 0,1 or S s a < value that qualfes the mportance that the term the legtmate queres s efne by the followng syntactc rules: ) q =< t, c > T H q Q ) ) v) q, p Q q p Q q, p Q q p Q q Q q Q q are only those obtane by applyng rules -v, nclusve. v) all legtmate queres Q 3. Evaluaton proceure T where [ ] t must have n the esre ocuments. Accorngly, the set Q of To evaluate these weghte Boolean queres we use a constructve bottom-up process base on the crteron of separablty (one of the most mportant propertes of the wsh lst) [22]. Ths process nclues two steps: - Frstly, the ocuments are evaluate accorng to ther relevance only to atoms of the query. In ths step, a partal relevance egree s assgne to each ocument wth respect to each atom n the query. - Seconly, the ocuments are evaluate accorng to ther relevance to Boolean combnatons of atomc components (ther partal relevance egree), an so on, workng n a bottom-up fashon untl the whole query s processe. In ths step, a total relevance egree s assgne to each ocument wth respect to the whole query. We represent the evaluaton proceure usng an evaluaton functon query, E obtans the relevance egree RSV of any E :. Depenng on the kn of Q D H D accorng to the followng rules: E ( < t, c >, ) = g(( F(, t ), c ) = RSV where g s a matchng functon that epens on both expresson 1.- oman an semantc nterpretaton assocate to c. 2.- E ( q p, ) = E( q, ) FUZZCONN AND E( p, ), where FUZZCONN AND s a fuzzy connectve that moels a combnaton behavour of values smlar to a t-norm. 3.- E ( q p, ) = E( q, ) FUZZCONN OR E( p, ), where FUZZCONN OR s a fuzzy connectve that moels a combnaton behavour of values smlar to a t-conorm. 4.- E ( q), ) = Neg( E( ( q), )), where Neg s a complement operator of fuzzy sets. ( We shoul pont out that the fuzzy connectves that apply n the evaluaton proceure are manly the famly of connectves of type OWA or IOWA [23,24] whose behavour can be controlle through an orness measure [23]. 3. A SOFTWARE TOOL TO TEACH FUZZY IR SYSTEMS BASED ON WEIGHTED QUERIES We have evelope ths software tool at the Faculty of Informaton an Lbrary Scence at the Unversty of Granaa as a useful fuzzy weghte query analyss tool (see The goal of ths software applcaton s to prove an envronment for emonstratng stuents the performance of fuzzy weghte queres an n such a way to a n ther learnng. Ths software tool s a Web-base applcaton that s mplemente n Java language. It s compose of three man moules: ) efnton moule of test collecton, ) formulaton moule of weghte queres, an ) a vsual executon moule of queres. 3.1 Defnton Moule of Test Collecton An expermental test collecton conssts of a atabase, a collecton of queres an relevance assessments ncatng whch ocuments are relevant n respect to a gven query [21]. Usually, the performance of a system s measure by means of the precson an recall acheve across the whole set of queres. As n [11,15] our goal s to encourage the analyss of the nvuals queres, an therefore, we only nee an nstructonal test collecton. However, the tool proves some stanar test collecton (ADI, CISI, CRANFIELD). We have ece to gve the capablty stuents to bul ther own test collectons (see Fgure 1),.e., toy test collectons, to analyze the performance of the fferent fuzzy weghte queres. In the efnton of test collecton they can establsh partcular queres an whch ocuments of the atabase match the relevance requrements of these queres.

4 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres FIGURE 1: Defnng a test collecton. 3.2 Formulaton Moule of Weghte Queres We have esgne a formulaton moule of weghte queres that allows stuents to efne ther weghte queres (see Fgure 2). To efne a weghte query they have to choose: search terms, Boolean connectves (An, Or, an Not), query structure, expresson oman of weghts (numercal or lngustc), semantc nterpretaton, an values of weghts. FIGURE 2: Defnng a weghte query. 3.3 Vsual Executon Moule of Queres We have mplemente an executon moule that allows measurng an vsualzng the performance of any query execute. Before to execute a weghte query a stuent has to choose fuzzy connectves that wll be assocate wth the Boolean connectves n the evaluaton proceure. Ths s one choosng a level of orness [23]. Ths moule generates performance feeback for the stuents by means of vsual tools. Ths feeback can be gven n both ways by showng nternal aspects of evaluatons of weghte queres, e.g. evaluaton trees, (see Fgures 3 an 4) or analyss of search results usng tratonal precson/recall curves. Furthermore, ths moule allows the comparson of fferent fferent weghte queres n the evaluaton process.

5 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres FIGURE 3: Evaluaton tree for all retreve ocuments. FIGURE 4: Evaluaton tree for a selecte ocument.

6 A Software Tool to Teach the Performance of Fuzzy IR Systems base on Weghte Queres 4. CONCLUSIONS In ths contrbuton we have presente a software tool to teach the use of fuzzy weghte queres n IR actvty. Our experence reveals that the use of ths tool enhances stuents learnng on fuzzy IR systems. Currently, we are workng to evelop a better set of tools for bulng fuzzy search engnes, that ntegrate sperng, nexng, searchng, an storage, to be apple n real stuatons. In such a way, we want to stmulate stuents creatvty an nnovaton an to mprove ther learnng. Atonally, we are esgnng a survey wth whch stuents can express ther experences an suggestons to mprove ths software tool. REFERENCES [1] Baeza-Yates R an Rbero-Neto B. (1999) Moern Informaton Retreval. Ason-Wesley. [2] Bonssone P P. (1997) Soft computng: the convergence of emergng reasonng technologes. Soft Computng, 1(1), pp [3] Borogna G an Pas G. (1993) Specal ssue: Management of mprecson an uncertanty. Journal of the Amercan Socety for Informaton Scence, 49(3), pp [4] Borogna G an Pas G. (1993) A Fuzzy Lngustc Approach Generalzng Boolean Informaton Retreval: A Moel an Its Evaluaton. Journal of the Amercan Socety for Informaton Scence, 44, pp [5] Caruso E. (1981) Computer as to learnng onlne retreval. Annual Revew of Informaton Scence an Technology, pp [6] Chau M Huang Z an Chen H. (2003) Teachng key topcs n computer scence an nformaton systems through a Web search engne proect. ACM Journal of Eucatonal Resources n Computng, 3(3), pp [7] Coron O an Herrera-Vema E. (2003) Specal ssue on soft computng applcatons to ntellgent nformaton retreval on the nternet. Internatonal Journal of Approxmate Reasonng, 34(2-3). [8] Crestan F an Pas G. (2003) Hanng vagueness, subectvty, an mprecson n nformaton access: an ntrouctonto to the specal ssue. Informaton Processng & Management, 39, pp [9] Crestan F an Pas G (Es). (2000) Soft Computng n Informaton Retreval: technques an applcatons. Stues n Fuzzness an Soft Computng Seres, vol. 50, Physca-Verlag. [10] Grffth J C an Norton N P. (1981) A computer assste nstructon programs for en users on an automate nformaton retreval systems. Proc. of the Secon Natonal Onlne Meetng, New York [11] Halttunen K an Sormunen E. (2000) Learnng nformaton retreval through an eucatonal game. Is Gamng suffcent for learnng?. Eucaton for Informaton, 18(4), pp [12] Herrera-Vema E. (2001) Moelng the retreval process for an nformaton retreval system usng an ornal fuzzy lngustc approach. Journal of the Amercan Socety for Informaton Scence an Technology, 52(6), pp [13] Herrera-Vema E an Pas G. (2006) Soft Approaches to Informaton Retreval an Informaton Access on the Web: An Introucton to the Specal Issue. Specal Issue on Soft Approaches to Informaton Retreval an Informaton Access on the Web. Journal of the Amercan Socety for Informaton Scence an Technology, 57(4), pp [14] Herrera-Vema E, Pas G an Crestan F. (2006) Soft Computng n Web Informaton Retreval: Moels an Applcatons. Stues n Fuzzness an Soft Computng Seres, vol. 197, Physca-Verlag. [15] Hull D. (1996) Stemmng algorthms: a case stuy for etale evaluaton. Journal of the Amercan Socety for Informaton Scence, 52(6), pp [16] Kraft D H an Buell D A. (1983) Fuzzy sets an generalze Boolean retreval systems. Internatonal Journal of Man-Machne Stues, 19, pp [17] Kraft D H, Borogna G an Pas G. (1994) An Extene Fuzzy Lngustc Approach to Generalze Boolean Informaton Retreval. Informaton Scences, 2, pp [18] Markey K an Atherton P. (1978) ONTAP onlne tranng an practce manual for ERIC ata base searchers. Syracuse, New York: ERIC Clearnghouse on Informaton Sources, Syracuse Unversty. [19] Nkravesh M, Loa V an Azvne B. (2002) Specal ssue on Fuzzy logc an the Internet (FLINT): Internet, worl we web an search engnes. Soft Computng, 6(5), pp [20] Salton G an McGll M H. (1984) {\em Introucton to moern nformaton retreval}. New York: McGraw-Hll. [21] Sparck Jones K an van Rsbergen C J. (1976) Informaton retreval test collectons. Journal of Documentaton, 32, pp [22] Waller W G an Kraft D H. (1979) A Mathematcal Moel of a Weghte Boolean Retreval System. Informaton Processng & Management, 15, pp [23] Yager R R. (1988) On orere weghte averagng aggregaton operators n multcrtera ecson makng. IEEE Transactons on Systems, Man, an Cybernetc, 18, pp [24] Yager R R an Flev D P. (1999) Inuce orere weghte averagng operators. IEEE Transacton on Systems, Man an Cybernetcs, 29, pp [25] Zaeh L A. (1975) The concept of a lngustc varable an ts applcatons to approxmate reasonng. Part I. Informaton Scences, 8, pp Part II. Informaton Scences, 8, pp Part III. Informaton Scences, 9, pp [26] Zaeh L A. (1997) What s soft computng?. Soft Computng, 1(1), pp. 1.