Approximate Retrieval from Multimedia Databases Using Relevance Feedback

Transcription

1

2 Approimte Retrievl from Multimedi Dtbses Using Relevnce Feedbck Ouri Wolfson An Lelescu Bo Xu Deprtment of Electricl Engineering nd Computer Science University of Illinois t Chicgo fwolfson,lelescu,bug@eecs.uic.edu Abstrct In this pper we ddress the problem of retrieving stored multimedi presenttions using relevnce feedbck. The multimedi presenttions my be produced by collbortive work. We introduce model of multimedi presenttions tht is conducive to content bsed retrievl using relevnce feedbck. We lso introduce lnguge for retrievl by content bsed on fuzzy logic, nd method for query refinement using relevnce feedbck. Introduction In this pper we ddress the problem of retrievl by content from dtbses tht contin multimedi presenttions. We first introduce model (i.e. n bstrction) of multimedi presenttion. Trditionlly, retrievl by content from such dtbses concentrted on retrievl of imges, nd of video/udio sequences ([, 5, 27]). In contrst, our model cptures the tetul, sptil, nd temporl ttributes of the presenttion. In our model, for emple, slide in the presenttion is represented by reltionl tuple contining the tet of the slide, the time t which the slide strts to be displyed on the screen nd the end time of this disply, nd the coordintes of the loction on the screen where the slide ppers. The video/udio prt of the presenttion is represented by the udio trnscription to tet, which cn be done using stndrd speech recognition techniques. This model is lso pproprite for cpturing collbortive work sessions. Thus, for emple, the contents of common whitebord is represented by tuple, nd ech prticipnt is represented by nother tuple in which the tet ttribute is his/her voice trnscript. The dtbse consists of qunts, where ech qunt represents multimedi presenttion unit (tht conveys some ide), or collbortive work session; This reserch ws supported in prt by grnts DARPA N , NATO CRG-96648, nd Hughes Reserch Lbs the reder cn ssume tht qunt lsts between 5 nd 2 minutes (lthough this ssumption is not importnt for our results). Net, we discuss the query lnguge for retrievl of qunts from the dtbse. The sme lnguge cn be used for filtering of qunts in n environment in which collbortive work sessions occur continuously [7] nd the user wnts to join sessions tht mtch his/her profile. The SQL lnguge is unstisfctory for retrieving nd filtering since it performs ect mtching. On the other hnd, mtching between the query nd qunt my hve to be stisfied in n pproimte sense. For emple, if the user is interested in qunts tht mention light-weight btteries nd strt t 2pm, he my find relevnt qunt tht strts t 2:pm nd mentions lkline btteries. The lnguge consists of conjunctive queries in which n tom, e.g. strt time = 2pm, is to be mtched in n pproimte sense using fuzzy logic pproch. The sme pproch to retrievl from multimedi dtbses ws used by Fgin ([, ]), lthough the focus in his works hs been on video nd imge retrievl. In the present pper, the tom strt time = 2pm is ssocited with tringulr fuzzy membership function (see [37, 35]) which normlizes the difference between the strting time of the qunt nd 2pm, by mpping this difference into rel number in the [,] intervl. The number indictes the degree of similrity between the strttime of the qunt nd 2pm. Similrly, tet = lkline btteries is mpped into rel number in the [,] intervl using stndrd tet retrievl techniques. Observe tht in our model, in contrst to [39, 36, 25, 38], the dt in the dtbse is crisp, i.e. the dtbse does not contin fuzzy terms such s erly, smll, etc. Trditionlly, in multimedi retrievl model ([23, 27,, 8]) the query is represented s point in multidimensionl spce, where ech dimension is feture (e.g. keyword for documents, color nd shpe for imges). Retrievl is performed using similrity function tht mesures the distnce in multidimensionl spce between the query point nd the point tht represents ech object in the dtbse. In

3 contrst, we model the query s set of similrity functions. Ech similrity function is the fuzzy membership function ssocited with ech tom. Further, the multimedi retrievl model considers weighted sum [27] or the Eucliden distnce [8] s the ggregtion function to combine query components, wheres we use the fuzzy ggregtion function min, s ccepted in the fuzzy logic literture (see [33, ]). For pproimte retrievl queries it is importnt to provide the user with itertive nd interctive query refinement mechnisms. We do so by using relevnce feedbck, i.e. the user lbels some of the retrieved qunts s relevnt or irrelevnt. Bsed on this feedbck the system utomticlly refines the query nd resubmits it to retrieve new set of qunts. The process continues until the system cnnot provide ny new qunts tht stisfy the ltest version of the query. Trditionlly, methods of query refinement bsed on user-feedbck use two techniques ([23, 24]), nmely query modifiction nd query re-weighting. The first technique involves either query point movement [8] or query epnsion [23]. The second method utomticlly djusts the reltive importnce of ech feture (i.e. weight) to the query. Our pproch to query refinement is not using ny of the bove methods. The min ide is to perform query refinement by modifying the similrity function for ech fuzzy tom in the query bsed on the relevnce feedbck. Specificlly, we propose n lgorithm to utomticlly djust the set of fuzzy membership functions. In ddition, we incorporte in the relevnce feedbck process eisting informtion retrievl techniques for tet [26]. Although relevnce feedbck is nturl for pproimte queries using fuzzy logic, nd it is well ccepted nd studied technique in tet retrievl (see [36, 29, 3]) nd in multimedi retrievl by content ([8, 22]), s fr s we know this is the first time query refinement technique is pplied in fuzzy logic systems. This pper is orgnized s follows. In section 2 we present the dt model for collbortive session. In section 3 we describe vgue query lnguge. In section 4 we present our pproch of query refinement by relevnce feedbck nd in section 5 we evlute the performnce by simultions. Section 6 includes some implementtion considertion nd section 7 summrizes relevnt work. We conclude in section 8. 2 DATA MODEL A collbortion session or multimedi presenttion consists of sequence of qunts, ech of which conveys n ide. A qunt is represented by set of medi objects (e.g. video, udio, viewgrphs nd other tet files). Ech object is presented to the user in ccordnce with certin sptil nd temporl ttributes. The sptil ttributes give the object s destintion loction on the screen (e.g. rectngulr window given by its Crtesin coordintes) nd its source loction in the network (e.g. IP ddress, host nme or city). The temporl ttributes re strt-time nd end-time, which specify the time intervl during which the object is on the screen. Ech medi object in the qunt is ssocited with some form of tetul informtion. For emple, voice trnscript is the tet ttribute for n udiovisul segment, nd the tet contined in PowerPoint slide is the tet ttribute for the slide. In summry, ech object O is represented by tuple with stndrd dtbse ttributes (e.g. title, uthor, dte, type, etc), tet ttribute tet, temporl ttributes strt time nd end time, nd possibly two sptil ttributes, source loction nd screen loction. Emple Consider qunt creted for clss lecture tht contins four medi objects: journl rticle, the lecture udio trck, nd two PowerPoint slides, slide nd slide2. In Figure the qunt s content is described using logicl visul screens to specify its sptio-temporl structure. At time t=2:pm the instructor strts the lecture, nd the voice trnscript is plyed continuously during the time intervl (2:pm, 2:2pm). Note tht the voice trnscript hs no screen loction ttribute. However its source loction sptil ttribute is During the instructor presenttion (i.e. while the voice trnscript is plyed), t time t2= 2:2pm journl rticle nd slide pper on the screen. The time intervl for slide is (2:2pm, 2:pm) nd its sptil ttributes re given by its loction on the screen nd source ddress respectively. At time t3=2:pm slide will dispper from the screen nd nother slide, slide2, is viewed t the sme screen loction during (2:pm, 2:2pm). Any modifiction in the set of ctive objects mrks screen chnge: the screen now contins the voice trnscript nd the journl s before, nd slide2 insted of slide. The reltion tht contins the qunt presented in this emple is given in Figure 2. The ID ttribute denotes the qunt s unique identifiction number. 2 Note tht in this pper we dopt the reltionl dt model. However, our results re not restricted to this model, nd in fct, they crry over even if the qunt is represented s comple object in the object oriented model. 3 THE QUERY LANGUAGE In this section we introduce fuzzy query lnguge, clled fuzzy-sql, tht llows the user to retrieve qunts from regulr dtbse (i.e. dtbse contining precise fcts), bsed on their content. The sme query lnguge is used for two different retrievl scenrios, nmely the offline scenrio (i.e. trditionl retrievl from dtbse), nd Note tht our source loction ttribute is sptil ttribute in the sense tht it cn be mpped to geogrphicl loction.

4 qunt.strt_time: 2:pm ID Dte 4/3/98 2:pm 2:2pm 2:pm 2:2pm Author John (,y) (,y) Informtion Retrievl G. Slton Introduction Topic Informtion Retrievl G. Slton Introduction Topic (,y ) (,y ) Topic Mchine lerning Topic2 Distributed operting systems Topic3 Dtbse mngement systems (2,y2) (2,y2) Btteries Bulbs User Group Figure. A lecture qunt Type journl Tet Informtion Retrievl G. Slton strt_time 2:2pm end_time 2:pm (2,y2 ) (2,y2 ) screen_loction (,y)(,y ) Introduction topic 4/3/98 John slide. Btteries 2:2pm 2:pm (2,y2)(2,y2 ) Bulbs 4/3/98 Dn slide2 2:pm 2:2pm (2,y2)(2,y2 ). User Group Topic null Lur voice 2:pm 2:2pm null Mchine lerning trnscript Topic2 Distributed operting systems Topic3 Dtbse mngement systems screen chnge screen chnge screen chnge source_loction Figure 2. A reltion representing the qunt in Emple the on-line scenrio (i.e. filter/trigger of incoming qunts). For the on-line scenrio, the qunts rrive continuously one t time (rther thn reside in dtbse) nd the retrievl process is reduced to independent binry decisions to ccept or reject ech qunt. Our gol is to provide high level lnguge such s SQL, tht trets both scenrios in uniform wy nd provides support for pproimte mtching. 3. Synt The fuzzy-sql lnguge is conjunctive SQL (or conjunctive reltionl clculus (see [34])), etended to ccommodte pproimte mtching. The SQL synt is etended in two wys ([2, 35]). The first etension generlizes the definition of toms (or tomic conditions) [34] to include the following pproimte comprison opertors: 2 f =, >, <g, i.e. pproimte-equl, pproimtebigger, pproimte-smller. An tom is clled n pproimte tom if it contins n pproimte opertor. An pproimte tom is clled tetul if it involves the TEXT ttribute. For nontetul pproimte toms, we distinguish between numeric toms, i.e. toms tht involve n ttribute with numeric domin, nd symbolic toms which do not do so. The only pproimte opertor llowed in tetul tom or in symbolic tom is =. The second etension of the SQL synt refers to the fct tht ech object in the dtbse stisfies query with pproimte toms with certin degree of similrity. Therefore, we introduce function clled score to mesure this vlue, nd prmeter T tht represents the cceptnce threshold. The synt of n etended SQL query is therefore: SELECT < ttributes> FROM <reltions> WHERE score(a nd A2... nd Am) >= T where ech Ai is n tom (crisp or pproimte), nd T is rel number from the [; ] intervl. Intuitively, the nswer to this query is set of tuples for which the degree of stisfction is greter thn or equl to T. The min motivtion behind using the threshold vlue insted of the trditionl top k (see [, 5]) is to llow us to provide unified lnguge for both the on-line nd the offline scenrios (note tht top k is meningless in the on-line cse). Emple 2 Consider the query in Figure 3. SELECT id FROM collb_session O,O2 WHERE score(o.uthor = "John D." AND O.tet ~= "multimedi retrievl nd presenttion" AND O2.strt_time ~= 2pm)>=.8 Figure 3. A fuzzy-sql query Intuitively, the query retrives the id of the qunts tht contin tuple O such tht: () the uthor ttribute vlue for O mtches John D. in n ect sense, nd (2) the tet vlue for O pproimtely mtches the phrse multimedi retrievl nd presenttion, nd (3) the strt time vlue for O is pproimtely 2pm. For n object to be retrieved, the ggregte (i.e. minimum) score of the bove toms must be t lest Semntics Let Q be fuzzy SQL query nd D be dtbse. Consider n ssignment for Q, i.e. tuple in the dtbse D from ech reltion mentioned in the FROM cluse of Q. The score of crisp (i.e. nonpproimte) tom is either or, depending on whether or not the tomic condition is stisfied by the ssignment. In order to ssign score to n pproimte tom A in query Q, we ssocite with A similrity function S: S ssigns score in the intervl

5 [,], which indictes how well the ssignment stisfies the tom; score of indictes perfect mtch. The overll score of the ssignment is given by n ggregtion function tht combines the ssignment s scores on ech tom. In this pper, we dopt the stndrd fuzzy-logic ggregtion function min ([8, 33]). The ssignment stisfies the query if the minimum score of ll toms is higher thn the threshold T. Net we discuss the similrity function ssocited with ech pproimte tom A in the query Q. We distinguish between tetul nd nontetul pproimte toms. (For emple, the second fuzzy tom in Figure 3 is tetul.) For tetul pproimte tom, S is stndrd tet similrity function such s the cosine [29]. It ssigns score to the ssignment which indictes how similr re the tets tht pper on both sides of the = opertor. For nontetul pproimte toms, we use fuzzy logic pproch ([37, 35]). A numeric tom with the pproimte opertor is ssocited with fuzzy membership function denoted s (), where is the difference between the vlues of the two opernds in the tom. For emple, given the fuzzy tom O.strt time = 2pm nd tuple O, is the difference between the strt time ttribute of O nd 2pm. The membership functions re defined s follows nd illustrted in Figure 4; nd b re clled the prmeters of ech membership function. =() = >() = <() = µ () ~= () 8 >< >: 8 < : 8 < : b if or b if =? if < < b? if < < b b if < if < if if < if if > µ () ~> (b) µ () ~< Figure 4. Fuzzy membership functions for the pproimte comprison opertors The cse of symbolic pproimte toms cn be reduced to the numeric cse by ssuming tht there eists similrity (or distnce) function tht gets s input ny two symbolic vlues from the domin of the tom, nd outputs the similrity between them. For emple, suppose tht the query contins the tom O.source loction = Sers Tower. If (c) for some tuples source loction is Chicgo, then the score of the tom produced by the similrity function my be.8. Thus, with this ssumption, the results of this pper pply to queries with symbolic pproimte toms s well. 4 QUERY REFINEMENT BY RELEVANCE FEEDBACK The min ide of our query refinement pproch is to itertively modify the membership functions ssocited with query, using the informtion provided by the user bout the relevnce of objects retrieved up to the current itertion. The overll gol of the modifiction is to retrieve ll the relevnt objects if enough itertions re requested by the user, while minimizing the number of irrelevnt objects tht re presented to the user. Hence, query refinement by relevnce feedbck is n itertive process involving user interction nd membership function modifiction. Ech itertion cn be summrized s follows:. Retrieve objects from the dtbse using the current query; 2. If there re new objects retrieved (i.e. objects tht were not mrked relevnt or irrelevnt by the user in previous itertions), present them to the user; Otherwise, go to step The user emines the objects presented. If the user does not request the net itertion, query refinement stops. Otherwise, the user mrks the objects s relevnt or irrelevnt nd step 4 is eecuted. 4. Eecute the Membership Function Modifiction (i.e. MFM) lgorithm to djust the query (see Section 4.2 below). The sme query refinement procedure cn be pplied for both on-line nd off-line retrievl scenrios. In the net two subsections we present the MFM lgorithm. In section 4. we introduce the preliminries of the lgorithm nd in section 4.2 we describe the lgorithm. 4. Preliminries In the following, for ese of eposition, we will ssume tht ech tom A i hs the form O:M i =, where M i is numeric ttribute. Ech A i is ssocited with tringulr membership function i () (see Figure 4()). Furthermore, since the lgorithm is symmetric for either side of i (), w.l.g. we will consider only the right side, i.e. we will ssume tht ll the vlues of M i re positive. We denote the membership function in this cse by i ( : b i ), where b i is the right bound. Given set of retrieved objects F, the relevnce ssignment of F is pir of disjoint subsets of F, nmely (F R ; F I ), where F R contins ll the objects mrked rele-

6 vnt nd F I contins ll the objects mrked irrelevnt. 2 A relevnce ssignment (F R ; F I ) is clled complete if F I is empty. At ech itertion the system mintins mrked objects collection (M OC) which ccumultes the mrked objects. Our lgorithm involves two types of membership function modifictions, nmely shrink nd epnd. The shrinkmodifiction chnges the prmeters of membership function such tht () the similrity of ech object tht hs been mrked relevnt is higher thn the threshold T ; (2) the similrity of ech object tht is presented but not mrked t this itertion is higher thn T ; nd (3) The number of objects tht hve been mrked irrelevnt nd tht hve similrities lower thn T is mimized. The epnd-modifiction increses the right bound of ech membership function i by positive constnt vlue c i, i.e. membership function i ( : b i ) becomes i ( : b i +c i ). We cll c i the epnsion constnt of i. The vlue of c i depends on the density of the objects long the is, nd therefore it is prmeter to the MFM lgorithm for ech fuzzy membership function. 4.2 Algorithm Description The MFM lgorithm modifies the current membership functions s follows. If no new objects re retrieved t this itertion or the relevnce ssignment is complete (i.e. ll the objects tht the user mrked t this itertion re relevnt), MFM epnds ech i ( : b i ) (epecting to retrieve more relevnt objects). Otherwise, it shrinks i ( : b i ). Below is the pseudo code of the MFM lgorithm. INPUT F, F R, F I, MOC, ech i ( : b i ) IF (there re no new objects retrieved (n object retrieved is new if it is in F but not in MOC)) epnd ech i ( : b i ) ELSE IF (the relevnce ssignment is complete) epnd ech i ( : b i ) ELSE F MOC = F [ MOC FOR ech i ( : b i ) find out in F MOC the non-irrelevnt (i.e. relevnt or not-mrked) object which is the gretest (i.e. the frthest from ) IF (there is such n object O) i ( : b i ) becomes i ( : O:A i =(? T )) ELSE //ll the objects in F M OC re irrelevnt i ( : b i ) becomes i ( : ) END END 2 The union of F R nd F I my be proper subset of F, i.e. the user does not necessrily mrk every object s relevnt or irrelevnt. Indeed, the user does not know priori how mny objects will be retrieved in n itertion, nd the retrieved set my be too lrge to emine in its entirety. END MOC = MOC [ F R [ F I END T µ () O O2 O3 O2.M/(-T) (N) (R) (I) b µ () 2 T R: Relevnt I: Irrelevnt N: Not Mrked O2 O (R) (N) O3 (I) O.M2/(-T) Figure 5. A shrink modifiction emple Note tht epnd-modifiction is firly strightforwrd. Therefore, in the following we choose to give n emple of the shrink-modifiction. Emple 3 Consider query with two toms A nd A2. Suppose tht t the beginning of n itertion, the two membership functions ssocited with these two toms re ( : b) nd 2( : b2) (see Figure 5). Assume tht the mrked object collection MOC is fo2g. Suppose tht there re three objects retrieved t this itertion, nmely O,O2 nd O3. Assume tht the relevnce ssignment is (;; fo3g), i.e. O3 is mrked irrelevnt nd O is not mrked (O2 is not presented to the user, since it is not new). Since the relevnce ssignment is not complete, MFM eecutes the shrink modifiction for both ( : b) nd 2( : b2). For, since O2 is the frthest non-irrelevnt object from, ( : b) is shrunk to O2:M =(? T ) (see Figure 5()). Thus, the score of O3 is lower thn T, nd the scores of O nd O2 re higher thn or equl to T. Similrly, the new bound of 2 is determined by O (see Figure 5(b)). After the shrink modifiction, the overll score of O3 is below T nd the overll scores of O nd O2 re bove or equl to T.2 5 EXPERIMENTS The min purpose of our eperiments is to evlute the performnce of the bove proposed MFM lgorithm. The evlution is conducted by compring it with the MindReder system described in [8]. We lso wnted to investigte the impct of the user itertion model on the performnce of our lgorithm. 5. Simultion environment We implemented our MFM lgorithm nd conducted eperiments using synthetic dt. We rndomly generted dtbse of 2 objects in n-dimensionl spce (i.e. objects with n ttributes) where n ws chosen to be either 2 b 2

7 or 5. For ech dimension the dt set is normlized in the intervl [; ]. We use these ttributes to build query tht hs n toms of the form M i = for i = ; 2; :::; n. The relevnce ssignments were generted for ech object using the following model. For ech dimension we set up two intervls [; ], (; ],. If n object flls into [; ] for ech dimension, then it is relevnt; otherwise it is irrelevnt. All the eperiments considered two prmeters: number of dimensions nd the relevnce ssignment model. For ll eperiments we mke the percentge of relevnt objects to be round 2%. Ech eperiment is conducted s follows. We strt with rbitrry membership functions. We ssume tht the user will mrk the retrieved objects from top to bottom until he meets the first irrelevnt object. Ech eperiment stops when ll the relevnt objects hve been mrked by the user. For MindReder the query is point in the n- dimensionl spce nd ech dimension is ssocited with weight[8]. Given set of objects with relevnce ssignments, MindReder estimtes the idel query nd weights. For the purpose of this eperiment, t the first itertion, we trin MindReder with the set of objects tht were retrieved nd mrked t the first itertion in our lgorithm. After tht, t ech itertion MindReder uses the previous query to retrieve top k objects nd then djusts the query nd the weights bsed the relevnce ssignment for them. At ech itertion we mke k the sme s the number of objects mrked for MFM t tht itertion. 5.2 Performnce Results cumultive recll+precision cumultive recll+precision () 5 toms MFM MindReder itertion (b) 2 toms MFM MindReder Comprison with MindReder.2 The comprison is conducted s follows. At ech itertion, we compute the cumultive recll+precision which is the sum of recll nd precision clculted bsed on the mrked objects collection up to tht itertion (i.e. MOC). The reson for this is tht we wnt to compre t ech itertion the increse in the number of relevnt objects retrieved nd the decrese in the number of irrelevnt objects retrieved. User s stisfction clerly depends minly on these two numbers since he/she wnts to receive s mny relevnt objects nd s few irrelevnt ones s possible. Figure 6 gives set of the comprison results. From Figure 6 we my see tht the performnce of MFM is consistently better thn tht of MindReder. Initilly, the performnce of MFM is very ner to tht of MindReder. However, the difference between them increses with the number of itertions. Regrdless of the number of dimensions, the MFM lgorithm is lmost perfect, i.e. the precision nd recll sum converges to itertion Figure 6. Comprison between top-bottom nd rndom relevnce feedbck The Influence of User s Interction The following eperiments illustrte two etreme emples of the user s mrking behvior for the MFM lgorithm. In the first scenrio, the user lwys mrks the rnked list of retrieved objects from top to bottom, nd in the second scenrio the user mrks rndomly from the rnk list. Figure 7 shows the result of our simultion for both mrking behviors. We see tht the top-bottom mode is much better thn the rndom one nd it converges much fster. Fortu-

8 ntely, top-bottom mrking is lso the most resonble user behvior. cumultive recll+precision cumultive recll+precision () 5 toms top-bottom rndom itertion (b) 2 toms top-bottom rndom itertion Figure 7. Comprison between top-bottom nd rndom relevnce feedbck 6 Implementtion Considertions Since the query lnguge is formulted in n etended- SQL dilect, the query processing is bsed on trnsforming it into one or more regulr SQL queries. For emple, consider the tom A=(O.strt time = 2pm) in query with threshold T. Let [; b] be the intervl for which () T, where () is the tom membership function. Then tom A is converted into rnge condition < O.strt time < b. We implemented the fuzzy query s wrpper round conventionl DBMS using INFORMIX. The min dvntge of using n object reltionl system is the fct tht it offers support for users to define ny new dt types nd opertions (i.e user defined functions) on them. The pproimte comprison opertors were implemented in the C lnguge. For tet ttributes, our implementtion uses commercilly vilble librry of functions designed to process tet using IR techniques. This pckge is provided by the Eclibur Tet DtBlde for INFORMIX. 7 RELEVANT WORK Vgue Queries in DBMS The need for vgue or imprecise queries hs been pointed out for mny yers nd hs been ddressed in the contet of fuzzy dtbses ([36, 25, 38]). Vgue queries re formulted by introducing definitions of fuzzy ttribute vlues (e.g young ) nd fuzzy comprtors similr to the pproimte opertors presented in this pper. In contrst, we ssume tht the dtbse stores only crisp informtion nd our query lnguge is intended to be used in combintion with eisting conventionl dtbse systems nd IR technology. None of the current work in fuzzy dtbses provides ny mens for query refinement. For crisp dtbses, there re two pproches in current literture which support vgue queries. None of them is bsed on fuzzy logic. First, the VAGUE system introduced in [2] is bsed on vrint of the vector spce model for dtbse reltions. The distnce between two tuples is computed by ggregting the individul distnces between the corresponding ttributes by mens of weighted Eucliden distnce mesure. In contrst, our ggregtion function is is given by min. Moreover, VAGUE doesn t use relevnce feedbck. Second, Fuhr introduced in ([4, 3]) method of integrting tet queries nd vgue queries in dtbses bsed on probbilistic model using relevnce feedbck. The min disdvntge there is tht the query lnguge is more restrictive due to independence ssumptions. Multimedi Retrievl Our concept of pproimte queries is similr to tht of content-bsed queries in multimedi pplictions [5]. Most recent work in this re focused on specific dt types such s tet, imges, video udio, etc. There is solid IR technology to retrieve documents bsed on their content ([29, 3]), nd reserch in imge processing led to systems such s QBIC [2], MARS [27]. In contrst, we introduced novel dt model tht combines tet, temporl nd sptil dt types for fuzzy retrievl of collbortion sessions. The ide of using multimedi retrievl model bsed on fuzzy sets ppers in ([, 22]). However, none of these works

9 uses relevnce feedbck. Relevnce Feedbck A different pproch to modify the membership function for fuzzy ttribute vlue (such s young ) ws introduced in [36]. The modifiction is motivted by the fct tht ech fuzzy term my hve different mening in the query. However, this modifiction does not involve relevnce feedbck. Relevnce feedbck methods hve been etensively studied in IR re ([3, 4, 26]). For the tet prt of our query refinement pproch, we ssume tht we cn plug in ny IR system tht supports relevnce feedbck, such s SMART [29]. Most of the reserch using relevnce feedbck in multimedi retrievl system is relted to the MARS system (see [23, 24, 27]) nd MindReder [8]. Different models for query reweighting nd query modifiction were eplored. There re three different query refinement strtegies, nmely query point movement [8], query epnsion model [23], nd query reweighting [27]. Query point movement moves the query towrd relevnt objects nd wy from the irrelevnt ones. Query epnsion dds more components to the query. Our pproch is completely different thn the bove methods. First, for query reweighting, if the weight on one dimension is incresed, it is decresed for nother one, but in our cse we epnd or shrink ll toms simultneously. Second, we do not move query nd we do not dd components to the query. The work in [28] considers n imge retrievl system tht supports multiple similrity mesures for given imge feture such s shpe. Relevnce feedbck mechnism is employed here to identify the similrity mesure tht ctully fits the user. In contrst, we use single similrity function for ech ttribute nd we modify this ccordingly. 8 Conclusion In this pper, we focused on the problem of retrievl by content from dtbses tht contin collbortive work sessions. We first introduced dt model tht cptures the tetul, sptil, nd temporl ttributes of multimedi sessions (qunts). Then we described fuzzy query lnguge tht uses pproimte comprison opertors to retrieve the qunts for two different scenrios, i.e. on-line nd off-line. For the on-line scenrio the qunts rrive continuously, wheres for the off-line scenrio they reside in dtbse. The min contribution of this pper is new pproch to perform query refinement bsed on user s relevnce feedbck by modifying the fuzzy membership function ssocited with ech tom. We proposed n lgorithm (i.e. MFM) to utomticlly djust the set of fuzzy membership functions in the query. The simultion results show tht the MFM lgorithm outperforms the MindReder system. Acknowledgements We wish to thnk Prof. Clement Yu nd Prof. Simon Ksif for helpfull discussions. References [] Bosc P., Glibourg M., Hmon G., Fuzzy querying with SQL: Etensions nd implementtion spects, Fuzzy Sets nd Systems, 28, 988. [2] Buckley C., Slton G., Optimiztion of Relevnce Feedbck Weights, in Proceedings of the 8th Annul Intl ACM SIGIR Conference, Setle, 995. [3] Buckley C., Slton G.,Alln J., The effect of dding relevnce feedbck informtion in relevnce feedbck environment in Proceedings of the 7th Annul Intl ACM SIGIR Conference, 996. [4] Croft B, Clln J., et ll, Integrting IR nd RDBMS Using Coopertive Indeing, in Proceedings of the 8th Annul Intl ACM SIGIR Conference, Setle, 995. [5] Do S., Shek E., Vellikl A., Muntz R., Zhng L., Potkonjk M., Wolfson O., Semntic Multicst: Intelligently Shring Collbortive Sessions, ACM Computing Surveys, 998. [6] Dubois D., Prde H., Criteri ggregtion nd rnking of lterntives in the frmework of fuzzy set theory, Fuzzy Sets nd Decision Anlysis (H.J. Zimmermnn, Zdeh L.A, Gines B., Eds). [7] Fgin R., Combining Fuzzy Informtion from Multiple Systems, in Proc. 5th ACM Symp. on Principles of Dtbse Systems, 996. [8] Fgin R., Fuzzy queries in multimedi dtbse systems, in Proceedings of ACM SIGMOD Conf, 998. [9] Fuhr N., Integrtion of probbilistic fct nd tet retrievl in Proceedings of 5th Annul Intl ACM SIGIR Conference, Denmrk, 992 [] Fuhr N., Rolleke Th., A Probbilistic Reltionl Algebr for Integrtion of Informtion Retrievl nd Dtbse Systems, in ACM Trnsctions on Informtion Systems, 5(), 997 [] Grosky W.I., Mnging multimedi informtion in dtbse systems, Communictions of the ACM, 4(2), 997.

10 [2] Hrmn D., Relevnce feedbck revisited, in Proceedings of 5th Annul Intl ACM SIGIR Conference, Denmrk, 992 [3] Ishikw Y., Subrmny R., Floutsos C., MindReder: Querying dtbses through multiple emples, Proceedings of the 24th VLDB Conference, New York, 998 [4] Korth H.F., Silbershtz A., Dtbse System Concepts, McGrw-Hill, Second Edition, 99. [5] Motro A. VAGUE: A user interfce to reltionl dtbses tht permits vgue queries. ACM Trnsctions on Office Informtion Systems, 6(3), 988. [6] Niblck W., et l. The QBIC Project: Querying Imges by Content Using Color, Teture nd Shpe in Proceedings SPIE, Sn Jose, CA, 993. [7] Orteg M., et l. Supporting similrity queries in MARS, in Proceedings of ACM Conference on Multimedi, 997. [8] Porkew K., Orteg M., Mehrotr S., Query Reformultion for Content Bsed Multimedi Retrievl in MARS, in Proceedings of ACM Conference on Multimedi, 999. [9] Porkew K., Mehrotr S., Orteg M., Chkrbrti K., Similrity Serch Using Multiple Emples in MARS in Intl Conference on Visul Informtion Retrievl, 999. [25] Slton G., Buckley C., Term weighting pproches in informtion retrievl, Informtion Processing nd Mngement, 24(5), 988. [26] Thole U., Zimmermnn J., Zysno P., On the suitbility of minimum nd product opertors for the intersection of fuzzy sets, Fuzzy Sets nd Systems, (2), 979. [27] J. D. Ullmn, Principles of Dtbse Systems, Computer Science Press, 982. [28] Yen J., Lngri R., Fuzzy Logic: Intelligence, Control nd Informtion, in Prentice Hll, 999. [29] Yu C., Meng W., Principles of Dtbse Query Processing for Advnced Applictions, Morgn Kufmnn Publishing, 998. [3] Zdeh L.A., Fuzzy Sets, Informtion nd Control, 8, 965. [3] Zemnkov M., Kndel A., Implementing imprecision in informtion systems, Informtion Sciences, 37, 985. [32] Zhng W., Yu C., Regn B., Nkjm H., Contet dependent interprettion of linguistic terms in fuzzy reltionl dtbses, IEEE Dt Enginning Conference, 995. [2] Prde H. Testmle C., Generlizing reltionl lgebr for tretment of incomplete or uncertin informtion nd vgue queries, Informtion Sciences, 34(2), 984. [2] Rochio J.J., Relevnce Feedbck in Informtion retrievl, in Slton Ed.. The Smrt Retrievl System Eperiments in utomtic document processing, Prentice Hll, Englewoods Cliffs N.J, 97. [22] Rui Y., Hung T.S., Mehrotr S., Content-bsed imge retrievl with relevnce feedbck in MARS, Proceedings of IEEE Intl Conference on Imge Processing, 997. [23] Rui Y., Mehrotr S., Orteg M., Automtic mtching tool selection using relevnce feedbck in MARS, in Intl Conference on Visul Informtion Retrievl, 997. [24] Slton G., The SMART Retrievl System-eperiments in utomtic dt processing, Prentice Hll, Englewood Clifs NJ, 97.