A personalized search using a semantic distance measure in a graph-based ranking model

Transcription

1 Article A personalized search sing a semantic distance measre in a graph-based ranking model Jornal of Information Science XX (X) pp The Athor(s) 2011 Reprints and Permissions: sagepb.co.k/jornalspermissions.nav DOI: / jis.sagepb.com Mariam Daod IRIT, Pal Sabatier University, 118 Rote Narbonne F-31062, Tolose cedex 9, France Lynda Tamine; Mohand Boghanem IRIT, Pal Sabatier University, 118 Rote Narbonne F-31062, Tolose cedex 9, France Abstract The goal of search personalization is to tailor search reslts to individal sers by taking into accont their profiles, which inclde their particlar interests and preferences. As these latter are mltiple and changing over time, personalization becomes effective when the search process takes into accont the crrent ser interest. This paper presents a search personalization approach that models a semantic ser profile and focses on a personalized docment ranking model based on an extended graph-based distance measre. Docments and ser profiles are both represented by graphs of concepts issed from predefined web ontology, namely the ODP 1. Personalization is then based on reordering the search reslts of related qeries according to a graph-based docment ranking model. This former is based on sing a graph-based distance measre combining Minimm Common Spergraph (MCS) and maximm common sbgraph (mcs) between the docment and the ser profile graphs. We extend this measre in order to take into accont a semantic recovery at exact and approximate concept level matching. Experimental reslts show the effectiveness of or personalized graph-based ranking model compared to Yahoo 2 and to different personalized ranking models performed sing classical graph-based measres or vector space similarity measres. Keywords Personalization; ser profile; graph formalism; ontology 1. Introdction Traditional information retrieval (IR) approaches consider that ser needs are described flly by the ser qery. These approaches are characterized as one size fits all providing the same reslts for the same keyword qeries even thogh these latter are sbmitted by different sers with different intentions. For example, the qery python may refer to python as a snake as well as the python programming langage. In more general context, the relevance decision is made independently of what the ser has searched before, ths by considering the qery as the main cle that specifies the ser information need. Personalized search aims at enhancing the ser qery with the ser profile to better meet the individal ser needs. The ser profile describes the ser interests and preferences that cold be explicitly set by the ser or gathered implicitly from the ser search history. The most challenging tasks for an effective search personalization are: (1) how to model an accrate ser profile and, (2) how to exploit it to improve the search. User profile representation models vary from very simple representation models based on bag-of-words [1] to more elaborated ones based on external semantic knowledge sch as ontologies or concept hierarchies [2, 3, 4]. A srvey on the research approaches that exploit the information extracted from Web logs (or qery logs) in personalized ser ontologies for personalizing search is presented in [43]. Tailoring the search reslts to a particlar ser consists of exploiting his/her profile in a Corresponding athor: Mariam Daod, IRIT, Pal Sabatier University, 118 Rote Narbonne F-31062, Tolose cedex 9, France, mariam_daod@live.fr

2 Mariam Daod, Lynda Tamine and Mohand Boghanem 2 personalized docment ranking by means of qery reformlation techniqes [2, 5], qery-docment matching [6, 7, 8] or reslt reordering [9, 4, 3]. In this paper, we exploit a semantic graph-based ser profile in a personalized docment ranking model. In this model, the docments and the ser profile are both represented as graphs of concepts and then personalization consists of reordering the docments sing a graph-based distance measre. This latter is based on a semantic extension of the graph-based distance measre combining maximm common sbgraph (mcs) and Minimm Common Spergraph (MCS). We extended this measre by introdcing an approximate concept matching considered throgh cross links connecting the two graphs. In this paper, we extend or previos work [42] by (1) stdying the accracy of the ser profile modeling (2) tning different parameters of the systems and stdying their impact on the personalized search effectiveness, (3) comparing or approach to other graph-based measres sed for personalized reranking as well as to a vector space matching model where docments and the ser profile are represented either by term lists or concept lists. The rest of this paper is organized as follows. In section 2, we present some related works on personalized docment ranking models and give an overview of exploiting a graph formalism for improving the search process. In section 3, we present a general overview of or contribtion; the graph representation model for docments and the ser profile and highlight or research objectives. In section 4, or personalized graph-based docment ranking model is detailed. In section 5, an experimental evalation setp and appropriate evalation protocols for the ser profile accracy and the personalized search effectiveness are described. In section 6, the evalation reslts of the ser profile accracy are presented. In section 7, the reslts of the personalized search effectiveness evalation compared to Yahoo search inclding a comparative stdy with other ranking models are presented. Finally, section 8 concldes the paper. 2. Related works In this section, we overview some significant related works on representing ser profile models, personalizing search sing a ser profile and exploiting a graph formalism for improving the retrieval effectiveness On modelling the ser profile A common aspect in most of ser modeling techniqes is to collect relevant sorces of evidence from the ser search history in order to model the ser profile. Generally, the ser search history is composed by the ser s docments of interest sch as recently browsed or jdged/assessed web pages [9, 4], a repository of interesting information sch as s, browsing featres or desktop information [10]. These data are then exploited in algorithms of data mining [4, 9, 11] or machine learning [12, 3, 13] strategies in order to bild the ser profile. The ser profile cold represent either short term ser interests [11, 4] or long term ones [14] and cold be represented by simple representation models based on simple terms [15, 6] or sing ontologies and concepts hierarchies [4, 9, 3]. Several works make distinction between short term ser interests [11, 4, 16] sed to better meet the ser needs in a crrent task, and long term ser interests [14] sed to improve the search for any qery. Generally, a short-term ser profile refers to the ser interests dring a short period of time and inferred from the recent search history. Some works sch as [11, 4] consider only the pre-qery session activity to define the search context. White et al. [16] consider the ser interest model for the crrent qery in addition to the session context defined by pre-qery session activity, and their combination to refer to the ser context, called intent model. A long-term ser profile holds persistent ser interests generally stable for a long time and inferred from the whole ser search history. Concerning the ser profile representation models, they are often based on a set of keyword vectors [15, 6] or classes of keyword vectors [1]. To overcome the limitations of the keyword-based representation models (absence of interrelations between keywords), some approaches model the ser interests by term and term relations [5] or hierarchical clsters of terms [17, 18, 19] issed from the ser s docments of interest. The benefit behind sch representation models is that the links between terms allow improving the retrieval effectiveness for similar qeries. Even if sch representations are well elaborated, they are leveraged from the ser history that is often limited and not sfficient to enconter new ser need. In order to integrate new knowledge in the ser profile model, some approaches make se of predefined semantic resorces to represent the ser profile as a set of concepts issed from predefined ontology [3, 20] or an instance of reference ontology [4, 9]. It is well known that ontology-based ser profile representations present several advantages compared to other representation models as they provide explicit semantic knowledge that facilitates the inference of new knowledge [21].

3 Mariam Daod, Lynda Tamine and Mohand Boghanem On personalizing search sing the ser profile Search personalization is achieved by integrating the ser profile in the qery reformlation process [2, 5, 11], qerydocment matching [6, 8, 7], personalized docment categorization [22] or docment reordering [4, 3, 9, 23]. Most of qery reformlation techniqes sed in personalized IR are based on adding or reweighting terms issed from the ser profile [2, 5, 11]. In [2], the qery is matched with the most appropriate pair of concepts of the ser profile; the first concept is the most similar one to the ser qery, and it is sed to add or enhance the weight of the qery terms while the second concept is the less similar one, it is sed to eliminate non relevant terms in the search. A more elaborated ser profile based on terms connected by different edge relations [5] (negation, sbstittion, etc.), allows adding, eliminating or sbstitting the qery terms with relevant ones. Qery refinement in [11] consists of adding weighted terms to the qery from the ser context defined by the ser search history sing Rocchio [24] algorithm. Added terms are selected according to a langage model that calclates the weight of a term in the previos qeries and the snippets of the clicked docments. Few works incorporate the ser profile in a qery-docment matching model. This latter consists of calclating the relevance score of a docment relative to both the ser qery and the ser profile. A bayesian model integrating a docment relevance score fnction is proposed in [6] in order to increase the score of the docment when its vocablary matches the ser profile one. In the same context, the proposed approach in [7] adjsts the scoring fnction parameters to the ser context sing genetic programming. Personalization based on reslt re-ranking consists of combining the content-based docment score with the personalized docment score [9, 4, 3] or with the personalized PageRank of the docment [23]. The personalized docment score is compted in [9, 4] sing the cosine similarity measre with the most similar concepts of the ser profile. A variant techniqe of personalized docment ranking is the personalized categorization of the search reslts. It consists of groping the search reslts into categories that describe the ser interests [22]. Personalization in [3] consists of combining personalized categorization and reslt re-ranking sing a voting-based merging scheme. Mapping the qery into a set of related concepts allows categorizing the search reslts into mltiple lists of retrieved docments. The new rank of a reslt is compted based on its rank in the list, the rank of the concept associated to the list and the similarity of the concept with respect to the qery. Personalized reslt-reranking based on PageRank is described in [23]. The ser selects his preferred pages from a set of hb pages and a personalized PageRank vector is compted for each ser interest sed to redirect the retrned web pages to the preferred ones. Chirita et al. [20] se a system in which a ser manally selects ODP categories as entries in his/her profile. Reranking search reslts is based on measring the similarity between a search reslt and the ser profile sing the node distance in a taxonomy concept tree, which means the search reslt mst associate with an ODP category On sing graphs for docment ranking The se of a graph formalism for docment ranking is investigated in several works for improving the retrieval effectiveness. We categorize these works into two broad categories based on the following principles: sing the graph as a semantic representation model for qeries and docments and exploiting it in a semantic docment retrieval model sing graph-based measres [25, 26, 27, 28,44], sing the graph to formalize hyperlinks between docments, clickthrogh relationships between qeries and docments [29], or semantic relationships between terms and docments [30, 31, 32] and then exploit the graph as an additional resorce for improving the retrieval effectiveness. Concerning the first category of works, graphs of docments or qeries are represented by concepts of WordNet linked by association rles, where each concept is defined by a set of descendant concepts [25] or by fzzy conceptal graphs sed for medical text indexing [44]. Ranking the docments with respect to a qery is based on calclating a conceptal similarity sing a graph-based measre that combines strctral and relational similarity degrees on WordNet. In the same context, qeries and docments are represented by concept sbtrees of WordNet [26]. Ranking the docments with respect to the ser qery is based on calclating the minimal sbtree that contains the docment sbtree H d and the qery sbtree H q at the same time and then calclating the relevance docment score sing a graphbased distance measre. This latter makes se of fzzy logic association rles sch as Diene, Gödel, or Lkasiewicz. In [27], a docment is represented as a sbgraph of a predefined ontology graph and an approximate graph matching approach is sed for content (address) extraction. The ontology constrction for the address domain tilized the DMTI GIS database, which provided all instance concepts, sch as Street Name, Street Type, City and Province, sed Qebec and Ontario, Canada and the is sed as a test Web docment set. The involved graph matching model considers a graph template for the address domain (city, contry,..) and calclates a similarity

4 Mariam Daod, Lynda Tamine and Mohand Boghanem 4 between a graph representing a segment text according to the nmber of shared nodes as well as the nmber of shared edges between graphs. Instead of sing only the shared nodes between graphs, a conceptal graph matching for semantic search in [28] exploits a similarity between concepts representing the entries of graphs and a similarity between relations based on their respective positions in the concept hierarchy from which they are issed (namely WordNet). The similarity between concepts is calclated in terms of the length path throgh the closest common parent concept. While the similarity score between two relations is 1 depending on whether a relation is a spertype of another relation and 0 otherwise. Concerning the second category of works, several graph-based ranking models similar to HITS algorithm [33] or PageRank [34] have been proposed in recent years, by focsing on sing the graph formalism as an additional resorce for improving the retrieval effectiveness. While the graph in HITS and PageRank formalizes relationships between the docments throgh the hyperlinks, it formalizes in some other approaches the relationships between qeries and clicked docments in the same searching session [29], or relationships between terms and docments [30, 31, 32]. 3. Research overview and objectives As mentioned above, the main challenging qestions for an effective search personalization are (1) how to model a ser profile and (2) how to exploit it in an effective way. Concerning the ser profile modeling, we consider a semantic graph-based ser profile [35] that represents a short-term ser interest in a specific search session. The ser profile is inferred from the ODP ontology 3, which is considered as a highly expressive grond to describe a semantic representation model of the ser profile. In order to exploit the ser profile in an effective way, we propose a personalized graph-based ranking model, which is the focs of the paper. Or research objectives and a general overview of or search personalization approach are given below Research objectives Or objective in this paper focses on a personalized docment ranking model where docments are reordered according to their similarity with the ser profile sing a graph-based distance measre. Compared to other related works that model ontology-based ser profiles [4, 9, 3, 22], or work is the first attempt to involve a graph-based representation for both the docments and the ser profile and a personalized graph-based ranking model for reordering the docments. Or intition behind this model is to captre term dependencies between terms in the docment content as well as between the concepts of interests in the ser profile model and to rank semantically the docments according to an exact and approximate concept level matching. Indeed, the main limitation of docment-profile keyword-based matching depends relatively of the docment length by treating mltiple topics in the docments in similar way. Concept-based matching allows overcoming the limitations of the term-based matching by exploiting the semantics in the docment and by discovering concept dependencies between mltiple topics that cold exist between the docment and the ser profile instead of estimating the shared terms between the docment and the ser profile. Compared to other related works, or approach has the new following featres: Most of personalized docment reranking approaches represent the docments and the ser profile by term vectors. Whereas or approach represents both the docments and the ser profile by graphs of concepts, Most of other personalized reranking methods ignore the semantic relationships between concepts of the ser profile by ranking the docments according to their similarity to the ser profile sing the cosine measre. Whereas or approach ranks semantically the docments compared to the ser profile according to a graphbased distance measre. The main research qestions that highlight or contribtion in the present paper are the following: How to exploit the graph formalism of the reference ontology to represent semantically the docments and the ser profile? o Concerning the ser profile modeling, we have previosly proposed [35] a graph-based representation model, inferred from the ODP ontology, o Concerning the docment graph, it is inferred by mapping the docment content on the ODP ontology and extract its associated graph. How to design a graph-based ranking model that ranks semantically the docments with respect to the ser profile? 3

5 Mariam Daod, Lynda Tamine and Mohand Boghanem 5 o o We se a semantic graph-based measre based on combining MCS and mcs measres [36] between the docment and the ser profile graphs. We extend this measre to take into accont a semantic recovery between the docment and the ser profile graphs at exact and approximate concept level matching General approach Or search personalization approach aims at exploiting the ser profile in a reslt reranking model. It consists of (1) representing both the docments and the ser profile, as graphs of concepts and then (2) reordering the search reslts of a given qery according to a graph-based docment ranking model sing the ser profile. This latter is bilt across previos qeries related to the crrent ser information need, which are groped in the same search session. The graph-based representation model of the ser profile and the docments and a general overview of or session-based personalized search sing a graph-based measre are detailed below A graph-based ser profile representation model The ser profile represents the ser interest in a crrent search session and is represented as a graph of weighted and interrelated concepts inferred from the ODP ontology [35]. It is bilt across related qeries by mapping the ser s docments of interest on reference ontology and then applying spreading concept activation for extracting the graph of the most relevant concepts. The overall process of generating the ser profile is detailed in two main steps: (1) bilding the ontological qery profile, (2) maintaining the ser profile in the same search session. Bilding the ontological qery profile: the qery profile represents the concepts extracted from the reference ontology with respect to a ser qery and the associated ser relevance feedback at the end of a search activity. Bilding the qery profile is based on two main steps. o Extracting a keyword vector: a keyword vector is extracted from the ser s docments of interest D R and then is mapped on the reference ontology to extract an initial weighted concept set. Each docment in D R is represented as a single term vector where the weight w td of term t in docment d is compted as follows: w td = tf d log(n/n t ). tf d is the freqency of term t in docment d, N is the total nmber of docments in the test docment collection and n t is the nmber of docments containing term t. We represent then the keyword vector by averaging the docments in D R. In order to represent the ser interests semantically as a set of concepts derived from the ontology, we map the keyword vector on the ODP ontology sing the cosine similarity measre. The reslt of mapping the qery context on the ontology is an initial set of concepts, called θs = {(c1, score(c1),..(ci, score(ci), )}. o Inferring the graph-based qery profile: based on the set of concepts θs, we attempt to bild a graph of semantically related concepts of ontology sing a spreading activation strategy. Spreading activation consists of propagating the weights of concepts having non-zero vales throgh different edge types of the ontology. We distingish the role of different edges in activating linked concepts in the score propagation. A set of graphs is then identified by groping interrelated concepts together and the qery profile is finally represented by the most relevant graph consisting of the most highly interrelated concepts. More details regarding the spreading activation process cold be fond in [35]. We mention that the concepts of the ODP are interrelated with different relationship types. An important distinction between taxonomies and ontology sch as the ODP ontology is that edges in taxonomy are all of the same type ( is-a links). While in the ODP ontology, edges can have additional semantic types (e.g., symbolic, related ), called cross links. Symbolic links are sed for mlticlassification in ODP. This mlticlassification link, like all mlticlassification links, brings the ser across the directory into a different category. It prevents the ser being at a page in ODP from having to traverse back (or p) some steps to the root of the directory and drills down from there to the topic the ser search for. Another link types are cross-references sch as those annotated with see also: and freqently sed in ODP. Sch links are not considered as symbolic links in the ODP RDF markp where they are represented with the related tag. The label of a see also: crossreference link informs the ser of the link target a priori, i.e., before the ser commits to following it. Maintaining the ser profile: it consists of combining qery profiles issed from the same search session. More precisely, it is based on accmlating the weights of possible common concepts and adding edges linking related concepts across related qeries.

6 Mariam Daod, Lynda Tamine and Mohand Boghanem 6 Formally, the graph strctre G=(V,E) of the ser profile has a hierarchical (tree) component composed of is-a links, and a non hierarchical component composed of cross links of different types predefined in the ODP ontology, where: V is a set of weighted nodes, representing the ser s concepts of interest, E is a set of edges between nodes in V, partitioned into three sbsets T, S and R, sch that T corresponds to the hierarchical component of the ser profile made of is-a links, S corresponds to the non-hierarchical component made of symbolic cross links and R corresponds to the non-hierarchical component made of related cross links. Figre 1 illstrates an example of a ser profile/docment that corresponds to the compter langage programming interest. In this example, the ser/docment profile G is defined by the following sets: V = {(c1, score(c1)), (c2, score(c2)),.., (c8, score(c8))}, S = {(c5, c4), (c5, c8), (c5, c6)}, T = {(c1, c2), (c1, c3), (c2, c4), (c2, c5), (c3, c6), (c3, c7), (c4, c8)}, R = {(c5, c3)}. Figre 1: A portion of an ontological ser profile in a graph-based representation A graph-based docment representation model Docments retrned with respect to a ser qery are represented by graphs of concepts and reordered according to a graph-based distance measre with respect the ser profile. Inferring the docment graph is described by the following steps: representing the docment content by a term vector d, k mapping the docment content on the ODP ontology by calclating the weight of docment d k in concept c j as follows: weight( c j ) cos( dk, c j ) where c j is a concept of the ontology represented as a vector of terms issed from the web pages associated to the concept and weight(c j ) is the weight associated to concept c j. More details on ontology concept representation are presented in [35]. connecting the top n weighted concepts sing different edge types of the reference ontology, and activating the associated general concepts in order to represent the docment content at both general and specific concept levels. Figre 1 cold also represent an example of a docment representation model sing the reference ontology Or session-based personalized search sing a graph-based distance measre Or search personalization approach integrates a short term ser profile modeling for representing the ser interest in a crrent search session. Search personalization is then based on exploiting the ser profile in a graph-based docment ranking model for reordering the search reslts of a related qery.

7 Mariam Daod, Lynda Tamine and Mohand Boghanem 7 The overall process of or session-based personalized search cold be described in algorithm 1 as follows. The algorithm incldes a session bondary recognition mechanism for groping related qeries in the same search session, a ser profile modeling across related qeries and a personalized ranking model that represents docments by graphs and makes se of a graph-based distance measre for reordering the docments. Session bondary recognition: each new sbmitted qery is considered in a session identification method proposed in or previos work [35] sing the Kendall rank correlation measre [35]. This latter qantifies a conceptal correlation ΔI between the ser profile G bilt in the crrent session and the new sbmitted qery q s+1 sbmitted by the ser at time s + 1. We choose a threshold σ and consider that the qeries are from the same session if the correlation is above the threshold. Algorithm 1: A general approach for a session-based personalized search sing a graph-based ser profile s 1 q : a qery sbmitted by the ser at time s + 1 s 1 R : the search reslts retrned by the system with respect to ser qery s G : a ser profile bilt at time s in a search session θ : a reference ontology s 1 q FOR EACH new sbmitted qery by ser do Session bondary test: Compte qery-profile conceptal correlation I ( q G s 1 s ) s s 1 q If ΔI σ then // No session bondary is detected: s 1 s 1 1) Re-rank the search reslts R of the qery q sing the ser profile R s 1 FOR EACH docment d in do Bild the docment graph representation G d sing the reference ontology θ G d project( d, ) Calclate the personalized docment score S ( G s, G ) s s S p ( Gd, G ) GraphDis( Gd, G ) Calclate the final docment score S f (d) s 1 s S f ( d) Si ( q, d) (1 ) S p ( Gd, G ) END FOR s 2) Update the ser profile sing the qery profile G G ELSE A new session is detected: s 1 s Reinitialize the ser profile by the qery profile G Gq END IF END FOR p d 1 s s 1 Gq s G User profile modeling: the ser profile is bilt across related qeries sing the relevant docments assessed by the ser as described in section When a new qery is groped in the crrent session, the ser profile is maintained by exploiting the recent ser relevance feedback. Otherwise, the ser profile is reinitialized and rebilt in the new session. Search personalization: when the qery is related to the search session, the ser profile bilt in the crrent session is sed to rerank the retrned docments according to a graph-based distance measre. The docment reordering process is achieved as follows: o bild the docment graph G d sing the ODP ontology (cf. section 3.2.2),

8 Mariam Daod, Lynda Tamine and Mohand Boghanem 8 o p d s o calclate the personalized docment score S ( G, G ) of each retrieved docment d sing a graphbased distance measre, combine the original docment score S i retrned by the system according to a content-based ranking model and its personalized score S p sing a reranking parameter 0 γ 1. When γ has a vale of 1, personalized score is not given any weight. If γ has a vale of 0, the original score is ignored and pre personalized score is considered. 4. A personalized graph-based ranking model sing a semantic ser profile Or search personalization approach exploits the ser profile in a personalized graph-based docment ranking model sing a semantic extension of the distance measre combining minimm common spergraph (MCS) and maximm common sbgraph (mcs). We review in this section the most common known graph-based distance measres, or motivations for sing the combined distance measre and or personalized graph-based docment ranking model Backgrond: Graph-Based Distance Measres The most common known graph-based distance measres are the maximm common sbgraph (mcs) [37], the minimm common spergraph (MCS) [37], or the combined measre sing MCS and mcs [36]. The maximm common sbgraph (mcs) of two graphs g 1 and g 2, is a sbgraph g of g 1 and g 2 based on common concepts and has among all the sbgraphs, the maximm nmber of nodes [38]. The graph-based distance measre based on the sbgraph mcs [37] is given by the following formla: mcs( g1, g2) d( g1, g2) 1 (2) max( g1, g2,) Where g 1 (resp. g 2 ) is the the nmber of nodes in g 1 (resp. in g 2 ). This formla gives lower distance for graphs having large mcs. The minimm common spergraph (MCS) of two graphs, g 1 and g 2, is a graph g that contains both g 1 and g 2 as sbgraphs and that has the minimm nmber of nodes and edges [38]. The distance measre based on only the spergraph (MCS) [38] is given in the following formla: g1 g2 MCS( g1, g2) d( g1, g2) 1 (3) max( g, g ) The distance measre based on combining MCS and mcs is given in [36] as follows: d MMCS ( g1, g2) MCS( g1, g2) mcs( g1, g2) (4) According to this measre, the distance between graphs is lower when the size of the spergraph is smaller and the size of the sbgraph is larger Motivations Or choice of sing the graph-based distance measre combining MCS and mcs in or personalized search model is based on the following assmption: a docment is ranked higher if it recovers the maximm of concepts of the ser profile at both specific and general levels. According to this assmption, docments presenting common general concepts are ranked lower than docments presenting general and specific concepts with the ser profile. Indeed, or choice cold be jstified by the following reasons: The Minimm Common Spergraph (MCS) allows measring the similarity between the docment and the ser profile at general levels even if the graph do not have common concepts. Indeed, the spergraph of two graphs is smaller when the root nodes of both graphs are close in the ontology. The maximm common sbgraph (mcs) allows measring the similarity between the docment and the ser profile at specific levels. Indeed, the sbgraph is larger when the two graphs have common concepts at specific levels, and conseqently this enlarge the sbgraph with common concepts at general levels. 1 2

9 Mariam Daod, Lynda Tamine and Mohand Boghanem 9 Thanks to the ODP ontology, the following assmption for calclating the combined distance measre based on MCS and mcs remains correct: The bigger is the sbgraph mcs between two graphs, the smaller is the spergraph MCS. Compared to previos cited work, or method overcomes the limitations of docment-profile term-based matching [9, 4]. Indeed, while term-based matching introdces the impact of noisy terms from both the docment and the ser profile, or approach alleviates this limitation by performing a graph-based matching at concept-level. Moreover, or approach integrates both common and similar concepts into the personalized score. The personalized approach in [20] se a distance measre between mlti-concept profile derived from the ODP and a docment/rl represented by one concept of the ontology, in which it is already classified. In this approach, the docment collection is restricted to the Web pages in the ontology and the docment-profile distance makes se of average distance in terms of nodes sing isa relationships. While the main advantage of or ranking is to consider a mlti-concept representation of the docment and a semantic similarity between the concepts via different kind of links instead of taking into accont only is-a links. This ranking gives higher score for a docment that has a graph similar or close to the ser profile graph than a docment that has a distant graph from the ser profile graph A graph-based ranking model based on a docment-profile semantic recovery We propose a graph-based ranking model that ranks semantically the docments with respect to a semantic ser profile. The involved ranking model calclates a personalized score of the docment sing an extension of the combined distance measre. We mention that the basic distance measre combining MCS and mcs in formla 4 can t deal with an approximate matching between the docment and the ser profile. Indeed, this measre gives similar distance for docments that have cross links with the ser profile and others that do not have cross links nor common concepts. This is de to the exact recovery assmption sing the common concepts to bild the sbgraph. That is why we arge to extend semantically the sbgraph of two graphs by taking into accont a relative docment-profile semantic recovery. Or intition at this level is to consider two types of recovery in the semantic distance measre: Exact recovery: refers to an exact similarity between the docment and the ser profile. It is calclated by the nmber of common concepts. Approximate recovery: refers to a relative similarity between the docment and the ser profile. It is calclated by the nmber of common related concepts connecting the two graphs, in other terms, those linked with cross links. We take into accont these two types of recovery when calclating the personalized docment rank. We extend first each of the docment s graph and the ser-profile s graph and then compte the distance measre based on the combination of MCS and mcs of the two extended graphs by giving more weight to the direct common concepts Extending the docment-profile common sbgraph sing cross links Formally, let G and G d the graphs representing respectively the ser profile and a docment. As shown in figre 2, G d is a graph of concepts representing docment d and G is a graph of concepts representing the ser profile. The set of concepts of graph G d connected to graph G with cross links represent the extension of graph G. Formally, we define d * the extended graph of graph G with respect to, as follows: G G d Where e ij is the edge linking concept c i to concept c j, S R is the set of symbolic and related concepts of the ODP * d* ontology (cf. Sect.3). We create the extended graph by the same manner as the graph. We obtain two d * G G * G d d* G c G / c i d G d j S extended graphs and that will be sed in the personalized docment ranking model. Figre 2 presents an illstrative example of an extended graph of g 1 from graph g 2 where concepts c 11 and c 14 from g 2 and their links to graph g 1 compose the extension. G e ij R (5) G

10 Mariam Daod, Lynda Tamine and Mohand Boghanem 10 Figre 2: A semantic extended graph throgh cross links Calclating the personalized docment score d * * We se the extended graphs G and G d to calclate the personalized relevance score of a docment based on a semantic distance combining MCS and mcs between the docment and the ser profile. The mcs of the two extended graphs contains initially common concepts called mcs cc and the activated concepts issed from the graph extension called mcs ca that are either the concepts linking the two graphs throgh cross links or the inner concepts linking the concepts of the sbgraph together. In order to distingish the role of the related concepts connected throgh the cross links relative to the direct common concepts between graphs, we sed a decay factor f ca. This factor is calclated atomatically based on the following assmption: the weight of the activated concepts compared to the common ones mst be redced as mch as we have symbolic or related edges connecting the graphs. Indeed, the nmber of activated concepts is high when the nmber of links between graphs or the nmber of common concepts is high. f ca is given as follows: LR fca, L R 1 LR is the set of cross edges linking concepts of the two graphs. Finally, in order to take into accont the graph size difference between the docments compared to the ser profile, we normalized the semantic distance measre between the graphs by dividing over MCS as follows: d * * d * * d * * MCS( G, Gd ) ( mcs( G, Gd ) fca mcsca( G, Gd ) ) S p( G, Gd ) (6) d * * MCS( G, G ) d 5. Experimental evalation setp In this section, we present the experimental evalation setp concerning the evalation objectives, the dataset and the evalation protocol Evalation objectives The experimental evalation of or approach is ndertaken throgh a hybrid evalation combining context simlation and ser stdy as adopted in [39] where we sed real ser data issed from the search log of Exalead 4 web search engine. The evalation methodology and measres are adopted for the evalating the effectiveness of contextal search technologies in the context of the QUAERO project 5. Experiments are condcted to achieve the following objectives: Evalate the ser profile accracy. This step is necessary as the ser profile accracy has an important impact on the personalized search effectiveness. The goal is to assess the relevance of the concepts that represent the ser profile with respect to the associated qeries. Analyzing the ser profile accracy is achieved in terms of

11 Mariam Daod, Lynda Tamine and Mohand Boghanem 11 the qality of the concept rank ordering, the precision and the recall considering different percentages of the top ranked concepts of the ser profile. Evalate the personalized search effectiveness. The goal is to evalate the effectiveness of or personalized search in front of a typical search performed by Yahoo, and that do not integrate any contextal element in the search process. Compare or personalized search approach to other ranking models. A comparative stdy is condcted to evalate the effect of or extended graph-based ranking model to other ranking models based on sing basic graph-based measres or vector-space similarity measres Dataset As there is no pblically available dataset for personalized search evalation prpose, we exploited a search log of a commercial web search engine, namely Exalead, and we extracted the search history of 10 sers collected along three months. Descriptions of search sessions, qeries and docment collection are given below Search sessions As or approach is based on personalizing search across sessions, each session is defined by a seqence of related qeries, where we have selected 25 ser search sessions for all the 10 sers as follows: Each session contains three qeries related to the same ser information need and are sbmitted by the same ser in a chronological order. Each qery has at least one clicked docment as it is the only sorce of evidence to bild the ser profile in a search session. We assme that a docment is relevant if it is clicked by the ser. A preliminary cleaning of the web search log leads to maintain only relevant clicked docments in the search sessions so as the ser profiles which are created from non biased resorce Qeries Qeries are extracted from the ser search sessions. In order to test the personalized search effectiveness along a ser search session, we have divided the qery set per session into a training qery set and a testing qery set. Training qery set: it is sed to learn the ser profile. The first two qeries of each search session are part of the training qery set. Therefore this set contains a total of 50 qeries. Test qery set: it is sed to evalate the retrieval performance. The last formlated ser qery of each session is part of the testing qery set. Therefore this set contains a total of 25 qeries, which is the minimm nmber reqired for validating experimental findings [45]. Qeries have an average length of 3 terms and the ser intent behind these qeries is mostly informational ( Risqes aditifs ) or transactional ( Le borg d oisans hotel ). We notice that related testing and training qeries in the same session are different where in some few cases a single common keyword exists and in other cases, the qeries have completely different keywords. In addition, as these qeries are extracted from a Web search log, this make the qery set consistent for evalating real personalized Web search. Table 1 presents some example of training and testing qeries derived from the same search sessions. Training qeries Q1: Oie, Q2: Oreille interne Q1: Pollinisation, Q2: Vanille cltre Q1:Cocktail menthe - rhm, Q2: cocktail mojito Q1: Arbre orme Q2Graphiose Tesing qery Q3: Risqes aditifs Q3: Abeille et gacho Q3: Cocktail margarita Q3: Lichens bioindicaters Table 1: Examples of training/ testing qeries derived from search sessions.

12 Mariam Daod, Lynda Tamine and Mohand Boghanem Docment collection The docment collection consists of collecting the top 50 reslts retrieved from the pblicly available Yahoo API 6 for each testing qery. Reslts are crawled and each one is represented by its complete content. In or evalation setting, these docments are sed only for reranking the search reslts sing the ser profile. In order to evalate the retrieval effectiveness, the relevance assessments for the testing qeries were given throgh a ser stdy. To do this, 5 compter science stdents of or lab 7 were presented with the set of top 50 reslts retrieved from Yahoo. Each participant was considered as the ser who has formlated the qery and asked to jdge whether each docment was relevant or not according to the sbject of a sbset of testing qeries Evalation protocol As stated above, we evalate both the ser profile accracy and the personalized search effectiveness. In order to achieve this objective, we propose appropriate evalation protocols described as follows How to evalate the ser profile accracy? The goal of evalating the ser profile is to validate their accracy and their exploitation for personalizing search as this evalation has a strong impact on the effectiveness of the personalized approach. Hence, we only look at evalating the ser profile accracy for validation prpose withot comparing its derivation techniqe to other methods. The protocol for evalating the ser profile accracy is inspired from [40] where the evalation is made at mono-profile level assigned for each ser. We adapt this protocol to take into accont mlti-profiles assigned for each ser where the evalation measres are averaged over the profile bilt for each ser. The methodology for evalating the ser profile accracy and the evalation measres are given below. Methodology. We recall that each ser i is associated to mltiple profiles bilt across the ser search sessions. Each profile is bilt in a search session across the training qeries and is represented as a graph of weighted concepts issed from the reference ontology. We are interested in evalating the graph content (the concepts representing the nodes of the graph) by ignoring its strctre as it is issed from reference ontology. i i i Pfl Pfl Pfl, Pfl,..., Pfl i. Each profile is Formally, we consider for each ser i, the set of n profiles i 1, 2 3 n composed of a list of ordered weighted concepts C C,.., C Pfl, j i j1 j 2 jm. For each ser i, the ser profile evalation protocol consists of the following steps: i Assessment step: for each profile Pfl j, one participant assessed each concept C jk as relevant or non-relevant with respect to the associated qeries in the search session. i Local accracy evalation step: this step consists of compting the accracy of each profile Pfl. Global accracy evalation step: this step consists of compting the average accracy of the profiles in Pfl bilt for a ser i across the whole search history. i j Evalation measres. In contrast to the direct se of some common known IR measres for evalating the ser profile accracy [40], where each ser has a mono-profile, we have compted for each ser i an average accracy vale of the set of profiles bilt across the ser search sessions. The evalation measres sed to calclate the ser profile accracy are the following: the average rank of non relevant concepts (ARnkNR( i )), the precision P@x( i ), the recall R@x( i ) at the top x concepts of the ser profile for each ser i and the F-measre averaged over all the sers. Given a ser profile, precision at top x concepts is compted as the fraction of the nmber of relevant concepts among the top x over x, and recall at top x concepts is compted as the fraction of the nmber of relevant concepts among the top x over the total nmber of relevant concepts in the ser profile How to evalate the personalized search effectiveness? The protocol for evalating the personalized search effectiveness consists of a training step and a testing step. Training step: This step consists of learning the ser profiles for each testing qery sing the clicked docments of the corresponding training qeries belonging to the same ser. j 6 search.cpan.org/perldoc?yahoo::search 7

13 Mariam Daod, Lynda Tamine and Mohand Boghanem 13 Testing step: This step consists of evalating the personalized retrieval effectiveness for each testing qery sing the ser profile compared to the baseline search performed by Yahoo search sing only the testing qery. Personalized search is based on reranking the top 50 reslts of Yahoo for each testing qery sing the appropriate ser profile and by combining for each docment its original rank (sorted by Yahoo) and its personalized rank calclated sing or graph-based distance measre. We se the precision at top 10 and top 20 docments (P10, P20) as an evalation metric. We pool together the qeries and jdgments of all the ten sers, so that the evalation reslt will be an average over the whole testing qeries. 6. User profile accracy evalation reslts We have condcted two series of experiments in order to evalate (1) the concept ranking qality and (2) the effect of the concept depth of the reference ontology on the ser profile accracy Concept ranking qality reslts In this experiment, we evalate the concept ranking qality of the ser profile according to the evalation protocol presented in section where concepts are ordered according to their weights in the ser profiles. We then analyze the evalation reslts in terms of the rank of non relevant concepts, the precision, the recall and the F-measre Analyzing the concept rank ordering Similar to the stdy condcted in [40], we analyze the accracy of the ser profile in terms of the average rank of non relevant concepts for each ser i. Table 2 presents for each ser i, the average profile size (Avgc( i )) which is the average nmber of concepts of the profiles in Pfl i associated to ser i, the average rank of non relevant concepts averaged over the set of profiles associated to ser i (ARnk NR ( i )), and the normalized average rank NoARnk NR ( i ) calclated by dividing the average rank of the non relevant concepts over the average profile size. A concept rank ordering that prodces a large vale of ARnk NR ( i ) ranks non relevant concepts frther down the list. As shown in table 2, the average nmber of concepts of the ser profile (avgc( i )) varies between 19 (User 4) and 83 (User 10) and the average rank of non relevant concepts varies between 14 (User 4) and 43 (User 10) with respect to the ser profile size (Avgc( i )). As a measre of the qality of the concept ordering, the normalized ARnk NR ( i ) gives an average rate of placing non relevant concepts frther down the ser profile representation. For all the sers, NoARnk NR ( i ) is above 50%, and reaches the highest vale of 78% (User 4). In average, the obtained vales highlight that the ser profiles were relevant at least at the highly weighted concepts. We mention that the qality of the ser profile measred by the average rank of non-relevant concepts is highly affected by the docment-concept matching accracy. Indeed, a more precise classification of the docments in the session history into the ontology contribtes to a better ser profile qality where non-relevant concepts wold be ranked down or exclded from the ser profile representation. Users Avgc(i) ARnkNR(i) NoARnkNR(i) User User User User User User User User User User Avg Table 2: Average rank of non relevant concepts per ser

14 Mariam Daod, Lynda Tamine and Mohand Boghanem Analyzing the ser profile precision and recall In this section, we analyze the ser profile accracy for each ser i in terms of the average precision and the average recall calclated at different percentages of the top ranked concepts of the ser profile. As it is difficlt to jdge all the concepts of the ODP ontology whether each one is relevant and not inclded in each ser profile, we compted the recall as proposed in [40] where recall at top x concepts is compted as the fraction of the nmber of relevant concepts among the top x over the total nmber of relevant concepts in the ser profile. This cold be an approximation of the global recall calclated over the entire ontology. Figre 3 and Figre 4 show respectively the precision and the recall calclated considering 10%, 20%, 30%,..., 100% of concepts of the ser profiles. Reslts show that the precision of the ser profiles is qite different between the sers and it is variable when considering differrent ctoff concept points. User profile accracy in terms of precision Precision 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percentages of the top ranked concepts ser1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 Figre 3: Precision calclated for each ser at different concept ctoff points User profile accracy in terms of recall 1,2 Recall 1 0,8 0,6 0,4 0,2 0 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percentages of the top ranked concepts ser1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 Figre 4: Recall calclated for each ser at different concept ctoff points Figre 5 shows the average F-measre calclated over all the sers and considering different percentages of the top ranked concepts of the ser profiles (10%, 20%, 30%,..., 100%). Reslts confirm the increasing crve of the ser profile accracy where accracy reaches the highest vale when we consider all the concepts of the ser profiles. Generally, the reslts obtained in this experiment deal with those obtained in [40] and jstify the se of all the concepts of the ser profile to calclate the personalized docment score.