Web Science and additionality

Transcription

1 Admin tuff... Lecture 1: EITN01 Web Intelligence and Information Retrieval Meage, lide, handout, lab manual and link: Contact: Ander Ardö, room: E:3119b (Tueday, Wedneday, Thurday) Ander Ardö EIT Electrical and Information Technology, Lund Univerity Lab at EIT, 4th floor, north, E:4119 Examination 1 / 56 Map of EIT - north end, floor 3 2 / 56 Web Science Web Science and additionality Ander Ardö Not the union of the dicipline Billboard But more than their interection EIT coure ervice dek 3 / 56 4 / 56

2 Related coure (at EIT) Lecture 1 agenda Chapter 1, 3, 4, (13) in Modern Information Retrieval LSI: 1 Information Retrieval 2 IR model 3 Weighting 4 Evaluation 5 / 56 6 / 56 Outline Introduction 1 Information Retrieval 2 IR model 3 Weighting 4 Evaluation Data retrieval which doc contain a et of keyword? well defined emantic a ingle erroneou object implie failure! Information retrieval - IR information about a ubject or topic emantic i frequently looe mall error are tolerated IR ytem: interpret content of information item match querie to document generate a ranking which reflect relevance notion of relevance: IMPORTANT 7 / 56 8 / 56

3 Cognitive IS & R model Information Retrieval - IR Information retrieval - IR root back to the 1950 ie trongly related to computer cience reearch Goal of IR i to retrieve all relevant document in a collection for a particular uer with a particular information need. IR i a field that ha exited ince computer were firt ued to count word. Statitical approach with the overall aumption that the ubject of a document can be expreed a the um of it word and term Advent of the Web: univeral repoitory of knowledge free (low cot) univeral acce no central editorial board Hard to locate pecific information: IR een a key to finding the olution! 9 / / 56 IR definition Definition of Information Retrieval (IR) Document an ordered et of term, expreing one or more ubject in natural language Collection a corpu of document Term a emantic unit, a word, a phrae, or the tem of a word Indexing electing term that repreent a document Databae an indexed Collection Query a quetion put to a Databae about one or more ubject Match the degree of imilarity between a query and a document Ranking a et of document, ordered by the degree of the match Recall the fraction of all relevant document retrieved Preciion the fraction of all retrieved document that are relevant Information retrieval (IR) i finding material (uually document) of an untructured nature (uually text) that atifie an information need from within large collection (uually tored on computer). The ad hoc retrieval problem: Given a uer information need and a collection of document, the IR ytem determine how well the document atify the query and return a ubet of relevant document to the uer. 11 / / 56

4 IR ytem IR proce Text object Information Need Repreentation Repreentation Document proceor Query proceor Query Indexed object Comparion Matching function Evaluation Relevance Feedback function Retrieved object Ranking capability 13 / 56 IR - imple approach 14 / 56 Indexing Find term in document and match them with term in the query (bag-of-word!!) The eential problem i to match the language in the query with the language in the document The core of IR and automatic indexing: Document that hare the ame vocabulary to a degree are related emantically! Indexing principle To repreent the content of a document To bring document with the ame content together Ait in evaluating the relevance of retrieved document Baically: create indexe to upport earching Manual indexing: aigned decriptor (typically controlled) Automatic indexing: extraction of keyword (uncontrolled) Depite it implicity and obviou theoretical flaw, thi approach i fiendihly effective!! (Mark Sanderon, Uni. Sheffield) What i the difference between manual/automatic indexing? 15 / / 56

5 Repreentation/Indexing Algorithmic IR Document Search requet Document text break into word document number word toplit and *field number non-toplit temming* Lexical analyi word temmed term weighting* word * Indicate optional operation term with weight aign document ID Index / databae 43 Relevance aement Recall bae Kalervo Järvelin, 2001 Databae Matching Evaluation Repreentation Repreentation Query Reult Evaluation Reult 17 / / 56 How to repreent indexed document? Inverted indexe Document aign document ID text break into word document number word toplit and *field number non-toplit temming* Lexical analyi word temmed term weighting* word doc t1 t2 t3 D D D D D D D D D D Term D1 D2 D3 D4 D5 D6 D7 t t t * Indicate optional operation term with weight Index / databae / / 56

6 Outline IR model - overview (ee alo fig 3.2) 1 Information Retrieval 2 IR model 3 Weighting 4 Evaluation U e r T a k Retrieval: Ad hoc Filtering Browing Claic model Boolean Vector pace Probabilitic Structured model Non-overlapping Lit Proximal node Browing Flat Structure Guided Hypertext Set theoretic Fuzzy Extended Boolean Algebraic Generalized vector LSI Neural Network Probabilitic Inference Network Belief Network 21 / / 56 Boolean model Drawback of Boolean model Simple model baed on et theory Document are et of term (they are either preent or abent) Querie pecified a Boolean expreion Precie emantic - document either match query or not Neat formalim Primary commercial IR model for 3 decade Retrieval baed on binary deciion criteria with no notion of partial matching No ranking of the document (abence of a grading cale) Query ha to be a Boolean expreion (awkward for mot uer) Boolean querie formulated by uer are often too implitic A a conequence, the Boolean model frequently return either too few or too many document in repone to a uer query 23 / / 56

7 IR model - vector pace model Vector pace model U e r T a k Retrieval: Ad hoc Filtering Browing Claic model Boolean Vector pace Probabilitic Structured model Non-overlapping Lit Proximal node Browing Flat Structure Guided Hypertext Set theoretic Fuzzy Extended Boolean Algebraic Generalized vector LSI Neural Network Probabilitic Inference Network Belief Network Key idea 1 - Each document (or query) i repreented by a vector Vector value are Boolean or weight Term Poition Value the carnivorou plant faq grow in beautiful n-dimenional pace n = total number of different term in collection Key idea 2 - Similarity between document and query i angle between their vector (proximity) ort hit-lit (ranking) 25 / / 56 Vector pace model - implified Vector pace - coine imilarity The imilarity between two vector i calculated by ue of the coine formula im(d j, q) = d j q d = wi,j w i,q j q w 2 i,j wi,q 2 Baically, it calculate the angle between two vector Aumption: Vector cloe (mall angle) document imilar Simplified binary matrix and vector in n-dimenional pace Auming a modet 10 6 document corpu, matrix Spare term/document matrix 27 / / 56

8 Vector pace model - problem? IR model - probabilitic model Ued a lot BUT No real theoretical bai auming a term pace. Vector more for viualization than having any real bai Mot imilarity meaure work about the ame Term are not really orthogonal dimenion Term are not independent of all other term Synonym/homonym generate noie No document tructure And yet Work urpriingly good! U e r T a k Retrieval: Ad hoc Filtering Browing Claic model Boolean Vector pace Probabilitic Structured model Non-overlapping Lit Proximal node Browing Flat Structure Guided Hypertext Set theoretic Fuzzy Extended Boolean Algebraic Generalized vector LSI Neural Network Probabilitic Inference Network Belief Network 29 / / 56 Probabilitic retrieval IR model - LSI BUT IR ytem make an uncertain gue if document are relevant Ue probability theory to reaon about uncertainty Rigorou formal model attempt to predict the probability that a given document will be relevant to a given query Rank retrieved document according to thi probability of relevance (Probability Ranking Principle) Rely on accurate etimate of probabilitie Experimental/reearch No major improvement compared to Vector pace model Not really ued U e r T a k Retrieval: Ad hoc Filtering Browing Claic model Boolean Vector pace Probabilitic Structured model Non-overlapping Lit Proximal node Browing Flat Structure Guided Hypertext Set theoretic Fuzzy Extended Boolean Algebraic Generalized vector LSI Neural Network Probabilitic Inference Network Belief Network 31 / / 56

9 Concept Concept Claic IR might lead to poor retrieval due to: unrelated document might be included in the anwer et relevant document that do not contain at leat one index term are not retrieved Reaoning: retrieval baed on index term i vague and noiy The uer information need i more related to concept and idea than to index term A document that hare concept with another document known to be relevant might be of interet Key idea: map document and querie into a lower dimenional pace Ue high level concept (fewer than the index term) a dimenion Retrieval in thi reduced concept pace hould be uperior to retrieval in the pace of index term 33 / / 56 Latent Semantic Indexing - LSI Singular Value Decompoition - SVD Dimenionality reduction of a term-document matrix Decompoition to dimenion uing Singular Value Decompoition (SVD) Map term and document to low dimenional repreentation in a new reduced vector pace Term are no longer bai vector The reduced vector pace reflect latent emantic relation between document and term The relation depend on complex and indirect aociation between term and document... one that come from an analyi of the tructure of the whole et of relation in the term/document matrix LSI find higher order co-occurrence tructure The new dimenion are latent concept The dicarded dimenion are redundant Retrieval i imilar to normal vector pace A = B C D 35 / / 56

10 LSI - pro and con Outline LSI i optimal: optimal olution for given dimenionality Caveat: Mathematically optimal i not necearily emantically optimal. Maintenance challenge Changing collection Re-compute in interval? Key benefit Enhance recall, addree ynonymy problem But can decreae preciion LSI ha been teted and found to be modetly effective with traditional tet collection. Permit compact torage/repreentation (vector are typically 100 element intead of ) 1 Information Retrieval 2 IR model 3 Weighting 4 Evaluation 37 / / 56 Weighting Weighting - within document How important i a term? Repetition i an indication of emphai The implet notion of an index weight i binary either a term i aociated with a document or it i not But it i natural to imagine degree of aboutne. Strength of term/document aociation = weight Weight can be aigned to both index term and query term Weighting i ued to expre aboutne and ultimately calculate imilarity between query and document within-document term-frequency count freq i,j : number of occurrence of term k i in document d j May be normalized by document length (len(d) : the total number of term-occurrence in document d) How often doe the term occur in the document? 39 / / 56

11 Weighting - adapting to the collection Weighting principle within-collection term-frequency count n i : the number of document in the collection in which a given term k i occur May be normalized by collection ize N: the total number of document in the collection How many document have the term? The term that hould be weighted mot trongly are thoe that occur relatively frequently in the document, but relatively rarely in the collection Term that occur frequently in a document are more indicative of the ubject of the document o term with high within-document term-frequency count hould be weighted more trongly Term that occur in many document are not good dicriminator (of relevant from irrelevant document) o term with high within-collection term-frequency count hould be weighted le trongly 41 / / 56 Weighting term frequency TF * IDF invere document frequency We have the following parameter to work with: Preence veru Frequency of occurrence in document Collection term document frequency Document length Collection ize Different weighting cheme work with the ame parameter, but uing different tatitic, thu yielding different reult! The mot widely ued i the o-called TF*IDF weighting cheme weight term highly if they are: frequent in the document (TF) AND infrequent in the collection a a whole (IDF) One (of many) formulation of TF * IDF: w i,j = TF i,j IDF i = f i,j log(n/n i ) Definition w i,j = weight of term i (k i ) in document j (d j ) TF i,j = term frequency (f i,j ) IDF i = invere document frequency (log(n/n i )) f i,j = normalized frequency of term i (k i ) in document j (d j ) N = total number of document in the collection n i = the number of document that contain term i (k i ) k i = term i d j = document j Depend on term, document, and collection 43 / / 56

12 TF * IDF: TF factor TF * IDF: IDF factor TF = the term frequency weight TF = freq i,j Variation/Normalization TF = (1 + log(freq i,j )) TF = f i,j = freq i,j /max i (freq i,j ) TF = freq i,j /len(d j ) TF = (N freq i,j )/ len(d j ) The TF weight for a term i varie from document to document. IDF = the invere document frequency weight IDF = 1/n i Variation/Normalization IDF = N/n i IDF = idf i = log(n/n i ): the logarithm moderate the influence of a large N (or a mall n i ) IDF = log(n/n i ) + 1: adding 1 prevent the IDF value from being 0 in cae where n i = N ome take the log to bae 2, other to bae The IDF weight for a term i i contant from document to document. But depend on the entire collection. 45 / / 56 Normalization Outline Normalize term weight o that longer document are not unfairly given more weight: Normalize with repect to the total number of document in the corpu: tf w i,j = i,j log(n/n i ) (tfk,j ) 2 (log(n/n k )) 2 Normalize againt the maximum document frequency (i.e., the mot document any term appear in). Normalize with repect to the number of document not containing a term N D(i) and add a contant of 0.5 to both numerator and denominator to moderate extreme value. 1 Information Retrieval 2 IR model 3 Weighting 4 Evaluation 47 / / 56

13 How compare IR ytem? Retrieval evaluation Search requet Relevance aement Document Repreentation Repreentation Databae Query Matching Reult Recall bae Evaluation Evaluation Reult Kalervo Järvelin, / / 56 Recall/Preciion Preciion-Recall graph From Manning, et. al., Introduction to Information Retrieval, / / 56

14 Preciion-Recall graph interpolated Recall/Preciion Real ytem from TREC. A recall preciion converely A recall preciion Why thi invere relationhip between recall and preciion? From Manning, et. al., Introduction to Information Retrieval, / / 56 Combine Preciion/Recall Text REtrieval Conference - TREC Combine them into F-core F = Which i bet Preciion or Recall? BOTH 2 preciion recall preciion + recall = harmonic mean reference collection of document, querie, and relevant judgment yearly workhop for evaluation of ytem Top 4, 2004: F = 2 preciion recall preciion + recall = harmonic mean Organization F Language Computer Corp National Univ. of Singapore Univ. of Wale, Bangor IBM Reearch / / 56