Knowledge Discovery and Data Mining 1 (VO) ( )
|
|
|
- Dwayne Ward
- 6 years ago
- Views:
Transcription
1 Knowledge Discovery and Data Mining 1 (VO) ( ) Data Matrices and Vector Space Model Denis Helic KTI, TU Graz Nov 6, 2014 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
2 Big picture: KDDM Probability Theory Linear Algebra Hardware & Programming Model Information Theory Statistical Inference Mathematical Tools Infrastructure Preprocessing Transformation Knowledge Discovery Process Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
3 Outline 1 Recap 2 Data Representation 3 Data Matrix 4 Vector Space Model: Document-Term Matrix (Running Example 5 Recommender Systems: Utility Matrix (Running Example 2) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
4 Recap Recap Recap Review of the preprocessing, feature extraction and selection phase Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
5 Recap Recap Preprocessing Initial phase of the Knowledge Discovery process... acquire the data to be analyzed e.g. by crawling the data from the Web... prepare the data e.g. by cleaning and removing outliers Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
6 Recap Recap Feature Extraction Example of features: Images colors, textures, contours,... Signals frequency, phase, samples, spectrum,... Time series ticks, trends, self-similarities,... Biomed dna sequence, genes,... Text words, POS tags, grammatical dependencies,... Features encode these properties in a way suitable for a chosen algorithm Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
7 Recap Feature Selection Recap Approach: select the sub-set of all features without redundant or irrelevant features Unsupervised, e.g. heuristics (black & white lists, score for ranking,... ) Supervised, e.g. using a training data set (calculate measure to assess how discriminative is a feature, e.g. information gain) Penalty for more features, e.g. regularization Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
8 Recap Text mining Recap Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
9 Data Representation Representing data Once when we have the features that we want: how can we represent them? Two representations: 1 Mathematical (model, analyze, and reason about data and algorithms in a formal way) 2 Representation in computers (data structures, implementation, performance) In this course we concentrate on mathematical models KDDM2 is about second representation Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
10 Data Representation Representing data Given: Preprocessed data objects as a set of features E.g. for text documents set of words, bigrams, n-grams,... Given: Feature statistics for each data object E.g. number of occurrences, magnitudes, ticks,... Find: Mathematical model for calculations E.g. similarity, distance, add, subtract, transform,... Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
11 Data Representation Representing data Let us think (geometrically) about features as dimensions (coordinates) in an m-dimensional space For two features we have 2D-space, for three features we have 3D-space, and so on For numeric features the feature statistics will be numbers coming from e.g. R Then each data object is a point in the m-dimensional space The value of a given feature is then the value for its coordinate in this m-dimensional space Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
12 Data Representation Representing data: Example Text documents Suppose we have the following documents: DocID d1 d2 d3 d4 d5 Document Chinese Chinese Chinese Chinese Tokyo Tokyo Beijing Beijing Beijing Chinese Tokyo Tokyo Tokyo Beijing Beijing Tokyo Tokyo Tokyo Tokyo Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
13 Data Representation Representing data: Example Now, we take words as features We take word occurrences as the feature values Three features: Beijing, Chinese, Tokyo Doc Feature Beijing Chinese Tokyo d d d d d Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
14 Data Representation Representing data: Example Geometric representation of data d4:(0,0,3) d3:(2,1,3) 3 d5:(0,0, d1:(0,2,0) d2:(3,2,2) 2 1 z(tokyo) x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
15 Data Matrix Representing data Now, an intuitive representation of the data is a matrix In a general case Columns correspond to features, i.e. dimensions or coordinates in an m-dimensional space Rows correspond to data objects, i.e. data points An element d ij in the i-th row and the j-th column is the j-th coordinate of the i-th data point The data is represented by a matrix D R n m, where n is the number of data points and m the number of features Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
16 Data Matrix Representing data For text Columns correspond to terms, words, and so on I.e. each word is a dimension or a coordinate in an m-dimensional space Rows correspond to documents An element d ij in the i-th row and the j-th column is the e.g. number of occurrences of the j-th word in the i-th document This matrix is called document-term matrix Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
17 Data Matrix Representing data: example D = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
18 Vector interpretation Alternative and very useful interpretation of data matrices is in terms of vectors Each data point, i.e. each row in the data matrix can be interpreted as a vector in an m-dimensional space This allows us to work with: 1 the vector direction, i.e. the line connecting the origin and a given data point 2 the vector magnitude (length), i.e. the distance from the origin and a given data point feature weighting 3 similarity, distance, correlations between vectors 4 manipulate vectors, and so on. Vector space model Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
19 Vector interpretation: Example Geometric representation of data d4:(0,0,3) d3:(2,1,3) 3 d5:(0,0, d1:(0,2,0) d2:(3,2,2) 2 1 z(tokyo) x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
20 Vector space model: feature weighting What we did so far is that we counted word occurrences and used these counts as the coordinates in a given dimension This weighting scheme is referred to as term frequency (TF) Three features: Beijing, Chinese, Tokyo Doc Feature Beijing Chinese Tokyo d d d d d Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
21 Vector space model: TF Please note the difference between the vector space model with TF weighting and e.g. Standard boolean model in information retrieval Standard boolean model uses binary feature values If the feature j occurs in the document i (regardless how many times) then d ij = 1 Otherwise d ij = 0 Allows very simple manipulation of the model, but does not capture the relative importance of the feature j for the document i If a feature occurs more frequently in a document then it is more important for that document and TF captures this intuition Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
22 Vector space model: feature weighting However, raw TF has a critical problem: all terms are considered equally important for assessing relevance In fact, certain terms have no discriminative power at all considering relevance For example, a collection of documents on the auto industry will have most probably term auto in every document We need to penalize terms which occur too often in the complete collection We start with the notion of document frequency (DF) of a term This is the number of documents that contain a given term Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
23 Vector space model: IDF If a term is discriminative then it will only appear in a small number of documents I.e. its DF will be low, or its inverse document frequency (IDF) will be high In practice we typically calculate IDF as: IDF j = log( N DF j ), where N is the number of documents and DF j is the document frequency of the term j. Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
24 Vector space model: TFxIDF Finally, we combine TF and IDF to produce a composite weight for a given feature TFxIDF ij = TF ij log( N DF j ) Now, this quantity will be high if the feature j occurs only a couple of times in the whole collection, and most of these occurrences are in the document i Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
25 TFxIDF: Example Text documents Suppose we have the following documents: DocID d1 d2 d3 d4 d5 Document Chinese Chinese Chinese Chinese Tokyo Tokyo Beijing Beijing Beijing Chinese Tokyo Tokyo Tokyo Beijing Beijing Tokyo Tokyo Tokyo Tokyo Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
26 TFxIDF: Example Three features: Beijing, Chinese, Tokyo TF: Doc Feature Beijing Chinese Tokyo d d d d d Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
27 TFxIDF: Example Three features: Beijing, Chinese, Tokyo DF: DF Beijing = 2 DF Chinese = 3 DF Tokyo = 4 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
28 TFxIDF: Example Three features: Beijing, Chinese, Tokyo IDF: IDF j = log( N DF j ) IDF Beijing = IDF Chinese = IDF Tokyo = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
29 TFxIDF: Example Three features: Beijing, Chinese, Tokyo TFxIDF: TFxIDF ij = TF ij log( N DF j ) Doc Feature Beijing Chinese Tokyo d d d d d Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
30 TFxIDF: Example Geometric representation of data 1.0 d4 d5 d1 d3 d z(tokyo) x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
31 TFxIDF: Example Geometric representation of data d4:(0,0,3) d3:(2,1,3) 3 d5:(0,0, d1:(0,2,0) d2:(3,2,2) 2 1 z(tokyo) x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
32 Vector space model: TFxIDF Further TF variants are possible Sub-linear TF scaling, e.g. TF or 1 + log(tf ) Unlikely that e.g. 20 occurrences of a single term carry 20 times the weight of a single occurrence Also very often TF-normalization by e.g. dividing by the maximal TF in the collection Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
33 Vector space model: vector manipulation Vector addition: concatenate two documents Vector subtraction: diff two documents Transform the vector space in a new vector space: projection and feature selection Similarity calculations Distance calculations Scaling for visualization Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
34 Vector space model: similarity and distance Similarity and distance allow us to order documents This is useful in many applications: 1 Relevance ranking (information retrieval) 2 Classification 3 Clustering 4 Transformation and projection Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
35 Properties of similarity and distance Definition Let D be the set of all data objects, e.g. all documents. Similarity and distance are functions sim : DxD R, dist : DxD R with the following properties: Symmetry Self-similarity (self-distance) sim(d i, d j ) = sim(d j, d i ) dist(d i, d j ) = dist(d j, d i ) sim(d i, d j ) sim(d i, d i ) = sim(d j, d j ) dist(d i, d j ) dist(d i, d i ) = dist(d j, d j ) = 0 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
36 Properties of similarity and distance Additional properties Some additional properties of the sim function: Normalization of similarity to the interval [0, 1] sim(d i, d j ) 0 sim(d i, d i ) = 1 Normalization of similarity to the interval [ 1, 1] sim(d i, d j ) 1 sim(d i, d i ) = 1 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
37 Properties of similarity and distance Additional properties Some additional properties of the dist functions: Triangle inequality dist(d i, d j ) dist(d i, d k ) + dist(d k, d j ) If triangle inequality is satisfied then dist function defines a norm on the vector space Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
38 Euclidean distance Norm We can apply the Euclidean or l 2 norm as a distance metric in the vector space model. x 2 = n i=1 x 2 i x 2 = x T x Euclidean distance of two vectors d i and d j is the length of their displacement vector x = d i d j Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
39 Euclidean distance: Example Geometric representation of data d4:(0,0,3) d3:(2,1,3) 3 d5:(0,0, d1:(0,2,0) d2:(3,2,2) 2 1 z(tokyo) x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
40 Euclidean distance d T 1 d T n d T D = 2. Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
41 Euclidean distance D = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
42 Euclidean distance dist = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
43 Cosine similarity Angle between vectors We can think of the angle ϕ between two vectors d i and d j as a measure of their similarity. Smaller angles mean that the vectors point in two directions that are close to each other Thus, smaller angles mean that the vectors are more similar Larger angles imply smaller similarity To obtain a numerical value from the interval [0, 1] we can calculate cosϕ Cosine of an angle can be also negative! Why do we always get the values from [0, 1] Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
44 Cosine similarity: Example Geometric representation of data d4:(0,0,3) d3:(2,1,3) 3 d5:(0,0, d1:(0,2,0) ϕ 2,3 d2:(3,2,2) 2 1 z(tokyo) ϕ 1, x(beijing) y(chinese) Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
45 Cosine similarity We can calculate the cosine similarity we make use of the dot product and Euclidean norm: sim(d i, d j ) = d T i d j d i 2 d j 2 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
46 Cosine similarity D = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
47 Cosine similarity sim = Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
48 Recommender Systems: Utility Matrix (Running Example 2) Representing data as matrices There are many other sources of the data that can be represented as large matrices We use also matrices to represent the social networks, etc. Measurement data, and much more In recommender systems, we represent user ratings of items as a utility matrix Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
49 Recommender Systems: Utility Matrix (Running Example 2) Recommender systems Recommender systems predict user responses to options E.g. recommend news articles based on prediction of user interests E.g. recommend products based on the predictions of what user might like Recommender systems are necessary whenever users interact with huge catalogs of items E.g. thousands, even millions of products, movies, music, and so on. Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
50 Recommender Systems: Utility Matrix (Running Example 2) Recommender systems These huge numbers of items arise because online you can interact also with the long tail items These are not available in retail because of e.g. limited shelf space Recommender systems try to support this interaction by suggesting certain items to the user The suggestions or recommendations are based on what the systems know about the user and about the items Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
51 Recommender Systems: Utility Matrix (Running Example 2) Formal model: the utility matrix In a recommender system there are two classes of entities: users and items Utility Function Let us denote the set of all users with U and the set of all items with I. We define the utility function u : UxI R, where R is a set of ratings and is a totally ordered set. For example, R = {1, 2, 3, 4, 5} set of star ratings, or R = [0, 1] set of real numbers from that interval. Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
52 Recommender Systems: Utility Matrix (Running Example 2) The Utility Matrix The utility function maps pairs of users and items to numbers These numbers can be represented by a utility matrix M R n m, where n is the number of users and m the number of items The matrix gives a value for each user-item pair where we know about the preference of that user for that item E.g. the values can come from an ordered set (1 to 5) and represent a rating that a user gave for an item Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
53 Recommender Systems: Utility Matrix (Running Example 2) The Utility Matrix We assume that the matrix is sparse This means that most entries are unknown The majority of the user preferences for specific items is unknown An unknown rating means that we do not have explicit information It does not mean that the rating is low Formally: the goal of a recommender system is to predict the blank in the utility matrix Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
54 Recommender Systems: Utility Matrix (Running Example 2) The Utility Matrix: example User Movie HP1 HP2 HP3 Hobbit SW1 SW2 SW3 A B C D 3 3 Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
55 Recommender Systems: Utility Matrix (Running Example 2) Recommender systems: short summary Three key problems in the recommender systems: Collecting data for the utility matrix 1 Explicit by e.g. collecting ratings 2 Implicit by e.g. interactions (users buy a product implies a high rating) Predicting missing values in the utility matrix 1 Problems are sparsity, cold start, and so on. 2 Content-based, collaborative filtering, matrix factorization Evaluating predictions 1 Training dataset, test dataset, error Denis Helic (KTI, TU Graz) KDDM1 Nov 6, / 55
Thanks to Jure Leskovec, Anand Rajaraman, Jeff Ullman
Thanks to Jure Leskovec, Anand Rajaraman, Jeff Ullman http://www.mmds.org Overview of Recommender Systems Content-based Systems Collaborative Filtering J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive
BBS654 Data Mining. Pinar Duygulu
BBS6 Data Mining Pinar Duygulu Slides are adapted from J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, http://www.mmds.org Mustafa Ozdal Example: Recommender Systems Customer X Buys Metallica
Instructor: Stefan Savev
LECTURE 2 What is indexing? Indexing is the process of extracting features (such as word counts) from the documents (in other words: preprocessing the documents). The process ends with putting the information
Introduction to Data Mining
Introduction to Data Mining Lecture #7: Recommendation Content based & Collaborative Filtering Seoul National University In This Lecture Understand the motivation and the problem of recommendation Compare
This lecture: IIR Sections Ranked retrieval Scoring documents Term frequency Collection statistics Weighting schemes Vector space scoring
This lecture: IIR Sections 6.2 6.4.3 Ranked retrieval Scoring documents Term frequency Collection statistics Weighting schemes Vector space scoring 1 Ch. 6 Ranked retrieval Thus far, our queries have all
Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University Infinite data. Filtering data streams
/9/7 Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them
String Vector based KNN for Text Categorization
458 String Vector based KNN for Text Categorization Taeho Jo Department of Computer and Information Communication Engineering Hongik University Sejong, South Korea tjo018@hongik.ac.kr Abstract This research
CS 124/LINGUIST 180 From Languages to Information
CS /LINGUIST 80 From Languages to Information Dan Jurafsky Stanford University Recommender Systems & Collaborative Filtering Slides adapted from Jure Leskovec Recommender Systems Customer X Buys CD of
Keyword Extraction by KNN considering Similarity among Features
64 Int'l Conf. on Advances in Big Data Analytics ABDA'15 Keyword Extraction by KNN considering Similarity among Features Taeho Jo Department of Computer and Information Engineering, Inha University, Incheon,
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
Information Retrieval. (M&S Ch 15)
Information Retrieval (M&S Ch 15) 1 Retrieval Models A retrieval model specifies the details of: Document representation Query representation Retrieval function Determines a notion of relevance. Notion
CS 124/LINGUIST 180 From Languages to Information
CS /LINGUIST 80 From Languages to Information Dan Jurafsky Stanford University Recommender Systems & Collaborative Filtering Slides adapted from Jure Leskovec Recommender Systems Customer X Buys CD of
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS6: Mining Massive Datasets Jure Leskovec, Stanford University http://cs6.stanford.edu Customer X Buys Metalica CD Buys Megadeth CD Customer Y Does search on Metalica Recommender system suggests Megadeth
Vector Space Models: Theory and Applications
Vector Space Models: Theory and Applications Alexander Panchenko Centre de traitement automatique du langage (CENTAL) Université catholique de Louvain FLTR 2620 Introduction au traitement automatique du
Encoding Words into String Vectors for Word Categorization
Int'l Conf. Artificial Intelligence ICAI'16 271 Encoding Words into String Vectors for Word Categorization Taeho Jo Department of Computer and Information Communication Engineering, Hongik University,
CS 124/LINGUIST 180 From Languages to Information
CS /LINGUIST 80 From Languages to Information Dan Jurafsky Stanford University Recommender Systems & Collaborative Filtering Slides adapted from Jure Leskovec Recommender Systems Customer X Buys Metallica
Information Retrieval
Information Retrieval Suan Lee - Information Retrieval - 06 Scoring, Term Weighting and the Vector Space Model 1 Recap of lecture 5 Collection and vocabulary statistics: Heaps and Zipf s laws Dictionary
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS6: Mining Massive Datasets Jure Leskovec, Stanford University http://cs6.stanford.edu //8 Jure Leskovec, Stanford CS6: Mining Massive Datasets High dim. data Graph data Infinite data Machine learning
Near Neighbor Search in High Dimensional Data (1) Dr. Anwar Alhenshiri
Near Neighbor Search in High Dimensional Data (1) Dr. Anwar Alhenshiri Scene Completion Problem The Bare Data Approach High Dimensional Data Many real-world problems Web Search and Text Mining Billions
Natural Language Processing
Natural Language Processing Information Retrieval Potsdam, 14 June 2012 Saeedeh Momtazi Information Systems Group based on the slides of the course book Outline 2 1 Introduction 2 Indexing Block Document
vector space retrieval many slides courtesy James Amherst
vector space retrieval many slides courtesy James Allan@umass Amherst 1 what is a retrieval model? Model is an idealization or abstraction of an actual process Mathematical models are used to study the
Informa(on Retrieval
Introduc)on to Informa)on Retrieval CS3245 Informa(on Retrieval Lecture 7: Scoring, Term Weigh9ng and the Vector Space Model 7 Last Time: Index Compression Collec9on and vocabulary sta9s9cs: Heaps and
COMP6237 Data Mining Searching and Ranking
COMP6237 Data Mining Searching and Ranking Jonathon Hare jsh2@ecs.soton.ac.uk Note: portions of these slides are from those by ChengXiang Cheng Zhai at UIUC https://class.coursera.org/textretrieval-001
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS46: Mining Massive Datasets Jure Leskovec, Stanford University http://cs46.stanford.edu /7/ Jure Leskovec, Stanford C46: Mining Massive Datasets Many real-world problems Web Search and Text Mining Billions
Basic techniques. Text processing; term weighting; vector space model; inverted index; Web Search
Basic techniques Text processing; term weighting; vector space model; inverted index; Web Search Overview Indexes Query Indexing Ranking Results Application Documents User Information analysis Query processing
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
Visualization and text mining of patent and non-patent data
of patent and non-patent data Anton Heijs Information Solutions Delft, The Netherlands http://www.treparel.com/ ICIC conference, Nice, France, 2008 Outline Introduction Applications on patent and non-patent
Boolean Model. Hongning Wang
Boolean Model Hongning Wang CS@UVa Abstraction of search engine architecture Indexed corpus Crawler Ranking procedure Doc Analyzer Doc Representation Query Rep Feedback (Query) Evaluation User Indexer
Chapter 2 Basic Structure of High-Dimensional Spaces
Chapter 2 Basic Structure of High-Dimensional Spaces Data is naturally represented geometrically by associating each record with a point in the space spanned by the attributes. This idea, although simple,
In = number of words appearing exactly n times N = number of words in the collection of words A = a constant. For example, if N=100 and the most
In = number of words appearing exactly n times N = number of words in the collection of words A = a constant. For example, if N=100 and the most common word appears 10 times then A = rn*n/n = 1*10/100
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS6: Mining Massive Datasets Jure Leskovec, Stanford University http://cs6.stanford.edu /7/0 Jure Leskovec, Stanford CS6: Mining Massive Datasets, http://cs6.stanford.edu High dim. data Graph data Infinite
Digital Libraries: Language Technologies
Digital Libraries: Language Technologies RAFFAELLA BERNARDI UNIVERSITÀ DEGLI STUDI DI TRENTO P.ZZA VENEZIA, ROOM: 2.05, E-MAIL: BERNARDI@DISI.UNITN.IT Contents 1 Recall: Inverted Index..........................................
Knowledge Discovery and Data Mining 1 (KU)
Knowledge Discovery and Data Mining 1 (KU) Simon Walk IICM, TU Graz October 22, 2015 Simon Walk (IICM) KDDM1 October 22, 2015 1 / 11 KDDM 1 (KU) - Introduction Introduction Institute for Information Systems
Informa(on Retrieval
Introduc)on to Informa)on Retrieval CS3245 Informa(on Retrieval Lecture 7: Scoring, Term Weigh9ng and the Vector Space Model 7 Last Time: Index Construc9on Sort- based indexing Blocked Sort- Based Indexing
Task Description: Finding Similar Documents. Document Retrieval. Case Study 2: Document Retrieval
Case Study 2: Document Retrieval Task Description: Finding Similar Documents Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade April 11, 2017 Sham Kakade 2017 1 Document
SOCIAL MEDIA MINING. Data Mining Essentials
SOCIAL MEDIA MINING Data Mining Essentials Dear instructors/users of these slides: Please feel free to include these slides in your own material, or modify them as you see fit. If you decide to incorporate
CS473: Course Review CS-473. Luo Si Department of Computer Science Purdue University
CS473: CS-473 Course Review Luo Si Department of Computer Science Purdue University Basic Concepts of IR: Outline Basic Concepts of Information Retrieval: Task definition of Ad-hoc IR Terminologies and
Chapter 6: Information Retrieval and Web Search. An introduction
Chapter 6: Information Retrieval and Web Search An introduction Introduction n Text mining refers to data mining using text documents as data. n Most text mining tasks use Information Retrieval (IR) methods
Introduction to Information Retrieval
Introduction Inverted index Processing Boolean queries Course overview Introduction to Information Retrieval http://informationretrieval.org IIR 1: Boolean Retrieval Hinrich Schütze Institute for Natural
COMP6237 Data Mining Making Recommendations. Jonathon Hare
COMP6237 Data Mining Making Recommendations Jonathon Hare jsh2@ecs.soton.ac.uk Introduction Recommender systems 101 Taxonomy of recommender systems Collaborative Filtering Collecting user preferences as
CSCI 599: Applications of Natural Language Processing Information Retrieval Retrieval Models (Part 1)"
CSCI 599: Applications of Natural Language Processing Information Retrieval Retrieval Models (Part 1)" All slides Addison Wesley, Donald Metzler, and Anton Leuski, 2008, 2012! Retrieval Models" Provide
Information Retrieval
Natural Language Processing SoSe 2014 Information Retrieval Dr. Mariana Neves June 18th, 2014 (based on the slides of Dr. Saeedeh Momtazi) Outline Introduction Indexing Block 2 Document Crawling Text Processing
Unsupervised Feature Selection for Sparse Data
Unsupervised Feature Selection for Sparse Data Artur Ferreira 1,3 Mário Figueiredo 2,3 1- Instituto Superior de Engenharia de Lisboa, Lisboa, PORTUGAL 2- Instituto Superior Técnico, Lisboa, PORTUGAL 3-
Machine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
Data Mining Lecture 2: Recommender Systems
Data Mining Lecture 2: Recommender Systems Jo Houghton ECS Southampton February 19, 2019 1 / 32 Recommender Systems - Introduction Making recommendations: Big Money 35% of Amazons income from recommendations
CS435 Introduction to Big Data Spring 2018 Colorado State University. 3/21/2018 Week 10-B Sangmi Lee Pallickara. FAQs. Collaborative filtering
W10.B.0.0 CS435 Introduction to Big Data W10.B.1 FAQs Term project 5:00PM March 29, 2018 PA2 Recitation: Friday PART 1. LARGE SCALE DATA AALYTICS 4. RECOMMEDATIO SYSTEMS 5. EVALUATIO AD VALIDATIO TECHIQUES
modern database systems lecture 4 : information retrieval
modern database systems lecture 4 : information retrieval Aristides Gionis Michael Mathioudakis spring 2016 in perspective structured data relational data RDBMS MySQL semi-structured data data-graph representation
Information Retrieval
Information Retrieval Natural Language Processing: Lecture 12 30.11.2017 Kairit Sirts Homework 4 things that seemed to work Bidirectional LSTM instead of unidirectional Change LSTM activation to sigmoid
Recommender Systems - Introduction. Data Mining Lecture 2: Recommender Systems
Recommender Systems - Introduction Making recommendations: Big Money 35% of amazons income from recommendations Netflix recommendation engine worth $ Billion per year And yet, Amazon seems to be able to
Clustering and Dimensionality Reduction. Stony Brook University CSE545, Fall 2017
Clustering and Dimensionality Reduction Stony Brook University CSE545, Fall 2017 Goal: Generalize to new data Model New Data? Original Data Does the model accurately reflect new data? Supervised vs. Unsupervised
Information Retrieval CS Lecture 06. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Information Retrieval CS 6900 Lecture 06 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Boolean Retrieval vs. Ranked Retrieval Many users (professionals) prefer
Information Retrieval: Retrieval Models
CS473: Web Information Retrieval & Management CS-473 Web Information Retrieval & Management Information Retrieval: Retrieval Models Luo Si Department of Computer Science Purdue University Retrieval Models
Automatic Data Analysis in Visual Analytics Selected Methods
Automatic Data Analysis in Visual Analytics Selected Methods Multimedia Information Systems 2 VU (SS 2015, 707.025) Vedran Sabol Know-Center March 15 th, 2016 2 Lecture Overview Visual Analytics Overview
10/10/13. Traditional database system. Information Retrieval. Information Retrieval. Information retrieval system? Information Retrieval Issues
COS 597A: Principles of Database and Information Systems Information Retrieval Traditional database system Large integrated collection of data Uniform access/modifcation mechanisms Model of data organization
Predicting Popular Xbox games based on Search Queries of Users
1 Predicting Popular Xbox games based on Search Queries of Users Chinmoy Mandayam and Saahil Shenoy I. INTRODUCTION This project is based on a completed Kaggle competition. Our goal is to predict which
Introduction to Information Retrieval
Introduction to Information Retrieval Mohsen Kamyar چهارمین کارگاه ساالنه آزمایشگاه فناوری و وب بهمن ماه 1391 Outline Outline in classic categorization Information vs. Data Retrieval IR Models Evaluation
Problem 1: Complexity of Update Rules for Logistic Regression
Case Study 1: Estimating Click Probabilities Tackling an Unknown Number of Features with Sketching Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox January 16 th, 2014 1
Basic Elements. Geometry is the study of the relationships among objects in an n-dimensional space
Basic Elements Geometry is the study of the relationships among objects in an n-dimensional space In computer graphics, we are interested in objects that exist in three dimensions We want a minimum set
Automatic Summarization
Automatic Summarization CS 769 Guest Lecture Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of Wisconsin, Madison February 22, 2008 Andrew B. Goldberg (CS Dept) Summarization
9/29/13. Outline Data mining tasks. Clustering algorithms. Applications of clustering in biology
9/9/ I9 Introduction to Bioinformatics, Clustering algorithms Yuzhen Ye (yye@indiana.edu) School of Informatics & Computing, IUB Outline Data mining tasks Predictive tasks vs descriptive tasks Example
Praxis Elementary Education: Mathematics Subtest (5003) Curriculum Crosswalk. Required Course Numbers. Test Content Categories.
Page 1 I. Numbers and Operations (40%) A. Understands the place value system 1. Writes numbers using base-10 numerals, number names, and expanded form 2. Composes and decomposes multi-digit numbers 3.
CSAP Achievement Levels Mathematics Grade 6 March, 2006
Advanced Performance Level 4 (Score range: 589 to 830) Students demonstrate equivalency among fractions decimals, and percents; apply number sense; solve real-world problems using geometric and numeric
Databases 2 (VU) ( / )
Databases 2 (VU) (706.711 / 707.030) MapReduce (Part 3) Mark Kröll ISDS, TU Graz Nov. 27, 2017 Mark Kröll (ISDS, TU Graz) MapReduce Nov. 27, 2017 1 / 42 Outline 1 Problems Suited for Map-Reduce 2 MapReduce:
Lecture on Modeling Tools for Clustering & Regression
Lecture on Modeling Tools for Clustering & Regression CS 590.21 Analysis and Modeling of Brain Networks Department of Computer Science University of Crete Data Clustering Overview Organizing data into
Text classification with Naïve Bayes. Lab 3
Text classification with Naïve Bayes Lab 3 1 The Task Building a model for movies reviews in English for classifying it into positive or negative. Test classifier on new reviews Takes time 2 Sentiment
CS452/552; EE465/505. Geometry Transformations
CS452/552; EE465/505 Geometry Transformations 1-26-15 Outline! Geometry: scalars, points & vectors! Transformations Read: Angel, Chapter 4 (study cube.html/cube.js example) Appendix B: Spaces (vector,
Today s topic CS347. Results list clustering example. Why cluster documents. Clustering documents. Lecture 8 May 7, 2001 Prabhakar Raghavan
Today s topic CS347 Clustering documents Lecture 8 May 7, 2001 Prabhakar Raghavan Why cluster documents Given a corpus, partition it into groups of related docs Recursively, can induce a tree of topics
Big Mathematical Ideas and Understandings
Big Mathematical Ideas and Understandings A Big Idea is a statement of an idea that is central to the learning of mathematics, one that links numerous mathematical understandings into a coherent whole.
Data Mining. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395
Data Mining Introduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 21 Table of contents 1 Introduction 2 Data mining
Document Clustering: Comparison of Similarity Measures
Document Clustering: Comparison of Similarity Measures Shouvik Sachdeva Bhupendra Kastore Indian Institute of Technology, Kanpur CS365 Project, 2014 Outline 1 Introduction The Problem and the Motivation
Introduction to Information Retrieval
Introduction to Information Retrieval (Supplementary Material) Zhou Shuigeng March 23, 2007 Advanced Distributed Computing 1 Text Databases and IR Text databases (document databases) Large collections
CLUSTERING, TIERED INDEXES AND TERM PROXIMITY WEIGHTING IN TEXT-BASED RETRIEVAL
STUDIA UNIV. BABEŞ BOLYAI, INFORMATICA, Volume LVII, Number 4, 2012 CLUSTERING, TIERED INDEXES AND TERM PROXIMITY WEIGHTING IN TEXT-BASED RETRIEVAL IOAN BADARINZA AND ADRIAN STERCA Abstract. In this paper
Information Retrieval
Natural Language Processing SoSe 2015 Information Retrieval Dr. Mariana Neves June 22nd, 2015 (based on the slides of Dr. Saeedeh Momtazi) Outline Introduction Indexing Block 2 Document Crawling Text Processing
Hierarchical Clustering
What is clustering Partitioning of a data set into subsets. A cluster is a group of relatively homogeneous cases or observations Hierarchical Clustering Mikhail Dozmorov Fall 2016 2/61 What is clustering
CPSC 340: Machine Learning and Data Mining. Recommender Systems Fall 2017
CPSC 340: Machine Learning and Data Mining Recommender Systems Fall 2017 Assignment 4: Admin Due tonight, 1 late day for Monday, 2 late days for Wednesday. Assignment 5: Posted, due Monday of last week
Data Mining. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1394
Data Mining Introduction Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1394 1 / 20 Table of contents 1 Introduction 2 Data mining
Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University
Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit
Recommendation Algorithms: Collaborative Filtering. CSE 6111 Presentation Advanced Algorithms Fall Presented by: Farzana Yasmeen
Recommendation Algorithms: Collaborative Filtering CSE 6111 Presentation Advanced Algorithms Fall. 2013 Presented by: Farzana Yasmeen 2013.11.29 Contents What are recommendation algorithms? Recommendations
Natural Language Processing Basics. Yingyu Liang University of Wisconsin-Madison
Natural Language Processing Basics Yingyu Liang University of Wisconsin-Madison Natural language Processing (NLP) The processing of the human languages by computers One of the oldest AI tasks One of the
Outlier Detection Using Unsupervised and Semi-Supervised Technique on High Dimensional Data
Outlier Detection Using Unsupervised and Semi-Supervised Technique on High Dimensional Data Ms. Gayatri Attarde 1, Prof. Aarti Deshpande 2 M. E Student, Department of Computer Engineering, GHRCCEM, University
CSE 6242 A / CX 4242 DVA. March 6, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CX 4242 DVA March 6, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Analyze! Limited memory size! Data may not be fitted to the memory of your machine! Slow computation!
The influence of social filtering in recommender systems
The influence of social filtering in recommender systems 1 Introduction Nick Dekkers 3693406 Recommender systems have become more and more intertwined in our everyday usage of the web. Think about the
CSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2009 Lecture 12 Google Bigtable
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2009 Lecture 12 Google Bigtable References Bigtable: A Distributed Storage System for Structured Data. Fay Chang et. al. OSDI
CS 6320 Natural Language Processing
CS 6320 Natural Language Processing Information Retrieval Yang Liu Slides modified from Ray Mooney s (http://www.cs.utexas.edu/users/mooney/ir-course/slides/) 1 Introduction of IR System components, basic
Models for Document & Query Representation. Ziawasch Abedjan
Models for Document & Query Representation Ziawasch Abedjan Overview Introduction & Definition Boolean retrieval Vector Space Model Probabilistic Information Retrieval Language Model Approach Summary Overview
Clustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
Information Retrieval. CS630 Representing and Accessing Digital Information. What is a Retrieval Model? Basic IR Processes
CS630 Representing and Accessing Digital Information Information Retrieval: Retrieval Models Information Retrieval Basics Data Structures and Access Indexing and Preprocessing Retrieval Models Thorsten
Supervised Random Walks
Supervised Random Walks Pawan Goyal CSE, IITKGP September 8, 2014 Pawan Goyal (IIT Kharagpur) Supervised Random Walks September 8, 2014 1 / 17 Correlation Discovery by random walk Problem definition Estimate
Recommender Systems - Content, Collaborative, Hybrid
BOBBY B. LYLE SCHOOL OF ENGINEERING Department of Engineering Management, Information and Systems EMIS 8331 Advanced Data Mining Recommender Systems - Content, Collaborative, Hybrid Scott F Eisenhart 1
Introduction to Data Mining
Introduction to JULY 2011 Afsaneh Yazdani What motivated? Wide availability of huge amounts of data and the imminent need for turning such data into useful information and knowledge What motivated? Data
Singular Value Decomposition, and Application to Recommender Systems
Singular Value Decomposition, and Application to Recommender Systems CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 Recommendation
K Nearest Neighbor Wrap Up K- Means Clustering. Slides adapted from Prof. Carpuat
K Nearest Neighbor Wrap Up K- Means Clustering Slides adapted from Prof. Carpuat K Nearest Neighbor classification Classification is based on Test instance with Training Data K: number of neighbors that
Grade Level Expectations for the Sunshine State Standards
for the Sunshine State Standards FLORIDA DEPARTMENT OF EDUCATION http://www.myfloridaeducation.com/ The seventh grade student: Number Sense, Concepts, and Operations knows word names and standard numerals
V2: Measures and Metrics (II)
- Betweenness Centrality V2: Measures and Metrics (II) - Groups of Vertices - Transitivity - Reciprocity - Signed Edges and Structural Balance - Similarity - Homophily and Assortative Mixing 1 Betweenness
Midterm Exam Search Engines ( / ) October 20, 2015
Student Name: Andrew ID: Seat Number: Midterm Exam Search Engines (11-442 / 11-642) October 20, 2015 Answer all of the following questions. Each answer should be thorough, complete, and relevant. Points
Cluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
Exploratory Analysis: Clustering
Exploratory Analysis: Clustering (some material taken or adapted from slides by Hinrich Schutze) Heejun Kim June 26, 2018 Clustering objective Grouping documents or instances into subsets or clusters Documents
James Mayfield! The Johns Hopkins University Applied Physics Laboratory The Human Language Technology Center of Excellence!
James Mayfield! The Johns Hopkins University Applied Physics Laboratory The Human Language Technology Center of Excellence! (301) 219-4649 james.mayfield@jhuapl.edu What is Information Retrieval? Evaluation
Information Retrieval (IR) Introduction to Information Retrieval. Lecture Overview. Why do we need IR? Basics of an IR system.
Introduction to Information Retrieval Ethan Phelps-Goodman Some slides taken from http://www.cs.utexas.edu/users/mooney/ir-course/ Information Retrieval (IR) The indexing and retrieval of textual documents.
International Journal of Advanced Research in Computer Science and Software Engineering
Volume 3, Issue 3, March 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue: