Knowledge Discovery and Data Mining 1 (KU)

Transcription

1 Knowledge Discovery and Data Mining 1 (KU) Simon Walk IICM, TU Graz October 22, 2015 Simon Walk (IICM) KDDM1 October 22, / 11

2 KDDM 1 (KU) - Introduction Introduction Institute for Information Systems & Computer Media Inffeldgasse 16c/I Office: D simon.walk@tugraz.at Simon Walk Research Interests: Knowledge & Data Mining Social Network Analysis Semantic Web & Ontologies Dynamical Systems & Complex Networks Machine Learning Simon Walk (IICM) KDDM1 October 22, / 11

3 KDDM 1 (KU) - Introduction Course Context & Goals Why should you be interested in KDDM1 (KU)? To consolidate and reinforce your (theoretical) knowledge obtained in KDDM1 (VO) with practical hands-on experience. Helps a LOT for the final exam! Good preparation for KDDM2! Feel like a data scientist! If interested: Continue with Master Project or Master s Thesis Simon Walk (IICM) KDDM1 October 22, / 11

4 KDDM 1 (KU) - Organization Course Organization You have to 1. form small groups of up to two students. 2. choose one of two practical assignments. 3. work on your chosen assignment. 4. give two presentations (in english) on the progress and results of your assignment. After forming a group, send one to simon.walk@tugraz.at and include the names and student ids (Matrikelnummern) of the group. All s have to include [KDDM1] in the subject! Simon Walk (IICM) KDDM1 October 22, / 11

5 Project 1 - Crawling, Cleaning and Clustering Objective: Group (semantically) similar pages of a website according to their most relevant terms! Write a web-crawler to collect pages/documents that contain text. Clean the crawled pages from all markup languages and unwanted content (e.g., HTML, JavaScript, etc.). Calculate similarities between the pages (i.e., by calculating similarities between the TF-IDF Vectors for each page) Group similar pages (i.e., by using a clustering algorithm, such as k-means) Hint: Python, scikit-learn 1, SciPy 2 and NumPy 3 already provide you with most of the functionality required to solve this task! Simon Walk (IICM) KDDM1 October 22, / 11

6 Project 1 - Crawling, Cleaning and Clustering A word of warning: Be careful when crawling websites! Don t hammer the servers or you might risk getting banned! Either select smaller websites for crawling (complete crawl) or choose an appropriate sampling strategy for selecting the pages to analyze! Rule of thumb: Your datasets should consist of, at least, 1,000 pages! Simon Walk (IICM) KDDM1 October 22, / 11

7 Project 2 - Movie Recommender Objective: Recommend similar movies to users, using matrix factorization! Crawl or download 4 a movie-ratings dataset. Create/Extract the required utility matrix and minimize noise (e.g., subtract averages). Perform UV Decomposition to obtain U R n d and V R d m with d = 2 or d = 3. Plot and interpret findings. Hint: Python, scikit-learn, SciPy and NumPy already provide you with many of the functions and tools required to solve this task! 4 We suggest to use MovieLens 100k Simon Walk (IICM) KDDM1 October 22, / 11

8 Project Presentations Will take place after Partial Exam 2 & 3 on and For prepare a 5-minute presentation (strict) with 3 slides: First slide: Dataset Second slide: Experimental Setup Third slide: preliminary results For prepare a 10-minute presentation (strict) with 5 slides: First slide: Introduction/Motivation Second slide: Methodology Third slide: Experimental setup Fourth slide: Results Fifth slide: Discussion Simon Walk (IICM) KDDM1 October 22, / 11

9 Project Presentations Send the slides to as PDF until :59 for presentation 1 and :59 for presentation 2. Subject of the must include [KDDM1]. Note that presentations that take longer than 5 or 10 minutes will be interrupted and stopped! Grading for the KU depends on your presentation and your results! Simon Walk (IICM) KDDM1 October 22, / 11

10 Questions? Simon Walk (IICM) KDDM1 October 22, / 11

11 Thanks! Simon Walk (IICM) KDDM1 October 22, / 11