IFCS 2009 Dresden 17/03/2009 Visualising a text with a tree cloud
|
|
|
- Ronald Shaw
- 6 years ago
- Views:
Transcription
1 IFCS 2009 Dresden 17/03/2009 Visualising a text with a tree cloud Philippe Gambette, Jean Véronis
2 Outline Tag and word clouds Enhanced tag clouds Tree clouds Construction steps Quality control
3 Tag clouds Built from a set of tags Font size related to frequency What is considered the first tag cloud, from D. Coupland: Microserfs, HarperCollins, Toronto, 1995
4 Tag clouds Built from a set of tags Font size related to frequency Gained popularity with Flickr Flickr's all time most popular tags
5 Word clouds Built from a set of words from a text Font size related to frequency Gained popularity with Wordle
6 Word clouds Built from a set of words from a text Font size related to frequency Gained popularity with Wordle GoogleImage(obama inaugural address wordle)
7 Word clouds Built from a set of words from a text Font size related to frequency Gained popularity with Wordle GoogleImage(obama inaugural address wordle)
8 Word clouds Built from a set of words from a text Font size related to frequency Gained popularity with Wordle GoogleImage(obama inaugural address wordle)
9 Enhanced tag / word clouds Add information from the text: color intensity to express recency in Amazon shared tags in red on del.icio.us group together cooccurring tags on the same line Hassan-Montero & Herrero-Solana, InScit'06 optimize blank space and semantic proximity Kaser & Lemire, WWW'07 topigraphy : 2D placement according to cooccurrence Fujimura, Fujimura, Matsubayashi, Yamada & Okuda, WWW'08
10 Enhanced tag / word clouds Add information from the text: color intensity to express recency in Amazon shared tags in red on del.icio.us group together cooccurring tags on the same line Hassan-Montero & Herrero-Solana, InScit'06 optimize blank space and semantic proximity Kaser & Lemire, WWW'07 topigraphy : 2D placement according to cooccurrence Fujimura, Fujimura, Matsubayashi, Yamada & Okuda, WWW'08
11 Enhanced tag / word clouds Add information from the text: color intensity to express recency in Amazon shared tags in red on del.icio.us group together cooccurring tags on the same line Hassan-Montero & Herrero-Solana, InScit'06 optimize blank space and semantic proximity Kaser & Lemire, WWW'07 topigraphy : 2D placement according to cooccurrence Fujimura, Fujimura, Matsubayashi, Yamada & Okuda, WWW'08
12 Enhanced tag / word clouds Add information from the text: color intensity to express recency in Amazon shared tags in red on del.icio.us group together cooccurring tags on the same line Hassan-Montero & Herrero-Solana, InScit'06 optimize blank space and semantic proximity Kaser & Lemire, WWW'07 topigraphy : 2D placement according to cooccurrence Fujimura, Fujimura, Matsubayashi, Yamada & Okuda, WWW'08
13 Enhanced tag / word clouds Add information from the text: color intensity to express recency in Amazon shared tags in red on del.icio.us group together cooccurring tags on the same line Hassan-Montero & Herrero-Solana, InScit'06 optimize blank space and semantic proximity Kaser & Lemire, WWW'07 topigraphy : 2D placement according to cooccurrence Fujimura, Fujimura, Matsubayashi, Yamada & Okuda, WWW'08
14 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: sense desamibiguation Grimmer (Wordmapper) Véronis (Hyperlex)
15 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: sense desamibiguation Grimmer (Wordmapper) Véronis (Hyperlex) Mayaffre, Quand travail, famille, et patrie cooccurrent dans le discours de Nicolas Sarkozy, JADT'08
16 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: sense desamibiguation Grimmer (Wordmapper) Véronis (Hyperlex) Brunet, Les séquences (suite), JADT'08
17 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: word sense disamibiguation Grimmer (Wordmapper) Véronis (Hyperlex) Barry, Viprey, Approche comparative des résultats d'exploration textuelle des discours de deux leaders africains Keita et Touré, JADT'08
18 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: word sense disamibiguation Grimmer (Wordmapper) Véronis (Hyperlex) Peyrat-Guillard, Analyse du discours syndical sur l entreprise, JADT'08
19 Extract semantic information from a text literature analysis: philological approach: only consider the text Brody discourse analysis: tree analysis or cooccurrence graph, geodesic projection text mining: semantic graph Brunet (Hyperbase), Viprey (Astartex) natural language processing: word sense disambiguation Grimmer (Wordmapper) Véronis (Hyperlex) Disambiguation of word barrage : dam, play-off, roadblock, police cordon. Véronis, HyperLex: Lexical Cartography for Information Retrieval, 2004
20 Tag cloud + tree = tree cloud SplitsTree: Huson 1998, Huson & Bryant 2006 Built with TreeCloud and GPL-licensed Treecloud in Python, available at
21 The first tree cloud Tree cloud of the blog posts containing Laurence Ferrari from 25/11/2007 to 10/12/2007, by Jean Véronis
22 Building a tree cloud extracting the words Extract words with frequency: stoplist? without stoplist with stoplist
23 Building a tree cloud extracting the words Extract words with frequency: lemmatization? groups words with similar meaning? no lemmatization... sometimes interesting 70 most frequent words in Obama's campaign speeches, winsize=30, distance=dice, NJ-tree.
24 Building a tree cloud dissimilarity matrix Many semantic distance formulas based on cooccurrence
25 Building a tree cloud dissimilarity matrix Many semantic distance formulas based on cooccurrence Text sliding window S sliding step s width w cooccurrence matrices O11, O12, O21, O22 semantic dissimilarity matrix chi squared, mutual information, liddel, dice, jaccard, gmean, hyperlex, minimum sensitivity, odds ratio, zscore, log likelihood, poisson-stirling... Evert, Statistics of words cooccurrences, PhD Thesis, 2005
26 Building a tree cloud dissimilarity matrix Transformations needed on the dissimilarity: transform similarity into dissimilarity linear normalization for positive matrices to get distances in [0,1] affine normalization for matrices with positive or negative numbers, to get distances in [α,1] (for example α=0.1)
27 Building a tree cloud tree reconstruction Many existing methods: Neighbor-Joining Addtree variants Quartet heuristic Saitou & Nei, 1987 Barthelemy & Luong, 1987 Cilibrasi & Vitanyi, 2007
28 Building a tree cloud tree decoration Choice of word sizes: computed directly from frequency (apply a log!) or computed from frequency ranking (exponential distribution) or statistical significance with respect to a reference corpus
29 Building a tree cloud tree decoration Colors: chronology old recent 150 most frequent words in Obama's campaign speeches, winsize=30, distance=oddsratio, color=chronology, NJ-tree. Built with TreeCloud and
30 Building a tree cloud tree decoration Colors: dispersion sparse dense standard deviation of the position too blue? 150 most frequent words in Obama's campaign speeches, winsize=30, distance=oddsratio, color=dispersion, NJ-tree.
31 Building a tree cloud tree decoration Colors: dispersion sparse dense standard deviation of the position word frequency too red? 150 most frequent words in Obama's campaign speeches, winsize=30, distance=oddsratio, color=norm-dispersion, NJ-tree.
32 Building a tree cloud tree decoration Edge color or thickness: quality of the induced cluster. 150 most frequent words in Obama's campaign speeches, winsize=30, distance=oddsratio, color=chronology, NJ-tree.
33 Quality control Is there an objective quality measure of tree clouds?
34 Quality control Is there an objective quality measure of tree clouds? What is the best method to build a tree cloud from my data?
35 Quality control Is there an objective quality measure of tree clouds? What is the best method to build a tree cloud from my data? Tree cloud variations if small changes? bootstrap to evaluate: - stability of the result - robustness of the method
36 Quality control Is there an objective quality measure of tree clouds? What is the best method to build a tree cloud from my data? Tree cloud variations if small changes? bootstrap to evaluate: - stability of the result - robustness of the method Still, is there a more direct method? arboricity to show whether the distance matrix fits with a tree, which should imply stability? Guénoche & Garreta, 2001 Guénoche & Darlu, 2009
37 Quality control bootstrap Randomly delete words with probability 50%. Built tree cloud of original text, and altered text. Compute similarity of both trees (1-normalized RobinsonFoulds) similarity 0,95 0,9 0,85 0,8 0,75 0,7 0,65 0,6 0,55 0,5 mi chisquared dice liddell gmean ms zscore poissonstirling jaccard hyperlex oddsratio loglikelihood 4 altered versions of 10 Obama's speeches, 3000 words in average, width=30, NJ-tree.
38 Quality control arboricity Relationship between bootstrap quality and arboricity: 80 arboricity (%) ,52 0,54 0,56 0,58 0,6 0,62 0,64 0,66 0,68 0,7 bootstrap quality
39 Quality control arboricity Relationship between bootstrap quality and arboricity: 80 arboricity (%) 70 correlation coefficient: 0.64 arboricity below 50%: danger! ,52 0,54 0,56 0,58 0,6 0,62 0,64 0,66 0,68 0,7 bootstrap quality
40 Perspectives Make the tool available on a web interface Evaluate tree clouds for discourse analysis Build the daily tree cloud of people popular on blogs, with
41 Thank you for your attention! Tree cloud of the words appearing twice or more in the IFCS 2009 call for paper lemmatization, width=20, distance=dice, NJ-tree
42 Tree clouds focused on one word Tree cloud of the neighborhood of McCain in Obama's campaign speeches -
43 Tree clouds focused on one word Tree cloud of the neighborhood of Bush in Obama's campaign speeches -
44 Tree clouds focused on one word Tree cloud of the neighborhood of world in Obama's campaign speeches -
Visualising a Text with a Tree Cloud
Author manuscript, published in "IFCS'09: International Federation of Classification Societies Conference, Dresde : Germany (2009)" DOI : 10.1007/978-3-642-10745-0_61 Visualising a Text with a Tree Cloud
Txt2vz: a new tool for generating graph clouds
Txt2vz: a new tool for generating graph clouds HIRSCH, L and TIAN, D Available from Sheffield Hallam University Research Archive (SHURA) at: http://shura.shu.ac.uk/6619/
WSI using Graphs of Collocations. Paper by: Ioannis P. Klapaftis and Suresh Manandhar Presented by: Ahmad R. Shahid
WSI using Graphs of Collocations Paper by: Ioannis P. Klapaftis and Suresh Manandhar Presented by: Ahmad R. Shahid Word Sense Induction (WSI) Identifying different senses (uses) of a word Finds applications
CHAPTER 2 TEXTURE CLASSIFICATION METHODS GRAY LEVEL CO-OCCURRENCE MATRIX AND TEXTURE UNIT
CHAPTER 2 TEXTURE CLASSIFICATION METHODS GRAY LEVEL CO-OCCURRENCE MATRIX AND TEXTURE UNIT 2.1 BRIEF OUTLINE The classification of digital imagery is to extract useful thematic information which is one
Label propagation with dams on large graphs using Apache Hadoop and Apache Spark
Label propagation with dams on large graphs using Apache Hadoop and Apache Spark ATTAL Jean-Philippe (1) MALEK Maria (2) (1) jal@eisti.eu (2) mma@eisti.eu October 19, 2015 Contents 1 Real-World Graphs
MEASURING SEMANTIC SIMILARITY BETWEEN WORDS AND IMPROVING WORD SIMILARITY BY AUGUMENTING PMI
MEASURING SEMANTIC SIMILARITY BETWEEN WORDS AND IMPROVING WORD SIMILARITY BY AUGUMENTING PMI 1 KAMATCHI.M, 2 SUNDARAM.N 1 M.E, CSE, MahaBarathi Engineering College Chinnasalem-606201, 2 Assistant Professor,
Evaluation Measures. Sebastian Pölsterl. April 28, Computer Aided Medical Procedures Technische Universität München
Evaluation Measures Sebastian Pölsterl Computer Aided Medical Procedures Technische Universität München April 28, 2015 Outline 1 Classification 1. Confusion Matrix 2. Receiver operating characteristics
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy
BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving
Distance based tree reconstruction. Hierarchical clustering (UPGMA) Neighbor-Joining (NJ)
Distance based tree reconstruction Hierarchical clustering (UPGMA) Neighbor-Joining (NJ) All organisms have evolved from a common ancestor. Infer the evolutionary tree (tree topology and edge lengths)
Toward the integration of informatic tools and GRID infrastructure for Assyriology text analysis
58 Rencontre Assyriologique Internationale (RAI) Private and State 16-20 July 2012 - Leiden Toward the integration of informatic tools and GRID infrastructure for Assyriology text analysis Giovanni Ponti,
A SOM-view of oilfield data: A novel vector field visualization for Self-Organizing Maps and its applications in the petroleum industry
A SOM-view of oilfield data: A novel vector field visualization for Self-Organizing Maps and its applications in the petroleum industry Georg Pölzlbauer, Andreas Rauber (Department of Software Technology
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
Stereo Vision II: Dense Stereo Matching
Stereo Vision II: Dense Stereo Matching Nassir Navab Slides prepared by Christian Unger Outline. Hardware. Challenges. Taxonomy of Stereo Matching. Analysis of Different Problems. Practical Considerations.
Distance-based Phylogenetic Methods Near a Polytomy
Distance-based Phylogenetic Methods Near a Polytomy Ruth Davidson and Seth Sullivant NCSU UIUC May 21, 2014 2 Phylogenetic trees model the common evolutionary history of a group of species Leaves = extant
Data Mining: Concepts and Techniques. Chapter March 8, 2007 Data Mining: Concepts and Techniques 1
Data Mining: Concepts and Techniques Chapter 7.1-4 March 8, 2007 Data Mining: Concepts and Techniques 1 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis Chapter 7 Cluster Analysis 3. A
5/15/16. Computational Methods for Data Analysis. Massimo Poesio UNSUPERVISED LEARNING. Clustering. Unsupervised learning introduction
Computational Methods for Data Analysis Massimo Poesio UNSUPERVISED LEARNING Clustering Unsupervised learning introduction 1 Supervised learning Training set: Unsupervised learning Training set: 2 Clustering
Online Social Networks and Media
Online Social Networks and Media Absorbing Random Walks Link Prediction Why does the Power Method work? If a matrix R is real and symmetric, it has real eigenvalues and eigenvectors: λ, w, λ 2, w 2,, (λ
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
Impact of Term Weighting Schemes on Document Clustering A Review
Volume 118 No. 23 2018, 467-475 ISSN: 1314-3395 (on-line version) url: http://acadpubl.eu/hub ijpam.eu Impact of Term Weighting Schemes on Document Clustering A Review G. Hannah Grace and Kalyani Desikan
Behavioral Data Mining. Lecture 18 Clustering
Behavioral Data Mining Lecture 18 Clustering Outline Why? Cluster quality K-means Spectral clustering Generative Models Rationale Given a set {X i } for i = 1,,n, a clustering is a partition of the X i
Query Suggestion. A variety of automatic or semi-automatic query suggestion techniques have been developed
Query Suggestion Query Suggestion A variety of automatic or semi-automatic query suggestion techniques have been developed Goal is to improve effectiveness by matching related/similar terms Semi-automatic
Local Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
University of Florida CISE department Gator Engineering. Clustering Part 5
Clustering Part 5 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville SNN Approach to Clustering Ordinary distance measures have problems Euclidean
DIMACS Tutorial on Phylogenetic Trees and Rapidly Evolving Pathogens. Katherine St. John City University of New York 1
DIMACS Tutorial on Phylogenetic Trees and Rapidly Evolving Pathogens Katherine St. John City University of New York 1 Thanks to the DIMACS Staff Linda Casals Walter Morris Nicole Clark Katherine St. John
Figure (5) Kohonen Self-Organized Map
2- KOHONEN SELF-ORGANIZING MAPS (SOM) - The self-organizing neural networks assume a topological structure among the cluster units. - There are m cluster units, arranged in a one- or two-dimensional array;
Computer Graphics. Coordinate Systems and Change of Frames. Based on slides by Dianna Xu, Bryn Mawr College
Computer Graphics Coordinate Systems and Change of Frames Based on slides by Dianna Xu, Bryn Mawr College Linear Independence A set of vectors independent if is linearly If a set of vectors is linearly
Thanks to Jure Leskovec, Anand Rajaraman, Jeff Ullman
Thanks to Jure Leskovec, Anand Rajaraman, Jeff Ullman http://www.mmds.org Overview of Recommender Systems Content-based Systems Collaborative Filtering J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive
Phylogenetics. Introduction to Bioinformatics Dortmund, Lectures: Sven Rahmann. Exercises: Udo Feldkamp, Michael Wurst
Phylogenetics Introduction to Bioinformatics Dortmund, 16.-20.07.2007 Lectures: Sven Rahmann Exercises: Udo Feldkamp, Michael Wurst 1 Phylogenetics phylum = tree phylogenetics: reconstruction of evolutionary
Overview of Clustering
based on Loïc Cerfs slides (UFMG) April 2017 UCBL LIRIS DM2L Example of applicative problem Student profiles Given the marks received by students for different courses, how to group the students so that
Edge and local feature detection - 2. Importance of edge detection in computer vision
Edge and local feature detection Gradient based edge detection Edge detection by function fitting Second derivative edge detectors Edge linking and the construction of the chain graph Edge and local feature
Large-Scale Face Manifold Learning
Large-Scale Face Manifold Learning Sanjiv Kumar Google Research New York, NY * Joint work with A. Talwalkar, H. Rowley and M. Mohri 1 Face Manifold Learning 50 x 50 pixel faces R 2500 50 x 50 pixel random
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
NATURAL LANGUAGE PROCESSING
NATURAL LANGUAGE PROCESSING LESSON 9 : SEMANTIC SIMILARITY OUTLINE Semantic Relations Semantic Similarity Levels Sense Level Word Level Text Level WordNet-based Similarity Methods Hybrid Methods Similarity
Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University Infinite data. Filtering data streams
/9/7 Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them
BOOLEAN MATRIX FACTORIZATIONS. with applications in data mining Pauli Miettinen
BOOLEAN MATRIX FACTORIZATIONS with applications in data mining Pauli Miettinen MATRIX FACTORIZATIONS BOOLEAN MATRIX FACTORIZATIONS o THE BOOLEAN MATRIX PRODUCT As normal matrix product, but with addition
CS 229 Final Project - Using machine learning to enhance a collaborative filtering recommendation system for Yelp
CS 229 Final Project - Using machine learning to enhance a collaborative filtering recommendation system for Yelp Chris Guthrie Abstract In this paper I present my investigation of machine learning as
Clustering. Robert M. Haralick. Computer Science, Graduate Center City University of New York
Clustering Robert M. Haralick Computer Science, Graduate Center City University of New York Outline K-means 1 K-means 2 3 4 5 Clustering K-means The purpose of clustering is to determine the similarity
Document Clustering using Feature Selection Based on Multiviewpoint and Link Similarity Measure
Document Clustering using Feature Selection Based on Multiviewpoint and Link Similarity Measure Neelam Singh neelamjain.jain@gmail.com Neha Garg nehagarg.february@gmail.com Janmejay Pant geujay2010@gmail.com
Information Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 4th, 2014 Wolf-Tilo Balke and José Pinto Institut für Informationssysteme Technische Universität Braunschweig The Cluster
Digital Libraries: Language Technologies
Digital Libraries: Language Technologies RAFFAELLA BERNARDI UNIVERSITÀ DEGLI STUDI DI TRENTO P.ZZA VENEZIA, ROOM: 2.05, E-MAIL: BERNARDI@DISI.UNITN.IT Contents 1 Recall: Inverted Index..........................................
Combining Appearance and Topology for Wide
Combining Appearance and Topology for Wide Baseline Matching Dennis Tell and Stefan Carlsson Presented by: Josh Wills Image Point Correspondences Critical foundation for many vision applications 3-D reconstruction,
Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
UNIBALANCE Users Manual. Marcin Macutkiewicz and Roger M. Cooke
UNIBALANCE Users Manual Marcin Macutkiewicz and Roger M. Cooke Deflt 2006 1 1. Installation The application is delivered in the form of executable installation file. In order to install it you have to
A Tabu Search Heuristic for the Generalized Traveling Salesman Problem
A Tabu Search Heuristic for the Generalized Traveling Salesman Problem Jacques Renaud 1,2 Frédéric Semet 3,4 1. Université Laval 2. Centre de Recherche sur les Technologies de l Organisation Réseau 3.
Cluster Analysis. Summer School on Geocomputation. 27 June July 2011 Vysoké Pole
Cluster Analysis Summer School on Geocomputation 27 June 2011 2 July 2011 Vysoké Pole Lecture delivered by: doc. Mgr. Radoslav Harman, PhD. Faculty of Mathematics, Physics and Informatics Comenius University,
Product Catalog. AcaStat. Software
Product Catalog AcaStat Software AcaStat AcaStat is an inexpensive and easy-to-use data analysis tool. Easily create data files or import data from spreadsheets or delimited text files. Run crosstabulations,
The Role of Information Scent in On-line Browsing:
The Role of Information Scent in On-line Browsing: Extensions of the ACT-R Utility and Concept Formation Mechanisms Peter Pirolli User Interface Research Area Supported in part by Office of Naval Research
Scaling species tree estimation methods to large datasets using NJMerge
Scaling species tree estimation methods to large datasets using NJMerge Erin Molloy and Tandy Warnow {emolloy2, warnow}@illinois.edu University of Illinois at Urbana Champaign 2018 Phylogenomics Software
Lecture Topic Projects 1 Intro, schedule, and logistics 2 Data Science components and tasks 3 Data types Project #1 out 4 Introduction to R,
Lecture Topic Projects 1 Intro, schedule, and logistics 2 Data Science components and tasks 3 Data types Project #1 out 4 Introduction to R, statistics foundations 5 Introduction to D3, visual analytics
Generating Useful Network-based Features for Analyzing Social Networks
Generating Useful Network-based Features for Analyzing Social Networks Abstract Recently, many Web services such as social networking services, blogs, and collaborative tagging have become widely popular.
Copyright 2011 by Dr. Andrew David Norte. All Rights Reserved.
Near-End Crosstalk Considerations For Coupled Microstriplines David Norte, PhD www.the-signal-and-power-integrity-institute.com Thornton, Colorado, 80234, USA Abstract This paper addresses the impact of
1 Homophily and assortative mixing
1 Homophily and assortative mixing Networks, and particularly social networks, often exhibit a property called homophily or assortative mixing, which simply means that the attributes of vertices correlate
Community Detection. Jian Pei: CMPT 741/459 Clustering (1) 2
Clustering Community Detection http://image.slidesharecdn.com/communitydetectionitilecturejune0-0609559-phpapp0/95/community-detection-in-social-media--78.jpg?cb=3087368 Jian Pei: CMPT 74/459 Clustering
Information Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering May 25, 2011 Wolf-Tilo Balke and Joachim Selke Institut für Informationssysteme Technische Universität Braunschweig Homework
Collective Entity Resolution in Relational Data
Collective Entity Resolution in Relational Data I. Bhattacharya, L. Getoor University of Maryland Presented by: Srikar Pyda, Brett Walenz CS590.01 - Duke University Parts of this presentation from: http://www.norc.org/pdfs/may%202011%20personal%20validation%20and%20entity%20resolution%20conference/getoorcollectiveentityresolution
Numerical Robustness. The implementation of adaptive filtering algorithms on a digital computer, which inevitably operates using finite word-lengths,
1. Introduction Adaptive filtering techniques are used in a wide range of applications, including echo cancellation, adaptive equalization, adaptive noise cancellation, and adaptive beamforming. These
Regression. Dr. G. Bharadwaja Kumar VIT Chennai
Regression Dr. G. Bharadwaja Kumar VIT Chennai Introduction Statistical models normally specify how one set of variables, called dependent variables, functionally depend on another set of variables, called
CSE 494: Information Retrieval, Mining and Integration on the Internet
CSE 494: Information Retrieval, Mining and Integration on the Internet Midterm. 18 th Oct 2011 (Instructor: Subbarao Kambhampati) In-class Duration: Duration of the class 1hr 15min (75min) Total points:
Example for calculation of clustering coefficient Node N 1 has 8 neighbors (red arrows) There are 12 connectivities among neighbors (blue arrows)
Example for calculation of clustering coefficient Node N 1 has 8 neighbors (red arrows) There are 12 connectivities among neighbors (blue arrows) Average clustering coefficient of a graph Overall measure
Introduction to Data Mining
Introduction to Data Mining Lecture #7: Recommendation Content based & Collaborative Filtering Seoul National University In This Lecture Understand the motivation and the problem of recommendation Compare
Spatial Outlier Detection
Spatial Outlier Detection Chang-Tien Lu Department of Computer Science Northern Virginia Center Virginia Tech Joint work with Dechang Chen, Yufeng Kou, Jiang Zhao 1 Spatial Outlier A spatial data point
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
CHAPTER 7 EXAMPLES: MIXTURE MODELING WITH CROSS- SECTIONAL DATA
Examples: Mixture Modeling With Cross-Sectional Data CHAPTER 7 EXAMPLES: MIXTURE MODELING WITH CROSS- SECTIONAL DATA Mixture modeling refers to modeling with categorical latent variables that represent
Visualization and Clustering of Tagged Music Data
Visualization and Clustering of Tagged Music Data Pascal Lehwark 1, Sebastian Risi 2, and Alfred Ultsch 3 1 Databionics Research Group, Philipps University Marburg pascal@indiji.com 2 Databionics Research
Introduction to Clustering and Classification. Psych 993 Methods for Clustering and Classification Lecture 1
Introduction to Clustering and Classification Psych 993 Methods for Clustering and Classification Lecture 1 Today s Lecture Introduction to methods for clustering and classification Discussion of measures
vector space retrieval many slides courtesy James Amherst
vector space retrieval many slides courtesy James Allan@umass Amherst 1 what is a retrieval model? Model is an idealization or abstraction of an actual process Mathematical models are used to study the
CS224W: Social and Information Network Analysis Project Report: Edge Detection in Review Networks
CS224W: Social and Information Network Analysis Project Report: Edge Detection in Review Networks Archana Sulebele, Usha Prabhu, William Yang (Group 29) Keywords: Link Prediction, Review Networks, Adamic/Adar,
Word Storms: Multiples of Word Clouds for Visual Comparison of Documents
Word Storms: Multiples of Word Clouds for Visual Comparison of Documents Quim Castella School of Informatics University of Edinburgh quim.castella@gmail.com Charles Sutton School of Informatics University
NAME: Sample Final Exam (based on previous CSE 455 exams by Profs. Seitz and Shapiro)
Computer Vision Prof. Rajesh Rao TA: Jiun-Hung Chen CSE 455 Winter 2009 Sample Final Exam (based on previous CSE 455 exams by Profs. Seitz and Shapiro) Write your name at the top of every page. Directions
Gene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
PERSONALIZED TAG RECOMMENDATION
PERSONALIZED TAG RECOMMENDATION Ziyu Guan, Xiaofei He, Jiajun Bu, Qiaozhu Mei, Chun Chen, Can Wang Zhejiang University, China Univ. of Illinois/Univ. of Michigan 1 Booming of Social Tagging Applications
Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects
Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
Standard and Convex NMF in Clustering UCI wine and sonar data
Standard and Convex NMF in Clustering UCI wine and sonar data ACS AISBITS 2016, October 19-21, Miȩdzyzdroje. Anna M. Bartkowiak aba@cs.uni.wroc.pl Institute of Computer Science, University of Wroclaw PL
Data Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining
Data Mining: Data Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach, Kumar Data Preprocessing Aggregation Sampling Dimensionality Reduction Feature subset selection Feature creation
BBS654 Data Mining. Pinar Duygulu. Slides are adapted from Nazli Ikizler
BBS654 Data Mining Pinar Duygulu Slides are adapted from Nazli Ikizler 1 Classification Classification systems: Supervised learning Make a rational prediction given evidence There are several methods for
Collaborative filtering based on a random walk model on a graph
Collaborative filtering based on a random walk model on a graph Marco Saerens, Francois Fouss, Alain Pirotte, Luh Yen, Pierre Dupont (UCL) Jean-Michel Renders (Xerox Research Europe) Some recent methods:
Random Number Generation and Monte Carlo Methods
James E. Gentle Random Number Generation and Monte Carlo Methods With 30 Illustrations Springer Contents Preface vii 1 Simulating Random Numbers from a Uniform Distribution 1 1.1 Linear Congruential Generators
Basic Statistical Terms and Definitions
I. Basics Basic Statistical Terms and Definitions Statistics is a collection of methods for planning experiments, and obtaining data. The data is then organized and summarized so that professionals can
Distance-based Methods: Drawbacks
Distance-based Methods: Drawbacks Hard to find clusters with irregular shapes Hard to specify the number of clusters Heuristic: a cluster must be dense Jian Pei: CMPT 459/741 Clustering (3) 1 How to Find
Robust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
Generating Useful Network-based Features for Analyzing Social Networks
Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008) Generating Useful Network-based Features for Analyzing Social Networks Jun Karamon and Yutaka Matsuo and Mitsuru Ishizuka
Road Map. Data types Measuring data Data cleaning Data integration Data transformation Data reduction Data discretization Summary
2. Data preprocessing Road Map Data types Measuring data Data cleaning Data integration Data transformation Data reduction Data discretization Summary 2 Data types Categorical vs. Numerical Scale types
OBJECT-CENTERED INTERACTIVE MULTI-DIMENSIONAL SCALING: ASK THE EXPERT
OBJECT-CENTERED INTERACTIVE MULTI-DIMENSIONAL SCALING: ASK THE EXPERT Joost Broekens Tim Cocx Walter A. Kosters Leiden Institute of Advanced Computer Science Leiden University, The Netherlands Email: {broekens,
Edge and corner detection
Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements
Chapter 2 Basic Structure of High-Dimensional Spaces
Chapter 2 Basic Structure of High-Dimensional Spaces Data is naturally represented geometrically by associating each record with a point in the space spanned by the attributes. This idea, although simple,
Web Page Recommender System based on Folksonomy Mining for ITNG 06 Submissions
Web Page Recommender System based on Folksonomy Mining for ITNG 06 Submissions Satoshi Niwa University of Tokyo niwa@nii.ac.jp Takuo Doi University of Tokyo Shinichi Honiden University of Tokyo National
Customer Clustering using RFM analysis
Customer Clustering using RFM analysis VASILIS AGGELIS WINBANK PIRAEUS BANK Athens GREECE AggelisV@winbank.gr DIMITRIS CHRISTODOULAKIS Computer Engineering and Informatics Department University of Patras
Department of Electronic Engineering FINAL YEAR PROJECT REPORT
Department of Electronic Engineering FINAL YEAR PROJECT REPORT BEngCE-2007/08-HCS-HCS-03-BECE Natural Language Understanding for Query in Web Search 1 Student Name: Sit Wing Sum Student ID: Supervisor:
Computational Databases: Inspirations from Statistical Software. Linnea Passing, Technical University of Munich
Computational Databases: Inspirations from Statistical Software Linnea Passing, linnea.passing@tum.de Technical University of Munich Data Science Meets Databases Data Cleansing Pipelines Fuzzy joins Data
Selection of Scale-Invariant Parts for Object Class Recognition
Selection of Scale-Invariant Parts for Object Class Recognition Gy. Dorkó and C. Schmid INRIA Rhône-Alpes, GRAVIR-CNRS 655, av. de l Europe, 3833 Montbonnot, France fdorko,schmidg@inrialpes.fr Abstract
Didacticiel - Études de cas
Subject In some circumstances, the goal of the supervised learning is not to classify examples but rather to organize them in order to point up the most interesting individuals. For instance, in the direct
CMSC 476/676 Information Retrieval Midterm Exam Spring 2014
CMSC 476/676 Information Retrieval Midterm Exam Spring 2014 Name: You may consult your notes and/or your textbook. This is a 75 minute, in class exam. If there is information missing in any of the question
INF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
DescriptorEnsemble: An Unsupervised Approach to Image Matching and Alignment with Multiple Descriptors
DescriptorEnsemble: An Unsupervised Approach to Image Matching and Alignment with Multiple Descriptors 林彥宇副研究員 Yen-Yu Lin, Associate Research Fellow 中央研究院資訊科技創新研究中心 Research Center for IT Innovation, Academia
Modelling image complexity by independent component analysis, with application to content-based image retrieval
Modelling image complexity by independent component analysis, with application to content-based image retrieval Jukka Perkiö 1,2 and Aapo Hyvärinen 1,2,3 1 Helsinki Institute for Information Technology,
Chapter 4: Text Clustering
4.1 Introduction to Text Clustering Clustering is an unsupervised method of grouping texts / documents in such a way that in spite of having little knowledge about the content of the documents, we can
10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
Non-linear dimension reduction
Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods
Data Partitioning and MapReduce
Data Partitioning and MapReduce Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Intelligent Decision Support Systems Master studies,