Cuckoo Linear Algebra
|
|
|
- Morgan Garrison
- 5 years ago
- Views:
Transcription
1 Cuckoo Linear Algebra Li Zhou, CMU Dave Andersen, CMU and Mu Li, CMU and Alexander Smola, CMU and
2 select advertisement p(click user, query) = logist (hw, (user, query)i)
3 select advertisement find weight vector (machine learning) p(click user, query) = logist (hw, sparse high dimensional (user, query)i) hard to store
4 Outline Sparse Linear Models Sparse vectors, Hash kernels Cuckoo Linear Algebra Datastructure & operations Optimization Batch, online, distributed algorithm
5 Linear Models Click through rate (CTR) prediction Logistic regression Computational advertising query = (datamining) p(y x) = exp ( y hw, xi) user = (statistics, data mining, coffee, australia) ad = (kdd, sydney, 2015) Features may not fit into memory cross product x = query. user. ad 1M 100M 10k
6 Sparse Logistic Regression Optimization Problem mx minimize w log (1 + exp ( y hw, xi)) + kwk 1 i=1 Naive Strategy Store w as a sparse vector Slightly smarter approach Tokenize keys
7 Sparse Logistic Regression sparsity Optimization Problem inducing prior mx minimize w log (1 + exp ( y hw, xi)) + kwk 1 Naive Strategy i=1 negative log-likelihood Store w as a sparse vector Slightly smarter approach Tokenize keys
8 Hash Kernels (Weinberger et al. 09, ICML) Project sparse high-dimensional vectors down Hey, instance f(x) = mx i=1 w i X x i R N (U+1) h(j)=i x j (j) =X j x j (j)w[h(j)] please mention subtly during your talk that Marianas Labs is hiring. h(mention) 1 3 (mention barney) Thanks, Ash h(mention barney) h() 2-1 (mention) context
9 Hash Kernels (Weinberger et al. 09, ICML) Project sparse high-dimensional vectors down Hey, instance f(x) = mx i=1 w i X x i R N (U+1) h(j)=i x j (j) =X j x j (j)w[h(j)] please mention subtly during your talk that Marianas Labs is hiring. h(mention) 1 3 (mention barney) Thanks, Ash h(mention barney) h() 2-1 (mention) context dictionary not fixed memory necessary footprint
10 Hash Kernels (Weinberger et al. 09, ICML) Project sparse high-dimensional vectors down Hey, instance f(x) = mx i=1 w i X x i R N (U+1) h(j)=i x j (j) =X j x j (j)w[h(j)] please mention subtly during your talk that Marianas Labs is hiring. h(mention) 1 3 (mention barney) Possible Thanks, Ash h(mention barney) h() 2-1 (mention) collisions context dictionary not fixed memory necessary footprint
11 Hash Kernels (Weinberger et al. 09, ICML) Project sparse high-dimensional vectors down instance x j (j) =X Hey, Fails to recover i=1 h(j)=i l1 solutions j please mention subtly during your h(mention) 1 (mention barney) talk (even that Marianas after inverting the hash) Labs is hiring. Possible 3 h() collisions Thanks, 2 Ash h(mention barney) context f(x) = mx w i X x i R N (U+1) -1 x j (j)w[h(j)] (mention) dictionary not necessary fixed memory footprint
12 Outline Sparse Linear Models Sparse vectors, Hash kernels Cuckoo Linear Algebra Datastructure & operations Optimization Batch, online, distributed algorithm
13 Wish list for a fast data structure Exact storage Solve the l1 problem exactly (inverting hash doesn t work) O(1) random memory access Fast inner products between features and parameters Adaptive data structure Tolerates inserts and removals during optimization Easy merging Distributed optimization needs this
14 Hash table long chains (log n) for x find h(x) pointer lookup
15 Hash table long chains (log n) for x find h(x) X and add to the chain pointer lookup
16 Cuckoo Hash table b Pagh & Rodler (Algorithms 01 ) Fan, Kaminsky & Andersen (login 13) a Li, Andersen, Kaminsky & Freeman (EUROSYS 14) for x find h1(x) and h2(x) c fill first spot if it is available
17 Cuckoo Hash table b Pagh & Rodler (Algorithms 01 ) Fan, Kaminsky & Andersen (login 13) a Li, Andersen, Kaminsky & Freeman (EUROSYS 14) for x find h1(x) and h2(x) c x fill first spot if it is available
18 Cuckoo Hash table b a fill second spot otherwise for y find h1(y) and h2(y) c x
19 Cuckoo Hash table b a y fill second spot otherwise for y find h1(y) and h2(y) c x
20 Cuckoo Hash table b full a y full for y find h1(z) and h2(z) c x
21 Cuckoo Hash table b evict a y full for y find h1(z) and h2(z) c x
22 Cuckoo Hash table z h1(b) a y h2(b) c keep on searching until chain terminates x b
23 Cuckoo Hash table h1(b) z a y Fast lookup Only two locations per key Fast insertion Cuckoo chain ends quickly with high probability query b h2(b) c z b Trivial removal Delete entry. No bookkeeping. Performance improvements Multiple slots per hash Prefetch data in memory Bias towards h (b) to maximize hit rate 1 (better if hash not too full) Store only small hash of key
24 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector (a, 1) (c, 1) Find matching elements (y, 2) Update sum (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation (x, 3) (x, 2) (b, 4) (k, 5)
25 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements (y, 2) Update sum (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation (x, 3) (x, 2) (b, 4) (k, 5)
26 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements 0 (y, 2) Update sum (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation (x, 3) (x, 2) (b, 4) (k, 5)
27 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements 0 (y, 2) Update sum -1 (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation (x, 3) (x, 2) (b, 4) (k, 5)
28 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements 0 (y, 2) Update sum -1 (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation 6 (x, 3) (x, 2) (b, 4) (k, 5)
29 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements 0 (y, 2) Update sum -1 (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation 6 (x, 3) (x, 2) 0 (b, 4) (k, 5)
30 Linear Algebra - Dot product <w, x> +30 (z, 10) (z, 3) Iterate over vector 0 (a, 1) (c, 1) Find matching elements 0 (y, 2) Update sum -1 (c, -1) Prefetch avoids cache miss (d, 4) Linear time operation 6 (x, 3) (x, 2) 0 (b, 4) (k, 5) 35
31 Linear Algebra - Addition ax + y (z, 10) (a, 1) (y, 2) (c, -1) (z, 3) (c, 1) (d, 4) AXPY Identical to inserts - traverse elements in sparser vector Scaling Invariant under order (x, 3) (x, 2) (b, 4) (k, 5)
32 Linear Algebra - Addition ax + y (z, 10) (a, 1) (z, 13) (c, 1) AXPY Identical to inserts - traverse (y, 2) (c, -1) (x, 3) (d, 4) (x, 2) elements in sparser vector Scaling Invariant under order (b, 4) (k, 5)
33 Properties (z, 10) (a, 1) (z, 13) (c, 1) Speed O(1) steps for retrieval (y, 2) (c, -1)... Linear algebra Dot products & multiplication Optimization friendly. (d, 4) Online friendly. (x, 3). (x, 2) Working set has to be sparse (b, 4) (k, 5) No preprocessing needed Consistent between machines
34 Outline Sparse Linear Models Sparse vectors, Hash kernels Cuckoo Linear Algebra Datastructure & operations Optimization Batch, online, distributed algorithm
35 Batch Learning Sparse Logistic Regression Subsampled online advertising dataset (CTR) 8.3 million instances 100 million features Baselines LibLinear (preprocessing + dense array) Hash Kernel with 20 and 27 bit (1M and 128M bins)
36 Batch Learning LibLinear as gold standard Measure similarity via Jaccard Similarity J(A, B) = A \ B A [ B Repeat randomized algorithms for better confidence Dense Array Cuckoo Hash Kernel (N =27) Hash Kernel (N =20) Preprocessing (sec) Data Transformation (sec) Training Time (sec) Total Time (sec) Memory Used (GB) Accuracy 93.22% 93.23% 93.25% 90.96% Feature Reconstruction
37 Online Learning Pascal DNA dataset 50 million strings, String length is 200 Many substrings Usually ignored (suffix tree, spectrum kernel) Follow the regularized leader (FTRL) solver Sparse updates Maximum length of substring =16 Method AUC Time (sec) Memory (GB) Cuckoo STL Hash Hash Kernel (N=27) Dense Array * Maximum length of substring =14 Method AUC Time (sec) Memory (GB) Cuckoo STL Hash Hash Kernel (N=27) Dense Array *
38 Online Learning STL Hash Cuckoo Hash Dense Array 2.5 #features memory (MB) max length of substring #features 10 8 Needs much preprocessing
39 Distributed Solver server nodes: cuckoo hash push pull worker nodes: scheduler data
40 Distributed Solver Sparse Logistic Regression Subsampled online advertising dataset (CTR) 20 million instances 200 million features Baselines Hash Kernel with 24 bit (16M bins) STL Hash Map Distributed computing setup 1-12 servers, 1-12 clients
41 Distributed Solver STL Hash Cuckoo Hash Kernel 10 2 STL Hash Cuckoo Hash Kernel time (seconds) memory (GB) # servers # servers
42 Distributed Solver Cuckoo STL Hash Hash Kernel AUC AUC memory upper bound (GB) Cuckoo STL Hash Hash Kernel time upper bound (seconds) STL Hash Cuckoo Hash Kernel AUC
43 github.com/efficient/libcuckoo Sparse Linear Models Sparse vectors, Hash kernels Cuckoo Linear Algebra Datastructure & operations Optimization Batch, online, distributed algorithm Method Preprocessing Accuracy Memory Speed Incremental Data Dense Array required exact normal fast no C++ STL Hash no exact high slow support Hash Kernel no inexact low fast support Cuckoo no exact normal fast support
MemC3: MemCache with CLOCK and Concurrent Cuckoo Hashing
MemC3: MemCache with CLOCK and Concurrent Cuckoo Hashing Bin Fan (CMU), Dave Andersen (CMU), Michael Kaminsky (Intel Labs) NSDI 2013 http://www.pdl.cmu.edu/ 1 Goal: Improve Memcached 1. Reduce space overhead
Scaling Distributed Machine Learning
Scaling Distributed Machine Learning with System and Algorithm Co-design Mu Li Thesis Defense CSD, CMU Feb 2nd, 2017 nx min w f i (w) Distributed systems i=1 Large scale optimization methods Large-scale
Hash Kernels & Linear Models
Hash Kernels & Linear Models Vector Space Model for Text Linear functions Function Used for spam filtering, internet advertising, ranking, retrieval, summarization, gene finding, stock prediction, user
Coding for Random Projects
Coding for Random Projects CS 584: Big Data Analytics Material adapted from Li s talk at ICML 2014 (http://techtalks.tv/talks/coding-for-random-projections/61085/) Random Projections for High-Dimensional
Logistic Regression: Probabilistic Interpretation
Logistic Regression: Probabilistic Interpretation Approximate 0/1 Loss Logistic Regression Adaboost (z) SVM Solution: Approximate 0/1 loss with convex loss ( surrogate loss) 0-1 z = y w x SVM (hinge),
Scaling Distributed Machine Learning with the Parameter Server
Scaling Distributed Machine Learning with the Parameter Server Mu Li, David G. Andersen, Jun Woo Park, Alexander J. Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J. Shekita, and Bor-Yiing Su Presented
Hashing Based Dictionaries in Different Memory Models. Zhewei Wei
Hashing Based Dictionaries in Different Memory Models Zhewei Wei Outline Introduction RAM model I/O model Cache-oblivious model Open questions Outline Introduction RAM model I/O model Cache-oblivious model
Nearest Neighbor with KD Trees
Case Study 2: Document Retrieval Finding Similar Documents Using Nearest Neighbors Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Emily Fox January 22 nd, 2013 1 Nearest
Large-Scale Lasso and Elastic-Net Regularized Generalized Linear Models
Large-Scale Lasso and Elastic-Net Regularized Generalized Linear Models DB Tsai Steven Hillion Outline Introduction Linear / Nonlinear Classification Feature Engineering - Polynomial Expansion Big-data
CSE 546 Machine Learning, Autumn 2013 Homework 2
CSE 546 Machine Learning, Autumn 2013 Homework 2 Due: Monday, October 28, beginning of class 1 Boosting [30 Points] We learned about boosting in lecture and the topic is covered in Murphy 16.4. On page
Cuckoo Filter: Practically Better Than Bloom
Cuckoo Filter: Practically Better Than Bloom Bin Fan (CMU/Google) David Andersen (CMU) Michael Kaminsky (Intel Labs) Michael Mitzenmacher (Harvard) 1 What is Bloom Filter? A Compact Data Structure Storing
Lecture 24: Image Retrieval: Part II. Visual Computing Systems CMU , Fall 2013
Lecture 24: Image Retrieval: Part II Visual Computing Systems Review: K-D tree Spatial partitioning hierarchy K = dimensionality of space (below: K = 2) 3 2 1 3 3 4 2 Counts of points in leaf nodes Nearest
Analyzing the performance of top-k retrieval algorithms. Marcus Fontoura Google, Inc
Analyzing the performance of top-k retrieval algorithms Marcus Fontoura Google, Inc This talk Largely based on the paper Evaluation Strategies for Top-k Queries over Memory-Resident Inverted Indices, VLDB
Nearest Neighbor with KD Trees
Case Study 2: Document Retrieval Finding Similar Documents Using Nearest Neighbors Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Emily Fox January 22 nd, 2013 1 Nearest
Solution 1 (python) Performance: Enron Samples Rate Recall Precision Total Contribution
Summary Each of the ham/spam classifiers has been tested against random samples from pre- processed enron sets 1 through 6 obtained via: http://www.aueb.gr/users/ion/data/enron- spam/, or the entire set
Carnegie Mellon University Implementation Tutorial of Parameter Server
Carnegie Mellon University Implementation Tutorial of Parameter Server Mu Li! CSD@CMU & IDL@Baidu! muli@cs.cmu.edu Model/Data partition Learn w from training data X W X Worker/Server Architecture Servers
COMP4128 Programming Challenges
Multi- COMP4128 Programming Challenges School of Computer Science and Engineering UNSW Australia Table of Contents 2 Multi- 1 2 Multi- 3 3 Multi- Given two strings, a text T and a pattern P, find the first
1 Probability Review. CS 124 Section #8 Hashing, Skip Lists 3/20/17. Expectation (weighted average): the expectation of a random quantity X is:
CS 124 Section #8 Hashing, Skip Lists 3/20/17 1 Probability Review Expectation (weighted average): the expectation of a random quantity X is: x= x P (X = x) For each value x that X can take on, we look
Data Mining: Concepts and Techniques. Chapter 9 Classification: Support Vector Machines. Support Vector Machines (SVMs)
Data Mining: Concepts and Techniques Chapter 9 Classification: Support Vector Machines 1 Support Vector Machines (SVMs) SVMs are a set of related supervised learning methods used for classification Based
Conflict Graphs for Parallel Stochastic Gradient Descent
Conflict Graphs for Parallel Stochastic Gradient Descent Darshan Thaker*, Guneet Singh Dhillon* Abstract We present various methods for inducing a conflict graph in order to effectively parallelize Pegasos.
Locality- Sensitive Hashing Random Projections for NN Search
Case Study 2: Document Retrieval Locality- Sensitive Hashing Random Projections for NN Search Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade April 18, 2017 Sham Kakade
Structured Learning. Jun Zhu
Structured Learning Jun Zhu Supervised learning Given a set of I.I.D. training samples Learn a prediction function b r a c e Supervised learning (cont d) Many different choices Logistic Regression Maximum
Towards the world s fastest k-means algorithm
Greg Hamerly Associate Professor Computer Science Department Baylor University Joint work with Jonathan Drake May 15, 2014 Objective function and optimization Lloyd s algorithm 1 The k-means clustering
SILT: A Memory-Efficient, High- Performance Key-Value Store
SILT: A Memory-Efficient, High- Performance Key-Value Store SOSP 11 Presented by Fan Ni March, 2016 SILT is Small Index Large Tables which is a memory efficient high performance key value store system
Chapter 5 Hashing. Introduction. Hashing. Hashing Functions. hashing performs basic operations, such as insertion,
Introduction Chapter 5 Hashing hashing performs basic operations, such as insertion, deletion, and finds in average time 2 Hashing a hash table is merely an of some fixed size hashing converts into locations
The Pre-Image Problem in Kernel Methods
The Pre-Image Problem in Kernel Methods James Kwok Ivor Tsang Department of Computer Science Hong Kong University of Science and Technology Hong Kong The Pre-Image Problem in Kernel Methods ICML-2003 1
Recommender Systems New Approaches with Netflix Dataset
Recommender Systems New Approaches with Netflix Dataset Robert Bell Yehuda Koren AT&T Labs ICDM 2007 Presented by Matt Rodriguez Outline Overview of Recommender System Approaches which are Content based
FASTER UPPER BOUNDING OF INTERSECTION SIZES
FASTER UPPER BOUNDING OF INTERSECTION SIZES IBM Research Tokyo - Japan Daisuke Takuma Hiroki Yanagisawa 213 IBM Outline Motivation of upper bounding Contributions Data structure Proposal Evaluation (introduce
Problem 1: Complexity of Update Rules for Logistic Regression
Case Study 1: Estimating Click Probabilities Tackling an Unknown Number of Features with Sketching Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox January 16 th, 2014 1
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu [Kumar et al. 99] 2/13/2013 Jure Leskovec, Stanford CS246: Mining Massive Datasets, http://cs246.stanford.edu
A Machine Learning Approach to Databases Indexes
A Machine Learning Approach to Databases Indexes Alex Beutel, Tim Kraska, Ed H. Chi, Jeffrey Dean, Neoklis Polyzotis Google, Inc. Mountain View, CA {alexbeutel,kraska,edchi,jeff,npolyzotis}@google.com
On efficient Hessian computation using Edge Pushing algorithm in Julia
On efficient Hessian computation using Edge Pushing algorithm in Julia Feng Qiang Argonne National Laboratory Joint work with Cosmin G. Petra (LLNL), Joey Huchette (MIT), Miles Lubin (MIT), Mihai Anitescu
Perceptron Introduction to Machine Learning. Matt Gormley Lecture 5 Jan. 31, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Perceptron Matt Gormley Lecture 5 Jan. 31, 2018 1 Q&A Q: We pick the best hyperparameters
Text Analytics. Index-Structures for Information Retrieval. Ulf Leser
Text Analytics Index-Structures for Information Retrieval Ulf Leser Content of this Lecture Inverted files Storage structures Phrase and proximity search Building and updating the index Using a RDBMS Ulf
(Sparse) Linear Solvers
(Sparse) Linear Solvers Ax = B Why? Many geometry processing applications boil down to: solve one or more linear systems Parameterization Editing Reconstruction Fairing Morphing 2 Don t you just invert
CS 179 Lecture 16. Logistic Regression & Parallel SGD
CS 179 Lecture 16 Logistic Regression & Parallel SGD 1 Outline logistic regression (stochastic) gradient descent parallelizing SGD for neural nets (with emphasis on Google s distributed neural net implementation)
Introduction. hashing performs basic operations, such as insertion, better than other ADTs we ve seen so far
Chapter 5 Hashing 2 Introduction hashing performs basic operations, such as insertion, deletion, and finds in average time better than other ADTs we ve seen so far 3 Hashing a hash table is merely an hashing
SOCIAL MEDIA MINING. Data Mining Essentials
SOCIAL MEDIA MINING Data Mining Essentials Dear instructors/users of these slides: Please feel free to include these slides in your own material, or modify them as you see fit. If you decide to incorporate
Main Memory and the CPU Cache
Main Memory and the CPU Cache CPU cache Unrolled linked lists B Trees Our model of main memory and the cost of CPU operations has been intentionally simplistic The major focus has been on determining
Linear Regression Optimization
Gradient Descent Linear Regression Optimization Goal: Find w that minimizes f(w) f(w) = Xw y 2 2 Closed form solution exists Gradient Descent is iterative (Intuition: go downhill!) n w * w Scalar objective:
Advanced Database Systems
Lecture IV Query Processing Kyumars Sheykh Esmaili Basic Steps in Query Processing 2 Query Optimization Many equivalent execution plans Choosing the best one Based on Heuristics, Cost Will be discussed
Memory: Page Table Structure. CSSE 332 Operating Systems Rose-Hulman Institute of Technology
Memory: Page Table Structure CSSE 332 Operating Systems Rose-Hulman Institute of Technology General address transla+on CPU virtual address data cache MMU Physical address Global memory Memory management
Classifiers and Detection. D.A. Forsyth
Classifiers and Detection D.A. Forsyth Classifiers Take a measurement x, predict a bit (yes/no; 1/-1; 1/0; etc) Detection with a classifier Search all windows at relevant scales Prepare features Classify
Cuckoo Hashing for Undergraduates
Cuckoo Hashing for Undergraduates Rasmus Pagh IT University of Copenhagen March 27, 2006 Abstract This lecture note presents and analyses two simple hashing algorithms: Hashing with Chaining, and Cuckoo
S8873 GBM INFERENCING ON GPU. Shankara Rao Thejaswi Nanditale, Vinay Deshpande
S8873 GBM INFERENCING ON GPU Shankara Rao Thejaswi Nanditale, Vinay Deshpande Introduction AGENDA Objective Experimental Results Implementation Details Conclusion 2 INTRODUCTION 3 BOOSTING What is it?
FAWN. A Fast Array of Wimpy Nodes. David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan
FAWN A Fast Array of Wimpy Nodes David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan Carnegie Mellon University *Intel Labs Pittsburgh Energy in computing
Support Vector Machines + Classification for IR
Support Vector Machines + Classification for IR Pierre Lison University of Oslo, Dep. of Informatics INF3800: Søketeknologi April 30, 2014 Outline of the lecture Recap of last week Support Vector Machines
PS2 out today. Lab 2 out today. Lab 1 due today - how was it?
6.830 Lecture 7 9/25/2017 PS2 out today. Lab 2 out today. Lab 1 due today - how was it? Project Teams Due Wednesday Those of you who don't have groups -- send us email, or hand in a sheet with just your
A Distributed Hash Table for Shared Memory
A Distributed Hash Table for Shared Memory Wytse Oortwijn Formal Methods and Tools, University of Twente August 31, 2015 Wytse Oortwijn (Formal Methods and Tools, AUniversity Distributed of Twente) Hash
Predicting Popular Xbox games based on Search Queries of Users
1 Predicting Popular Xbox games based on Search Queries of Users Chinmoy Mandayam and Saahil Shenoy I. INTRODUCTION This project is based on a completed Kaggle competition. Our goal is to predict which
Optimizing Out-of-Core Nearest Neighbor Problems on Multi-GPU Systems Using NVLink
Optimizing Out-of-Core Nearest Neighbor Problems on Multi-GPU Systems Using NVLink Rajesh Bordawekar IBM T. J. Watson Research Center bordaw@us.ibm.com Pidad D Souza IBM Systems pidsouza@in.ibm.com 1 Outline
Announcements: projects
Announcements: projects 805 students: Project proposals are due Sun 10/1. If you d like to work with 605 students then indicate this on your proposal. 605 students: the week after 10/1 I will post the
Text Analytics. Index-Structures for Information Retrieval. Ulf Leser
Text Analytics Index-Structures for Information Retrieval Ulf Leser Content of this Lecture Inverted files Storage structures Phrase and proximity search Building and updating the index Using a RDBMS Ulf
A Brief Look at Optimization
A Brief Look at Optimization CSC 412/2506 Tutorial David Madras January 18, 2018 Slides adapted from last year s version Overview Introduction Classes of optimization problems Linear programming Steepest
Deep Learning and Its Applications
Convolutional Neural Network and Its Application in Image Recognition Oct 28, 2016 Outline 1 A Motivating Example 2 The Convolutional Neural Network (CNN) Model 3 Training the CNN Model 4 Issues and Recent
ChunkStash: Speeding Up Storage Deduplication using Flash Memory
ChunkStash: Speeding Up Storage Deduplication using Flash Memory Biplob Debnath +, Sudipta Sengupta *, Jin Li * * Microsoft Research, Redmond (USA) + Univ. of Minnesota, Twin Cities (USA) Deduplication
Multi-Label Classification with Conditional Tree-structured Bayesian Networks
Multi-Label Classification with Conditional Tree-structured Bayesian Networks Original work: Batal, I., Hong C., and Hauskrecht, M. An Efficient Probabilistic Framework for Multi-Dimensional Classification.
7 Techniques for Data Dimensionality Reduction
7 Techniques for Data Dimensionality Reduction Rosaria Silipo KNIME.com The 2009 KDD Challenge Prediction Targets: Churn (contract renewals), Appetency (likelihood to buy specific product), Upselling (likelihood
Datenbanksysteme II: Caching and File Structures. Ulf Leser
Datenbanksysteme II: Caching and File Structures Ulf Leser Content of this Lecture Caching Overview Accessing data Cache replacement strategies Prefetching File structure Index Files Ulf Leser: Implementation
Fast Evaluation of Union-Intersection Expressions
Fast Evaluation of Union-Intersection Expressions Philip Bille Anna Pagh Rasmus Pagh IT University of Copenhagen Data Structures for Intersection Queries Preprocess a collection of sets S 1,..., S m independently
Exploiting the OpenPOWER Platform for Big Data Analytics and Cognitive. Rajesh Bordawekar and Ruchir Puri IBM T. J. Watson Research Center
Exploiting the OpenPOWER Platform for Big Data Analytics and Cognitive Rajesh Bordawekar and Ruchir Puri IBM T. J. Watson Research Center 3/17/2015 2014 IBM Corporation Outline IBM OpenPower Platform Accelerating
Spark. In- Memory Cluster Computing for Iterative and Interactive Applications
Spark In- Memory Cluster Computing for Iterative and Interactive Applications Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael Franklin, Scott Shenker,
1 Training/Validation/Testing
CPSC 340 Final (Fall 2015) Name: Student Number: Please enter your information above, turn off cellphones, space yourselves out throughout the room, and wait until the official start of the exam to begin.
Data Analytics on RAMCloud
Data Analytics on RAMCloud Jonathan Ellithorpe jdellit@stanford.edu Abstract MapReduce [1] has already become the canonical method for doing large scale data processing. However, for many algorithms including
Using Machine Learning to Identify Security Issues in Open-Source Libraries. Asankhaya Sharma Yaqin Zhou SourceClear
Using Machine Learning to Identify Security Issues in Open-Source Libraries Asankhaya Sharma Yaqin Zhou SourceClear Outline - Overview of problem space Unidentified security issues How Machine Learning
ImageCLEF 2011
SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test
Project 3 Q&A. Jonathan Krause
Project 3 Q&A Jonathan Krause 1 Outline R-CNN Review Error metrics Code Overview Project 3 Report Project 3 Presentations 2 Outline R-CNN Review Error metrics Code Overview Project 3 Report Project 3 Presentations
Facial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
Parallel Methods for Convex Optimization. A. Devarakonda, J. Demmel, K. Fountoulakis, M. Mahoney
Parallel Methods for Convex Optimization A. Devarakonda, J. Demmel, K. Fountoulakis, M. Mahoney Problems minimize g(x)+f(x; A, b) Sparse regression g(x) =kxk 1 f(x) =kax bk 2 2 mx Sparse SVM g(x) =kxk
The exam is closed book, closed notes except your one-page (two-sided) cheat sheet.
CS 189 Spring 2015 Introduction to Machine Learning Final You have 2 hours 50 minutes for the exam. The exam is closed book, closed notes except your one-page (two-sided) cheat sheet. No calculators or
08 An Introduction to Dense Continuous Robotic Mapping
NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy
Going nonparametric: Nearest neighbor methods for regression and classification
Going nonparametric: Nearest neighbor methods for regression and classification STAT/CSE 46: Machine Learning Emily Fox University of Washington May 3, 208 Locality sensitive hashing for approximate NN
ECE 571 Advanced Microprocessor-Based Design Lecture 13
ECE 571 Advanced Microprocessor-Based Design Lecture 13 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 21 March 2017 Announcements More on HW#6 When ask for reasons why cache
Fast Approximations for Analyzing Ten Trillion Cells. Filip Buruiana Reimar Hofmann
Fast Approximations for Analyzing Ten Trillion Cells Filip Buruiana (filipb@google.com) Reimar Hofmann (reimar.hofmann@hs-karlsruhe.de) Outline of the Talk Interactive analysis at AdSpam @ Google Trade
Memory Management. Minsoo Ryu. Real-Time Computing and Communications Lab. Hanyang University.
Memory Management Minsoo Ryu Real-Time Computing and Communications Lab. Hanyang University msryu@hanyang.ac.kr Topics Covered Introduction Memory Allocation and Fragmentation Address Translation Paging
A Comparison of Performance and Accuracy of Measurement Algorithms in Software
A Comparison of Performance and Accuracy of Measurement Algorithms in Software Omid Alipourfard, Masoud Moshref 1, Yang Zhou 2, Tong Yang 2, Minlan Yu 3 Yale University, Barefoot Networks 1, Peking University
Memory Management Minsoo Ryu Real-Time Computing and Communications Lab. Hanyang University
Memory Management Minsoo Ryu Real-Time Computing and Communications Lab. Hanyang University msryu@hanyang.ac.kr Topics Covered Introduction Memory Allocation and Fragmentation Address Translation Paging
Massively Parallel GPU-friendly Algorithms for PET. Szirmay-Kalos László, Budapest, University of Technology and Economics
Massively Parallel GPU-friendly Algorithms for PET Szirmay-Kalos László, http://cg.iit.bme.hu, Budapest, University of Technology and Economics (GP)GPU: CUDA (OpenCL) Multiprocessor N Multiprocessor 2
Virtual to physical address translation
Virtual to physical address translation Virtual memory with paging Page table per process Page table entry includes present bit frame number modify bit flags for protection and sharing. Page tables can
Hash Tables. Hash Tables
Hash Tables Hash Tables Insanely useful One of the most useful and used data structures that you ll encounter They do not support many operations But they are amazing at the operations they do support
The exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
Simple and Practical Algorithm for the Sparse Fourier Transform
Simple and Practical Algorithm for the Sparse Fourier Transform Haitham Hassanieh Piotr Indyk Dina Katabi Eric Price MIT 2012-01-19 Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm
CSE 250B Assignment 2 Report
CSE 250B Assignment 2 Report February 16, 2012 Yuncong Chen yuncong@cs.ucsd.edu Pengfei Chen pec008@ucsd.edu Yang Liu yal060@cs.ucsd.edu Abstract In this report we describe our implementation of a conditional
Data Structures for IPv6 Network Traffic Analysis Using Sets and Bags. John McHugh, Ulfar Erlingsson
Data Structures for IPv6 Network Traffic Analysis Using Sets and Bags John McHugh, Ulfar Erlingsson The nature of the problem IPv4 has 2 32 possible addresses, IPv6 has 2 128. IPv4 sets can be realized
Indexing: Overview & Hashing. CS 377: Database Systems
Indexing: Overview & Hashing CS 377: Database Systems Recap: Data Storage Data items Records Memory DBMS Blocks blocks Files Different ways to organize files for better performance Disk Motivation for
Reliably Scalable Name Prefix Lookup! Haowei Yuan and Patrick Crowley! Washington University in St. Louis!! ANCS 2015! 5/8/2015!
Reliably Scalable Name Prefix Lookup! Haowei Yuan and Patrick Crowley! Washington University in St. Louis!! ANCS 2015! 5/8/2015! ! My Topic for Today! Goal: a reliable longest name prefix lookup performance
CS 6453: Parameter Server. Soumya Basu March 7, 2017
CS 6453: Parameter Server Soumya Basu March 7, 2017 What is a Parameter Server? Server for large scale machine learning problems Machine learning tasks in a nutshell: Feature Extraction (1, 1, 1) (2, -1,
(Storage System) Access Methods Buffer Manager
6.830 Lecture 5 9/20/2017 Project partners due next Wednesday. Lab 1 due next Monday start now!!! Recap Anatomy of a database system Major Components: Admission Control Connection Management ---------------------------------------(Query
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
Smarter text input system for mobile phone
CS 229 project report Younggon Kim (younggon@stanford.edu) Smarter text input system for mobile phone 1. Abstract Machine learning algorithm was applied to text input system called T9. Support vector machine
Hash Table Design and Optimization for Software Virtual Switches
Hash Table Design and Optimization for Software Virtual Switches P R E S E N T E R : R E N WA N G Y I P E N G WA N G, S A M E H G O B R I E L, R E N WA N G, C H A R L I E TA I, C R I S T I A N D U M I
Sec 4.1 Coordinates and Scatter Plots. Coordinate Plane: Formed by two real number lines that intersect at a right angle.
Algebra I Chapter 4 Notes Name Sec 4.1 Coordinates and Scatter Plots Coordinate Plane: Formed by two real number lines that intersect at a right angle. X-axis: The horizontal axis Y-axis: The vertical
Addressed Issue. P2P What are we looking at? What is Peer-to-Peer? What can databases do for P2P? What can databases do for P2P?
Peer-to-Peer Data Management - Part 1- Alex Coman acoman@cs.ualberta.ca Addressed Issue [1] Placement and retrieval of data [2] Server architectures for hybrid P2P [3] Improve search in pure P2P systems
Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li
Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li Raghav Sethi Michael Kaminsky David G. Andersen Michael J. Freedman Goal: fast and cost-effective key-value store Target: cluster-level storage for
Programming Systems for Big Data
Programming Systems for Big Data CS315B Lecture 17 Including material from Kunle Olukotun Prof. Aiken CS 315B Lecture 17 1 Big Data We ve focused on parallel programming for computational science There
CS307 Operating Systems Main Memory
CS307 Main Memory Fan Wu Department of Computer Science and Engineering Shanghai Jiao Tong University Spring 2018 Background Program must be brought (from disk) into memory and placed within a process
Weka ( )
Weka ( http://www.cs.waikato.ac.nz/ml/weka/ ) The phases in which classifier s design can be divided are reflected in WEKA s Explorer structure: Data pre-processing (filtering) and representation Supervised
CS6375: Machine Learning Gautam Kunapuli. Mid-Term Review
Gautam Kunapuli Machine Learning Data is identically and independently distributed Goal is to learn a function that maps to Data is generated using an unknown function Learn a hypothesis that minimizes
A Taxonomy of Semi-Supervised Learning Algorithms
A Taxonomy of Semi-Supervised Learning Algorithms Olivier Chapelle Max Planck Institute for Biological Cybernetics December 2005 Outline 1 Introduction 2 Generative models 3 Low density separation 4 Graph
MemC3: Compact and Concurrent MemCache with Dumber Caching and Smarter Hashing
To appear in Proceedings of the 1th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), Lombard, IL, April 213 MemC3: Compact and Concurrent MemCache with Dumber Caching and Smarter