MLlib. Distributed Machine Learning on. Evan Sparks. UC Berkeley
|
|
|
- Bathsheba Bradford
- 5 years ago
- Views:
Transcription
1 MLlib & ML base Distributed Machine Learning on Evan Sparks UC Berkeley January 31st, 2014 Collaborators: Ameet Talwalkar, Xiangrui Meng, Virginia Smith, Xinghao Pan, Shivaram Venkataraman, Matei Zaharia, Rean Griffith, John Duchi, Joseph Gonzalez, Michael Franklin, Michael I. Jordan, Tim Kraska UC Berkeley
2 Problem: Scalable implementa.ons difficult for ML Developers
3 Problem: Scalable implementa.ons difficult for ML Developers
4 Problem: Scalable implementa.ons difficult for ML Developers
5 Problem: ML is difficult for End Users Too many algorithms
6 Problem: ML is difficult for End Users Too many knobs Too many algorithms
7 Problem: ML is difficult for End Users Too many knobs Too many algorithms Difficult to debug
8 Problem: ML is difficult for End Users Too many knobs Too many algorithms Difficult to debug Doesn t scale
9 Problem: ML is difficult Fast for End Users Too many knobs Reliable Provable Accurate Too many algorithms Difficult to debug Doesn t scale
10 ML Experts MLbase Systems Experts
11 ML Experts MLbase Systems Experts 1. Easy scalable ML development (ML Developers) 2. User- friendly ML at scale (End Users)
12 Matlab Stack
13 Matlab Stack Single Machine
14 Matlab Stack Lapack Single Machine Lapack: low- level Fortran linear algebra library
15 Matlab Stack Matlab Interface Lapack Single Machine Lapack: low- level Fortran linear algebra library Matlab Interface Higher- level abstrac]ons for data access / processing More extensive func]onality than Lapack Leverages Lapack whenever possible
16 Matlab Stack Matlab Interface Lapack Single Machine Lapack: low- level Fortran linear algebra library Matlab Interface Higher- level abstrac]ons for data access / processing More extensive func]onality than Lapack Leverages Lapack whenever possible Similar stories for R and Python
17 MLbase Stack Matlab Interface Lapack Single Machine
18 MLbase Stack Matlab Interface Lapack Single Machine Runtime(s)
19 MLbase Stack Matlab Interface Lapack Single Machine Runtime(s) Spark Spark: cluster compu]ng system designed for itera]ve computa]on
20 MLbase Stack Matlab Interface Lapack Single Machine MLlib Runtime(s) Spark Spark: cluster compu]ng system designed for itera]ve computa]on MLlib: produc]on- quality ML library in Spark
21 MLbase Stack Matlab Interface Lapack Single Machine MLI MLlib Runtime(s) Spark Spark: cluster compu]ng system designed for itera]ve computa]on MLlib: produc]on- quality ML library in Spark MLI: experimental API for simplified feature extrac]on and algorithm development
22 MLbase Stack ML Optimizer Matlab Interface Lapack Single Machine MLI MLlib Runtime(s) Spark Spark: cluster compu]ng system designed for itera]ve computa]on MLlib: produc]on- quality ML library in Spark MLI: experimental API for simplified feature extrac]on and algorithm development ML Op.mizer: a declara]ve layer to simplify access to large- scale ML
23 Overview MLlib Collabora.ve Filtering ALS Details
24 MLlib Classifica.on: Regression: Collabora.ve Filtering: Clustering / Explora.on: Op.miza.on Primi.ves: Interopera.lity: Logis]c Regression, Linear SVM (+L1, L2), Decision Trees, Naive Bayes Linear Regression (+Lasso, Ridge) Alterna]ng Least Squares K- Means, SVD SGD, Parallel Gradient Scala, Java, PySpark (0.9)
25 MLlib Classifica.on: Regression: Collabora.ve Filtering: Clustering / Explora.on: Op.miza.on Primi.ves: Logis]c Regression, Linear SVM (+L1, L2), Decision Trees, Naive Bayes Linear Regression (+Lasso, Ridge) Alterna]ng Least Squares K- Means, SVD SGD, Parallel Gradient Interopera.lity: Scala, Java, PySpark (0.9) Included within Spark codebase Unlike Mahout/Hadoop Part of Spark 0.8 release Con]nued support via Spark project Community involvement has been terrific: ALS with implicit feedback (0.8.1), Naive Bayes (0.9), SVD (0.9), Decision Trees (soon!)
26 MLlib Performance
27 MLlib Performance Wall.me: elapsed ]me to execute task
28 MLlib Performance Wall.me: elapsed ]me to execute task Weak scaling fix problem size per processor ideally: constant wall]me as we grow cluster
29 MLlib Performance Wall.me: elapsed ]me to execute task Weak scaling fix problem size per processor ideally: constant wall]me as we grow cluster Strong scaling fix total problem size ideally: linear speed up as we grow cluster
30 MLlib Performance Wall.me: elapsed ]me to execute task Weak scaling fix problem size per processor ideally: constant wall]me as we grow cluster Strong scaling fix total problem size ideally: linear speed up as we grow cluster EC2 Experiments m2.4xlarge instances, up to 32 machine clusters
31 Logis]c Regression - Weak Scaling
32 Logis]c Regression - Weak Scaling Full dataset: 200K images, 160K dense features
33 Logis]c Regression - Weak Scaling 10 8 MLbase MLlib VW Ideal relative walltime # machines Full dataset: 200K images, 160K dense features Similar weak scaling
34 Logis]c Regression - Weak Scaling 10 8 MLbase MLlib VW Ideal relative walltime # machines Full dataset: 200K images, 160K dense features Similar weak scaling MLlib within a factor of 2 of VW s wall]me
35 Logis]c Regression - Weak Scaling relative walltime MLbase MLlib VW Ideal walltime (s) n=6k, d=160k n=12.5k, d=160k n=25k, d=160k n=50k, d=160k n=100k, d=160k n=200k, d=160k # machines 0 MLbase MLlib VW Matlab Full dataset: 200K images, 160K dense features Similar weak scaling MLlib within a factor of 2 of VW s wall]me
36 Logis]c Regression - Strong Scaling
37 Logis]c Regression - Strong Scaling Fixed Dataset: 50K images, 160K dense features
38 Logis]c Regression - Strong Scaling MLlib MLbase VW Ideal speedup # machines Fixed Dataset: 50K images, 160K dense features MLlib exhibits bejer scaling proper]es
39 Logis]c Regression - Strong Scaling speedup MLlib MLbase VW Ideal walltime (s) Machine 2 Machines 4 Machines 8 Machines 16 Machines 32 Machines # machines MLbase MLlib VW Matlab Fixed Dataset: 50K images, 160K dense features MLlib exhibits bejer scaling proper]es MLlib faster than VW with 16 and 32 machines
40 ALS - Wall]me
41 ALS - Wall]me Dataset: Scaled version of Neklix data (9X in size) Cluster: 9 machines
42 ALS - Wall]me System Wall.me (seconds) Matlab Dataset: Scaled version of Neklix data (9X in size) Cluster: 9 machines
43 ALS - Wall]me System Wall.me (seconds) Matlab Mahout 4206 Dataset: Scaled version of Neklix data (9X in size) Cluster: 9 machines
44 ALS - Wall]me System Wall.me (seconds) Matlab Mahout 4206 GraphLab 291 MLlib 481 Dataset: Scaled version of Neklix data (9X in size) Cluster: 9 machines MLlib an order of magnitude faster than Mahout
45 ALS - Wall]me System Wall.me (seconds) Matlab Mahout 4206 GraphLab 291 MLlib 481 Dataset: Scaled version of Neklix data (9X in size) Cluster: 9 machines MLlib an order of magnitude faster than Mahout MLlib within factor of 2 of GraphLab
46 Deployment Considera]ons
47 Deployment Considera]ons Vowpal Wabbit, GraphLab Data prepara]on specific to each program Non- trivial setup on cluster No fault tolerance
48 Deployment Considera]ons Vowpal Wabbit, GraphLab Data prepara]on specific to each program Non- trivial setup on cluster No fault tolerance MLlib Reads files from HDFS Launch/compile/run on cluster with a few commands RDD s provide fault tolerance naturally
49 Deployment Considera]ons Vowpal Wabbit, GraphLab Data prepara]on specific to each program Non- trivial setup on cluster No fault tolerance MLlib Reads files from HDFS Launch/compile/run on cluster with a few commands RDD s provide fault tolerance naturally Part of Spark s swiss army knife ecosystem
50 Deployment Considera]ons Vowpal Wabbit, GraphLab Data prepara]on specific to each program Non- trivial setup on cluster No fault tolerance MLlib Reads files from HDFS Launch/compile/run on cluster with a few commands RDD s provide fault tolerance naturally Part of Spark s swiss army knife ecosystem Shark, Spark Streaming, Graph- X, BlinkDB, etc.
51 Vision MLlib Collabora.ve Filtering ALS Details
52 Matrix Comple]on
53 Matrix Comple]on Goal: Recover a matrix from a subset of its entries
54 Matrix Comple]on Goal: Recover a matrix from a subset of its entries
55 Matrix Comple]on Goal: Recover a matrix from a subset of its entries
56 Reducing Degrees of Freedom
57 Reducing Degrees of Freedom Problem: Impossible without additional information n m
58 Reducing Degrees of Freedom Problem: Impossible without additional information mn degrees of freedom n r n r m = m Low-rank
59 Reducing Degrees of Freedom Problem: Impossible without additional information mn degrees of freedom Solution: Assume small # of factors determine preference O(m + n) degrees of freedom n r n r m = m Low-rank
60 Alterna]ng Least Squares =
61 Alterna]ng Least Squares =
62 Alterna]ng Least Squares =
63 Alterna]ng Least Squares =
64 Alterna]ng Least Squares = training error for first user = ( - ) + ( - )
65 Alterna]ng Least Squares = training error for first user = ( - ) + ( - ) ALS: alternate between updating user and movie factors
66 Alterna]ng Least Squares = training error for first user = ( - ) + ( - ) ALS: alternate between updating user and movie factors update first user by finding that minimizes training error
67 Alterna]ng Least Squares = training error for first user = ( - ) + ( - ) ALS: alternate between updating user and movie factors update first user by finding that minimizes training error reduces to standard linear regression problem
68 Alterna]ng Least Squares... = training error for first user = ( - ) + ( - ) ALS: alternate between updating user and movie factors update first user by finding that minimizes training error reduces to standard linear regression problem can update all users in parallel!
69 Alterna]ng Least Squares = training error for first user = ( - ) + ( - )
70 Alterna]ng Least Squares = H > M W training error for first user = ( - ) + ( - )
71 Alterna]ng Least Squares = H > M W training error for first user = ( - ) + ( - ) = X (M 1j W 1 H > j ) 2 (1,j)2
72 Alterna]ng Least Squares = H > M W training error for first user = ( - ) + ( - ) = X (M 1j W 1 H > j ) 2 (1,j)2 W 1 =(H > 1 H 1 ) 1 H > 1 M > 1 1
73 Exercise Today 19
74 Exercise Today Load 1,000,000 ra]ngs from MovieLens. 19
75 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. 19
76 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. Split into training/valida]on. 19
77 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. Split into training/valida]on. Fit a model. 19
78 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. Split into training/valida]on. Fit a model. Validate and tune hyperparameters. 19
79 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. Split into training/valida]on. Fit a model. Validate and tune hyperparameters. Get YOUR recommenda]ons. 19
80 Exercise Today Load 1,000,000 ra]ngs from MovieLens. Get YOUR ra]ngs. Split into training/valida]on. Fit a model. Validate and tune hyperparameters. Get YOUR recommenda]ons. Great example of a Spark applica]on! 19
81 Vision MLlib Collabora.ve Filtering ALS Details
82 Three Kinds of ALS Broadcast Everything Data Parallel Fully Parallel
83 Three Kinds of ALS Broadcast Everything Data Parallel Fully Parallel
84 Broadcast Everything Ratings Movie! User! Master Workers
85 Broadcast Everything Ratings Movie! User! Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
86 Broadcast Everything Ratings Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. Movie! User! At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
87 Broadcast Everything Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. Ratings Movie! User! At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
88 Broadcast Everything Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. Ratings Movie! User! At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
89 Broadcast Everything Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. Ratings Movie! User! At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
90 Broadcast Everything Master loads (small) data file and ini]alizes models. Master broadcasts data and ini]al models. Ratings Movie! User! At each itera]on, updated models are broadcast again. Works OK for small data. Master Workers Lots of communica]on overhead - doesn t scale well. Ships with Spark Examples
91 Three Kinds of ALS Broadcast Everything Data Parallel Fully Parallel
92 Data Parallel Ratings Movie! Ratings User! Ratings Master Workers Ratings
93 Data Parallel Ratings Workers load data Movie! Ratings User! Ratings Master Workers Ratings
94 Data Parallel Ratings Workers load data Movie! Ratings Master broadcasts ini]al models User! Ratings Master Workers Ratings
95 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Master Workers Ratings
96 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Master Workers Ratings
97 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Works on large datasets Master Workers Ratings
98 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Works on large datasets Master Workers Ratings Works well for smaller models. (low K)
99 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Works on large datasets Master Workers Ratings Works well for smaller models. (low K)
100 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Works on large datasets Master Workers Ratings Works well for smaller models. (low K)
101 Data Parallel Ratings Workers load data Movie! User! Ratings Ratings Master broadcasts ini]al models At each itera]on, updated models are broadcast again Much bejer scaling Works on large datasets Master Workers Ratings Works well for smaller models. (low K)
102 Three Kinds of ALS Broadcast Everything Data Parallel Fully Parallel
103 Fully Parallel Ratings Movie! User! Ratings Movie! User! Ratings Movie! User! Master Workers Ratings Movie! User!
104 Fully Parallel Ratings Movie! User! Workers load data Ratings Movie! User! Ratings Movie! User! Master Workers Ratings Movie! User!
105 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! Ratings Movie! User! Master Workers Ratings Movie! User!
106 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! At each itera]on, models are shared via join between workers. Ratings Movie! User! Master Workers Ratings Movie! User!
107 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! Ratings Movie! User! At each itera]on, models are shared via join between workers. Much bejer scalability. Master Workers Ratings Movie! User!
108 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! Ratings Movie! User! At each itera]on, models are shared via join between workers. Much bejer scalability. Works on large datasets Master Workers Ratings Movie! User!
109 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! At each itera]on, models are shared via join between workers. Ratings Movie! User! Much bejer scalability. Works on large datasets Master Workers Ratings Movie! User! Works on big models (higher K)
110 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! At each itera]on, models are shared via join between workers. Ratings Movie! User! Much bejer scalability. Works on large datasets Master Workers Ratings Movie! User! Works on big models (higher K)
111 Fully Parallel Ratings Movie! User! Workers load data Models are instan]ated at workers. Ratings Movie! User! At each itera]on, models are shared via join between workers. Ratings Movie! User! Much bejer scalability. Works on large datasets Master Workers Ratings Movie! User! Works on big models (higher K)
112 Three Kinds of ALS Broadcast Everything Data Parallel Fully Parallel
113 Three Kinds of ALS Four Broadcast Everything Data Parallel Fully Parallel Blocked
114 ML Optimizer MLI MLlib Runtime(s) Spark ML Op.mizer: a declara]ve layer to simplify access to large- scale ML MLI: experimental API for simplified feature extrac]on and algorithm development MLlib: produc]on- quality ML library in Spark Spark: cluster compu]ng system designed for itera]ve computa]on
115 ML Optimizer MLI MLlib Runtime(s) Spark ML Op.mizer: a declara]ve layer to simplify access to large- scale ML MLI: experimental API for simplified feature extrac]on and algorithm development MLlib: produc]on- quality ML library in Spark Spark: cluster compu]ng system designed for itera]ve computa]on ML base
116 ML Optimizer MLI MLlib Runtime(s) Spark ML Op.mizer: a declara]ve layer to simplify access to large- scale ML MLI: experimental API for simplified feature extrac]on and algorithm development MLlib: produc]on- quality ML library in Spark Spark: cluster compu]ng system designed for itera]ve computa]on THANKS! QUESTIONS? ML base
Evan Sparks and Ameet Talwalkar UC Berkeley. UC Berkeley
ML base Evan Sparks and Ameet Talwalkar UC Berkeley UC Berkeley Three Converging Trends Three Converging Trends Big Data Three Converging Trends Big Data Distributed Compu2ng Three Converging Trends Machine
Evan Sparks and Ameet Talwalkar UC Berkeley
ML base www.mlbase.org Evan Sparks and Ameet Talwalkar UC Berkeley Collaborators: Tim Kraska 2, Virginia Smith 1, Xinghao Pan 1, Shivaram Venkataraman 1, Matei Zaharia 1, Rean Griffith 3, John Duchi 1,
Evan Sparks and Ameet Talwalkar UC Berkeley
ML base www.mlbase.org Evan Sparks and Ameet Talwalkar UC Berkeley Collaborators: Tim Kraska 2, Virginia Smith 1, Xinghao Pan 1, Shivaram Venkataraman 1, Matei Zaharia 1, Rean Griffith 3, John Duchi 1,
Scaled Machine Learning at Matroid
Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh http://reza-zadeh.com Machine Learning Pipeline Learning Algorithm Replicate model Data Trained Model Serve Model Repeat entire pipeline Scaling
Big Data Infrastructures & Technologies
Big Data Infrastructures & Technologies Spark and MLLIB OVERVIEW OF SPARK What is Spark? Fast and expressive cluster computing system interoperable with Apache Hadoop Improves efficiency through: In-memory
Distributed Machine Learning" on Spark
Distributed Machine Learning" on Spark Reza Zadeh @Reza_Zadeh http://reza-zadeh.com Outline Data flow vs. traditional network programming Spark computing engine Optimization Example Matrix Computations
MLI - An API for Distributed Machine Learning. Sarang Dev
MLI - An API for Distributed Machine Learning Sarang Dev MLI - API Simplify the development of high-performance, scalable, distributed algorithms. Targets common ML problems related to data loading, feature
Summary of Big Data Frameworks Course 2015 Professor Sasu Tarkoma
Summary of Big Data Frameworks Course 2015 Professor Sasu Tarkoma www.cs.helsinki.fi Course Schedule Tuesday 10.3. Introduction and the Big Data Challenge Tuesday 17.3. MapReduce and Spark: Overview Tuesday
Integration of Machine Learning Library in Apache Apex
Integration of Machine Learning Library in Apache Apex Anurag Wagh, Krushika Tapedia, Harsh Pathak Vishwakarma Institute of Information Technology, Pune, India Abstract- Machine Learning is a type of artificial
An Introduction to Apache Spark
An Introduction to Apache Spark 1 History Developed in 2009 at UC Berkeley AMPLab. Open sourced in 2010. Spark becomes one of the largest big-data projects with more 400 contributors in 50+ organizations
Distributed Computing with Spark
Distributed Computing with Spark Reza Zadeh Thanks to Matei Zaharia Outline Data flow vs. traditional network programming Limitations of MapReduce Spark computing engine Numerical computing on Spark Ongoing
SCALABLE, LOW LATENCY MODEL SERVING AND MANAGEMENT WITH VELOX
THE MISSING PIECE IN COMPLEX ANALYTICS: SCALABLE, LOW LATENCY MODEL SERVING AND MANAGEMENT WITH VELOX Daniel Crankshaw, Peter Bailis, Joseph Gonzalez, Haoyuan Li, Zhao Zhang, Ali Ghodsi, Michael Franklin,
Execu&on Templates: Caching Control Plane Decisions for Strong Scaling of Data Analy&cs
Execu&on Templates: Caching Control Plane Decisions for Strong Scaling of Data Analy&cs Omid Mashayekhi Hang Qu Chinmayee Shah Philip Levis July 13, 2017 2 Cloud Frameworks SQL Streaming Machine Learning
MLlib and Distributing the " Singular Value Decomposition. Reza Zadeh
MLlib and Distributing the " Singular Value Decomposition Reza Zadeh Outline Example Invocations Benefits of Iterations Singular Value Decomposition All-pairs Similarity Computation MLlib + {Streaming,
Fast, Interactive, Language-Integrated Cluster Computing
Spark Fast, Interactive, Language-Integrated Cluster Computing Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael Franklin, Scott Shenker, Ion Stoica www.spark-project.org
Matrix Computations and " Neural Networks in Spark
Matrix Computations and " Neural Networks in Spark Reza Zadeh Paper: http://arxiv.org/abs/1509.02256 Joint work with many folks on paper. @Reza_Zadeh http://reza-zadeh.com Training Neural Networks Datasets
MLBase, MLLib and GraphX
MLBase, MLLib and GraphX 2015 Professor Sasu Tarkoma www.cs.helsinki.fi MLBase and MLlib: Vision Same system for Exploring data interactively using Spark Spark standalone programs Spark streaming for problems
DATA SCIENCE USING SPARK: AN INTRODUCTION
DATA SCIENCE USING SPARK: AN INTRODUCTION TOPICS COVERED Introduction to Spark Getting Started with Spark Programming in Spark Data Science with Spark What next? 2 DATA SCIENCE PROCESS Exploratory Data
Distributed Computing with Spark and MapReduce
Distributed Computing with Spark and MapReduce Reza Zadeh @Reza_Zadeh http://reza-zadeh.com Traditional Network Programming Message-passing between nodes (e.g. MPI) Very difficult to do at scale:» How
Analytic Cloud with. Shelly Garion. IBM Research -- Haifa IBM Corporation
Analytic Cloud with Shelly Garion IBM Research -- Haifa 2014 IBM Corporation Why Spark? Apache Spark is a fast and general open-source cluster computing engine for big data processing Speed: Spark is capable
Gradient Descent. Michail Michailidis & Patrick Maiden
Gradient Descent Michail Michailidis & Patrick Maiden Outline Mo4va4on Gradient Descent Algorithm Issues & Alterna4ves Stochas4c Gradient Descent Parallel Gradient Descent HOGWILD! Mo4va4on It is good
Submitted to: Dr. Sunnie Chung. Presented by: Sonal Deshmukh Jay Upadhyay
Submitted to: Dr. Sunnie Chung Presented by: Sonal Deshmukh Jay Upadhyay Submitted to: Dr. Sunny Chung Presented by: Sonal Deshmukh Jay Upadhyay What is Apache Survey shows huge popularity spike for Apache
Asynchronous and Fault-Tolerant Recursive Datalog Evalua9on in Shared-Nothing Engines
Asynchronous and Fault-Tolerant Recursive Datalog Evalua9on in Shared-Nothing Engines Jingjing Wang, Magdalena Balazinska, Daniel Halperin University of Washington Modern Analy>cs Requires Itera>on Graph
Spark. In- Memory Cluster Computing for Iterative and Interactive Applications
Spark In- Memory Cluster Computing for Iterative and Interactive Applications Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael Franklin, Scott Shenker,
Apache Spark 2.0. Matei
Apache Spark 2.0 Matei Zaharia @matei_zaharia What is Apache Spark? Open source data processing engine for clusters Generalizes MapReduce model Rich set of APIs and libraries In Scala, Java, Python and
A Parallel R Framework
A Parallel R Framework for Processing Large Dataset on Distributed Systems Nov. 17, 2013 This work is initiated and supported by Huawei Technologies Rise of Data-Intensive Analytics Data Sources Personal
Apache Spark 2 X Cookbook Cloud Ready Recipes For Analytics And Data Science
Apache Spark 2 X Cookbook Cloud Ready Recipes For Analytics And Data Science We have made it easy for you to find a PDF Ebooks without any digging. And by having access to our ebooks online or by storing
Chapter 1 - The Spark Machine Learning Library
Chapter 1 - The Spark Machine Learning Library Objectives Key objectives of this chapter: The Spark Machine Learning Library (MLlib) MLlib dense and sparse vectors and matrices Types of distributed matrices
Towards an OpBmizer for MLbase
Towards an OpBmizer for MLbase Ameet Talwalkar Collaborators: Evan Sparks, Michael Franklin, Michael I. Jordan, Tim Kraska UC Berkeley Problem: Scalable implementa.ons difficult for ML Developers Problem:
a Spark in the cloud iterative and interactive cluster computing
a Spark in the cloud iterative and interactive cluster computing Matei Zaharia, Mosharaf Chowdhury, Michael Franklin, Scott Shenker, Ion Stoica UC Berkeley Background MapReduce and Dryad raised level of
Shark. Hive on Spark. Cliff Engle, Antonio Lupher, Reynold Xin, Matei Zaharia, Michael Franklin, Ion Stoica, Scott Shenker
Shark Hive on Spark Cliff Engle, Antonio Lupher, Reynold Xin, Matei Zaharia, Michael Franklin, Ion Stoica, Scott Shenker Agenda Intro to Spark Apache Hive Shark Shark s Improvements over Hive Demo Alpha
Intelligent Services Serving Machine Learning
Intelligent Services Serving Machine Learning Joseph E. Gonzalez jegonzal@cs.berkeley.edu; Assistant Professor @ UC Berkeley joseph@dato.com; Co-Founder @ Dato Inc. Contemporary Learning Systems Big Data
Apache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context
1 Apache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context Generality: diverse workloads, operators, job sizes
Spark, Shark and Spark Streaming Introduction
Spark, Shark and Spark Streaming Introduction Tushar Kale tusharkale@in.ibm.com June 2015 This Talk Introduction to Shark, Spark and Spark Streaming Architecture Deployment Methodology Performance References
Overview. Prerequisites. Course Outline. Course Outline :: Apache Spark Development::
Title Duration : Apache Spark Development : 4 days Overview Spark is a fast and general cluster computing system for Big Data. It provides high-level APIs in Scala, Java, Python, and R, and an optimized
Resilient Distributed Datasets
Resilient Distributed Datasets A Fault- Tolerant Abstraction for In- Memory Cluster Computing Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael Franklin,
Spark. Cluster Computing with Working Sets. Matei Zaharia, Mosharaf Chowdhury, Michael Franklin, Scott Shenker, Ion Stoica.
Spark Cluster Computing with Working Sets Matei Zaharia, Mosharaf Chowdhury, Michael Franklin, Scott Shenker, Ion Stoica UC Berkeley Background MapReduce and Dryad raised level of abstraction in cluster
CSE 444: Database Internals. Lecture 23 Spark
CSE 444: Database Internals Lecture 23 Spark References Spark is an open source system from Berkeley Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. Matei
COSC 6339 Big Data Analytics. Introduction to Spark. Edgar Gabriel Fall What is SPARK?
COSC 6339 Big Data Analytics Introduction to Spark Edgar Gabriel Fall 2018 What is SPARK? In-Memory Cluster Computing for Big Data Applications Fixes the weaknesses of MapReduce Iterative applications
MapReduce Spark. Some slides are adapted from those of Jeff Dean and Matei Zaharia
MapReduce Spark Some slides are adapted from those of Jeff Dean and Matei Zaharia What have we learnt so far? Distributed storage systems consistency semantics protocols for fault tolerance Paxos, Raft,
2/4/2019 Week 3- A Sangmi Lee Pallickara
Week 3-A-0 2/4/2019 Colorado State University, Spring 2019 Week 3-A-1 CS535 BIG DATA FAQs PART A. BIG DATA TECHNOLOGY 3. DISTRIBUTED COMPUTING MODELS FOR SCALABLE BATCH COMPUTING SECTION 1: MAPREDUCE PA1
Lecture 11 Hadoop & Spark
Lecture 11 Hadoop & Spark Dr. Wilson Rivera ICOM 6025: High Performance Computing Electrical and Computer Engineering Department University of Puerto Rico Outline Distributed File Systems Hadoop Ecosystem
Using Existing Numerical Libraries on Spark
Using Existing Numerical Libraries on Spark Brian Spector Chicago Spark Users Meetup June 24 th, 2015 Experts in numerical algorithms and HPC services How to use existing libraries on Spark Call algorithm
Research challenges in data-intensive computing The Stratosphere Project Apache Flink
Research challenges in data-intensive computing The Stratosphere Project Apache Flink Seif Haridi KTH/SICS haridi@kth.se e2e-clouds.org Presented by: Seif Haridi May 2014 Research Areas Data-intensive
Cloud, Big Data & Linear Algebra
Cloud, Big Data & Linear Algebra Shelly Garion IBM Research -- Haifa 2014 IBM Corporation What is Big Data? 2 Global Data Volume in Exabytes What is Big Data? 2005 2012 2017 3 Global Data Volume in Exabytes
Chapter 4: Apache Spark
Chapter 4: Apache Spark Lecture Notes Winter semester 2016 / 2017 Ludwig-Maximilians-University Munich PD Dr. Matthias Renz 2015, Based on lectures by Donald Kossmann (ETH Zürich), as well as Jure Leskovec,
Using Numerical Libraries on Spark
Using Numerical Libraries on Spark Brian Spector London Spark Users Meetup August 18 th, 2015 Experts in numerical algorithms and HPC services How to use existing libraries on Spark Call algorithm with
Big Data Infrastructures & Technologies Hadoop Streaming Revisit.
Big Data Infrastructures & Technologies Hadoop Streaming Revisit ENRON Mapper ENRON Mapper Output (Excerpt) acomnes@enron.com blake.walker@enron.com edward.snowden@cia.gov alex.berenson@nyt.com ENRON Reducer
Overcoming the Barriers of Graphs on GPUs: Delivering Graph Analy;cs 100X Faster and 40X Cheaper
Overcoming the Barriers of Graphs on GPUs: Delivering Graph Analy;cs 100X Faster and 40X Cheaper November 18, 2015 Super Compu3ng 2015 The Amount of Graph Data is Exploding! Billion+ Edges! 2 Graph Applications
Backtesting with Spark
Backtesting with Spark Patrick Angeles, Cloudera Sandy Ryza, Cloudera Rick Carlin, Intel Sheetal Parade, Intel 1 Traditional Grid Shared storage Storage and compute scale independently Bottleneck on I/O
Cloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
Spark & Spark SQL. High- Speed In- Memory Analytics over Hadoop and Hive Data. Instructor: Duen Horng (Polo) Chau
CSE 6242 / CX 4242 Data and Visual Analytics Georgia Tech Spark & Spark SQL High- Speed In- Memory Analytics over Hadoop and Hive Data Instructor: Duen Horng (Polo) Chau Slides adopted from Matei Zaharia
Stream Processing on IoT Devices using Calvin Framework
Stream Processing on IoT Devices using Calvin Framework by Ameya Nayak A Project Report Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science Supervised
Scalable Machine Learning in R. with H2O
Scalable Machine Learning in R with H2O Erin LeDell @ledell DSC July 2016 Introduction Statistician & Machine Learning Scientist at H2O.ai in Mountain View, California, USA Ph.D. in Biostatistics with
Scalable Tools - Part I Introduction to Scalable Tools
Scalable Tools - Part I Introduction to Scalable Tools Adisak Sukul, Ph.D., Lecturer, Department of Computer Science, adisak@iastate.edu http://web.cs.iastate.edu/~adisak/mbds2018/ Scalable Tools session
BigDataBench- S: An Open- source Scien6fic Big Data Benchmark Suite
BigDataBench- S: An Open- source Scien6fic Big Data Benchmark Suite Xinhui Tian, Shaopeng Dai, Zhihui Du, Wanling Gao, Rui Ren, Yaodong Cheng, Zhifei Zhang, Zhen Jia, Peijian Wang and Jianfeng Zhan INSTITUTE
The Datacenter Needs an Operating System
UC BERKELEY The Datacenter Needs an Operating System Anthony D. Joseph LASER Summer School September 2013 My Talks at LASER 2013 1. AMP Lab introduction 2. The Datacenter Needs an Operating System 3. Mesos,
Creating a Recommender System. An Elasticsearch & Apache Spark approach
Creating a Recommender System An Elasticsearch & Apache Spark approach My Profile SKILLS Álvaro Santos Andrés Big Data & Analytics Solution Architect in Ericsson with more than 12 years of experience focused
Data Analytics and Machine Learning: From Node to Cluster
Data Analytics and Machine Learning: From Node to Cluster Presented by Viswanath Puttagunta Ganesh Raju Understanding use cases to optimize on ARM Ecosystem Date BKK16-404B March 10th, 2016 Event Linaro
Machine Learning Crash Course: Part I
Machine Learning Crash Course: Part I Ariel Kleiner August 21, 2012 Machine learning exists at the intersec
Apache SystemML Declarative Machine Learning
Apache Big Data Seville 2016 Apache SystemML Declarative Machine Learning Luciano Resende About Me Luciano Resende (lresende@apache.org) Architect and community liaison at Have been contributing to open
2/26/2017. Originally developed at the University of California - Berkeley's AMPLab
Apache is a fast and general engine for large-scale data processing aims at achieving the following goals in the Big data context Generality: diverse workloads, operators, job sizes Low latency: sub-second
RESILIENT DISTRIBUTED DATASETS: A FAULT-TOLERANT ABSTRACTION FOR IN-MEMORY CLUSTER COMPUTING
RESILIENT DISTRIBUTED DATASETS: A FAULT-TOLERANT ABSTRACTION FOR IN-MEMORY CLUSTER COMPUTING Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael J. Franklin,
CSC 261/461 Database Systems Lecture 24. Spring 2017 MW 3:25 pm 4:40 pm January 18 May 3 Dewey 1101
CSC 261/461 Database Systems Lecture 24 Spring 2017 MW 3:25 pm 4:40 pm January 18 May 3 Dewey 1101 Announcements Term Paper due on April 20 April 23 Project 1 Milestone 4 is out Due on 05/03 But I would
Spark. In- Memory Cluster Computing for Iterative and Interactive Applications
Spark In- Memory Cluster Computing for Iterative and Interactive Applications Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael Franklin, Scott Shenker,
CS 6453: Parameter Server. Soumya Basu March 7, 2017
CS 6453: Parameter Server Soumya Basu March 7, 2017 What is a Parameter Server? Server for large scale machine learning problems Machine learning tasks in a nutshell: Feature Extraction (1, 1, 1) (2, -1,
15.1 Data flow vs. traditional network programming
CME 323: Distributed Algorithms and Optimization, Spring 2017 http://stanford.edu/~rezab/dao. Instructor: Reza Zadeh, Matroid and Stanford. Lecture 15, 5/22/2017. Scribed by D. Penner, A. Shoemaker, and
A very short introduction
A very short introduction General purpose compute engine for clusters batch / interactive / streaming used by and many others History developed in 2009 @ UC Berkeley joined the Apache foundation in 2013
A Distributed Data- Parallel Execu3on Framework in the Kepler Scien3fic Workflow System
A Distributed Data- Parallel Execu3on Framework in the Kepler Scien3fic Workflow System Ilkay Al(ntas and Daniel Crawl San Diego Supercomputer Center UC San Diego Jianwu Wang UMBC WorDS.sdsc.edu Computa3onal
Re- op&mizing Data Parallel Compu&ng
Re- op&mizing Data Parallel Compu&ng Sameer Agarwal Srikanth Kandula, Nicolas Bruno, Ming- Chuan Wu, Ion Stoica, Jingren Zhou UC Berkeley A Data Parallel Job can be a collec/on of maps, A Data Parallel
Processing of big data with Apache Spark
Processing of big data with Apache Spark JavaSkop 18 Aleksandar Donevski AGENDA What is Apache Spark? Spark vs Hadoop MapReduce Application Requirements Example Architecture Application Challenges 2 WHAT
Efficient Memory and Bandwidth Management for Industrial Strength Kirchhoff Migra<on
Efficient Memory and Bandwidth Management for Industrial Strength Kirchhoff Migra
Unifying Big Data Workloads in Apache Spark
Unifying Big Data Workloads in Apache Spark Hossein Falaki @mhfalaki Outline What s Apache Spark Why Unification Evolution of Unification Apache Spark + Databricks Q & A What s Apache Spark What is Apache
Scalable Automated Model Search
Scalable Automated Model Search Evan Sparks Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2014-122 http://www.eecs.berkeley.edu/pubs/techrpts/2014/eecs-2014-122.html
Index. bfs() function, 225 Big data characteristics, 2 variety, 3 velocity, 3 veracity, 3 volume, 2 Breadth-first search algorithm, 220, 225
Index A Anonymous function, 66 Apache Hadoop, 1 Apache HBase, 42 44 Apache Hive, 6 7, 230 Apache Kafka, 8, 178 Apache License, 7 Apache Mahout, 5 Apache Mesos, 38 42 Apache Pig, 7 Apache Spark, 9 Apache
Jialin Liu, Evan Racah, Quincey Koziol, Richard Shane Canon, Alex Gittens, Lisa Gerhardt, Suren Byna, Mike F. Ringenburg, Prabhat
H5Spark H5Spark: Bridging the I/O Gap between Spark and Scien9fic Data Formats on HPC Systems Jialin Liu, Evan Racah, Quincey Koziol, Richard Shane Canon, Alex Gittens, Lisa Gerhardt, Suren Byna, Mike
KNIME for the life sciences Cambridge Meetup
KNIME for the life sciences Cambridge Meetup Greg Landrum, Ph.D. KNIME.com AG 12 July 2016 What is KNIME? A bit of motivation: tool blending, data blending, documentation, automation, reproducibility More
IBM Data Science Experience White paper. SparkR. Transforming R into a tool for big data analytics
IBM Data Science Experience White paper R Transforming R into a tool for big data analytics 2 R Executive summary This white paper introduces R, a package for the R statistical programming language that
Why do we need graph processing?
Why do we need graph processing? Community detection: suggest followers? Determine what products people will like Count how many people are in different communities (polling?) Graphs are Everywhere Group
Machine Learning for Large-Scale Data Analysis and Decision Making A. Distributed Machine Learning Week #9
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Distributed Machine Learning Week #9 Today Distributed computing for machine learning Background MapReduce/Hadoop & Spark Theory
Hadoop 2.x Core: YARN, Tez, and Spark. Hortonworks Inc All Rights Reserved
Hadoop 2.x Core: YARN, Tez, and Spark YARN Hadoop Machine Types top-of-rack switches core switch client machines have client-side software used to access a cluster to process data master nodes run Hadoop
Twitter data Analytics using Distributed Computing
Twitter data Analytics using Distributed Computing Uma Narayanan Athrira Unnikrishnan Dr. Varghese Paul Dr. Shelbi Joseph Research Scholar M.tech Student Professor Assistant Professor Dept. of IT, SOE
Specialist ICT Learning
Specialist ICT Learning APPLIED DATA SCIENCE AND BIG DATA ANALYTICS GTBD7 Course Description This intensive training course provides theoretical and technical aspects of Data Science and Business Analytics.
Distributed Machine Learning: An Intro. Chen Huang
: An Intro. Chen Huang Feature Engineering Group, Data Mining Lab, Big Data Research Center, UESTC Contents Background Some Examples Model Parallelism & Data Parallelism Parallelization Mechanisms Synchronous
Performance Evaluation of a MongoDB and Hadoop Platform for Scientific Data Analysis
Performance Evaluation of a MongoDB and Hadoop Platform for Scientific Data Analysis Elif Dede, Madhusudhan Govindaraju Lavanya Ramakrishnan, Dan Gunter, Shane Canon Department of Computer Science, Binghamton
Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL. May 2015
Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document
Machine Learning at the Limit
Machine Learning at the Limit John Canny*^ * Computer Science Division University of California, Berkeley ^ Yahoo Research Labs @GTC, March, 2015 My Other Job(s) Yahoo [Chen, Pavlov, Canny, KDD 2009]*
Distributed Computation Models
Distributed Computation Models SWE 622, Spring 2017 Distributed Software Engineering Some slides ack: Jeff Dean HW4 Recap https://b.socrative.com/ Class: SWE622 2 Review Replicating state machines Case
Accelerating Spark Workloads using GPUs
Accelerating Spark Workloads using GPUs Rajesh Bordawekar, Minsik Cho, Wei Tan, Benjamin Herta, Vladimir Zolotov, Alexei Lvov, Liana Fong, and David Kung IBM T. J. Watson Research Center 1 Outline Spark
Analytics in Spark. Yanlei Diao Tim Hunter. Slides Courtesy of Ion Stoica, Matei Zaharia and Brooke Wenig
Analytics in Spark Yanlei Diao Tim Hunter Slides Courtesy of Ion Stoica, Matei Zaharia and Brooke Wenig Outline 1. A brief history of Big Data and Spark 2. Technical summary of Spark 3. Unified analytics
Apache Spark and Scala Certification Training
About Intellipaat Intellipaat is a fast-growing professional training provider that is offering training in over 150 most sought-after tools and technologies. We have a learner base of 600,000 in over
Blended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a)
Blended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a) Cloudera s Developer Training for Apache Spark and Hadoop delivers the key concepts and expertise need to develop high-performance
Benchmarking Apache Flink and Apache Spark DataFlow Systems on Large-Scale Distributed Machine Learning Algorithms
Benchmarking Apache Flink and Apache Spark DataFlow Systems on Large-Scale Distributed Machine Learning Algorithms Candidate Andrea Spina Advisor Prof. Sonia Bergamaschi Co-Advisor Dr. Tilmann Rabl Co-Advisor
The Stratosphere Platform for Big Data Analytics
The Stratosphere Platform for Big Data Analytics Hongyao Ma Franco Solleza April 20, 2015 Stratosphere Stratosphere Stratosphere Big Data Analytics BIG Data Heterogeneous datasets: structured / unstructured
Decentralized and Distributed Machine Learning Model Training with Actors
Decentralized and Distributed Machine Learning Model Training with Actors Travis Addair Stanford University taddair@stanford.edu Abstract Training a machine learning model with terabytes to petabytes of
Webinar Series TMIP VISION
Webinar Series TMIP VISION TMIP provides technical support and promotes knowledge and information exchange in the transportation planning and modeling community. Today s Goals To Consider: Parallel Processing
Big Data. Big Data Analyst. Big Data Engineer. Big Data Architect
Big Data Big Data Analyst INTRODUCTION TO BIG DATA ANALYTICS ANALYTICS PROCESSING TECHNIQUES DATA TRANSFORMATION & BATCH PROCESSING REAL TIME (STREAM) DATA PROCESSING Big Data Engineer BIG DATA FOUNDATION
Machine learning library for Apache Flink
Machine learning library for Apache Flink MTP Mid Term Report submitted to Indian Institute of Technology Mandi for partial fulfillment of the degree of B. Tech. by Devang Bacharwar (B2059) under the guidance
A Tutorial on Apache Spark
A Tutorial on Apache Spark A Practical Perspective By Harold Mitchell The Goal Learning Outcomes The Goal Learning Outcomes NOTE: The setup, installation, and examples assume Windows user Learn the following:
Lambda Architecture with Apache Spark
Lambda Architecture with Apache Spark Michael Hausenblas, Chief Data Engineer MapR First Galway Data Meetup, 2015-02-03 2015 MapR Technologies 2015 MapR Technologies 1 Polyglot Processing 2015 2014 MapR