CSC 4510 Machine Learning
|
|
|
- Adela Cameron
- 6 years ago
- Views:
Transcription
1 4: Regression (con.nued) CSC 4510 Machine Learning Dr. Mary Angela Papalaskari Department of CompuBng Sciences Villanova University Course website: The slides in this presentabon are adapted from: The Stanford online ML course hmp:// class.org/ 1
2 Last Bme IntroducBon to linear regression IntuiBon least squares approximabon IntuiBon gradient descent algorithm Hands on: Simple example using excel 2
3 Today How to apply gradient descent to minimize the cost funcbon for regression linear algebra refresher 3
4 Reminder: sample problem Housing Prices (Portland, OR) Price (in 1000s of dollars) Size (feet 2 ) 4
5 Reminder: NotaBon Training set of housing prices (Portland, OR) Size in feet 2 (x) Notation: m = Number of training examples x s = input variable / features y s = output variable / target variable Price ($) in 1000's (y)
6 Reminder: Learning algorithm for hypothesis funcbon h Training Set Learning Algorithm Linear Hypothesis: Size of house Univariate linear regression) h Estimate price 6
7 Reminder: Learning algorithm for hypothesis funcbon h Training Set Learning Algorithm Linear Hypothesis: Size of house Univariate linear regression) h Estimate price 7
8 Gradient descent algorithm Linear Regression Model 8
9 Today How to apply gradient descent to minimize the cost funcbon for regression 1. a closer look at the cost funcbon 2. applying gradient descent to find the minimum of the cost funcbon linear algebra refresher 9
10 Hypothesis: Parameters: Cost Function: Goal: 10
11 Hypothesis: Simplified θ 0 = 0 Parameters: Cost Function: Goal: 11
12 (for fixed θ 1 this is a function of x) (function of the parameter θ 1 ) θ 0 = y x h θ (x) = x 12
13 (for fixed θ 1 this is a function of x) (function of the parameter θ 1 ) θ 0 = 0 3 y x h θ (x) = 0.5x 13
14 (for fixed θ 1 this is a function of x) (function of the parameter θ 1 ) θ 0 = 0 3 y x h θ (x) = 0 14
15 What if θ 0 0? Hypothesis: Parameters: Cost Function: Goal: 15
16 (for fixed θ 0, θ 1, this is a function of x) (function of the parameters θ 0, θ 1 ) Price ($) in 1000 s Size in feet 2 (x) h θ (x) = x 16
17 17
18 (for fixed θ 0, θ 1, this is a function of x) (function of the parameters θ 0, θ 1 ) 18
19 (for fixed θ 0, θ 1, this is a function of x) (function of the parameters θ 0, θ 1 ) 19
20 (for fixed θ 0, θ 1, this is a function of x) (function of the parameters θ 0, θ 1 ) 20
21 (for fixed θ 0, θ 1, this is a function of x) (function of the parameters θ 0, θ 1 ) 21
22 Today How to apply gradient descent to minimize the cost funcbon for regression 1. a closer look at the cost funcbon 2. applying gradient descent to find the minimum of the cost funcbon linear algebra refresher 22
23 Have some function Want Gradient descent algorithm outline: Start with some Keep changing to reduce until we hopefully end up at a minimum 23
24 Have some function Want Gradient descent algorithm 24
25 Have some function Want Gradient descent algorithm learning rate 25
26 If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge. 26
27 at local minimum Current value of 27
28 Gradient descent can converge to a local minimum, even with the learning rate α fixed. 28
29 Gradient descent algorithm Linear Regression Model 29
30 Gradient descent algorithm update and simultaneously 30
31 J(θ 0,θ 1 ) θ 1 θ 0 31
32 J(θ 0,θ 1 ) θ 1 θ 0 32
33 33
34 (for fixed, this is a function of x) (function of the parameters ) 34
35 (for fixed, this is a function of x) (function of the parameters ) 35
36 (for fixed, this is a function of x) (function of the parameters ) 36
37 (for fixed, this is a function of x) (function of the parameters ) 37
38 (for fixed, this is a function of x) (function of the parameters ) 38
39 (for fixed, this is a function of x) (function of the parameters ) 39
40 (for fixed, this is a function of x) (function of the parameters ) 40
41 (for fixed, this is a function of x) (function of the parameters ) 41
42 (for fixed, this is a function of x) (function of the parameters ) 42
43 Batch Gradient Descent Batch : Each step of gradient descent uses all the training examples. Alternative: process part of the dataset for each step of the algorithm. The slides in this presentabon are adapted from: The Stanford online ML course hmp:// class.org/ 43
44 What s next? We are not in univariate regression anymore: Size (feet 2 ) Number of bedrooms Number of floors Age of home Price ($1000) (years)
45 What s next? We are not in univariate regression anymore: Size (feet 2 ) Number of bedrooms Number of floors Age of home Price ($1000) (years)
46 What s next? We are not in univariate regression anymore: Size (feet 2 ) Number of bedrooms Number of floors Age of home Price ($1000) (years)
47 Today How to apply gradient descent to minimize the cost funcbon for regression 1. a closer look at the cost funcbon 2. applying gradient descent to find the minimum of the cost funcbon linear algebra refresher 47
48 Matrix: Rectangular array of numbers Matrix Elements (entries of matrix), entry in the row, column. Dimension of matrix: number of rows x number of columns eg: 4 x 2 48
49 Vector: An n x 1 matrix. element 1-indexed vs 0-indexed: 49
50 Matrix Addition 50
51 Scalar Multiplication 51
52 Combination of Operands 52
53 Matrix-vector multiplication 53
54 Details: m x n matrix (m rows, n columns) n x 1 matrix (n-dimensional vector) m-dimensional vector To get, multiply s row with elements of vector, and add them up. 54
55 Example 55
56 House sizes: 56
57 Example 57
58 Details: m x n matrix (m rows, n columns) n x o matrix (n rows, o columns) m x o matrix The column of the matrix is obtained by multiplying with the column of. (for i = 1,2,,o) 58
59 Example 59
60 House sizes: Have 3 competing hypotheses: Matrix Matrix 60
61 Let and be matrices. Then in general, (not commutative.) E.g. 61
62 Let Let Compute Compute 62
63 Identity Matrix Denoted (or ). Examples of identity matrices: 2 x 2 3 x 3 For any matrix, 4 x 4 63
64 Not all numbers have an inverse. Matrix inverse: If A is an m x m matrix, and if it has an inverse, Matrices that don t have an inverse are singular or degenerate 64
65 Matrix Transpose Example: Let Then be an m x n matrix, and let is an n x m matrix, and 65
CSC 4510 Machine Learning
5: Mul'variate Regression CSC 4510 Machine Learning Dr. Mary Angela Papalaskari Department of CompuBng Sciences Villanova Course website: www.csc.villanova.edu/~map/4510/ The slides in this presentabon
Programming Exercise 1: Linear Regression
Programming Exercise 1: Linear Regression Machine Learning Introduction In this exercise, you will implement linear regression and get to see it work on data. Before starting on this programming exercise,
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 20 2018 Review Solution for multiple linear regression can be computed in closed form
REGRESSION ANALYSIS : LINEAR BY MAUAJAMA FIRDAUS & TULIKA SAHA
REGRESSION ANALYSIS : LINEAR BY MAUAJAMA FIRDAUS & TULIKA SAHA MACHINE LEARNING It is the science of getting computer to learn without being explicitly programmed. Machine learning is an area of artificial
CS535 Big Data Fall 2017 Colorado State University 10/10/2017 Sangmi Lee Pallickara Week 8- A.
CS535 Big Data - Fall 2017 Week 8-A-1 CS535 BIG DATA FAQs Term project proposal New deadline: Tomorrow PA1 demo PART 1. BATCH COMPUTING MODELS FOR BIG DATA ANALYTICS 5. ADVANCED DATA ANALYTICS WITH APACHE
Linear Regression QuadraJc Regression Year
Linear Regression Regression Given: n Data X = x (1),...,x (n)o where x (i) 2 R d n Corresponding labels y = y (1),...,y (n)o where y (i) 2 R September Arc+c Sea Ice Extent (1,000,000 sq km) 9 8 7 6 5
Machine Learning Lecture-1
Machine Learning Lecture-1 Programming Club Indian Institute of Technology Kanpur pclubiitk@gmail.com February 25, 2016 Programming Club (IIT Kanpur) ML-1 February 25, 2016 1 / 18 Acknowledgement This
CSC 411: Lecture 02: Linear Regression
CSC 411: Lecture 02: Linear Regression Raquel Urtasun & Rich Zemel University of Toronto Sep 16, 2015 Urtasun & Zemel (UofT) CSC 411: 02-Regression Sep 16, 2015 1 / 16 Today Linear regression problem continuous
x = 12 x = 12 1x = 16
2.2 - The Inverse of a Matrix We've seen how to add matrices, multiply them by scalars, subtract them, and multiply one matrix by another. The question naturally arises: Can we divide one matrix by another?
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 18 2018 Logistics HW 1 is on Piazza and Gradescope Deadline: Friday, Sept. 28, 2018 Office
AH Matrices.notebook November 28, 2016
Matrices Numbers are put into arrays to help with multiplication, division etc. A Matrix (matrices pl.) is a rectangular array of numbers arranged in rows and columns. Matrices If there are m rows and
Vector: A series of scalars contained in a column or row. Dimensions: How many rows and columns a vector or matrix has.
ASSIGNMENT 0 Introduction to Linear Algebra (Basics of vectors and matrices) Due 3:30 PM, Tuesday, October 10 th. Assignments should be submitted via e-mail to: matlabfun.ucsd@gmail.com You can also submit
COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS18. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions
COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS18 Lecture 2: Linear Regression Gradient Descent Non-linear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Simplest
Gradient Descent - Problem of Hiking Down a Mountain
Gradient Descent - Problem of Hiking Down a Mountain Udacity Have you ever climbed a mountain? I am sure you had to hike down at some point? Hiking down is a great exercise and it is going to help us understand
Linear Regression Implementation
Linear Regression Implementation 1 When you experience regression, you go back in some way. The process of regressing is to go back to a less perfect or less developed state. Modeling data is focused on
Computational Intelligence (CS) SS15 Homework 1 Linear Regression and Logistic Regression
Computational Intelligence (CS) SS15 Homework 1 Linear Regression and Logistic Regression Anand Subramoney Points to achieve: 17 pts Extra points: 5* pts Info hour: 28.04.2015 14:00-15:00, HS i12 (ICK1130H)
Rapid growth of massive datasets
Overview Rapid growth of massive datasets E.g., Online activity, Science, Sensor networks Data Distributed Clusters are Pervasive Data Distributed Computing Mature Methods for Common Problems e.g., classification,
3 Types of Gradient Descent Algorithms for Small & Large Data Sets
3 Types of Gradient Descent Algorithms for Small & Large Data Sets Introduction Gradient Descent Algorithm (GD) is an iterative algorithm to find a Global Minimum of an objective function (cost function)
CS545 Contents IX. Inverse Kinematics. Reading Assignment for Next Class. Analytical Methods Iterative (Differential) Methods
CS545 Contents IX Inverse Kinematics Analytical Methods Iterative (Differential) Methods Geometric and Analytical Jacobian Jacobian Transpose Method Pseudo-Inverse Pseudo-Inverse with Optimization Extended
Linear Regression & Gradient Descent
Linear Regression & Gradient Descent These slides were assembled by Byron Boots, with grateful acknowledgement to Eric Eaton and the many others who made their course materials freely available online.
OUTLINES. Variable names in MATLAB. Matrices, Vectors and Scalar. Entering a vector Colon operator ( : ) Mathematical operations on vectors.
1 LECTURE 3 OUTLINES Variable names in MATLAB Examples Matrices, Vectors and Scalar Scalar Vectors Entering a vector Colon operator ( : ) Mathematical operations on vectors examples 2 VARIABLE NAMES IN
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 22 2019 Outline Practical issues in Linear Regression Outliers Categorical variables Lab
Homework 5. Due: April 20, 2018 at 7:00PM
Homework 5 Due: April 20, 2018 at 7:00PM Written Questions Problem 1 (25 points) Recall that linear regression considers hypotheses that are linear functions of their inputs, h w (x) = w, x. In lecture,
Therefore, after becoming familiar with the Matrix Method, you will be able to solve a system of two linear equations in four different ways.
Grade 9 IGCSE A1: Chapter 9 Matrices and Transformations Materials Needed: Straightedge, Graph Paper Exercise 1: Matrix Operations Matrices are used in Linear Algebra to solve systems of linear equations.
Basics of Java Programming variables, assignment, and input
Basics of Java Programming variables, assignment, and input CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/
COMPUTATIONAL INTELLIGENCE (CS) (INTRODUCTION TO MACHINE LEARNING) SS16. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions
COMPUTATIONAL INTELLIGENCE (CS) (INTRODUCTION TO MACHINE LEARNING) SS16 Lecture 2: Linear Regression Gradient Descent Non-linear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Regression
ITI Introduction to Computing II Winter 2018
ITI 1121. Introduction to Computing II Winter 2018 Assignment 1 (Last modified on 27 janvier 2018) Deadline: February 2, 11:30 pm [ PDF ] Learning objectives Applying basic object oriented programming
All lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
A Brief Look at Optimization
A Brief Look at Optimization CSC 412/2506 Tutorial David Madras January 18, 2018 Slides adapted from last year s version Overview Introduction Classes of optimization problems Linear programming Steepest
PetShop (BYU Students, SIGGRAPH 2006)
Now Playing: PetShop (BYU Students, SIGGRAPH 2006) My Mathematical Mind Spoon From Gimme Fiction Released May 10, 2005 Geometric Objects in Computer Graphics Rick Skarbez, Instructor COMP 575 August 30,
Matrices. A Matrix (This one has 2 Rows and 3 Columns) To add two matrices: add the numbers in the matching positions:
Matrices A Matrix is an array of numbers: We talk about one matrix, or several matrices. There are many things we can do with them... Adding A Matrix (This one has 2 Rows and 3 Columns) To add two matrices:
6. Linear Discriminant Functions
6. Linear Discriminant Functions Linear Discriminant Functions Assumption: we know the proper forms for the discriminant functions, and use the samples to estimate the values of parameters of the classifier
HMC CS 158, Fall 2017 Problem Set 3 Programming: Regularized Polynomial Regression
HMC CS 158, Fall 2017 Problem Set 3 Programming: Regularized Polynomial Regression Goals: To open up the black-box of scikit-learn and implement regression models. To investigate how adding polynomial
Today. Today. Introduction. Matrices. Matrices. Computergrafik. Transformations & matrices Introduction Matrices
Computergrafik Matthias Zwicker Universität Bern Herbst 2008 Today Transformations & matrices Introduction Matrices Homogeneous Affine transformations Concatenating transformations Change of Common coordinate
Gradient Descent Optimization Algorithms for Deep Learning Batch gradient descent Stochastic gradient descent Mini-batch gradient descent
Gradient Descent Optimization Algorithms for Deep Learning Batch gradient descent Stochastic gradient descent Mini-batch gradient descent Slide credit: http://sebastianruder.com/optimizing-gradient-descent/index.html#batchgradientdescent
Machine Learning Basics. Sargur N. Srihari
Machine Learning Basics Sargur N. srihari@cedar.buffalo.edu 1 Overview Deep learning is a specific type of ML Necessary to have a solid understanding of the basic principles of ML 2 Topics Stochastic Gradient
Algorithms in everyday life. Algorithms. Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs
Data and Algorithms Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs Algorithms in everyday life CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari
06: Logistic Regression
06_Logistic_Regression 06: Logistic Regression Previous Next Index Classification Where y is a discrete value Develop the logistic regression algorithm to determine what class a new input should fall into
Optimization. Industrial AI Lab.
Optimization Industrial AI Lab. Optimization An important tool in 1) Engineering problem solving and 2) Decision science People optimize Nature optimizes 2 Optimization People optimize (source: http://nautil.us/blog/to-save-drowning-people-ask-yourself-what-would-light-do)
ECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 22, 2016 Course Information Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/ ECS289G_Fall2016/main.html My office: Mathematical Sciences
0_PreCNotes17 18.notebook May 16, Chapter 12
Chapter 12 Notes BASIC MATRIX OPERATIONS Matrix (plural: Matrices) an n x m array of elements element a ij Example 1 a 21 = a 13 = Multiply Matrix by a Scalar Distribute scalar to all elements Addition
Maths for Signals and Systems Linear Algebra in Engineering. Some problems by Gilbert Strang
Maths for Signals and Systems Linear Algebra in Engineering Some problems by Gilbert Strang Problems. Consider u, v, w to be non-zero vectors in R 7. These vectors span a vector space. What are the possible
Programming Exercise 3: Multi-class Classification and Neural Networks
Programming Exercise 3: Multi-class Classification and Neural Networks Machine Learning Introduction In this exercise, you will implement one-vs-all logistic regression and neural networks to recognize
Matrix Computations and " Neural Networks in Spark
Matrix Computations and " Neural Networks in Spark Reza Zadeh Paper: http://arxiv.org/abs/1509.02256 Joint work with many folks on paper. @Reza_Zadeh http://reza-zadeh.com Training Neural Networks Datasets
Limited view X-ray CT for dimensional analysis
Limited view X-ray CT for dimensional analysis G. A. JONES ( GLENN.JONES@IMPERIAL.AC.UK ) P. HUTHWAITE ( P.HUTHWAITE@IMPERIAL.AC.UK ) NON-DESTRUCTIVE EVALUATION GROUP 1 Outline of talk Industrial X-ray
Linear Algebra Review
CS 1674: Intro to Computer Vision Linear Algebra Review Prof. Adriana Kovashka University of Pittsburgh January 11, 2018 What are images? (in Matlab) Matlab treats images as matrices of numbers To proceed,
Machine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
SOME CONCEPTS IN DISCRETE COSINE TRANSFORMS ~ Jennie G. Abraham Fall 2009, EE5355
SOME CONCEPTS IN DISCRETE COSINE TRANSFORMS ~ Jennie G. Abraham Fall 009, EE5355 Under Digital Image and Video Processing files by Dr. Min Wu Please see lecture10 - Unitary Transform lecture11 - Transform
Transformation. Jane Li Assistant Professor Mechanical Engineering & Robotics Engineering
RBE 550 MOTION PLANNING BASED ON DR. DMITRY BERENSON S RBE 550 Transformation Jane Li Assistant Professor Mechanical Engineering & Robotics Engineering http://users.wpi.edu/~zli11 Announcement Project
Systems of Inequalities and Linear Programming 5.7 Properties of Matrices 5.8 Matrix Inverses
5 5 Systems and Matrices Systems and Matrices 5.6 Systems of Inequalities and Linear Programming 5.7 Properties of Matrices 5.8 Matrix Inverses Sections 5.6 5.8 2008 Pearson Addison-Wesley. All rights
Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs
Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova
Image Registration Lecture 4: First Examples
Image Registration Lecture 4: First Examples Prof. Charlene Tsai Outline Example Intensity-based registration SSD error function Image mapping Function minimization: Gradient descent Derivative calculation
2D transformations: An introduction to the maths behind computer graphics
2D transformations: An introduction to the maths behind computer graphics Lecturer: Dr Dan Cornford d.cornford@aston.ac.uk http://wiki.aston.ac.uk/dancornford CS2150, Computer Graphics, Aston University,
Optimization Plugin for RapidMiner. Venkatesh Umaashankar Sangkyun Lee. Technical Report 04/2012. technische universität dortmund
Optimization Plugin for RapidMiner Technical Report Venkatesh Umaashankar Sangkyun Lee 04/2012 technische universität dortmund Part of the work on this technical report has been supported by Deutsche Forschungsgemeinschaft
(Creating Arrays & Matrices) Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB (Creating Arrays & Matrices) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional
Physics 326G Winter Class 2. In this class you will learn how to define and work with arrays or vectors.
Physics 326G Winter 2008 Class 2 In this class you will learn how to define and work with arrays or vectors. Matlab is designed to work with arrays. An array is a list of numbers (or other things) arranged
Distributed Machine Learning" on Spark
Distributed Machine Learning" on Spark Reza Zadeh @Reza_Zadeh http://reza-zadeh.com Outline Data flow vs. traditional network programming Spark computing engine Optimization Example Matrix Computations
Algorithms in everyday life. Algorithms. Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs
Data and Algorithms Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs Algorithms in everyday life CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari
Perceptron: This is convolution!
Perceptron: This is convolution! v v v Shared weights v Filter = local perceptron. Also called kernel. By pooling responses at different locations, we gain robustness to the exact spatial location of image
Deep Neural Networks Optimization
Deep Neural Networks Optimization Creative Commons (cc) by Akritasa http://arxiv.org/pdf/1406.2572.pdf Slides from Geoffrey Hinton CSC411/2515: Machine Learning and Data Mining, Winter 2018 Michael Guerzhoy
Perceptron Introduction to Machine Learning. Matt Gormley Lecture 5 Jan. 31, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Perceptron Matt Gormley Lecture 5 Jan. 31, 2018 1 Q&A Q: We pick the best hyperparameters
CSE 547: Machine Learning for Big Data Spring Problem Set 2. Please read the homework submission policies.
CSE 547: Machine Learning for Big Data Spring 2019 Problem Set 2 Please read the homework submission policies. 1 Principal Component Analysis and Reconstruction (25 points) Let s do PCA and reconstruct
Introduction to MATLAB
CHEE MATLAB Tutorial Introduction to MATLAB Introduction In this tutorial, you will learn how to enter matrices and perform some matrix operations using MATLAB. MATLAB is an interactive program for numerical
Conditional Statements
Conditional Statements CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/ Some
Chapter 1: Number and Operations
Chapter 1: Number and Operations 1.1 Order of operations When simplifying algebraic expressions we use the following order: 1. Perform operations within a parenthesis. 2. Evaluate exponents. 3. Multiply
CSC 1051 Data Structures and Algorithms I. Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University
Last Class CSC 1051 Data Structures and Algorithms I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/ Some slides in this
Derek Bridge School of Computer Science and Information Technology University College Cork
CS468: Artificial Intelligence I Ordinary Least Squares Regression Derek Bridge School of Computer Science and Information Technology University College Cork Initialization In [4]: %load_ext autoreload
4. Linear Algebra. In maple, it is first necessary to call in the linear algebra package. This is done by the following maple command
4. Linear Algebra Vectors and Matrices Maple has this wonderful linear algebra package. It has to do with vectors and matrices. A vector is simply an array of numbers written as () where a matrix is written
CS 395T Lecture 12: Feature Matching and Bundle Adjustment. Qixing Huang October 10 st 2018
CS 395T Lecture 12: Feature Matching and Bundle Adjustment Qixing Huang October 10 st 2018 Lecture Overview Dense Feature Correspondences Bundle Adjustment in Structure-from-Motion Image Matching Algorithm
Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs
Algorithms and Java basics: pseudocode, variables, assignment, and interactive programs CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova
Affine Transformation. Edith Law & Mike Terry
Affine Transformation Edith Law & Mike Terry Graphic Models vs. Images Computer Graphics: the creation, storage and manipulation of images and their models Model: a mathematical representation of an image
Humanoid Robotics. Least Squares. Maren Bennewitz
Humanoid Robotics Least Squares Maren Bennewitz Goal of This Lecture Introduction into least squares Use it yourself for odometry calibration, later in the lecture: camera and whole-body self-calibration
An array is a collection of data that holds fixed number of values of same type. It is also known as a set. An array is a data type.
Data Structures Introduction An array is a collection of data that holds fixed number of values of same type. It is also known as a set. An array is a data type. Representation of a large number of homogeneous
Scaled Machine Learning at Matroid
Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh http://reza-zadeh.com Machine Learning Pipeline Learning Algorithm Replicate model Data Trained Model Serve Model Repeat entire pipeline Scaling
CSC 1051 Algorithms and Data Structures I. Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University
Algorithms CSC 1051 Algorithms and Data Structures I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/ Some slides in this
Independent systems consist of x
5.1 Simultaneous Linear Equations In consistent equations, *Find the solution to each system by graphing. 1. y Independent systems consist of x Three Cases: A. consistent and independent 2. y B. inconsistent
CS 179 Lecture 16. Logistic Regression & Parallel SGD
CS 179 Lecture 16 Logistic Regression & Parallel SGD 1 Outline logistic regression (stochastic) gradient descent parallelizing SGD for neural nets (with emphasis on Google s distributed neural net implementation)
Selection Statements and operators
Selection Statements and operators CSC 1051 Data Structures and Algorithms I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/
Today. Gradient descent for minimization of functions of real variables. Multi-dimensional scaling. Self-organizing maps
Today Gradient descent for minimization of functions of real variables. Multi-dimensional scaling Self-organizing maps Gradient Descent Derivatives Consider function f(x) : R R. The derivative w.r.t. x
CSE 546 Machine Learning, Autumn 2013 Homework 2
CSE 546 Machine Learning, Autumn 2013 Homework 2 Due: Monday, October 28, beginning of class 1 Boosting [30 Points] We learned about boosting in lecture and the topic is covered in Murphy 16.4. On page
Arrays, Part 3: Multidimensional arrays
Arrays, Part 3: Multidimensional arrays CSC 1051 Data Structures and Algorithms I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/
Array Creation ENGR 1187 MATLAB 2
Array Creation ENGR 1187 MATLAB 2 Today s Topics Arrays: What are they? Vectors Matrices Creating Arrays Today s Topics Arrays: What are they? Vectors Matrices Creating Arrays Arrays: What are they? Before
Understanding Andrew Ng s Machine Learning Course Notes and codes (Matlab version)
Understanding Andrew Ng s Machine Learning Course Notes and codes (Matlab version) Note: All source materials and diagrams are taken from the Coursera s lectures created by Dr Andrew Ng. Everything I have
CPSC 340: Machine Learning and Data Mining. Multi-Class Classification Fall 2017
CPSC 340: Machine Learning and Data Mining Multi-Class Classification Fall 2017 Assignment 3: Admin Check update thread on Piazza for correct definition of trainndx. This could make your cross-validation
3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
CSC 1051 Villanova University. CSC 1051 Data Structures and Algorithms I. Course website:
Repetition CSC 1051 Data Structures and Algorithms I Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/1051/ Some slides in this
Linear Regression Optimization
Gradient Descent Linear Regression Optimization Goal: Find w that minimizes f(w) f(w) = Xw y 2 2 Closed form solution exists Gradient Descent is iterative (Intuition: go downhill!) n w * w Scalar objective:
Arrays, Matrices and Determinants
Arrays, Matrices and Determinants Spreadsheet calculations lend themselves almost automatically to the use of arrays of values. Arrays in Excel can be either one- or two-dimensional. For the solution of
Machine Learning Basics: Stochastic Gradient Descent. Sargur N. Srihari
Machine Learning Basics: Stochastic Gradient Descent Sargur N. srihari@cedar.buffalo.edu 1 Topics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation Sets
Machine Learning and Computational Statistics, Spring 2016 Homework 1: Ridge Regression and SGD
Machine Learning and Computational Statistics, Spring 2016 Homework 1: Ridge Regression and SGD Due: Friday, February 5, 2015, at 6pm (Submit via NYU Classes) Instructions: Your answers to the questions
Matrix Inverse 2 ( 2) 1 = 2 1 2
Name: Matrix Inverse For Scalars, we have what is called a multiplicative identity. This means that if we have a scalar number, call it r, then r multiplied by the multiplicative identity equals r. Without
STAT R Overview. R Intro. R Data Structures. Subsetting. Graphics. January 11, 2018
January 11, 2018 Why use R? R is: a free public domain implementation of S, the standard among (academic) professional statisticians, available for Windows, Mac, and Linux, an object-oriented and functional
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix.
MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix. Row echelon form A matrix is said to be in the row echelon form if the leading entries shift to the
Parallel Stochastic Gradient Descent
University of Montreal August 11th, 2007 CIAR Summer School - Toronto Stochastic Gradient Descent Cost to optimize: E z [C(θ, z)] with θ the parameters and z a training point. Stochastic gradient: θ t+1
Graphics and Interaction Transformation geometry and homogeneous coordinates
433-324 Graphics and Interaction Transformation geometry and homogeneous coordinates Department of Computer Science and Software Engineering The Lecture outline Introduction Vectors and matrices Translation
COMP30019 Graphics and Interaction Transformation geometry and homogeneous coordinates
COMP30019 Graphics and Interaction Transformation geometry and homogeneous coordinates Department of Computer Science and Software Engineering The Lecture outline Introduction Vectors and matrices Translation
Figure (5) Kohonen Self-Organized Map
2- KOHONEN SELF-ORGANIZING MAPS (SOM) - The self-organizing neural networks assume a topological structure among the cluster units. - There are m cluster units, arranged in a one- or two-dimensional array;
2 Second Derivatives. As we have seen, a function f (x, y) of two variables has four different partial derivatives: f xx. f yx. f x y.
2 Second Derivatives As we have seen, a function f (x, y) of two variables has four different partial derivatives: (x, y), (x, y), f yx (x, y), (x, y) It is convenient to gather all four of these into
MATLAB COURSE FALL 2004 SESSION 1 GETTING STARTED. Christian Daude 1
MATLAB COURSE FALL 2004 SESSION 1 GETTING STARTED Christian Daude 1 Introduction MATLAB is a software package designed to handle a broad range of mathematical needs one may encounter when doing scientific