Linear Regression: One-Dimensional Case
|
|
|
- Ann Stewart
- 6 years ago
- Views:
Transcription
1 Linear Regression: One-Dimensional Case Given: a set of N input-response pairs The inputs (x) and the responses (y) are one dimensional scalars Goal: Model the relationship between x and y (CS5350/6350) Linear Models for Regression September 6, / 17
2 Linear Regression: One-Dimensional Case Let s assume the relationship between x and y is linear (CS5350/6350) Linear Models for Regression September 6, / 17
3 Linear Regression: One-Dimensional Case Let s assume the relationship between x and y is linear Linear relationship can be defined by a straight line with parameter w Equation of the straight line: y = wx (CS5350/6350) Linear Models for Regression September 6, / 17
4 Linear Regression: One-Dimensional Case The line may not fit the data exactly (CS5350/6350) Linear Models for Regression September 6, / 17
5 Linear Regression: One-Dimensional Case The line may not fit the data exactly But we can try making the line a reasonable approximation (CS5350/6350) Linear Models for Regression September 6, / 17
6 Linear Regression: One-Dimensional Case The line may not fit the data exactly But we can try making the line a reasonable approximation Error for the pair (x i, y i )pair:e i = y i wx i (CS5350/6350) Linear Models for Regression September 6, / 17
7 Linear Regression: One-Dimensional Case The line may not fit the data exactly But we can try making the line a reasonable approximation Error for the pair (x i, y i )pair:e i = y i wx i The total squared error: E = N i=1 e2 i = N i=1 (y i wx i ) 2 (CS5350/6350) Linear Models for Regression September 6, / 17
8 Linear Regression: One-Dimensional Case The line may not fit the data exactly But we can try making the line a reasonable approximation Error for the pair (x i, y i )pair:e i = y i wx i The total squared error: E = N i=1 e2 i = N i=1 (y i wx i ) 2 The best fitting line is defined by w minimizing the total error E (CS5350/6350) Linear Models for Regression September 6, / 17
9 Linear Regression: One-Dimensional Case The line may not fit the data exactly But we can try making the line a reasonable approximation Error for the pair (x i, y i )pair:e i = y i wx i The total squared error: E = N i=1 e2 i = N i=1 (y i wx i ) 2 The best fitting line is defined by w minimizing the total error E Just requires a little bit of calculus to find it (take derivative, equate to zero..) (CS5350/6350) Linear Models for Regression September 6, / 17
10 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data (CS5350/6350) Linear Models for Regression September 6, / 17
11 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? (CS5350/6350) Linear Models for Regression September 6, / 17
12 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y (CS5350/6350) Linear Models for Regression September 6, / 17
13 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y Linear regression uses the sum-of-squared error notion of closeness (CS5350/6350) Linear Models for Regression September 6, / 17
14 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y Linear regression uses the sum-of-squared error notion of closeness Similar intuition carries over to higher dimensions too (CS5350/6350) Linear Models for Regression September 6, / 17
15 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y Linear regression uses the sum-of-squared error notion of closeness Similar intuition carries over to higher dimensions too Fitting a D-dimensional hyperplane to the data (CS5350/6350) Linear Models for Regression September 6, / 17
16 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y Linear regression uses the sum-of-squared error notion of closeness Similar intuition carries over to higher dimensions too Fitting a D-dimensional hyperplane to the data Hard to visualize in pictures though.. (CS5350/6350) Linear Models for Regression September 6, / 17
17 Analogy to line fitting: In higher dimensions, we will fit hyperplanes For 2-dim. inputs, linear regression fits a 2-dim. plane to the data Many planes are possible. Which one is the best? Intuition: Choose the one which is (on average) closest to the responses Y Linear regression uses the sum-of-squared error notion of closeness Similar intuition carries over to higher dimensions too Fitting a D-dimensional hyperplane to the data Hard to visualize in pictures though.. The hyperplane is defined by parameters w (a D 1 weight vector) (CS5350/6350) Linear Models for Regression September 6, / 17
18 (Formally) Given training data D = {(x 1, y 1 ),...,(x N, y N )} Inputs x i : D-dimensional vectors (R D ), responses y i : scalars (R) (CS5350/6350) Linear Models for Regression September 6, / 17
19 (Formally) Given training data D = {(x 1, y 1 ),...,(x N, y N )} Inputs x i : D-dimensional vectors (R D ), responses y i : scalars (R) The linear model: response is a linear function of the model parameters y = f (x, w) =b + M w j φ j (x) j=1 (CS5350/6350) Linear Models for Regression September 6, / 17
20 (Formally) Given training data D = {(x 1, y 1 ),...,(x N, y N )} Inputs x i : D-dimensional vectors (R D ), responses y i : scalars (R) The linear model: response is a linear function of the model parameters y = f (x, w) =b + M w j φ j (x) j=1 w j s and b are the model parameters (b is an offset) Parameters define the mapping from the inputs to responses (CS5350/6350) Linear Models for Regression September 6, / 17
21 (Formally) Given training data D = {(x 1, y 1 ),...,(x N, y N )} Inputs x i : D-dimensional vectors (R D ), responses y i : scalars (R) The linear model: response is a linear function of the model parameters y = f (x, w) =b + M w j φ j (x) j=1 w j s and b are the model parameters (b is an offset) Parameters define the mapping from the inputs to responses Each φ j is called a basis function Allows change of representation of the input x (often desired) (CS5350/6350) Linear Models for Regression September 6, / 17
22 The linear model: φ =[φ 1,...φ M ] y = b + M w j φ j (x) =b + w T φ(x) j=1 w =[w 1,...,w M ], the weight vector (to learn using the training data) (CS5350/6350) Linear Models for Regression September 6, / 17
23 The linear model: φ =[φ 1,...φ M ] y = b + M w j φ j (x) =b + w T φ(x) j=1 w =[w 1,...,w M ], the weight vector (to learn using the training data) We consider the simplest case: φ(x) = x φ j (x) isthej-th feature of the data (total D features, so M = D) (CS5350/6350) Linear Models for Regression September 6, / 17
24 The linear model: φ =[φ 1,...φ M ] y = b + M w j φ j (x) =b + w T φ(x) j=1 w =[w 1,...,w M ], the weight vector (to learn using the training data) We consider the simplest case: φ(x) = x φ j (x) isthej-th feature of the data (total D features, so M = D) The linear model becomes y = b + D w j x j = b + w T x j=1 (CS5350/6350) Linear Models for Regression September 6, / 17
25 The linear model: φ =[φ 1,...φ M ] y = b + M w j φ j (x) =b + w T φ(x) j=1 w =[w 1,...,w M ], the weight vector (to learn using the training data) We consider the simplest case: φ(x) = x φ j (x) isthej-th feature of the data (total D features, so M = D) The linear model becomes y = b + D w j x j = b + w T x j=1 Note: Nonlinear relationships between x and y can be modeled using suitably chosen φ j s (more when we cover Kernel Methods) (CS5350/6350) Linear Models for Regression September 6, / 17
26 Given training data D = {(x 1, y 1 ),...,(x N, y N )} Fit each training example (x i, y i ) using the linear model y i = b + w T x i (CS5350/6350) Linear Models for Regression September 6, / 17
27 Given training data D = {(x 1, y 1 ),...,(x N, y N )} Fit each training example (x i, y i ) using the linear model y i = b + w T x i A bit of notation abuse: write w =[b, w], write x i =[1, x i ] y i = w T x i (CS5350/6350) Linear Models for Regression September 6, / 17
Lecture 3: Convex sets
Lecture 3: Convex sets Rajat Mittal IIT Kanpur We denote the set of real numbers as R. Most of the time we will be working with space R n and its elements will be called vectors. Remember that a subspace
Support vector machines. Dominik Wisniewski Wojciech Wawrzyniak
Support vector machines Dominik Wisniewski Wojciech Wawrzyniak Outline 1. A brief history of SVM. 2. What is SVM and how does it work? 3. How would you classify this data? 4. Are all the separating lines
Introduction to Supervised Learning
Introduction to Supervised Learning Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA 01003 February 17, 2014 Abstract This document introduces the
Lecture notes on the simplex method September We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2017 CS 6820: Algorithms Lecture notes on the simplex method September 2017 1 The Simplex Method We will present an algorithm to solve linear programs of the form maximize subject
Support Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
7.3 3-D Notes Honors Precalculus Date: Adapted from 11.1 & 11.4
73 3-D Notes Honors Precalculus Date: Adapted from 111 & 114 The Three-Variable Coordinate System I Cartesian Plane The familiar xy-coordinate system is used to represent pairs of numbers (ordered pairs
Data Mining: Concepts and Techniques. Chapter 9 Classification: Support Vector Machines. Support Vector Machines (SVMs)
Data Mining: Concepts and Techniques Chapter 9 Classification: Support Vector Machines 1 Support Vector Machines (SVMs) SVMs are a set of related supervised learning methods used for classification Based
Numerical Optimization
Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y
Data Analysis 3. Support Vector Machines. Jan Platoš October 30, 2017
Data Analysis 3 Support Vector Machines Jan Platoš October 30, 2017 Department of Computer Science Faculty of Electrical Engineering and Computer Science VŠB - Technical University of Ostrava Table of
1 Case study of SVM (Rob)
DRAFT a final version will be posted shortly COS 424: Interacting with Data Lecturer: Rob Schapire and David Blei Lecture # 8 Scribe: Indraneel Mukherjee March 1, 2007 In the previous lecture we saw how
CS261: A Second Course in Algorithms Lecture #7: Linear Programming: Introduction and Applications
CS261: A Second Course in Algorithms Lecture #7: Linear Programming: Introduction and Applications Tim Roughgarden January 26, 2016 1 Preamble With this lecture we commence the second part of the course,
As a consequence of the operation, there are new incidences between edges and triangles that did not exist in K; see Figure II.9.
II.4 Surface Simplification 37 II.4 Surface Simplification In applications it is often necessary to simplify the data or its representation. One reason is measurement noise, which we would like to eliminate,
Non-Bayesian Classifiers Part II: Linear Discriminants and Support Vector Machines
Non-Bayesian Classifiers Part II: Linear Discriminants and Support Vector Machines Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2007 c 2007,
All lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
Lecture 3: Linear Classification
Lecture 3: Linear Classification Roger Grosse 1 Introduction Last week, we saw an example of a learning task called regression. There, the goal was to predict a scalar-valued target from a set of features.
Topic 4: Support Vector Machines
CS 4850/6850: Introduction to achine Learning Fall 2018 Topic 4: Support Vector achines Instructor: Daniel L Pimentel-Alarcón c Copyright 2018 41 Introduction Support vector machines (SVs) are considered
More on Classification: Support Vector Machine
More on Classification: Support Vector Machine The Support Vector Machine (SVM) is a classification method approach developed in the computer science field in the 1990s. It has shown good performance in
Lab 2: Support vector machines
Artificial neural networks, advanced course, 2D1433 Lab 2: Support vector machines Martin Rehn For the course given in 2006 All files referenced below may be found in the following directory: /info/annfk06/labs/lab2
Support vector machines
Support vector machines When the data is linearly separable, which of the many possible solutions should we prefer? SVM criterion: maximize the margin, or distance between the hyperplane and the closest
- Introduction P. Danziger. Linear Algebra. Algebra Manipulation, Solution or Transformation
Linear Algera Linear of line or line like Algera Manipulation, Solution or Transformation Thus Linear Algera is aout the Manipulation, Solution and Transformation of line like ojects. We will also investigate
Basis Functions. Volker Tresp Summer 2017
Basis Functions Volker Tresp Summer 2017 1 Nonlinear Mappings and Nonlinear Classifiers Regression: Linearity is often a good assumption when many inputs influence the output Some natural laws are (approximately)
Learning from Data Linear Parameter Models
Learning from Data Linear Parameter Models Copyright David Barber 200-2004. Course lecturer: Amos Storkey a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ 2 chirps per sec 26 24 22 20
Chakra Chennubhotla and David Koes
MSCBIO/CMPBIO 2065: Support Vector Machines Chakra Chennubhotla and David Koes Nov 15, 2017 Sources mmds.org chapter 12 Bishop s book Ch. 7 Notes from Toronto, Mark Schmidt (UBC) 2 SVM SVMs and Logistic
ECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
Support Vector Machines
Support Vector Machines SVM Discussion Overview. Importance of SVMs. Overview of Mathematical Techniques Employed 3. Margin Geometry 4. SVM Training Methodology 5. Overlapping Distributions 6. Dealing
Radial Basis Function Networks
Radial Basis Function Networks As we have seen, one of the most common types of neural network is the multi-layer perceptron It does, however, have various disadvantages, including the slow speed in learning
SUPPORT VECTOR MACHINES
SUPPORT VECTOR MACHINES Today Reading AIMA 18.9 Goals (Naïve Bayes classifiers) Support vector machines 1 Support Vector Machines (SVMs) SVMs are probably the most popular off-the-shelf classifier! Software
Support Vector Machines.
Support Vector Machines srihari@buffalo.edu SVM Discussion Overview 1. Overview of SVMs 2. Margin Geometry 3. SVM Optimization 4. Overlapping Distributions 5. Relationship to Logistic Regression 6. Dealing
Optimal Separating Hyperplane and the Support Vector Machine. Volker Tresp Summer 2018
Optimal Separating Hyperplane and the Support Vector Machine Volker Tresp Summer 2018 1 (Vapnik s) Optimal Separating Hyperplane Let s consider a linear classifier with y i { 1, 1} If classes are linearly
Support Vector Machines.
Support Vector Machines srihari@buffalo.edu SVM Discussion Overview. Importance of SVMs. Overview of Mathematical Techniques Employed 3. Margin Geometry 4. SVM Training Methodology 5. Overlapping Distributions
Kernel Methods & Support Vector Machines
& Support Vector Machines & Support Vector Machines Arvind Visvanathan CSCE 970 Pattern Recognition 1 & Support Vector Machines Question? Draw a single line to separate two classes? 2 & Support Vector
, etc. Let s work with the last one. We can graph a few points determined by this equation.
1. Lines By a line, we simply mean a straight curve. We will always think of lines relative to the cartesian plane. Consider the equation 2x 3y 4 = 0. We can rewrite it in many different ways : 2x 3y =
Constrained Optimization with Calculus. Background Three Big Problems Setup and Vocabulary
Constrained Optimization with Calculus Background Three Big Problems Setup and Vocabulary Background Information In unit 3, you learned about linear programming, in which all constraints and the objective
Restricted-Orientation Convexity in Higher-Dimensional Spaces
Restricted-Orientation Convexity in Higher-Dimensional Spaces ABSTRACT Eugene Fink Derick Wood University of Waterloo, Waterloo, Ont, Canada N2L3G1 {efink, dwood}@violetwaterlooedu A restricted-oriented
1 The Axiom of Extensionality
1 The Axiom of Extensionality Primitive notion: Set A set is a group, a collection, or an aggregate of things. In fact, the words set, group, collection, and aggregate are all synonyms denoting the same
CHAPTER 5 SYSTEMS OF EQUATIONS. x y
page 1 of Section 5.1 CHAPTER 5 SYSTEMS OF EQUATIONS SECTION 5.1 GAUSSIAN ELIMINATION matrix form of a system of equations The system 2x + 3y + 4z 1 5x + y + 7z 2 can be written as Ax where b 2 3 4 A [
Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning
Robot Learning 1 General Pipeline 1. Data acquisition (e.g., from 3D sensors) 2. Feature extraction and representation construction 3. Robot learning: e.g., classification (recognition) or clustering (knowledge
Introduction to Support Vector Machines
Introduction to Support Vector Machines CS 536: Machine Learning Littman (Wu, TA) Administration Slides borrowed from Martin Law (from the web). 1 Outline History of support vector machines (SVM) Two classes,
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
LOGISTIC REGRESSION FOR MULTIPLE CLASSES
Peter Orbanz Applied Data Mining Not examinable. 111 LOGISTIC REGRESSION FOR MULTIPLE CLASSES Bernoulli and multinomial distributions The mulitnomial distribution of N draws from K categories with parameter
Support Vector Machines
Support Vector Machines . Importance of SVM SVM is a discriminative method that brings together:. computational learning theory. previously known methods in linear discriminant functions 3. optimization
Vocabulary Unit 2-3: Linear Functions & Healthy Lifestyles. Scale model a three dimensional model that is similar to a three dimensional object.
Scale a scale is the ratio of any length in a scale drawing to the corresponding actual length. The lengths may be in different units. Scale drawing a drawing that is similar to an actual object or place.
We will be sketching 3-dimensional functions. You will be responsible for doing this both by hand and with Mathematica.
Review polar coordinates before 9.7. Section 9.6 Functions and Surfaces We will be sketching 3-dimensional functions. You will be responsible for doing this both b hand and with Mathematica. Remember:
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2017
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2017 Assignment 3: 2 late days to hand in tonight. Admin Assignment 4: Due Friday of next week. Last Time: MAP Estimation MAP
12 Classification using Support Vector Machines
160 Bioinformatics I, WS 14/15, D. Huson, January 28, 2015 12 Classification using Support Vector Machines This lecture is based on the following sources, which are all recommended reading: F. Markowetz.
Introduction to Immersion, Embedding, and the Whitney Embedding Theorems
Introduction to Immersion, Embedding, and the Whitney Embedding Theorems Paul Rapoport November 23, 2015 Abstract We give an overview of immersion in order to present the idea of embedding, then discuss
Math background. 2D Geometric Transformations. Implicit representations. Explicit representations. Read: CS 4620 Lecture 6
Math background 2D Geometric Transformations CS 4620 Lecture 6 Read: Chapter 2: Miscellaneous Math Chapter 5: Linear Algebra Notation for sets, functions, mappings Linear transformations Matrices Matrix-vector
This research aims to present a new way of visualizing multi-dimensional data using generalized scatterplots by sensitivity coefficients to highlight
This research aims to present a new way of visualizing multi-dimensional data using generalized scatterplots by sensitivity coefficients to highlight local variation of one variable with respect to another.
Data mining with Support Vector Machine
Data mining with Support Vector Machine Ms. Arti Patle IES, IPS Academy Indore (M.P.) artipatle@gmail.com Mr. Deepak Singh Chouhan IES, IPS Academy Indore (M.P.) deepak.schouhan@yahoo.com Abstract: Machine
Supervised vs. Unsupervised Learning. Supervised vs. Unsupervised Learning. Supervised vs. Unsupervised Learning. Supervised vs. Unsupervised Learning
Overview T7 - SVM and s Christian Vögeli cvoegeli@inf.ethz.ch Supervised/ s Support Vector Machines Kernels Based on slides by P. Orbanz & J. Keuchel Task: Apply some machine learning method to data from
A Short SVM (Support Vector Machine) Tutorial
A Short SVM (Support Vector Machine) Tutorial j.p.lewis CGIT Lab / IMSC U. Southern California version 0.zz dec 004 This tutorial assumes you are familiar with linear algebra and equality-constrained optimization/lagrange
Support Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
Logistic Regression and Gradient Ascent
Logistic Regression and Gradient Ascent CS 349-02 (Machine Learning) April 0, 207 The perceptron algorithm has a couple of issues: () the predictions have no probabilistic interpretation or confidence
Linear Models. Lecture Outline: Numeric Prediction: Linear Regression. Linear Classification. The Perceptron. Support Vector Machines
Linear Models Lecture Outline: Numeric Prediction: Linear Regression Linear Classification The Perceptron Support Vector Machines Reading: Chapter 4.6 Witten and Frank, 2nd ed. Chapter 4 of Mitchell Solving
Let be a function. We say, is a plane curve given by the. Let a curve be given by function where is differentiable with continuous.
Module 8 : Applications of Integration - II Lecture 22 : Arc Length of a Plane Curve [Section 221] Objectives In this section you will learn the following : How to find the length of a plane curve 221
Propositional Calculus: Boolean Algebra and Simplification. CS 270: Mathematical Foundations of Computer Science Jeremy Johnson
Propositional Calculus: Boolean Algebra and Simplification CS 270: Mathematical Foundations of Computer Science Jeremy Johnson Propositional Calculus Topics Motivation: Simplifying Conditional Expressions
Unit Maps: Grade 8 Math
Real Number Relationships 8.3 Number and operations. The student represents and use real numbers in a variety of forms. Representation of Real Numbers 8.3A extend previous knowledge of sets and subsets
Foundations of Computing
Foundations of Computing Darmstadt University of Technology Dept. Computer Science Winter Term 2005 / 2006 Copyright c 2004 by Matthias Müller-Hannemann and Karsten Weihe All rights reserved http://www.algo.informatik.tu-darmstadt.de/
Monroe County School District Elementary Pacing Guide
Date Taught: Third Grade Unit 1: Number, Operations, and Base 10 Timeline: August5 - September 16 CMA: 3.NBT.2 Fluently add and subtract within 1000 using strategies and algorithms based on place value,
Chap.12 Kernel methods [Book, Chap.7]
![Chap.12 Kernel methods [Book, Chap.7]](/thumbs/83/87786394.jpg "Chap.12 Kernel methods [Book, Chap.7]")
Chap.12 Kernel methods [Book, Chap.7] Neural network methods became popular in the mid to late 1980s, but by the mid to late 1990s, kernel methods have also become popular in machine learning. The first
Inverse and Implicit functions
CHAPTER 3 Inverse and Implicit functions. Inverse Functions and Coordinate Changes Let U R d be a domain. Theorem. (Inverse function theorem). If ϕ : U R d is differentiable at a and Dϕ a is invertible,
Machine Learning for NLP
Machine Learning for NLP Support Vector Machines Aurélie Herbelot 2018 Centre for Mind/Brain Sciences University of Trento 1 Support Vector Machines: introduction 2 Support Vector Machines (SVMs) SVMs
Computer Graphics II
Computer Graphics II Autumn 2018-2019 Outline 1 Transformations Outline Transformations 1 Transformations Orthogonal Projections Suppose we want to project a point x onto a plane given by its unit-length
Mathematics Assessment Anchor Glossary Grades 3 & 4
Mathematics Assessment Anchor Glossary Grades 3 & 4 The definitions for this glossary were taken from one or more of the following sources: Webster s Dictionary, various mathematics dictionaries, the PA
3 RD GRADE MATH-COMMON CORE PACING GUIDE 1ST 9 WEEKS Standard I Can Statements Date
3 RD GRADE MATH-COMMON CORE PACING GUIDE 1ST 9 WEEKS 2013-2014 Standard I Can Statements Date Taught Operations and Algebraic Thinking (OA) 3.OA.1. Interpret products of whole numbers, e.g., interpret
SYSTEMS OF NONLINEAR EQUATIONS
SYSTEMS OF NONLINEAR EQUATIONS Widely used in the mathematical modeling of real world phenomena. We introduce some numerical methods for their solution. For better intuition, we examine systems of two
Lecture Linear Support Vector Machines
Lecture 8 In this lecture we return to the task of classification. As seen earlier, examples include spam filters, letter recognition, or text classification. In this lecture we introduce a popular method
1-Nearest Neighbor Boundary
Linear Models Bankruptcy example R is the ratio of earnings to expenses L is the number of late payments on credit cards over the past year. We would like here to draw a linear separator, and get so a
EXTREME POINTS AND AFFINE EQUIVALENCE
EXTREME POINTS AND AFFINE EQUIVALENCE The purpose of this note is to use the notions of extreme points and affine transformations which are studied in the file affine-convex.pdf to prove that certain standard
Research in Computational Differential Geomet
Research in Computational Differential Geometry November 5, 2014 Approximations Often we have a series of approximations which we think are getting close to looking like some shape. Approximations Often
The end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
Introduction to Machine Learning
Introduction to Machine Learning Isabelle Guyon Notes written by: Johann Leithon. Introduction The process of Machine Learning consist of having a big training data base, which is the input to some learning
Geometric Computations for Simulation
1 Geometric Computations for Simulation David E. Johnson I. INTRODUCTION A static virtual world would be boring and unlikely to draw in a user enough to create a sense of immersion. Simulation allows things
Learning via Optimization
Lecture 7 1 Outline 1. Optimization Convexity 2. Linear regression in depth Locally weighted linear regression 3. Brief dips Logistic Regression [Stochastic] gradient ascent/descent Support Vector Machines
Final Report: Dynamic Dubins-curve based RRT Motion Planning for Differential Constrain Robot
Final Report: Dynamic Dubins-curve based RRT Motion Planning for Differential Constrain Robot Abstract This project develops a sample-based motion-planning algorithm for robot with differential constraints.
3.NBT.1 Use place value understanding to round whole numbers to the nearest 10 or 100.
Date Taught: Monroe County School District Third Grade Unit 1: Number, Operations, and Base 10 Timeline: August 3 - September 28 CMA: September 24-28 3.NBT.2 Fluently add and subtract within 1000 using
VECTOR SPACE CLASSIFICATION
VECTOR SPACE CLASSIFICATION Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, Introduction to Information Retrieval, Cambridge University Press. Chapter 14 Wei Wei wwei@idi.ntnu.no Lecture
THE DNA INEQUALITY POWER ROUND
THE DNA INEQUALITY POWER ROUND Instructions Write/draw all solutions neatly, with at most one question per page, clearly numbered. Turn in the solutions in numerical order, with your team name at the upper
A Poorly Conditioned System. Matrix Form
Possibilities for Linear Systems of Equations A Poorly Conditioned System A Poorly Conditioned System Results No solution (inconsistent) Unique solution (consistent) Infinite number of solutions (consistent)
See the course website for important information about collaboration and late policies, as well as where and when to turn in assignments.
COS Homework # Due Tuesday, February rd See the course website for important information about collaboration and late policies, as well as where and when to turn in assignments. Data files The questions
Updated: January 11, 2016 Calculus III Section Math 232. Calculus III. Brian Veitch Fall 2015 Northern Illinois University
Math 232 Calculus III Brian Veitch Fall 2015 Northern Illinois University 12.5 Equations of Lines and Planes Definition 1: Vector Equation of a Line L Let L be a line in three-dimensional space. P (x,
REVIEW I MATH 254 Calculus IV. Exam I (Friday, April 29) will cover sections
REVIEW I MATH 254 Calculus IV Exam I (Friday, April 29 will cover sections 14.1-8. 1. Functions of multivariables The definition of multivariable functions is similar to that of functions of one variable.
Homework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in class hard-copy please)
Virginia Tech. Computer Science CS 5614 (Big) Data Management Systems Fall 2014, Prakash Homework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in
Unit Maps: Grade 8 Math
Real Number Relationships 8.3 Number and operations. The student represents and use real numbers in a variety of forms. Representation of Real Numbers 8.3A extend previous knowledge of sets and subsets
CS229 Lecture notes. Raphael John Lamarre Townshend
CS229 Lecture notes Raphael John Lamarre Townshend Decision Trees We now turn our attention to decision trees, a simple yet flexible class of algorithms. We will first consider the non-linear, region-based
Feature Extractors. CS 188: Artificial Intelligence Fall Some (Vague) Biology. The Binary Perceptron. Binary Decision Rule.
CS 188: Artificial Intelligence Fall 2008 Lecture 24: Perceptrons II 11/24/2008 Dan Klein UC Berkeley Feature Extractors A feature extractor maps inputs to feature vectors Dear Sir. First, I must solicit
Vector Algebra Transformations. Lecture 4
Vector Algebra Transformations Lecture 4 Cornell CS4620 Fall 2008 Lecture 4 2008 Steve Marschner 1 Geometry A part of mathematics concerned with questions of size, shape, and relative positions of figures
Lecture 4. If P1(x1,y1) and P2(x2,y2) are points on a non-vertical line, then the slope m of the line is defined by
Lines Lecture 4 In this section we shall discuss ways to measure the "steepness" or "slope" of a line in the plane. The ideas we develop here will be important when we discuss equations and graphs of straight
CS 559: Machine Learning Fundamentals and Applications 9 th Set of Notes
1 CS 559: Machine Learning Fundamentals and Applications 9 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Overview
CS570: Introduction to Data Mining
CS570: Introduction to Data Mining Classification Advanced Reading: Chapter 8 & 9 Han, Chapters 4 & 5 Tan Anca Doloc-Mihu, Ph.D. Slides courtesy of Li Xiong, Ph.D., 2011 Han, Kamber & Pei. Data Mining.
Calculus (Math 1A) Lecture 1
Calculus (Math 1A) Lecture 1 Vivek Shende August 23, 2017 Hello and welcome to class! I am Vivek Shende I will be teaching you this semester. My office hours Starting next week: 1-3 pm on tuesdays; 2-3
CS 8520: Artificial Intelligence
CS 8520: Artificial Intelligence Machine Learning 2 Paula Matuszek Spring, 2013 1 Regression Classifiers We said earlier that the task of a supervised learning system can be viewed as learning a function
Weeks 1 through 6 Aug. 6- Sept. 13 Weeks indicate introduction of standards
1 Weeks 1 through 6 Aug. 6- Sept. 13 Weeks indicate introduction of standards Review Addition and Subtraction with Regrouping I can add numbers with regrouping. I can subtract numbers with regrouping.
Lecture 9: Support Vector Machines
Lecture 9: Support Vector Machines William Webber (william@williamwebber.com) COMP90042, 2014, Semester 1, Lecture 8 What we ll learn in this lecture Support Vector Machines (SVMs) a highly robust and
Fractals: Self-Similarity and Fractal Dimension Math 198, Spring 2013
Fractals: Self-Similarity and Fractal Dimension Math 198, Spring 2013 Background Fractal geometry is one of the most important developments in mathematics in the second half of the 20th century. Fractals
Level 3 will generally
Same as Level4 understanding of the standard multiplication algorithm by analyzing partial products or errors use models and strategies to represent and solve realworld division without use of the standard
Supervised Learning (contd) Linear Separation. Mausam (based on slides by UW-AI faculty)
Supervised Learning (contd) Linear Separation Mausam (based on slides by UW-AI faculty) Images as Vectors Binary handwritten characters Treat an image as a highdimensional vector (e.g., by reading pixel
Iterative Methods for Solving Linear Problems
Iterative Methods for Solving Linear Problems When problems become too large (too many data points, too many model parameters), SVD and related approaches become impractical. Iterative Methods for Solving
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Digits. Value The numbers a digit. Standard Form. Expanded Form. The symbols used to show numbers: 0,1,2,3,4,5,6,7,8,9
Digits The symbols used to show numbers: 0,1,2,3,4,5,6,7,8,9 Value The numbers a digit represents, which is determined by the position of the digits Standard Form Expanded Form A common way of the writing