Homework 3. (a) The following code imports the data and standardizes the vector u, called U, include("readclassjson.jl")
|
|
- Arnold Johnston
- 5 years ago
- Views:
Transcription
1 EE104, Spring S. Boyd & S. Lall Homework 3 1. Predicting power demand. Power utilities need to predict the power demanded by consumers. A possible predictor is an auto-regressive (AR) model, which predicts power demands based on a history of power demand data, p t for t = 1,..., T. Here, p t is the power usage in kilowatt hours (kwh) during time interval t. The AR predictor uses standardized power data z t to predict future power demands as an affine function of historical power data. In particular, we set x i = (1, z i+(h 1),..., z i ) y i = z h+i, i = 1,..., T h, and predict future standardized power demands z h+i = y i θ T x i. You will select parameter vector θ R h+1 with ridge regression. The data in power_demand_data.json contains p, the hourly electric power demands for California between July 2015 and December 2017 (the times corresponding to the power demands can be found in dates). Carry out ridge regression for the AR problem on this data, by standardizing the data, computing the data records x i and targets y i, and then solving the ridge regression problem. Although more sophisticated validation techniques are possible, use the first 17,000 data points (approximately two years worth of data) to set the model parameters, and validate the regularization weight λ using the remaining records, with h = 336 (corresponding to two weeks of data). After selecting a reasonable value of λ, compute your final θ by ridge regression on the entire dataset. Provide a plot of validation RMSE versus λ, and a plot of the components of your optimal θ. Comment briefly on the results. Solution. (a) The following code imports the data and standardizes the vector u, called U, include("readclassjson.jl") data = readclassjson("california_hourly_demand.json") U = data["u"] U -= mean(u) U /= norm(u)/sqrt(length(u)) (b) The data matrix can be written row-by-row as x i = (z i+m, z i+m 1, z i+m 2,..., z i ), i = 1,..., T M 1 and y i = z i+m+1. Therefore n = T M 1. 1
2 (c) The plot of the train and test errors is found below Train error Test error RMSE lambda We found that λ.38 yielded the lowest test error. The code for fitting this follows using PyPlot include("readclassjson.jl") data = readclassjson("power_demand_data.json") U = data["p"] U -= mean(u) U /= norm(u)/sqrt(length(u)) M = 336 n = size(u,1) - M X = zeros(n, M+1) y = zeros(n) n_test = 365*24 n_train = n - n_test 2
3 for i=1:n X[i,:] = [1; U[i:i+M-1]] y[i] = U[i+M] println("finished construcing matrix") X_train = X[1:n_train,:] y_train = y[1:n_train] X_test = X[n_train+1:,:] y_test = y[n_train+1:] n_lambda = 20 poss_lambda = logspace(-3, 4, 20) train_err = zeros(n_lambda) test_err = zeros(n_lambda) best_test_rmse = Inf best_test_theta = nothing for idx=1:n_lambda theta = [X_train; zeros(m) sqrt(poss_lambda[idx])*eye(m)] \ [y_train; zeros train_err[idx] = norm(x_train * theta - y_train)/sqrt(n_train) test_err[idx] = norm(x_test * theta - y_test)/sqrt(n_test) println("train $(train_err[idx]) and test $(test_err[idx]) for lambda = $(poss_lambda[idx])") if test_err[idx] < best_test_rmse best_test_rmse = test_err[idx] best_test_theta = theta semilogx(poss_lambda, train_err, label="train error") semilogx(poss_lambda, test_err, label="test error") xlabel("lambda") ylabel("rmse") leg() 3
4 savefig("predict_demand_rmse.pdf") close() figure() plot(1:m+1, reverse(best_test_theta)) xlabel("hours") ylabel("theta") savefig("predict_demand_theta.pdf") show() close() (d) Plotting the parameter θ gives the figure below theta hours We can see the top 10 variables (without the constant) as
5 which make some sense. The two most important predictors are the previous two hours, continued by the energy usage at the same time period in the previous 24 hours ±1 hour. Immediately after, we can see that the next most important features are those corresponding to the usage over the same period 168/24 = 7 days ago (i.e., a week ago). The rest of the features are explained similarly. 2. Least squares gradient descent. Gradient descent is an iterative method that can be used to minimize the average loss of a parameter vector on a dataset. In this problem, we consider the least squares linear regression problem, where empirical risk is the mean-square error, L(θ) = 1 n (θ T x i y i ) 2, n with parameter vector θ R d. i=1 (a) Find L(θ) and implement gradient(x, y, theta), which returns L(θ) from an input data matrix X, target vector y, and current value of the parameter vector theta. (b) Using the function you wrote in part (a), implement gradient descent for the least squares linear regression problem. Generate data with the code below and experiment with various choices of the parameters θ 1, h 1, ɛ, and k max. srand(1234) n, d = 200, 10 X = randn(n, d) y = X * randn(d) +.1*randn(n) On the data above, report θ and a plot of L(θ k ) versus iteration. (c) Using the data in part (b), compute θ using X\y. How does it compare to your iterative solution? Solution. (a) Writing the ERM loss in vectorized form, L(θ) = 1 n n (θ T x i y i ) 2 = 1 n Xθ y 2 2. i=1 We know the derivative of this from EE103, which is just L(θ) = 2 n XT (Xθ y). (b) The following function implements the gradient method in the quadratic loss case 5
6 function quadratic_grad_method(x, y, theta_init) curr_theta = theta_init k_max = 300 eps_min =.01 step_size = 1 for _ = 1:k_max val_f = norm(x * curr_theta - y)^2/n grad_f = step_size*(2*x *(X*curr_theta - y)/n) # Stop if small gradient if norm(grad_f) < 1e-5 new_theta = curr_theta - step_size*grad_f if norm(x * new_theta - y)^2/n <= val_f step_size *= 1.2 curr_theta = new_theta else step_size /= 2 (c) In this case, we find that our quadratic_grad_method implementation terminates in around 14 steps and the difference between the loss computed with the gradient method and the one computed with the usual least squares approach is around The complete code for this problem is srand(1234) n, d = 200, 10 X = randn(n, d) y = X * randn(d) +.1*randn(n) function quadratic_grad_method(x, y, theta_init) curr_theta = theta_init k_max = 300 eps_min =.01 step_size = 1 6
7 for _ = 1:k_max val_f = norm(x * curr_theta - y)^2/n println("current val : $(val_f)") grad_f = step_size*(2*x *(X*curr_theta - y)/n) # Stop if small gradient if norm(grad_f) < 1e-5 new_theta = curr_theta - step_size*grad_f if norm(x * new_theta - y)^2/n <= val_f step_size *= 1.2 curr_theta = new_theta else step_size /= 2 ls_theta = X\y gm_theta = quadratic_grad_method(x, y, zeros(d)) loss_ls = norm(x*ls_theta - y)^2/n loss_gm = norm(x*gm_theta - y)^2/n println("difference of losses is $(loss_gm - loss_ls)") 7
Homework 1. for i=1:n curr_dist = norm(x - X[i,:])
EE104, Spring 2017-2018 S. Boyd & S. Lall Homework 1 1. Nearest neighbor predictor. We have a collection of n observations, x i R d, y i R, i = 1,..., n. Based on these observations, the nearest neighbor
More informationMachine Learning and Computational Statistics, Spring 2016 Homework 1: Ridge Regression and SGD
Machine Learning and Computational Statistics, Spring 2016 Homework 1: Ridge Regression and SGD Due: Friday, February 5, 2015, at 6pm (Submit via NYU Classes) Instructions: Your answers to the questions
More informationProgramming Exercise 5: Regularized Linear Regression and Bias v.s. Variance
Programming Exercise 5: Regularized Linear Regression and Bias v.s. Variance Machine Learning May 13, 212 Introduction In this exercise, you will implement regularized linear regression and use it to study
More informationNONPARAMETRIC REGRESSION SPLINES FOR GENERALIZED LINEAR MODELS IN THE PRESENCE OF MEASUREMENT ERROR
NONPARAMETRIC REGRESSION SPLINES FOR GENERALIZED LINEAR MODELS IN THE PRESENCE OF MEASUREMENT ERROR J. D. Maca July 1, 1997 Abstract The purpose of this manual is to demonstrate the usage of software for
More informationMachine Learning and Computational Statistics, Spring 2015 Homework 1: Ridge Regression and SGD
Machine Learning and Computational Statistics, Spring 2015 Homework 1: Ridge Regression and SGD Due: Friday, February 6, 2015, at 4pm (Submit via NYU Classes) Instructions: Your answers to the questions
More informationCS294-1 Assignment 2 Report
CS294-1 Assignment 2 Report Keling Chen and Huasha Zhao February 24, 2012 1 Introduction The goal of this homework is to predict a users numeric rating for a book from the text of the user s review. The
More informationModel selection and validation 1: Cross-validation
Model selection and validation 1: Cross-validation Ryan Tibshirani Data Mining: 36-462/36-662 March 26 2013 Optional reading: ISL 2.2, 5.1, ESL 7.4, 7.10 1 Reminder: modern regression techniques Over the
More informationBoolean Classification
EE04 Spring 08 S. Lall and S. Boyd Boolean Classification Sanjay Lall and Stephen Boyd EE04 Stanford University Boolean classification Boolean classification I supervised learning is called boolean classification
More informationVariations on Regression Models. Prof. Bennett Math Models of Data Science 2/02/06
Variations on Regression Models Prof. Bennett Math Models of Data Science 2/02/06 Outline Steps in modeling Review of Least Squares model Model in E & K pg 24-29 Aqualsol version of E&K Other loss functions
More informationComputational Intelligence (CS) SS15 Homework 1 Linear Regression and Logistic Regression
Computational Intelligence (CS) SS15 Homework 1 Linear Regression and Logistic Regression Anand Subramoney Points to achieve: 17 pts Extra points: 5* pts Info hour: 28.04.2015 14:00-15:00, HS i12 (ICK1130H)
More informationAravind Baskar A ASSIGNMENT - 3
Aravind Baskar A0136344 ASSIGNMENT - 3 EE5904R/ME5404 Neural Networks Homework #3 Q1. Function Approximation with RBFN (20 Marks) Solution a): RBFN-Exact Interpolation Method: Number of hidden units =
More informationk-nn classification with R QMMA
k-nn classification with R QMMA Emanuele Taufer file:///c:/users/emanuele.taufer/google%20drive/2%20corsi/5%20qmma%20-%20mim/0%20labs/l1-knn-eng.html#(1) 1/16 HW (Height and weight) of adults Statistics
More informationCSE 158 Lecture 2. Web Mining and Recommender Systems. Supervised learning Regression
CSE 158 Lecture 2 Web Mining and Recommender Systems Supervised learning Regression Supervised versus unsupervised learning Learning approaches attempt to model data in order to solve a problem Unsupervised
More informationIn stochastic gradient descent implementations, the fixed learning rate η is often replaced by an adaptive learning rate that decreases over time,
Chapter 2 Although stochastic gradient descent can be considered as an approximation of gradient descent, it typically reaches convergence much faster because of the more frequent weight updates. Since
More informationMidterm Examination , Fall 2011
Midterm Examination 36-350, Fall 2011 Instructions: Provide all answers in your bluebook. Only work in the bluebook will be graded. Clearly indicate which problem each answer goes with. There are 100 points
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 20 2018 Review Solution for multiple linear regression can be computed in closed form
More informationModel Answers to The Next Pixel Prediction Task
Model Answers to The Next Pixel Prediction Task December 2, 25. (Data preprocessing and visualization, 8 marks) (a) Solution. In Algorithm we are told that the data was discretized to 64 grey scale values,...,
More informationDivide and Conquer Kernel Ridge Regression
Divide and Conquer Kernel Ridge Regression Yuchen Zhang John Duchi Martin Wainwright University of California, Berkeley COLT 2013 Yuchen Zhang (UC Berkeley) Divide and Conquer KRR COLT 2013 1 / 15 Problem
More informationCSE 446 Bias-Variance & Naïve Bayes
CSE 446 Bias-Variance & Naïve Bayes Administrative Homework 1 due next week on Friday Good to finish early Homework 2 is out on Monday Check the course calendar Start early (midterm is right before Homework
More informationHomework 3: Solutions
Homework 3: Solutions Statistics 43 Fall 207 Data Analysis: Note: All data analysis results are provided by Yixin Chen STAT43 - HW3 Yixin Chen Data Analysis in R Pipeline (a) Since the purpose is to find
More information6.867 Machine Learning
6.867 Machine Learning Problem set 3 Due Tuesday, October 22, in class What and how to turn in? Turn in short written answers to the questions explicitly stated, and when requested to explain or prove.
More informationHMC CS 158, Fall 2017 Problem Set 3 Programming: Regularized Polynomial Regression
HMC CS 158, Fall 2017 Problem Set 3 Programming: Regularized Polynomial Regression Goals: To open up the black-box of scikit-learn and implement regression models. To investigate how adding polynomial
More informationOnline Learning. Lorenzo Rosasco MIT, L. Rosasco Online Learning
Online Learning Lorenzo Rosasco MIT, 9.520 About this class Goal To introduce theory and algorithms for online learning. Plan Different views on online learning From batch to online least squares Other
More informationFinal Exam. Advanced Methods for Data Analysis (36-402/36-608) Due Thursday May 8, 2014 at 11:59pm
Final Exam Advanced Methods for Data Analysis (36-402/36-608) Due Thursday May 8, 2014 at 11:59pm Instructions: you will submit this take-home final exam in three parts. 1. Writeup. This will be a complete
More informationCSE 190, Spring 2015: Midterm
CSE 190, Spring 2015: Midterm Name: Student ID: Instructions Hand in your solution at or before 7:45pm. Answers should be written directly in the spaces provided. Do not open or start the test before instructed
More informationAssignment 5: Collaborative Filtering
Assignment 5: Collaborative Filtering Arash Vahdat Fall 2015 Readings You are highly recommended to check the following readings before/while doing this assignment: Slope One Algorithm: https://en.wikipedia.org/wiki/slope_one.
More informationConstrained optimization
Constrained optimization A general constrained optimization problem has the form where The Lagrangian function is given by Primal and dual optimization problems Primal: Dual: Weak duality: Strong duality:
More informationMachine Learning. Support Vector Machines. Fabio Vandin November 20, 2017
Machine Learning Support Vector Machines Fabio Vandin November 20, 2017 1 Classification and Margin Consider a classification problem with two classes: instance set X = R d label set Y = { 1, 1}. Training
More informationMATH 829: Introduction to Data Mining and Analysis Model selection
1/12 MATH 829: Introduction to Data Mining and Analysis Model selection Dominique Guillot Departments of Mathematical Sciences University of Delaware February 24, 2016 2/12 Comparison of regression methods
More informationStat 4510/7510 Homework 6
Stat 4510/7510 1/11. Stat 4510/7510 Homework 6 Instructions: Please list your name and student number clearly. In order to receive credit for a problem, your solution must show sufficient details so that
More informationLogistic Regression and Gradient Ascent
Logistic Regression and Gradient Ascent CS 349-02 (Machine Learning) April 0, 207 The perceptron algorithm has a couple of issues: () the predictions have no probabilistic interpretation or confidence
More informationPackage RAMP. May 25, 2017
Type Package Package RAMP May 25, 2017 Title Regularized Generalized Linear Models with Interaction Effects Version 2.0.1 Date 2017-05-24 Author Yang Feng, Ning Hao and Hao Helen Zhang Maintainer Yang
More informationProgramming Exercise 1: Linear Regression
Programming Exercise 1: Linear Regression Machine Learning Introduction In this exercise, you will implement linear regression and get to see it work on data. Before starting on this programming exercise,
More informationLab 10 - Ridge Regression and the Lasso in Python
Lab 10 - Ridge Regression and the Lasso in Python March 9, 2016 This lab on Ridge Regression and the Lasso is a Python adaptation of p. 251-255 of Introduction to Statistical Learning with Applications
More informationPackage nodeharvest. June 12, 2015
Type Package Package nodeharvest June 12, 2015 Title Node Harvest for Regression and Classification Version 0.7-3 Date 2015-06-10 Author Nicolai Meinshausen Maintainer Nicolai Meinshausen
More informationProgramming Exercise 3: Multi-class Classification and Neural Networks
Programming Exercise 3: Multi-class Classification and Neural Networks Machine Learning Introduction In this exercise, you will implement one-vs-all logistic regression and neural networks to recognize
More informationSimple Model Selection Cross Validation Regularization Neural Networks
Neural Nets: Many possible refs e.g., Mitchell Chapter 4 Simple Model Selection Cross Validation Regularization Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February
More informationLearning Meanings for Sentences with Recursive Autoencoders
Learning Meanings for Sentences with Recursive Autoencoders Tsung-Yi Lin and Chen-Yu Lee Department of Electrical and Computer Engineering University of California, San Diego {tsl008, chl260}@ucsd.edu
More informationLab 9 - Linear Model Selection in Python
Lab 9 - Linear Model Selection in Python March 7, 2016 This lab on Model Validation using Validation and Cross-Validation is a Python adaptation of p. 248-251 of Introduction to Statistical Learning with
More informationCS535 Big Data Fall 2017 Colorado State University 10/10/2017 Sangmi Lee Pallickara Week 8- A.
CS535 Big Data - Fall 2017 Week 8-A-1 CS535 BIG DATA FAQs Term project proposal New deadline: Tomorrow PA1 demo PART 1. BATCH COMPUTING MODELS FOR BIG DATA ANALYTICS 5. ADVANCED DATA ANALYTICS WITH APACHE
More informationThe exam is closed book, closed notes except your one-page (two-sided) cheat sheet.
CS 189 Spring 2015 Introduction to Machine Learning Final You have 2 hours 50 minutes for the exam. The exam is closed book, closed notes except your one-page (two-sided) cheat sheet. No calculators or
More informationCSE446: Linear Regression. Spring 2017
CSE446: Linear Regression Spring 2017 Ali Farhadi Slides adapted from Carlos Guestrin and Luke Zettlemoyer Prediction of continuous variables Billionaire says: Wait, that s not what I meant! You say: Chill
More informationOptimization Plugin for RapidMiner. Venkatesh Umaashankar Sangkyun Lee. Technical Report 04/2012. technische universität dortmund
Optimization Plugin for RapidMiner Technical Report Venkatesh Umaashankar Sangkyun Lee 04/2012 technische universität dortmund Part of the work on this technical report has been supported by Deutsche Forschungsgemeinschaft
More informationEE 511 Linear Regression
EE 511 Linear Regression Instructor: Hanna Hajishirzi hannaneh@washington.edu Slides adapted from Ali Farhadi, Mari Ostendorf, Pedro Domingos, Carlos Guestrin, and Luke Zettelmoyer, Announcements Hw1 due
More information3 Types of Gradient Descent Algorithms for Small & Large Data Sets
3 Types of Gradient Descent Algorithms for Small & Large Data Sets Introduction Gradient Descent Algorithm (GD) is an iterative algorithm to find a Global Minimum of an objective function (cost function)
More information1 Training/Validation/Testing
CPSC 340 Final (Fall 2015) Name: Student Number: Please enter your information above, turn off cellphones, space yourselves out throughout the room, and wait until the official start of the exam to begin.
More informationRegularization Methods. Business Analytics Practice Winter Term 2015/16 Stefan Feuerriegel
Regularization Methods Business Analytics Practice Winter Term 2015/16 Stefan Feuerriegel Today s Lecture Objectives 1 Avoiding overfitting and improving model interpretability with the help of regularization
More informationComputation of the Constrained Infinite Time Linear Quadratic Optimal Control Problem
Computation of the Constrained Infinite Time Linear Quadratic Optimal Control Problem July 5, Introduction Abstract Problem Statement and Properties In this paper we will consider discrete-time linear
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 18 2018 Logistics HW 1 is on Piazza and Gradescope Deadline: Friday, Sept. 28, 2018 Office
More informationCollaborative Filtering Applied to Educational Data Mining
Collaborative Filtering Applied to Educational Data Mining KDD Cup 200 July 25 th, 200 BigChaos @ KDD Team Dataset Solution Overview Michael Jahrer, Andreas Töscher from commendo research Dataset Team
More information2. Linear Regression and Gradient Descent
Pattern Recognition And Machine Learning - EPFL - Fall 2015 Emtiyaz Khan, Timur Bagautdinov, Carlos Becker, Ilija Bogunovic & Ksenia Konyushkova 2. Linear Regression and Gradient Descent 2.1 Goals The
More informationPackage RPEnsemble. October 7, 2017
Version 0.4 Date 2017-10-06 Title Random Projection Ensemble Classification Package RPEnsemble October 7, 2017 Author Maintainer Timothy I. Cannings Implements the methodology
More informationHW7 Solutions. Gabe Hope. May The accuracy of the nearest neighbor classifier was: 0.9, and the classifier took 2.55 seconds to run.
HW7 Solutions Gabe Hope May 2017 1 Problem 1 1.1 Part a The accuracy of the nearest neighbor classifier was: 0.9, and the classifier took 2.55 seconds to run. 1.2 Part b The results using random projections
More informationPackage obliquerf. February 20, Index 8. Extract variable importance measure
Package obliquerf February 20, 2015 Title Oblique Random Forests from Recursive Linear Model Splits Version 0.3 Date 2012-08-10 Depends R (>= 2.0.0), stats, ROCR, pls, mda, e1071 Author Bjoern Menze and
More informationProgramming Exercise 4: Neural Networks Learning
Programming Exercise 4: Neural Networks Learning Machine Learning Introduction In this exercise, you will implement the backpropagation algorithm for neural networks and apply it to the task of hand-written
More informationCSC 411: Lecture 02: Linear Regression
CSC 411: Lecture 02: Linear Regression Raquel Urtasun & Rich Zemel University of Toronto Sep 16, 2015 Urtasun & Zemel (UofT) CSC 411: 02-Regression Sep 16, 2015 1 / 16 Today Linear regression problem continuous
More informationLogistic Regression with a Neural Network mindset
Logistic Regression with a Neural Network mindset Welcome to your first (required) programming assignment! You will build a logistic regression classifier to recognize cats. This assignment will step you
More informationLecture Linear Support Vector Machines
Lecture 8 In this lecture we return to the task of classification. As seen earlier, examples include spam filters, letter recognition, or text classification. In this lecture we introduce a popular method
More informationLogistic Regression. Abstract
Logistic Regression Tsung-Yi Lin, Chen-Yu Lee Department of Electrical and Computer Engineering University of California, San Diego {tsl008, chl60}@ucsd.edu January 4, 013 Abstract Logistic regression
More informationYelp Recommendation System
Yelp Recommendation System Jason Ting, Swaroop Indra Ramaswamy Institute for Computational and Mathematical Engineering Abstract We apply principles and techniques of recommendation systems to develop
More informationPS 6: Regularization. PART A: (Source: HTF page 95) The Ridge regression problem is:
Economics 1660: Big Data PS 6: Regularization Prof. Daniel Björkegren PART A: (Source: HTF page 95) The Ridge regression problem is: : β "#$%& = argmin (y # β 2 x #4 β 4 ) 6 6 + λ β 4 #89 Consider the
More informationAlternating Direction Method of Multipliers
Alternating Direction Method of Multipliers CS 584: Big Data Analytics Material adapted from Stephen Boyd (https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdf) & Ryan Tibshirani (http://stat.cmu.edu/~ryantibs/convexopt/lectures/21-dual-meth.pdf)
More informationLecture 26: Missing data
Lecture 26: Missing data Reading: ESL 9.6 STATS 202: Data mining and analysis December 1, 2017 1 / 10 Missing data is everywhere Survey data: nonresponse. 2 / 10 Missing data is everywhere Survey data:
More informationCOMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS18. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions
COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS18 Lecture 2: Linear Regression Gradient Descent Non-linear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Simplest
More informationMachine Problem 8 - Mean Field Inference on Boltzman Machine
CS498: Applied Machine Learning Spring 2018 Machine Problem 8 - Mean Field Inference on Boltzman Machine Professor David A. Forsyth Auto-graded assignment Introduction Mean-Field Approximation is a useful
More informationOverfitting. Machine Learning CSE546 Carlos Guestrin University of Washington. October 2, Bias-Variance Tradeoff
Overfitting Machine Learning CSE546 Carlos Guestrin University of Washington October 2, 2013 1 Bias-Variance Tradeoff Choice of hypothesis class introduces learning bias More complex class less bias More
More informationThe exam is closed book, closed notes except your one-page (two-sided) cheat sheet.
CS 189 Spring 2015 Introduction to Machine Learning Final You have 2 hours 50 minutes for the exam. The exam is closed book, closed notes except your one-page (two-sided) cheat sheet. No calculators or
More informationProblem Set 5. MAS 622J/1.126J: Pattern Recognition and Analysis. Due Monday, 8 November Resubmission due, 12 November 2010
Problem Set 5 MAS 622J/1.126J: Pattern Recognition and Analysis Due Monday, 8 November 2010. Resubmission due, 12 November 2010 Note: All instructions to plot data or write a program should be carried
More informationPenalized regression Statistical Learning, 2011
Penalized regression Statistical Learning, 2011 Niels Richard Hansen September 19, 2011 Penalized regression is implemented in several different R packages. Ridge regression can, in principle, be carried
More informationGeology Geomath Estimating the coefficients of various Mathematical relationships in Geology
Geology 351 - Geomath Estimating the coefficients of various Mathematical relationships in Geology Throughout the semester you ve encountered a variety of mathematical relationships between various geologic
More informationConvex Optimization / Homework 2, due Oct 3
Convex Optimization 0-725/36-725 Homework 2, due Oct 3 Instructions: You must complete Problems 3 and either Problem 4 or Problem 5 (your choice between the two) When you submit the homework, upload a
More informationPackage SAM. March 12, 2019
Type Package Title Sparse Additive Modelling Version 1.1.1 Date 2019-02-18 Package SAM March 12, 2019 Author Haoming Jiang, Yukun Ma, Xinyu Fei, Tuo Zhao, Xingguo Li, Han Liu, and Kathryn Roeder Maintainer
More informationHW3: Multiple Linear Regression
STAT 391 - INTRO STAT DATA SCI UW Spring Quarter 2017 Néhémy Lim HW3: Multiple Linear Regression Programming assignment. Directions. Comment all functions to receive full credit. Provide a single Python
More informationHomework 5. Due: April 20, 2018 at 7:00PM
Homework 5 Due: April 20, 2018 at 7:00PM Written Questions Problem 1 (25 points) Recall that linear regression considers hypotheses that are linear functions of their inputs, h w (x) = w, x. In lecture,
More informationLecture 12: convergence. Derivative (one variable)
Lecture 12: convergence More about multivariable calculus Descent methods Backtracking line search More about convexity (first and second order) Newton step Example 1: linear programming (one var., one
More informationApplied Regression Modeling: A Business Approach
i Applied Regression Modeling: A Business Approach Computer software help: SAS SAS (originally Statistical Analysis Software ) is a commercial statistical software package based on a powerful programming
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 24 2019 Logistics HW 1 is due on Friday 01/25 Project proposal: due Feb 21 1 page description
More informationNo more questions will be added
CSC 2545, Spring 2017 Kernel Methods and Support Vector Machines Assignment 2 Due at the start of class, at 2:10pm, Thurs March 23. No late assignments will be accepted. The material you hand in should
More informationRegularized Committee of Extreme Learning Machine for Regression Problems
Regularized Committee of Extreme Learning Machine for Regression Problems Pablo Escandell-Montero, José M. Martínez-Martínez, Emilio Soria-Olivas, Josep Guimerá-Tomás, Marcelino Martínez-Sober and Antonio
More informationPenalizied Logistic Regression for Classification
Penalizied Logistic Regression for Classification Gennady G. Pekhimenko Department of Computer Science University of Toronto Toronto, ON M5S3L1 pgen@cs.toronto.edu Abstract Investigation for using different
More informationOptimization and least squares. Prof. Noah Snavely CS1114
Optimization and least squares Prof. Noah Snavely CS1114 http://cs1114.cs.cornell.edu Administrivia A5 Part 1 due tomorrow by 5pm (please sign up for a demo slot) Part 2 will be due in two weeks (4/17)
More information14. League: A factor with levels A and N indicating player s league at the end of 1986
PENALIZED REGRESSION Ridge and The LASSO Note: The example contained herein was copied from the lab exercise in Chapter 6 of Introduction to Statistical Learning by. For this exercise, we ll use some baseball
More informationComputerlinguistische Anwendungen Support Vector Machines
with Scikitlearn Computerlinguistische Anwendungen Support Vector Machines Thang Vu CIS, LMU thangvu@cis.uni-muenchen.de May 20, 2015 1 Introduction Shared Task 1 with Scikitlearn Today we will learn about
More informationSUPERVISED LEARNING WITH SCIKIT-LEARN. How good is your model?
SUPERVISED LEARNING WITH SCIKIT-LEARN How good is your model? Classification metrics Measuring model performance with accuracy: Fraction of correctly classified samples Not always a useful metric Class
More information1. Approximation and Prediction Problems
Neural and Evolutionary Computing Lab 2: Neural Networks for Approximation and Prediction 1. Approximation and Prediction Problems Aim: extract from data a model which describes either the depence between
More informationNote Set 4: Finite Mixture Models and the EM Algorithm
Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for
More informationCS 237: Probability in Computing
CS 237: Probability in Computing Wayne Snyder Computer Science Department Boston University Lecture 25: Logistic Regression Motivation: Why Logistic Regression? Sigmoid functions the logit transformation
More informationLab 16 - Multiclass SVMs and Applications to Real Data in Python
Lab 16 - Multiclass SVMs and Applications to Real Data in Python April 7, 2016 This lab on Multiclass Support Vector Machines in Python is an adaptation of p. 366-368 of Introduction to Statistical Learning
More informationPackage SSLASSO. August 28, 2018
Package SSLASSO August 28, 2018 Version 1.2-1 Date 2018-08-28 Title The Spike-and-Slab LASSO Author Veronika Rockova [aut,cre], Gemma Moran [aut] Maintainer Gemma Moran Description
More informationAMS : Combinatorial Optimization Homework Problems - Week V
AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear
More informationPackage svmpath. R topics documented: August 30, Title The SVM Path Algorithm Date Version Author Trevor Hastie
Title The SVM Path Algorithm Date 2016-08-29 Version 0.955 Author Package svmpath August 30, 2016 Computes the entire regularization path for the two-class svm classifier with essentially the same cost
More informationUnderstanding Andrew Ng s Machine Learning Course Notes and codes (Matlab version)
Understanding Andrew Ng s Machine Learning Course Notes and codes (Matlab version) Note: All source materials and diagrams are taken from the Coursera s lectures created by Dr Andrew Ng. Everything I have
More informationMachine Learning Techniques for Detecting Hierarchical Interactions in GLM s for Insurance Premiums
Machine Learning Techniques for Detecting Hierarchical Interactions in GLM s for Insurance Premiums José Garrido Department of Mathematics and Statistics Concordia University, Montreal EAJ 2016 Lyon, September
More informationPackage bayescl. April 14, 2017
Package bayescl April 14, 2017 Version 0.0.1 Date 2017-04-10 Title Bayesian Inference on a GPU using OpenCL Author Rok Cesnovar, Erik Strumbelj Maintainer Rok Cesnovar Description
More informationECE251DN: Homework #3 Solutions
ECE251DN: Homework #3 Solutions Problem 3.7.2 (a) In this problem, we only have one null constraint. So N 1 j C = V(ψ ) = [e 2 ψ,..., 1,..., e j N 1 2 ψ ] T The weights of the least squares approximation
More informationGeneralized least squares (GLS) estimates of the level-2 coefficients,
Contents 1 Conceptual and Statistical Background for Two-Level Models...7 1.1 The general two-level model... 7 1.1.1 Level-1 model... 8 1.1.2 Level-2 model... 8 1.2 Parameter estimation... 9 1.3 Empirical
More informationPredicting Bus Arrivals Using One Bus Away Real-Time Data
Predicting Bus Arrivals Using One Bus Away Real-Time Data 1 2 3 4 5 Catherine M. Baker Alexander C. Nied Department of Computer Science Department of Computer Science University of Washington University
More informationLecture : Neural net: initialization, activations, normalizations and other practical details Anne Solberg March 10, 2017
INF 5860 Machine learning for image classification Lecture : Neural net: initialization, activations, normalizations and other practical details Anne Solberg March 0, 207 Mandatory exercise Available tonight,
More informationWeek 3: Perceptron and Multi-layer Perceptron
Week 3: Perceptron and Multi-layer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, Fitzhugh-Nagumo model and Izhikevich model. This week,
More information06: Logistic Regression
06_Logistic_Regression 06: Logistic Regression Previous Next Index Classification Where y is a discrete value Develop the logistic regression algorithm to determine what class a new input should fall into
More information