Non-linear models. Basis expansion. Overfitting. Regularization.
|
|
- Allison Collins
- 5 years ago
- Views:
Transcription
1 Non-linear models. Basis epansion. Overfitting. Regularization. Petr Pošík Czech Technical Universit in Prague Facult of Electrical Engineering Dept. of Cbernetics Non-linear models Basis epansion Two spaces Remarks How to evaluate a predictive model? 7 Model evaluation Training and testing error Overfitting Bias vs Variance Crossvalidation How to determine a suitable model fleibilit How to prevent overfitting? Regularization Ridge Lasso Summar 9 Competencies
2 When a linear model is not enough... / Basis epansion a.k.a. feature space straightening. Wh? Linear decision boundar (or linear regression model) ma not be fleible enough to perform accurate classification (regression). The algorithms for fitting linear models can be used to fit (certain tpe of) non-linear models! How? Let s define a new multidimensional image space F. Feature vectors are transformed into this image space F (new features are derived) using mapping Φ: z = Φ(), = (,,..., D ) z = (Φ (), Φ (),..., Φ G ()), while usuall D G. In the image space, a linear model is trained. However, this is equivalent to training a non-linear model in the original space. f G (z) = w z + w z +...+w G z G + w f() = f G (Φ()) = w Φ ()+w Φ ()+...+w G Φ G ()+w P. Pošík c 7 Artificial Intelligence 3 / Two coordinate sstems Transformation into a high-dimensional image space Feature space Image space = (,,..., D ) z = (z, z,..., z G ) z = log z = 3 z 3 = e... Training a linear model in the image space f() = w log + w 3 + w 3 e +...+w f G (z) = w z + w z + w 3 z w P. Pošík c 7 Artificial Intelligence / Non-linear model in the feature space
3 Two coordinate sstems: simple graphical Transformation eample into a high-dimensional image space D Feature space D Image space Training a linear model in the image space P. Pošík c 7 Artificial Intelligence 5 / Non-linear model in the feature space Basis epansion: remarks Advantages: Universal, generall usable method. Disadvantages: We must define what new features shall form the high-dimensional space F. The eamples must be reall transformed into the high-dimensional space F. When too much derived features is used, the resulting models are prone to overfitting (see net slides). For certain tpe of algorithms, there is a method how to perform the basis epansion without actuall carring out the mapping! (See the net lecture.) P. Pošík c 7 Artificial Intelligence / 3
4 How to evaluate a predictive model? 7 / Model evaluation Fundamental question: What is a good measure of model qualit from the machine-learning standpoint? We have various measures of model error: For regression tasks: MSE, MAE,... For classification tasks: misclassification rate, measures based on confusion matri,... Some of them can be regarded as finite approimations of the Baes risk. Are these functions good approimations when measured on the data the models were trained on? 3.5 f() = f() = f() = f() =. + (.3) + (.7 ) + (.5 3 ) Using MSE onl, both models are equivalent!!! Using MSE onl, the cubic model is better than linear!!! A basic method of evaluation is model validation on a different, independent data set from the same source, i.e. on testing data. P. Pošík c 7 Artificial Intelligence 8 / Validation on testing data Eample: Polnomial regression with varring degree: X U(, 3) Y X + N(, ) Poln om d e g.:, tr. e rr.: 8.3 9, te s t. e rr.:.9 Poln om d e g.:, tr. e rr.:. 3, te s t. e rr.:.8 Poln om d e g.:, tr. e rr.:. 7, te s t. e rr.: Poln om d e g.: 3, tr. e rr.:. 5, te s t. e rr.: Poln om d e g.: 5, tr. e rr.:., te s t. e rr.: Poln om d e g.: 9, tr. e rr.:.5 5, te s t. e rr.: P. Pošík c 7 Artificial Intelligence 9 /
5 Training and testing error Tra in in g e rror Te s tin g e rror MSE Polnom de gre e The training error decreases with increasing model fleibilit. The testing error is minimal for certain degree of model fleibilit. P. Pošík c 7 Artificial Intelligence / Overfitting Definition of overfitting: Let H be a hpotheses space. Let h H and h H be different hpotheses from this space. Let Err Tr (h) be an error of the hpothesis h measured on the training dataset (training error). Let Err Tst (h) be an error of the hpothesis h measured on the testing dataset (testing error). We sa that h is overfitted if there is another h for which Err Tr (h ) < Err Tr (h ) Err Tst (h ) > Err Tst (h ) Model Error Testing data Model Fleibilit When overfitted, the model works well for the training data, but fails for new (testing) data. Overfitting is a general phenomenon affecting all kinds of inductive learning of models with tunable fleibilit. We want models and learning algorithms with a good generalization abilit, i.e. we want models that encode onl the relationships valid in the whole domain, not those that learned the specifics of the training data, i.e. we want algorithms able to find onl the relationships valid in the whole domain and ignore specifics of the training data. P. Pošík c 7 Artificial Intelligence / 5
6 Bias vs Variance Poln om d e g.:, tr. e rr.:. 3, te s t. e rr.:.8 Poln om d e g.:, tr. e rr.:. 7, te s t. e rr.:.9 5 Poln om d e g.: 9, tr. e rr.:.5 5, te s t. e rr.: High bias: model not fleible enough (Underfit) Just right (Good fit) High variance: model fleibilit too high (Overfit) Testing data High bias problem: Err Tr (h) is high Err Tst (h) Err Tr (h) Model Error High variance problem: Err Tr (h) is low Err Tst (h) >> Err Tr (h) Model Fleibilit P. Pošík c 7 Artificial Intelligence / Crossvalidation How to estimate the true error of a model on new, unseen data? Simple crossvalidation: Split the data into training and testing subsets. Train the model on training data. Evaluate the model error on testing data. K-fold crossvalidation: Split the data into k folds (k is usuall 5 or ). In each iteration: Use k folds to train the model. Use fold to test the model, i.e. measure error. Iter. Training Training Testing Iter. Training Testing Training Iter. k Testing Training Training Aggregate (average) the k error measurements to get the final error estimate. Train the model on the whole data set. Leave-one-out (LOO) crossvalidation: k = T, i.e. the number of folds is equal to the training set size. Time consuming for large T. P. Pošík c 7 Artificial Intelligence 3 /
7 How to determine a suitable model fleibilit Simpl test models of varing compleities and choose the one with the best testing error, right? The testing data are used here to tune a meta-parameter of the model. The testing data are used to train (a part of) the model, thus essentiall become part of training data. The error on testing data is no longer an unbiased estimate of model error; it underestimates it. A new, separate data set is needed to estimate the model error. Using simple crossvalidation:. : use cca 5 % of data for model building.. Validation data: use cca 5 % of data to search for the suitable model fleibilit. 3. Train the suitable model on training + validation data.. Testing data: use cca 5 % of data for the final estimate of the model error. Using k-fold crossvalidation. : use cca 75 % of data to find and train a suitable model using crossvalidation.. Testing data: use cca 5 % of data for the final estimate of the model error. The ratios are not set in stone, there are other possibilities, e.g. ::, etc. P. Pošík c 7 Artificial Intelligence / How to prevent overfitting?. Feature selection: Reduce the number of features. Select manuall, which features to keep. Tr to identif a suitable subset of features during learning phase (man feature selection methods eist; none is perfect).. Regularization: Keep all the features, but reduce the magnitude of their weights w. Works well, if we have a lot of features each of which contributes a bit to predicting. P. Pošík c 7 Artificial Intelligence 5 / 7
8 Regularization / Ridge regularization (a.k.a. Tikhonov regularization) Ridge regularization penalizes the size of the model coefficients: Training and testing errors as functions of regularization parameter: Modification of the optimization criterion: J(w) = T T i= ( ) +α (i) h w ( (i) D ) w d. d= Training error Testing error The solution is given b a modified Normal equation w = (X T X+αI) X T MSE..5 As α, w ridge w OLS. As α, w ridge. OLS - ordinar least squares. Just a simple multiple linear regression Regularization factor The values of coefficients (weights w) as functions of regularization parameter: 5 Coefficient size Regularization factor P. Pošík c 7 Artificial Intelligence 7 / Lasso regularization Lasso regularization penalizes the size of the model coefficients: Training and testing errors as functions of regularization parameter: Modification of the optimization criterion: J(w) = T T i= ( ) +α (i) h w ( (i) D ) w d. d= As α, Lasso regularization decreases the number of non-zero coefficients, effectivel also performing feature selection and creating sparse models. MSE Training error Testing error Regularization factor The values of coefficients as functions of regularization parameter:.7..5 Coefficient size Regularization factor P. Pošík c 7 Artificial Intelligence 8 / 8
9 Summar 9 / Competencies After this lecture, a student shall be able to... eplain the reason for doing basis epansion (feature space straightening), and describe its principle; show the effect of basis epansion with a linear model on a simple eample for both classification and regression settings; implement user-defined basis epansions in certain programming language; list advantages and disadvantages of basis epansion; eplain wh the error measured on the training data is not a good estimate of the epected error of the model for new data, and whether it under- or overestimates the true error; eplain basic methods to get unbiased estimate of the true model error (testing data, k-fold crossvalidation, LOO crossvalidation); describe the general form of dependenc of the model training and testing errors on the model compleit/fleibilit/capacit; define overfitting; discuss high bias and high variance problems of models; eplain how to proceed if a suitable model compleit must be chosen as part of the training process; list basic methods of overfitting prevention; describe the principles of ridge (Tikhonov) and lasso regularizations and their effects on the model parameters. P. Pošík c 7 Artificial Intelligence / 9
Bias-Variance Decomposition Error Estimators
Bias-Variance Decomposition Error Estimators Cross-Validation Bias-Variance tradeoff Intuition Model too simple does not fit the data well a biased solution. Model too comple small changes to the data,
More informationBias-Variance Decomposition Error Estimators Cross-Validation
Bias-Variance Decomposition Error Estimators Cross-Validation Bias-Variance tradeoff Intuition Model too simple does not fit the data well a biased solution. Model too comple small changes to the data,
More informationPerformance Evaluation
Performance Evaluation Dan Lizotte 7-9-5 Evaluating Performance..5..5..5..5 Which do ou prefer and wh? Evaluating Performance..5..5 Which do ou prefer and wh?..5..5 Evaluating Performance..5..5..5..5 Performance
More informationCross-validation for detecting and preventing overfitting
Cross-validation for detecting and preventing overfitting Andrew W. Moore/Anna Goldenberg School of Computer Science Carnegie Mellon Universit Copright 2001, Andrew W. Moore Apr 1st, 2004 Want to learn
More informationCross-validation for detecting and preventing overfitting
Cross-validation for detecting and preventing overfitting Note to other teachers and users of these slides. Andrew would be delighted if ou found this source material useful in giving our own lectures.
More informationLeveling Up as a Data Scientist. ds/2014/10/level-up-ds.jpg
Model Optimization Leveling Up as a Data Scientist http://shorelinechurch.org/wp-content/uploa ds/2014/10/level-up-ds.jpg Bias and Variance Error = (expected loss of accuracy) 2 + flexibility of model
More informationCross-validation for detecting and preventing overfitting
Cross-validation for detecting and preventing overfitting Note to other teachers and users of these slides. Andrew would be delighted if ou found this source material useful in giving our own lectures.
More informationPerformance Estimation and Regularization. Kasthuri Kannan, PhD. Machine Learning, Spring 2018
Performance Estimation and Regularization Kasthuri Kannan, PhD. Machine Learning, Spring 2018 Bias- Variance Tradeoff Fundamental to machine learning approaches Bias- Variance Tradeoff Error due to Bias:
More informationCross-validation. Cross-validation is a resampling method.
Cross-validation Cross-validation is a resampling method. It refits a model of interest to samples formed from the training set, in order to obtain additional information about the fitted model. For example,
More informationLinear Model Selection and Regularization. especially usefull in high dimensions p>>100.
Linear Model Selection and Regularization especially usefull in high dimensions p>>100. 1 Why Linear Model Regularization? Linear models are simple, BUT consider p>>n, we have more features than data records
More informationNonparametric Methods Recap
Nonparametric Methods Recap Aarti Singh Machine Learning 10-701/15-781 Oct 4, 2010 Nonparametric Methods Kernel Density estimate (also Histogram) Weighted frequency Classification - K-NN Classifier Majority
More informationDecision Trees. Petr Pošík. Czech Technical University in Prague Faculty of Electrical Engineering Dept. of Cybernetics
Decision Trees Petr Pošík Czech Technical University in Prague Faculty of Electrical Engineering Dept. of Cybernetics This lecture is largely based on the book Artificial Intelligence: A Modern Approach,
More informationCSE446: Linear Regression. Spring 2017
CSE446: Linear Regression Spring 2017 Ali Farhadi Slides adapted from Carlos Guestrin and Luke Zettlemoyer Prediction of continuous variables Billionaire says: Wait, that s not what I meant! You say: Chill
More informationIntroduction to Automated Text Analysis. bit.ly/poir599
Introduction to Automated Text Analysis Pablo Barberá School of International Relations University of Southern California pablobarbera.com Lecture materials: bit.ly/poir599 Today 1. Solutions for last
More informationOverfitting, Model Selection, Cross Validation, Bias-Variance
Statistical Machine Learning Notes 2 Overfitting, Model Selection, Cross Validation, Bias-Variance Instructor: Justin Domke Motivation Suppose we have some data TRAIN = {(, ), ( 2, 2 ),..., ( N, N )} that
More informationModel Complexity and Generalization
HT2015: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Generalization Learning Curves Underfit Generalization
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 20 2018 Review Solution for multiple linear regression can be computed in closed form
More informationA Rational Shift in Behavior. Translating Rational Functions. LEARnIng goals
. A Rational Shift in Behavior LEARnIng goals In this lesson, ou will: Analze rational functions with a constant added to the denominator. Compare rational functions in different forms. Identif vertical
More informationDiscussion: Clustering Random Curves Under Spatial Dependence
Discussion: Clustering Random Curves Under Spatial Dependence Gareth M. James, Wenguang Sun and Xinghao Qiao Abstract We discuss the advantages and disadvantages of a functional approach to clustering
More informationLecture 7. CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler. Outline. Machine Learning: Cross Validation. Performance evaluation methods
CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 7 Machine Learning: Cross Validation Outline Performance evaluation methods test/train sets cross-validation k-fold Leave-one-out 1 A
More information2017 ITRON EFG Meeting. Abdul Razack. Specialist, Load Forecasting NV Energy
2017 ITRON EFG Meeting Abdul Razack Specialist, Load Forecasting NV Energy Topics 1. Concepts 2. Model (Variable) Selection Methods 3. Cross- Validation 4. Cross-Validation: Time Series 5. Example 1 6.
More informationCSE Data Mining Concepts and Techniques STATISTICAL METHODS (REGRESSION) Professor- Anita Wasilewska. Team 13
CSE 634 - Data Mining Concepts and Techniques STATISTICAL METHODS Professor- Anita Wasilewska (REGRESSION) Team 13 Contents Linear Regression Logistic Regression Bias and Variance in Regression Model Fit
More informationChapter 3. Interpolation. 3.1 Introduction
Chapter 3 Interpolation 3 Introduction One of the fundamental problems in Numerical Methods is the problem of interpolation, that is given a set of data points ( k, k ) for k =,, n, how do we find a function
More informationEE 511 Linear Regression
EE 511 Linear Regression Instructor: Hanna Hajishirzi hannaneh@washington.edu Slides adapted from Ali Farhadi, Mari Ostendorf, Pedro Domingos, Carlos Guestrin, and Luke Zettelmoyer, Announcements Hw1 due
More informationCOMPUTATIONAL INTELLIGENCE SEW (INTRODUCTION TO MACHINE LEARNING) SS18. Lecture 6: k-nn Cross-validation Regularization
COMPUTATIONAL INTELLIGENCE SEW (INTRODUCTION TO MACHINE LEARNING) SS18 Lecture 6: k-nn Cross-validation Regularization LEARNING METHODS Lazy vs eager learning Eager learning generalizes training data before
More informationLecture on Modeling Tools for Clustering & Regression
Lecture on Modeling Tools for Clustering & Regression CS 590.21 Analysis and Modeling of Brain Networks Department of Computer Science University of Crete Data Clustering Overview Organizing data into
More informationMulticollinearity and Validation CIVL 7012/8012
Multicollinearity and Validation CIVL 7012/8012 2 In Today s Class Recap Multicollinearity Model Validation MULTICOLLINEARITY 1. Perfect Multicollinearity 2. Consequences of Perfect Multicollinearity 3.
More informationOverfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Earl Stopping Rich Caruana CALD, CMU 5 Forbes Ave. Pittsburgh, PA 53 caruana@cs.cmu.edu Steve Lawrence NEC Research Institute 4 Independence
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 24 2019 Logistics HW 1 is due on Friday 01/25 Project proposal: due Feb 21 1 page description
More informationLECTURE 6: CROSS VALIDATION
LECTURE 6: CROSS VALIDATION CSCI 4352 Machine Learning Dongchul Kim, Ph.D. Department of Computer Science A Regression Problem Given a data set, how can we evaluate our (linear) model? Cross Validation
More informationModel Assessment and Selection. Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer
Model Assessment and Selection Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 Model Training data Testing data Model Testing error rate Training error
More informationLasso Regression: Regularization for feature selection
Lasso Regression: Regularization for feature selection CSE 416: Machine Learning Emily Fox University of Washington April 12, 2018 Symptom of overfitting 2 Often, overfitting associated with very large
More informationLecture 27: Review. Reading: All chapters in ISLR. STATS 202: Data mining and analysis. December 6, 2017
Lecture 27: Review Reading: All chapters in ISLR. STATS 202: Data mining and analysis December 6, 2017 1 / 16 Final exam: Announcements Tuesday, December 12, 8:30-11:30 am, in the following rooms: Last
More informationHyperparameters and Validation Sets. Sargur N. Srihari
Hyperparameters and Validation Sets Sargur N. srihari@cedar.buffalo.edu 1 Topics in Machine Learning Basics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation
More informationLecture 13: Model selection and regularization
Lecture 13: Model selection and regularization Reading: Sections 6.1-6.2.1 STATS 202: Data mining and analysis October 23, 2017 1 / 17 What do we know so far In linear regression, adding predictors always
More informationChapter 5: Polynomial Functions
Chapter : Polnomial Functions Section.1 Chapter : Polnomial Functions Section.1: Eploring the Graphs of Polnomial Functions Terminolog: Polnomial Function: A function that contains onl the operations of
More informationTopics in Machine Learning-EE 5359 Model Assessment and Selection
Topics in Machine Learning-EE 5359 Model Assessment and Selection Ioannis D. Schizas Electrical Engineering Department University of Texas at Arlington 1 Training and Generalization Training stage: Utilizing
More informationFitting a transformation: Feature-based alignment April 30 th, Yong Jae Lee UC Davis
Fitting a transformation: Feature-based alignment April 3 th, 25 Yong Jae Lee UC Davis Announcements PS2 out toda; due 5/5 Frida at :59 pm Color quantization with k-means Circle detection with the Hough
More informationCSE446: Linear Regression. Spring 2017
CSE446: Linear Regression Spring 2017 Ali Farhadi Slides adapted from Carlos Guestrin and Luke Zettlemoyer Prediction of continuous variables Billionaire says: Wait, that s not what I meant! You say: Chill
More informationGradient LASSO algoithm
Gradient LASSO algoithm Yongdai Kim Seoul National University, Korea jointly with Yuwon Kim University of Minnesota, USA and Jinseog Kim Statistical Research Center for Complex Systems, Korea Contents
More informationTIPS4RM: MHF4U: Unit 1 Polynomial Functions
TIPSRM: MHFU: Unit Polnomial Functions 008 .5.: Polnomial Concept Attainment Activit Compare and contrast the eamples and non-eamples of polnomial functions below. Through reasoning, identif attributes
More informationOverfitting. Machine Learning CSE546 Carlos Guestrin University of Washington. October 2, Bias-Variance Tradeoff
Overfitting Machine Learning CSE546 Carlos Guestrin University of Washington October 2, 2013 1 Bias-Variance Tradeoff Choice of hypothesis class introduces learning bias More complex class less bias More
More informationPartial Fraction Decomposition
Section 7. Partial Fractions 53 Partial Fraction Decomposition Algebraic techniques for determining the constants in the numerators of partial fractions are demonstrated in the eamples that follow. Note
More informationINTRODUCTION TO MACHINE LEARNING. Measuring model performance or error
INTRODUCTION TO MACHINE LEARNING Measuring model performance or error Is our model any good? Context of task Accuracy Computation time Interpretability 3 types of tasks Classification Regression Clustering
More informationUnit I - Chapter 3 Polynomial Functions 3.1 Characteristics of Polynomial Functions
Math 3200 Unit I Ch 3 - Polnomial Functions 1 Unit I - Chapter 3 Polnomial Functions 3.1 Characteristics of Polnomial Functions Goal: To Understand some Basic Features of Polnomial functions: Continuous
More informationLast time... Bias-Variance decomposition. This week
Machine learning, pattern recognition and statistical data modelling Lecture 4. Going nonlinear: basis expansions and splines Last time... Coryn Bailer-Jones linear regression methods for high dimensional
More informationModeling and Simulation Exam
Modeling and Simulation am Facult of Computers & Information Department: Computer Science Grade: Fourth Course code: CSC Total Mark: 75 Date: Time: hours Answer the following questions: - a Define the
More informationLECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS
LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Neural Networks Classifier Introduction INPUT: classification data, i.e. it contains an classification (class) attribute. WE also say that the class
More informationNetwork Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Sources Hastie, Tibshirani, Friedman: The Elements of Statistical Learning James, Witten, Hastie, Tibshirani: An Introduction to Statistical Learning Andrew Ng:
More informationCSE 446 Bias-Variance & Naïve Bayes
CSE 446 Bias-Variance & Naïve Bayes Administrative Homework 1 due next week on Friday Good to finish early Homework 2 is out on Monday Check the course calendar Start early (midterm is right before Homework
More informationLesson 2.1 Exercises, pages 90 96
Lesson.1 Eercises, pages 9 96 A. a) Complete the table of values. 1 1 1 1 1. 1 b) For each function in part a, sketch its graph then state its domain and range. For : the domain is ; and the range is.
More information5.2 Graphing Polynomial Functions
Locker LESSON 5. Graphing Polnomial Functions Common Core Math Standards The student is epected to: F.IF.7c Graph polnomial functions, identifing zeros when suitable factorizations are available, and showing
More informationStat 342 Exam 3 Fall 2014
Stat 34 Exam 3 Fall 04 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed There are questions on the following 6 pages. Do as many of them as you can
More informationMachine Learning: An Applied Econometric Approach Online Appendix
Machine Learning: An Applied Econometric Approach Online Appendix Sendhil Mullainathan mullain@fas.harvard.edu Jann Spiess jspiess@fas.harvard.edu April 2017 A How We Predict In this section, we detail
More informationUsing Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
More informationy = f(x) x (x, f(x)) f(x) g(x) = f(x) + 2 (x, g(x)) 0 (0, 1) 1 3 (0, 3) 2 (2, 3) 3 5 (2, 5) 4 (4, 3) 3 5 (4, 5) 5 (5, 5) 5 7 (5, 7)
0 Relations and Functions.7 Transformations In this section, we stud how the graphs of functions change, or transform, when certain specialized modifications are made to their formulas. The transformations
More informationCS273 Midterm Exam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014
CS273 Midterm Eam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014 Your name: Your UCINetID (e.g., myname@uci.edu): Your seat (row and number): Total time is 80 minutes. READ THE
More informationInvestigation Free Fall
Investigation Free Fall Name Period Date You will need: a motion sensor, a small pillow or other soft object What function models the height of an object falling due to the force of gravit? Use a motion
More informationGLOBAL EDITION. Interactive Computer Graphics. A Top-Down Approach with WebGL SEVENTH EDITION. Edward Angel Dave Shreiner
GLOBAL EDITION Interactive Computer Graphics A Top-Down Approach with WebGL SEVENTH EDITION Edward Angel Dave Shreiner This page is intentionall left blank. 4.10 Concatenation of Transformations 219 in
More informationSystems of Linear Equations
Sstems of Linear Equations Gaussian Elimination Tpes of Solutions A linear equation is an equation that can be written in the form: a a a n n b The coefficients a i and the constant b can be real or comple
More informationDiversity visualization in evolutionary algorithms
Diversit visualization in evolutionar algorithms Jan Drchal drchaj@fel.cvut.cz Miroslav Šnorek snorek@fel.cvut.cz Abstract: Evolutionar Algorithms (EAs) are well-known nature-inspired optimization methods.
More informationDimensionality Reduction, including by Feature Selection.
Dimensionality Reduction, including by Feature Selection www.cs.wisc.edu/~dpage/cs760 Goals for the lecture you should understand the following concepts filtering-based feature selection information gain
More informationBig Data Methods. Chapter 5: Machine learning. Big Data Methods, Chapter 5, Slide 1
Big Data Methods Chapter 5: Machine learning Big Data Methods, Chapter 5, Slide 1 5.1 Introduction to machine learning What is machine learning? Concerned with the study and development of algorithms that
More informationBoosted Optimization for Network Classification. Bioinformatics Center Kyoto University
Boosted Optimization for Network Classification Timothy Hancock Hiroshi Mamitsuka Bioinformatics Center Kyoto University 2 of 22 Motivation We want to construct a classifier that has good performance where
More informationDetermining the 2d transformation that brings one image into alignment (registers it) with another. And
Last two lectures: Representing an image as a weighted combination of other images. Toda: A different kind of coordinate sstem change. Solving the biggest problem in using eigenfaces? Toda Recognition
More information5.2 Graphing Polynomial Functions
Name Class Date 5.2 Graphing Polnomial Functions Essential Question: How do ou sketch the graph of a polnomial function in intercept form? Eplore 1 Investigating the End Behavior of the Graphs of Simple
More informationRandom Forest A. Fornaser
Random Forest A. Fornaser alberto.fornaser@unitn.it Sources Lecture 15: decision trees, information theory and random forests, Dr. Richard E. Turner Trees and Random Forests, Adele Cutler, Utah State University
More informationArtificial Neural Networks MLP, RBF & GMDH
Artificial Neural Networks MLP, RBF & GMDH Jan Drchal drchajan@fel.cvut.cz Computational Intelligence Group Department of Computer Science and Engineering Faculty of Electrical Engineering Czech Technical
More informationSection 4.3 Features of a Line
Section.3 Features of a Line Objectives In this section, ou will learn to: To successfull complete this section, ou need to understand: Identif the - and -intercepts of a line. Plotting points in the --plane
More information10601 Machine Learning. Model and feature selection
10601 Machine Learning Model and feature selection Model selection issues We have seen some of this before Selecting features (or basis functions) Logistic regression SVMs Selecting parameter value Prior
More informationChapter 7: Numerical Prediction
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases SS 2016 Chapter 7: Numerical Prediction Lecture: Prof. Dr.
More informationVariable Selection 6.783, Biomedical Decision Support
6.783, Biomedical Decision Support (lrosasco@mit.edu) Department of Brain and Cognitive Science- MIT November 2, 2009 About this class Why selecting variables Approaches to variable selection Sparsity-based
More informationTight Clustering: a method for extracting stable and tight patterns in expression profiles
Statistical issues in microarra analsis Tight Clustering: a method for etracting stable and tight patterns in epression profiles Eperimental design Image analsis Normalization George C. Tseng Dept. of
More informationPredictor Selection Algorithm for Bayesian Lasso
Predictor Selection Algorithm for Baesian Lasso Quan Zhang Ma 16, 2014 1 Introduction The Lasso [1] is a method in regression model for coefficients shrinkage and model selection. It is often used in the
More informationMachine Learning using Matlab. Lecture 3 Logistic regression and regularization
Machine Learning using Matlab Lecture 3 Logistic regression and regularization Presentation Date (correction) 10.07.2017 11.07.2017 17.07.2017 18.07.2017 24.07.2017 25.07.2017 Project proposals 13 submissions,
More informationModeling with CMU Mini-FEA Program
Modeling with CMU Mini-FEA Program Introduction Finite element analsis (FEA) allows ou analze the stresses and displacements in a bod when forces are applied. FEA determines the stresses and displacements
More information2.3 Polynomial Functions of Higher Degree with Modeling
SECTION 2.3 Polnomial Functions of Higher Degree with Modeling 185 2.3 Polnomial Functions of Higher Degree with Modeling What ou ll learn about Graphs of Polnomial Functions End Behavior of Polnomial
More informationImplicit differentiation
Roberto s Notes on Differential Calculus Chapter 4: Basic differentiation rules Section 5 Implicit differentiation What ou need to know alread: Basic rules of differentiation, including the chain rule.
More informationTransformations of Functions. 1. Shifting, reflecting, and stretching graphs Symmetry of functions and equations
Chapter Transformations of Functions TOPICS.5.. Shifting, reflecting, and stretching graphs Smmetr of functions and equations TOPIC Horizontal Shifting/ Translation Horizontal Shifting/ Translation Shifting,
More information3-2. Families of Graphs. Look Back. OBJECTIVES Identify transformations of simple graphs. Sketch graphs of related functions.
3-2 BJECTIVES Identif transformations of simple graphs. Sketch graphs of related functions. Families of Graphs ENTERTAINMENT At some circuses, a human cannonball is shot out of a special cannon. In order
More informationEssential Question How many turning points can the graph of a polynomial function have?
.8 Analzing Graphs of Polnomial Functions Essential Question How man turning points can the graph of a polnomial function have? A turning point of the graph of a polnomial function is a point on the graph
More information1.2. Characteristics of Polynomial Functions. What are the key features of the graphs of polynomial functions?
1.2 Characteristics of Polnomial Functions In Section 1.1, ou eplored the features of power functions, which are single-term polnomial functions. Man polnomial functions that arise from real-world applications
More information3.2 Polynomial Functions of Higher Degree
71_00.qp 1/7/06 1: PM Page 6 Section. Polnomial Functions of Higher Degree 6. Polnomial Functions of Higher Degree What ou should learn Graphs of Polnomial Functions You should be able to sketch accurate
More informationOnline Homework Hints and Help Extra Practice
Evaluate: Homework and Practice Use a graphing calculator to graph the polnomial function. Then use the graph to determine the function s domain, range, and end behavior. (Use interval notation for the
More informationCSC 411: Lecture 02: Linear Regression
CSC 411: Lecture 02: Linear Regression Raquel Urtasun & Rich Zemel University of Toronto Sep 16, 2015 Urtasun & Zemel (UofT) CSC 411: 02-Regression Sep 16, 2015 1 / 16 Today Linear regression problem continuous
More informationAutomatic basis selection for RBF networks using Stein s unbiased risk estimator
Automatic basis selection for RBF networks using Stein s unbiased risk estimator Ali Ghodsi School of omputer Science University of Waterloo University Avenue West NL G anada Email: aghodsib@cs.uwaterloo.ca
More informationMachine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
More informationChapter 1. Limits and Continuity. 1.1 Limits
Chapter Limits and Continuit. Limits The its is the fundamental notion of calculus. This underling concept is the thread that binds together virtuall all of the calculus ou are about to stud. In this section,
More informationCSC 411 Lecture 4: Ensembles I
CSC 411 Lecture 4: Ensembles I Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 04-Ensembles I 1 / 22 Overview We ve seen two particular classification algorithms:
More informationClustering Part 2. A Partitional Clustering
Universit of Florida CISE department Gator Engineering Clustering Part Dr. Sanja Ranka Professor Computer and Information Science and Engineering Universit of Florida, Gainesville Universit of Florida
More informationNUMERICAL PERFORMANCE OF COMPACT FOURTH ORDER FORMULATION OF THE NAVIER-STOKES EQUATIONS
Published in : Communications in Numerical Methods in Engineering (008 Commun.Numer.Meth.Engng. 008; Vol : pp 003-019 NUMERICAL PERFORMANCE OF COMPACT FOURTH ORDER FORMULATION OF THE NAVIER-STOKES EQUATIONS
More informationDerivatives 3: The Derivative as a Function
Derivatives : The Derivative as a Function 77 Derivatives : The Derivative as a Function Model : Graph of a Function 9 8 7 6 5 g() - - - 5 6 7 8 9 0 5 6 7 8 9 0 5 - - -5-6 -7 Construct Your Understanding
More informationPredictive Analytics: Demystifying Current and Emerging Methodologies. Tom Kolde, FCAS, MAAA Linda Brobeck, FCAS, MAAA
Predictive Analytics: Demystifying Current and Emerging Methodologies Tom Kolde, FCAS, MAAA Linda Brobeck, FCAS, MAAA May 18, 2017 About the Presenters Tom Kolde, FCAS, MAAA Consulting Actuary Chicago,
More informationk-nearest Neighbors + Model Selection
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University k-nearest Neighbors + Model Selection Matt Gormley Lecture 5 Jan. 30, 2019 1 Reminders
More informationModel selection and validation 1: Cross-validation
Model selection and validation 1: Cross-validation Ryan Tibshirani Data Mining: 36-462/36-662 March 26 2013 Optional reading: ISL 2.2, 5.1, ESL 7.4, 7.10 1 Reminder: modern regression techniques Over the
More informationPattern Recognition for Neuroimaging Data
Pattern Recognition for Neuroimaging Data Edinburgh, SPM course April 2013 C. Phillips, Cyclotron Research Centre, ULg, Belgium http://www.cyclotron.ulg.ac.be Overview Introduction Univariate & multivariate
More informationUsing a Table of Values to Sketch the Graph of a Polynomial Function
A point where the graph changes from decreasing to increasing is called a local minimum point. The -value of this point is less than those of neighbouring points. An inspection of the graphs of polnomial
More informationmodel order p weights The solution to this optimization problem is obtained by solving the linear system
CS 189 Introduction to Machine Learning Fall 2017 Note 3 1 Regression and hyperparameters Recall the supervised regression setting in which we attempt to learn a mapping f : R d R from labeled examples
More informationLecture 25: Review I
Lecture 25: Review I Reading: Up to chapter 5 in ISLR. STATS 202: Data mining and analysis Jonathan Taylor 1 / 18 Unsupervised learning In unsupervised learning, all the variables are on equal standing,
More informationMoving Beyond Linearity
Moving Beyond Linearity Basic non-linear models one input feature: polynomial regression step functions splines smoothing splines local regression. more features: generalized additive models. Polynomial
More information