Symbolic Regression or:
|
|
|
- Warren Cook
- 5 years ago
- Views:
Transcription
1 Symbolic Regression or: How I Learned to Worry About my Machine Learning Models Fabrício Olivetti de França Universidade Federal do ABC folivetti@ufabc.edu.br
2 First there was data Huge amount of data can be collected nowadays due to: Increase of available storage space New techniques for data collection (not necessarily computer related) New ways to process large quantity of raw data
3 and then the models were created With such humongous quantity of data, we had to mine some answers from them! So we ve created mathematical models that can answer things like: This sample belongs to this category, but this does not! These set of samples defines a group Given these variables, the expected response is x
4 The Good, Linear Regression: ŷ=w*x+b Clear interpretation: for every xi, we (de)increase the total value by wi. Surprisingly good enough to fit many real world samples But if the samples present a nonlinear relationship
5 the Bad, ŷ = mean(y) -- regression ŷ = mode(y) -- classification Incredibly simple and incredibly low accuracy!
![h2) w3 W3] ŷ = tanh(w4. h3) It gets the job done!](/docs-images/93/114170747/images/6-1.jpg "It can fit anything! But what does the answer mean?")
6 and the Deep Learning h1 = [tanh (w1. x) w1 W1] h2 = [tanh (w2. h1) w2 W2] h3 = [tanh (w3. h2) w3 W3] ŷ = tanh(w4. h3) It gets the job done! It can fit anything! But what does the answer mean? Is the data right?
7 The answer to the life, the universe and everything 42
8 The answer to the life, the universe and everything 42 BUT WHAT IS THE QUESTION?
9 Current state of Data Science in a picture
10 But isn t science about learning how things work rather getting a numerical result?
11 A debt situation A Machine Learning model can help answer: - This client will pay his debt? - How much will he owe?
12 A debt situation This can support the decision process to whether the bank will lend money or not! And how much!
13 A debt situation Wouldn t it be more interesting to understand why a given profile won t pay back? Is it something the bank can fix? If it can, more money!
14 And what if the data is wrong?
15 And thus the answer is wrong?
16 Judge Dredd Can a Machine Learning model estimate recurrence of criminal? Can also estimate the number of jail time a judge should give? Futurama: Season 01, Episode 05
17 Judge Dredd The model will be fitted to the current data, the current data will include collected statistics regarding past crimes and judges mistakes, social prejudice, and the Machine will learn that.
18 The meaning of life Sometimes the interesting part of the study is to discover what equations govern a given natural process.
19 Symbolic Regression Regression model that searches the space of mathematical expressions for one (or more) that: Fits the sampled data Is simple!
20 Symbolic Regression Regression model that searches the space of mathematical expressions for one (or more) that: Fits the sampled data Is simple!
21 The Simple Life of a Mathematical Model How can we measure the simplicity of a Mathematical Equation? There is no formalization yet! But we all agree that: The smaller the better Less nonlinearity is simpler
22 Genetic Programming Evolving computer programs described as expressions: + * 3 λx (3*x) + sin(x) sin x x
23 Genetic Programming Simple steps: Combine expressions Change expressions Select expressions Repeat
24 Combine Expressions + + s * 3 - x s c x x x s c x * 3 x
25 Change Expressions + s * 3 + x x s 3 x x
26 Select Expressions Sample the expressions with probability proportional to the simplicity and inversely proportional to the fitting error. Expected to find a good expression sometime in the near future...
27 The near future This algorithm usually take hours to fit simple data with just a few variables...days for complex data...and we usually end up with: e *id(x_1^-4.0 * x_2^3.0 * x_3^1.0) *id(x_1^-4.0 * x_2^3.0 * x_3^2.0) *id(x_1^-4.0 * x_2^3.0 * x_3^3.0) *id(x_1^-4.0 * x_2^3.0 * x_3^4.0) *cos(x_1^-3.0 * x_2^-1.0) *cos(x_1^-3.0) *cos(x_1^-3.0 * x_3^1.0) *cos(x_1^-3.0 * x_2^1.0) *cos(x_1^-2.0 * x_2^-2.0) *log(x_1^-2.0 * x_2^-2.0) *cos(x_1^-2.0 * x_2^-1.0) *cos(x_1^-2.0 * x_2^-1.0 * x_3^1.0) *cos(x_1^-2.0) *cos(x_1^-2.0 * x_3^1.0) *cos(x_1^-2.0 * x_3^2.0) *cos(x_1^-2.0 * x_2^1.0) *cos(x_1^-2.0 * x_2^1.0 * x_3^1.0) *id(x_1^-2.0 * x_2^2.0) *sin(x_1^-2.0 * x_2^2.0) *cos(x_1^-2.0 * x_2^2.0) *tan(x_1^-2.0 * x_2^2.0) *log1p(x_1^-2.0 * x_2^2.0) *cos(x_1^-2.0 * x_2^2.0 * x_3^1.0) *tan(x_1^-2.0 * x_2^2.0 * x_3^1.0) +
28 Why?
29 How to improve it? Many solutions were devised: - Array-based representation - Semantic operators that minimize the observable change - Search space restriction
30 How to improve it? Many solutions were devised: - Array-based representation - Semantic operators that minimize the observable change - Search space restriction through representation
31 Interaction-Transformation
32 Data Structure: Expression Poly = [Int] -- coefficients of a polynomial transformation Term = (Poly, λ) -- the polynomial applied to a function Expression = {Term} -- a set of terms
33 What it can and what cannot
34 Simple algorithm Start with linear expression Repeat: Generate all interactions Generate all transformations Incrementally add terms as long as they reduce mse Weights w are adjusted as a linear regression.
35 Interaction Poly1 = [1, 1] -- x1. x2 Poly2 = [0, 2] -- x2^2 Poly1 + Poly2 = [1, 3] -- x1. x2^3 Poly1 - Poly2 = [1, -1] -- x1 / x2
36 Transformation (Poly1, id) [(Poly1, sin), (Poly1, cos), (Poly1, tan), ]
37 Marathon Time Prediction Data: recorded the training data of marathoners from Prague. Km4weeks: total Kms run in the last 4 weeks Sp4weeks: avg. speed in the last 4 weeks Marathon time: target variable
38 Results Multi Layer Perceptron: tanh activation 50 x 50 x 10 hidden neurons MSE: Linear Regression: ŷ = speed MSE: Symbolic Regression: ŷ = sin(speed) speed km/speed cos(km) tan(km. speed) MSE: 0.029
39 Results Canonical Genetic Programming: MSE: (after half an hour)
40 Results Symbolic Regression also found a solution with MSE = 0.02, but with 20 terms (still much simpler than Neural Network).
41 Some insights Evidently, km / speed (fourth term) gives us the expected time! The first part of the expression. Within the speed range, gives us:
42 Some insights This might indicate that: The best athletes run faster (duh). A middle class of athletes keep a lower speed, probably to keep it constant. The lower class tries to increase their speed at the beginning, raising the average speed, but with a penalty by the final half of the marathon.
43 Final Remarks Symbolic Regression can be (in the near future) a very competitive approach to advanced model fitting. The main advantage is the expressiveness of the model that can help us detect biases in the data and understand what the model is doing.
44 Final Remarks So far, there is no SR algorithm capable of dealing with high dimensions while competing with Neural Network regarding minimization of error and speed.
Ensemble methods in machine learning. Example. Neural networks. Neural networks
Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you
CSE 158 Lecture 2. Web Mining and Recommender Systems. Supervised learning Regression
CSE 158 Lecture 2 Web Mining and Recommender Systems Supervised learning Regression Supervised versus unsupervised learning Learning approaches attempt to model data in order to solve a problem Unsupervised
Lecture 3: Linear Classification
Lecture 3: Linear Classification Roger Grosse 1 Introduction Last week, we saw an example of a learning task called regression. There, the goal was to predict a scalar-valued target from a set of features.
Clustering algorithms and autoencoders for anomaly detection
Clustering algorithms and autoencoders for anomaly detection Alessia Saggio Lunch Seminars and Journal Clubs Université catholique de Louvain, Belgium 3rd March 2017 a Outline Introduction Clustering algorithms
Machine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
Download Free Pictures & Wallpaper from the Internet
Download Free Pictures & Wallpaper from the Internet D 600 / 1 Millions of Free Graphics and Images at Your Fingertips! Discover How To Get Your Hands on Them Almost any type of document you create can
Analytical model A structure and process for analyzing a dataset. For example, a decision tree is a model for the classification of a dataset.
Glossary of data mining terms: Accuracy Accuracy is an important factor in assessing the success of data mining. When applied to data, accuracy refers to the rate of correct values in the data. When applied
The Problem of Overfitting with Maximum Likelihood
The Problem of Overfitting with Maximum Likelihood In the previous example, continuing training to find the absolute maximum of the likelihood produced overfitted results. The effect is much bigger if
Outline. CS 6776 Evolutionary Computation. Numerical Optimization. Fitness Function. ,x 2. ) = x 2 1. , x , 5.0 x 1.
Outline CS 6776 Evolutionary Computation January 21, 2014 Problem modeling includes representation design and Fitness Function definition. Fitness function: Unconstrained optimization/modeling Constrained
Math 3 Coordinate Geometry Part 2 Graphing Solutions
Math 3 Coordinate Geometry Part 2 Graphing Solutions 1 SOLVING SYSTEMS OF EQUATIONS GRAPHICALLY The solution of two linear equations is the point where the two lines intersect. For example, in the graph
Linear Models. Lecture Outline: Numeric Prediction: Linear Regression. Linear Classification. The Perceptron. Support Vector Machines
Linear Models Lecture Outline: Numeric Prediction: Linear Regression Linear Classification The Perceptron Support Vector Machines Reading: Chapter 4.6 Witten and Frank, 2nd ed. Chapter 4 of Mitchell Solving
Lecture #11: The Perceptron
Lecture #11: The Perceptron Mat Kallada STAT2450 - Introduction to Data Mining Outline for Today Welcome back! Assignment 3 The Perceptron Learning Method Perceptron Learning Rule Assignment 3 Will be
More on Neural Networks. Read Chapter 5 in the text by Bishop, except omit Sections 5.3.3, 5.3.4, 5.4, 5.5.4, 5.5.5, 5.5.6, 5.5.7, and 5.
More on Neural Networks Read Chapter 5 in the text by Bishop, except omit Sections 5.3.3, 5.3.4, 5.4, 5.5.4, 5.5.5, 5.5.6, 5.5.7, and 5.6 Recall the MLP Training Example From Last Lecture log likelihood
Chap.12 Kernel methods [Book, Chap.7]
![Chap.12 Kernel methods [Book, Chap.7]](/thumbs/83/87786394.jpg "Chap.12 Kernel methods [Book, Chap.7]")
Chap.12 Kernel methods [Book, Chap.7] Neural network methods became popular in the mid to late 1980s, but by the mid to late 1990s, kernel methods have also become popular in machine learning. The first
Using the HP 38G in upper school: preliminary thoughts
Using the HP 38G in upper school: preliminary thoughts Barry Kissane Murdoch University The following examples have been devised to show some of the range of ways in which the HP 38G graphics calculator
Simple Model Selection Cross Validation Regularization Neural Networks
Neural Nets: Many possible refs e.g., Mitchell Chapter 4 Simple Model Selection Cross Validation Regularization Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February
A Neural Network Model Of Insurance Customer Ratings
A Neural Network Model Of Insurance Customer Ratings Jan Jantzen 1 Abstract Given a set of data on customers the engineering problem in this study is to model the data and classify customers
Week 3: Perceptron and Multi-layer Perceptron
Week 3: Perceptron and Multi-layer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, Fitzhugh-Nagumo model and Izhikevich model. This week,
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Sources Hastie, Tibshirani, Friedman: The Elements of Statistical Learning James, Witten, Hastie, Tibshirani: An Introduction to Statistical Learning Andrew Ng:
The MathType Window. The picture below shows MathType with all parts of its toolbar visible: Small bar. Tabs. Ruler. Selection.
Handle MathType User Manual The MathType Window The picture below shows MathType with all parts of its toolbar visible: Symbol palettes Template palettes Tabs Small bar Large tabbed bar Small tabbed bar
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 15
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 15 Slides adapted from Jordan Boyd-Graber Machine Learning: Chenhao Tan Boulder 1 of 21 Logistics HW3 available on Github, due on October
DEEP LEARNING IN PYTHON. Introduction to deep learning
DEEP LEARNING IN PYTHON Introduction to deep learning Imagine you work for a bank You need to predict how many transactions each customer will make next year Example as seen by linear regression Age Bank
CPSC 340: Machine Learning and Data Mining. More Linear Classifiers Fall 2017
CPSC 340: Machine Learning and Data Mining More Linear Classifiers Fall 2017 Admin Assignment 3: Due Friday of next week. Midterm: Can view your exam during instructor office hours next week, or after
LinReg 2.06 Manual. DePauw University: Physics Department 120 Physics I and 130 Physics II
LinReg 2.06 Manual DePauw University: Physics Department 120 Physics I and 130 Physics II Updated March 23 rd, 2011 1 About LinReg: LinReg is a program used extensively in DePauw s physics laboratory classes.
Chapter 7: Analytic Trigonometry
Chapter 7: Analytic Trigonometry 7. Trigonometric Identities Below are the basic trig identities discussed in previous chapters. Reciprocal csc(x) sec(x) cot(x) sin(x) cos(x) tan(x) Quotient sin(x) cos(x)
Why Use Graphs? Test Grade. Time Sleeping (Hrs) Time Sleeping (Hrs) Test Grade
Analyzing Graphs Why Use Graphs? It has once been said that a picture is worth a thousand words. This is very true in science. In science we deal with numbers, some times a great many numbers. These numbers,
LESSON 1: Trigonometry Pre-test
LESSON 1: Trigonometry Pre-test Instructions. Answer each question to the best of your ability. If there is more than one answer, put both/all answers down. Try to answer each question, but if there is
A Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
Introduction to Engineering gii
25.108 Introduction to Engineering gii Dr. Jay Weitzen Lecture Notes I: Introduction to Matlab from Gilat Book MATLAB - Lecture # 1 Starting with MATLAB / Chapter 1 Topics Covered: 1. Introduction. 2.
MATLAB QUICK START TUTORIAL
MATLAB QUICK START TUTORIAL This tutorial is a brief introduction to MATLAB which is considered one of the most powerful languages of technical computing. In the following sections, the basic knowledge
CS1114 Section 8: The Fourier Transform March 13th, 2013
CS1114 Section 8: The Fourier Transform March 13th, 2013 http://xkcd.com/26 Today you will learn about an extremely useful tool in image processing called the Fourier transform, and along the way get more
SEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah
Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Reference Most of the slides are taken from the third chapter of the online book by Michael Nielson: neuralnetworksanddeeplearning.com
Using Microsoft Excel
About Excel Using Microsoft Excel What is a Spreadsheet? Microsoft Excel is a program that s used for creating spreadsheets. So what is a spreadsheet? Before personal computers were common, spreadsheet
Precalculus: Graphs of Tangent, Cotangent, Secant, and Cosecant Practice Problems. Questions
Questions 1. Describe the graph of the function in terms of basic trigonometric functions. Locate the vertical asymptotes and sketch two periods of the function. y = 3 tan(x/2) 2. Solve the equation csc
^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held
Rudolf Kruse Christian Borgelt Frank Klawonn Christian Moewes Matthias Steinbrecher Pascal Held Computational Intelligence A Methodological Introduction ^ Springer Contents 1 Introduction 1 1.1 Intelligent
The problem we have now is called variable selection or perhaps model selection. There are several objectives.
STAT-UB.0103 NOTES for Wednesday 01.APR.04 One of the clues on the library data comes through the VIF values. These VIFs tell you to what extent a predictor is linearly dependent on other predictors. We
Neural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
Basis Functions. Volker Tresp Summer 2016
Basis Functions Volker Tresp Summer 2016 1 I am an AI optimist. We ve got a lot of work in machine learning, which is sort of the polite term for AI nowadays because it got so broad that it s not that
Knowledge Discovery and Data Mining. Neural Nets. A simple NN as a Mathematical Formula. Notes. Lecture 13 - Neural Nets. Tom Kelsey.
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
Knowledge Discovery and Data Mining
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
AMS 27L LAB #1 Winter 2009
AMS 27L LAB #1 Winter 2009 Introduction to MATLAB Objectives: 1. To introduce the use of the MATLAB software package 2. To learn elementary mathematics in MATLAB Getting Started: Log onto your machine
Table of Laplace Transforms
Table of Laplace Transforms 1 1 2 3 4, p > -1 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Heaviside Function 27 28. Dirac Delta Function 29 30. 31 32. 1 33 34. 35 36. 37 Laplace Transforms
Vocabulary Unit 2-3: Linear Functions & Healthy Lifestyles. Scale model a three dimensional model that is similar to a three dimensional object.
Scale a scale is the ratio of any length in a scale drawing to the corresponding actual length. The lengths may be in different units. Scale drawing a drawing that is similar to an actual object or place.
CSC 578 Neural Networks and Deep Learning
CSC 578 Neural Networks and Deep Learning Fall 2018/19 7. Recurrent Neural Networks (Some figures adapted from NNDL book) 1 Recurrent Neural Networks 1. Recurrent Neural Networks (RNNs) 2. RNN Training
CS273 Midterm Exam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014
CS273 Midterm Eam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014 Your name: Your UCINetID (e.g., myname@uci.edu): Your seat (row and number): Total time is 80 minutes. READ THE
Week 3 Video 4. Automated Feature Generation Automated Feature Selection
Week 3 Video 4 Automated Feature Generation Automated Feature Selection Automated Feature Generation The creation of new data features in an automated fashion from existing data features Multiplicative
Green Globs And Graphing Equations
Green Globs And Graphing Equations Green Globs and Graphing Equations has four parts to it which serve as a tool, a review or testing device, and two games. The menu choices are: Equation Plotter which
CSE Data Mining Concepts and Techniques STATISTICAL METHODS (REGRESSION) Professor- Anita Wasilewska. Team 13
CSE 634 - Data Mining Concepts and Techniques STATISTICAL METHODS Professor- Anita Wasilewska (REGRESSION) Team 13 Contents Linear Regression Logistic Regression Bias and Variance in Regression Model Fit
Lecture 25: Review I
Lecture 25: Review I Reading: Up to chapter 5 in ISLR. STATS 202: Data mining and analysis Jonathan Taylor 1 / 18 Unsupervised learning In unsupervised learning, all the variables are on equal standing,
The Addition Formulas in Trigonometry. Scott Fallstrom Faculty Director, Math Learning Center
The Addition Formulas in Trigonometry Scott Fallstrom Faculty Director, Math Learning Center Why not the usual? In Mathematics, we know that the distributive property allows 7(x + 5) = 7x + 35 With derivatives,
CSI31 Lecture 5. Topics: 3.1 Numeric Data Types 3.2 Using the Math Library 3.3 Accumulating Results: Factorial
CSI31 Lecture 5 Topics: 3.1 Numeric Data Types 3.2 Using the Math Library 3.3 Accumulating Results: Factorial 1 3.1 Numberic Data Types When computers were first developed, they were seen primarily as
Structured Completion Predictors Applied to Image Segmentation
Structured Completion Predictors Applied to Image Segmentation Dmitriy Brezhnev, Raphael-Joel Lim, Anirudh Venkatesh December 16, 2011 Abstract Multi-image segmentation makes use of global and local features
Programming in QBasic
Programming in QBasic Second lecture Constants In QBASIC: Constants In QBASIC division into three types: 1. Numeric Constants: there are two types of numeric constants: Real: the numbers used may be written
Week 4: Day 1: Unit circle and meaning!
Week : Day 1: Unit circle and meaning! SAT QOD (5 min) Mission Questions (10 min) o Should lead right into unit circle and what the heck it means It begins! (0 min) o The unit circle has radius 1. o Fill
Data Mining and Analytics
Data Mining and Analytics Aik Choon Tan, Ph.D. Associate Professor of Bioinformatics Division of Medical Oncology Department of Medicine aikchoon.tan@ucdenver.edu 9/22/2017 http://tanlab.ucdenver.edu/labhomepage/teaching/bsbt6111/
Generating Functions
6.04/8.06J Mathematics for Computer Science Srini Devadas and Eric Lehman April 7, 005 Lecture Notes Generating Functions Generating functions are one of the most surprising, useful, and clever inventions
Unit 1. Word Definition Picture. The number s distance from 0 on the number line. The symbol that means a number is greater than the second number.
Unit 1 Word Definition Picture Absolute Value The number s distance from 0 on the number line. -3 =3 Greater Than The symbol that means a number is greater than the second number. > Greatest to Least To
AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 2 Wednesday, January 27th 1 Overview In our previous lecture we discussed several applications of optimization, introduced basic terminology,
Lecture 11 Unbounded Arrays
Lecture 11 Unbounded Arrays 15-122: Principles of Imperative Computation (Spring 2018) Rob Simmons, Frank Pfenning Arrays have efficient O(1) access to elements given an index, but their size is set at
LESSON 1: Trigonometry Pre-test
LESSON 1: Trigonometry Pre-test Instructions. Answer each question to the best of your ability. If there is more than one answer, put both/all answers down. Try to answer each question, but if there is
An introduction to plotting data
An introduction to plotting data Eric D. Black California Institute of Technology February 25, 2014 1 Introduction Plotting data is one of the essential skills every scientist must have. We use it on a
Neural Nets for Adaptive Filter and Adaptive Pattern Recognition
Adaptive Pattern btyoung@gmail.com CSCE 636 10 February 2010 Outline Adaptive Combiners and Filters Minimal Disturbance and the Algorithm Madaline Rule II () Published 1988 in IEEE Journals Bernard Widrow
Higher. The Wave Equation. The Wave Equation 146
Higher Mathematics UNIT OUTCOME 4 The Wave Equation Contents The Wave Equation 146 1 Expressing pcosx + qsinx in the form kcos(x a 146 Expressing pcosx + qsinx in other forms 147 Multiple Angles 148 4
Text Input and Conditionals
Text Input and Conditionals Text Input Many programs allow the user to enter information, like a username and password. Python makes taking input from the user seamless with a single line of code: input()
Logistic Regression: Probabilistic Interpretation
Logistic Regression: Probabilistic Interpretation Approximate 0/1 Loss Logistic Regression Adaboost (z) SVM Solution: Approximate 0/1 loss with convex loss ( surrogate loss) 0-1 z = y w x SVM (hinge),
Open Access Research on the Prediction Model of Material Cost Based on Data Mining
Send Orders for Reprints to reprints@benthamscience.ae 1062 The Open Mechanical Engineering Journal, 2015, 9, 1062-1066 Open Access Research on the Prediction Model of Material Cost Based on Data Mining
Notes on Multilayer, Feedforward Neural Networks
Notes on Multilayer, Feedforward Neural Networks CS425/528: Machine Learning Fall 2012 Prepared by: Lynne E. Parker [Material in these notes was gleaned from various sources, including E. Alpaydin s book
RATIOS AND PROPORTIONS. Lesson 2.2
RATIOS AND PROPORTIONS Lesson 2.2 DO NOW TEACHING POINT Mathematicians will be able to determine proportionality using a graph and rate table STANDARD 7.RP.2a Recognize and represent proportional relationships
Neural Networks. By Laurence Squires
Neural Networks By Laurence Squires Machine learning What is it? Type of A.I. (possibly the ultimate A.I.?!?!?!) Algorithms that learn how to classify data The algorithms slowly change their own variables
CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION
75 CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION 6.1 INTRODUCTION Counter propagation network (CPN) was developed by Robert Hecht-Nielsen as a means to combine an unsupervised Kohonen
Optimization and least squares. Prof. Noah Snavely CS1114
Optimization and least squares Prof. Noah Snavely CS1114 http://cs1114.cs.cornell.edu Administrivia A5 Part 1 due tomorrow by 5pm (please sign up for a demo slot) Part 2 will be due in two weeks (4/17)
Applying Machine Learning to Real Problems: Why is it Difficult? How Research Can Help?
Applying Machine Learning to Real Problems: Why is it Difficult? How Research Can Help? Olivier Bousquet, Google, Zürich, obousquet@google.com June 4th, 2007 Outline 1 Introduction 2 Features 3 Minimax
Functions and Inverses ID1050 Quantitative & Qualitative Reasoning
Functions and Inverses ID1050 Quantitative & Qualitative Reasoning Function and Inverse Function Gun Analogy Functions take a number, perform an operation on it, and return another number. The inverse
Functions. Jason Filippou UMCP. Jason Filippou UMCP) Functions / 19
Functions Jason Filippou CMSC250 @ UMCP 06-22-2016 Jason Filippou (CMSC250 @ UMCP) Functions 06-22-2016 1 / 19 Outline 1 Basic definitions and examples 2 Properties of functions 3 The pigeonhole principle
PV211: Introduction to Information Retrieval
PV211: Introduction to Information Retrieval http://www.fi.muni.cz/~sojka/pv211 IIR 15-1: Support Vector Machines Handout version Petr Sojka, Hinrich Schütze et al. Faculty of Informatics, Masaryk University,
Supervised Learning with Neural Networks. We now look at how an agent might learn to solve a general problem by seeing examples.
Supervised Learning with Neural Networks We now look at how an agent might learn to solve a general problem by seeing examples. Aims: to present an outline of supervised learning as part of AI; to introduce
Trigonometric Graphs Dr. Laura J. Pyzdrowski
1 Names: About this Laboratory In this laboratory, we will examine trigonometric functions and their graphs. Upon completion of the lab, you should be able to quickly sketch such functions and determine
1 Introduction to Using Excel Spreadsheets
Survey of Math: Excel Spreadsheet Guide (for Excel 2007) Page 1 of 6 1 Introduction to Using Excel Spreadsheets This section of the guide is based on the file (a faux grade sheet created for messing with)
PARTICIPANT REGISTRATION GUIDE
PARTICIPANT REGISTRATION GUIDE Please note: This guide talks about standard configurations. All events are different and you may not see all the options we discuss within the event in which you re participating.
How to predict IMDb score
How to predict IMDb score Jiawei Li A53226117 Computational Science, Mathematics and Engineering University of California San Diego jil206@ucsd.edu Abstract This report is based on the dataset provided
Lecture 4 CSE July 1992
Lecture 4 CSE 110 6 July 1992 1 More Operators C has many operators. Some of them, like +, are binary, which means that they require two operands, as in 4 + 5. Others are unary, which means they require
Chapter 10. Conclusion Discussion
Chapter 10 Conclusion 10.1 Discussion Question 1: Usually a dynamic system has delays and feedback. Can OMEGA handle systems with infinite delays, and with elastic delays? OMEGA handles those systems with
Machine Learning for NLP
Machine Learning for NLP Support Vector Machines Aurélie Herbelot 2018 Centre for Mind/Brain Sciences University of Trento 1 Support Vector Machines: introduction 2 Support Vector Machines (SVMs) SVMs
Boosting Simple Model Selection Cross Validation Regularization. October 3 rd, 2007 Carlos Guestrin [Schapire, 1989]
![Boosting Simple Model Selection Cross Validation Regularization. October 3 rd, 2007 Carlos Guestrin [Schapire, 1989]](/thumbs/85/91370094.jpg "Boosting Simple Model Selection Cross Validation Regularization. October 3 rd, 2007 Carlos Guestrin [Schapire, 1989]")
Boosting Simple Model Selection Cross Validation Regularization Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 3 rd, 2007 1 Boosting [Schapire, 1989] Idea: given a weak
Lecture 16: High-dimensional regression, non-linear regression
Lecture 16: High-dimensional regression, non-linear regression Reading: Sections 6.4, 7.1 STATS 202: Data mining and analysis November 3, 2017 1 / 17 High-dimensional regression Most of the methods we
Data mining with Support Vector Machine
Data mining with Support Vector Machine Ms. Arti Patle IES, IPS Academy Indore (M.P.) artipatle@gmail.com Mr. Deepak Singh Chouhan IES, IPS Academy Indore (M.P.) deepak.schouhan@yahoo.com Abstract: Machine
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
scikit-learn (Machine Learning in Python)
scikit-learn (Machine Learning in Python) (PB13007115) 2016-07-12 (PB13007115) scikit-learn (Machine Learning in Python) 2016-07-12 1 / 29 Outline 1 Introduction 2 scikit-learn examples 3 Captcha recognize
Translation of graphs (2) The exponential function and trigonometric function
Lesson 35 Translation of graphs (2) The exponential function and trigonometric function Learning Outcomes and Assessment Standards Learning Outcome 2: Functions and Algebra Assessment Standard Generate
Keywords Fuzzy, Set Theory, KDD, Data Base, Transformed Database.
Volume 6, Issue 5, May 016 ISSN: 77 18X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Fuzzy Logic in Online
Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda
Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda 1 Observe novel applicability of DL techniques in Big Data Analytics. Applications of DL techniques for common Big Data Analytics problems. Semantic indexing
XP: Backup Your Important Files for Safety
XP: Backup Your Important Files for Safety X 380 / 1 Protect Your Personal Files Against Accidental Loss with XP s Backup Wizard Your computer contains a great many important files, but when it comes to
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining
Index. Umberto Michelucci 2018 U. Michelucci, Applied Deep Learning,
A Acquisition function, 298, 301 Adam optimizer, 175 178 Anaconda navigator conda command, 3 Create button, 5 download and install, 1 installing packages, 8 Jupyter Notebook, 11 13 left navigation pane,
Introduction to SNNS
Introduction to SNNS Caren Marzban http://www.nhn.ou.edu/ marzban Introduction In this lecture we will learn about a Neural Net (NN) program that I know a little about - the Stuttgart Neural Network Simulator
IMPORTANT WORDS AND WHAT THEY MEAN
MOBILE PHONES WHAT IS DATA Data is Internet. It can let you do lots of different things on your phone or tablet. You can send or receive texts, emails or photos, listen to music, watch TV shows, movies
INTRODUCTION TO DATA MINING
INTRODUCTION TO DATA MINING 1 Chiara Renso KDDLab - ISTI CNR, Italy http://www-kdd.isti.cnr.it email: chiara.renso@isti.cnr.it Knowledge Discovery and Data Mining Laboratory, ISTI National Research Council,
Polynomial Models Studio October 27, 2006
Polynomial Models Studio October 27, 26 A. Download the data spreadsheet, open it, and select the tab labeled Murder. This has the FBI Uniform Crime Statistics reports of Murder and non-negligent manslaughter
Neural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer