Neural Networks. Aarti Singh & Barnabas Poczos. Machine Learning / Apr 24, Slides Courtesy: Tom Mitchell
|
|
- Marjorie Terry
- 5 years ago
- Views:
Transcription
1 Neura Networks Aarti Singh & Barnabas Poczos Machine Learning / Apr 24, 2014 Sides Courtesy: Tom Mitche 1
2 Logis0c Regression Assumes the foowing func1ona form for P(Y X): Logis1c func1on appied to a inear func1on of the data Logistic function (or Sigmoid): ogit (z) z 2
3 Logis0c Regression is a Linear Cassifier! Assumes the foowing func1ona form for P(Y X): Decision boundary: 1 1 (Linear Decision Boundary) 3
4 Training Logis0c Regression How to earn the parameters w 0, w 1, w d? Training Data Maximum (Condi1ona) Likeihood Es1mates Discrimina1ve phiosophy Don t waste effort earning P(X), focus on P(Y X) that s a that malers for cassifica1on! 4
5 Op0mizing convex func0on Max Condi1ona og- ikeihood = Min Nega1ve Condi1ona og- ikeihood Nega1ve Condi1ona og- ikeihood is a convex func1on Gradient Descent (convex) Gradient: Update rue: Learning rate, η>0 5
6 Logis0c func0on as a Graph Sigmoid Unit d d d
7 Neura Networks to earn f: X à Y f can be a non- inear func1on X (vector of) con1nuous and/or discrete variabes Y (vector of) con1nuous and/or discrete variabes Neura networks - Represent f by network of ogis1c/sigmoid units: Sigmoid Unit Output ayer, Y Hidden ayer, H Input ayer, X
8 Neura Network trained to distinguish vowe sounds using 2 formants (features) Output ayer Hidden ayer Input ayer Two ayers of ogistic units Highy non-inear decision surface
9 Neura Network trained to drive a car! Weights to output units from the hidden unit Weights of each pixe for one hidden unit
10
11 Predic0on using Neura Networks Prediction Given neura network (hidden units and weights), use it to predict the abe of a test point Forward Propagation Start from input ayer For each subsequent ayer, compute output of sigmoid unit Sigmoid unit: 1-Hidden ayer, 1 output NN: o h
12 M(C)LE Training for Neura Networks Consider regression probem f:xà Y, for scaar Y y = f(x) + ε assume noise N(0,σ ε ), iid deterministic Let s maximize the conditiona data ikeihood Learned neura network Train weights of a units to minimize sum of squared errors of predicted network outputs
13 MAP Training for Neura Networks Consider regression probem f:xà Y, for scaar Y y = f(x) + ε noise N(0,σ ε ) deterministic Gaussian P(W) = N(0,σΙ) n P(W) c i w i 2 Train weights of a units to minimize sum of squared errors of predicted network outputs pus weight magnitudes
14 E Mean Square Error d For Neura Networks, E[w] no onger convex in w
15 Training Neura Networks d d d Differentiabe
16 Error Gradient for a Sigmoid Unit y y y y y y y Sigmoid Unit d d d
17 Using a training data D y y
18 (MLE) Using Forward propagation y k y k = target output (abe) o k/h = unit output (obtained by forward propagation) w ij = wt from i to j Note: if i is input variabe, o i = x i o
19 Objective/Error no onger convex in weights
20 Deaing with OverfiUng Our earning agorithm invoves a parameter n=number of gradient descent iterations How do we choose n to optimize future error? (note: simiar issue for ogistic regression, decision trees, ) e.g. the n that minimizes error rate of neura net over future data
21 Deaing with OverfiUng Our earning agorithm invoves a parameter n=number of gradient descent iterations How do we choose n to optimize future error? Separate avaiabe data into training and vaidation set Use training to perform gradient descent n ß number of iterations that optimizes vaidation set error
22 K- fod Cross- vaida0on Idea: train mutipe times, eaving out a disjoint subset of data each time for test. Average the test set accuracies. Partition data into K disjoint subsets For k=1 to K testdata = kth subset h ß cassifier trained* on a data except for testdata accuracy(k) = accuracy of h on testdata end FinaAccuracy = mean of the K recorded testset accuracies * might withhod some of this to choose number of gradient decent steps
23 Leave- one- out Cross- vaida0on This is just k-fod cross vaidation eaving out one exampe each iteration Partition data into K disjoint subsets, each containing one exampe For k=1 to K testdata = kth subset h ß cassifier trained* on a data except for testdata accuracy(k) = accuracy of h on testdata end FinaAccuracy = mean of the K recorded testset accuracies * might withhod some of this to choose number of gradient decent steps
24 Deaing with OverfiUng Cross-vaidation Reguarization sma weights impy NN is inear (ow VC dimension) Logistic output Contro number of hidden units ow compexity Σw i x i
25
26
27
28
29
30
31 eft strt right up w 0
32 Semantic Memory Mode Based on ANN s [McCeand & Rogers, Nature 2003] No hierarchy given. Train with assertions, e.g., Can(Canary,Fy)
33 Humans act as though they have a hierarchica memory organization 1. Victims of Semantic Dementia progressivey ose knowedge of objects But they ose specific detais first, genera properties ater, suggesting hierarchica memory organization 2. Chidren appear to earn genera categories and properties first, foowing the same hierarchy, top down *. NonLiving Thing Pant Living Fish Anima Bird Canary * some debate remains on this.
34 Memory deterioration foows semantic hierarchy [McCeand & Rogers, Nature 2003]
35
36 Ar0ficia Neura Networks: Summary Ac1vey used to mode distributed computa1on in brain Highy non- inear regression/cassifica1on Vector- vaued inputs and outputs Poten1ay miions of parameters to es1mate - overfiwng Hidden ayers earn intermediate representa1ons how many to use? Predic1on Forward propaga1on Gradient descent (Back- propaga1on), oca minima probems Coming back in new form as deep beief networks (probabiis1c interpreta1on)
Neural Networks. Aarti Singh. Machine Learning Nov 3, Slides Courtesy: Tom Mitchell
Neura Networks Aarti Singh Machine Learning 10-601 Nov 3, 2011 Sides Courtesy: Tom Mitche 1 Logis0c Regression Assumes the foowing func1ona form for P(Y X): Logis1c func1on appied to a inear func1on of
More informationLogis&c Regression. Aar$ Singh & Barnabas Poczos. Machine Learning / Jan 28, 2014
Logis&c Regression Aar$ Singh & Barnabas Poczos Machine Learning 10-701/15-781 Jan 28, 2014 Linear Regression & Linear Classifica&on Weight Height Linear fit Linear decision boundary 2 Naïve Bayes Recap
More informationNearest Neighbor Learning
Nearest Neighbor Learning Cassify based on oca simiarity Ranges from simpe nearest neighbor to case-based and anaogica reasoning Use oca information near the current query instance to decide the cassification
More informationMobile App Recommendation: Maximize the Total App Downloads
Mobie App Recommendation: Maximize the Tota App Downoads Zhuohua Chen Schoo of Economics and Management Tsinghua University chenzhh3.12@sem.tsinghua.edu.cn Yinghui (Catherine) Yang Graduate Schoo of Management
More informationLecture Notes for Chapter 4 Part III. Introduction to Data Mining
Data Mining Cassification: Basic Concepts, Decision Trees, and Mode Evauation Lecture Notes for Chapter 4 Part III Introduction to Data Mining by Tan, Steinbach, Kumar Adapted by Qiang Yang (2010) Tan,Steinbach,
More informationResearch of Classification based on Deep Neural Network
2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) Research of Emai Cassification based on Deep Neura Network Wang Yawen Schoo of Computer Science and Engineering Xi
More informationSpace-Time Trade-offs.
Space-Time Trade-offs. Chethan Kamath 03.07.2017 1 Motivation An important question in the study of computation is how to best use the registers in a CPU. In most cases, the amount of registers avaiabe
More informationLayer-Specific Adaptive Learning Rates for Deep Networks
Layer-Specific Adaptive Learning Rates for Deep Networks arxiv:1510.04609v1 [cs.cv] 15 Oct 2015 Bharat Singh, Soham De, Yangmuzi Zhang, Thomas Godstein, and Gavin Tayor Department of Computer Science Department
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
More informationMachine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
More informationNeural Networks and Deep Learning
Neural Networks and Deep Learning Example Learning Problem Example Learning Problem Celebrity Faces in the Wild Machine Learning Pipeline Raw data Feature extract. Feature computation Inference: prediction,
More informationTransformation Invariance in Pattern Recognition: Tangent Distance and Propagation
Transformation Invariance in Pattern Recognition: Tangent Distance and Propagation Patrice Y. Simard, 1 Yann A. Le Cun, 2 John S. Denker, 2 Bernard Victorri 3 1 Microsoft Research, 1 Microsoft Way, Redmond,
More informationGradient Descent. Michail Michailidis & Patrick Maiden
Gradient Descent Michail Michailidis & Patrick Maiden Outline Mo4va4on Gradient Descent Algorithm Issues & Alterna4ves Stochas4c Gradient Descent Parallel Gradient Descent HOGWILD! Mo4va4on It is good
More informationCMPT 882 Week 3 Summary
CMPT 882 Week 3 Summary! Artificial Neural Networks (ANNs) are networks of interconnected simple units that are based on a greatly simplified model of the brain. ANNs are useful learning tools by being
More informationFastest-Path Computation
Fastest-Path Computation DONGHUI ZHANG Coege of Computer & Information Science Northeastern University Synonyms fastest route; driving direction Definition In the United states, ony 9.% of the househods
More informationNatural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu
Natural Language Processing CS 6320 Lecture 6 Neural Language Models Instructor: Sanda Harabagiu In this lecture We shall cover: Deep Neural Models for Natural Language Processing Introduce Feed Forward
More informationLinear Regression & Gradient Descent
Linear Regression & Gradient Descent These slides were assembled by Byron Boots, with grateful acknowledgement to Eric Eaton and the many others who made their course materials freely available online.
More informationAutomatic Grouping for Social Networks CS229 Project Report
Automatic Grouping for Socia Networks CS229 Project Report Xiaoying Tian Ya Le Yangru Fang Abstract Socia networking sites aow users to manuay categorize their friends, but it is aborious to construct
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More informationSimple Model Selection Cross Validation Regularization Neural Networks
Neural Nets: Many possible refs e.g., Mitchell Chapter 4 Simple Model Selection Cross Validation Regularization Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February
More informationForgot to compute the new centroids (-1); error in centroid computations (-1); incorrect clustering results (-2 points); more than 2 errors: 0 points.
Probem 1 a. K means is ony capabe of discovering shapes that are convex poygons [1] Cannot discover X shape because X is not convex. [1] DBSCAN can discover X shape. [1] b. K-means is prototype based and
More informationConvolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
More informationStages of (Batch) Machine Learning
Evalua&on Stages of (Batch) Machine Learning Given: labeled training data X, Y = {hx i,y i i} n i=1 Assumes each x i D(X ) with y i = f target (x i ) Train the model: model ß classifier.train(x, Y ) x
More informationMACHINE learning techniques can, automatically,
Proceedings of Internationa Joint Conference on Neura Networks, Daas, Texas, USA, August 4-9, 203 High Leve Data Cassification Based on Network Entropy Fiipe Aves Neto and Liang Zhao Abstract Traditiona
More informationDistance Weighted Discrimination and Second Order Cone Programming
Distance Weighted Discrimination and Second Order Cone Programming Hanwen Huang, Xiaosun Lu, Yufeng Liu, J. S. Marron, Perry Haaand Apri 3, 2012 1 Introduction This vignette demonstrates the utiity and
More informationCOMPRESSIVE sensing (CS), which aims at recovering
D-Net: Deep Learning pproach for Compressive Sensing RI Yan Yang, Jian Sun, Huibin Li, and Zongben u ariv:705.06869v [cs.cv] 9 ay 07 bstract Compressive sensing (CS) is an effective approach for fast agnetic
More informationNaïve Bayes, Gaussian Distributions, Practical Applications
Naïve Bayes, Gaussian Distributions, Practical Applications Required reading: Mitchell draft chapter, sections 1 and 2. (available on class website) Machine Learning 10-601 Tom M. Mitchell Machine Learning
More informationAgent architectures. Francesco Amigoni
Francesco Amigoni Designing inteigent agents An agent is defined by its agent function f() that maps a sequence of perceptions to an action p(0), p(1),..., p(t) f() a(t) AGENT perceptions actions ENVIRONMENT
More informationA Fast-Convergence Decoding Method and Memory-Efficient VLSI Decoder Architecture for Irregular LDPC Codes in the IEEE 802.
A Fast-Convergence Decoding Method and Memory-Efficient VLSI Decoder Architecture for Irreguar LDPC Codes in the IEEE 82.16e Standards Yeong-Luh Ueng and Chung-Chao Cheng Dept. of Eectrica Engineering,
More informationSensitivity Analysis of Hopfield Neural Network in Classifying Natural RGB Color Space
Sensitivity Anaysis of Hopfied Neura Network in Cassifying Natura RGB Coor Space Department of Computer Science University of Sharjah UAE rsammouda@sharjah.ac.ae Abstract: - This paper presents a study
More informationML4Bio Lecture #1: Introduc3on. February 24 th, 2016 Quaid Morris
ML4Bio Lecture #1: Introduc3on February 24 th, 216 Quaid Morris Course goals Prac3cal introduc3on to ML Having a basic grounding in the terminology and important concepts in ML; to permit self- study,
More informationUnsupervised Learning: Clustering
Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning
More informationMore Relation Model: Functional Dependencies
More Reation Mode: Functiona Dependencies Lecture #7 Autumn, 2001 Fa, 2001, LRX #07 More Reation Mode: Functiona Dependencies HUST,Wuhan,China 152 Functiona Dependencies X -> A = assertion about a reation
More informationLecture 20: Neural Networks for NLP. Zubin Pahuja
Lecture 20: Neural Networks for NLP Zubin Pahuja zpahuja2@illinois.edu courses.engr.illinois.edu/cs447 CS447: Natural Language Processing 1 Today s Lecture Feed-forward neural networks as classifiers simple
More informationLogistic Regression. Abstract
Logistic Regression Tsung-Yi Lin, Chen-Yu Lee Department of Electrical and Computer Engineering University of California, San Diego {tsl008, chl60}@ucsd.edu January 4, 013 Abstract Logistic regression
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationImage Segmentation Using Semi-Supervised k-means
I J C T A, 9(34) 2016, pp. 595-601 Internationa Science Press Image Segmentation Using Semi-Supervised k-means Reza Monsefi * and Saeed Zahedi * ABSTRACT Extracting the region of interest is a very chaenging
More informationOpening the Black Box Data Driven Visualizaion of Neural N
Opening the Black Box Data Driven Visualizaion of Neural Networks September 20, 2006 Aritificial Neural Networks Limitations of ANNs Use of Visualization (ANNs) mimic the processes found in biological
More informationPerceptron as a graph
Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 10 th, 2007 2005-2007 Carlos Guestrin 1 Perceptron as a graph 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-6 -4-2
More informationSequential Learning of Layered Models from Video
Sequentia Learning of Layered Modes from Video Michais K. Titsias and Christopher K. I. Wiiams Schoo of Informatics, University of Edinburgh, Edinburgh EH1 2QL, UK M.Titsias@sms.ed.ac.uk, c.k.i.wiiams@ed.ac.uk
More information5940 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 13, NO. 11, NOVEMBER 2014
5940 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 13, NO. 11, NOVEMBER 014 Topoogy-Transparent Scheduing in Mobie Ad Hoc Networks With Mutipe Packet Reception Capabiity Yiming Liu, Member, IEEE,
More informationD-Separation. b) the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C.
D-Separation Say: A, B, and C are non-intersecting subsets of nodes in a directed graph. A path from A to B is blocked by C if it contains a node such that either a) the arrows on the path meet either
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationWhat is machine learning?
Machine learning, pattern recognition and statistical data modelling Lecture 12. The last lecture Coryn Bailer-Jones 1 What is machine learning? Data description and interpretation finding simpler relationship
More informationLearning to Generate with Memory
Chongxuan Li LICX14@MAILS.TSINGHUA.EDU.CN Jun Zhu DCSZJ@TSINGHUA.EDU.CN Bo Zhang DCSZB@TSINGHUA.EDU.CN Dept. of Comp. Sci. & Tech., State Key Lab of Inte. Tech. & Sys., TNList Lab, Center for Bio-Inspired
More informationAarti Singh. Machine Learning / Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg
Spectral Clustering Aarti Singh Machine Learning 10-701/15-781 Apr 7, 2010 Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg 1 Data Clustering Graph Clustering Goal: Given data points X1,, Xn and similarities
More informationMachine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
More informationArithmetic Coding. Prof. Ja-Ling Wu. Department of Computer Science and Information Engineering National Taiwan University
Arithmetic Coding Prof. Ja-Ling Wu Department of Computer Science and Information Engineering Nationa Taiwan University F(X) Shannon-Fano-Eias Coding W..o.g. we can take X={,,,m}. Assume p()>0 for a. The
More informationSTA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! h0p://www.cs.toronto.edu/~rsalakhu/ Lecture 3 Parametric Distribu>ons We want model the probability
More informationMachine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 9, 2012 Today: Graphical models Bayes Nets: Inference Learning Readings: Required: Bishop chapter
More informationSequential Learning of Layered Models from Video
Sequentia Learning of Layered Modes from Video Michais K. Titsias and Christopher K. I. Wiiams Schoo of Informatics, University of Edinburgh, Edinburgh EH1 2QL, UK M.Titsias@sms.ed.ac.uk, c.k.i.wiiams@ed.ac.uk
More informationNeural Networks: Learning. Cost func5on. Machine Learning
Neural Networks: Learning Cost func5on Machine Learning Neural Network (Classifica2on) total no. of layers in network no. of units (not coun5ng bias unit) in layer Layer 1 Layer 2 Layer 3 Layer 4 Binary
More informationMultiple Plane Phase Retrieval Based On Inverse Regularized Imaging and Discrete Diffraction Transform
Mutipe Pane Phase Retrieva Based On Inverse Reguaried Imaging and Discrete Diffraction Transform Artem Migukin, Vadimir Katkovnik, and Jaakko Astoa Department of Signa Processing, Tampere University of
More informationPattern Classification Algorithms for Face Recognition
Chapter 7 Pattern Classification Algorithms for Face Recognition 7.1 Introduction The best pattern recognizers in most instances are human beings. Yet we do not completely understand how the brain recognize
More informationQuaternion Support Vector Classifier
Quaternion Support Vector Cassifier G. López-Gonzáez, Nancy Arana-Danie, and Eduardo Bayro-Corrochano CINVESTAV - Unidad Guadaajara, Av. de Bosque 1145, Coonia e Bajo, Zapopan, Jaisco, México {geopez,edb}@gd.cinvestav.mx
More informationClustering. Image segmentation, document clustering, protein class discovery, compression
Clustering CS 444 Some material on these is slides borrowed from Andrew Moore's machine learning tutorials located at: Clustering The problem of grouping unlabeled data on the basis of similarity. A key
More informationA probabilistic fuzzy method for emitter identification based on genetic algorithm
A probabitic fuzzy method for emitter identification based on genetic agorithm Xia Chen, Weidong Hu, Hongwen Yang, Min Tang ATR Key Lab, Coege of Eectronic Science and Engineering Nationa University of
More informationLecture outline Graphics and Interaction Scan Converting Polygons and Lines. Inside or outside a polygon? Scan conversion.
Lecture outine 433-324 Graphics and Interaction Scan Converting Poygons and Lines Department of Computer Science and Software Engineering The Introduction Scan conversion Scan-ine agorithm Edge coherence
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining
More informationMulti-Manifold Deep Metric Learning for Image Set Classification
Muti-Manifod Deep Metric Learning for Image Set Cassification Jiwen Lu1, Gang Wang1,2, Weihong Deng3, Pierre Mouin1,4, and Jie Zhou5 1 Advanced Digita Sciences Center, Singapore 2 Schoo of Eectrica and
More informationNeural Network Enhancement of the Los Alamos Force Deployment Estimator
Missouri University of Science and Technoogy Schoars' Mine Eectrica and Computer Engineering Facuty Research & Creative Works Eectrica and Computer Engineering 1-1-1994 Neura Network Enhancement of the
More informationSupervised Learning in Neural Networks (Part 2)
Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (back-propagation training algorithm) The input signals are propagated in a forward direction on a layer-bylayer basis. Learning
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationRadial Basis Function Networks: Algorithms
Radial Basis Function Networks: Algorithms Neural Computation : Lecture 14 John A. Bullinaria, 2015 1. The RBF Mapping 2. The RBF Network Architecture 3. Computational Power of RBF Networks 4. Training
More informationA METHOD FOR GRIDLESS ROUTING OF PRINTED CIRCUIT BOARDS. A. C. Finch, K. J. Mackenzie, G. J. Balsdon, G. Symonds
A METHOD FOR GRIDLESS ROUTING OF PRINTED CIRCUIT BOARDS A C Finch K J Mackenzie G J Basdon G Symonds Raca-Redac Ltd Newtown Tewkesbury Gos Engand ABSTRACT The introduction of fine-ine technoogies to printed
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationDiagnosing Breast Cancer with a Neural Network
Undergraduate Journa of Mathematica Modeing: One + Two Voume 7 017 Spring 017 Issue Artice 4 Diagnosing Breast Cancer with a Neura Network John Cuen University of South Forida Advisors: Arcadii Grinshpan,
More informationAn Introduction to Design Patterns
An Introduction to Design Patterns 1 Definitions A pattern is a recurring soution to a standard probem, in a context. Christopher Aexander, a professor of architecture Why woud what a prof of architecture
More informationSupport Vector Machines.
Support Vector Machines srihari@buffalo.edu SVM Discussion Overview 1. Overview of SVMs 2. Margin Geometry 3. SVM Optimization 4. Overlapping Distributions 5. Relationship to Logistic Regression 6. Dealing
More informationAutomatic Hidden Web Database Classification
Automatic idden Web atabase Cassification Zhiguo Gong, Jingbai Zhang, and Qian Liu Facuty of Science and Technoogy niversity of Macau Macao, PRC {fstzgg,ma46597,ma46620}@umac.mo Abstract. In this paper,
More informationDeep Learning & Neural Networks
Deep Learning & Neural Networks Machine Learning CSE4546 Sham Kakade University of Washington November 29, 2016 Sham Kakade 1 Announcements: HW4 posted Poster Session Thurs, Dec 8 Today: Review: EM Neural
More informationMachine Learning in Telecommunications
Machine Learning in Telecommunications Paulos Charonyktakis & Maria Plakia Department of Computer Science, University of Crete Institute of Computer Science, FORTH Roadmap Motivation Supervised Learning
More informationDeformable Part Models
Deformable Part Models References: Felzenszwalb, Girshick, McAllester and Ramanan, Object Detec@on with Discrimina@vely Trained Part Based Models, PAMI 2010 Code available at hkp://www.cs.berkeley.edu/~rbg/latent/
More informationGradient Descent Optimization Algorithms for Deep Learning Batch gradient descent Stochastic gradient descent Mini-batch gradient descent
Gradient Descent Optimization Algorithms for Deep Learning Batch gradient descent Stochastic gradient descent Mini-batch gradient descent Slide credit: http://sebastianruder.com/optimizing-gradient-descent/index.html#batchgradientdescent
More informationBinarized support vector machines
Universidad Caros III de Madrid Repositorio instituciona e-archivo Departamento de Estadística http://e-archivo.uc3m.es DES - Working Papers. Statistics and Econometrics. WS 2007-11 Binarized support vector
More informationMachine Learning Classifiers and Boosting
Machine Learning Classifiers and Boosting Reading Ch 18.6-18.12, 20.1-20.3.2 Outline Different types of learning problems Different types of learning algorithms Supervised learning Decision trees Naïve
More informationFunctions. 6.1 Modular Programming. 6.2 Defining and Calling Functions. Gaddis: 6.1-5,7-10,13,15-16 and 7.7
Functions Unit 6 Gaddis: 6.1-5,7-10,13,15-16 and 7.7 CS 1428 Spring 2018 Ji Seaman 6.1 Moduar Programming Moduar programming: breaking a program up into smaer, manageabe components (modues) Function: a
More informationFormulation of Loss minimization Problem Using Genetic Algorithm and Line-Flow-based Equations
Formuation of Loss minimization Probem Using Genetic Agorithm and Line-Fow-based Equations Sharanya Jaganathan, Student Member, IEEE, Arun Sekar, Senior Member, IEEE, and Wenzhong Gao, Senior member, IEEE
More informationKnowledge Discovery and Data Mining. Neural Nets. A simple NN as a Mathematical Formula. Notes. Lecture 13 - Neural Nets. Tom Kelsey.
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
More informationLRBM: A Restricted Boltzmann Machine based Approach for Representation Learning on Linked Data
LRBM: A Restricted Botzmann Machine based Approach for Representation Learning on Linked Data Kang Li, Jing Gao, Suxin Guo, Nan Du, Xiaoyi Li and Aidong Zhang Department of Computer Science and Engineering
More informationJOINT IMAGE REGISTRATION AND EXAMPLE-BASED SUPER-RESOLUTION ALGORITHM
JOINT IMAGE REGISTRATION AND AMPLE-BASED SUPER-RESOLUTION ALGORITHM Hyo-Song Kim, Jeyong Shin, and Rae-Hong Park Department of Eectronic Engineering, Schoo of Engineering, Sogang University 35 Baekbeom-ro,
More informationWHILE estimating the depth of a scene from a single image
JOURNAL OF L A T E X CLASS FILES, VOL. 4, NO. 8, AUGUST 05 Monocuar Depth Estimation using Muti-Scae Continuous CRFs as Sequentia Deep Networks Dan Xu, Student Member, IEEE, Eisa Ricci, Member, IEEE, Wani
More informationLogistic Regression. May 28, Decision boundary is a property of the hypothesis and not the data set e z. g(z) = g(z) 0.
Logistic Regression May 28, 202 Logistic Regression. Decision Boundary Decision boundary is a property of the hypothesis and not the data set. sigmoid function: h (x) = g( x) = P (y = x; ) suppose predict
More informationDense Multimodal Fusion for Hierarchically Joint Representation
Dense Mutimoda Fusion for Hierarchicay Joint Representation Di Hu Schoo of Computer Science Northwestern Poytechnica University Feiping Nie Schoo of Computer Science Northwestern Poytechnica University
More informationKnowledge Discovery and Data Mining
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
More informationSemi- Supervised Learning
Semi- Supervised Learning Aarti Singh Machine Learning 10-601 Dec 1, 2011 Slides Courtesy: Jerry Zhu 1 Supervised Learning Feature Space Label Space Goal: Optimal predictor (Bayes Rule) depends on unknown
More informationAssignment # 5. Farrukh Jabeen Due Date: November 2, Neural Networks: Backpropation
Farrukh Jabeen Due Date: November 2, 2009. Neural Networks: Backpropation Assignment # 5 The "Backpropagation" method is one of the most popular methods of "learning" by a neural network. Read the class
More informationIntro to Programming & C Why Program? 1.2 Computer Systems: Hardware and Software. Hardware Components Illustrated
Intro to Programming & C++ Unit 1 Sections 1.1-3 and 2.1-10, 2.12-13, 2.15-17 CS 1428 Fa 2017 Ji Seaman 1.1 Why Program? Computer programmabe machine designed to foow instructions Program instructions
More informationNeural Networks. Theory And Practice. Marco Del Vecchio 19/07/2017. Warwick Manufacturing Group University of Warwick
Neural Networks Theory And Practice Marco Del Vecchio marco@delvecchiomarco.com Warwick Manufacturing Group University of Warwick 19/07/2017 Outline I 1 Introduction 2 Linear Regression Models 3 Linear
More informationNotes and Announcements
Notes and Announcements Midterm exam: Oct 20, Wednesday, In Class Late Homeworks Turn in hardcopies to Michelle. DO NOT ask Michelle for extensions. Note down the date and time of submission. If submitting
More informationChapter Multidimensional Direct Search Method
Chapter 09.03 Mutidimensiona Direct Search Method After reading this chapter, you shoud be abe to:. Understand the fundamentas of the mutidimensiona direct search methods. Understand how the coordinate
More informationLearning to Learn Second-Order Back-Propagation for CNNs Using LSTMs
Learning to Learn Second-Order Bac-Propagation for CNNs Using LSTMs Anirban Roy SRI Internationa Meno Par, USA anirban.roy@sri.com Sinisa Todorovic Oregon State University Corvais, USA sinisa@eecs.oregonstate.edu
More informationA HYBRID FEATURE SELECTION METHOD BASED ON FISHER SCORE AND GENETIC ALGORITHM
Journa of Mathematica Sciences: Advances and Appications Voume 37, 2016, Pages 51-78 Avaiabe at http://scientificadvances.co.in DOI: http://dx.doi.org/10.18642/jmsaa_7100121627 A HYBRID FEATURE SELECTION
More informationComplex Human Activity Searching in a Video Employing Negative Space Analysis
Compex Human Activity Searching in a Video Empoying Negative Space Anaysis Shah Atiqur Rahman, Siu-Yeung Cho, M.K.H. Leung 3, Schoo of Computer Engineering, Nanyang Technoogica University, Singapore 639798
More informationLayout Conscious Approach and Bus Architecture Synthesis for Hardware-Software Co-Design of Systems on Chip Optimized for Speed
Layout Conscious Approach and Bus Architecture Synthesis for Hardware-Software Co-Design of Systems on Chip Optimized for Speed Nattawut Thepayasuwan, Member, IEEE and Aex Doboi, Member, IEEE Abstract
More informationBacking-up Fuzzy Control of a Truck-trailer Equipped with a Kingpin Sliding Mechanism
Backing-up Fuzzy Contro of a Truck-traier Equipped with a Kingpin Siding Mechanism G. Siamantas and S. Manesis Eectrica & Computer Engineering Dept., University of Patras, Patras, Greece gsiama@upatras.gr;stam.manesis@ece.upatras.gr
More informationFACE RECOGNITION WITH HARMONIC DE-LIGHTING. s: {lyqing, sgshan, wgao}jdl.ac.cn
FACE RECOGNITION WITH HARMONIC DE-LIGHTING Laiyun Qing 1,, Shiguang Shan, Wen Gao 1, 1 Graduate Schoo, CAS, Beijing, China, 100080 ICT-ISVISION Joint R&D Laboratory for Face Recognition, CAS, Beijing,
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationBayesian model ensembling using meta-trained recurrent neural networks
Bayesian model ensembling using meta-trained recurrent neural networks Luca Ambrogioni l.ambrogioni@donders.ru.nl Umut Güçlü u.guclu@donders.ru.nl Yağmur Güçlütürk y.gucluturk@donders.ru.nl Julia Berezutskaya
More informationLearning from Data. COMP61011 : Machine Learning and Data Mining. Dr Gavin Brown Machine Learning and Op<miza<on Research Group
Learning from Data COMP61011 : Machine Learning and Data Mining Dr Gavin Brown Machine Learning and Op
More information