Lecture 5: Multilayer Perceptrons
|
|
- Jean Malone
- 5 years ago
- Views:
Transcription
1 Lecture 5: Multlayer Perceptrons Roger Grosse 1 Introducton So far, we ve only talked about lnear models: lnear regresson and lnear bnary classfers. We noted that there are functons that can t be represented by lnear models; for nstance, lnear regresson can t represent quadratc functons, and lnear classfers can t represent XOR. We also saw one partcular way around ths ssue: by defnng features, or bass functons. E.g., lnear regresson can represent a cubc polynomal f we use the feature map ψ(x) = (1, x, x 2, x 3 ). We also observed that ths sn t a very satsfyng soluton, for two reasons: 1. The features need to be specfed n advance, and ths can requre a lot of engneerng work. 2. It mght requre a very large number of features to represent a certan set of functons; e.g. the feature representaton for cubc polynomals s cubc n the number of nput features. In ths lecture, and for the rest of the course, we ll take a dfferent approach. We ll represent complex nonlnear functons by connectng together lots of smple processng unts nto a neural network, each of whch computes a lnear functon, possbly followed by a nonlnearty. In aggregate, these unts can compute some surprsngly complex functons. By hstorcal accdent, these networks are called multlayer perceptrons. Some people would clam that the methods covered n ths course are really just adaptve bass functon representatons. I ve never found ths a very useful way of lookng at thngs. 1.1 Learnng Goals Know the basc termnology for neural nets Gven the weghts and bases for a neural net, be able to compute ts output from ts nput Be able to hand-desgn the weghts of a neural net to represent functons lke XOR Understand how a hard threshold can be approxmated wth a soft threshold Understand why shallow neural nets are unversal, and why ths sn t necessarly very nterestng 1
2 Fgure 1: A multlayer perceptron wth two hdden layers. Left: wth the unts wrtten out explctly. Rght: representng layers as boxes. 2 Multlayer Perceptrons In the frst lecture, we ntroduced our general neuron-lke processng unt: a = φ j w j x j + b, where the x j are the nputs to the unt, the w j are the weghts, b s the bas, φ s the nonlnear actvaton functon, and a s the unt s actvaton. We ve seen a bunch of examples of such unts: Lnear regresson uses a lnear model, so φ(z) = z. In bnary lnear classfers, φ s a hard threshold at zero. In logstc regresson, φ s the logstc functon σ(z) = 1/(1 + e z ). A neural network s just a combnaton of lots of these unts. Each one performs a very smple and stereotyped functon, but n aggregate they can do some very useful computatons. For now, we ll concern ourselves wth feed-forward neural networks, where the unts are arranged nto a graph wthout any cycles, so that all the computaton can be done sequentally. Ths s n contrast wth recurrent neural networks, where the graph can have cycles, so the processng can feed nto tself. These are much more complcated, and we ll cover them later n the course. The smplest knd of feed-forward network s a multlayer perceptron (MLP), as shown n Fgure 1. Here, the unts are arranged nto a set of layers, and each layer contans some number of dentcal unts. Every unt n one layer s connected to every unt n the next layer; we say that the network s fully connected. The frst layer s the nput layer, and ts unts take the values of the nput features. The last layer s the output layer, and t has one unt for each value the network outputs (.e. a sngle unt n the case of regresson or bnary classfaton, or K unts n the case of K-class classfcaton). All the layers n between these are known as hdden layers, because we don t know ahead of tme what these unts should compute, and ths needs to be dscovered durng learnng. The unts MLP s an unfortunate name. The perceptron was a partcular algorthm for bnary classfcaton, nvented n the 1950s. Most multlayer perceptrons have very lttle to do wth the orgnal perceptron algorthm. 2
3 Fgure 2: An MLP that computes the XOR functon. All actvaton functons are bnary thresholds at 0. n these layers are known as nput unts, output unts, and hdden unts, respectvely. The number of layers s known as the depth, and the number of unts n a layer s known as the wdth. As you mght guess, deep learnng refers to tranng neural nets wth many layers. As an example to llustrate the power of MLPs, let s desgn one that computes the XOR functon. Remember, we showed that lnear models cannot do ths. We can verbally descrbe XOR as one of the nputs s 1, but not both of them. So let s have hdden unt h 1 detect f at least one of the nputs s 1, and have h 2 detect f they are both 1. We can easly do ths f we use a hard threshold actvaton functon. You know how to desgn such unts t s an exercse of desgnng a bnary lnear classfer. Then the output unt wll actvate only f h 1 = 1 and h 2 = 0. A network whch does ths s shown n Fgure 2. Let s wrte out the MLP computatons mathematcally. Conceptually, there s nothng new here; we just have to pck a notaton to refer to varous parts of the network. As wth the lnear case, we ll refer to the actvatons of the nput unts as x j and the actvaton of the output unt as y. The unts n the lth hdden layer wll be denoted h (l). Our network s fully connected, so each unt receves connectons from all the unts n the prevous layer. Ths means each unt has ts own bas, and there s a weght for every par of unts n two consecutve layers. Therefore, the network s computatons can be wrtten out as: = φ (1) j h (1) = φ (2) j h (2) y = φ (3) j j x j + b (1) w (1) w (2) j h(1) w (3) j h(2) j + b (2) j + b (3) (1) Termnology for the depth s very nconsstent. A network wth one hdden layer could be called a one-layer, two-layer, or three-layer network, dependng f you count the nput and output layers. Note that we dstngush φ (1) and φ (2) because dfferent layers may have dfferent actvaton functons. Snce all these summatons and ndces can be cumbersome, we usually 3
4 wrte the computatons n vectorzed form. Snce each layer contans multple unts, we represent the actvatons of all ts unts wth an actvaton vector h (l). Snce there s a weght for every par of unts n two consecutve layers, we represent each layer s weghts wth a weght matrx W (l). Each layer also has a bas vector b (l). The above computatons are therefore wrtten n vectorzed form as: h (1) = φ (1) ( W (1) x + b (1)) h (2) = φ (2) ( W (2) h (1) + b (2)) y = φ (3) ( W (3) h (2) + b (3)) (2) When we wrte the actvaton functon appled to a vector, ths means t s appled ndependently to all the entres. Recall how n lnear regresson, we combned all the tranng examples nto a sngle matrx X, so that we could compute all the predctons usng a sngle matrx multplcaton. We can do the same thng here. We can store all of each layer s hdden unts for all the tranng examples as a matrx H (l). Each row contans the hdden unts for one example. The computatons are wrtten as follows (note the transposes): H (1) = φ (1) ( XW (1) + 1b (1) ) H (2) = φ (2) ( H (1) W (2) + 1b (2) ) Y = φ (3) ( H (2) W (3) + 1b (3) ) (3) If t s hard to remember when a matrx or vector s transposed, fear not. You can usually fgure t out by makng sure the dmensons match up. These equatons can be translated drectly nto NumPy code whch effcently computes the predctons over the whole dataset. 3 Feature Learnng We already saw that lnear regresson could be made more powerful usng a feature mappng. For nstance, the feature mappng ψ(x) = (1, x, x 2, x e ) can represent thrd-degree polynomals. But statc feature mappngs were lmted because t can be hard to desgn all the relevant features, and because the mappngs mght be mpractcally large. Neural nets can be thought of as a way of learnng nonlnear feature mappngs. E.g., n Fgure 1, the last hdden layer can be thought of as a feature map ψ(x), and the output layer weghts can be thought of as a lnear model usng those features. But the whole thng can be traned end-to-end wth backpropagaton, whch we ll cover n the next lecture. The hope s that we can learn a feature representaton where the data become lnearly separable: 4
5 Fgure 3: Left: Some tranng examples from the MNIST handwrtten dgt dataset. Each nput s a grayscale mage, whch we treat as a 784- dmensonal vector. Rght: A subset of the learned frst-layer features. Observe that many of them pck up orented edges. Consder tranng an MLP to recognze handwrtten dgts. (Ths wll be a runnng example for much of the course.) The nput s a grayscale mage, and all the pxels take values between 0 and 1. We ll gnore the spatal structure, and treat each nput as a 784-dmensonal vector. Ths s a multway classfcaton task wth 10 categores, one for each dgt class. Suppose we tran an MLP wth two hdden layers. We can try to understand what the frst layer of hdden unts s computng by vsualzng the weghts. Each hdden unt receves nputs from each of the pxels, whch means the weghts feedng nto each hdden unt can be represented as a 784- dmensonal vector, the same as the nput sze. In Fgure 3, we dsplay these vectors as mages. In ths vsualzaton, postve values are lghter, and negatve values are darker. Each hdden unt computes the dot product of these vectors wth the nput mage, and then passes the result through the actvaton functon. So f the lght regons of the flter overlap the lght regons of the mage, and the dark regons of the flter overlap the dark regon of the mage, then the unt wll actvate. E.g., look at the thrd flter n the second row. Ths corresponds to an orented edge: t detects vertcal edges n the upper rght part of the mage. Ths s a useful sort of feature, snce t gves nformaton about the locatons and orentaton of strokes. Many of the features are smlar to ths; n fact, orented edges are a very commonly learned by the frst layers of neural nets for vsual processng tasks. It s harder to vsualze what the second layer s dong. We ll see some trcks for vsualzng ths n a few weeks. We ll see that hgher layers of a neural net can learn ncreasngly hgh-level and complex features. Later on, we ll talk about convolutonal networks, whch use the spatal structure of the mage. 4 Expressve Power Lnear models are fundamentally lmted n ther expressve power: they can t represent functons lke XOR. Are there smlar lmtatons for MLPs? It depends on the actvaton functon. 5
6 Fgure 4: Desgnng a bnary threshold network to compute a partcular functon. 4.1 Lnear networks Deep lnear networks are no more powerful than shallow ones. The reason s smple: f we use the lnear actvaton functon φ(x) = x (and forget the bases for smplcty), the network s functon can be expanded out as y = W (L) W (L 1) W (1) x. But ths could be vewed as a sngle lnear layer wth weghts gven by W = W (L) W (L 1) W (1). Therefore, a deep lnear network s no more powerful than a sngle lnear layer,.e. a lnear model. 4.2 Unversalty As t turns out, nonlnear actvaton functons gve us much more power: under certan techncal condtons, even a shallow MLP (.e. one wth a sngle hdden layer) can represent arbtrary functons. Therefore, we say t s unversal. Let s demonstrate unversalty n the case of bnary nputs. We do ths usng the followng game: suppose we re gven a functon mappng nput vectors to outputs; we wll need to produce a neural network (.e. specfy the weghts and bases) whch matches that functon. The functon can be gven to us as a table whch lsts the output correspondng to every possble nput vector. If there are D nputs, ths table wll have 2 D rows. An example s shown n Fgure 4. For convenence, let s suppose these nputs are ±1, rather than 0 or 1. All of our hdden unts wll use a hard threshold at 0 (but we ll see shortly that these can easly be converted to soft thresholds), and the output unt wll be lnear. Our strategy wll be as follows: we wll have 2 D hdden unts, each of whch recognzes one possble nput vector. We can then specfy the functon by specfyng the weghts connectng each of these hdden unts to the outputs. For nstance, suppose we want a hdden unt to recognze the nput ( 1, 1, 1). Ths can be done usng the weghts ( 1, 1, 1) and bas 2.5, and ths unt wll be connected to the output unt wth weght 1. (Can you come up wth the general rule?) Usng these weghts, any nput pattern wll produce a set of hdden actvatons where exactly one of the unts s actve. The weghts connectng nputs to outputs can be set based on the nput-output table. Part of the network s shown n Fgure 4. Ths argument can easly be made nto a rgorous proof, but ths course won t be concerned wth mathematcal rgor. 6
7 Unversalty s a neat property, but t has a major catch: the network requred to represent a gven functon mght have to be extremely large (n partcular, exponental). In other words, not all functons can be represented compactly. We desre compact representatons for two reasons: 1. We want to be able to compute predctons n a reasonable amount of tme. 2. We want to be able to tran a network to generalze from a lmted number of tranng examples; from ths perspectve, unversalty smply mples that a large enough network can memorze the tranng set, whch sn t very nterestng. 4.3 Soft thresholds In the prevous secton, our actvaton functon was a step functon, whch gves a hard threshold at 0. Ths was convenent for desgnng the weghts of a network by hand. But recall from last lecture that t s very hard to drectly learn a lnear classfer wth a hard threshold, because the loss dervatves are 0 almost everywhere. The same holds true for multlayer perceptrons. If the actvaton functon for any unt s a hard threshold, we won t be able to learn that unt s weghts usng gradent descent. The soluton s the same as t was n last lecture: we replace the hard threshold wth a soft one. Does ths cost us anythng n terms of the network s expressve power? No t doesn t, because we can approxmate a hard threshold usng a soft threshold. In partcular, f we use the logstc nonlnearty, we can approxmate a hard threshold by scalng up the weghts and bases: 4.4 The power of depth If shallow networks are unversal, why do we need deep ones? One mportant reason s that deep nets can represent some functons more compactly than shallow ones. For nstance, consder the party functon (on bnary-valued nputs): { 1 f f par (x 1,..., x D ) = j x j s odd (4) 0 f t s even. We won t prove ths, but t requres an exponentally large shallow network to represent the party functon. On the other hand, t can be computed by a deep network whose sze s lnear n the number of nputs. Desgnng such a network s a good exercse. 7
Machine Learning 9. week
Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below
More informationComplex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following.
Complex Numbers The last topc n ths secton s not really related to most of what we ve done n ths chapter, although t s somewhat related to the radcals secton as we wll see. We also won t need the materal
More informationClassification / Regression Support Vector Machines
Classfcaton / Regresson Support Vector Machnes Jeff Howbert Introducton to Machne Learnng Wnter 04 Topcs SVM classfers for lnearly separable classes SVM classfers for non-lnearly separable classes SVM
More informationSupport Vector Machines
/9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.
More informationCMPS 10 Introduction to Computer Science Lecture Notes
CPS 0 Introducton to Computer Scence Lecture Notes Chapter : Algorthm Desgn How should we present algorthms? Natural languages lke Englsh, Spansh, or French whch are rch n nterpretaton and meanng are not
More informationFor instance, ; the five basic number-sets are increasingly more n A B & B A A = B (1)
Secton 1.2 Subsets and the Boolean operatons on sets If every element of the set A s an element of the set B, we say that A s a subset of B, or that A s contaned n B, or that B contans A, and we wrte A
More informationMathematics 256 a course in differential equations for engineering students
Mathematcs 56 a course n dfferental equatons for engneerng students Chapter 5. More effcent methods of numercal soluton Euler s method s qute neffcent. Because the error s essentally proportonal to the
More informationOutline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1
4/14/011 Outlne Dscrmnatve classfers for mage recognton Wednesday, Aprl 13 Krsten Grauman UT-Austn Last tme: wndow-based generc obect detecton basc ppelne face detecton wth boostng as case study Today:
More informationProblem Set 3 Solutions
Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n,
More informationEdge Detection in Noisy Images Using the Support Vector Machines
Edge Detecton n Nosy Images Usng the Support Vector Machnes Hlaro Gómez-Moreno, Saturnno Maldonado-Bascón, Francsco López-Ferreras Sgnal Theory and Communcatons Department. Unversty of Alcalá Crta. Madrd-Barcelona
More informationSupport Vector Machines
Support Vector Machnes Decson surface s a hyperplane (lne n 2D) n feature space (smlar to the Perceptron) Arguably, the most mportant recent dscovery n machne learnng In a nutshell: map the data to a predetermned
More informationHermite Splines in Lie Groups as Products of Geodesics
Hermte Splnes n Le Groups as Products of Geodescs Ethan Eade Updated May 28, 2017 1 Introducton 1.1 Goal Ths document defnes a curve n the Le group G parametrzed by tme and by structural parameters n the
More informationMachine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law)
Machne Learnng Support Vector Machnes (contans materal adapted from talks by Constantn F. Alfers & Ioanns Tsamardnos, and Martn Law) Bryan Pardo, Machne Learnng: EECS 349 Fall 2014 Support Vector Machnes
More informationParallelism for Nested Loops with Non-uniform and Flow Dependences
Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr
More informationS1 Note. Basis functions.
S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type
More informationChapter 6 Programmng the fnte element method Inow turn to the man subject of ths book: The mplementaton of the fnte element algorthm n computer programs. In order to make my dscusson as straghtforward
More informationLearning the Kernel Parameters in Kernel Minimum Distance Classifier
Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department
More informationSLAM Summer School 2006 Practical 2: SLAM using Monocular Vision
SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,
More information6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour
6.854 Advanced Algorthms Petar Maymounkov Problem Set 11 (November 23, 2005) Wth: Benjamn Rossman, Oren Wemann, and Pouya Kheradpour Problem 1. We reduce vertex cover to MAX-SAT wth weghts, such that the
More informationProgramming in Fortran 90 : 2017/2018
Programmng n Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Exercse 1 : Evaluaton of functon dependng on nput Wrte a program who evaluate the functon f (x,y) for any two user specfed values
More information3D vector computer graphics
3D vector computer graphcs Paolo Varagnolo: freelance engneer Padova Aprl 2016 Prvate Practce ----------------------------------- 1. Introducton Vector 3D model representaton n computer graphcs requres
More informationFace Recognition University at Buffalo CSE666 Lecture Slides Resources:
Face Recognton Unversty at Buffalo CSE666 Lecture Sldes Resources: http://www.face-rec.org/algorthms/ Overvew of face recognton algorthms Correlaton - Pxel based correspondence between two face mages Structural
More informationSequential search. Building Java Programs Chapter 13. Sequential search. Sequential search
Sequental search Buldng Java Programs Chapter 13 Searchng and Sortng sequental search: Locates a target value n an array/lst by examnng each element from start to fnsh. How many elements wll t need to
More informationAnnouncements. Supervised Learning
Announcements See Chapter 5 of Duda, Hart, and Stork. Tutoral by Burge lnked to on web page. Supervsed Learnng Classfcaton wth labeled eamples. Images vectors n hgh-d space. Supervsed Learnng Labeled eamples
More informationLecture #15 Lecture Notes
Lecture #15 Lecture Notes The ocean water column s very much a 3-D spatal entt and we need to represent that structure n an economcal way to deal wth t n calculatons. We wll dscuss one way to do so, emprcal
More informationToday s Outline. Sorting: The Big Picture. Why Sort? Selection Sort: Idea. Insertion Sort: Idea. Sorting Chapter 7 in Weiss.
Today s Outlne Sortng Chapter 7 n Wess CSE 26 Data Structures Ruth Anderson Announcements Wrtten Homework #6 due Frday 2/26 at the begnnng of lecture Proect Code due Mon March 1 by 11pm Today s Topcs:
More information12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification
Introducton to Artfcal Intellgence V22.0472-001 Fall 2009 Lecture 24: Nearest-Neghbors & Support Vector Machnes Rob Fergus Dept of Computer Scence, Courant Insttute, NYU Sldes from Danel Yeung, John DeNero
More informationA Binarization Algorithm specialized on Document Images and Photos
A Bnarzaton Algorthm specalzed on Document mages and Photos Ergna Kavalleratou Dept. of nformaton and Communcaton Systems Engneerng Unversty of the Aegean kavalleratou@aegean.gr Abstract n ths paper, a
More informationCompiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz
Compler Desgn Sprng 2014 Regster Allocaton Sample Exercses and Solutons Prof. Pedro C. Dnz USC / Informaton Scences Insttute 4676 Admralty Way, Sute 1001 Marna del Rey, Calforna 90292 pedro@s.edu Regster
More informationCS 534: Computer Vision Model Fitting
CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust
More informationR s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes
SPH3UW Unt 7.3 Sphercal Concave Mrrors Page 1 of 1 Notes Physcs Tool box Concave Mrror If the reflectng surface takes place on the nner surface of the sphercal shape so that the centre of the mrror bulges
More informationArray transposition in CUDA shared memory
Array transposton n CUDA shared memory Mke Gles February 19, 2014 Abstract Ths short note s nspred by some code wrtten by Jeremy Appleyard for the transposton of data through shared memory. I had some
More informationFace Detection with Deep Learning
Face Detecton wth Deep Learnng Yu Shen Yus122@ucsd.edu A13227146 Kuan-We Chen kuc010@ucsd.edu A99045121 Yzhou Hao y3hao@ucsd.edu A98017773 Mn Hsuan Wu mhwu@ucsd.edu A92424998 Abstract The project here
More informationCHARUTAR VIDYA MANDAL S SEMCOM Vallabh Vidyanagar
CHARUTAR VIDYA MANDAL S SEMCOM Vallabh Vdyanagar Faculty Name: Am D. Trved Class: SYBCA Subject: US03CBCA03 (Advanced Data & Fle Structure) *UNIT 1 (ARRAYS AND TREES) **INTRODUCTION TO ARRAYS If we want
More informationAnalysis of Continuous Beams in General
Analyss of Contnuous Beams n General Contnuous beams consdered here are prsmatc, rgdly connected to each beam segment and supported at varous ponts along the beam. onts are selected at ponts of support,
More informationMath Homotopy Theory Additional notes
Math 527 - Homotopy Theory Addtonal notes Martn Frankland February 4, 2013 The category Top s not Cartesan closed. problem. In these notes, we explan how to remedy that 1 Compactly generated spaces Ths
More informationOutline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:
Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A
More informationImage Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline
mage Vsualzaton mage Vsualzaton mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and Analyss outlne mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and
More informationModule Management Tool in Software Development Organizations
Journal of Computer Scence (5): 8-, 7 ISSN 59-66 7 Scence Publcatons Management Tool n Software Development Organzatons Ahmad A. Al-Rababah and Mohammad A. Al-Rababah Faculty of IT, Al-Ahlyyah Amman Unversty,
More informationContent Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers
IOSR Journal of Electroncs and Communcaton Engneerng (IOSR-JECE) e-issn: 78-834,p- ISSN: 78-8735.Volume 9, Issue, Ver. IV (Mar - Apr. 04), PP 0-07 Content Based Image Retreval Usng -D Dscrete Wavelet wth
More informationCourse Introduction. Algorithm 8/31/2017. COSC 320 Advanced Data Structures and Algorithms. COSC 320 Advanced Data Structures and Algorithms
Course Introducton Course Topcs Exams, abs, Proects A quc loo at a few algorthms 1 Advanced Data Structures and Algorthms Descrpton: We are gong to dscuss algorthm complexty analyss, algorthm desgn technques
More informationInverse Kinematics (part 2) CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016
Inverse Knematcs (part 2) CSE169: Computer Anmaton Instructor: Steve Rotenberg UCSD, Sprng 2016 Forward Knematcs We wll use the vector: Φ... 1 2 M to represent the array of M jont DOF values We wll also
More informationTN348: Openlab Module - Colocalization
TN348: Openlab Module - Colocalzaton Topc The Colocalzaton module provdes the faclty to vsualze and quantfy colocalzaton between pars of mages. The Colocalzaton wndow contans a prevew of the two mages
More informationWriter Identification using a Deep Neural Network
Wrter Identfcaton usng a Deep Neural Network Jun Chu and Sargur Srhar Department of Computer Scence and Engneerng Unversty at Buffalo, The State Unversty of New York Buffalo, NY 1469, USA {jchu6, srhar}@buffalo.edu
More informationIntro. Iterators. 1. Access
Intro Ths mornng I d lke to talk a lttle bt about s and s. We wll start out wth smlartes and dfferences, then we wll see how to draw them n envronment dagrams, and we wll fnsh wth some examples. Happy
More informationCluster Analysis of Electrical Behavior
Journal of Computer and Communcatons, 205, 3, 88-93 Publshed Onlne May 205 n ScRes. http://www.scrp.org/ournal/cc http://dx.do.org/0.4236/cc.205.350 Cluster Analyss of Electrcal Behavor Ln Lu Ln Lu, School
More informationFEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur
FEATURE EXTRACTION Dr. K.Vjayarekha Assocate Dean School of Electrcal and Electroncs Engneerng SASTRA Unversty, Thanjavur613 41 Jont Intatve of IITs and IISc Funded by MHRD Page 1 of 8 Table of Contents
More informationA Bilinear Model for Sparse Coding
A Blnear Model for Sparse Codng Davd B. Grmes and Rajesh P. N. Rao Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA 98195-2350, U.S.A. grmes,rao @cs.washngton.edu Abstract
More informationSolving two-person zero-sum game by Matlab
Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by
More informationRecognizing Faces. Outline
Recognzng Faces Drk Colbry Outlne Introducton and Motvaton Defnng a feature vector Prncpal Component Analyss Lnear Dscrmnate Analyss !"" #$""% http://www.nfotech.oulu.f/annual/2004 + &'()*) '+)* 2 ! &
More information2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements
Module 3: Element Propertes Lecture : Lagrange and Serendpty Elements 5 In last lecture note, the nterpolaton functons are derved on the bass of assumed polynomal from Pascal s trangle for the fled varable.
More informationAccounting for the Use of Different Length Scale Factors in x, y and z Directions
1 Accountng for the Use of Dfferent Length Scale Factors n x, y and z Drectons Taha Soch (taha.soch@kcl.ac.uk) Imagng Scences & Bomedcal Engneerng, Kng s College London, The Rayne Insttute, St Thomas Hosptal,
More informationON SOME ENTERTAINING APPLICATIONS OF THE CONCEPT OF SET IN COMPUTER SCIENCE COURSE
Yordzhev K., Kostadnova H. Інформаційні технології в освіті ON SOME ENTERTAINING APPLICATIONS OF THE CONCEPT OF SET IN COMPUTER SCIENCE COURSE Yordzhev K., Kostadnova H. Some aspects of programmng educaton
More informationA Modified Median Filter for the Removal of Impulse Noise Based on the Support Vector Machines
A Modfed Medan Flter for the Removal of Impulse Nose Based on the Support Vector Machnes H. GOMEZ-MORENO, S. MALDONADO-BASCON, F. LOPEZ-FERRERAS, M. UTRILLA- MANSO AND P. GIL-JIMENEZ Departamento de Teoría
More informationCHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION
48 CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 3.1 INTRODUCTION The raw mcroarray data s bascally an mage wth dfferent colors ndcatng hybrdzaton (Xue
More informationAn Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices
Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal
More informationOn Some Entertaining Applications of the Concept of Set in Computer Science Course
On Some Entertanng Applcatons of the Concept of Set n Computer Scence Course Krasmr Yordzhev *, Hrstna Kostadnova ** * Assocate Professor Krasmr Yordzhev, Ph.D., Faculty of Mathematcs and Natural Scences,
More informationFast Feature Value Searching for Face Detection
Vol., No. 2 Computer and Informaton Scence Fast Feature Value Searchng for Face Detecton Yunyang Yan Department of Computer Engneerng Huayn Insttute of Technology Hua an 22300, Chna E-mal: areyyyke@63.com
More informationParallel matrix-vector multiplication
Appendx A Parallel matrx-vector multplcaton The reduced transton matrx of the three-dmensonal cage model for gel electrophoress, descrbed n secton 3.2, becomes excessvely large for polymer lengths more
More informationFeature Reduction and Selection
Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components
More informationRandom Kernel Perceptron on ATTiny2313 Microcontroller
Random Kernel Perceptron on ATTny233 Mcrocontroller Nemanja Djurc Department of Computer and Informaton Scences, Temple Unversty Phladelpha, PA 922, USA nemanja.djurc@temple.edu Slobodan Vucetc Department
More informationLecture 13: High-dimensional Images
Lec : Hgh-dmensonal Images Grayscale Images Lecture : Hgh-dmensonal Images Math 90 Prof. Todd Wttman The Ctadel A grayscale mage s an nteger-valued D matrx. An 8-bt mage takes on values between 0 and 55.
More informationAssignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009.
Farrukh Jabeen Algorthms 51 Assgnment #2 Due Date: June 15, 29. Assgnment # 2 Chapter 3 Dscrete Fourer Transforms Implement the FFT for the DFT. Descrbed n sectons 3.1 and 3.2. Delverables: 1. Concse descrpton
More informationInsertion Sort. Divide and Conquer Sorting. Divide and Conquer. Mergesort. Mergesort Example. Auxiliary Array
Inserton Sort Dvde and Conquer Sortng CSE 6 Data Structures Lecture 18 What f frst k elements of array are already sorted? 4, 7, 1, 5, 1, 16 We can shft the tal of the sorted elements lst down and then
More informationFeature-Based Matrix Factorization
Feature-Based Matrx Factorzaton arxv:1109.2271v3 [cs.ai] 29 Dec 2011 Tanq Chen, Zhao Zheng, Quxa Lu, Wenan Zhang, Yong Yu {tqchen,zhengzhao,luquxa,wnzhang,yyu}@apex.stu.edu.cn Apex Data & Knowledge Management
More informationWhy visualisation? IRDS: Visualization. Univariate data. Visualisations that we won t be interested in. Graphics provide little additional information
Why vsualsaton? IRDS: Vsualzaton Charles Sutton Unversty of Ednburgh Goal : Have a data set that I want to understand. Ths s called exploratory data analyss. Today s lecture. Goal II: Want to dsplay data
More informationNAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics
Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson
More informationCAN COMPUTERS LEARN FASTER? Seyda Ertekin Computer Science & Engineering The Pennsylvania State University
CAN COMPUTERS LEARN FASTER? Seyda Ertekn Computer Scence & Engneerng The Pennsylvana State Unversty sertekn@cse.psu.edu ABSTRACT Ever snce computers were nvented, manknd wondered whether they mght be made
More informationLearning-based License Plate Detection on Edge Features
Learnng-based Lcense Plate Detecton on Edge Features Wng Teng Ho, Woo Hen Yap, Yong Haur Tay Computer Vson and Intellgent Systems (CVIS) Group Unverst Tunku Abdul Rahman, Malaysa wngteng_h@yahoo.com, woohen@yahoo.com,
More informationSUMMARY... I TABLE OF CONTENTS...II INTRODUCTION...
Summary A follow-the-leader robot system s mplemented usng Dscrete-Event Supervsory Control methods. The system conssts of three robots, a leader and two followers. The dea s to get the two followers to
More informationClassification algorithms on the cell processor
Rochester Insttute of Technology RIT Scholar Works Theses Thess/Dssertaton Collectons 8-1-2008 Classfcaton algorthms on the cell processor Mateusz Wyganowsk Follow ths and addtonal works at: http://scholarworks.rt.edu/theses
More informationRelated-Mode Attacks on CTR Encryption Mode
Internatonal Journal of Network Securty, Vol.4, No.3, PP.282 287, May 2007 282 Related-Mode Attacks on CTR Encrypton Mode Dayn Wang, Dongda Ln, and Wenlng Wu (Correspondng author: Dayn Wang) Key Laboratory
More informationA Network for Extracting the Locations of Point Clusters Using Selective Attention 1
INTERNATIONAL COMPUTER SCIENCE INSTITUTE 1947 Center Street Sute 600 Berkeley, Calforna 94704 (415) 643 4274 FAX (415) 643-7684 A Network for Extractng the Locatons of Pont Clusters Usng Selectve Attenton
More informationSubspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;
Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features
More informationPrivate Information Retrieval (PIR)
2 Levente Buttyán Problem formulaton Alce wants to obtan nformaton from a database, but she does not want the database to learn whch nformaton she wanted e.g., Alce s an nvestor queryng a stock-market
More informationHarmonic Coordinates for Character Articulation PIXAR
Harmonc Coordnates for Character Artculaton PIXAR Pushkar Josh Mark Meyer Tony DeRose Bran Green Tom Sanock We have a complex source mesh nsde of a smpler cage mesh We want vertex deformatons appled to
More informationSupport Vector Machines. CS534 - Machine Learning
Support Vector Machnes CS534 - Machne Learnng Perceptron Revsted: Lnear Separators Bnar classfcaton can be veed as the task of separatng classes n feature space: b > 0 b 0 b < 0 f() sgn( b) Lnear Separators
More informationLecture notes: Histogram, convolution, smoothing
Lecture notes: Hstogram, convoluton, smoothng Hstogram. A plot o the ntensty dstrbuton n an mage. requency (# occurrences) ntensty The ollowng shows an example mage and ts hstogram: I we denote a greyscale
More informationAn Optimal Algorithm for Prufer Codes *
J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,
More informationCSE 326: Data Structures Quicksort Comparison Sorting Bound
CSE 326: Data Structures Qucksort Comparson Sortng Bound Steve Setz Wnter 2009 Qucksort Qucksort uses a dvde and conquer strategy, but does not requre the O(N) extra space that MergeSort does. Here s the
More informationRadial Basis Functions
Radal Bass Functons Mesh Reconstructon Input: pont cloud Output: water-tght manfold mesh Explct Connectvty estmaton Implct Sgned dstance functon estmaton Image from: Reconstructon and Representaton of
More informationReview of approximation techniques
CHAPTER 2 Revew of appromaton technques 2. Introducton Optmzaton problems n engneerng desgn are characterzed by the followng assocated features: the objectve functon and constrants are mplct functons evaluated
More informationUser Authentication Based On Behavioral Mouse Dynamics Biometrics
User Authentcaton Based On Behavoral Mouse Dynamcs Bometrcs Chee-Hyung Yoon Danel Donghyun Km Department of Computer Scence Department of Computer Scence Stanford Unversty Stanford Unversty Stanford, CA
More informationCSCI 104 Sorting Algorithms. Mark Redekopp David Kempe
CSCI 104 Sortng Algorthms Mark Redekopp Davd Kempe Algorthm Effcency SORTING 2 Sortng If we have an unordered lst, sequental search becomes our only choce If we wll perform a lot of searches t may be benefcal
More informationActive Contours/Snakes
Actve Contours/Snakes Erkut Erdem Acknowledgement: The sldes are adapted from the sldes prepared by K. Grauman of Unversty of Texas at Austn Fttng: Edges vs. boundares Edges useful sgnal to ndcate occludng
More informationComparing Image Representations for Training a Convolutional Neural Network to Classify Gender
2013 Frst Internatonal Conference on Artfcal Intellgence, Modellng & Smulaton Comparng Image Representatons for Tranng a Convolutonal Neural Network to Classfy Gender Choon-Boon Ng, Yong-Haur Tay, Bok-Mn
More informationAssembler. Building a Modern Computer From First Principles.
Assembler Buldng a Modern Computer From Frst Prncples www.nand2tetrs.org Elements of Computng Systems, Nsan & Schocken, MIT Press, www.nand2tetrs.org, Chapter 6: Assembler slde Where we are at: Human Thought
More informationSorting Review. Sorting. Comparison Sorting. CSE 680 Prof. Roger Crawfis. Assumptions
Sortng Revew Introducton to Algorthms Qucksort CSE 680 Prof. Roger Crawfs Inserton Sort T(n) = Θ(n 2 ) In-place Merge Sort T(n) = Θ(n lg(n)) Not n-place Selecton Sort (from homework) T(n) = Θ(n 2 ) In-place
More informationBrave New World Pseudocode Reference
Brave New World Pseudocode Reference Pseudocode s a way to descrbe how to accomplsh tasks usng basc steps lke those a computer mght perform. In ths week s lab, you'll see how a form of pseudocode can be
More informationReading. 14. Subdivision curves. Recommended:
eadng ecommended: Stollntz, Deose, and Salesn. Wavelets for Computer Graphcs: heory and Applcatons, 996, secton 6.-6., A.5. 4. Subdvson curves Note: there s an error n Stollntz, et al., secton A.5. Equaton
More informationCSE 326: Data Structures Quicksort Comparison Sorting Bound
CSE 326: Data Structures Qucksort Comparson Sortng Bound Bran Curless Sprng 2008 Announcements (5/14/08) Homework due at begnnng of class on Frday. Secton tomorrow: Graded homeworks returned More dscusson
More informationOptimization Methods: Integer Programming Integer Linear Programming 1. Module 7 Lecture Notes 1. Integer Linear Programming
Optzaton Methods: Integer Prograng Integer Lnear Prograng Module Lecture Notes Integer Lnear Prograng Introducton In all the prevous lectures n lnear prograng dscussed so far, the desgn varables consdered
More informationA Fast Content-Based Multimedia Retrieval Technique Using Compressed Data
A Fast Content-Based Multmeda Retreval Technque Usng Compressed Data Borko Furht and Pornvt Saksobhavvat NSF Multmeda Laboratory Florda Atlantc Unversty, Boca Raton, Florda 3343 ABSTRACT In ths paper,
More informationSENSITIVITY ANALYSIS IN LINEAR PROGRAMMING USING A CALCULATOR
SENSITIVITY ANALYSIS IN LINEAR PROGRAMMING USING A CALCULATOR Judth Aronow Rchard Jarvnen Independent Consultant Dept of Math/Stat 559 Frost Wnona State Unversty Beaumont, TX 7776 Wnona, MN 55987 aronowju@hal.lamar.edu
More informationCS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS46: Mnng Massve Datasets Jure Leskovec, Stanford Unversty http://cs46.stanford.edu /19/013 Jure Leskovec, Stanford CS46: Mnng Massve Datasets, http://cs46.stanford.edu Perceptron: y = sgn( x Ho to fnd
More informationAdaptive Regression in SAS/IML
Adaptve Regresson n SAS/IML Davd Katz, Davd Katz Consultng, Ashland, Oregon ABSTRACT Adaptve Regresson algorthms allow the data to select the form of a model n addton to estmatng the parameters. Fredman
More informationSmoothing Spline ANOVA for variable screening
Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory
More informationAPPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT
3. - 5. 5., Brno, Czech Republc, EU APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT Abstract Josef TOŠENOVSKÝ ) Lenka MONSPORTOVÁ ) Flp TOŠENOVSKÝ
More informationArtificial Intelligence (AI) methods are concerned with. Artificial Intelligence Techniques for Steam Generator Modelling
Artfcal Intellgence Technques for Steam Generator Modellng Sarah Wrght and Tshldz Marwala Abstract Ths paper nvestgates the use of dfferent Artfcal Intellgence methods to predct the values of several contnuous
More informationHigh-Boost Mesh Filtering for 3-D Shape Enhancement
Hgh-Boost Mesh Flterng for 3-D Shape Enhancement Hrokazu Yagou Λ Alexander Belyaev y Damng We z Λ y z ; ; Shape Modelng Laboratory, Unversty of Azu, Azu-Wakamatsu 965-8580 Japan y Computer Graphcs Group,
More information